..:n..n..~mmzaw§l 5.1%. 5.". Hugh. ‘ 5214.. .. Rio?! 5...; 2...? :1 EFT»? .315 .. ..r13uiv . . an ,. ,3! .. . E. 1.:- .I: / till ”7“ ”Eng WCHI u a RAM \llllllli‘l Michigan State Unlverslty ll“ ll l\\\\ l; \lll This is to certify that the thesis entitled THE EFFECTS OF EVALUATIVE FEEDBACK AND TASK DIFFICULTY ON LEARNING AND TRAINING PERFORMANCE presented by Rebecca J. Toney has been accepted towards fulfillment of the requirements for M'A- degree in Psychology Q g 54 g g. ”z /&I Date July 13, 1998 0-7639 MS U is an Affirmative Action/Equal Opportunity Institution PLACE IN RETURN BOX to remove this checkout from your record. TO AVOID FINES return on or before date due. MAY BE RECALLED with earlier due date if requested. DATE DUE DATE DUE DATE DUE 1“ WM THE EFFECTS OF EVALUATIVE FEEDBACK AND TASK DIFFICULTY ON LEARNING AND TRAINING PERFORMANCE By Rebecca J. Toney A THESIS Submitted to Michigan State University In partial fulfillment of the requirements for the degree of MASTER OF ARTS Department of Psychology 1998 ABSTRACT THE EFFECTS OF EVALUATIVE FEEDBACK AND TASK DIFFICULTY ON LEARNING AND TRAINING PERFORMANCE By Rebecca J. Toney Trainees are frequently required to learn difficult tasks, which require efficacy, effort, and knowledge. Social Learning Theory (Bandura, 1991) suggests that positive feedback is best for increasing trainees’ efficacy. Control Theory (Carver & Scheier, 1982) suggests that negative feedback is best for facilitating effort and learning. To capitalize on the potential benefits of both approaches, this research examined feedback shifted from positive to negative. Three conditions of feedback were examined: (1) positive, (2) negative, and (3) positive shifted to negative. Feedback was crossed with two levels of task difficulty. Results suggested that, for more difficult tasks, performance for positive feedback trainees began to plateau toward the end of the task. Trainees in the positive shifted to negative condition experienced the greatest increases in goal setting and learning, and the least decreases in time spent studying in later trials of a more difficult task. A process model explicating indirect effects of feedback on learning and training performance received support. Implications for research and practice are discussed. ACKNOWLEDGEMENTS Thanks to Steve Kozlowski, my chair, for guidance and direction in getting started, for flexibility in reviewing drafts (when he was in Australia), and for pushing me when I needed it. He provided thoughtful insights and evaluations that helped make the thesis better, and self-efficacy boosts that helped me feel better about the thesis. Thanks to my committee members, Neal Schmitt and Rick DeShon, for assisting with complex power analysis issues and regression questions. They also asked the difficult questions that made me think in a way that produced a better quality product. Thanks to my colleagues who supported me and believed in me even at those times when I did not, and who celebrated the important milestones with me. Thanks to my mother, father, and sister, who actually understand what I/O psychology is and what I do. They have supported me with encouragement in all my life endeavors that have led me to this point. Thanks to my husband, John, who has been with me every step of the way. He has encouraged me through all of the ups and downs of the past three years. Whether I needed enormous amounts of time to work, a shoulder to cry on, or humor to keep me sane, he was always there to provide what I needed. (He even helped by suggesting an alternative title for this thesis: The Effects of Psychological Mumbo Jumbo on the New Employee.) Thanks to God for the energy, talent, and supportive persons in my life, without which Icould not accomplish anything. iii TABLE OF CONTENTS LIST OF TABLES ............................................................................................................. vi LIST OF FIGURES .......................................................................................................... viii INTRODUCTION ............................................................................................................... 1 Theoretical Foundation ............................................................................................ 7 Social Learning Theory ............................................................................... 7 Control Theory ............................................................................................ 9 Advantages and Disadvantages of Positive Feedback .............................. 12 Advantages and Disadvantages of Negative Feedback ............................. 15 Integration, Model Development, and Hypotheses ............................................... 17 Combining Positive and Negative Feedback ............................................. 18 Overall Model and Construct Development .............................................. 21 Individual Difference Constructs .................................................. 21 Independent Variable Constructs .................................................. 24 Process Variable Constructs .......................................................... 26 Effort Constructs ........................................................................... 28 Outcome Constructs ...................................................................... 28 Hypotheses ................................................................................................ 29 Direct Effects ................................................................................. 29 Interaction Effects ......................................................................... 34 Effects of Feedback Over Time ..................................................... 39 METHOD .......................................................................................................................... 40 Design .................................................................................................................... 40 Overview ................................................................................................... 40 The Task .................................................................................................... 4O Participants ............................................................................................................ 43 Procedure ............................................................................................................... 44 Manipulations ........................................................................................................ 45 Feedback Sign ........................................................................................... 45 Task Difficulty .......................................................................................... 46 Measures ................................................................................................................ 46 RESULTS .......................................................................................................................... 52 Preliminary Analyses ............................................................................................ 52 Direct Effects ......................................................................................................... 55 Interaction Effects ................................................................................................. 68 Effects of Feedback Over Time ............................................................................. 78 Integrated Summary Mode] ................................................................................... 88 iv DISCUSSION ................................................................................................................... 93 Contribution of Ramped Feedback ........................................................................ 93 Contribution of Process Model ............................................................................. 96 Unexpected Results ............................................................................................... 98 Boundary Conditions ............................................................................................. 99 Implications for Research and Practice ............................................................... 102 Conclusion ........................................................................................................... 105 APPENDD( A: Power Analysis ...................................................................................... 107 APPENDIX B: Consent and Debriefing Forms .............................................................. l 1 1 APPENDIX C: Feedback Manipulation FASTBACK Screens ...................................... 1 l3 APPENDD( D: Measures ................................................................................................ l 14 REFERENCES ................................................................................................................ 126 Table 1. Table 2. Table 3. Table 4. Table 5. Table 6. Table 7. Table 8. Table 9. Table 10. Table 11. Table 12. Table 13. Table 14. Table 15. Table 16. Table 17. Table 18. LIST OF TABLES ANCOVA Between Subjects Effects for feedback and task difficulty on goal commitment ........................................................................................... 52 RM-ANCOVA for feedback and task difficulty on goal commitment .............. 53 AN COVA for feedback on feedback perceptions .............................................. 55 RM-MANCOVA for feedback and task difficulty on dependent measures ...... 56 Means, Standard Deviations, and Correlations for Variables used in Regression Analyses ........................................................................................... 57 Summary of Hypotheses and Results ................................................................. 60 Categorical regression for the effect of feedback on self-efficacy ..................... 61 Categorical regression for the effect of feedback on self-satisfaction ............... 62 Categorical regression for the effect of task difficulty on perceived task difficulty ............................................................................................................. 63 Categorical regression for the effect of feedback on perceived task difficulty ............................................................................................................ 64 Regression equation for the effect of self—efficacy on goal setting .................. 64 Regression equation for the effect of self-satisfaction on goal setting ............ 65 Regression equation for the effect of self-efficacy on time spent studying ..... 66 Regression equation for the effect of self-satisfaction on time spent studying ............................................................................................................. 66 Regression equation for the effect of goal setting on training performance 67 Regression equation for the effect of time spent studying on learning ............ 68 Regression equation for the effect of learning on training performance .......... 69 Regression equation for the interaction effect of feedback and attributions on self-efficacy ................................................................................................. 69 vi Table 19. Table 20. Table 21. Table 22. Table 23. Table 24. Table 25. Table 26. Table 27. Table 28. Table 29. Table 30. Regression equation for the interaction effect of feedback and attributions on self-satisfaction ............................................................................................ 70 Regression equation for the interaction effect of self-efficacy and perceived task difficulty on goal setting ........................................................... 72 Regression equation for the interaction effect of self-efficacy and perceived task difficulty on time spent studying .............................................. 72 Regression equation for the interaction effect of self-satisfaction and perceived task difficulty on goal setting ........................................................... 73 Regression equation for the interaction effect of self-satisfaction and perceived task difficulty on time spent studying .............................................. 73 Regression equation for the interaction effect of self-efficacy and objective task difficulty on goal setting ............................................................ 74 Regression equation for the interaction effect of goal setting and objective task difficulty on training performance ............................................. 76 Regression equation for the interaction effect of time spent studying and objective task difficulty on learning ................................................................. 76 Regression equation for the interaction effect of goal setting and learning on training performance ...................................................................... 77 Regression equation for the effect of self-efficacy and self-satisfaction on goal setting ................................................................................................... 90 Regression equation for the effect of self-efficacy and self-satisfaction on time spent studying ...................................................................................... 90 Regression equation for the effect of goal setting and learning on performance ...................................................................................................... 91 vii LIST OF FIGURES Figure 1. The Self-Regulation Process (adapted from Bandura, 1991) ............................. 8 Figure 2. The Negative Feedback Loop (adapted from Carver & Scheier, 1982, 1990) 9 Figure 3. Overall Model .................................................................................................... 22 Figure 4. Direct Effects Model .......................................................................................... 30 Figure 5. Interactions Model ............................................................................................. 35 Figure 6. Interaction between Perceived Task Difficulty and Self-Efficacy .................... 36 Figure 7. Interaction between Perceived Task Difficulty and Self-Satisfaction ............... 37 Figure 8. Goal Commitment by Feedback Group ............................................................. 53 Figure 9. Goal Commitment by Task Difficulty Group .................................................... 54 Figure 10. Figure 11. Figure 12. Figure 13. Figure 14. Figure 15. Figure 16. Figure 17. Figure 18. Figure 19. Figure 20. Figure 21. Interaction of Feedback and Attributions on Self-Efficacy ............................. 7] Interaction of Self-Efficacy and Objective Task Difficulty on Goal Setting... 75 Interaction of Time Spent Studying and Objective Task Difficulty on Learning ........................................................................................................... 77 Self-Efficacy over Time (Residuals) ............................................................... 79 Self-Satisfaction over Time (Residuals) .......................................................... 81 Perceived Task Difficulty over Time (Residuals) ........................................... 82 Goal Setting over Time (Residuals) ................................................................. 84 Time Spent Studying over Time (Residuals) ................................................... 85 Learning over Time (Residuals) ...................................................................... 86 Performance over Time (Residuals) ................................................................ 87 Direct Effects Summary Model ....................................................................... 89 Complete Summary Model .............................................................................. 92 viii INTRODUCTION Change in technology is occurring throughout all types of industry (Howard, 1995), requiring many employees to learn novel, complex tasks. These advances in technology are demanding that employees learn advanced cognitive skills and possess high expertise in order to perform the complex tasks successfully (Kozlowski, Weissbein, Brown, Toney, & Mullins, 1997; Kozlowski, Gully, Smith, Nason, & Brown, 1995). Training becomes an important vehicle for facilitating the process of learning these skills. Technological innovations have also extended into the administration of training to employees. In fact, the complex nature of the tasks being learned in some instances will necessitate the use of advanced technology training. Embedded training systems are now being developed and used which allow the trainer to provide hi gh-fidelity training, and on-line feedback in great amount and detail. Understanding the implications of providing such feedback becomes increasingly important with the advent of these advanced training systems. A number of issues surround the interest in training for novel, complex tasks, including what kind of goals promote learning and performance (Kozlowski et al., 1995; Locke & Latham, 1990), whether the trainee should have a learning or a performance orientation (Bouffard, Boisvert, Vezeau, & Larouche, 1995; Dweck, 1986; Elliott & Dweck, 1988), what the role of ability is in the learning process (Kanfer, 1996; Kanfer & Ackerrnan, 1989; Ree & Earles, 1991; Schmidt, Hunter, Outerbridge, & Goff, 1988), and what kind of augmented feedback is most important to provide trainees (Kluger & DeNisi, 1996; Kozlowski et al., 1997). The purpose of the present study will be to explore further the issues of feedback relevancy in relation to learning novel, complex tasks. Training people on complex or difficult1 tasks presents some problems for trainers. Initially, any task that is new and difficult may take a long period of time for a trainee to learn the skills required for successful performance. As the trainee is struggling to learn the difficult task, she is likely to face some failures early in the learning process. Because she is still learning the basic mechanics of the task, she is unable to perform quickly, smoothly, or efficiently enough to meet a minimum standard. Thus, when her performance is compared with the standards for skilled workers on the task, it is decisively inadequate. Whether by her own interpretation, or by the construal of a trainer or supervisor, her performance on the task is often evaluated as a failure. Such early failure may lead the trainee to lose motivation for exerting further effort toward learning the task. The trainee may also come to believe that she is unable to perform the task adequately even in the future. This belief in one’s capability to perform is termed self-efficacy (Bandura, 1977). In this manner, the trainee’s own reaction to the process of learning a new, difficult task may hinder her from experiencing further learning and the improved performance which naturally comes with practice performing a task. As described above, the effects of evaluative feedback on complex task learning and performance occur through a motivational process. Motivation will be discussed ' Task difficulty can be considered a component of task complexity (Wood, 1986). A more in-depth discussion of the distinction between these constructs will be presented in a later section. From this point forward, the term task difficulty will be used whenever possible; however, in a few cases the term complexity is necessary to remain consistent with previous literature involving this construct. throughout this paper as a latent, conceptual process. Constructs such as self-efficacy serve as indicators of this underlying process. The question of interest is how evaluative feedback indirectly affects learning and performance through its direct effects on variables within the process model. The focus of the present study will be to describe more specifically the motivational effects brought about by the model as a whole. Self-regulation models (Bandura & Cervone, 1983, 1986; Carver & Scheier, 1982, 1990) can be particularly helpful in trying to understand this underlying motivational process. Feedback is usually a central component of self-regulation models. Feedback can be defined as . . information (provided) regarding some aspect of one’s task performance” (Kluger & DeNisi, 1996, p. 255). Within self-regulation, feedback has both process and outcome implications. Process feedback (i.e. information that tells trainees how they are performing) is typically considered necessary for learning. Without receiving some feedback indicating what a trainee did or did not do with respect to the training task, it is questionable whether that trainee could learn the task. Outcome feedback (i.e. information that tells trainees how well they are performing) is not strictly necessary for learning, but can have important evaluative implications relevant to the motivational process that affects learning (Kozlowski et al., 1997). Models of self- regulation tend to involve the use of outcome, rather than process, feedback. As the present study will draw on self-regulation research, outcome feedback will be the focus. Outcome feedback usually reflects a positive or negative sign, depending on whether a trainee is performing well or not. The sign of the feedback provided to trainees has been shown to be particularly relevant in bolstering the self-efficacy and motivation of the trainees (Bandura & Cervone, 1983; Karl, O’Leary-Kelly, & Martocchio, 1993; Latham & Locke, 1991; Podsakoff & Fahr, 1989). In this direction, positive feedback has been routinely heralded as the best feedback to give trainees to aid in improving training performance (Kluger & DeNisi, 1996). The notion is that trainees who believe they are performing well will not lose their motivation to perform or their beliefs that they can perform. There is some question, however, as to whether positive feedback is always best for trainees. First, positive feedback has been shown to improve performance on efiort- based tasks (Bandura & Cervone, 1983, 1986), but has not been studied in relation to improving learning for difficult tasks which require more than just physical effort. For example, Kluger and DeNisi (1996) offer a comprehensive review of the feedback literature, yet their conclusions and propositions relate only to performance outcomes. Second, positive feedback leads to increases in both self-efficacy and satisfaction with performance (Bandura & Cervone, 1983; Bandura & Jourden, 1991; Hysong & Quinones, 1997). While self—efficacy clearly increases as performance increases, self- satisfaction has a curvilinear relationship with performance, such that at a certain level of satisfaction the trainee becomes complacent and performance no longer increases and may even decrease (Bandura & Cervone, 1983). Thus, there is an inherent contradiction: If positive feedback causes both self-efficacy and self-satisfaction to increase, opposing forces will eventually be operating on perforrnance--one motivating and one demotivating. Third, negative feedback has also been shown to have a motivating effect on performance (Bandura & Cervone, 1986; Carver & Scheier, 1982). This evidence suggests that positive feedback does not exclusively explain increases in performance. To elaborate, when a trainee is learning a novel or difficult task, she is likely to face some failures early in the learning process. To get through this initial learning phase, the trainee needs to have a strong sense of self-efficacy. Without such a belief that she will (eventually) succeed at the task, she will not be motivated to continue learning and practicing. As the trainee continues, experience with the task will likely bring more success in performing the task, and hence satisfaction with that successful performance. With repeated success and satisfaction, the trainee will likely become complacent and reduce efforts toward learning the task. At this point the trainee should have substantial self-efficacy for the task, and needs a different kind of motivation to continue learning and practicing. This motivating force may serve to reduce the trainee’s satisfaction with her performance of the task. This explanation of what a trainee should be experiencing throughout the course of training demonstrates that time plays a critical role in training. Because training takes place over an extended period of time, particularly for difficult tasks, trainees go through a range of experiences, and have different needs at different times. Administration of feedback to trainees can be tailored to the different stages the trainee experiences during training, to help enhance the learning process. The above discussion suggests that the best feedback to give trainees may be a combination of positive and negative feedback over the course of training. Such a combination would be most useful to trainees when positive feedback is presented first, and replaced later by negative feedback. Positive feedback may benefit the trainee during the initial stages of the training, because of its tendency to boost the trainee’s self-efficacy at the time when early failures that decrease efficacy are most likely. Negative feedback may benefit the trainee during the later stages of training, when the trainee begins to encounter the complacency that would result from continued positive feedback. This negative feedback would arouse a feeling of dissatisfaction that would counteract complacency. The level of difficulty of the task also affects the trainee’s experiences during training. A more difficult task will generally lead a trainee to put more effort toward learning and performing than will a less difficult task. However, a trainee will also perceive the level of task difficulty differently throughout the training. Initially, the task will seem extremely difficult. As the trainee has gained experience and success with the task, it will likely seem less difficult. When the trainee perceives the task as easier, she may begin to decrease her efforts to learn and practice the task. In order to maintain motivation to continue learning, the trainee needs an experience that will make the task appear more challenging. Thus, the actual difficulty of the task and the trainee’s perception of the task’s difficulty also demonstrate the critical role that time plays during training. While the “objective” difficulty of the task is maintained at a constant level throughout the training, the trainee’s perception of the task’s difficulty changes over time. A trainee’s perception of the task’s difficulty is also affected by the sign of the feedback: Negative feedback connotes a more difficult task than does positive feedback. By presenting negative feedback to trainees later in the task, as it is perceived as less difficult, the trainee will perceive an increase in the task’s difficulty. This change in perception of task difficulty will motivate the trainee to invest more effort in learning and practicing the task. These introductory ideas will be further expanded in the next section. First, relevant theories will be identified and reviewed. The two major theories that speak to the effects of feedback sign on the processes that subsequently lead to learning and performance are Social Learning Theory and Control Theory. Social learning theory (Bandura, 1991) proposes benefits of positive feedback; control theory (Carver & Scheier, 1982, 1990) proposes benefits of negative feedback. Second, arguments for the benefits of positive feedback to learning and performance will be presented, followed by a discussion of the limitations of positive feedback provision. Third, the benefits and limitations of negative feedback will be presented. Fourth, these two approaches will be integrated, along with a model and hypotheses that propose to resolve the apparent paradox between the different approaches. Theoretical Foundation Social Learning Theory Social learning theory is a comprehensive organization of the social and cognitive elements involved in the learning process developed by Bandura (1991). The central process in this meta-theory is self-regulation, a process involving observing, evaluating and reacting to one’s own behaviors (Figure l). The initial phase of the process involves self-observation, or self-monitoring. Self-monitoring functions to provide information for later phases of the process where behavior is compared to personal standards, or goals. The effect self-monitoring has on other parts of the process can, in part, depend on the inforrnativeness of the outcome feedback received. The judgment, or self-evaluation, phase involves comparing information obtained in the self-monitoring phase to personal standards, or goals. Most goals cannot be measured in an absolute sense, and therefore, people make social comparisons to the performance of others, or self-comparisons to their own previous performance. A limiting factor of the effects of self-evaluation is the Self— Self- Self- M onitoring Evaluation Reaction Includes Includes Includes reception of comparison reaction to feedback. ofgoals to comparison feedback. via positive or negative affect. Figure l. The Self-Regulation Process (adapted from Bandura, 1991). extent to which the person attributes her behavior internally, to her own capabilities, or externally to uncontrollable circumstances. Self-reaction involves the affective reaction one has when behavior is compared to a standard or goal. Self-satisfaction is the positive reaction to performance that had exceeded a goal; self-dissatisfaction is the negative reaction to sub—standard performance. The central construct to social learning theory is self-efficacy. As previously stated, self-efficacy can be described as one’s beliefs about one’s capability to successfully execute a behavior to reach a desired outcome (Bandura, 1977). The first two phases of self-regulation--self-monitoring and self-evaluating--are affected by self- efficacy in important ways. First, self-efficacy mediates the effect of social comparison feedback on subsequent performance. Second, self-efficacy affects the level of goals or standards that individuals set. The more efficacious they are, the higher the goal they set for themselves, and the better they perform. This pattern continues as a cyclic process, spiraling upward. Social learning theory necessitates the use of both goals and feedback in the self- regulation process. A number of assertions are made concerning the motivational implications of social learning theory. One unique motivational hypothesis of this theory is that, as standards are attained, they are re-adjusted upward. Another interesting motivational assertion is that, for complex (or difficult) tasks, self—satisfaction from positive feedback is actively sought; for simple (or easy) tasks, self-dissatisfaction from negative feedback is avoided (Bandura, 1991). Control Theory Control theory is another meta-theory that also details the process of self- regulation. However, the central focus of control theory is the negative feedback loop (Figure 2; Carver & Scheier, 1982, 1990). The self-regulation process begins with an Reference Value i ‘ tI Comparator 1 Input Output Function Function l Impact on L I Environment T I Disturbance Figure 2. The Negative Feedback Loop (adapted from Carver & Scheier, 1982, 1990). input function, or the sensing of a present condition. Feedback provided to trainees would be one example of an input function. The next step is a comparison of the perceived input to a reference value, or existing standard (Kluger & DeNisi, 1996). A training goal could be the comparison standard to which the feedback is compared. This comparison is what creates the positive or negative feedback sign. When a discrepancy is perceived between the input and the standard, an output function—-or behavior-~is produced. The purpose of this behavior is to reduce the perceived discrepancy. If feedback indicates that a goal has not been reached, the output function may be either to increase effort or to decrease effort and withdraw from the task. If feedback indicates that the goal has been surpassed--also a discrepancy--the output function may be to decrease effort. The output function will then impact the environment, producing a new input function. Thus, based on the behavior to change effort, new feedback would be provided. Disturbances, or forces outside the self-regulation system, may also impact the environment to produce a new input function. However, the focus of the present study will remain on changes within the system. This feedback 100p is continuously moving toward the goal of reducing discrepancies, which is why it is referred to as a negative feedback loopz. The loop is also capable of simultaneously operating at several different levels, from the subconscious “muscular” level of sensation to a more global, “self-image” level of principles (Carver & Scheier, 1982). The highest level operating at any given time is 2 The term “negative” in reference to a feedback loop is different from the term “negative” in reference to feedback sign. The negative loop means that all departures from a standard are always “negated,” or reduced, whether they are positive discrepancies (above the goal) or negative discrepancies (below the goal). 10 likely the level to which the individual is currently attentive, which is typically a moderate level (Carver & Scheier, 1982; Kluger & DeNisi, 1996). Usually individuals engage in this discrepancy reduction process rather automatically, without being consciously aware of their self-regulation. Kluger and DeNisi suggest that feedback interventions work by re-directing an individual’s attention to the self-regulatory process. Directing attention to the process in this way may result in an increased tendency for trainees to compare performance feedback to goals (Carver & Scheier, 1982), an enhancement of the self—regulation process. Control theory clearly indicates that when a discrepancy reduction is perceived by an individual as too large to be within her capacity to reduce it, she will withdraw from the task (Carver & Scheier, 1982; 1990). Such a discrepancy would likely occur following the presentation of extremely negative feedback, or possibly any prolonged negative feedback at the initial stages of learning a difficult task. In contrast to social learning theory, control theory generally assumes that all discrepancies will motivate reduction, including a positive discrepancy resulting from a surpassed goal. It is suggested that the individual will “coast” for a while with the surpassed goal, and devote extra effort and attention to a different goal (Carver & Scheier, 1990). The feedback system is slow—acting, and individuals do not shift goals--either upward or downward-- very frequently. Once positive feedback has led the trainee to exceed a goal, continued positive feedback will only produce further discrepancies that must be reduced by behaviors such as decreasing effort. This effect may be viewed as similar to the previously discussed effect of self-satisfaction, resulting from an exceeded goal, which has the apparent impact of reducing motivation or effort toward a task. 11 Advantages and Disadvantages of Positive Feedback Benefits of Poflve Feedba_cl;. Social Ieaming theory would suggest that the best feedback to give trainees in order to improve training performance is positive feedback. The reasoning for this is simple and direct. First, positive feedback raises self-efficacy. Past performance on a task is the primary influence of a person’s self-efficacy for future performance (Bandura, 1991; Gist & Mitchell, 1992; Silver, Mitchell, & Gist, 1995). Positive feedback is an unambiguous indicator of the trainee’s past success with task performance. The realization of past success leads to an increase in self—efficacy for future performance. Second, positive feedback raises self-satisfaction--or creates positive affect. Self-regulation literature suggests that when a standard is achieved, the achievement leads to positive affect (Bandura & Cervone, 1983). Positive feedback is evidence that a standard has been met or surpassed. Therefore, meeting the standard leads to positive affect--self-satisfaction. Third, both high self-efficacy and high self-satisfaction--which have arisen from positive feedback--increase motivation, as demonstrated through the goals trainees set for themselves. High judgment of capability (self-efficacy) has been demonstrated to lead to higher goal setting (Locke & Latham, 1990; Wood & Bandura, 1989). Similarly, self- regulation literature suggests that when a standard is achieved (associated with self- satisfaction), 3 higher goal is set (Bandura, 1991; Bandura & Cervone, 1986). The increase in motivation, via goal setting, presumably translates to increases in both Ieaming and performance (barring any unexpected constraints; Latham & Locke, 1991). Limigltions of Positive Feedbzgk. There are, however, some limitations to the ability of social learning theory to adequately explain the effects of positive feedback on 12 learnin g and training performance. First, continued positive feedback may eventually lead to demotivation and complacency on the part of the trainee. The relationship between self-efficacy and effort has been demonstrated to be both positive and linear. As self-efficacy for the task increases, effort devoted to the task also increases--which in most cases will subsequently lead to increased performance. The more a trainee believes she can complete a task, the more effort she will put toward completing it. For example, the trainee may continue to set higher goals. However, the relationship between self- satisfaction and effort is curvilinear. At early stages, increasing self-satisfaction with task performance results in increasing effort toward task performance. But eventually a threshold is reached at which higher self-satisfaction will be demotivating (Bandura & Cervone, 1983) and less effort will be expended. When a trainee is continually satisfied with her performance, she may begin putting in less effort because she has been making satisfactory progress with respect to learning and performance. She has already achieved her goal, and no longer needs to put forth effort toward achieving it. Control theory would suggest that the trainee will maintain the current goal, and put the extra effort toward other goals (Carver & Scheier, 1982). Therefore continued positive feedback would ultimately lead to increases in effort via self-efficacy mechanisms, yet lead to counteracting decreases in effort due to complacent satisfaction. This contradiction would suggest that perhaps giving only positive feedback to trainees Ieaming difficult tasks may not be the best feedback strategy. An additional issue concerns the role of a trainee’s attributions for performance in the effect of positive feedback on self-efficacy and selfvsatisfaction. Previously it was noted that attributions may be a limiting factor in the self-evaluation process (Bandura, 13 1991). When a trainee attributes positive feedback to her own capabilities in producing the successful performance, the relationship with feedback is strong for both self-efficacy and self-satisfaction. However, if the trainee attributes the positive feedback and successful performance to some external event, such as “a lucky break,” then the relationship is not strong (Bandura, 1977). There is no reason to believe the trainee will increase in self-efficacy for the task, or be satisfied with the performance. 80 positive feedback would tend to be most beneficial when attributions by the trainee are internal; the feedback would tend to be less beneficial if the attributions are external. Another issue that questions the “blanket” superiority of positive feedback is the objective difficulty of the task, and the trainee’s perception of the task difficulty. Both objective complexity and perceived complexity have been found to individually influence Ieaming and training performance (Maynard & Hakel, 1997). There are reasons to believe that feedback may influence the trainee’s perception of task difficulty. Negative feedback would make a task appear more difficult; positive feedback would make a task appear easier. Both objective task difficulty and perceived task difficulty may moderate the effects of feedback on Ieaming and performance. With respect to simple versus complex tasks, positive feedback and self- satisfaction may be more motivating for complex tasks; negative feedback and self— dissatisfaction may be more motivating for simple tasks (Bandura, 1991; Karoly, 1993; Waldersee & Luthans, 1994). This relationship is suggested because more difficult and challenging tasks would require more positive feedback in order for the trainee to sustain effort and motivation in the face of challenges, whereas for a simple task complacency would set in early and dissatisfaction would become an effective motivator. Another 14 study suggests that high self-efficacy and satisfaction may lead to greater effort and performance for tasks perceived as difficult, but not for those perceived as easy (Salomon, 1984). This relationship can be framed as one of “overconfidence” when a trainee has high self-efficacy for a task perceived as easy. In such a situation the trainee would not likely put forth much effort, which might harm the trainee’s performance, particularly if the perception of difficulty was misleading and the task was not as easy as it seemed. Advantaggs and Disadvantages of Negative Feedback Benefits of Negative Feedback. Control theory (Carver & Scheier, 1982) indicates that, similarly to positive feedback, giving trainees negative feedback has the capability to improve leamin g and training performance. According to control theory moderately negative feedback creates dissatisfaction, which is motivating to the trainee. In this manner, negative feedback indicates that a standard has not been met. Self- regulation literature suggests that when a standard is not achieved the lack of achievement leads to negative affect (Bandura & Cervone, 1983, 1986). Therefore, failure to meet the standard leads to negative affect in the form of self-dissatisfaction. Trainees attempt to reduce self-dissatisfaction by increasing their motivation and/or effort (Bandura & Cervone, 1986). Moderately negative discrepancies from a standard lead to motivation and goal commitment because they are generally perceived as surmountable. Large negative discrepancies lead to decreased self-efficacy and, ultimately, goal abandonment (Bandura & Cervone, 1983, 1986). Another advantage of providing trainees with negative feedback is that it will provoke metacognitive activity and enhance self-regulation. Self-regulation literature 15 (Carver & Scheier, 1990; Karoly, 1993; Latham & Locke, 1991; Lord & Levy, 1994) and error management training literature (Frese & Altmann, 1989; Ivancic & Hesketh, 1995) suggest that when a discrepancy from a standard is produced, the trainee will engage in self-monitoring and metacognitive reflection in an attempt to determine what produced the discrepancy and how it may be resolved. Negative feedback indicates that a standard has not been met, producing a discrepancy. When the discrepancy is encountered the trainee will engage in “controlled processing,” where task behavior is closely monitored with respect to the goal (Ivancic & Hesketh, I995; Salomon & Perkins, 1989; Schmidt & B jork, 1992; Wofford & Goodwin, 1990). This controlled processing leads to “mindful abstraction,” which is the “conscious and effortful abstraction of common elements between situations” (Ivancic & Hesketh, 1995, p. 108). “Mindful abstraction” can be considered as a type of metacognition, or self-regulation. Similarly, Carver & Scheier (1982) note that discrepancies lead to self-directed attention, or a shift of attention to the standard and standard-relevant behavior. This shift of attention leads to improved self- regulation. As noted previously, Kluger and DeNisi (1996) suggest that this is the manner in which feedback interventions work--by re-directing attention to self-regulatory processes. Lord and Levy (1994) indicate that learning occurs when the discrepancy is resolved. Thus, using negative feedback to evoke metacognitive activity and self- regulation on the part of the trainee will likely lead to improved Ieaming of the task. Limitations of Negative Feedback. The disadvantages of providing trainees with negative feedback are more intuitive than the disadvantages of positive feedback. Particularly if there is no established sense of self-efficacy (such as at the initial stages of learning), a trainee receiving negative feedback will experience decreased self-efficacy. l6 Prolonged negative feedback may decrease the trainee’s self-efficacy to the point where she no longer believes she will ever have the capacity to perform the task. At this point, the trainee withdraws all effort toward learning or performing the task. A situation of withdrawal from the task is the most important limitation of using negative feedback in training. Attributions are limiting to the presentation of negative feedback, as well as positive feedback, because they alter the nature of the expected relationships. When a trainee attributes negative feedback to her own capabilities in producing the unsuccessful performance, the relationship with feedback is strong for both self-efficacy and self- satisfaction (Bandura, 1977). Both self-efficacy and self-satisfaction will decrease. However, if the trainee attributes the negative feedback and unsuccessful performance to some external event, such as “bad luck,” then the relationship is not strong. There is no reason to believe the trainee will lose self-efficacy for the task, or be wholly dissatisfied with the performance (Ivancic & Hesketh, 1995). In this way, negative feedback would be detrimental when attributions by the trainee are internal; the feedback would not be detrimental if the attributions are external. Integration, Model DeveIOpment, and Hypotheses Recall that there is an inherent paradox in presenting exclusively positive feedback to trainees in order to enhance Ieaming and task performance. Positive evaluative feedback will increase trainees’ self-efficacy, which will lead to increased effort toward performing. However, positive feedback will also increase self—satisfaction, which will lead to strong effort at first but will later give way to complacency, leading to decreased effort toward Ieaming and performing. Effort cannot both increase and 17 decrease at the same time and be expected to have an overall positive effect on learning and performing. Therefore, exclusively positive feedback may not be the best method by which to enhance Ieaming and performance. While negative feedback has the potential to induce effort toward Ieaming and performance by creating self-dissatisfaction, it also has the potential to destroy self-efficacy and induce withdrawal from the task when prolonged. Thus, providing exclusively negative feedback is also not a strong method for enhancing leamin g and performing. However, a combination of positive and negative feedback could capitalize on the benefits of both, while avoiding the associated disadvantages. Relative to effort-based situations, Ieaming situations have characteristics that cause the perceived difficulty of a task to shift throughout training. Trainees beginning to learn a novel and difficult task will perceive it as more difficult, but as they gain experience and success with the task they will perceive it as less difficult. As a task is perceived as becoming easier the trainees will likely lose motivation to continue learning and practicing the task. However, receiving negative feedback would create the perception that the task has increased in difficulty. A shift in feedback from positive to negative would help compensate for the shift in the perception of task difficulty from more difficult to less difficult. Combining Positive and Negative Feedggk It is proposed that sequencing first positive and then negative feedback over the course of the training will help resolve some of the issues that limit either social learning theory or control theory from fully explaining the effects of feedback sign on the process that leads to learning and training performance. It is expected that the inclusion of both 18 positive and negative feedback will lead to greater overall benefit than either type of feedback provided separately. The following section summarizes the benefits of ramping from positive to negative feedback during the course of training. First, using both positive and negative feedback will allow the trainee to attain motivational benefits from both the linear efficacy/performance relationship and the curvilinear satisfaction/performance relationship. Positive feedback will provide an initial boost to self-efficacy, which will lead to increasing effort and performance. Negative feedback will create subsequent dissatisfaction, which will be motivating, rather than allowing trainees to become complacent with continued positive feedback and self- satisfaction. Second, by provoking metacognitive activity and self-regulation with negative feedback, Ieaming will increase. Performance is the only outcome factor addressed by the proponents of positive feedback. Third, literature suggests that failure feedback should be avoided early in a task, as attentional resources may then be devoted to off-task cognitions (Kuhl & Koch, 1984; Kluger & DeNisi, 1996; Mikiluncer, 1989) and self-efficacy will decline (Silver et al., 1995). Starting trainees by giving positive feedback will build their self-efficacy. Starting trainees with negative feedback would likely lower self-efficacy and lead to withdrawal from the task because “true” negative performance at the beginning of a difficult task will be represented by more extreme negative feedback, whereas later in the task it will more likely be represented by moderately negative feedback. In other words, negative discrepancies are larger early in skill acquisition relative to later in skill acquisition. Therefore, the ramped feedback is only recommended to proceed from positive to negative feedback, and not from negative to positive (Kozlowski et al., 1997). 19 Ramping trainees from positive to negative feedback may also aid in explaining the effect of positive and negative feedback on tasks perceived as having differing levels of difficulty. In fact, the combination of positive and negative feedback is expected to affect the perception of a task’s difficulty level. As previously discussed, positive feedback and self-satisfaction may only be motivating for complex tasks; negative feedback and self-dissatisfaction may only be motivating for simple tasks (Bandura, 1991; Karoly, 1993; Waldersee & Luthans, 1994). High self-efficacy and satisfaction may lead to greater effort/performance for tasks perceived as difficult, but not for those perceived as easy (“overconfidence”; Salomon, 1984). As the trainee gains practice on a task and receives initial self-efficacy boosts through positive feedback, the task will be perceived as easier. Negative feedback can be introduced when the task is beginning to be perceived as easier and complacency and overconfidence are setting in. The negative feedback would create dissatisfaction, thereby giving the trainee another boost of motivation. Also. the negative feedback would create the perception that the task is more difficult, thereby requiring more effort toward Ieaming and performance. Thus, negative feedback counteracts decreases in motivation in two different ways. An overall model of the process of feedback sign affecting the outcomes of learning and training performance will be presented and briefly explained. The constructs involved in the model will then each be described individually. Once the constructs have been identified, specific hypotheses will be made concerning direct effects, interactions, and changes over time. 20 Overall Model yd Construct Development The model shown in Figure 3 depicts the entire mediated and moderated process involved in feedback affecting Ieaming and training performance. There are five distinct sets of variables that make up the model: (1) individual difference variables, (2) independent variables, (3) process variables, (4) effort variables, and (5) outcome variables. These sets of variables create a four-step progression through the model. First, cognitive ability and mastery/performance orientation are identified as individual difference variables that may directly affect such process variables as self-efficacy and perceived task difficulty. While these individual difference variables are not of interest substantively in the model, they are identified so that they can be used as covariates in analyses. Second, feedback and objective task difficulty are identified as independent variables that will have direct effects on the process variables. Attributions are also an independent variable, but will serve as a moderator of the effects of feedback on self- efficacy and self-satisfaction. Third, the process variables--self—efficacy, self- satisfaction, and perceived task difficultyuwill primarily affect the effort variables, goal setting and time spent studying. Fourth, the effort variables are expected to affect the outcome variables, learning and training performance. The presentation of the constructs, and later, the hypotheses will follow this progression through the model. Individual Difference Constructs Cognitive Ability. Different levels of mental abilities and skills are brought to the training setting by each trainee. These individual differences can have important effects on training and transfer outcomes. While cognitive ability is not a central component of the model, these differences should be considered when evaluating the model. 2] .582 =ago .m ease llllll a ....... .. .I u IIIIIII . IIIIIIIIII M . . _ _ IIIIIIIIII J. . _ n _ _ . IIIIIIIII a a m _ _ . ESE:— . . _ flirt: "n .u : .aa ._ n . . _ . 5—305:— " n 02:07.0 ” n _ u x u .n : . . _ . . . u . u u 2.2... n . _ _ . . _ _ _ 22:25 _ . _ . u . _ . . _ — — — — — BO—uUIhh—uflm — — — u u u (L 9:233 _ w‘ n . n . _ *l 2.25 93:. _ _ _ _ . . _ . _ . _ _ _ _ _ _ _ . . . . _ _ . . . . . _ . _ . _ . . . _ _ _ _ . _ . 5.1.520 . . n n V _ u _ . 3.3.53.0.— . _ . _ . a 8:2 . _ . m 4 9.55.25 _ u . . . C a _ _ easiest... L. - . . . . . _ . " 9:239 “ n u u . u . . _ _ _ _ . . n _ 22.3.53 n u l__ _ _ . _ _ _ . .L p IIIIIII _ _ _ _ _ _ lllllllll mot—3...?» _ _ _ _ — IIIII L rII v _A— _ n“ lllL r llllll moss—:3, 3:0.— 2. _ IIIIIIIIII 2; .n =— - IIIIIIII mQ—Dflm 309:0 0‘ III mo_£a_._a> mmvuchn— a mu_e_a._a> 9.2.3— 2.33:0 12.—zit:— 22 Well-documented is the fact that general cognitive ability predicts training performance (Ree & Earles, 1992; Schmidt, Hunter, Outerbridge, & Goff, 1988). Learning outcomes may also be affected by a trainee’s cognitive ability. All things being equal, trainees with high cognitive ability would be expected to learn more and in a shorter period of time than trainees with low cognitive ability. This is because “attentional resources”—-which are largely determined by cognitive ability--are central to the skill acquisition process (Ackerman, 1992; Kanfer & Ackerman, 1989; Norman & Bobrow, 1975). Masteg/Perfonnance Orientation. Individuals who possess a mastery orientation are more likely to adopt goals focused on improving task learning (Ames & Archer, 1988; Elliot & Dweck, 1988; Boyle & Klimoski, 1996; Duda & Nicholls, 1992). Mastery-oriented individuals tend to experience positive affect, increased motivation, and active self-monitoring (Dweck & Leggett, 1988). Mastery-oriented individuals also tend to maintain motivation in a Ieaming environment, provided successful performance is construed by trainees as a skill that can be improved (Bandura, 1991). Performance-oriented individuals are instead concerned with demonstrating competence to some other person or persons. They adopt goals focused on perforrnin g the task successfully (Ames & Archer, 1988; Dweck & Leggett, 1988). Orientation toward performance may operate to suppress self-regulation and other cognitive processes that are stimulated by the adoption of a learning orientation (Schraw, Horn, Thomdike-Christ, & Bruning, 1995). Individuals with a performance orientation may be more likely to withdraw from the training task when faced with failure (Ivancic & 23 Hesketh, 1995), and may also have less self—efficacy (Martocchio, 1994). Both mastery and performance orientations appear to be predictive of task leamin g. Independent Variable Constructs Feedback Provision. Feedback is information that is made available to the trainee, separate from what information they get simply from interacting with the task. Feedback can be given with respect to knowledge the trainee has learned during the training, or with respect to task performance. Feedback can be framed in many different ways. Feedback can be purely descriptive, with only informational elements, or it can be interpretational, providing some explanation of that descriptive information. A typical type of interpretive feedback is feedback that is evaluative. Evaluative feedback can be normatively referenced, velocity referenced, or labeled according to (positive/negative) sign (Kluger & DeNisi, 1996; Kozlowski et al., 1997). Norrnatively referenced feedback occurs when trainees are told how they are doing, “compared to others who have performed this task.” The normative reference can also take on a positive or negative sign, depending on whether they are performing better or worse than others, as demonstrated in an experiment performed by Podsakoff and Fahr (1989). However, the feedback may or may not be based on trainees’ actual performance. One way that the descriptive aspect of feedback can vary is its accuracy. Usually veridical, accurate feedback is recommended (Hunter-Blanks, Ghatala, Pressley, & Levin, 1988; Lindsley, Brass, & Thomas, 1995), but in some instances non-veridical feedback will be preferred for its potential motivational gain (Kozlowski et al., 1997). Objective Task Difficulty. Task difficulty is similar to, yet not entirely redundant with task complexity. Task complexity has several different dimensions, as outlined by 24 Wood (1986). Component, coordinative, and dynamic complexity represent three dimensions within the complexity construct. Component complexity is defined by the number of acts or information cues involved in completing the task. Coordinative complexity is defined by the strength of relationships between the acts, information cues, and products of the task. Dynamic complexity is defined by changes that occur, to which an individual must adapt, that alter the nature of the relationships between the acts, information cues, and products of the task. Gardner (1982) has suggested that one element that may cause a task to be difficult is the amount of information processing required. This is consistent with the component complexity dimension of task complexity. Therefore, task difficulty can be thought of as a part of task complexity, and will share many of the same effects. Attributions. Attributions are the causes to which behavior is ascribed. The attributed causes may or may not be the actual causes of the behavior, but are what the individual believes are the causes. Attributions are generally classified along three lines: Locus, stability, and controllability (Weiner, 1985). Stability refers to whether the attributed cause is stable or unstable. For example, the difficulty of a task may be considered a stable, constant cause for performance but effort put into the task may be considered an unstable, changing cause. Controllability refers to whether the trainee perceives the attributed cause as being under her control. Using the same examples, effort is likely to be seen as a controllable cause of performance, while task difficulty is likely to be seen as an uncontrollable cause. The locus of attributions refers to whether the cause is considered to be a factor within the person (internal), or a factor within the environment (external). This intemal/external locus distinction is important to the 25 moderating effect of attributions on the relationship of feedback to self-efficacy and self- satisfactions. Silver and colleagues (1995) have found their most robust effects for intemal/external causes moderating the relationship between past performance and self- efficacy. Process Variable Constructs Self-efficacy. Self-efficacy is an individual’s beliefs about whether she can perform a task or behavior. In this way, self-efficacy can be thought of as competency beliefs one holds about oneself. While self-efficacy is most often directly relevant to a particular task, it can be generalized to other tasks and situations. Self-efficacy also changes over time and with experience on a task (Bandura, 1977). Past performance on a task is considered to be the best indicator of future performance (Gist & Mitchell, 1992; Silver et al., 1995). Thus, successful experiences typically lead to increases in self- efficacy; failures tend to undermine it. Traditional conceptualizations of self-efficacy frame the construct as little more than a simple intention for future performance based on past performance. Study participants are asked about their capability beliefs for a (typically) effort-based task. For example, in studies completed by Bandura and Cervone (1983, 1986) a participant would rate her efficacy for attaining a certain level of performance on an ergometer, which is an exercise device that measures physical effort at pushing and pulling arm levers. If the participant believes she can do well at the task, she will intend to do well, and will put forth more effort until she is no longer physically able. The belief that one can complete the task translates quite easily into an intention, and subsequently, into successful performance. There is a relatively direct relationship between effort and performance. 26 However, self-efficacy can also be conceptualized as a more complex psychological construct that captures a broader self-perception of competence. For tasks that are cognitively difficult rather than strictly effort-based this representation of self-efficacy as a self-perception of task-relevant competence is more appropriate than traditional representations of self-efficacy as simple intentions to put forth effort toward the task. Self-satisfaction. Self-satisfaction is an affective reaction to behavior, as opposed to self-efficacy which is a cognitive reaction to behavior. Satisfaction with oneself is a direct result of comparing one’s performance with one’s goal for that performance (Bandura & Cervone, 1983). When a goal is attained or surpassed, a person will experience the positive affect of satisfaction. When a goal is not attained, a person will experience the negative affect of dissatisfaction. Based on other moderating factors, this self-satisfaction or self-dissatisfaction can be either motivating or demotivating (Bandura, I991; Carver & Scheier, 1990). The motivating potential of these affective reactions will have a direct impact on the level of effort that the individual will put toward performance. Perceived task difficulty. Perceived task difficulty has been demonstrated to predict performance over and above that which is predicted by actual task difficulty (Maynard & Hakel, 1997). This construct represents the way the trainee views the task in terms of difficulty. While it is partly determined by the objective difficulty of the task, other variables also influence perceived task difficulty: A trainee’s ability, her experience with the task, and feedback she receives about how well she has performed the task. For example, a difficult task may not seem as difficult to someone who is very intelligent or someone who has handled such difficult tasks before. Conversely, an easy task may seem more difficult to someone who has received feedback indicating poor task 27 performance. In summary, perceived task difficulty is a construct related to but not the exactly the same as objective task difficulty. Effort Constructs Goal Setting. While there are many variables that can represent the effort construct, two will be reviewed here. The first is the level of self-set goals, which represents the effort the trainee intends to put toward task performance. Goal setting has been shown to augment training performance (Kluger & DeNisi, 1996). Obviously there are some potential moderators of this relationship, including ability and task difficulty. However, generally, if a trainee does not intend to put forth much effort she will not choose goals that would require her to put forth such effort. Time Sgent Studying. Goal setting is the typical effort variable chosen, but it tends to only reflect on performance. In situations where Ieaming is an important outcome, separate from performance, effort toward Ieaming should also be measured. For a difficult or novel task, there is a necessity for learners to study task information and materials prior to opportunities to practice or perform the task. The amount of time learners spend studying these task materials can constitute an indication of their effort to learn the task. Outcome Constructs LearnirLg and Training Performance. The outcome variable that is most salient in training studies is training performance. Training performance can be construed as a skill, or what the trainee does with respect to the task. Often forgotten is Ieaming itself as an outcome variable in training studies (Kozlowski et al., 1997). Leaming can be construed as knowledge, or what a trainee knows with respect to the task. Both training 28 performance and leaming--doing and knowing--are an integral part of the self-regulatory process, particularly for providing an outcome from which a feedback perception can be made. These outcomes are also important, however, in examining how performance varies as a result of manipulating feedback and the associated changes in other process variables involved in self-regulation. Mes—es Direct Effects The direct effects of this model are presented in Figure 4. These direct effects represent what is expected to result at the end of the feedback process. Three types of feedback are examined with respect to the model--only positive feedback (Positive), only negative feedback (Negative), and positive ramped to negative feedback (Positive/Negative). Because positive feedback has a demonstrated positive linear relationship to self-efficacy of trainees, trainees receiving any kind of positive feedback should experience higher self-efficacy than trainees not receiving positive feedback (Karl et al., 1993; Latham, & Locke, 1991). Also, the linear nature of the relationship allows self-efficacy accrued with positive feedback to be better maintained in the face of subsequent negative feedback, as when trainees receive Positive/Negative feedback. Thus, at the end of the feedback process Positive and Positive/Negative feedback should both produce higher self-efficacy than Negative feedback. Hypothesis 1a: Trainees receiving Positive feedback will experience higher self- efficacy than trainees receiving Negative feedback. Hypothesis 1 b: Trainees receiving Positive/Negative feedback will experience higher self-eflicacy than trainees receiving Negative feedback. 29 eucaEeeueon— «559;. mam—:34 ”£33m 3on 02:. wfizom .an EEEE a; 1933.3.— ..oBz seem 525 .4 23E 5.53.2:— =33££~am 9.75 3.8::— £5 a»; 228.30 guaaeaam 30 Self-satisfaction is related to positive feedback such that positive feedback produces increased self-satisfaction and negative feedback creates self-dissatisfaction (Podsakoff & Fahr, 1989). This is a linear relationship, similar to that of the relationship of feedback to self-efficacy. However, satisfaction accrued with positive feedback is unlikely to be maintained when faced with negative feedback, as when trainees receive Positive/Negative feedback. Positive feedback is expected to produce higher self- satisfaction at the end of the feedback process than the other types of feedback. Hypothesis 2a: Trainees receiving Positive feedback will experience higher self- satisfaction than trainees receiving Negative feedback. Hypothesis 2b: Trainees receiving Positive feedback will experience higher self- satisfaction than trainees receiving Positive/Negative feedback. Objective task difficulty and perceived task difficulty share a linear relationship. The task’s objective difficulty will directly affect the trainee’s perception of the task’s difficulty. As would be expected, a difficult task will usually be perceived as more difficult, and an easy task will usually be perceived as less difficult (Huber, 1985). The explication of this hypothesis is necessary to highlight the notion that objective task difficulty is not the only variable that influences perceived task difficulty. How trainees view the difficulty of the task is additionally affected by the feedback the trainees receive. A difficult task, followed by feedback indicating success, may not seem quite so difficult. Given failure feedback, trainees may perceive an easy task to be very difficult. Thus, positive and negative feedback can alter a trainees’ perception of the task’s difficulty, independent of how difficult the task actually is. 31 Hypothesis 3a: Trainees Ieaming a more difiicult task will perceive the task to be more difi‘icult than those trainees Ieaming a less difi‘icult task. Hypothesis 3b: Trainees receiving Negative feedback will perceive the task to be more diflicult than trainees receiving Positive feedback. Hypothesis 3c: Trainees receiving Negative feedback will perceive the task to be more difi‘icult than trainees receiving Positive/Negative feedback. The goals that trainees set for subsequent performance attempts will be directly influenced by self-efficacy (Bandura, 1988; Bandura & Cervone, 1983; 1986; Cervone & Peake, 1986). Logically, trainees who believe they can accomplish the task will set a high goal and trainees who do not believe they can accomplish a task will set a low goal (Bandura, 1991; Karoly, 1993; Locke, Frederick, Lee, & Bobko, 1984; Taylor, Locke, Lee, & Gist, 1984; Wood, Bandura, & Bailey, 1990). How satisfied trainees are with past performance will also directly influence the goals that trainees set for subsequent performance attempts. However, the effects in this case are curvilinear. If a trainee is very satisfied with her performance, there is no need for her to set a higher goal, whereas a trainee who is only moderately satisfied will likely increase the goal she sets (Podsakoff & Fahr, 1989). Also, trainees who are very dissatisfied will likely withdraw from the task and cease to set goals. Hypothesis 4a: Trainees with higher self-efiicacy will set higher goals. Hypothesis 4b: Trainees with high and low self-satisfaction will set lower goals; trainees with moderate self-satisfaction will set higher goals. The time trainees spend studying for performance attempts will increase as the trainees’ self-efficacy increases. While this construct has not typically been examined in 32 training literature, the logic leading to this hypothesis is relatively straightforward. If a trainee believes she can do something, she will spend more time preparing herself, whereas if she doesn’t think she can do it, she will not waste time preparing herself. However, trainees who are very satisfied with past performance will be unlikely to spend much time preparing for the task; they are at an acceptable level--why should they put in any more effort? Trainees who are moderately dissatisfied will increase their efforts to reach an acceptable level of performance. Trainees who are extremely dissatisfied will likely withdraw from the task and cease to spend time studying. Hypothesis 5a: Trainees with higher self-efficacy will spend more time studying. Hypothesis 5b: Trainees with high and low self-satisfaction will spend less time studying; trainees with moderate self-satisfaction will spend more time studying. Goal setting and time spent studying will directly affect the outcomes of Ieaming and training performance. The proposed relationships are positive and linear. When low goals are set, even if they are achieved, performance remains low. Higher goals must be set, and then achieved, for performance to reach high levels (Latham & Lee, 1986; Locke, Shaw, Saari, & Latham, 1981; Mento, Steel & Karren, 1987). As long as ability is controlled, the more time a trainee spends studying task material, the more she will learn about that task. The trainee is unlikely to learn much unless time is spent studying. Hypothesis 6: Trainees who set higher goals will achieve higher performance. Hypothesis 7: Trainees who spend more time studying will achieve higher Ieaming. The learning outcome will also directly affect the performance outcome. For a cognitively—based task where Ieaming is necessary to performance, there will be a linear 33 positive relationship between learning and performance. Regardless of all other motivational processes that may affect performance, if a trainee has not learned the material critical to performance, the trainee will not be able to perform the task. Hypothesis 8.“ Trainees who exhibit higher Ieaming will achieve higher performance. The process of the effects of feedback on Ieaming and training performance is not a simple process that can be adequately described by a model of direct effects and mediation. While this model (Figure 4) suggests the basic framework of the process, there are some important moderating relationships that can be identified. Interaction Effects The moderating effects proposed in this section will be overlaid on the previous model, and are presented in Figure 5. As in any complex process, there are a number of interactions among the variables in this model, particularly stemming from trainee attributions, perceived task difficulty. and objective task difficulty. First, the attributions the trainee makes concerning the reason for the positive or negative feedback may moderate the effect that feedback has on self-efficacy and self-satisfaction. As already explained, trainees will tend to make either internal or external attributions for the feedback they receive. When trainees receive positive feedback, making internal attributions is favorable to maintaining and/or boosting self-efficacy and self-satisfaction. However, when feedback is negative, making external attributions helps trainees to preserve self-efficacy and self-satisfaction. It is most beneficial when a trainee attributes positive performance to her own ability, but attributes negative performance to some external circumstance. 34 ace-«Estua— ”552,—. wet—.594 five—2 32682:— .n oSwE «5.22.5 OI” Eon—m 05:. ‘@ 3.32::— 5.58.2:— swan. 333.3.— 5233.5 flow I by :8... 258.25 gaaaeaam 2335:: 35 Hypothesis 9a: When feedback is positive, internal attributions will lead to higher self-efficacy than external attributions; when feedback is negative, internal attributions will lead to lower self-efficacy than external attributions. Hypothesis 9!): When feedback is positive, internal attributions will lead to higher self-satisfaction than external attributions; when feedback is negative, internal attributions will lead to lower self—satisfaction than external attributions. The perceived difficulty of the task will moderate the effects of self—efficacy and self-satisfaction on trainee effort. In both cases, perceptions of difficulty will moderate the strength of the relationships. Consider the relationship of self-efficacy to goal setting and time spent studying. As the trainee perceives the task difficulty to increase, the relationship between self-efficacy and effort will become stronger (Figure 6). Task Perceived More Difficult High Task Perceived Less Difficult Goal Setting and Time Spent Studying Low Low High Self-Efficacy Figure 6. Interaction between Perceived Task Difficulty and Self—Efficacy. 36 Specifically, when a task is perceived as more difficult, higher self-efficacy will lead to even higher goal setting and more time studying than when a task is perceived as less difficult. Similarly, lower self-efficacy will lead to even lower goal setting and less time spent studying when the task is perceived as more difficult. Hypothesis 10a: When a trainee perceives the task difliculty as higher, self- efiicacy will have a stronger efi‘ect on goal setting and time spent studying. Next consider the relationship of self-satisfaction to goal setting and time spent studying. As the trainee perceives the task difficulty to increase, the relationship between self-satisfaction and effort will become weaker (Figure 7). This is different from the expectation of the self-efficacy relationship. This interaction is complicated by the curvilinear relationship of self-satisfaction and effort. First, examine the initial half of the curvilinear relationship, where satisfaction increases from low to medium. Task Perceived Less Difficult High Task Perceived J More Difficult Goal Setting and Time Spent Studying Low Low High Self-Satisfaction Figure 7. Interaction between Perceived Task Difficulty and Self-Satisfaction. 37 When a task is perceived as more difficult, higher self-satisfaction will lead to lower goal setting and time studying than when a task is perceived as less difficult. Next, examine the latter half of the curvilinear relationship, where satisfaction increases from medium to high. When a task is perceived as more difficult, higher self-satisfaction will lead to higher goal setting and time studying than when a task is perceived as less difficult. In other words, only if a trainee is either very satisfied or very dissatisfied with performance on a more difficult task, will effort be higher than if the task were less difficult. When the task produces moderate levels of satisfaction less difficult tasks will receive higher effort. (It should be noted that this hypothesis is largely exploratory, as very little research exists in this area.) Hypothesis 101): When a trainee perceives task difficulty as higher, self- satisfaction will have a weaker effect on goal setting and time spent studying. The objective difficulty of the task has been demonstrated in past literature to moderate the relationship between effort and outcomes. While this moderator is not central to the current study, it may be useful to look for a replication of this established effect. As the difficulty of a task increases it requires more than simply putting in extra effort to perform well. Therefore the effort variable, goal setting, does not exhibit as strong of a relationship with the outcome variable, training performance. However, the opposite effect might be expected for the moderation of the effort to Ieaming relationship. As task difficulty increases, more time spent studying will allow greater improvements in Ieaming than when a task is less difficult. The more difficult a task is, the more study time that is needed to effectively learn the task. 38 Hypothesis 1 1 a: When task difficulty is high, goal setting will have a weaker effect on performance. Hypothesis 11b: When task difliculty is high, time spent studying will have a stronger efifect on Ieaming. Knowledge is also needed to create the link to performance. The Ieaming outcome variable is an additional moderator to the effort-performance relationship. As a trainee’s knowledge of the task increases, as demonstrated through Ieaming, the relationship between goal setting and performance will be stronger. Hypothesis 12: When Ieaming is high, goal setting will have a stronger efifect on performance. Effects of Feedback Over Time Because training takes place over an extended period of time and feedback can be changed during the course of training from positive to negative, changes over time in the effects of the feedback are particularly relevant to examine. The manner in which feedback affects some of the process variables differentially over time may additionally demonstrate the benefits of giving different feedback to trainees at different times during training. Self-efficacy will increase during early stages of training, then be maintained during later stages for trainees experiencing first positive, then negative feedback (Kozlowski et al., 1997). This represents a stronger effect than that of providing only negative feedback, as with negative feedback self-efficacy would decrease during early stages. Self-satisfaction would decrease during later stages of training for those receiving feedback sequenced from positive to negative. Because this would counteract the 39 complacency that would likely set in with the provision of only positive feedback, this also represents a stronger effect. When taken into consideration together, self-efficacy and self-satisfaction are most strongly affected by Positive/Negative feedback. The perceived level of task difficulty is expected to decrease over time for all feedback groups. However, Positive/Negative feedback will cause an increase in perceived difficulty following the introduction of negative feedback. This increase in perceived difficulty is expected to additionally create dissatisfaction and motivate greater effort. Goal setting and time spent studying would both be expected to increase more during later stages of training for feedback that is sequenced from positive to negative. The dissatisfaction with negative feedback would prompt this effect, which would not be present for trainees receiving only positive feedback, or for trainees receiving negative feedback who would have likely withdrawn effort by later stages of training. The outcomes of Ieaming and performance would also be expected to change during the training. As a result of these prior processes, the greatest increases over time for both Ieaming and performance would be expected for those trainees receiving positive, then negative feedback. Hypothesis 13: Positive/Negative feedback should have stronger efi‘ects over time on (a) self-ejficacy, (b) self-satisfaction, (c) perceived task difiiculty, (d) goal- setting, (e) time spent studying, (f) Ieaming, and (g) performance, relative to Positive or Negative feedback only. 40 METHOD Mn Overview. The present study incorporated a crossed 3 x 2 factorial design with repeated measures. One factor, task difficulty, included two levels: More Difficult and Less Difficult. The other factor, feedback, included three levels: Positive, Negative, and Positive/Negative. Each participant experienced eight training trials, in four blocks of two trials each, creating the repeated measure portion of the study. The Task. A task was chosen that allowed the researcher to manipulate its difficulty level. A PC-based simulated naval radar tracking task known as TEAMS/TANDEM provides high psychological fidelity for complex and difficult decision making and information processing tasks. This task was originally developed by the Naval Air Warfare Center Training Systems Division (NAWCTSD) and the University of Central Florida (UCF). Following extensive design modifications conducted in collaboration with a Michigan State University (MSU) research group, TEAMS/TANDEM (Version 8.10 has been developed as a dynamic, self-contained, and completely novel task environment. TEAMS/TANDEM allows researchers to explore the process of trainees learning a difficult task and developing adaptive expertise. TEAMS/ TANDEM was designed to include events that unfold in real time that can be scripted by the researcher. The researcher also has control over what information trainees may access, what decisions they need to make, and how the performance of these actions will be scored. 41 The design of the program to be used in the present study is the focus of the remainder of this section. This design will be referred to as the Tactical Action Game (TAG) to distinguish it from the main version that can be programmed to many different specifications. TAG began with the “operator” seated facing a simulated radar display on a PC screen. The operator’s “Own Ship” was situated in the center of the radar display, and represented the base to be defended from enemy attack. Numerous targets surrounded Own Ship on the radar display, to be identified and engaged by the operator. The operator identified each target by first clicking on it with a mouse, and then collecting information about the target from pull-down menus located in the upper right corner of the screen. Three engagement subdecisions were made by the operator: The target’s type, class, and intent. Each of these subdecisions was made by examining different cues on a pull-down menu that described different characteristics of the target. The operator learned these cue values and what they indicated about the target. Once these three subdecisions were made to identify the target, the operator engaged the target by either shooting or clearing it from the screen. This final engagement decision was also made by using a pull-down menu. Correct final engagement decisions were rewarded with a score of 100 points; incorrect decisions incurred a penalty of -100 points. Operators were able to determine why their engagement was correct or incorrect by holding down the right mouse button when engaging. This action provided the operator with information detailing the correct type, class, and intent subdecisions. To ensure that operators were selecting and engaging targets, a penalty was also incurred for allowing a target to penetrate a “defensive perimeter” located 10 nautical miles from Own Ship. This perimeter was 42 depicted as a shaded circle surrounding Own Ship. Allowing a target to enter this perimeter before it was engaged resulted in a loss of 100 points. TEAMS/TANDEM has the capability to record almost every action that trainees take within the context of the game. These records can be used as measurements of a wide variety of dependent variables. A feedback software program called FASTBACK was developed by the MSU research team as a companion to TEAMS/TANDEM. TEAMS/TANDEM allows the researchers to access the recorded performance information from the trials, but trainees typically only have knowledge of their total score. FASTBACK allowed the researcher to set up the simulation so that, following practice, the trainee could view any number of the recorded pieces of information. The feedback appeared after the trial on a sequenced set of screens that the trainee scrolled through. Participants Participants included 330 volunteers from courses at a large Midwestern university, and received class credit for their three hours of participation. Data from thirty-four participants were dropped prior to analyses for the following reasons: 8 participants experienced technological problems or experimenter errors; 8 participants did not follow directions or cheated on some portion of the experiment; 9 participants withdrew from the experiment early (1) or were missing substantial amounts of data (e. g., entire scales, or computer-recorded trial performance scores); 4 participants never spent any time studying the materials (I fell asleep), which evidenced a clear lack of motivation or understanding of the nature of the task; and 5 participants could not read or speak English well enough to understand the task. After completing some preliminary 43 analyses, 6 participants were identified as outliers on the goal setting measure. Throughout all eight trials of the experiment these six set a goal of 2500 points, the highest possible point total. While this goal was theoretically possible, in a practical sense it could not be reached. Thus these six participants may have not clearly understood the nature of the task or the goal setting exercise, and were dropped from all analyses. The final sample size used in analyses was 290. Appendix A presents a power analysis that indicated the need for approximately 250 participants for the current study design. Procedure First an introduction to the experiment was given, which included a brief description of the task to be completed. Next, participants were asked to sign a consent form, indicating that the expectations and risks of the experiment were adequately explained and their agreement to participate (Appendix B). Participants then took the working memory test and a mastery/performance orientation questionnaire. A demonstration of the TAG game was presented by the experimenter and each trainee had an opportunity to practice playing the TAG game during a five-minute familiarization trial. An additional purpose of this familiarization trial was to give the trainee a basis from which to make a judgment concerning what goal to set for the first training trial. The first of the eight training trials began. Before each of the eight trials, trainees were given three minutes during which they could study the materials presented on- screen which explained the functioning of the TAG game and which contained the listing of cue values they must learn in order to perform the task. Immediately following the study period, trainees were asked to set a goal for the upcoming trial and completed the goal commitment measure. The trainee then completed the five-minute training trial. At the conclusion of the trial, FASTBACK feedback screens appeared which allowed the trainee to view feedback information. After every two training trials, trainees completed several additional measures. Trainees completed the perceived difficulty, self-efficacy, self-satisfaction, and attributions measures, and had eight minutes during which the Ieaming test was administered. When all eight trials had been administered and all measures completed, the participants had finished the experiment. They were debriefed as to the nature of the experiment and questions they had were answered by the experimenter (Appendix B). Manipulations Feedback Sign. The participants were presented with score feedback from their actual performance on the simulation. However, in order to manipulate the sign of the feedback, they also received non-veridical, normative feedback preceding the score information. Participants received feedback that either stated that they were doing better than others who have previously performed the task or that they were not doing as well as others who have previously performed the task. Research indicated that a moderate negative discrepancy is well-represented by feedback indicating a performance score 14 to 19 percent below the goal, or below the average (Bandura & Cervone, 1986; Podsakoff & Fahr, 1989). A moderate positive discrepancy was presumed to fall in a similar range above the average performance score. Thus, a 16 percent discrepancy was chosen as a mean for both positive and negative feedback. The discrepancies over the four trial blocks varied slightly around this mean. Participants in the Positive feedback condition received the statement that they were doing better than (roughly) 66 percent of others 45 who have played TAG, following all eight of the training trials. Participants in the Negative feedback condition received the statement that they were doing worse than (roughly) 66 percent of others who have played TAG, following all eight of the training trials. Participants in the Positive/Negative feedback condition received the statement that they were doing better than 66 percent of others following the first four training trials. They then received the statement that they were doing worse than 66 percent of others following the last four trials. The feedback screens presented to participants as a part of the feedback manipulation are presented in Appendix C. Task Difficulty. The task was designed to vary in component complexity, or difficulty. The difficulty varied based on the amount of information processing required by the task. Thus, with respect to TAG, the number of cues needed to make each of the three engagement subdecisions reflected the necessary amount of information processing. There were two levels of task difficulty. The Less Difficult task involved 27 cue values to be learned (three values for each of three cues, under each of the three subdecisions). The More Difficult task involved 45 cue values to be learned (three values for each of five cues, under three subdecisions). Measures Working memorLtest. This test allowed analyses to control for those trainees who learn and perform better simply because they possess greater ability to remember basic information; it functions only as a covariate in the analyses. Kyllonen & Christal (1990) maintain that working memory is a construct similar, but not identical, to reasoning ability. Working memory is one of four sources that contribute to individual differences on cognitive tasks. The other three sources include breadth of declarative 46 knowledge, breadth of procedural knowledge, and processing speed. The task was designed to involve mostly information processing and cue memorization; hence, working memory capacity the most likely source of differences in cognitive ability relevant to Ieaming this task. These authors have identified and developed a number of different tests of working memory capacity. The one that has been investigated the most and is apparently the most successful measure of this memory construct is called the alphabet recoding task. The alphabet recoding task was modified from a version introduced by Woltz (1988). It is this modified version of the task that was used in the present study. The computer-administered task involved 10 items, with three practice items, each composed of three screens. The first screen presented the examinee with three randomly-chosen (with some constraints; see Kyllonen & Christal, 1990 for a full review) letters of the alphabet, such as “L P B.” The second screen presented either a positive or negative number, such as “+3,” indicating that the examinee should determine what letters of the alphabet are three letters after the original letters “L P B.” The third screen presented a multiple-choice selection of letter groupings, one of which would be the correct response, “O S E.” After each item was completed, the computer program presented right-wron g feedback to the examinee. Mastery/Performance Orientation. A l6-item questionnaire was used to measure the participants’ Ieaming and performance orientations (Button, Mathieu, & Zajac, 1996). Response options to the questionnaire items were on a five-point Likert-type scale ranging from “strongly disagree” (1) to “strongly agree” (5). The mastery orientation portion of this measure resulted in a coefficient alpha of .76, and the performance 47 orientation portion of this measure had an alpha of .79. This questionnaire is reproduced in Appendix D. Self-segoals. The trainees were asked to set a goal before completing each of eight training trials. The goals were based on what performance score they expected to attain on the upcoming trial. They were then asked to record on a sheet of paper the goal they had set for that trial. The format for recording goals is presented in Appendix D. Goal commitment. The goal commitment measure served to ensure that all conditions were equally motivated to attend to the goals they had set for themselves. Before each of the eight training trials, after trainees had self-set goals, a seven-item measure of goal commitment was taken (Hollenbeck, Williams, & Klein, 1989). Responses options were on a five-point Likert-type scale ranging from “strongly disagree” (1) to “strongly agree” (5). One item was dropped due to a low inter-item correlation with the other scale items. This 6-item scale demonstrated coefficient alphas ranging from .85 to .93 (Kozlowski et al., 1995, 1996), and is reproduced in Appendix D. Time Spent Studying. A manual containing the information trainees needed to learn was presented via computer screens. Trainees viewed a table of contents, from which they could then go directly to any page of the manual. Thus trainees could spend study time looking at the learning material they considered most relevant, and they could exit the manual as soon as they had finished studying. The computer recorded the amount of time (in seconds) each trainee spent looking at each page of the manual. A maximum time for studying the manual was imposed at three minutes. Prior experience with this task indicated that three minutes would not produce any ceiling effects with regard to time spent studying. Three of the pages in the manual were designated as 48 containing critical information that trainees needed to memorize. The total time a trainee spent looking at these three critical pages was computed to represent the time spent studying for each trainee. Self-efficacy. Following even-numbered trials, trainees completed an 8-item measure of self-efficacy. This measure assessed self-efficacy with a Likert-type scale rather than ratings of confidence about particular aspects of the task (Hysong & Quinones, 1997; Lee & Bobko, 1994). The self-efficacy measure was used here as a reflection of the conceptualization of self-efficacy as a complex self-perception, as discussed previously. Thus it can be viewed as a measure of “task-specific self- perceptions of efficacy.” Response options ranged from “strongly disagree” (1) to “strongly agree” (5). Coefficient alphas for this measure ranged from .89 to .95. The self-efficacy scale is presented in Appendix D. Self-satisfaction. Following even-numbered trials, trainees completed a 5-item measure of self-satisfaction. This scale is similar to one used by Podsakoff & Fahr (1989) but had been modified to reflect satisfaction rather than dissatisfaction. Likert- type response options were on a 13-point scale, ranging from “strongly disagree” (1) to “strongly agree” (13). One item was dropped due to a low inter-item correlation with the other scale items. Coefficient alphas for this 4-item measure ranged from .88 to .92. The self-satisfaction scale is reproduced in Appendix D. Attributions. Following even-numbered trials, trainees completed a 6-item measure developed for this research indicating the causes to which they attributed their performance. The attribution items were restricted to internal versus external locus. Response options were on a five-point Likert-type scale ranging from “strongly disagree” 49 ( l) to “strongly agree” (5). ). One item was dropped due to a low inter-item correlation with the other scale items. Coefficient alphas for this 5-item measure ranged from .61 to .76. This scale is presented in Appendix D. Perceived difficulty. Following even-numbered trials, trainees completed a 4- item measure of perceived task difficulty. This scale had response options on a five-point Likert-type scale ranging from “strongly disagree” (1) to “strongly agree” (5). The scale was modified from a version produced by Maynard and Hakel (1997), to focus the questions on task difficulty rather than task complexity. The measure reflected acceptable internal consistency reliability, with alphas ranging from .81 to .93. This scale is reproduced in Appendix D. Feedback Perceptions. At the end of the experiment, after completion of the final set of measures, participants completed a 3-item measure of their perceptions of the feedback they received throughout the experiment. These items asked participants how accurate and believable they felt their feedback was. This scale had response options on a five-point Likert-type scale ranging from “strongly disagree” (1) to “strongly agree” (5). and demonstrated internal reliability of .84. This scale is presented in Appendix D. Learning test. The trainees’ learning was assessed with a criterion-referenced multiple-choice test of cue value knowledge, as the test included an item for each cue needed to accurately perform the task. The number of items in the test varied by the difficulty condition that the trainees were in. The more difficult condition had a test of 45 items, reflecting the 45 cue values the trainees need to learn. The less difficult condition had a test of only 27 items, reflecting the 27 cue values necessary for learning in that condition. One item was dropped from the more difficult condition test, as the 50 response options did not contain a completely correct response. Coefficient alphas for this measure ranged from .77 to .84. The Ieaming test for the more difficult condition (45 items) is presented in Appendix D. Task performance. The engagement of targets reflected the most salient performance aspect of the task. To measure basic task performance, the score based on targets engaged correctly and incorrectly was calculated. Participants received 100 points for each target engaged correctly, and lost 100 points for each target engaged incorrectly. However, another part of performance was engaging targets before they crossed the defensive perimeter. Therefore, 100 points was also subtracted for each target that crossed the defensive perimeter. The cumulative total score for each training trial was used as the measure of task performance. 51 RESULTS Preliminary Analyses Goal Commitment. Measures of goal commitment were taken to ensure that trainees in all groups were equally committed to their goals, so that this could be excluded as a reason that results differed among groups. A repeated-measures ANCOVA was performed with ability, mastery orientation, and performance orientation as covariates. A test of between-subjects effects demonstrated no significant effect of goal commitment on either groups receiving different feedback or on groups receiving different levels of task difficulty (Table 1). However, tests of within-subjects effects demonstrated significant effects of goal commitment (Table 2) over time for both feedback (F=2.036, p=.01) and task difficulty (F=4.l74, p<.01). Trainees receiving Positive feedback were more likely to maintain a consistent level of goal commitment throughout training, whereas those receiving Negative feedback or Positive/Negative feedback decreased their level of goal commitment toward the end of training (Figure 8). Table 1. ANCOVAa Between Subjects Effects for feedback and task difficulty on goal commitment. Effect Sum of Squares df Mean Square F Independent Variables Feedback (FB) 8.866 2 4.433 1.875 Task Difficulty (TD) .852 l .852 .360 Interactions FB x TD 6.126 2 3.063 1.295 a Covariates included ability, mastery orientation, and performance orientation. * p < .05; **p < .01 52 Table 2. RM-ANCOVAa for feedback and task difficulty on goal commitment. Effect Sum of Squares df Mean Square F Independent Variables Feedback (FB) 3.921 14 .280 2.036** Task Difficulty (TD) 4.020 7 .574 4.174** Interactions FBxTD 2.721 14 .194 1.413 " Covariates included ability, mastery orientation, and performance orientation. * p < .05; **p < .01 4.0 ’5 en E t E bk Feedback E 3.6- \ {>— o \ c: s . . U \ \ __ _Posrtrve "‘ 3.5. . 8 O Negative Q ---- 3.4 1 ' 1 g . , O Pos/Neg 1 2 3 4 5 6 7 8 Trial Figure 8. Goal Commitment by Feedback Group 53 4.0 E Q) E E E u Task Diff 8 DLess Diff O __ 3.5 O . iééiééiéMomD‘” Trial Figure 9. Goal Commitment by Task Difficulty Group Trainees who were in the Less Difficult task condition were also more likely to maintain a consistent level of goal commitment while those in the More Difficult task condition decreased commitment to goals over time (Figure 9). Because of these differential effects between groups over time, goal commitment was used as a covariate in all analyses of the hypotheses. Feedback Perceptions. Measures of trainees’ perceptions of the feedback they received were taken to ensure that all groups perceived the feedback as being equally credible and believable. A factorial ANCOVA revealed that trainees receiving different levels of task difficulty did not perceive the feedback differently (F=3.50), but trainees receiving different feedback did perceive the feedback differently (F=l7.54, p<01; Table 3). Inspection of the means (without covariates) provided evidence that trainees 54 Table 3. ANCOVAa for feedback on feedback perceptions. Effect Sum of Squares df Mean Sgare F Independent Variables Feedback (FB) 36.766 2 18.383 17.544** Task Difficulty (TD) 3.665 ‘ 1 3.665 3.498 Interactions FB x TD .929 2 .464 .443 a Covariates included ability, mastery orientation, and performance orientation. * p < .05; **p < .01 receiving all positive feedback (M=3.37) perceived the feedback to be more believable than those receiving all negative feedback (M=2.74) and positive/negative feedback (M=2.54). The fact that the negative and combined feedback group perceived the feedback to be less believable may explain why the goal commitment of these groups fell significantly more than the positive feedback group. Due to the significant differences among feedback groups in the perceptions of the feedback’s credibility, this variable was used as a covariate in all subsequent analyses. RM-MANCOVA. A repeated-measures MANCOVA was performed to examine the overall effects that the feedback and task difficulty manipulations had on the entire set of dependent measures over time (Table 4). The MANCOVA demonstrated a significant effect for both feedback (F=2.24, p<.01) and task difficulty (F=25.89, p<.01). The manipulations were also found to have significant effects over time: feedback (F=3.49, p<.01), task difficulty (F=13.97, p<.01). Significant overall effects suggested that regression analyses exploring each of the proposed hypotheses were appropriate. Direct Effects Means, standard deviations, and correlations for all variables used in the regression analyses are presented in Table 5. Individual regression analyses were carried 55 Table 4. RM-MANCOVAa for feedback and task difficulty on dependent measuresb. Effect Pillais Trace df F Independent Variables Feedback (FB) .1 10 14, 540 2.24** Task Difficulty (TD) .403 7, 269 25.89** Trial Block .692 7, 273 8744““ Interactions FB x TD .113 14,540 2.32** Block x FB .164 14, 548 3.49** Block x TD .264 7, 273 13.97** Block x FB x TD .068 14, 548 1.37 " Covariates include ability, mastery orientation, performance orientation, goal commitment, and feedback perceptions. b Dependent measures include self-efficacy, self-satisfaction, perceived task difficulty, goal setting, time spent studying, training performance, and Ieaming. * p < .05; **p < .01 out for each of the specific hypotheses. A summary of the results presented in the following section is contained in Table 6. The variables included in the regressions represented a combination of the final two trial blocks. Process variables (self-efficacy, self-satisfaction, and perceived task difficulty) were averaged over trial blocks three and four. Effort variables (goal setting and time spent studying) and the performance outcome variable were averaged over trials six, seven, and eight3. The Ieaming outcome variable was the percentage of correct responses available from the second knowledge test completed at the end of trial four. For each categorical regression feedback was contrast coded to reflect the contrast called for in the hypothesis. 3 Trial five was not included because the Positive/Negative feedback group had not yet “experienced” the shift to negative feedback until after setting a goal and performing trial five. The first instance of negative feedback was presented following the performance of trial five. 56 dog—£8 E 82. 5: 2.3 2:. .2258 8:55 a .5852 5: Eu :5 632:; SUB—coo. .2: 2.. 8 E5555 2 032.5, 53.58 558 05. .wombacm 505:6 E com: 203 .2... 2.5% 52:58 05 .5 22258 528.5 .5852 832.? 53—58 m .2: 2:- . tann- n8.- 4%.. 2... 8o.- mm2-4 85% 855.88 .2 team.- a8.. N4...- 8... 2...- 45.2 3.4.. wees: .2 :8... “8.- 28. 8.. 2. 2.4.4 2.: 2.32m 25% as; .0. tan.- 2o.- 8o.- 48. 2... 4.84 358 2.20m .80 .2 :8“.- 25. 8o.- ..44.. .2. 2.. Gem 28.-5-28 .4. tang.- tca. .- 8..- tea. 8.. NZ 8.x 8.6525228 .2 5..- ....s..- 25.. 3&2. 42. me. on... 82.3.22 .2 .149... E..- . 5.- 48. N4... 8. Nam Esau-5 3.3 8285.. ._ . .2 ..- 4.... am... 8.. 42. 2.. 5m 205.558 .80 . o. .8. t8...- Roc 18m. SN. 2... 23 .8588“. V.2543... a *2... N2..- 2.... .8. mm... mm. 43 5.35.5 855.88 .w a8. 9....- .8.. Re. 48.- .4. 8.4 8.55.5 5as... 5 2... 4o..- o2..- oao. 8o. m2 mom 5...... a 8.. 48. 8o. 4.... 2....- -- -- 5.855 .5 .m 8.. :8... :8...- ooo. -- -- 2 2 .2825. 4 v.2335". .4 8.. :84.- tmow. .. -- $02 as 82.8% m 8248". .m 8.. 1.34. -- -- 92.8.. a... 8... N V3.5.4.5... .N 8.. -- -- .302 as S... . V338...”. .. m 4 m N . am :82 $85.22. 55053. E vow: 832:3 .8 mecca—250 use £53250 BEESm £5.22 .m 03:. 57 .8229... E 08: .0: 2.3 0.8 .8550 02.55 a .5858 .0: 20 :5 658:; x0388 .2... on. o. Ecowosto m. 032:; .6303. 5.58 of. 883...... 505:... E 08: 203 .2. anew x0338 on. .0 282.50 2.88:0 58050.. 832.2, x0308. m .2: 2:- 4 1&8. $.40- :20 N8.- ..2 T 48.- :SN. 88:88.. .2 .82. .42.- :20 .4...- *2 ..- 08. :88. 8.543 2 .8. 48. .82. :88. .8. m8. ...2 T 8.83m .88 2:; .o. :88. 2.4.0- :28. m8; 8..- .8. 1.2. 25.8 .80 .2 :84. .88.- 1. .m. 1.8.. .82.- :8.. 84.. 88.5....8 .4. 3.8. :88.- .z..2. 4... 8...- 8... 18.6.. 8.8.2.8...8 .2 8.. 8.. 28.4. :48. 2... $8.. 42. 82.852 .2 8.. 1.8.. 3.84m. 42. 54... 2... 3.8.85 84.4 8288. .2 8.. 1.84m. .82 .- :20 8... 8.5.5.80 .80 . o. 8.. c8. 8.. .8. ”8.888.. 8888”. .8 OO.— NNO. wmo. COSMEOEO oocaehotom .w 8.. vvor 5.8.520 x5822 .N- 8.. 2.5.4 .o 5.8...5 .5 .m 2 o. .8885. 4 8848”. .4 302 .m> woémom. m x0308”. .m 307:5; .m> won: N x0303“. .N 4302 .w> mom. _ x0308”— . ~— : o_ a w h 0 60.500. m 03....- 58 .8222: E :02. .0: 2.3 0:: .22.:00 00:52: a .5352 .0: E: :5 632:; 0.08.08: .2: 0:. 0. _a:0w0£5 m. 032.2, .0303: 550-. 2F 83.35:: 520:... E :8: 203 .2. 2.09% 0.03.000: 0:. :0 £22.50 505:... .5350. 332:; x0303: m 7:: 2:- .-. 8.— w— than. 8.— 2 30mm... :40.- co.— @— «*th. **mmv. **O©Nr co; m— «*mmv. **_OM. maor **_mv. co; :— *uvcv. **mmN. *mm_r **mNm. **onm. co. m— ou:aE5to: .w. wEEfiJ .2 8.82m .88 as; .c. w:_..0m :50 .m— xoaoEm-tom .4. :0..0&2.mm-:_uw .2 80.80.02 .m— 0.8....0 0.8% 48.88: .. . 525.5500 .50 . o— m:0_.:005m 0.03000"— . 5.22:0:0 00555.5: . 5.35:0 50.2.2 . 0...? . 0.85.0 8.4... . 2 0. 3:05:20. 4 x0308: . $02 .m> 3235:. m 0.03000: . 3.0280: .w> 5:. N x0308”. . 4302 .w> mom. _ x0308”. . .33.:00. m 03...: 59 Table 6. Summary of Hypotheses and Results. Summary of Results Hyp # Indep. Variable(s) Dep. Variable(s) Analyses Results la* Feedback Self-efficacy Categ. Pos > Neg 1b Regress. Pos/Neg = Neg 2a Feedback Self-satisfaction Categ. Pos = Neg 2b* Regress. Pos > Pos/Neg 321* Objective task Perceived task Categ. More Difficult > difficulty difficulty Regress. Less Difficult 3b-c Feedback Perceived task Categ. Non-Sig. difficulty Regress. 43* Self-efficacy Goal setting Regress. Sig. pos correlation 4b Self-satisfaction Goal setting Polynom. Sig. pos correlation Regress. 5a Self—efficacy Time spent Regress. Sig. neg correlation studying 5b Self-satisfaction Time spent Polynom. Sig. neg correlation studying Regress. 6* Goal setting Performance Regress. Sigios correlation 7* Time spent studying Learning Regress. Sig. pos correlation 8* Learning Performance Regress. Sig. pos correlation 9a Feedback Self-efficacy Mod. Sig. IV and Attributions Regress. sig. interaction 9b Feedback Self-satisfaction Mod. Sig. IV and Attributions Regress. non-sig. interaction 10a Self-efficacy Goal setting Mod. Sig. IV and Perceived task Time spent Regress. non-sig. interaction difficulty studying 10b Self-satisfaction Goal setting Mod. Sig. IV and Perceived task Time spent Regress. non-sig. interaction difficulty studying 1 1a Goal setting Performance Mod. Sig. IV and Objective task Regress. non-sig. interaction difficulty 1 1b* Time spent studying Learning Mod. Sig. IV and Objective task Regress. sig. interaction difficulty 12 Goal setting Performance Mod. Sig. IV and non-sig. Learning Regress. interaction l3a-g* Feedback Self-eff, Self-sat, RM- Sig. Mancova Time PTD, Goal setting, Mancova for Feedback over Time study, Learn, Perform Time * Hypothesis received support 60 Hypothesis 1. The first hypothesis suggested that trainees receiving any positive feedback would experience higher self-efficacy than those receiving only negative feedback. A categorical regression (Table 7) supported hypothesis 1a, such that Positive feedback trainees reported higher self-efficacy than Negative feedback trainees (AR2=.017, p<.05; 5:162, p<.01), after controlling for covariates“. Hypothesis 1b was not supported, as Positive/Negative feedback trainees did not report higher self-efficacy than Negative feedback trainees (B=-.083). Hypothesis 2. The second hypothesis predicted that higher self-satisfaction would result from trainees receiving only positive feedback than from trainees receiving any negative feedback. A categorical regression (Table 8) supported hypothesis 2b, such that Positive feedback trainees reported higher self-satisfaction than Positive/Negative Table 7. Categorical regression for the effect of feedback on self-efficacy. Step: Variable(s) R2 df A R2 Adf Ba 1: Individual Differences .305** 5, 281 -— -- Ability .107* Mastery Orientation .009 Performance Orientation -.]O4* Goal Commitment .445** Feedback Perceptions .082 2: Feedback Manipulation .323** 7, 279 .017* 2, 279 Feedback 1b .162** Feedback 3C -.083 3The BS refer to standardized regression coefficients associated with each step of the hierarchical regression. b Contrast coded variable set up to compare Positive feedback to Negative feedback. C Contrast coded variable set up to compare Positive/Negative feedback to Negative feedback. *p < .05; **p < .01 4 Covariates included Ability, Mastery Orientation, Performance Orientation, Goal Commitment, and Feedback Perceptions for all analyses. 61 Table 8. Categorical regression for the effect of feedback on self-satisfaction. Step: Variable(s) R2 df A R2 Adf Ba 1: Individual Differences .075** 5, 281 -- -- Ability .176** Mastery Orientation -.O71 Performance Orientation -.062 Goal Commitment . 148* Feedback Perceptions .096 2: Feedback Manipulation .l32** 5, 289 .057** 2, 289 Feedback 1b .034 Feedback 2c .236** “The Bs refer to standardized regression coefficients associated with each step of the hierarchical regression. b Contrast coded variable set up to compare Positive feedback to Negative feedback. C Contrast coded variable set up to compare Positive feedback to Positive/Negative feedback. *p < .05; **p < .0] feedback trainees (AR2=.057, p<.01; (3:236, p<.01), after controlling for covariates. However, although Positive feedback trainees reported higher satisfaction than Negative feedback trainees, this difference did not reach significance (B=.034). Thus, hypothesis 2a was not supported. Hypothesis 3. The first part of hypothesis 3 stated that more difficult tasks would be perceived as more difficult, and vice versa. A categorical regression (Table 9) supported hypotheses 3a, such that trainees in the More Difficult condition reported higher perceptions of task difficulty than trainees in the Less Difficult condition (AR2=.122, p<.Ol; [3:357, p<.01), after controlling for covariates. Hypothesis 3 also suggested that feedback would affect the trainees’ perception of the task’s difficulty, such that trainees receiving any negative feedback would have perceived the task to be more 62 Table 9. Categorical regression for the effect of task difficulty on perceived task difficulty. Step: Variable(s) R2 df A R2 Adf [3" 1: Individual Differences .121** 5, 281 -— -- Ability .006 Mastery Orientation -.049 Performance Orientation .202** Goal Commitment .203** Feedback Perceptions .l93** 2: Difficulty Manipulation .243** 6, 280 .122** 1, 280 Task Difficulty .357** “The BS refer to standardized regression coefficients associated with each step of the hierarchical regression. * p < .05; **p < .01 difficult. However, a categorical regression (Table 10) demonstrated no effect for feedback on perceived task difficulty (AR“=.001), after controlling for covariates. Thus, hypotheses 3b and 3c were not supported. Hypothesis 4. First, a positive relationship was expected between self-efficacy and goal setting. Self-efficacy was found to have a significant effect on goal setting (AR“=.149, p < .01; (3:463, p<.01), after controlling for covariates (Table 11). This supported hypothesis 4a, such that trainees reporting higher self-efficacy set higher goals. Second, a curvilinear relationship was expected between satisfaction and goal setting. Trainees were expected to set higher goals at moderate levels of satisfaction, and set lower goals at extremely high or low satisfaction. Self-satisfaction demonstrated a significant linear effect on goal setting (AR“=.O69, p<.01; [3:274, p<.01), after controlling for covariates (Table 12). However, the quadratic term did not explain any further variance beyond the linear term (AR“=.000). Thus, the curvilinear nature of hypothesis 4b was not supported; however, there was a demonstrated positive relationship between satisfaction and goal setting. 63 Table 10. Categorical regression for the effect of feedback on perceived task difficulty. Step: Variable(s) R2 df A R2 Adf B“ 1: Individual Differences .121** 5, 281 -- -- Ability .006 Mastery Orientation -.O49 Performance Orientation .202** Goal Commitment .203** Feedback Perceptions .193** 2: Feedback Manipulation .122** 7, 279 .001 2, 279 Feedback 1b -.O46 Feedback 3c .023 “The Bs refer to standardized regression coefficients associated with each step of the hierarchical regression. b Contrast coded variable set up to compare Positive feedback to Negative feedback. ° Contrast coded variable set up to compare Positive/Negative feedback to Negative feedback. * p < .05; **p < .01 Table 1 1. Regression equation for the effect of self-efficacy on goal setting. Step: Variable(s) R“ df A R2 Adf B“ 1: Individual Differences .133** 5, 281 -- -- Ability .l63** Mastery Orientation .018 Performance Orientation -.07l Goal Commitment .278** Feedback Perceptions -.185** 2: Self-Efficacy .282** 6, 280 .149** 1, 280 .463** “The Bs refer to standardized regression coefficients associated with each step of the hierarchical regression. *p < .05; **p < .01 64 Table 12. Regression equation for the effect of self-satisfaction on goal setting. Step: Variable(s) R2 df A R2 Adf a“ 1: Individual Differences .133** 5, 281 -- -- Ability .163** Mastery Orientation .018 Performance Orientation -.071 Goal Commitment .278** Feedback Perceptions -.185** 2: Self-Satisfaction .202** 6, 280 .069** 1, 280 .274** 3: Self-Satisfaction .203** 7, 279 .000 l, 279 .087 quadratic term “The Bs refer to standardized regression coefficients associated with each step of the hierarchical regression. * p < .05; **p < .01 Hypothesis 5. The hypotheses involving the process variables and time spent studying mirrored those involving goal setting. Self-efficacy was expected to demonstrate a positive relationship with time spent studying. However, self-efficacy was found to have a significant negative effect on time spent studying (AR“=.O76, p<.01; B=- .330, p<.01), after controlling for covariates (Table 13). Thus, hypothesis 5a was not supported, as the relationship between self-efficacy and study time was opposite of that expected. Those trainees who felt more capable (high self-efficacy) put less effort into studying. Satisfaction was expected to demonstrate a curvilinear relationship with study time, such that those trainees who were moderately satisfied would have spent more time studying than those who were extremely satisfied or dissatisfied. Like self-efficacy, self- satisfaction also demonstrated a significant negative effect on time spent studying (AR“=.033; B=-. 189, p<.01) after controlling for covariates (Table 14). The quadratic term explained no variance beyond the linear term (AR2=.OOO). Thus, hypothesis 5b was not supported, although there was a significant link between satisfaction and time spent studying. 65 Table 13. Regression equation for the effect of self-efficacy on time spent studying. Step: Variable(s) R2 df A R2 Adf B“ 1: Individual Differences .045* 5, 281 -- -— Ability -.103 Mastery Orientation .066 Performance Orientation -.011 Goal Commitment .058 Feedback Perceptions .136* 2: Self-Efficacy .121** 6, 280 .076** 1, 280 -.330** “The BS refer to standardized regression coefficients associated with each step of the hierarchical regression. * p < .05; **p < .01 Table 14. Regression equation for the effect of self-satisfaction on time spent studying. Step: Variable(s) R2 df A R2 Adf [3“ 1: Individual Differences .045* S, 281 -- -- Ability -.103 Mastery Orientation .066 Performance Orientation -.01 1 Goal Commitment .058 Feedback Perceptions .136* 2: Self-Satisfaction .078** 6, 280 .O33** 1, 280 -.189** 3: Self-Satisfaction .078** 7, 279 .000 l, 279 .071 (piadratic term “The Bs refer to standardized regression coefficients associated with each step of the hierarchical regression. * p < .05; **p < .01 66 Hypothesis find 7. Greater levels of effort were expected to improve outcomes. Higher effort demonstrated through goal setting was predicted to improve performance outcomes more than lower goal setting. Self-set goals were found to have a strong, significant effect on training performance (AR“=.600, p < .01; [3:83 1, p<.01), after controlling for covariates (Table 15). This supported hypothesis 6, such that trainees setting higher goals also achieved higher performance scores than their counterparts selecting lower goals. Higher effort, demonstrated through increased time spent studying, was predicted to increase learning outcomes. Time spent studying was found to have a significant effect on learning (AR2:01 1, p < .05; [3:1 17, p<.05), after controlling for covariates and for task difficulty“ (Table 16). This supported hypothesis 7, such that trainees who spent more time studying achieved greater learning. Table 15. Regression equation for the effect of goal setting on training performance. Step: Variable(s) R2 df A R“ Adf [3“ 1: Individual Differences .1 19** 5, 280 -- -- Ability .243** Mastery Orientation -.071 Performance Orientation -.094 Goal Commitment .207** Feedback Perceptions -.105 2: Goal-Setting .719** 6, 279 .600** l, 279 .831** “The BS refer to standardized regression coefficients associated with each step of the hierarchical regression. * p < .05; **p < .01 5 Task difficulty was used as a covariate in all regressions involving the learning outcome variable. The nature of the different difficulty levels of the task determined, in part, how much could be learned. 67 Table 16. Regression equation for the effect of time spent studying on learning. Step: Variable(s) R2 df A R“ Adf t3“ 1: Individual Differences .189** 5, 281 -- -- Ability .249** Mastery Orientation -.020 Performance Orientation —.086 Goal Commitment .331 ** Feedback Perceptions -.123* 2. Objective Task Difficulty .287** 6, 280 .O98** 1, 280 -.320** 3: Time Spent Studying .298** 7,279 .011* 1, 279 .117* “The BS refer to standardized regression coefficients associated with each step of the hierarchical regression. * p < .05: **p < .01 Hypothesis 8. Difficult information processing tasks such as this one require some level of learning to achieve adequate performance. Thus it was proposed that the learning outcome variable would positively affect the performance outcome variable. Learning was found to have a Significant effect on performance (AR“=.072, p < .01; 8:317, p<.01), after controlling for covariates and task difficulty (Table 17). This supported hypothesis 8, as trainees who learned more also performed better. Interaction Effects Hypothesis 9. Trainees making internal attributions about their performance were expected to have higher self-efficacy than those making external attributions, when they received positive feedback. However, when feedback was negative, trainees making internal attributions would have lower self-efficacy than those making external attributions. Similar effects were predicted for self-satisfaction. The observed interactions were all in the predicted direction, but only one reached significance (Tables 18-19). The regression step including both contrast-coded interactions for attributions and feedback on self-efficacy was not Significant (AR“=.010, .000) after controlling for 68 Table 17. Regression equation for the effect of learning on training performance. Step: Variable(s) R2 df A R2 Adf B“ 1: Individual Differences .119** 5, 280 -- -— Ability .243** Mastery Orientation -.O71 Performance Orientation -.094 Goal Commitment .207** Feedback Perceptions -.105 2. Objective Task Difficulty .394** 6, 279 .275** 1, 279 -.535** 3: Leaming .466** 7, 278 .072** l, 278 .317** “ The BS refer to standardized regression coefficients associated with each step of the hierarchical regression. * p < .05; **p < .01 Table 18. Regression equation for the interaction effect of feedback and attributions on self-efficacy. Step: Variable(s) R2 df A R2 Adf B“ 1: Individual Differences .305** 5, 281 -- -- Ability .107* Mastery Orientation .099 Performance Orientation -. 104* Goal Commitment .445** Feedback Perceptions .082 2: Independent Variables .368** 8, 278 .O63** 3, 278 Feedback 1" .144* Feedback 3c -.056 Attributions .254** 3: Interactions (and IVs) .379** 10, 276 .010 2, 276 Feedback 1 -.537 Feedback 3 .448 Attributions .276** Feedback 1*Attributions .685* Feedback 3*Attributions -.499 “The BS refer to standardized regression coefficients associated with each step of the hierarchical regression. “ Contrast coded variable set up to compare Positive feedback to Negative feedback. ° Contrast coded variable set up to compare Positive/Negative feedback to Negative feedback. * p < .05; **p < .01 69 Table 19. Regression equation for the interaction effect of feedback and attributions on self-satisfaction. Step: Variable(s) R2 df A R“ Adf [3" 1: Individual Differences .075** 5, 281 -- -- Ability .176** Mastery Orientation -.071 Performance Orientation -.O62 Goal Commitment .148 Feedback Perceptions .096 2: Independent Variables .179** 8, 278 .105** 3, 278 Feedback 1b .042 Feedback 2C .208** Attributions .259** 3: Interactions .l80** 10, 276 .000 2, 276 Feedback 1*Attributions .088 Feedback 2*Attributions .019 “The BS refer to Standardized regression coefficients associated with each step of the hierarchical regression. “ Contrast coded variable set up to compare Positive feedback to Negative feedback. “ Contrast coded variable set up to compare Positive feedback to Positive/Negative feedback. * p < .05; **p < .01 covariates and direct effects of feedback and attributions. However, the beta for one of the interactions was Significant (B=.685, p<.05). Attributions strengthened the relationship between feedback and self-efficacy; the interaction is graphed in Figure 10. A direct effect of attributions was found for both self-efficacy (B=.254, p<.01) and self- satisfaction (B=.259, p<.01), such that making internal attributions was associated with higher efficacy and satisfaction. Hypothesis 10. Self-efficacy was predicted to have a stronger effect on goal setting and time Spent studying when the task was perceived to be more difficult. There was no interaction between perceived difficulty and self-efficacy on goal setting or time Spent studying (AR“=.OO7, .004) after controlling for covariates and direct effects 70 >‘ Feedback 0 __ .. . 8 ‘ Poe/N on c a: 1 1 0 ..... g i.) 0 Negative 3 — — o m 0 ° Positive 1 2 3 4 5 6 Attributions Figure 10. Interaction of Feedback and Attributions on Self-Efficacy (Tables 20-21). Altemately, self-satisfaction was predicted to have a weaker effect on goal setting and time Spent studying when the task was perceived as more difficult. The interactions between perceived task difficulty and satisfaction on goal setting and time spent studying were also not significant (AR“=.005, .009) after controlling for covariates and direct effects (Tables 22-23). However, there were significant direct effects for perceived task difficulty on goal setting (B=-.237, -.321, p<.01). The more difficult a task appeared, the lower the goals the trainee set. There was also one direct effect for perceived task difficulty on time spent studying (B=.167, p<.01). The more difficult a task appeared, the more time the trainee spent studying. 71 Table 20. Regression equation for the interaction effect of self-efficacy and perceived task difficulty on goal setting. Step: Variable(s) R2 df A R2 Adf B“ 1: Individual Differences .133** 5, 281 -- -- Ability .l63** Mastery Orientation .018 Performance Orientation -.O71 Goal Commitment .278** Feedback Perceptions -.185** 2: Independent Variables .320** 7, 279 .187** 2, 279 Self—Efficacy .336** Perceived Difficulty -.237** 3: Interactions .327** 8, 278 .007 l, 278 Efficacy*Perceived -.46O Difficulty “The BS refer to Standardized regression coefficients associated with each step of the hierarchical regression. * p < .05; **p < .01 Table 21. Regression equation for the interaction effect of self-efficacy and perceived task difficulty on time spent studying. Step: Variable(s) R2 df A R2 Adf B“ 1: Individual Differences 045* 5, 281 -- -- Ability -. 103 Mastery Orientation .066 Performance Orientation —.01 1 Goal Commitment .058 Feedback Perceptions .136* 2: Independent Variables .127** 7, 279 .082** 2, 279 Self-Efficacy -.279** Perceived Difficulty .096 3: Interactions .l3l** 8, 278 .004 l, 278 Efficacy*Perceived -.322 Difficulty “The BS refer to standardized regression coefficients associated with each Step of the hierarchical regression. * p < .05; **p < .01 72 Table 22. Regression equation for the interaction effect of self-satisfaction and perceived task difficulty on goal setting. Step: Variable(s) R2 (If A R7 Adf B“ 1: Individual Differences .133** 5, 281 -— -— Ability .l63** Mastery Orientation .018 Performance Orientation -.O71 Goal Commitment .278** Feedback Perceptions -.185** 2: Independent Variables .282** 7, 279 .149** 2, 279 Self-Satisfaction .l68** Perceived Difficulty -.321** 3: Interactions .287 8, 278 .005 1, 278 Satisfaction*Perceived .279 Difficulty “The BS refer to standardized regression coefficients associated with each step of the hierarchical regression. * p < .05; **p < .01 Table 23. Regression equation for the interaction effect of self-satisfaction and perceived task difficulty on time Spent Studying. Step: Variable(s) R2 df A R“2 Adf B“ 1: Individual Differences .045* 5, 281 -- -- Ability -.103 Mastery Orientation .066 Performance Orientation -.Oll Goal Commitment .058 Feedback Perceptions .136* 2: Independent Variables .100** 7, 279 .055** 2, 279 Self-Efficacy -. 134* Perceived Difficulty .167** 3: Interactions .108** 8, 278 .009 1, 278 Satisfaction*Perceived -.374 Difficulty “The BS refer to Standardized regression coefficients associated with each step of the hierarchical regression. * p < .05; **p < .01 73 Because perceived task difficulty was only affected by objective task difficulty, and it seemed to operate as simply a weaker version of objective task difficulty, these four regressions were run using objective task difficulty to create the interaction. Only one regression was significant in this case (Table 24). Objective task difficulty interacted with self—efficacy in its effect on goal setting, after controlling for covariates (AR“=.019, p<.01, B=-.725, p<.01). However, this did not lend support to the original hypothesis, as the interaction was in the opposite direction of what was expected. Task difficulty was proposed to strengthen the relationship between efficacy and goal setting, but in this study it was observed to weaken the relationship (Figure 1 1). Table 24. Regression equation for the interaction effect of self-efficacy and objective task difficulty on goal setting. Step: Variable(s) R2 df A R2 Adf B“ 1: Individual Differences .133** 5, 281 -- -- Ability .l63** Mastery Orientation .018 Performance Orientation -.O71 Goal Commitment .278** Feedback Perceptions -.185** 2: Independent Variables .417** 7, 279 .284** 2, 279 Self-Efficacy .313** Objective Difficulty -.396** 3: Interactions (and IVs) .436** 8, 278 .Ol9** 1, 278 Self-Efficacy .797** Objective Difficulty .301 Efficacy*Objective -.725** Difficulty “ The Bs refer to standardized regression coefficients associated with each step of the hierarchical regression. * p < .05; **p < .01 74 3000 2000~ DD 1000‘ .E *5 m 7:: 0 O 0. Task Diff ° More Diff "000 , . . 1 , 0 Less Diff 0 1 2 3 4 5 (5 Self-Efficacy Figure 1 1. Interaction of Self-Efficacy and Objective Task Difficulty on Goal Setting Hypotheses 11 and 12. Goal setting was predicted to have a weaker effect on performance when a task was more difficult. The observed interaction between difficulty and goal setting was not significant (AR“=.OOO) after controlling for covariates and direct effects (Table 25); hypothesis 11a was not supported. Time spent Studying was predicted to have a stronger effect on learning when a task was more difficult. The interaction between difficulty and study time was in the predicted direction and significant (AR“=.012, p < .05; B=.480, p<.05), after controlling for covariates and direct effects (Table 26). A graphical representation of this interaction is shown in Figure 12. A direct effect of task difficulty was found for both performance and Ieaming (B=-.182, -.362), such that the more difficult the task, the lower the performance. 75 Table 25. Regression equation for the interaction effect of goal setting and objective task difficulty on training performance. Step: Variable(s) R2 df A R2 Adf B“ 1: Individual Differences .119** 5, 280 -- -- Ability .243** Mastery Orientation -.O71 Performance Orientation -.094 Goal Commitment .207** Feedback Perceptions -.105 2: Independent Variables .742** 7, 278 .623** 2, 278 Goal Setting .734** Objective Difficulty -.182** 3: Interactions .742** 8, 277 .000 1, 277 Goal*Objective .000 Difficulty “ The BS refer to standardized regression coefficients associated with each step of the hierarchical regression. * p < .05; **p < .01 Table 26. Regression equation for the interaction effect of time spent studying and objective task difficulty on learning. Step: Variable(s) R2 df A R2 Adf B“ 1: Individual Differences .189** 5, 281 -- —- Ability .249** Mastery Orientation -.020 Performance Orientation -.086 Goal Commitment .331** Feedback Perceptions -.123* 2: Independent Variables .298** 7, 279 .109** 2, 279 Time Spent Studying .117* Objective Difficulty -.362** 3: Interactions (and IVS) .311** 8, 278 .012* 1,278 Time Spent Studying -.273 Objective Difficulty —.515** Study*Objective .480* Difficulty “ The BS refer to standardized regression coefficients associated with each step of the hierarchical regression. * p < .05; **p < .01 76 120 100* 80‘ Learning (Percentage correct) 4o. . T35: D!” o More Diff 0 . . 0 Less Diff -1oo o 100 200 Time spent Studying Figure 12. Interaction of Time Spent Studying and Objective Task Difficulty on Learning Table 27. Regression equation for the interaction effect of goal setting and learning on training performance. Step: Variable(s) R“ df A R2 Adf B“ 1: Individual Differences .1 19** 5, 280 -- -- Ability .243** Mastery Orientation -.O7l Performance Orientation -.O94 Goal Commitment .207** Feedback Perceptions -.105 2. Objective Task Difficulty .394** 6, 279 .275** 1, 279 -.535** 2: Independent Variables .763** 8, 277 .370** 2, 277 Goal Setting .695** Learning .175** 3: Interactions .763** 9, 276 .000 1, 276 Goal*Leaming .087 “The BS refer to standardized regression coefficients associated with each Step of the hierarchical regression. * p < .05; **p < .01 77 Goal setting was expected to have a stronger effect on performance when leamin g is high. However, the observed interaction was not significant (AR“=.OOO) after controlling for covariates, task difficulty, and direct effects of learning and goal setting (Table 27). Effects of Feedback Over Time Hypothesis 13. In general, Positive/Negative feedback (relative to Positive or Negative feedback only) was expected to have Stronger effects over time on all of the dependent variables: self-efficacy, self-satisfaction, perceived task difficulty, goal- setting, time spent studying, learning, and performance. The initial RM-MANCOVA (Table 4) demonstrated Significant effects for feedback over time on this dependent variable set. Univariate tests Showed that the feedback effect was Significant for self- efficacy and self-satisfaction. Mean residual values (after removing the effects of covariates) were plotted for each hypothesized variable at the end of each of the four trial blocks. The apparent trends over time are described in the remainder of this section. Self-efficacy was proposed to increase early with positive feedback, while those with early negative feedback would decrease in efficacy. Over time, efficacy would be maintained when negative feedback was introduced to trainees who received early positive feedback. This pattern was observed more clearly in the Less Difficult condition (Figure 13). After four trials, efficacy was similar for the Positive and Positive/Negative feedback groups, and higher than for the Negative feedback group. After eight trials the Positive/Negative feedback group had maintained efficacy at a level lower than the Positive group, but higher than the Negative group. The unexpected finding here was that Negative feedback trainees did not decrease in efficacy. These trainees simply 78 Less Difficult Condition —D— Positive — -O - Negative - - fi- - -Positive/Negative " l .5 j I Time 1 Time 2 Time 3 Time 4 More Difficult Condition —-D— Positive — -O — Negative ._ _ _ _ .0. _ 22.-.8; ........ - - At - - Positive/Negative _ _. V-.,—_ __ __ A.-- _ A. :. .— —1.5 t l Time 1 Time 2 Time 3 Time 4 Figure 13. Self-Efficacy over Time (Residuals) 79 started with lower levels of efficacy, but they did increase throughout the experiment despite receiving continued negative feedback. In the More Difficult task condition, the pattern was similar, but the Positive/Negative feedback group experienced more of a drop in efficacy following the introduction of negative feedback. At the end of eight trials, however, their efficacy was lower than that of the Negative group. AS expected, over time satisfaction of the Negative feedback group was lower than that of the Positive feedback group. The expected drop in satisfaction for the Positive/Negative group was most clearly observed for the More Difficult task, but was also evident in the Less Difficult condition (Figure 14). For the first half of the trials, Positive/Negative trainees reported satisfaction levels that mirrored the Positive trainees. Following the introduction of negative feedback, their satisfaction levels dropped to mirror the Negative trainees. Again, the unexpected finding was that satisfaction increased for the Negative feedback group. In fact, by the end of eight trials, the Positive/Negative group reported lower satisfaction than that of the Negative group, particularly in the Less Difficult condition. Perceived task difficulty did decrease over time as expected (Figure 15), for the Less Difficult task, and the decline was uniform for all feedback groups. For the More difficult condition, as expected, the Positive/Negative trainees did perceive an increase in difficulty following the introduction of negative feedback. However, the Positive trainees experienced a Similar increase in perceived difficulty; it is not clear why this result was observed. Goal setting and time Spent studying were both expected to increase most during later trials for the Positive/Negative feedback group. Goal setting increased fairly 80 Less Difficult Condition 7 5 3 1 A ----- /:‘i .- __ _ib1 —D—Positive ’-”’ """"""" Anna-”R --O-Negative '1 - - A - ~ Positive/Negative -3 -5 -7 l l Time 1 Time 2 Time 3 Time 4 More Difficult Condition 7 5 _ , 3 _ , —D—Positive - -O- Negative - - A - - Positive/Negative -7 t I Time 1 Time 2 Time 3 Time 4 Figure 14. Self-Satisfaction over Time (Residuals) 81 Less Difficult Condition —D— Positive - -O - Negative - - A - - Positive/Negative -1.5 ”2.5 T I Time 1 Time 2 Time 3 Time 4 More Difficult Condition 15- _- mw___-zflfin- ,, —D-— Positive - '0- Negative -O.5 -* :— ._ —M______A - - A - - Positive/Negative -l.5- A- : A . - - -_.. -2.5 t A Time 1 Time 2 Time 3 Time 4 Figure 15. Perceived Task Difficulty over Time (Residuals) 82 uniformly for all three groups in the Less Difficult condition in later trials. The expected effect was somewhat apparent in the More Difficult condition (Figure 16). Between trial blocks 3 and 4, the Positive and Negative groups actually decreased their goal setting, while the Positive/Negative group slightly increased their goal setting. However, at the end of all trials, all three groups were setting approximately the same level of goals. Time spent studying was found to decrease rather than increase over time for all groups under Less Difficult conditions (Figure 17). Decreases in study time were uniform across feedback conditions, but Positive trainees finished with higher levels of studying. In the More Difficult condition, Negative feedback trainees decreased in time spent studying during later trials, while Positive and Positive/Negative trainees experienced slight increases. Positive/Negative trainees were clearly spending more time studying at the end of six and eight trials. As a result of these prior processes unfolding over time, it was proposed that Positive/Negative trainees would experience the greatest increases in learning and performance. For the Less Difficult condition, it was Negative trainees who achieved increases in learning, with the Positive/Negative trainees achieving similar levels of learning (Figure 18), but were slightly decreasing. Positive trainees decrease in learning over time, as was expected. However, in the More Difficult condition, Positive/Negative trainees did see the greatest gains in learning. In this condition Positive trainees finished with a higher level of learning, but the level is maintained, rather than increased. Performance is similar for all feedback groups in both task difficulty conditions (Figure 19). However, Positive feedback trainees began to plateau or decrease in performance relative to the Negative and Positive/Negative groups. 83 Less Difficult Condition 800 600 400 200 —-D— Positive - O - Negative -200 - - A - - Positive/Negative -400 -600 -800 . . Time 1 Time 2 Time 3 Time 4 More Difficult Condition —D— Positive - -O - Negative - - A - - Positive/Negative Time 1 Time 2 Time 3 Time 4 Figure 16. Goal Setting over Time (Residuals) 84 Less Difficult Condition 100 60 20 —D— Positive N + ‘4 - '0 - Negative -20 - . . . an - - -A- - - Positive/Negative ~60 - 100 u : Time 1 Time 2 Time 3 Time 4 More Difficult Condition 100 60 _ __ __ __ .Li .,____ __. 2 —-D—Positive - '0- Negative -20 - - ~ - W“ V" “m -~ -- — —- ‘—— - - 1A - - Positive/Negative -60 - , ,_-, .477 -2 -100 u u Time 1 Time 2 Time 3 Time 4 Figure 17. Time Spent Studying over Time (Residuals) 85 Less Difficult Condition —-D—Positive - -O - Negative - - t - - Positive/Negative -5 -10 -15 Time 2 Time 4 More Difficult Condition 20 15 - — — ~ 10 - k - —* 5 _ _ , W, fi_________ W" ,__ ,___ —D—Positive — 0- Negative 0 - T— _ fl“ — _ fl _ fl - - fi- - - Positive/Negative -5 , T'.T'.":’.-r :- n ism-“**ggi'; ' . . -10 - v — ~ -15 : Time 2 Time 4 Figure 18. Learning over Time (Residuals) 86 Less Difficult Condition —U-— Positive - '0 - Negative _200 - - t - - Positive/Negative -400 -600 -800 , , Time 1 Time 2 Time 3 Time 4 More Difficult Condition —-0— Positive - '0 - Negative - - -A- - - Positive/Negative Time 1 Time 2 Time 3 Time 4 Figure 19. Performance over Time (Residuals) 87 Integrated Summary Model The proposed model depicted a process by which evaluative feedback affected the self-evaluative process, which in turn influenced effort, which then affected outcome variables. To provide an integrated summary of the results uncovered through the hypothesis tests, a modified model was formed. This model included all variables that had significant effects when tested by hypotheses. A test of this path model was performed using hierarchical regressions. First a path model of the direct effects is presented. Second, a path model including both direct effects and interaction effects is presented, and changes in direct effects from the first model to the second model will be noted. Also, the unhypothesized direct effects of attributions will be included in this second model. Figure 20 depicts the direct effects in the modified model. As noted in the first two hypotheses, Positive feedback had a positive effect on self-efficacy6 (Table 7) and on self-satisfaction7 (Table 8). Self-efficacy and self-satisfaction were together entered into a regression with goal setting and a regression with time spent studying. Self-efficacy demonstrated a significant positive effect with each effort variable, but self-satisfaction was no longer a significant predictor (Tables 28 and 29). Self—efficacy and self- satisfaction operated similarly; however, self-efficacy had stronger effects than self- satisfaction. Time spent studying demonstrated a significant path to learning (Table 16) and learning together with goal setting had significant paths to performance (Table 30). 6 This effect is only for the contrast between Positive feedback and Negative feedback. 7 This effect is only for the contrast between Positive feedback and Positive/Negative feedback. 88 .382 QEEsm £85m 885 .8 223m ma: 9.35% 3on 3:: 55.5%; 2% 9:5ou €2.33. 8:.«Eueuoa— ”£55.; .335H 5% wfizom :80 89 Table 28. Regression equation for the effect of self-efficacy and self-satisfaction on goal setting. Step: Variable(s) R2 df A R2 Adf Ba 1: Individual Differences .133** 5, 28] -- -- Ability .l63** Mastery Orientation .018 Performance Orientation -.O7l Goal Commitment .278** Feedback Perceptions -.185** 2: Independent Variables .286** 7, 279 .153** 2, 279 Self-Efficacy .414** Self-satisfaction .077 3The [35 refer to standardized regression coefficients associated with each step of the hierarchical regression. * p < .05; **p < .01 Table 29. Regression equation for the effect of self-efficacy and self-satisfaction on time spent studying. Step: Variable(s) R2 df A R2 Adf [3“ 1: Individual Differences .045* 5, 281 -- -- Ability -. 103 Mastery Orientation .066 Performance Orientation -.01 1 Goal Commitment .058 Feedback Perceptions .136* 2: Independent Variables .l22** 7, 279 .077** 2, 279 Self-Efficacy -.30l ** Self-satisfaction -.O46 aThe Bs refer to standardized regression coefficients associated with each step of the hierarchical regression. * p < .05; **p < .01 90 Table 30. Regression equation for the effect of goal setting and Ieaming on performance. Step: Variable(s) R2 df A R2 Adf [3“ 1: Individual Differences .119** 5, 280 -- -- Ability .243** Mastery Orientation -.O7l Performance Orientation -.094 Goal Commitment .207** Feedback Perceptions -.105 2: Independent Variables .749** 7, 278 .630** 2, 278 Goal Setting .761** Learning .205** 3The BS refer to standardized regression coefficients associated with each step of the hierarchical regression. * p < .05; **p < .01 Figure 21 presents the complete summary model, adding direct and interaction paths for attributions and task difficulty. Attributions had relatively strong direct paths to self-efficacy and self-satisfaction (Tables 18 and 19). Additionally, one path was significant for the interaction between attributions and feedback on self-efficacy (Table 18). With the inclusion of these paths, however, the direct path from feedback to self- efficacy changed from positive and significant to negative and non-significant. The introduction of the interaction and direct effects may reveal a suppressor relationship among these variables. Objective task difficulty was found to interact with self-efficacy in predicting goal setting (Table 24). Objective task difficulty was also found to interact with time spent studying in predicting learning (Table 26). When this interaction was introduced it also changed the direct path from time spent studying to learning, such that the path coefficient changed from positive and significant to negative and non-significant. Again, this change may be evidence for a suppressor relationship. 91 .382 bag—5m owe—aEeU ._N 2:3”. 3.3.25 i2. **Nm: *5“. “.333... ma... «Eh—55 «:25 95:. 553.325 Eom wEand 5K: **mN. 3:5:283m ”___—:9; **eh. 95.9w =30 23.232 92 In addition to the direct effects represented in Figure 21, indirect effects can be calculated for goal setting, time spent studying, learning, and training performance. The direct effect of self-efficacy on goal setting (.78) is of much greater magnitude than the indirect effects of feedback and attributions on goal setting (.34). However, these indirect effects on goal setting are not insubstantial. The indirect effects on time spent studying (- .15) are less than half of the magnitude of the indirect effects on goal setting. The indirect effects on learning are considerably smaller (.04) than the indirect effects on either of the effort variables. Total indirect effects on training performance are of greater magnitude (-.29) than of those on learning. The indirect effects on performance are negative due to the strong interaction effect of task difficulty on self-efficacy and the strong direct effect of task difficulty on Ieaming. Calculation of these indirect effects demonstrate that the effects of feedback on the self-regulation processes represented by this model diminish as they are mediated by increasing numbers of variables. However, the total indirect effects on the final outcome variable, performance, are still of substantial magnitude (-.29). The model presented in Figure 21 represents an integrated summary of the significant effects from individual hypothesis tests. The resulting summary model is remarkably similar to the model as it was originally proposed. The main differences from the proposed model are the direct effects of attributions, the lack of self-satisfaction effects, and the lack of perceived task difficulty effects. These results will be discussed further in the following section. 93 DISCUSSION This study intended to explore the effects of evaluative outcome feedback on Ieaming and performance for cognitively difficult tasks. Two meta-theories in the literature have directed research in this area: social cognitive theory and control theory. Social cognitive theory proposes that positive feedback is necessary when learning difficult tasks in order to boost self-efficacy. Self-efficacy in turn provides the mechanisms for learning and performance improvement by enhancing effort toward these outcomes. Social cognitive theory thus suggests a loop wherein positive feedback leads to increases in training outcomes, which then provide further positive feedback. Control theory alternately specifies negative feedback as the motivating force behind increased learning and performance. Low self-satisfaction and metacognition produced by negative feedback will drive the trainee toward greater effort, Ieaming and performance. Contribution of Ramped Feedback These two theories have opposing predictions of how feedback will effect increases in training outcomes. Social cognitive theory would recommend positive feedback, while control theory would advocate negative feedback. However, an important element of training is that it takes place over time. These two apparently competing theories can both contribute to the process of increasing Ieaming and performance, which takes place over time. The benefits of positive feedback as predicted by social cognitive theory would be most beneficial during the early stages of learning a difficult task. Positive feedback would give trainees an early boost in efficacy to help them avoid becoming discouraged by early failures. The benefits of negative feedback as 94 predicted by control theory would be most beneficial during later stages of training, when adequate efficacy for the difficult task has been established. At this point, prompting self-regulation and metacognition through dissatisfaction with learning and performance discrepancies will be essential. Thus, the two seemingly different lines of research can be reconciled when the effects of feedback over time are considered. The present study effectively combined these two theories into a model of Ieaming and training performance. This study predicted specific changes in how feedback would affect the process of increasing training outcomes. An innovation of this study was to take advantage of those predicted changes by shifting the presentation of feedback over time from positive to negative. The best evidence of the success of this strategy occurred in the More Difficult task condition, as was expected. Some of the results presented in the previous section demonstrate the beneficial effects of ramping feedback from positive to negative over the course of training. In Figure 19 one can see that, for a More Difficult task, Positive feedback produces greater performance at early and middle stages of the task, yet performance plateaus and actually begins to decrease at later stages of the task. However, when feedback is shifted from Positive to Negative midway through a task, performance is better maintained. Early gains in performance are maintained with the positive feedback presented early in the task, but complacency does not set in, as feedback shifts to negative. Positive/Negative feedback also demonstrated the greatest increase in learning over time for a More Difficult task (Figure 18). 95 The greater gains in outcomes for Positive/Negative trainees in the More Difficult condition can be traced to the antecedent effort and process variables. Goal setting does not decrease for this group in later training trials (Figure 16), and this group also maintained their study time (Figure 17). This group became much less satisfied with their performance following the shift to negative feedback (Figure 14), which affected their levels of effort accordingly. The only unexpected effect was the large drop in self- efficacy following the introduction of negative feedback (Figure 13). However, this did not seem to adversely affect the pattern of results unfolding over time for the outcome variables, and self-efficacy seemed to operate in a manner similar to self-satisfaction insofar as it prompted increases in effort. Contribution of Process Model Much of feedback research implies direct effects of feedback on performance outcomes. This research is based on feedback that is typically descriptive and functions as knowledge of results (KOR) feedback. However, even KOR feedback is invariably evaluated in some fashion, by either trainees or trainers. Identifying the effects of evaluative feedback on the underlying motivational process that leads to Ieaming and performance will contribute to the feedback literature. The feedback presented in this study was evaluative and had its impact on training outcomes not directly, but through a motivational process. This study attempted to model the process by which evaluative feedback impacts training outcomes for cognitively difficult tasks. Literature suggests that self-efficacy, self-satisfaction, and goal setting are variables included in the motivational process by which feedback affects performance in effort-based tasks (i.e. Bandura, 1991; Locke & 96 Latham, 1990). To model this process for a more cognitively-loaded task, the model was elaborated to include a knowledge—based effort variable (time spent studying) and a knowledge-based outcome (learning). These variables parallel the traditional effort and outcome variables, goal setting and performance. Thus there are two tracks by which self-efficacy and self-satisfaction have effects: Through goal setting to performance, and through time spent studying to learning. By incorporating these knowledge-based variables, the traditional feedback model becomes a more complete explanatory mechanism, particularly for cognitively-loaded tasks. Learning played an important role in the model because it affects, yet is a separate construct from, performance. Learning mediated the effect of time spent studying on performance. Time spent studying is a new way to examine effort, particularly effort toward learning. The relationship between time spent studying and learning was strengthened for tasks that are more difficult. This result intuitively makes sense, as more difficult tasks typically involve more cognitive effort, and an increase in such cognitive effort will be more necessary in order to achieve the desired outcome (Ieaming). Task difficulty usually affects the effort to performance relationship by weakening it. However, such an interactive effect was not found between goal setting and performance in this study. The cognitive nature of the task may be such that this interaction would not be expected to hold as it does in more purely physical effort-based tasks. In physical effort tasks, the more difficult the task, simply increasing effort will not lead to increased performance as it would in an easier task. But in a difficult cognitive effort task, the interaction is instead found between the knowledge-based effort and outcome variables, and greater effort is more likely to pay off with increased learning, which enhances performance. 97 Unexpected Results One unexpected result was the similarity in functioning of the process variables, self-efficacy and self-satisfaction. Self-efficacy was expected to have positive linear effects on the effort variables and self-satisfaction was expected to have curvilinear effects. Instead, they each produced the same effects: Positive linear relationships with goal setting, and negative linear relationships with time spent studying. A confirmatory principal components factor analysis with varimax rotation yielded two separate factors for these items. So while they are factorially separate constructs and have only a correlation of .55 (p<.01), self—efficacy and self-satisfaction are functioning in parallel. Additionally, in the summary model (Figure 21), when combined with self-efficacy in regression analyses, self-satisfaction no longer has significant direct effects on the effort variables. These results suggest that while there is a conceptual difference between self- efficacy and self-satisfaction, they do not appear to operate through different motivational mechanisms. The process may need more time to unfold and reveal the separation of these constructs. A literature review uncovered little information to formulate a hypothesis concerning the effect of self-efficacy on time spent studying. In light of this lack of information, self-efficacy was proposed to have similar positive effects on both effort variables-~goal setting and time spent studying. Opposite results were found; those with higher self-efficacy set higher goals but spent less time studying. This result is consistent with the expectation that overconfidence would lead to complacency resulting from continued positive feedback. 98 Contrary to what was expected, the perceived task difficulty variable was not affected by feedback. Perceived difficulty also did not cause any interactions with the process variables, thus becoming little more than a manipulation check for the task difficulty variable. Perceived task difficulty functioned in this study as a weaker version of objective task difficulty (see correlations in Table 5), and its weakness is likely the reason it did not interact with efficacy and satisfaction. It is unclear why feedback did not affect perceived task difficulty. Perhaps the discounting evidenced in the perceptions of feedback played a role here. Boundary Conditions One element of the present study that should be kept in mind when interpreting the results is the difficulty of manipulating evaluative feedback while also providing veridical process KOR. This study attempted to manipulate the sign of normative outcome feedback. Separating out this aspect of the feedback from other descriptive, evaluative, and process aspects of the feedback was a challenge that was met with only moderate success. While pilot participants reported finding the feedback believable, the feedback perceptions from the study sample were less optimistic. Participants who were given positive feedback were significantly more likely to report believing that the feedback they were given was accurate; those who received either negative or shifting feedback were more skeptical of its accuracy. (This also likely contributed to the decrease in goal commitment for these two groups.) Part of the lack of believability may be attributed to design factors. Participants could have been instructed more clearly as to the extremely difficult nature of the task, and warned that many capable participants perform poorly while beginning to learn the 99 task. Participants also could have been informed that many people experience significant performance improvements about halfway through the task training, thus making a shift in feedback halfway through the task more realistic (e. g. if they are suddenly doing much worse than others, they must not have experienced the improvements that others did). However, part of the lack of believability is likely due to human nature. Many college students have become accustomed to excelling in whatever ability-related tasks they perform. For such people, to be told that they are performing better than 66 percent would seem like negative feedback; that they are performing worse than 66 percent just does not seem plausible to them. Also, people generally like to believe that they are “above average”; to consistently perform at a below average level is something most people would prefer to not believe about themselves. Despite the potential discounting of the negative feedback, results were reasonably consistent with the hypothesized model of the process by which feedback affects learning and performance. An alternative explanation for the feedback perceptions is that participants did not really consider whether the feedback was believable or not until they were prompted to do so by the questionnaire items. Only at this point did those who received negative feedback make this “external attribution” that they did not believe the feedback to be veridical. In this study participants receiving negative feedback were significantly more likely to make external attributions (r=.l33, p<.05) for their performance. Making external attributions may allow the trainee to maintain a reasonable level of efficacy and satisfaction, and continue to put effort toward performance. 100 Another boundary condition involves the time spent studying and Ieaming measures. The links in the model involving Ieaming may be even stronger than were observed, as ceiling effects within the learning measure may have weakened its apparent relationship to time spent studying and performance. However, the ceiling effects are not due to a flawed measure; the knowledge test was designed to tap every piece of information that trainees needed to know to perform the task. Those trainees who reached the ceiling on this measure did so because they acquired all of the knowledge possible for the task. Similarly, the links between time spent studying and its antecedent and consequent variables may have been weakened by the floor effects present in the measurement of this effort variable. Again, this does not reflect a design flaw, as a floor effect represents those trainees who spent the minimum possible time studying (zero seconds). A caveat that always must be noted when interpreting results is the nature of a lab study. This study was completed entirely in the laboratory, necessitating (and enabling) the manipulations previously discussed. An issue central to the use of a lab study is the sample. The participants in this study were all college students completing the experiment for course credit, and over 90 percent were between the ages of 18 and 23. Another central issue is the setting. These participants were unlikely to have experienced the same motivation as trainees learning a difficult task relevant to their job. Thus, the generalizability of these results to a field setting are not as appropriate as if the study had been conducted in a field setting with a sample of trainees. However, the purpose of this study was to use existing theory in the self-regulation and feedback literature, which has been based on both lab and field studies, to predict what could happen in a laboratory 101 situation. Cook and Campbell (1979) note that multiple replications of a study will enhance the study’s external validity. Additionally, in a field setting the manipulations in this study would not have been appropriate. Without manipulating the sign of the feedback, many of the hypothesized relationships could not have been explored. Without manipulating the difficulty of the task, the parameters of the model’s effectiveness could not have been explored. Implications for Researclgnd Practice The present study offers an innovative approach to presenting feedback in training, such that evaluative aspects of the feedback change over time. This shifting of feedback from positive to negative during the course of training demonstrated promising results. However, little research exists that involves changing feedback over time (i.e. Bandura & Jourden, 1991). Thus, the way in which the shift in feedback should occur is an issue to be further explored. Three important questions arise when considering the rampin g of feedback throughout training: (1) When is the appropriate time to shift the feedback? (2) How gradual or abrupt should the shift be? and (3) What kind of positive feedback should be presented initially in order to effect increases in efficacy? The first two questions are related. The present study changed the feedback halfway through the eight-trial training experiment. However, the plateau effects expected from Positive feedback and the beneficial effects expected from Positive/Negative feedback were only beginning to manifest themselves at the end of the eight trials. Perhaps a longer experiment would have allowed the process to play out more fully and would have strengthened the observed effects. The optimal time to introduce negative feedback would appear to be when the benefits of positive feedback 102 begin diminishing. The present study also abruptly switched from positive feedback that was set at an average of 16% above the norm to feedback that was 16% below the norm. A more gradual introduction of negative feedback might have different effects on efficacy and satisfaction. The basis of this research is that very difficult tasks will inevitably produce negative feedback during early stages. If positive feedback is necessary in early task learning, future research needs to determine the best method of providing that feedback. The third question from above refers to whether the positive feedback provided to boost efficacy should be veridical or non-veridical. The present study demonstrates one option: Providing non-veridical normative information in addition to veridical descriptive feedback. In this way, descriptive feedback that might receive a negative evaluation by the trainee can be supplemented with positive evaluative feedback from the trainer. Another option for providing initial positive feedback is to find a source of feedback other than performance--one that is both veridical and positive. Measuring knowledge- based constructs (time spent studying and Ieaming) may provide a means by which veridical positive feedback can be given to trainees early in training. Trainees will be increasing their learning from the very beginning of the training, and this feedback would be more positive in nature. These and other related questions concerning the administration of shifting feedback need to be addressed in future research. Unexpected results uncovered in this study also point to opportunities for future research. First, task withdrawal was predicted for trainees receiving negative feedback, but was not observed. There was no prior specification for when this withdrawal effect might have set in. The present study, again, may not have been long enough to observe 103 this effect. However, trends were that Negative feedback trainees were increasing in self-efficacy and satisfaction. One post-hoc hypothesis for why trainees receiving continued negative feedback did not withdraw from the task is that they chose an additional reference point from which to base their self-evaluations and intentions toward effort. In the present study the trainees may have reverted to velocity feedback (Kluger & DeNisi, 1991; Kozlowski et al., 1997), wherein they made self-evaluations based on their own past performance and whether they achieved their own goal. Exploring the effects of different sources of feedback may help to determine the manner in which trainees combine the various sources of descriptive and evaluative feedback that are available from a task. Future studies should ask trainees to report the different sources of feedback they are using to make their attributions and evaluations. Second, the roles of some variables in the model need to be further explicated. As noted earlier, self-efficacy and self-satisfaction were established as separate constructs but were observed to operate in functionally similar ways. Future research may uncover distinct characteristics of these two constructs. The attributional interactions proposed in this study were in the expected direction, but most did not reach significance. This may have been the result of low reliability for the measure. Efforts should be made to refine attributional measurement scales. Direct effects, however, were present and (as might have been expected) internal attributions were associated with high efficacy and satisfaction. The present study examined only the internal versus external locus of attributions. Additional dimensions of attributions, particularly controllability, should be explored in future feedback research. 104 The implications of this research for training practitioners are broad. The capabilities of embedded training systems make possible almost any means of providing feedback. This study suggests that trainers need to begin taking advantage of the benefits these systems can offer in order to provide the best possible training environment. Embedded systems can monitor and produce a wide assortment of feedback indicators, which are likely to vary in their evaluative impact when presented to the trainee. With these systems, trainers can readily manage the task of changing feedback to fit the changing needs of learners throughout the training process. Conclusion The present study provides initial support for the practice of providing trainees with positive feedback at the early stages of a difficult task in order to boost their efficacy, and then transitioning to negative feedback in order to stimulate additional learning and performance increases. As technology for embedding feedback into the workplace becomes more powerful, and many different kinds of feedback are available to present trainees throughout their training, it becomes increasingly important that researchers and practitioners be aware of the impacts that providing such feedback will have on training outcomes. With more and more training involving the Ieaming of difficult and complex tasks, past notions of appropriate feedback provision may need to be altered in order to best leverage the learning process over time. The model presented here represents a first step toward identifying how feedback can be presented differentially over time in order to motivate trainees to learn as much and perform as well as they can during training. 105 APPENDICES 106 APPENDIX A Power Analysis All power analyses are based on effect size estimates and required sample size estimates listed in Cohen (1988). With use of the sample sizes listed below, the effects sizes will be detected with 80 percent power for a one-tailed test ((1:05). Direct Effects Hla & 1b: Feedback Sign to Self-Efficacy r = 06-22 (Nease, Mudgett, & Quinones, 1997) r = .31-.60 (Mudgett & Quinones, 1997) r = -.10 (Podsakoff & Fahr, 1989) The effect sizes range from small to large, but tend to center around medium (r z .26), requiring 102 subjects. H2a & 2b: Feedback Sign to Self-Satisfaction r = .l9-.46 (Mudgett & Quinones, 1997) r = —.55 (Podsakoff & Fahr, 1989) These effect sizes range from medium to large (r z .40) suggesting a sample size of 37. H3a: Objective Task Difficulty to Perceived Task Difficulty r = .34 (Maynard & Hakel, 1997) Again, this medium effect size suggests sample size of 56. H3b & 3c: Feedback Sign to Perceived Task Difficulty No effect size for this relationship was found in the literature. The best estimate that can be made would be based on the relationship between cognitive ability to perceived task difficulty, = -.28 (Maynard & Hakel, 1997). This relationship would be expected to have a similar effect size and in the same direction. This is a medium effect size, requiring 85 subjects. H4a: Self-Efficacy to Goal Setting r = .29 (Bandura & Jourden, 1991) r = .52 (Wood, Bandura, & Bailey, 1990) r = 41-65 (Wood & Bandura, 1989) r = .69 (Podsakoff & Fahr, 1989) r = .48 (Bandura & Cervone, 1986) r = .73 (Bandura & Wood, 1989) While most of these effects sizes would be considered large (r z .50), a meta-analysis by Hysong and Quinones (1997) suggests that the type of self-efficacy measure that will be used in the present study generates somewhat lower effect sizes, more closely corresponding to a medium effect size (r z .30). A medium effect size would necessitate a sample of 68. 107 H4b: Self-Satisfaction to Goal Setting r = .52 (Bandura & Cervone, 1986) r = .22 (Podsakoff & Fahr, 1989) This medium effect size (r z .37) would require 47 subjects. H5a: Self-Efficacy to Time Spent Studying r = .01-.29 (Brown, 1996) The majority of the effect sizes reported in Brown (1996) were in a range from .14 to .29. This small to medium effect size (r = .23) would require 153 subjects. H5b: Self-Satisfaction to Time Spent Studyirg r = .15-.33 (Brown, 1996) This small to medium effect size (r z .24) would require 1 19 subjects. H6: Goal Setting to Performance r = .37 (Kluger & DeNisi, 1996) r = .50 (Wood & Bandura, 1989) r = .39 (Bandura & Jourden, 1991) r = .73 (Podsakoff & Fahr, 1989) r = .43 (Wood, Bandura, & Bailey, 1990) r = .77 (Bandura & Wood, 1989) The large effect sizes (r z .50) indicated here would require 22 subjects. H7: Time Spent Stuging to Learning r = .07-.47 (Brown, 1996) This link is not well established in the literature, and in this one study exhibited effect sizes from small to large (r = .10-.50). I will estimate the effect size to be medium which, would require 94 subjects. H8: Learning to Training Performance r = .40 (Brown, 1996) This medium to large effect size suggests sample size of 37. 108 Interactions Power to test mode] interactions will be assessed using the following formulas found in Cohen ( 1977). The first will be used when there is no information on covariates, the second will be used to partial out covariates. L1-R2 L1- 2 =—(—2——""¥+w+u+1 N=—(2—fl§i+w+u+z+l RAB—RA RAB—RA Power is specified at .80 and alpha is set at .05 for a one-tailed test. H9a: Attributions Moderating Feedback to Self-Efficacy Feedback Sign to Self-Efficacy: r z .26 Cognitive Ability to Self-Efficacy: r z .02 (Brown, 1996) Mastery/Performance to Self-Efficacy: r z .37 (Brown, 1996) RA2 = .07 R32 = .03 (conservatively estimated to be small) RC2 = .14 L=7.85 w=1 u=1 z=2 N = 199 H9b: Attributions Moderating Feedback to Self-Satisfaction Feedback Sign to Self-Satisfaction: r z .40 Cognitive Ability to Self-Satisfaction: r == -.07 (Brown, 1996) Mastery/Performance to Self-Satisfaction: r x .30 (Brown, 1996) RA2 = .16 RB2 = .03 (conservatively estimated to be small) RC2 = .09 L=7.85 w=1 u=1 z=2 N = 193 H 10a: Perceived Task Difficulty Moderating Self-Efficagv to Goal Setting Self-Efficacy to Goal Setting: r =- .30 RA2 = .09 RB2 = .03 (conservatively estimated to be small) L = 7.85 w = l u = l N = 233 H10a: Perceived Task Difficulty Moderating Self-Efficacy to Time Spent Studying Self-Efficacy to Time Spent Studying: r z .23 RA2 = .05 R32 = .03 (conservatively estimated to be small) L = 7.85 w = 1 u = l N = 244 H10b: Perceivedfiisk Difficulty Moderating Self-Satisfaction to Goal Setting Self-Satisfaction to Goal Setting: r = .37 RA2 = .14 R32 = .03 (conservatively estimated to be small) L = 7.85 w = 1 u = l N = 220 109 H10b: Perceived Task Difficulty Moderating Self-Satisfaction to Time Spent Studying Self-Satisfaction to Time Spent Studying: r z .24 R,2 = .06 R32 = .03 (conservatively estimated to be small) L = 7.85 w = 1 u = 1 N = 241 H1 la: Objective Task Difficulty Moderating Time Spent Studying to Learning Time Spent Studying to Learning: r z .27 Cognitive Ability to Learning: r z .07 (Brown, 1996) Mastery/Performance to Learning: r z -.34 (Brown, 1996) RA2 = .07 R32 = .03 (conservatively estimated to be small) RC2 = .12 L=7.85 w=1 u=1 z=2 N = 235 H1 1b: Objective Task Difficulty Moderating Goal Setting to Performance H12: Leaming Moderating Goal Setting to Performance Goal Setting to Performance: r z .50 Cognitive Ability to Performance: r z .08 (Brown, 1996) Mastery/Performance to Performance: z -.14 (Brown, 1996) RA2 = .25 R32 = .03 (conservatively estimated to be small) RC2 = .03 L=7.85 w=1 u=1 z=2 N = 186 MANCOVA H13: In order to estimate an effect size for the overall MANCOVA, the following conservative procedure was used: The estimated R2 for the dependent variable with the strongest relationship was taken, r = .77 (Bandura & Wood, 1989) for the goal setting to performance relationship, along with an estimated r = .40 (Brown, 1996) for the learning to performance relationship. These two relationships produce an R2 of .77. This is a conservative figure to use in estimating the MANCOVA effect size, because adding any additional dependent variables would result in a larger R2. An R2 of .77 corresponds roughly to an f2 of 1.12, a large effect size. Using the equations in Cohen’s (1988) chapter on multivariate analyses, a sample size of 127 would be required to detect effects of this size with 80 percent power and a one-tailed test (0t=.05). 110 APPENDIX B Consent and Debriefing Forms Informed Consent Tactical Action Game (TAG) The study in which you are about to participate investigates your learning and performance on the Tactical Action Game (TAG). TAG is a computer-simulated, radar tracking task that you will trained to use, and then you will practice. Using the computer mouse, you will assess the attributes of contacts that appear on your screen and decide what action should be taken for each contact. You will be also asked to answer questions which help us understand your task performance and learning. Your participation in this study requires that you stay in our lab for three (3) hours. While your participation is completely voluntary, the success of this study depends on your making a commitment to pay attention and put forth your best effort during the session. No risks or discomforts are anticipated as a result of this study, other than those associated with working on a challenging task on a computer for this length of time. Awards will be distributed at the conclusion of the study, expected to be sometime midway through Spring Semester. Ten $25.00 awards will be given to randomly selected participants. If you win, you will be contacted at the address and phone number you indicate below. Instructions for claiming the award through the Department of Psychology Business Office will be provided when you are contacted. At the end of your involvement, you will be provided with feedback explaining the purpose of this research in more detail. Participation is strictly voluntary. You are free to discontinue the study at any time for any reason without penalty. Your responses will be completely confidential; your name is only taken to notify you if you have won an award. You are free to ask any questions you might have about this study at any time. You may ask questions about the outcome of the study at any time by contacting Rebecca Toney at 353-2880. Consent: I have been fully informed of the above-described study and its possible risks. I give permission for my participation in this study. I know that the investigator and his/her associates will be available to answer any questions I may have. I understand that I am free to withdraw this consent and discontinue participation in this study at any time without penalty. Name: Phone #: Address: Signature: Date: 111 Debriefing Form Tactical Action Game (TAG) The study in which you just participated was designed to examine the effects of feedback and training on learning and decision-making processes. During this study, you operated the TAG radar simulation. TAG simulates the difficult physical performance, information processing, and decision-making demands of fast-paced, critical tasks. TAG required you to gather information about objects on the screen, make decisions, and set goals based on the feedback you received. The information gathered during the study will be used to link your task performance and task knowledge to the feedback you received. If you have any questions about this study or would like to receive a copy of the results when they are complete, please notify the investigator now or call the investigator at a later time. Thank you for participating in this study. Investigator: Rebecca Toney 353-2880 112 APPENDIX C Feedback Manipulation FASTBACK Screens Positive Feedback For this past practice period: You have scored B E T T E R than _% of others who have completed the same number of TAG trials as you. Your total score: Negtive Feedback For this past practice period: You have scored W O R S E than % of others who have completed the same number of TAG trials as you. Your total score: 113 APPENDIX D Measures General Information 1. What is your sex? (1) Male (2) Female 2. What is your age? (1) less than 18 yrs (2) 18-19 yrs (3) 20-21 yrs (4) 22-23 yrs (5) greater than 23 yrs 3. What is your overall grade point average? (1) 0 to 1.0 (2) 1.1 to 2.0 (3) 2.1 to 3.0 (4) 3.1 to 4.0 (5) >40 4. Have you been to this particular lab before? (1) Yes (2) No 5. Are you left or right handed? (1) Left (2) Right 6. Do you play with video games? (1) Never (2) Rarely (3) Sometimes (4) Frequently (5) Always Mastery/Performance Orientation Scale This set of questions asks you to describe how you feel about each of the following statements. Please use the scale shown below to make your ratings. Strongly Strongly Disagree Disagree Neutral Agree Agree <----| l l l 1 > (1) (2) (3) (4) (5) The opportunity to do challenging work is important to me. I do my best when I’m working on a fairly difficult task. I try hard to improve on my past performance. 0. When I have difficulty solving a problem, I enjoy trying different approaches to see which one will work. 1 1. The opportunity to learn new things is important to me. 12. The opportunity to extend the range of my abilities is important to me. 13. I prefer to work on tasks that force me to learn new things. 14. When I fail to complete a difficult task, I plan to try harder the next time I work on it. 15. The things I enjoy the most are the things I do the best. 16. I feel smart when I can do something better than most other people. 17. I like to be fairly confident that I can successfully perform a task before I attempt it. 18. I am happiest at work when I perform tasks on which I know I won’t make any errors. 19. I feel smart when I do something without making any mistakes. 20. I prefer to do things that I can do well rather than things that I do poorly. 21. The opinions others have about how well I can do certain things are important to me. 22. I like to work on tasks that I have done well on in the past. “509°.“ 114 Self-Set Goal Measure Think about what score you would like to finish with on this next trial of TAG. Write down your Score Goal here: points Goal Commitment Scale This set of questions asks you to describe how you feel about the goals you have just set. Please use the scale shown below to make your ratings. Strongly Strongly Disagree Disagree Neutral Agree Agree < 1 1 1 1 1 > (1) (2) (3) (4) (5) 23. I take my goal for this task session seriously. 24. I’m willing to put forth effort to work toward my goal. 25.* It is quite likely that my goal for this session may need to be revised, depending on how things go. 26. I care about my goal for this session. 27. I am committed to pursuing my goal for this session. 28. It wouldn’t take much to make me abandon my goal for the session. 29. I think my goal is worth pursuing. * This question was omitted from the scale used in analyses due to unreliability. Self-Efficacy Scale This set of questions asks you to describe how you feel about your capabilities for playing TAG. Please use the scale shown below to make your ratings. Strongly Strongly Disagree Disagree Neutral Agree Agree < 1 1 1 1 1 > (1) (2) (3) (4) (5) 30. I can meet the challenges of this simulation. 31. I am confident in my understanding of how information cues are related to decisions. 32. I can deal with decisions under ambiguous conditions. 33. I am certain that I can manage the requirements of this task. 34. I believe I will fare well in this task if the workload is increased. 35. I am confident that I can cope with this simulation if it becomes more complex. 36. I believe I can develop methods to handle changing aspects of this task. 37. I am certain I can cope with task components competing for my time. 115 Self-Satisfaction Scale This set of questions asks you to describe how you feel about your performance on the last TAG practice period. Please circle the number on the scale shown below each question to make your ratings. 38. I am satisfied with my overall performance on this task. Extremely Neither Satisfied Extremely Satisfied Satisfied nor Dissatisfied Dissatisfied Dissatisfied <--l- I I 1 | l 1 1 1 1 1 1 1 > 13 12 11 10 9 8 7 6 5 4 3 2 l 39. I am pleased with how I am doing. Extremely Neither Satisfied Extremely Satisfied Satisfied nor Dissatisfied Dissatisfied Dissatisfied <--l ------- I ------- l- 1 l 1 l 1 1 1 1 1 |--> 13 12 11 10 9 8 7 6 5 4 3 2 l 40. My current performance satisfies me. Extremely Neither Satisfied Extremely Satisfied Satisfied nor Dissatisfied Dissatisfied Dissatisfied <--l ------- l ------- | l |-- 1 1 1-- 1 1 1 1 1 > 13 12 11 10 9 8 7 6 5 4 3 2 1 41. I am happy with my performance at this point. Extremely Neither Satisfied Extremely Satisfied Satisfied nor Dissatisfied Dissatisfied Dissatisfied <--| I 1 | I 1 1 1 1 1 1 1 1 > 13 12 11 10 9 8 7 6 5 4 3 2 l 42.*I would be happier if I were performing better than I am now. Extremely Neither Satisfied Extremely Satisfied Satisfied nor Dissatisfied Dissatisfied Dissatisfied <--l- I- 1 | 1 l 1 1 1 1 1 1 1 > 13 12 11 10 9 8 7 6 5 4 3 2 l * This question was omitted from the scale used in analyses due to unreliability. 116 Attribution Scale This set of questions asks you to describe how you feel about your performance on the last TAG practice period. Please use the scale shown below to make your ratings. Strongly Strongly Disagree Disagree Neutral Agree Agree <----1 1 1 1 1 > (1) (2) (3) (4) (5) 43. The cause of my performance involved something I did. 44. The cause of my performance was go_t related to what I did. 45. My performance was mostly caused by the amount of effort I put into the task. 46. My performance was mostly caused by how good I was at the task. 47. My performance was mostly caused by luck. 48.* My performance was mostly caused by how difficult the task was. * This question was omitted from the scale used in analyses due to unreliability. Perceived Task Difficulty Scale This set of questions asks you to describe how you feel about the TAG game. Please use the scale shown below to make your ratings. Strongly Strongly Disagree Disagree Neutral Agree Agree <----1 1 1 1 1 > (1) (2) (3) (4) (5) 49. I found this to be a difficult task. 50. This task was mentally demanding. 51. This task required a lot of thought and information processing. 52. I found this to be a challenging task. Feedback Pergtions Scale This set of questions asks you to describe how you feel now that you are finished playing the TAG game. Please use the scale shown below to make your ratings. Strongly Strongly Disagree Disagree Neutral Agree Agree <----1 1 1 1- 1 > (1) (2) (3) (4) (5) 53. I think the feedback I received accurately described my performance. 54. I believe the feedback I received was correct. 55. My score compared to others who have played TAG was believable. 117 Knowledge Test for Five Cues The following is a knowledge test about TAG. Please use the Scantron sheet to answer the following questions. Bubble in the correct leieg for each question, making sure that the question numbers match. 56. 2|: 57. 58. 59. 60. 61. 62. If a target’s Speed is 40 knots, what does this suggest about the target? a. The target is Air b. The target is Surface c. The target is Civilian d. The target is Military If a target’s Speed is 25 knots, what does this suggest about the target? a. The target is Air b. The target is Surface c. The target is Submarine d. The target is Unknown If a target’s Speed is 20 knots, what does this suggest about the target? a. The target is Civilian b. The target is Military c. The target is Surface d. The target is Submarine If a target’s Altitude/Depth is > 0 feet, what does this suggest about the target? a. The target is Civilian b. The target is Military c. The target is Air d. The target is Submarine If a target’s Altitude/Depth is 0 feet, what does this suggest about the target? a. The target is Air b. The target is Submarine c. The target is Surface d. The target is Unknown If a target’s Altitude/Depth is < 0 feet, what does this suggest about the target? a. The target is Air b. The target is Submarine c. The target is Civilian d. The target is Military An Air target has what Signal Strength? a. Low b. Medium c. Moderate (1. High 118 63. 66. 69. A Surface target has what Signal Strength? a. Low b Medium c. Moderate d High A Submarine target has what Signal Strength? a. Low b. Medium c Moderate d High An Air target has what Climb/Dive Rate? > 0 feet < 0 feet > 35 feet < 35 feet 999‘.” A Surface target has what Climb/Dive Rate? a Medium b. Unknown c 0 feet d 25-35 feet A Submarine target has what Climb/Dive Rate? a < 0 feet b. > 0 feet c 0 feet d 0-24 feet A Communication Time of 39 seconds indicates that the target is likely: a Air b. Surface c Submarine (1 Unknown A Communication Time of 52 seconds indicates that the target is likely: a Air b. Surface c Submarine d Unknown 119 70. 71. 72. 73. 75. A Communication Time of 90 seconds indicates that the target is likely: a. Air b Surface c. Submarine d Unknown A Maneuvering Pattern of Code Foxtrot indicates the target is likely: a Air b. Military c Surface d Civilian A Maneuvering Pattern of Code Echo indicates the target is likely: a Civilian b. Military c Unknown d Undetectable A Maneuvering Pattern of Code Delta indicates the target is likely: a Hostile b. Military c Peaceful d Civilian A Civilian target has what Intelligence sensors on board? a Private b. Platform c None (1 Radar If a target’s Intelligence sensors are Unavailable, the target is likely: Civilian Military Undetectable Unknown 999‘!” A Military target has what Intelligence sensors on board? a Platform b. Private c None d Radar 120 77. 78. 80. 81. 83. A Civilian target has what Initial Bearing? a. 000-090 degrees b 091-270 degrees c. 091-180 degrees d 271-359 degrees An Unknown target has what Initial Bearing? a. 000-090 degrees b. 091-270 degrees c. 091-180 degrees d 271-359 degrees A Military target has what Initial Bearing? a. 000-090 degrees b. 091-270 degrees c. 091-180 degrees d 271-359 degrees A Blue Lagoon Direction of Origin indicates the target is likely: a. Hostile b. Military c. Peaceful d Civilian An Unknown target would have which Direction of Origin? 9’ Blue Lagoon Echo Red Sea Unknown 11.0.57 Military target would have which Direction of Origin? A a. Blue Lagoon b. Echo c Red Sea d Platform A Civilian target has what Initial Range? a <0nm b. 0-20 nm c. 21-100 nm d. > 20 nm 121 84. 85. 86. 87. 88. >l< If a target has an Initial Range of 24 nm, what does this suggest about the target? a. Military b. Unknown c. Civilian d. Submarine A Military target has what Initial Range? a 0-20 nm b. 0-24 nm c 25-35 nm (1 > 100 nm What Threat Level would be assigned to a Peaceful target? a. 0 b. 1 c. 2 d. 3 A Threat Level of 2 would indicate what kind of target? a. Peaceful b. Civilian c. Hostile d. Unknown What Threat Level would be assigned to a Hostile target? a. l b. 2 c. 3 d 10 89. A Peaceful target will demonstrate what form of Countermeasures? a. Jamming b. None c. Unknown (1. Undetected 90. A target with Unknown Countermeasures indicates: a. The target is Hostile b. The target is None c. The target is Peaceful d. The target is Unknown 122 91 92 93 95. 96. . A Hostile target will demonstrate what form of Countermeasures? a. Radar b. None c. Jamming d. Platform . A target with a Clean Missile Lock indicates: a. The target is Hostile b The target is Military c. The target is Peaceful d The target is Unknown . A target with an Undetected Missile Lock indicates: a. The target is Hostile b. The target is Military c. The target is Peaceful d. The target is Unknown . A target with a Locked Missile Lock indicates: a. The target is Hostile b The target is Military c. The target is Peaceful d The target is Unknown What is the Response of a Peaceful target? a. No Response b. Undetected c. Inaudible d Given What is the Response of an Unknown target? a. No Response b Undetected c. Inaudible d Given What is the Response of a Hostile target? a. No Response b. Undetected c. Inaudible d. Given 123 98. A Peaceful target will demonstrate what form of Electronic Warfare? a. Jamming b. None c. Unknown (1. Radar * 99. An Unknown target will demonstrate what form of Electronic Warfare? a. Jamming b. None c. Unknown d. Undetected 100.** A Hostile target will demonstrate what form of Electronic Warfare? * a. Radar b. None c. Jamming (1. Platform * Denotes the correct answer. ** This question was omitted from the scale used in analyses due to an incomplete correct answer. The correct answer should have read “Big Bulge Radar.” 124 REFERENCES 125 REFERENCES Ackennan P. L. (1992). Predicting individual differences in complex skill acquisition: Dynamics of ability determinants. Journal of Applied Psychology, 77, 589-614. Ames, C., & Archer, J. (1988). Achievement goals in the classroom: Students’ learning strategies and motivation processes. Journal of Educational Psychology. 80, 260- 267. Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191-215. Bandura, A. (1991). Social cognitive theory of self-regulation. Organizational Behavior and Human Decision Processes, 50, 248-287. Bandura, A., & Cervone, D. (1983). Self-evaluative and self-efficacy mechanisms governing the motivational effects of goal systems. Journal of Personality and Social Psychology, 45. 1017-1028. Bandura, A., & Cervone, D. (1986). Differential engagement of self-reactive influences in cognitive motivation. Organizational Behavior and Human Decision Making Processes, 38, 92-113. Bandura, A., & Jourden, F. J. (1991). Self-regulatory mechanisms governing the impact of social comparison on complex decision making. Journal of Personality and Social Psychology, 60, 941-951. Bandura, A. & Wood, R. (1989). Effects of perceived controllability and performance standards on self-regulation of complex decision making. Journal of Personality and Social Psychology, 56, 805-814. Bouffard, T., Boisvert, J ., Vezeau, C., & Larouche, C. (1995). The impact of goal orientation on self-regulation and performance among college students. British Journal of Educational Psychology, 65, 317-329. Boyle, K. A., & Klimoski, R. J. (1995, May). Toward an understanding of goal orientation in the traininfigcontext. Paper presented at the Tenth Annual Conference of the Society for Industrial and Organizational Psychology, Orlando, FL. Brown, K. G. (1996). Motivational and informational conseguences of errors in early skill acquisition: The effects of individual differencesfiand training strategy on perceptions of negative feedback. Unpublished master’s thesis, Michigan State University, East Lansing. 126 Button, S. B., Mathieu, J. E., & Zajac, D. M. (1996). Goal orientation in organizational research: A conceptual and empirical foundation. Organizational Behavior and Human Decision Processes, 67. 26-48. Carver, C. S., & Scheier, M. F. (1982). Control theory: A useful conceptual framework for personality-social, clinical, and health psychology. Psychological Bulletin, 2;, 111-135. Carver, C. S., & Scheier, M. F. (1990). Origins and functions of positive and negative affect: A control-process view. Psychological Bulletin. 97. 19-35. Cervone, D., & Peake, P. K. (1986). Anchoring, efficacy, and action: The influence of judgmental heuristics on self-efficacy judgments and behavior. Journal of Personality and Social Psychology. 50. 492-501. Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159. Cook, T. D. & Campbell, D. T. (1979). Quasi-exmrimentation: Design and analysis issues for field settings. Boston: Houghton Mifflin. Duda, J. L., & Nicholls, J. G. (1992). Dimensions of achievement motivation in schoolwork and sport. Journal of Educational Psychology. 84. 290-299. Dweck, C. S., & Leggett, E. L. (1988). A social-cognitive approach to motivation and personality. Psychological Review, 95, 256-273. Dweck, C. S. (1986). Motivational processes affecting learning. American Psychologist, 41, 1040-1048. Elliott, E. S., & Dweck, C. S. (1988). Goals: An approach to motivation and achievement. Journal of Personality and Social Psychology. 54. 5-12. Frese, M., & Altmann, A. (1989). The treatment of errors in learning and training. In L. Bainbridge & S. A. R. Quintanilla (Eds), Developing skills with information technology (pp. 658-682). New York: Wiley. Gardner, D. G. (1982). An empirical test of activation theory-based predictions. Academy of Management Proceedings 1982, 136-140. Gist, M. E., & Mitchell, T. R. (1992). Self-Efficacy: A theoretical analysis of its determinants and malleability. Academy of Management Review, 17, 183-211. Hollenbeck, J. R., Williams, C. R., & Klein, H. J. (1989). An empirical examination of antecedents of commitment to difficult goals. Journal of Applied Psychology, 74, 18-23. Howard, A. (1995). A framework of work change. In a. Howard (Ed.), The nature of work (pp. 3-44). San Fransico: Jossey-Bass. 127 Huber, V. L. (1985). Effects of task difficulty, goal setting, and strategy on performance of a heuristic task. Journal of Applied Psychology, 70, 492-504. Hunter-Blanks, P., Ghatala, E. S., Pressley, M., & Levin, J. R. (1988). Comparison of monitoring during study and during testing on a sentence learning task. Journal of Educational Psychology. 80. 279-283. Hysong, S. J ., & Quinones, M. A. (1997). The relationship between self-efficacy and performance: A meta-analysis. Paper presented at the Twelfth Annual Conference of the Society for Industrial and Organizational Psychology, St. Louis, MO. Ivancic, K., & Hesketh, B. (1995). Making the best of errors during training. Training Research Journal, 1. 103-125. Kanfer, R. (1996). Self-regulatory and other non-ability determinants of skill acquisition. In P. M. Gollwitzer & J. A. Bargh (Eds), The psychology of action: Linking cognition and motivation to behavior (pp. 404-423). New York, Guilford Press. Kanfer, R., & Ackerman, P. L. (1989). Motivation and cognitive abilities: An integrative/aptitude-treatment interaction approach to skill acquisition. Journal of Applied Psychology, 74L 657-690. Karl, K. A., O'Leary-Kelly, A. M., & Martocchio, J. J. (1993). The impact of feedback and self-efficacy on performance in training. Journal of Organizational Behavior. 1_4_, 379-394. Karoly, P. (1993). Mechanisms of self-regulation: A systems view. Annual Review of Psychology, 44, 23-52. Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin. 1 1% 254-284. Kozlowski, S. W. J., Weissbein, D. A., Brown, K. G., Toney, R. J., & Mullins, M. E. (1997, May). Guiding the development of deployable shipboard training systems: Enhancing skill acquisition, adaptability, and effectiveness. (Final Report 1.1; Contract No. N61339-96-K-0005). Michigan State University, East Lansing. Kozlowski, S. W. J., Gully, S. M., Smith, E. M., Brown, K. G., Mullins, M. E., & Williams, A. E. (1996, April). Sequenced mastery goals and advance organizers: Enhancing the effects of practice. In K. Smith-Jentsch (Chair), When. how. and why does practice make perfect? Symposium conducted at the 11th Annual Conference of the Society for Industrial and Organizational Psychology, San Diego, CA. 128 Kozlowski, S. W. J., Gully, S. M., Smith, E. M., Nason, E. R., & Brown, K. G. (1995, May). Learning objectives and training for complex tasks: The effects of sequenced mastery goals apd_advance organizers on learning outcomes and skill generalizations. Paper presented at the 10th Annual Conference of the Society for Industrial and Organizational Psychology, Orlando, FL. Kuhl, J ., & Koch, B. (1984). Motivational determinants of motor performance: The hidden second task. Psychological Research, 46, 143-153. Kyllonen, P. C., & Christal, R. E. (1990). Reasoning ability is (little more than) working-memory capacity?! Intelligence. 14. 389-433. Latham, G. P., & Lee, T. W. (1986). Goal Setting. In E. A. Locke (Ed.), Generalizing from laboratory to field settings. Lexington, MA: Lexington Books. Latham, G. P., & Locke, E. A. (1991). Self-regulation through goal setting. Organizational Behaviorfiand Human Decision Processes, 50, 212-247. Lee, C., & Bobko, P. (1994). Self-efficacy beliefs: Comparison of five measures. journal of Applied Psychology. 79. 364-369. Lindsley, D. H., Brass, D. J ., & Thomas, J. B. (1995). Efficacy-performance spirals: A multilevel perspective. Academy of Management Review, 20, 645-678. Locke, E. A., Frederick, E., Lee, C., & Bobko, P. (1984). Effect of self-efficacy, goals, and task strategies on task performance. Journal of Applied Psychology. 69, 241- 251. Locke, E. A., & Latham, G. P. (1990). A theory of goal setting and task performance. Englewood Cliffs, NJ: Prentice Hall. Locke, E. A., Shaw, K. N., Saari, L. M., & Latham, G. P. (1981). Goal setting and task performance: 1969-1980. Psychological Bulletin. 9Q 125-152. Lord, R. G., & Levy, P. E. (1994). Moving from cognition to action: A control theory perspective. Applied Psychology: An International Review. 43. 335-367. Martocchio, J. J. (1994). Effects of conceptions of ability on anxiety, self-efficacy, and Ieaming in training. Journal of Applied Psychology, 79, 819-825. Maynard, D.C., & Hakel, M. D. (in press). The effects of objective and subjective task complexity on performance. Human Performance. Mento, A. J., Steel, R. P., & Karren, R. J. (1987). A meta-analytic study of the effects of goal setting on task performance: 1966-1984. Organizational Behaviior and Human Decision Processes,39. 52-83. 129 Mikiluncer, M. (1989). Cognitive interference and learned helplessness: The effects of off-task cognitions on performance following unsolvable problems. Journal of Personality and Social Psychology, 57, 129-135. Mudgett, B., & Quinones, M. A. (1997, April). The effect of personality and feedback type on goal revision and goal commitment. In K. J. Williams (Chair), Reactions to goal-perforrnance discrepancies: Effects on personalifity and performance context. Symposium conducted at the Twelfth Annual Conference of the Society for Industrial and Organizational Psychology, St. Louis, MO. Nease, A. A., Mudgett, B., & Quinones, M. A. (1997, April). Effects of feedback sign and self-efficacy on acceptance of feedbac_k. Paper presented at the Twelfth Annual Conference of the Society for Industrial and Organizational Psychology, St. Louis, MO. Norman D. A., & Bobrow, D. G. (1975). On data-limited and resource-limited processes. Cognitive Psychology. 7, 44-64. Podsakoff, P. M. & Fahr, J. (1989). Effects of feedback sign and credibility on goal setting and task performance. Organizational Behavior and Human Decision Processes, 44, 45-67. Ree, M. J., & Earles, J. A. (1992). Intelligence is the best predictor of job performance. Current Directions in Psychological Science. 1. 86-89. Salomon, G. (1984). Television is "easy" and print is "tough": The differential investment of mental effort in Ieaming as a function of perceptions and attributions. Journal of Educational Psychology, 76, 647-658. Salomon, G., & Perkins, D. N. (1989). Rocky roads to transfer: Rethinking mechanisms of a neglected phenomenon. Educational Psychologist. 24. 1 13-142. Schmidt, R. A., & Bjork, R. A. (1992). New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training. Psychological Science 3 207-217. Schmidt, F. L., Hunter, J. E., Outerbridge, A. N., & Goff, S. (1988). Joint relation of experience and ability with job performance: Test of three hypotheses. Journal of Applied Psychologyjl 46-57. Schraw, G., Horn, C., Thomdike-Christ, T., & Bruning, R. (1995). Academic goal orientations and student classroom achievement. Contemporag Educational Psychology, E, 359-368. Silver, W. S., Mitchell, T. R., & Gist, M. E. (1995). Responses to successful and unsuccessful performance: The moderating effect of self-efficacy on the relationship between performance and attributions. Organizational Behavior and Human Decision Processes, 62, 286-299. 130 Taylor, M. S., Locke, E. A., Lee, C., & Gist, M. E. (1984). Type A behavior and faculty research productivity: What are the mechanisms? Organizatiogl belLvior and Human Performance. 34, 402-418. Waldersee, R., & Luthans, F. (1994). The impact of positive and corrective feedback on customer service performance. Journal of Organizational Behavior. 15. 83-95. Weiner, B. (1985). An attributional theory of achievement motivation and emotion. Psychological Review. 92. 548-573. Wofford, J. C., & Goodwin, V. L. (1990). Effects of feedback on cognitive processing and choice of decision style. Journal of Applied Psychology, 75, 603-612. Woltz, D. J. (1988). An investigation of the role of working memory in procedural skill acquisition. Journal of Experimental Psychology: General, 117. 319-331. Wood, R. E. (1986). Task complexity: Definition of the construct. Organizational Behavior and Human Decision Processes. 37, 60-82. Wood, R. E., & Bandura, A. (1989). Impact of conceptions of ability of self- regulatory mechanisms and complex decision making. Journal of Personality and Social Psychology, 56, 407-415. Wood, R., Bandura, A., & Bailey, T. (1990). Mechanisms governing organizational performance in complex decision-making environments. Organizational Behavior and Human Decision Processes. 46. 181-201. 131 "11111111111tilt