Iv... 4 . 1:7 I: . . 3.... .x .41... .th lit) 5 34‘: ,4 a I. . 9.. 3.3:! a: 1 .:. T . vials»... I . i...aflmhu.mxt§fui$uiwu. v6 . .4 $5.“. .1. 3:31 .3va} {A l a I). . sl « r .2! . Iui “ mflmramafiwmn a ‘53-. 1:1».I 5.14 . I. It -. lf-q ‘.::.- o! a,“ .. II . ‘3 fiifililyt‘ 5.“..6. i.d. .fiLx‘v ..A.H‘§.Il:vflli 4:1.- I. Janivlfinsfll tut-xnn‘ t iiltaii’hil- ‘l , .. .§." lat-1194'; .4: I: 1.4 a.“ .. : u: _ . 3 u. . . . . . :4, , ., . WI... {@3312 f 5.51 .asquuumnfimfimmwf . .. . fie“; , n .xwafifiésafif . . V 56.". 1...» ,. 13.1.... . 2.» :P. 3.3.. 2. ciao/o LIBRARY Michigan State University This is to certify that the thesis entitled AN EXAMINATION OF THE INTERACTIVE EFFECTS OF GOAL SETTING AND METACOGNITION ON LEARNER DIRECTED SKILL ACQUISITION presented by Dustin K. Jundt has been accepted towards fulfillment of the requirements for the degree In . Psychology Liz/é / Major Pr sor’s Signature 7/30/2 0 9.5 Date MSU is an Affirmative Action/Equal Opportunity Institution PLACE IN RETURN Box to remove this checkout from your record. To AVOID FINES return on or before date due. MAY BE RECALLED with earlier due date if requested. DATE DUE DATE DUE DATE DUE [6‘3er :4 612.2 I 2 — 2/05 c:/ClRC/DateDue.indd-p. 15 AN EXAMINATION OF THE INTERACTIVE EFFECTS OF GOAL SETTING AND METACOGNIT ION ON LEARNER DIRECTED SKILL ACQUISITION By Dustin K. Jundt A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of MASTER OF ARTS Department of Psychology 2005 ABSTRACT AN EXAMINATION OF THE INTERACTIVE EFFECTS OF GOAL SETTING AND METACOGNITION ON LEARNER DIRECTED SKILL ACQUISITION By Dustin K. Jundt Two self-regulatory processes related to skill acquisition, goal setting and metacognition, were examined in this study. Goal setting studies typically examine the motivational impact of challenging, specific goals, while metacognition studies focus primarily on the strategic and regulatory benefits of making plans, monitoring behavior and cognitions, and evaluating leaming. These two processes are rarely considered simultaneously in terms of potential concurrent influences on skill acquisition, however. This study was designed to explicitly examine the interactive effects of goal setting and a metacognition training intervention on declarative knowledge and skill-based performance on a moderately complex webpage building task. The predicted multiplicative effects of challenging, specific goals and the metacognition intervention on declarative knowledge gain and performance were not supported. While the metacognition intervention lead to higher levels of strategic/metacognitive activity, goals were not related to motivation. Furthermore, metacognition and motivation did not interact to influence declarative knowledge gain or performance. ACKNOWLEDGMENTS I would like to thank Dr. Daniel Ilgen for serving as my advisor. Thank you for your ideas and for helping to provide me with clarity on many of the issues I faced while working on this project. I am especially thankful for your patience and in letting me work through all of the issues I encountered while still keeping me on track. I would also like to thank Dr. Steve Kozlowski and Dr. Kevin Ford for serving on my committee and helping me to better understand some of the important issues related to this study. iii TABLE OF CONTENTS LIST OF TABLES .............................................................................................................. vi LIST OF FIGURES ........................................................................................................... vii INTRODUCTION ............................................................................................................... 1 Goals ........................................................................................................................ 3 Metacognition .......................................................................................................... 4 Simultaneous Consideration of Goals and Metacognition ....................................... 5 GOAL SETTING THEORY ................................................................................................ 7 METACOGNITION .......................................................................................................... 19 Metacognitive Knowledge ..................................................................................... 21 Metacognitive Control ........................................................................................... 22 Metacognition and Performance ............................................................................ 28 Metacognitive Interventions .................................................................................. 33 GOAL SETTING AND METACOGNITION ................................................................... 4O MODELS AND HYPOTHESES ....................................................................................... 43 METHOD .......................................................................................................................... 50 Participants ............................................................................................................ 50 Design .................................................................................................................... 50 Complex Task ........................................................................................................ 51 Manipulations and Measures ................................................................................. 52 Goals .......................................................................................................... 52 Metacognitive Intervention ........................................................................ 54 Strategic Activities ..................................................................................... 55 Task Activity .............................................................................................. 57 Declarative Knowledge .............................................................................. 58 Task Performance ...................................................................................... 59 Control Variables ....................................................................................... 60 Procedure ............................................................................................................... 60 RESULTS .......................................................................................................................... 62 Control Variables ................................................................................................... 62 Hypothesis 1 ........................................................................................................... 62 Hypothesis 2 ........................................................................................................... 65 Hypothesis 3 ........................................................................................................... 68 Hypothesis 4 ........................................................................................................... 69 DISCUSSION .................................................................................................................... 72 Limitations ............................................................................................................. 74 iv Design and Operational Limitations .......................................................... 76 Execution Limitations ................................................................................ 81 Findings and Related Limitations .......................................................................... 84 Hypothesis 1 ............................................................................................... 84 Hypothesis 2 ............................................................................................... 85 Hypothesis 3 ............................................................................................... 86 Hypothesis 4 ............................................................................................... 87 An Alternative Model ............................................................................................ 89 Learning Points and Related Research Directions ................................................. 92 APPENDICES Appendix A: Goal Manipulations .......................................................................... 98 Appendix B: Metacognition Intervention — Overview and Description ................ 99 Appendix C: List of Self-Questions Given in the Metacognitive Intervention... 101 Appendix D: Strategic Behaviors Scale ............................................................... 102 Appendix E: Declarative Knowledge Quiz .......................................................... 103 Appendix F: Webpage Building Objectives ........................................................ 108 Appendix G: Computer Experience Scale ........................................................... 112 Appendix H: Consent Form ................................................................................. 113 Appendix I: Debriefing Form .............................................................................. 115 Appendix J: Tables .............................................................................................. 117 Appendix K: Figures ............................................................................................ 129 REFERENCES ................................................................................................................ 135 LIST OF TABLES Table 1: Means, Standard Deviations, Intercorrelations, and Internal Consistency Reliabilities for Key Variables in the Study .................................................................... 1 17 Table 2: Effects of Goa] Condition and Mastery Goal Orientation On Practice Activities (n =109) ........................................................................................................... 118 Table 3: Effects of Goal Condition and Goal Commitment On Practice Activities (n=109) ............................................................................................................ 119 Table 4: Effects of Goal Condition and Attention to Assigned Goals on Practice Activities (n=109) ............................................................................................................ 120 Table 5: Effects of Goal Condition and the Metacognitive Intervention on Metacognitive Activity (n=1 14) ...................................................................................... 121 Table 6: Effects of Goal Condition and the Metacognitive Intervention on Repeated Submodule Page Visits (n=114) ...................................................................................... 122 Table 7: Effects of the Metacognitive Intervention and Mastery Goal Orientation on Metacognitive Activity (n=114) ...................................................................................... 123 Table 8: Effects of the Metacognitive Intervention and Computer Experience on Metacognitive Activity (n=111) ...................................................................................... 124 Table 9: Effects of Metacognitive Activity and Practice Activities on Webpage Score (n=105) .................................................................................................................. 125 Table 10: Effects of Metacognitive Activity and Mastery Goal Orientation on Webpage Score (n=105) .................................................................................................. 126 Table 11: Effects of Metacognitive Activity and Practice Activities on Declarative Knowledge (n=108) ...................................................................................... 127 Table 12: Effects of Metacognitive Activity and Mastery Goal Orientation on Declarative Knowledge (n=113) ...................................................................................... 128 LIST OF FIGURES Figure 1: Goal Setting Effects on Task Performance ...................................................... 129 Figure 2: Hypothesized Effects of a Metacognitive Intervention on the Goal-Performance Relationship on a Complex Task. ..................................................... 130 Figure 3: Hypothesized Interaction Between Goal Condition and Presence or Absence of the Metacognitive Intervention on the Level of Strategic Behaviors ........... 131 Figure 4: Hypothesized Interactive Effects of Strategic and Task Activities on Learning and Task Performance (Collapsed and Shown as General Performance) ........ 132 Figure 5: A Model of the Interactive Effects of Goal Orientations and a Metacognitive Intervention on Metacognitive Activity and Learning Outcomes. Taken from Schmidt & Ford (2003) ................................................................................ 133 Figure 6: An Alternative Post-Hoe Model of the Impact of a Metacognitive Intervention and Mastery Goal Orientation on Skill Acquisition on a Complex Task....134 vii Introduction Researchers in Industrial/Organizational psychology and related fields have noted that the days of having static “jobs” where one repeatedly performs the same tasks are coming to an end (Ilgen & Pulakos, 1999; Smith, Ford, & Kozlowski, 1997). As the nature of work becomes more complex, the expectation that one can go to work day in and day out and do the same set of tasks is rapidly becoming more fiction than reality. Because of this shift to less static jobs, changes need to be made in order to ensure that employees are adequately prepared to do the myriad tasks that are required of them. To do this, employers need ways of delivering training in a manner that is not only effective and efficient but also available on an as-needed basis. This dramatic increase in demand for flexible, efficient training has created somewhat of a dilemma. Keeping employees up to date on the skills and competencies they need is important if organizations want to stay competitive, but the necessary training can be quite resource intensive. To combat this, researchers and organizations have begun to explore different methods of delivering training, including methods in which the employee or trainee is in charge of their own learning. This type of training has become more popular recently due to the decreasing costs, increasing availability, and inherent flexibility associated with computers. Researchers have taken note of this shift in training and over the past few decades have conducted a substantial number of studies with the intent of examining what effects the increase of control has on individuals’ learning and performance. (see Milheim & Martin 1991; Steinberg, 1977, 1989; and Williams 1996 for reviews). This research, however, has been plagued by theoretical, analytical, and definitional problems (Reeves, 1993) as well as large amounts of divergence in findings. Results from the learner control research run the gamut from suggesting that giving learners control increases learning, motivation, and self-efficacy to suggesting that giving control to learners actually decreases learning, to suggesting that learner controlled training produces essentially the same results as traditional types of training. Furthermore, a large amount of the learner control research has focused on direct comparisons between learner controlled training and traditional training in terms of which one is “better”. Little attention has been given to what actually can be done to make learner controlled training more effective. Recent research has began to move away from this “horserace” approach, contending that learner controlled, computer-based training is here to stay and that the important thing now is to figure out how to make it work better (Williams, 1996). While researchers have not yet focused a great deal of attention on explicitly examining ways to improve performance and learning in learner controlled situations, research on performance and learning in complex task settings is likely applicable. In fact, learner controlled settings may be best thought of as a subcategory or type of more general complex task setting, rather than something totally unique. With this being said, it is important to note that although researchers have examined performance and learning in complex task settings, the mechanisms responsible for success are not well understood. Research from a number of different areas may be relevant to attempting to increase performance and learning in these types of settings. Many approaches to performance and learning in complex settings take a self- regulation perspective in which goals are the central focus and strategies and tactics are devised and enacted to achieve them. Specifically, two areas currently receiving attention from researchers that focus on discrete aspects of self-regulation are goal setting and metacognition. Goals Goal setting research deals primarily with goals as precursors to direct action. The notion is that goals contain a sort of “hot” component that mobilizes energy and directs effort. The types of mechanisms through which goals are thought to influence performance reflect this. Specifically, researchers commonly suggest that there are four of these mechanisms: effort, direction, persistence, and strategy use (Latham & Pinder, 2005; Locke, Shaw, Saari, & Latham, 1981; Locke & Latham 1990a; 19900). The first three goal setting mechanisms are traditionally referred to as direct mechanisms (Locke, Shaw, Saari, & Latham, 1981; Earley, Wojnaroski, & Prest, 1987) and are motivational in nature, as they deal directly with the mobilization of energy. Effort can simply be thought of as how hard a person is working toward achieving their goal at any given time; direction refers where this effort is applied. Persistence is a combination of effort and direction of attention and can be thought of as the level of effort directed in a given direction over time (Locke et al., 1981). Challenging, specific goals impact performance via these mechanisms by increasing effort and directing action toward the activities specified by the goal and away from goal-irrelevant activities (Locke & Latham, 1990c). A fourth mechanism by which goals are thought to influence performance is through strategy use or plans (Locke et al., 1981). Strategies and plans are thought of as indirect mechanisms in that specific goals do not directly impact them. Rather, as Locke et al. (1981) suggest: “Although strategy development is motivated by goals, the mechanism itself is cognitive in essence; it involves skill development or creative problem solving (p. 132).” Strategies and plans are thought to provide the layout for how mobilized, directed energy will be used over time. If individuals are not accomplishing their goals, they should realize it (via feedback) and then direct effort toward finding or devising more appropriate strategies for the task(s) at hand (Locke et al., 1981; Locke & Latham 1990c; Terborg, 1976). These strategies, in turn, should lead to higher levels of performance. Metacognition Metacognition deals explicitly with strategy development, implementation, and revision and can be loosely defined as the knowledge about one’s own cognitions and the monitoring and regulation of behavior based upon it (F lavell, 1979; Brown, 1987). Research on metacognition has been spurred on by the rather robust finding that adults often fail to monitor their thinking and behavior (Gamer & Alexander, 1989). This research has focused primarily on the processes involved in successfully gaining knowledge from past experience and how people use that knowledge to make plans for and regulate the success of future performance. Metacognition is a process by which individuals use prior knowledge to select strategies, make plans, monitor their performance or learning in real time, and evaluate the success of their strategies and plans afier performing (Ertmer & Newby, 1996). These actions, then, are thought to create knowledge regarding the usefulness of different strategies in a wide variety of different performance or learning situations. This knowledge can then be used in the future to help the individual to make appropriate plans and select useful strategies for performance (Schraw, 1998). Researchers examining metacognition suggest that in order to monitor and evaluate the success of strategies and overall performance, there must be a reference point for these judgments to be centered around. This reference point is traditionally thought to come in the form of a goal. While the importance of having a goal or reference point is acknowledged, metacognition research does not typically deal explicitly with how these goals are devised or the impact they have on the performer’s motivation. Instead, this research focus on a more “cold” cognitive process that is concerned with a somewhat mechanical strategy selection and performance monitoring process. Simultaneous Consideration of Goals and Metacognition As can be seen by these brief overviews, there is some of overlap between goal- setting theory and metacognition theory. Both deal with self-regulation in general and acknowledge the importance of both motivation (in the form of goals) and cognition (in terms of strategy development and monitoring). However, both literatures fail to provide solid overall perspectives in that they focus primarily on one part or the other, without explicitly detailing the importance of both motivation and higher-order cognition at the same time. This is a problem in that a narrowed point of view can arise when considering each of these literatures independently. I believe that in order to gain a more complete understanding of performance and learning on complex tasks, in particular those of a learner-controlled nature, one must consider the impact of both goal setting and metacognition concurrently. It may be the case that in order to perform well in complex task environments, learners need to not only have specific, difficult goals to increase their level of mobilized energy, but they may also need to use this energy systematically to devise strategies and plans in the manner suggested by metacognition theory. High levels of motivation may be useless if one cannot select and implement appropriate strategies. Similarly, having the ability to make plans and monitor their success may not impact performance if one does not have the motivation to do so. Therefore, I believe it is important that individuals performing or learning in complex task situations have both the motivation associated with specific, difficult goals, as well as the strategic benefits associated with metacognition. To support this assertion, I will thoroughly review the literature in both areas pertaining to complex task performance, present a model based on this review that describes how I believe goals and metacognition will impact performance, and design a study to test the validity of this model. Goal Setting Theory Decades of research comprised of hundreds of individual studies have supported the primary tenants of goal setting theory: Challenging, specific goals lead to high levels of performance (Locke & Latham, 1990a). Numerous reviews and meta analyses have supported these assertions across a wide variety of different tasks, settings, time spans, nationalities, performance criteria, etc. . .(Locke & Latham, 1990a; 1990b: Locke et al., 1981; Mento, Steel, & Karren, 1987; Wood, Mento, & Locke, 1987). While the findings regarding goal setting theory are robust, they are based predominantly on research using simple tasks (Wood et al., 1987). In simple tasks, strategies do not necessarily play an important role due to the fact that mobilization of effort will likely be directly translated into to higher levels of performance (Campbell, 1991; Locke & Latham, 1990b; Wood & Locke, 1990). This may not the case on more complex tasks, however. Complex tasks have requirements that are much more thorough and cognitively demanding than do simple tasks Wood (1986) describes three types different types of task complexity: Component complexity, coordinative complexity, and dynamic complexity. Tasks with high levels of component complexity have a large number of distinct acts to perform and information cues to process. Tasks with these characteristics are complex because individuals need to be aware of an able to perform many more activities and process more types of information than on simpler tasks. Coordinative complexity refers to the strength of relationships between different aspects of the task and the timing or order they need to be performed in. Tasks higher in component complexity require individuals to not only know how to do a number of different things but also what sequence and at what time they should do them. Finally, dynamic complexity occurs when the cause and effect relationships between different components of the task change over time. Performance on a dynamically complex task requires individuals to know about components and sequencing as well as possible changes over time on both of these. As can be seen by this brief examination of the nature of task complexity, simply working harder and exerting more effort will likely not be sufficient for increasing performance. Indeed, on complex tasks, the data suggests that strategy differences may play a much more important role in determining performance effectiveness (Campbell, 1991; Wood & Locke, 1990). In a meta-analysis, Wood et al. (1987) found that task complexity moderated the effects of challenging and specific goals such that the impact of challenging and specific goals on performance decreased as task complexity increased. This finding corroborated early theory regarding the lowered impact of goals on complex task performance (e. g. Campbell, 1984; Earley, 1985) and has spurred researchers to more closely examine the nature of this relationship. Wood and Locke (1990) attempted to more clearly explain how goals may impact performance on tasks of varying levels of complexity by focusing on different types of task plans and strategies (see also Campbell, 1991, for a similar approach) and their importance in the performance process. These authors approached plans in a slightly different way, suggesting that all of the aforementioned goal-perforrnance mediating mechanisms can be thought of as plans or strategies in their own right. Wood and Locke (1990) suggested that the direct mechanisms of effort, direction, and persistence could be thought of as Stored Universal Plans (SUP’s) that apply to nearly every type of task to a certain extent. It is thought that at some point in time, nearly everyone learns that these strategies are important for performance. These SUP’s, then, are used so often that they become automatized or “built into goals” (Wood & Locke, 1990, p.77). Without mobilized energy, one will likely not be successful on performing a task regardless of its level of complexity. This line of thinking, of course, is nothing new, and although these mechanisms were referred to as plans or strategies, they are still primarily motivational in nature. Wood and Locke (1990) described other types of plans that may impact the goal- perfonnance relationship as well. The first are referred to as Stored Task Specific Plans (STSP’s). These are plans that individuals have learned through instruction, modeling, practice, or use on previous tasks and are thought to be more cognitive and skillful than SUP’s. Individuals are likely to first attempt to use STSP’s on tasks that require more than just directed energy. STSP’s are thought to be good for tasks that are well practiced or somewhat familiar to the performer and can be specific (e. g. a set of distinct steps for completing a task) or general (e. g. “use common sense”) in nature (Wood & Locke, 1990). STSP’s are suggested to be most directly related to performance on tasks of moderate complexity where effort is not enough for performance. On these types of tasks, goals likely lead to effort, which in turn may yield appropriate strategy selection based on past experience. On the most complex tasks, even a wide variety of STSP’s may not be enough, especially if the task is novel to the performer. When the performer deems STSP’s inapplicable, New Task Specific Plans (NTSP’s) must be devised (Wood & Locke, 1990). Generating NTSP’s is more difficult to do than selecting STSP’s due to the fact that the individual either has to create a totally new plan or must find an existing plan or strategy that he or she is not aware of. Also, as Wood & Locke (1990) point out: “People seem loath to abandon STSP’s which have served them well for a long period of time”(p. 81). This would indicate that in many cases, it is difficult for people to even get to the point of trying to find NTSP’s. The generation or discovery of NTSP’s, then, likely takes a large amount of effort, which still may not lead to successful strategy development (Wood & Locke, 1990). Even with the motivating effects of challenging, specific goals, the individual simply may not be able to come up with an appropriate NTSP for a novel task. This relates to my earlier assertion that simply considering goals or strategies on their own is insufficient. Understanding and considering the motivational impact of goals is important, but if researchers do not also consider ways in which this motivation can be translated into proper strategy development and use, they cannot gain a thorough understanding of the mechanisms that impact performance in complex settings. While researchers such as Wood & Locke (1990) and Campbell (1991) do discuss strategies more formally as part of the goal setting process, they do not provide a great degree of clarity pertaining to exactly how these strategies are developed or what type of knowledge the individual uses to select strategies or make plans. These researchers focus primarily on goals as providing motivation and driving individuals to try to find better task strategies. This lack of theoretical comprehensiveness and clarity regarding the relationship between goals, strategies, and complex task performance manifests itself in the form of inconsistent research findings. Researchers have conducted a number of studies directed specifically at examining how (or it) challenging, specific goals impact performance on complex tasks. Their results, however, lead to no clear conclusions. For example, in both laboratory and 10 field experiments Earley et a1. (1987) found that the predictions of goal-setting theory (e. g. Locke, 1968; Locke et al., 1981) hold up to a moderate extent on complex tasks. Challenging, specific goals were related to higher levels of performance on a marketing task (laboratory) and across a handful of different jobs for employees in a service organization or a moving company. F urtherrnore, goals were found to impact performance both directly and indirectly through effort and planning in the laboratory sample. Also, in a study using a managerial decision making simulation, Cervone, J iwani, & Wood (1991) found that challenging, specific goals had a small impact on initial performance. This effect, however, disappeared soon after participants became more familiar with task. The results of these studies are somewhat consistent with the Wood et al. (1987) meta-analysis that suggested a small, but still positive impact of challenging, specific goals on complex task performance. The majority of studies, however, have found that challenging, specific goals have little impact on complex task performance or that they may even lead to poorer performance. For example, Kanfer & Ackennan (1989) and Steele-Johnson & Kanfer (1992) found that goals set early in the skill acquisition process had no overall effects on performance on an Air Traffic Controller simulation. Similarly, Cervone & Wood (1995) and Wood, Bandura, & Bailey (1990) found no impact of challenging, specific goals on individuals’ performance on a managerial decision making task. Huber (1985) examined the impact of goals on participants’ ability to solve a difficult maze task and found that participants given challenging, specific goals actually performed worse than did those given “do your best” (DYB) goals. Across three different experiments, Earley, Connolly, & Ekegren (1989) found similar results when 11 using a stock value prediction task. Mone & Shalley (1995) used an organizational staffing simulation and found that people given DYB goals were able to make more decisions in the same time frame than were those given challenging, specific goals and that both groups made equally accurate decisions. Latham & Seijts (1999) examined the impact of challenging, specific distal and proximal performance goals on a managerial decision making simulation dealing with production. Similar to other studies, they found challenging, specific distal performance goals banned performance compared to DYB goals, but they also suggested that proximal performance goals given along with distal performance goals lead to the highest levels of performance. The proximal goals, however, were likely confounded with the amount of task information given to participants, thus clouding the interpretation of their usefulness. Indeed, Seijts & Latham (2001) were not able to replicate findings pertaining to paired proximal and distal goals on a class scheduling task, although they did replicate the finding that those given distal performance goals performed worse than those given DYB goals. Four general explanations have been given to explain why challenging, specific goals may not impact complex task performance in the same way as they impact simple task performance. Based on a resource allocation perspective, Kanfer and Ackerman (1989) suggested that goals initiate self-regulatory activities such as performance monitoring and evaluation. This, of course, is somewhat similar to what was suggested by Wood & Locke (1990). Kanfer & Ackerman (1989), however, proposed that individuals have a fixed pool of attentional resources to work with and regulatory activities, especially early in learning or skill acquisition, rob resources that could be better used for exploring different strategy options. These suggestions were based on 12 findings in three studies conducted by Kanfer & Ackerman (1989) and are supported by Steele-Johnson & Kanfer (1992) and Kanfer, Ackerman, Murtha, Dugdale, & Nelson (1994). A second explanation comes form Earley, Connolly, & Ekegren (1989) and Wood and Locke (1990) and suggests challenging, specific goals create a perceived pressure to immediately perform well. Basically, these authors suggested that if people do not perform well initially they panic and begin to rapidly switch strategies in a haphazard manner, without appropriately testing the usefulness of the strategies or making small strategic modifications when necessary. Strategy shifting, then, results in poorer performance than would be expected from someone who had a more systematic method of testing the usefulness of his or her given strategies. Afore mentioned studies by Huber (1985), Earley, Connolly, & Lee (1989), and Wood et al. (1990) support these assertions. A third explanation comes from DeShon & Alexander (1996), who made a distinction between explicit and implicit learning on complex tasks and suggested that challenging, specific goals may impact performance differently on each. Successfiil performance on explicit learning tasks requires individuals to consciously form representations of the problem, devise or select strategies, test their appropriateness, and make changes if necessary. In these types of tasks goals should lead individuals to direct more effort and attention to strategy development and performance monitoring, which in turn may lead to better performance much in the manner suggested by Wood & Locke (1990). DeShon & Alexander (1996) suggested that implicit learning tasks, on the other hand, require “the acquisition of knowledge concerning stimulus covariation learned through repeated exposure to the problem exemplars without intention or awareness” (p. 13 19). Goals are thought to be irrelevant or even detrimental on tasks of this type as they may lead to the use of explicit learning strategies, which are simply not functional on these types of tasks and likely interfere with unconscious, implicit learning. This is an interesting distinction that is made relevant due to the fact that many of the studies examining goal effects on complex tasks (e. g. Cervone et al., 1991; Cervone & Wood, 1995; Barley, Connolly, & Ekegren, 1989; Earley, Connolly, & Lee, 1989; Wood, et al., 1990; Mone & Shalley, 1995) use tasks that may be more suited for implicit learning (DeShon & Alexander, 1996). In these tasks, individuals are often required to consider the covariation between cues and information pertaining to a number of different areas. In order to perform well, the information from these sources or cues must be properly weighted and combined. Learning, and subsequently performance, is assessed over a number of different trials that are completed fairly rapidly. DeShon & Alexander’s (1996) results suggest that this explicit vs. implicit learning distinction may indeed be important when considering the impact of challenging, specific goals on performance. While they could not directly compare performance on explicit vs. implicit learning tasks (due to the fact that different tasks had to be used), DeShon & Alexander (1996) did find that challenging, specific goals lead to lower levels of performance on an implicit learning task and higher levels of performance on an explicit learning task, just as they had hypothesized. Finally, research contrasting learning and performance achievement goals may also provide some insight into the reasons why traditional outcome goals do not lead to high levels of performance on complex tasks. Learning, or mastery, achievement goals focus on self-referenced skill gain or learning, while performance goals focus more on 14 exhibiting competence and comparisons with others (Dweck & Leggett, 1998). Briefly, researchers focusing on the leaming/performance achievement goal distinction suggest that performance goals may limit the availability of cognitive resources (e. g. Kanfer & Ackerman, 1989) and lead to a focus on surface-level aspects of performance rather than declarative knowledge gain. Mastery goals, on the other hand, may be more effective in helping people gain complex skills as they focus on learning deep elements of the task, continuous learning, and higher levels of intrinsic motivation. A number of studies have examined the impact of learning and performance achievement goals in skill-acquisition related settings. One consistent finding from this research suggests that mastery goals lead to higher levels of intrinsic motivation (e. g. Ames & Archer, 1998; Duda & Nicholls, 1992; Elliot & Harackiewicz, 1994; Harackiewicz & Elliot, 1993). In relation to task performance, Kozlowski et al. (2001) manipulated performance vs. mastery training goals in a complex computer-simulation study and found that mastery goals were related to higher levels of knowledge structure coherence and self-efficacy, which were both related to higher levels of transfer performance on the task. Gist & Stevens (1998) found that mastery goals were related to higher levels of learning and skill transfer on a negotiation task when the goals were paired with stressful practice conditions. Furthermore, Winters and Latham (1996) and Seijts & Latham (2001) found support for the usefulness of learning goals, in this case specific learning goals. Their results suggest that specific leaming goals be instituted in order to increase levels of strategy exploration and performance. In some cases, mastery goals have also been found to be related to the use of cognitive and metacognitive learning strategies (6. g. Pintrich, 1989; Pintrich & 15 Garcia, 1991 ). Results from the Educational Psychology literature, however, provide mixed support for the relationships between mastery goals and graded performance: A number of studies reveal positive effects while many others yield null effects (see Elliot & Church, 1997). Overall, the research on mastery versus performance goals reveals some positive findings regarding the usefulness of mastery goals in skill acquisition settings, although these findings are by no means ubiquitous. This study does not explicitly focus on the mastery vs. performance goal distinction, however, as much of the theory on goal setting effects revolves primarily around perforrnance-related outcome goals, which in this case were chosen for examination because of their potential for creating motivation and the objective referent standards that they provide. As mentioned earlier, a key component of self-regulatory based theories is the notion of some kind of reference point to which achievement or progress can be compared. Outcome goals clearly provide this type of standard, which was proposed to be important in helping direct attention and effort on the task. Furthermore, it is likely the case that in skill acquisition-based settings, a balance of both learning and performance are important. While gaining knowledge is certainly desirable, one has to ultimately be able to perform a given task at a certain level. Providing people with too much freedom (via mastery goals) may counteract this. A common thread occurring in all of these explanations is the notion that goals and regulatory activities are related but goals do not lead directly to appropriate strategy selection and performance monitoring processes on complex tasks. This is consistent with my suggestion that goals alone may be necessary but not sufficient for achieving high levels of performance in a complex task situation. The importance of systematic l6 strategy development and monitoring must be considered as well. Based on the divergence in research findings discussed above and the notion that goals do not lead directly to optimal strategy selection and performance monitoring, some researchers have attempted to identify boundary conditions that may moderate the success of challenging, specific goals on complex task performance. Kanfer et a1. (1994) examined the impact of challenging, specific goals on performance on an Air Traffic Controller task under two different conditions of practice. The first was a massed practice condition in which the subject completed multiple practice blocks with little or no time between them. The second was a spaced practice condition where the individuals were given free time between the practice blocks. Based on the Kanfer & Ackerman (1989) resource allocation model, Kanfer et al. (1994) suggested that free time would free attentional resources for performance monitoring and reflection during the off-task periods and not interfere with or take away resources needed for skill acquisition. Consistent with their predictions, Kanfer et al. (1994) found that challenging, specific goals led to increased performance in the spaced practice condition and decreased performance in the massed practice condition as compared to DYB goals. Barley, Connolly, & Lee (1989) took a different approach and provided two types of strategy selection assistance to participants. The first was a strategy reduction aid that provided a list of 10 possible strategies (including the correct one) for performing a stock value prediction task. The second involved training individuals with proper procedures for testing and modifying their own strategies. The basic premise of this study was that goals would mobilize effort, which in turn would lead to higher performance when the 17 individuals were provided with assistance in developing or finding appropriate task strategies. Indeed, their results suggested that individuals with strategy selection assistance of some sort and challenging, specific goals outperformed those given DYB goals. This is consistent with the logic Earley, Connolly, & Ekegren (1989) offered for explaining deficits in the goal-perfonnance relationship on complex tasks. Based on this body of research, a few tentative conclusions regarding the impact of challenging, specific goals on the performance of complex tasks can be made. First of all, consistent with past theory on goal setting (Locke et al., 1981; Wood & Locke, 1990) goals are not likely to impact performance positively unless the individual can translate the effort and energy associated with them into successful strategies for performance. As the results reported by Early, Connolly, & Lee (1989) suggest, some mechanism or training may need to be in place to help people find appropriate strategies; effort in not enough. Kanfer et al.’s (1994) results suggest that spaced (rather than blocked) practice may be one of these mechanisms in that it allows people time to reflect on the usefulness of their plans or strategies without taking away attentional resources needed for task performance. Finally, DeShon & Alexander (1996) provide that one must consider the type of task when setting goals so that the goals do not lead to the use of explicit self- regulation strategies that may interfere with necessary implicit learning. 18 Metacognition As stated earlier, metacognition is a process by which individuals use metacognitive knowledge to select strategies. Also, individuals use metacognitive control to devise plans, monitor their performance or learning in real time, and evaluate the success of their strategies and plans (Ertrner & Newby, 1996). To appreciate the impact metacognition can have on leaming and performance, it is essential to understand what exactly is involved in the process of metacognition and how it evolved as a construct. The idea that people can monitor and evaluate their own thinking and learning processes is hardly new. Philosophers and psychologists have long wondered about the nature of things metacognitive, even though they did not use the same term to describe them. The following quote from Speannan (1923, p. 52-53) does a good job of highlighting this: “Such a cognizing of cognition itself was already announced by Plato. Aristotle likewise posited a separate power whereby, over and above actually seeing and hearing, the psyche becomes aware of doing so. Later authors, such as Strato, Galen, Alexander of Aphrodisias, and in particular Plotinus, amplified the doctrine, designating the processes of cognizing one’s own cognition by several specific names. Much later, especial stress was laid on this power of reflection, as it was now called by Locke.” While one can see that these past thinkers and researchers contemplated the nature of higher order thinking, a well-developed manner of describing the nature of these “cognized cognitions” was not within their grasp. In particular, many people had trouble with the notion that one can simultaneously be both the thinker and the observer. This 19 dilemma is eloquently stated by the philosopher Comte in his suggestion that “The thinker cannot divide himself into two, of whom one reasons whilst the other observes him reason (Cited in Nelson, 1996).” This statement has been referred to as Comte’s paradox and many researchers and philosophers tried in vain to work around it or find solutions to it. More recent thinkers have taken a number of different approaches to solving the problems associated with Comte’s paradox (Nelson, 1996). One approach, taken by the philosopher Albert Tarski (1956), suggested that two (or more) different levels of reference simultaneously exist in humans and that the differences in the foci of each of these levels could be used to explain why someone can simultaneously do and observe at the same time. The introduction of the construct of metacognition was spurred on by thinking of this manner done by Flavell and his colleagues in a series of works in the 1970’s (Flavell, Friedrichs, & Hoyt, 1970; F lavell, 1976; Flavell 1979) in which they applied this type of solution to cognition and memory. Using Tarski’s levels distinction, cognition can be thought of as an object level representation or “cognitions about something”. Metacognition, then, is a representation and organization of these cognitions or “cognitions about cognitions”. Information is thought to flow back and forth between the meta level and the object level. The type and function of information flowing upward (from object to meta level) is different than the type of information flowing downward (from meta to object level), however, and will be discussed later at greater length. A number of different models and theories regarding metacognition have been proposed by researchers over the past few decades, with many of them taking different vieWpoints and disagreeing on what exactly the nature of the construct should be. For 20 example, some theorists view metacognition as part of a more general type of self- regulation (e. g. Bouffard-Bouchard, Parent, & Larivee, 1993) while others take a somewhat opposing view and suggest that self-regulation is one of the key components of metacognition (e. g. Ertrner & Newby, 1996). While one can hardly suggest that a unified framework exists regarding the nature of metacognition, there are two key elements that are common to a large number of conceptualizations of metacognition: Metacognitive knowledge and metacognitive control processes (Schraw & Moshman, 1995; Winne, 1995; 1996). Metacognitive Knowledge Metacognitive knowledge refers to broadly to what one knows about cognition and the cognitive demands of the task at hand (Schraw & Moshman, 1995). Metacognitive knowledge is abstracted from experience in the domain of thinking and can be constructed based on either forethought or prior experience in multiple types of varied situations. This knowledge is constantly being revised and enlarged during the learning process. Traditionally, three major types of knowledge have been thought of to be useful with regard to metacognition. These are declarative knowledge (knowledge about things), procedural knowledge (knowledge regarding how to do things), and conditional knowledge (knowledge regarding when and why) (Schraw, 1998). Declarative knowledge refers to knowledge about oneself as a learner, knowledge about the demands of the task at hand, or knowledge regarding what additional factors may impact performance. This knowledge can take many different forms and refer to a number of different things. Examples of declarative knowledge include knowledge regarding what types of environment one likes to study in, knowledge regarding what 21 types of strategies could be used on a given task, knowledge pertaining to how easy or difficult a given task may be, or knowledge regarding the level of understanding one has about a given topic. Procedural knowledge can be thought of as knowledge pertaining to the execution or use of different skills or strategies. This is different than declarative knowledge in that one may know of a particular strategy but not know how to properly implement it. Procedural knowledge, then, addresses the question “how?”. A person with high levels of procedural knowledge is not only well aware of a number of different elements or strategies that are important for successful performance (declarative knowledge), but he or she is also able to use this knowledge to his/her benefit. Conditional knowledge is directly related to both declarative knowledge and procedural knowledge in that it is knowledge regarding which strategies or actions are the most appropriate in any given situation. In order to have a high level of conditional knowledge one must not only have a large bank of knowledge regarding what factors influence performance (declarative knowledge) and how to execute different strategies or do different tasks (procedural knowledge), but he or she must also know how these two relate to each other in order to determine what strategies are useful in different situations. Conditional knowledge is likely the most important aspect of metacognitive knowledge in that it is the most proximal precursor to direct action. Metacognitive Control The second key aspect of metacognition is commonly referred to as metacognitive control. Unlike metacognitive knowledge, which is basically static, metacognitive control is a set of active processes that deal with the regulation and control of one’s 22 behavior or thinking (Ertmer & Newby; 1996). From a motivational perspective, one can think of these processes as a central determinant of the direction of behavior. Metacognitive control deals with the manifestation and use of metacognitive knowledge. One important thing to note is that even though many people have knowledge based on previous experiences, many often fail to implement that knowledge to direct firture behavior and successfully accomplish their goals (Garner & Alexander, 1989; Osman and Hannafin, 1992). Metacognitive control processes are very important to both learning and performance in this respect (Reeve & Brown, 1985). There are three primary dimensions of metacognitive control processes; they are commonly referred to as planning, monitoring, and evaluating. Planning refers to a number of strategic behaviors done before actual task engagement. Ertmer and Newby (1996) identify three major tasks that are essential to the planning process. The first is setting or accepting a goal. This goal can come in the form of a self-set goal based on knowledge of the task at hand and what is required for successfiil performance or in the form of a goal set by someone else (e. g. supervisor, manager, etc...) who has a stake in the related learning or performance outcomes. If the task is something that the person has previous experience with, they can draw from their metacognitive knowledge bank regarding the task characteristics. Should the task he a novel one, the person may compare it with similar tasks they have experience with or use forethought to identify what the task demands may be. Forethought may often be partially based on what is referred to by Nelson and Narens (1990) as an ease-of-leaming judgment. These judgments refer to how easy or difficult the individual believes it will be to perform the task at hand. Ease-of—learning judgments are largely inferential and can 23 signal one to think more (or less) about the clarity and depth of goals necessary for successful performance. Once an individual has a goal (or goals) in mind, his or her focus is shifted toward identifying strategies for accomplishing that goal. This decision is based primarily on the declarative knowledge possessed regarding task demands and appropriate strategies as well as procedural and conditional knowledge regarding how well their available strategies match the task demands and specific environmental conditions. The proper selection and sequencing of strategies can be very important in that it lays the foundation for successful performance. Having a clear set of strategies also allows gives the individual freedom to use his or her cognitive resources for on-line regulation purposes, which can be vital during the early stages of skill acquisition and learning (Kanfer & Ackerman, 1989). These strategies can take a number of different forms, as pointed out by Ertmer and Newby (1996). For example, they suggest that successful strategy development could center not just around what a person should be doing but also around what type of environment would be the best for pursuing one’s goal (assuming that one has control over this). The final step in the planning process involves identifying potential obstacles that may stand in the way of successfully achieving one’s goal. This again involves forethought, although at a somewhat deeper level than in the goal setting/acceptance stage. One must again rely on previous stores of metacognitive knowledge to identify what types of problems may occur and how he or she can prepare his or herself to get around them. To a certain degree, one can think of this step as contingency planning. For example, individuals might decide that they need to get a final report done for work 24 on the following Monday, so they will spend some extra time in the office on Friday evening. This would be the plan or strategy. What would happen, however, if a computer crashed or close friends showed up in town at the last minute? Those more metacognitively oriented would be expected to be more likely to have planned for potential disturbances such as these and have contingency plans in place that would increase the probability of successfully attaining their original goals. Monitoring is the second key aspect of the metacognitive control process and is suggested by Schraw and Moshman (1995, p. 355) to refer to “one’s on-line awareness of comprehension and task performance.” Monitoring is the part of the metacognitive process where the difference between cognition and metacognition can be most eloquently illustrated. Reconsidering to the afore mentioned distinction between the object (cognitive) level and the meta level helps to highlight the notion that the information flowing between the two levels is not the same (Nelson, 1996). Monitoring, which can be either conscious or unconscious, results in the flow of information from the lower object level up to the higher meta level. Here, the object level information one has gathered is organized and suggestions for changes in behavior are made if any discrepancies are found. These suggestions are then put in place at the object level; this is the control aspect and represents the downward flow of information. As can be seen, this is a fairly complex process that involves many steps. In particular, there are three major steps that the metacognitively oriented performer is constantly going through while actually performing the task. The first of these steps involves actively monitoring one’s own performance in order to observe exactly what actions are being performed. This involves a number of 25 different types of judgments, with one of the most important being what Nelson and Narens (1990) refer to as a judgment of leaming (J OL). These are simply judgments of how well one is learning the content at hand. If one extrapolates to a task performance situation, these can be thought of as judgments regarding how well one is currently performing. These judgments are important in that they provide the basis for detection of possible discrepancies between current levels of performance and targeted levels as specified by plans and goals. The second major step involved in the monitoring process is the comparison of one’s current progress to his or her estimated progress. This type of judgment revolves not only around performance but also around how well one’s pre-selected strategies are working. Should one be performing or learning adequately, there is no need for revisions to the initial plan. The performer can simply continue to monitor his or her behavior. However, discrepancies can and do occur, and when this is the case the individual must rely upon one of their pre-formed contingencies or reformulate plans in order to continue performing successfully. This kind of refonnulating and re-planning is the third major step involved in monitoring and is somewhat different from that involved in the planning stage in that it occurs in real time while the performer is working on the task. It is also different in that the performer now has more information to work with and is better able to develop contingencies for any problems that may arise in future learning or performance. All of this monitoring and on-line planning can, however, become overwhelming at times. While there is a natural temporal ordering between planning, monitoring, and evaluating, one does not necessarily progress through these stages linearly. At any time, 26 people utilizing metacognitive control processes can shift back and forth between different stages in the control process in order to facilitate leanring or task performance. This may happen when one encounters an unforeseen obstacle or realizes that the strategies planned out prior to task engagement are not working. The person may stop actively working on the task (if he or she has the discretion to do so) and start over in the planning stage. While this sounds like a cumbersome process, it may take only a few minutes (or seconds) for the learner to make a new plan and get back to performing the task. Evaluating is the final step of the metacognitive control process and it occurs toward the conclusion of the skill acquisition or goal striving process. This step involves assessing both the process employed to attain one’s goals and the product achieved (Ertmer & Newby, 1996). An important distinction should be made at this point between a performer who utilizes metacognitive control processes and one who does not. Frequently, people simply evaluate the success of their performance only. For example, they may ask themselves “Did I do a good job?” Performers who are actively engaging in metacognitive processes, however, would not only assess the success of their goal striving, but also the effectiveness and efficiency of the process used along the way. They would be concerned with how well their planned strategies worked, how well they adjusted to any problems that they encountered, and how efficiently they used their time. All of this is done in order modify their plans in the case that they encounter a similar task or situation in the future. This, of course, will likely be the case in an organizational setting where they are required to be learning on a nearly continuous basis. 27 This process of reflection is key in building future metacognitive knowledge. It is difficult to understand how to use different strategies (procedural knowledge) and when and where different strategies are effective (conditional knowledge) if one does not reflect upon the times he or she attempted to use them and how successful or unsuccessful they were in those situations. It is not difficult to see how the reciprocal processes of planning, monitoring, evaluating, and reflecting could lead to increased levels of metacognitive knowledge, which in tum could help one better develop goals, strategies, and plans for future performance. As can be seen by this explanation of the elements involved in metacognition, strategies are paramount. Metacognitive knowledge is the basis for strategies and metacognitive control processes are used to make plans, monitor their usefulness, and make necessary modifications or changes. What is not explicitly considered, however, is where the energy or motivation to do all of this comes from. While metacognition theorists do consider the importance of a goal as a reference point, they do not traditionally consider the importance of goals for providing necessary energy and effort. Without this effort, it is unlikely that people will use the metacognitive knowledge or skills they may possess. Nevertheless, researchers have been able to find links between metacognition and outcomes such as learning and skill-based performance. Metacognition and Performance A number of researchers have examined the impact that metacognition can have on the ability of an individual to perform well in different types of settings. These studies have primarily focused on skill acquisition or learning and deal primarily with students who are performing in classroom environments. The primary notion driving these studies 28 is that individuals with greater metacognitive skills should be able to learn or acquire skills better because they can quickly identify the problems they are having and remedy them by implementing or devising different, more appropriate strategies. Metacognition has often been treated as a static individual difference variable and has been measured without manipulation. For example, Pintrich and DeGroot (1990) conducted a study in which they examined the relationship between high levels of metacognition, in the form of planning, performance monitoring, and effort management strategies, and a number of different classroom outcomes for middle school students. They found that high levels of metacognition were correlated with higher GPA as well as with better performance on a number of different aspects of classroom achievement such as exam performance and proficiency in writing class reports. Furthermore, their results suggested that high levels of metacognition explained unique variance in their criteria of interest above and beyond that explained by motivational factors (i.e. self-efficacy, intrinsic task value, and test-taking anxiety) as well as cognitive strategy use. Landine and Stewart (1998) found similar results in correlational study and concluded that high levels of metacognition, defined broadly in this case, were positively related to GPA in high school students. A Minnaert and J anssen (1999) investigated the impact of metacognition, in the form of regulatory control processes, on college student academic performance over and above that of intelligence. In particular, Minnaert and J anssen (1999) tested whether or not varying levels of regulatory control could explain unique variance in academic performance over and above that predicted by an individual’s verbal, numerical, and diagrammatic intelligence. Across levels of each type of intelligence, these researchers 29 found that people with higher levels of metacognition were more academically successful than were those who exhibited low levels of metacognition. Swanson (1990) conducted a somewhat similar study with elementary school students and found corroborating results. In this study, however, the focus was more on metacognitive knowledge than on regulatory control processes. Swanson’s (1990) results suggested that high levels of metacognitive knowledge, which was assessed by asking questions regarding how knowledge stores influence future performance, were related to problem solving ability on two different tasks across both high and low levels of aptitude. Furthermore, he found an interaction between aptitude and metacognition such that individuals who were high on both solved problems better than did those who had only high levels of aptitude or high levels of metacognition. Like much of the research done on goal setting, these studies primarily focused on tasks or learning objectives that were not tremendously complex or were somewhat familiar to the performers. The relationship between metacognition in its different forms and learning or problem solving may become more variable as the task becomes more complex or more novel, such as that which would be faced in a learner controlled training environment. Researchers coming from a resource allocation perspective, such as Kanfer and Ackerman (1989), would likely suggest that expending effort on regulatory activities early on in the process of skill acquisition might hamper performance because the attentional resources used for regulation may be better spent on simply trying to learn the basics of the tasks. This would be especially true for complex tasks. If this were the case, we would expect to find that the use of metacognitive strategies might actually impair initial learning or performance. 30 An altemative explanation, put forth from the viewpoint of Early, Connolly, and Ekegren (1989) however, would suggest that high initial task demands might inhibit the effectiveness of strategy search and selection. If this is the case, then, it is likely that people with high levels of metacognitive skill may be better able to perform well initially, even though the task demands are high. The strategic benefits of using metacognition may guide attention in a more systematic way, thus reducing sporadic and maladaptive strategy search and selection. A handful of recent studies have examined the impact of high levels of metacognition on learning and problem solving in novel, complex environments. As part of a larger overall model, Ford, Smith, Weissbein, Gully, & Salas (1998) examined the impact of metacognition on performance, where performance was a number of different learning outcomes, in a situation in which the learner was given a high degree of control over the training procedure. Their study utilized college students who performed a complex military-based decision making task. They found that the self- reported use of metacognitive control processes was positively related to levels of declarative knowledge gained in training and higher levels of training performance. These outcomes, in turn, were related to higher levels of performance on a transfer task. Clause, Delbridge, Schmitt, Chan, & Jennings (2001) investigated the relationship between metacognitive control processes and performance in an employment testing situation. More specifically, these researchers conducted a field study in which they examined the ability of people applying for entry-level law enforcement positions to apply knowledge regarding departmental procedures to a number of different simulated situations. Clause et al. (2001) found that applicants who utilized metacognitive control 31 when preparing for the employment test exhibited more effort in preparation and were more likely to utilize deep, complex leaming strategies. In their model, metacognition exerted an indirect influence on test performance through these two factors. Taken together, these two studies support the viewpoint of Barley, Connolly, & Ekegren (1989) who suggest that high levels of metacognition can benefit performance in novel or difficult learning or problem solving situations. A study conducted by Pokay & Blumenfeld (1990) provides an exception to the positive link between metacognition and performance. This work lends some support to Kanfer & Ackennan’s (1989) resource allocation interpretation of complex task performance. Pokay & Blumenfeld’s (1990) study examined the impact of metacognition on the classroom performance of high- school geometry students at two different points in time. These researchers found that the use of metacognitive control strategies early on in the process of geometry skill acquisition (at the beginning of the semester) was actually somewhat detrimental to performance. However, as time progressed and the students became more familiar with the material, the use of metacognitive strategies was related to higher levels of performance. Based on these findings, one can draw the conclusion that the use of metacognition, in particular metacognitive control strategies, in learning or problem solving environments is likely beneficial to the learners’ performance. This seems to hold true even when the task is novel or complex, although a person likely needs some basic knowledge about the task before metacognitive planning and control strategies become viable. These results need to be interpreted cautiously, however. In many of these studies, metacognition was examined in settings where performance was very goal 32 directed (e. g. classrooms, job application tests, etc. . .). Because of this, it is difficult to understand the usefulness of metacognition when people are not striving toward a well- defined, challenging goal and do not necessarily have the motivation associated with these types of goals. Regardless of this, it should come as no surprise that researchers have attempted to develop interventions and training programs aimed at enhancing people’s levels of metacognition during leaming and problem solving endeavors. Metacognitive Interventions As has been emphasized throughout this section, the primary purpose of metacognitive interventions is to ensure that the techniques taught to individuals to help them regulate their learning can be transferred to and successfully used on tasks other than those in which the techniques were originally taught (Reeve & Brown, 1985). As Reeve and Brown (1985) clearly state, “. . .the aim [of metacognitive training] is the generalization of skills.” Unfortunately, researchers have not addressed ability of interventions to lead to strategies that generalize across of a number of tasks, although many studies focusing on interventions have found promising results for their given tasks. Interventions and training programs come in a number of different forms and levels of complexity and have been shown to elicit various levels of success on different types of leaming tasks. These intervention studies have focused primarily on metacognitive control processes, especially those related to performance. This comes as no surprise because monitoring skills do not require stores of knowledge or past experience and are likely easier to acquire than declarative, procedural, and conditional task knowledge. Studies examining metacognitive control interventions are reviewed next 33 In a set of studies examining a method called “reciprocal teaching”, Brown and Palincsar (Brown & Palincsar, 1982; 1985; Palincsar & Brown, 1984) examined the effect of a metacognitively oriented training program on the reading skills of poor learners in junior high school. This method encouraged the student and the teacher to take turns presenting passages and leading dialogue related to them in order to get the student to engage in processes such as self-review, questioning, and clarifying meaning, all of which are related to metacognitive control. The results showed that students participating in the reciprocal teaching program showed large gains in terms of both classroom and standardized tests of reading comprehension. Another successful procedure was used in a number of studies conducted by King and colleagues (King, 1989; King, 1990; King 1991a; King, 1992; King & Rosenshine, 1993). This procedure involved teaching learners to use generic question stems such as “How are and similar” or “How does affect ...” to generate questions during leaming that would help them to monitor their comprehension as well as integrate and recall previous knowledge more successfully. These questioning strategies have been found to lead to greater increases in understanding and comprehension as compared to controls across a variety of different ages ranging from 5th graders to college students and have been especially helpful for students of average ability (Rosenshine, Meister, & Chapman, 1996). Berardi-Coletta, Buyer, Dominowski, & Rellinger (1995) utilized a technique in which they asked participants questions aimed directly at assessing strategy use and monitoring during performance on a series of fairly simplistic problem solving tasks. These questions such as “how do you know that this is a good move?” were thought to 34 focus participants on monitoring their success and planning upcoming behaviors. Participants who were asked these questions performed better on the problem solving tasks (solved the problems in less steps) than did students who were asked questions about surface level elements of the problem facing them, participants simply asked to think aloud while performing their task, and a control group who simply worked on solving the problems quietly and were asked no questions. King (1991b) utilized questions similar to those used by Berardi-Coletta et al. (1995) but required the students to ask the questions of themselves and their partners. In the King (1991b) study, pairs of 5th grade students performed a series of relatively simplistic figural-spatial problems in one of three conditions. In the guided self- questioning conditions, students were given a problem-solving prompt card with questions pertaining to the metacognitive control processes of planning (e.g. “what is our plan”), monitoring (e. g. “are we on the right track”), and evaluating (e. g. “what would we do differently next time”), that they were instructed to ask each other and themselves throughout the performance periods. In the second condition, students were simply instructed to ask their partners questions, but were given no guidance in the type of questions to ask. The third condition was a control with no explicit instructions on questioning. Students in the guided questioning condition outperformed students in the other two conditions on tests of written problem solving ability and the ability to solve a problem on a novel task. As stated earlier, these studies focused primarily on training planning and performance monitoring and evaluation skills, skills that are primarily related to metacognitive control. A few other studies, however, have been somewhat broader in 35 scope. These studies have focused on providing individuals with metacognitive knowledge, primarily in the form of task strategies (declarative knowledge) and procedural knowledge regarding how to appropriately use these strategies. This is likely more difficult as declarative and procedural knowledge require time to build up and conditional knowledge, which involves knowing what strategies to use in different situations, cannot be built without experience using strategies in different situations. These studies are reviewed next. Scardamalia, Bereiter, and colleagues (Bereiter & Scardamalia, 1982; Scardamalia & Bereiter, 1982; Scardamalia, Bereiter, & Steinbach, 1984) have conducted one of the most thorough metacognitive training programs with the goal of improving the written composition skills of young students via training both metacognitive knowledge (e. g. strategy selection and development) and control (e. g. performance monitoring) processes. Their training program involved a comprehensive combination of direct strategy instruction for reconciling conflicting ideas when writing, presenting students with cues to stimulate self—questioning during composition planning, and reciprocal modeling by both students and teachers of “think aloud” strategies thought to help extemalize the writer’s cognition. The results of their training program suggest that students gained an increased ability to reflect on ideas and problems and the ability to better structure their written compositions. In a similar but less thorough study, Delclos and Hanington (1991) examined the ability of 5th and 6th grade students to solve problems when given (a) task training only (b) task training plus basic problem solving training or (c) task training, basic problem solving training, and self-monitoring training via self-questioning strategies. Basic 36 problem solving training can be thought of as metacognitive knowledge training in that it helped provide students with declarative and procedural knowledge regarding different types of problem solving strategies. Students in the self-monitoring training condition received a booklet with questions similar to those used by King (1991b) aimed at helping them monitor their performance at different stages of task completion. As expected, students receiving both the problem solving training and the self-monitoring training were able to solve more difficult problems correctly than students in either of the other two conditions. While the metacognitive training procedures and interventions reviewed here have been successful, their success may not be directly generalizable to organizational settings. The majority of the studies used elementary or junior-high school students or deal with performance on simple problem solving tasks or academic material, neither of which parallel the complexity of tasks that people in modern organizations must learn how to perform. Others use interventions so complex and time-consuming that it would be difficult to justify implementing them in organizations (i.e. Brown & Palincsar, 1982; Palincsar & Brown, 1984; Bereiter & Scardamalia, 1982; Scardamalia et al., 1984). Furthermore, these studies deal mostly with classroom situations in which the learner and teacher are working together or the learner receives a large amount of up-front instruction regarding the task; they do not examine the use of metacognitive interventions or training in settings where the learner is primarily responsible for their own success. Research has begun to emerge, however, that is focused on being more organizationally relevant. Schmidt and Ford (2003) conducted a study in which they examined the impact of metacognitive interventions on a number of different learning and performance 37 outcomes. The task used in this study is involved training college students on how to use software designed for constructing an intemet webpage. The training program was designed so that learners had a high degree of control over what information to study, the sequence with which they progressed through the material, and the amount of time they spent on each part of the training and on the training as a whole. Schmidt and Ford’s (2003) metacognitive training intervention consisted of three parts, all of which were aimed at increasing the performance monitoring and evaluation ability of the learners. The first utilized self-questioning strategies and generic question stems similar to those used by King and colleagues (King, 1989; King, 1990; King 19913; King, 1992; King & Rosenshine, 1993). The second part of the metacognitive training intervention instructed individuals to delay self-questioning briefly after studying based on research suggesting that delayed judgments of learning are much more accurate than those made immediately (e. g. Nelson & Dunlosky, 1991). The third part consisted of prompting trainees every 10 minutes throughout training to assess their current level of understanding of the material that they had already covered. Schmidt and Ford’s (2003) results suggested that their metacognitive training intervention interacted with individuals’ perfonnance-avoidance goal orientations (e. g. Elliot & Church, 1997; VandeWalle, 1997) to impact metacognitive activity. More specifically, individuals not afraid of demonstrating incompetence were more receptive to the metacognitive intervention. Metacognitive activity, then, was related to higher levels of declarative knowledge regarding the webpage design task and higher levels of performance on the actual task of building a webpage. No direct, overall effect of their intervention on performance was found, however. 38 Taken as a whole, this research suggests that metacognitive interventions have been successful in a number of different areas and on a number of different tasks. While these findings are encouraging to a certain degree, they do not present a clear picture as to what kind of impact metacognitive interventions have in organizational settings. As Schmidt and Ford (2003) show, there may be boundary conditions that determine the impact that metacognitive interventions have on the use of metacognitive control processes and subsequently learning and performance. Identifying boundary conditions may be the key to successfully implementing these types of interventions, especially in settings where learners are in control of their own progress. While metacognitive interventions quite likely have beneficial effects on performance in these types of settings, contingencies in addition to those suggested by Schmidt and Ford (2003) likely exist. One of these contingencies may be the type of goals that the individuals given metacognitive interventions are pursuing. 39 Goal Setting and Metacognition Goal setting theory and research (e. g. Locke, 1968; Locke et al., 1981; Wood & Locke, 1990) clearly suggest that goals impact levels of effort and energy exerted. This research and theory, however, does little to specify how energy is translated into successful strategy selection and use on complex tasks. Simply assuming that people are able to select, implement, and monitor the utility of strategies is problematic, as research suggests that adults often fail to monitor behavior or use existing self-regulatory skills (6. g. Garner & Alexander, 1989). Unlike simple tasks where working hard leads to better performance, effort and persistence are not sufficient for successfirl performance on more complex tasks. This is illustrated by Earley, Connolly, & Lee (1989), who provide empirical support for the notion that effort derived from challenging, specific goals is only translated into successful performance on a complex task when performers have some sort of help with selecting and implementing strategies. Clearly, more attention needs to be given to how these strategies can be devised and implemented successfully, especially when the learner or performer is in control and is not receiving outside assistance. Traditional goal setting research simply does not provide strong enough theory pertaining to these potentially important determinants of complex task performance. Research and theory on metacognition (e. g. Ertmer & Newby, 1996; Nelson, 1996; Schraw & Moshman, 1995), however, provide considerable insight into the ways with which individuals select, implement, and monitor the usefulness of learning and performance strategies. Those in the field of metacognition, however, seem to overlook the impact of goals as energizers of behavior. In metacognition theory, goals are 40 primarily considered as referent standards used to evaluate learning or performance success against. This is the opposite side of the coin. Strategies and regulatory processes are explicitly focused on, while motivation to actually learn or perform well is relegated to being a background concern. This is troublesome as initial strategy selection and monitoring of strategy effectiveness based on Judgments of Learning (Nelson & Narens, 1990) may not lead to high levels of learning and performance on a complex task if a person is simply “trying to do their best”. Challenging, specific goals and metacognitive skills, specifically those related to strategy selection, performance monitoring, and evaluating, may compliment each other in leading to high performance and learning on complex tasks. This may be especially true when an individual is primarily responsible for the success of his or her learning and performance. Challenging, specific goals should lead to high levels of effort and persistence. Those who are capable of using metacognitive skills to select strategies, monitor their usefulness, and make changes when necessary, then, should better harness this energy. As Wood, et a1. (1990. p. 183) state: “on complex tasks, increased effort will not be translated into productivity gains unless effective strategies are developed for deploying that effort productively.” Little research has been done examining the potentially interactive influences of goal-setting and metacognitive interventions on learning and performance in complex task settings. One study, however, conducted by Ridley, Schutz, Glanz, and Weinstein (1992) did take an initial step in this direction by examining the interaction between goal setting (challenging, specific self-set goal vs. no goal) and high vs. low metacognitive awareness as measured by the Private Self—Consciousness scale (F eni gstein, Scheier, & 41 Buss, 1975). Their results suggest that neither goal setting nor high vs. low “metacognition” showed significant main effects on performance on a medical decision simulation. However, the two did interact such that highly “metacognitive” pe0ple who received the goal setting intervention performed the best overall. While these results are encouraging, they need to be interpreted with caution. First of all, the measure of metacognition used in this study did not focus explicitly on metacognitive knowledge or control, but rather on overall propensity to reflect upon one’s own thoughts and actions (Fenigstein et al., 1975). While this may be similar in some ways to metacognitive control processes, it likely taps a construct much less systematic than metacognition as discussed here. This is illustrated by the fact that Private Self-Consciousness scale was reported by Ridley et al. (1989) to correlate only .21 with the metacognition scale of the Motivated Strategies for Learning Scale (Pintrich et al., 1988), which is one of the more widely used self-report measures of metacognitive control skills. The focus of this study is on examining the impact of goals and metacognition on learning and performance on a complex task in a learner-controlled training environment. This goes beyond previous research in that it incorporates the potential benefits of both goal setting and metacognitive interventions in an environment where they should be very useful. On novel, complex tasks, especially when an individual is in control of their learning, the motivational benefits of goal setting and the strategic benefits associated with using metacognitive control processes are likely more important than in settings where tasks are simple or individuals are given external strategy or assistance or instruction. 42 Models and Hypotheses Figure 1 presents a general model of the impact of goals on task performance. This model is consistent with Goal Setting Theory (e. g. Locke & Latham, 1990a; 1990b). Implicit in this model are the mediating mechanisms effort, persistence, and direction, which should exert their influences on performance concurrently. These mechanisms deal not only with how hard people work but also by how long effort is maintained and what direction it is exerted in. Energy can be expended on either strategic or task activities, and both are thought to be important determinants of task performance. On simple tasks, high levels of energy expended have been shown to be sufficient for increased task performance (e. g. Locke et al., 1981). Energy expended can be translated directly into successful performance on simple tasks primarily because the strategies are straightforward or well known. Individuals do not need to devise new strategies or exert large amounts of effort finding existing strategies; the strategies for successful performance are fairly self-evident. Working hard on task activities is of primary importance as almost anyone is capable of finding and using appropriate strategies. Indeed, as Wood and Locke (1990) suggest, effort and persistence can be thought of as plans or strategies in and of themselves on simple tasks. Task related activity is not sufficient for high levels of performance on complex tasks, however, as appropriate strategies are not as easily accessible (Wood et al., 1987; Wood & Locke, 1990) On complex tasks, strategic behaviors should play a much more important role in determining successful performance. Because proper strategies are much more difficult to ascertain on complex tasks, simply working hard on task related activities should not lead 43 directly to high levels of performance. Substantial energy will need to be expended on strategic activities such as strategy selection and performance monitoring as well. Referring back to Figure 1, this would suggest that the effect of task activity on performance may decrease or even drop to zero as tasks become more complex, while the importance of strategic activities will remain. These strategic activities, however, are not likely to be successfully executed without some kind of assistance (e. g. Barley, Connolly, & Lee, 1989; Kanfer et al., 1994). In this study, the metacognitive intervention will serve as the strategic assistance. Figure 2 is a model designed to illustrate the hypothesized influences of challenging, specific goals and metacognitive interventions on declarative knowledge levels and performance on a complex task. These outcomes are both knowledge and skill-based and are consistent with suggestions by Kragier, Ford, and Salas (1993) to evaluate multiple training outcomes. These dual outcomes were selected based on Baldwin & Ford’s (1988) assertion that learning about the task (declarative knowledge) is not a sufficient end in and of itself; people must be able to apply this knowledge to the task that was the target of the learning or training procedure successfully (task performance). Based on the literature reviewed above and the discussion of the potential interactive effects of goal setting and metacognition on performance, the primary hypotheses for this study are as follows: Hypothesis 1: Challenging, specific goals with lead to higher levels of task related activities. 44 The basic premise of this hypothesis is that challenging, specific goals provide unambiguous performance targets and that people accepting these goals will not be satisfied until they reach them or get as close to them as possible. High levels of energy expended, then, will be exerted in order to achieve these goals. “Do your best” goals, on the other hand, provide ambiguous targets for performance that are open to the interpretation of the performer. Without a specific reference point, performers may not be as likely to exert high levels of energy and may simply quit when they feel they have worked “hard enough” or have done “well enough”. One of the ways energy expended is thought to manifest itself is through higher levels of task related activities. Specific, difficult goals have been shown to direct attention toward task-related activities and away from unrelated activities (Locke & Latham, 1990b; Locke, et al., 1981). Because of this, people working harder should be spending much of this effort directly on sub-tasks that they believe are relevant to completing the overall task at hand successfully. These types of activities can include such things aslreviewing relevant information and completing practice activities, among others. Hypothesis 2: Goal type will interact with the presence or absence of a metacognitive intervention to influence the amount of strategic behaviors. Specifically, type of goal given to a participant may lead to different levels of energy exerted. Specifically, challenging and specific goals should lead to more energy expended, which in turn is expected to be translated into more optimal strategy selection and monitoring processes when individuals receive metacognitive training aimed at helping them select appropriate strategies, monitor their performance, and evaluate their 45 utility. In a sense, people receiving metacognitive interventions should be able to translate effort into working “smarter” as well as working harder. As described previously, motivation in the form of energy expended is not sufficient to lead to proper strategy selection and evaluation on complex tasks (Barley, Connolly, & Lee, 1989). Metacognitive interventions aimed at helping people select strategies and monitor and evaluate their success, such as the one to be utilized in this study, may also be insufficient if people are not sufficiently motivated to use them. Therefore, high levels of strategic behaviors are predicted to occur when people are both sufficiently motivated (because of challenging, specific goals) and able to translate that motivation into making plans and monitoring and evaluating their effectiveness (because they received the metacognitive intervention). Figure 3 shows the hypothesized interaction between goal type and receiving or not receiving the metacognitive intervention on the level of strategic behaviors. Hypothesis 3: Strategic behavior levels and task activity levels will interact to influence levels of skill-based performance. Earley, Connolly, & Lee (1989) and Barley, Connolly, & Ekegren (1989) both found that less strategy search (which they equated with better overall strategies) was related to higher levels of performance on a stock value prediction task. Earley et al. (1987) found that better strategies, in the form of better task planning, were related to higher levels of performance on business marketing simulation in a laboratory setting and in a field setting with employees at different service organizations. Clause et al. (2001) found that learning strategies such as organization and elaboration were related to higher test performance for entry-level law enforcement applicants attempting to apply 46 knowledge regarding departmental procedures to a number of different simulated situations. Also, Huber (1985) found that across varying levels of complexity of a maze- solving task, individuals who utilized a better strategy performed better. Task related activities are also likely to be related to higher levels of task performance on many types of task. Typically, task related activities are thought to be tied directly to effort, direction, and persistence, three of the goal-perfonnance mediating mechanisms originally suggested by goal setting theory (Locke, 1968; Locke & Latham, 1990a). On tasks of low or moderate complexity, goals should be translated into performance primarily through their influence on these mechanisms, which are likely manifested as higher levels of task related activities. As Wood and Locke (1990) suggest, on less complex tasks proper strategies may simply be working hard and completing task related activities. Wood et al.’s (1987) meta-analysis, however, suggests that this relationship is not as strong on complex tasks. As reviewed earlier, there are no clear conclusions as to the effects of difficult, specific goals on complex task performance or learning, although the most likely case is that there is a small effect at best. While both Strategic activities and task activities (to a lesser extent) should lead to higher levels of performance, it is likely the case that they will also have interactive effects. Specifically, individuals with specific, difficult goals and metacognitive training should perform better due to the motivation to work hard not only on the task itself but also on making plans, monitoring and evaluating their effectiveness, and making changes when necessary. Task activities are necessary to familiarize the individuals with the information and skills necessary to perform well while strategic activities should provide 47 a way to organize and monitor the use of task related activities in a successful manner. Therefore, individuals who exhibit high levels of strategic and task related activities should outperform those who exhibit high levels of strategic activities only, task activities only, or low levels of both. Figure 4 shows the hypothesized interactive effects of strategic and task activities on task performance. Hypothesis 4: Declarative task knowledge should mediate the relationship between the Strategic behavior x task activity interaction and skill-based performance. In a setting like the one to be used in this experiment where the learner has control of the pace, sequence, and content of the training, one would expect that strategic activities aimed at progressing through the training in an optimal way should lead to higher levels of learning about the task (declarative task knowledge). Strategic activities for a task of this type would involve moving through the material in a systematic way so that basic, fundamental knowledge is studied first and later knowledge can be built upon that base. Another type of strategic behavior may involve moving back and forth between different parts of the training to ensure that all aspects are well understood. Learning should be reflected by how well the learner will able to answer questions pertaining to key processes involved in actually completing the task. F urthennore, strategic activities aimed at proceeding through the training should relate to higher levels of skill-based performance, in this case building an intemet webpage. Any increases in learning or comprehension gained by using better strategies to proceed through the training should translate into performance gains. 48 One would also expect that task related activities should be related to higher levels of declarative knowledge gained about the task at hand. In this study specifically, task related activities consist primarily of practicing how to do basic elements of task and acquiring some of the firndamental skills necessary to build a webpage. If a person is actually practicing how to do specific skills or complete certain parts of a webpage, one can reasonably assume that they are learning about how to do that given skill in the process. In a setting where learning is a fundamental part of performance and skill acquisition, one can easily see how difficult it would be to perform well without learning the important declarative aspects of the task at hand. Indeed, learning theorists have long suggested that before one can learn how or when to appropriately do something (i.e. procedural and conditional knowledge), they must first understand the basic declarative knowledge regarding the task or skill. 49 Method Participants One hundred twenty Michigan State University undergraduate students, recruited via the MSU Psychology student subject pool, participated in this study. Students received course credit commensurate with the amount of time their given experimental session was scheduled to last, which was 150 or 180 minutes, depending on the condition. Participants ranged from 18-38 years of age (mean = 19.7, sd = 2.27) and were predominantly (68.4%) female. Six participants were not able to complete the entire study due to a computer network malfunction. Their data were not analyzed, thereby reducing the N to 1 14. Furthermore, incomplete data were obtained for a number of participants due to various technical errors, and the analyses reported were done using pair-wise deletion, which left an analyzed N ranging from 105-114 participants, depending on the analysis. Design A two x two between-subjects design was used in which goals (specific, challenging vs. “do your best”) and the presence or absence of the metacognitive intervention were manipulated, thus creating four conditions in which the participants received either (1) specific, challenging goals and no metacognitive intervention, (2) specific, challenging goals and the metacognitive intervention, (3) “Do your best” goals and no metacognitive intervention, or (4) “Do your best goals” and the metacognitive intervention. All participants were randomly assigned to a condition based on the experimental session attended. 50 Complex Task The task was a variation of the task used by Schmidt and Ford (2003). Participants completed the task on computers equipped with Macromedia Drearnweaver MX webpage design software. The task was an intemet-based webpage training program in which individuals were given complete control over the content they viewed, the sequence of content they proceeded through, and the speed or pace with which they proceeded through the content. The content was broken up into six modules (Titles/keywords, Tables, Publishing, Links, Graphics, and General Formatting), each of which contained information that was important to the proper construction and publication of a webpage. These modules were each broken up into sub-modules that contained specific information on how to complete various different tasks. After completing the training, participants completed a 30-item declarative knowledge quiz designed to assess how much they learned about declarative elements of the webpage building task. The final part of the session consisted of having the participants build an actual webpage by attempting to complete 83 webpage building objectives. Based on Wood’s (1986) model of task complexity, this task was thought to be moderately complex. As there was a lot of information regarding a number of different topic areas, this task should have had fairly high levels of component complexity. Also, information from these areas needed to be integrated together in order for participants to perform well on the final webpage building task. Furthermore, in order to learn how to do the task properly, the information gathered from some of the sub-modules needed to 51 be properly sequenced. These two characteristics of the task suggest a low to moderate level of coordinative complexity. This task had little or no dynamic complexity as the information and standards for performance did not change. Overall, this suggests that the task was moderately complex. Indeed, Wood et a1. (1987) described classroom learning as being a moderately complex task. This task certainly contained many elements of classroom leaming (e. g. many types of information that needed to be learned and integrated), but it was likely. even more complex because of its novelty and lack of pre- imposed structure. Manipulations and Measures Goals The type of goal assigned to participants was based on the experimental condition in which they participated. The goals were set primarily in relation to the final skill- based performance task, which consisted of having the participants build a webpage by completing as many of the 83 webpage building objectives as possible in 35 minutes (see Appendix A for goal manipulations). The content of these objectives will be explained later. Participants in the “do your best” goal condition received goals asking them to simply try to do their best in performing on the final webpage building task. Those in the specific, challenging goal condition were assigned an outcome-based goal for the webpage building task. Based on initial pilot data, the outcome-based goal was set at a level of difficulty thought to be attainable only by the top 10-15% of performers as per recommendations by Wood and Bandura (1989). The final goal was to complete 75 of the 83 objectives in 35 minutes. 52 It is important to note here that as an outcome-based goal, this goal differed somewhat from those traditionally used in goal-setting studies that utilize more simple tasks. On more simple tasks, outcome based goals provide a clear referent against which feedback about performance can easily be judged. For example, if a person’s goal is to generate 50 uses for a particular object (a common task for goal-setting studies), the person knows how many uses he or she has generated and can easily gauge how many more are necessary for goal accomplishment. Indeed, performance feedback is thought to be important in determining the effectiveness of goals (Locke & Latham, 1990b). The goals set in this study, however, provided a less clear referent standard. The goals were set in relation to the final webpage building task, which was separated in time from the webpage training period. The goals were framed in terms of completing webpage building objectives, which were explained to participants as the same skills or tasks that were available in the training program. It should be noted that, while consistent with the kinds of goals often seen in training programs, these goals did not provide the subjects with clean links to specific behaviors needed to learn webpage building tasks during training nor to behaviors that would impact performance on the final webpage building task. Furthermore, there was no explicit feedback from either the experimenter or doing the task about learning these tasks and skills in the training period. Limits in the meaning of feedback for guiding behavior change may have created potential difficulties for subjects in terms of understanding how to meet their goals. The relevance of the way the goals were operationalized will be discussed further when interpreting the results of the study. 53 Metacognitive Intervention The metacognitive intervention consisted of two different parts, each of which was aimed at helping the participants successfully use metacognitive control processes to guide and promote learning and, subsequently, performance. The first part of the intervention involved giving the participants a brief overview of the important aspects of metacognition and informing them of the potential usefulness of making plans, monitoring their levels of knowledge and understanding, and evaluating the success of the different strategies they could use while going through the webpage training program (see Appendix B for the intervention passage used by the experimenter). The intervention was intended to help participants form an accurate representation of the potential usefulness of metacognitive control strategies. The second part of the intervention focused on self-questioning and involved providing participants with a card containing a set of questions pertaining to planning, monitoring, and evaluating that they could ask themselves throughout the training program (see Appendix C). These questions are similar to those found to be successful in previous research (e. g. King, 1991b; Delclos and Harrington, 1991; Schraw, 1998). Participants were required to use these questions to help them progress through the training program. Before beginning the webpage training program, participants who received the intervention were instructed to spend approximately 5-10 minutes thinking about the planning questions and writing down their responses. Upon beginning the webpage training, participants were prompted every 15 minutes to respond to the monitoring self-questions. When the participants who received the intervention were finished with the webpage training program, they were asked to spend roughly five 54 minutes thinking about the evaluating questions and writing down their responses. Furthermore, individuals were informed that they should briefly delay self-questioning, as responses to delayed self-questions have been shown to be much more accurate than ' responses to self-questioning done immediately after covering new information (Nelson & Dunlosky, 1991). Strategic Activities Strategic activities in the training period were assessed in two ways. Repeated page visits. One operationalization of strategic activities involved an objective count of the number of actual sub-module pages visited by each participant more than once. Participants were free to View as many (or as few) sub-modules of the training program as they wished. Furthermore, participants were also able to repeatedly view the same sub-module if they wished to reexamine its content. A database was set up that automatically recorded when each person visited each page in real time. In order for a page visit to be deemed valid, the participant must have viewed the sub-module for at least four seconds. Four seconds was used as a cutoff for two reasons. Conceptually, a cutoff needed to be placed so that random page visits caused by participants rapidly clicking through the training program would not be counted because participants could not view the content of the page when they rapidly clicking through them. Based on examination by the experimenter, four seconds seemed to be the bare minimum an individual could spend looking at a page before realizing what the content of the page actually was. Empirically, there was a sharp cut-off in the frequency of page visits once times became 4 seconds or longer. This suggests that many people rapidly clicked through different parts of the training and possibly did not examine the content in these 55 pages. Examination of the data revealed that many of the recorded page visits that were less than 4 seconds occurred in conjunction with strings of others, suggesting that some participants did indeed have periods where they rapidly clicked through modules without examining the content. The rationale for the number of page visits four seconds or greater as an operationalization of strategic behaviors was that individuals thinking more about how to complete the training properly, or individuals who monitored their learning more consistently, should not only have viewed all of the modules, but they should also have viewed modules multiple times if they realized that they did not remember or fully understand instructions or information that they covered earlier in the training program. Based on the amount of knowledge contained in the training program, it was deemed that this would likely be the case. Furthermore, these individuals should have been more likely to review the pages to ensure they covered the content in each of the sub-modules. Participants who were not as strategic in their approach to the task should have been more likely to have went through the different modules and sub-modules in the order that they were presented and should have viewed each sub-module only once because they were not monitoring their comprehension and understanding. Metacognitive activity. Strategic activities were also measured via self-reports on a 19-item scale that assessed the use of metacognitive planning and monitoring activities. This scale was adapted from Schmidt and Ford (2003). These items are listed in Appendix D. Participants responded on a five-point scale ranging from “strongly disagree” (1) to “strongly agree” (5). This scale had high levels of internal consistency reliability overall (a = .87). The measure was included as an alternative to the objective 56 measure of repeated sub—module page visits and as a basis for establishing convergent validity. These two measures of strategic activities, however, were uncorrelated (r = 0.04, p = n.s.). Task Activity Task activity was Operationalized as the number of practice tasks completed by participants while they went through the webpage training. Seven separate opportunities were provided within the sub-modules of the training program for people to practice building different parts of a webpage. These practice tasks were accessed by clicking on links that read, “to practice this task, please click here”, where the word “here” served as a link to a practice task. Participants were allowed to complete as many of these tasks as they saw fit but were not required to complete any of them. Furthermore, participants were allowed to complete any individual practice task multiple times if they chose to do so. These practice tasks were fairly easy to complete as instructions for how to complete them were displayed directly on the screen. The number of practice tasks completed was designed to serve as an objective indicator of how much energy individuals exerted toward working directly on the task. I coded the number of practice tasks that each individual completed. While the instructions suggested that each practice activity be saved and labeled separately for ease and accuracy of coding, this procedure was not followed by many of the participants. Therefore, coding required a close examination of the data for each individual. Three separate decision rules were used to decide what constituted a valid practice attempt. The first decision rule was that practice activities did not need to be completed correctly. There only needed to be evidence that the participant attempted some parts of the task. 57 The second decision rule was that any activity that was saved in a unique file as per the instructions was counted. The final decision rule was that unique practice activities were counted as valid when saved in the same file as other practice activities, provided that there was evidence that the activities were done uniquely for that file and not simply copied or retained from the files generated for other practice attempts. Any practice activity that met one of these three standards was coded as a practice activity. While the decision rules discussed above greatly simplified and standardized coding, some instances were slightly ambiguous and the coding was left up to my judgment. To determine the reliability of the coding, I initially coded the number of practice activities for each participant. I then waited for approximately one week before randomly sampling 16 participants and blindly re-coding the number of practice activities completed for each of them. The results suggested a perfect correspondence with the initial coding in terms of total number of practice activities completed as well as how often each participant completed each individual practice activity. Declarative Knowledge Declarative knowledge of the task was assessed by a 30-item multiple-choice quiz that focused on major topics presented in the training (see Appendix E). These questions were aimed at all different aspects of the task in order to assess a wide range of the knowledge necessary to build a webpage. Participants’ overall number of items correct served as the index of their declarative knowledge. The mean declarative knowledge level of the sample was 23.7 (sd = 6.7), suggesting that there was ample variance in this measure. Furthermore, only one individual obtained a perfect score and nobody scored worse than 12, suggesting that 58 floor or ceiling effects related to this variable as a criteria were not likely to be major issues. Task Performance Task performance was assessed by having the participants construct an actual webpage using the commands and tasks described in the training program. Participants were given a list of 83 individual objectives necessary to complete a webpage (see Appendix F). Each objective focused on having a participant execute one of the skills that was contained in the training system (c. g. insert a table, format text, etc. . .). This webpage was originally designed by the author to include nearly all of the activities described in the training program. The level of task performance was calculated as the number of these tasks correctly completed by each participant in 35 minutes. The 35- minute time frame was imposed primarily because of the nature of the software being used for webpage construction. Even if participants did not learn how to complete many of the necessary activities, it was feasible that they could simply figure them out by trial- and-error. Based on initial pilot data and the judgment of the primary investigator, 35 minutes was deemed to be a reasonable amount of time for participants to complete the objectives necessary to build the webpage if they learned how to sufficiently complete most of the tasks presented in the training program. The mean task performance score of the sample was 56.1 (sd = 21.4), suggesting that there was ample variance in this measure. Furthermore, only three individuals obtained perfect scores and only three people scored worse than 15, suggesting that floor or ceiling effects were not problematic. 59 Control Variables Participants’ prior experience with building webpages was assessed by 3three items (“I have a lot of experience building webpages”, “I have built many of my own webpages”, and “I do not have any experience at all building webpages”). Participants responded on a five-point scale ranging from “strongly disagree” (1) to “strongly agree” (5). Computer experience was also examined as a potential covariate and was assessed using an adapted version of the Potosky & Bobko (1998) measure (see Appendix G). Cognitive ability, as measured by self-report ACT/SAT scores, was also included as a potential control variable. Ability was calculated by standardizing self-reported ACT or SAT scores based on normative data. Although researchers have shown that metacognition, mostly in the form of metacognitive control processes, is related to performance or learning independent of cognitive ability, examining the potential incremental effects of the metacognitive intervention over and above cognitive ability was still potentially interesting. Procedure Once participants arrived at their scheduled session and gave their informed consent (see Appendix H for consent form), they completed a questionnaire that assessed their demographic characteristics (age and gender), previous webpage building experience, level of computer experience, self-report ACT or SAT scores, trait goal orientation, and personality characteristics. Next, the participants were given a general overview of the task including a brief synopsis of the types of information contained in each of the training modules and general instructions on how to operate and navigate through the training system. Once this was completed, participants in the appropriate 60 conditions were given the first part of the metacognitive intervention. After these initial overviews and instructions, participants were assigned the appropriate goals for their given condition. Once goals were assigned, participants completed the brief goal commitment questionnaire and began the webpage training. When participants decided that they were finished with the training program, they were asked to fill out a brief measure that assessed the cognitive effort they exerted in proceeding through the training program, the amount of off-task thoughts they had while proceeding through the training program, the amount of attention they paid to their assigned goal, and their metacognitive activity. Once this questionnaire was completed, each participant was offered a five minute break where and was encouraged to think about the task and reflect upon what he or she did. After this time had passed, the declarative knowledge quiz was administered. Upon completing the quiz, each participant was given a sheet with a list of objectives necessary for building the final webpage, as well as a sample of the webpage to serve as a guide, and was instructed to complete the task. Once the participants completed all of the objectives or the 35-minute time limit expired, they were debriefed and thanked for their participation in the study (see Appendix I for debriefing form). 61 Results Means, standard deviations, intercorrelations, and alpha reliabilities (on diagonal when applicable) for the variables of interest are in Table 1. Control Variables Webpage building experience, computer experience, and cognitive ability were simultaneously used as control variables for the tests of each of the hypotheses. As a whole, these variables only functioned as significant covariates for the analysis regarding Hypothesis 3 and parts of the analyses pertaining to Hypothesis 4. The pattern of significant results obtained using these variables as controls, however, was not different from the results obtained when these variables were not included as controls. Because of this, the analyses and results reported do not include webpage building experience, computer experience, and cognitive ability as control variables. Hypothesis 1 Hypothesis 1 suggested that challenging, specific goals would be related to higher levels of task related activities than “do your best” goals. The correlation between goal condition and task related activities as indexed by the practice activities variable reported in Table 1 suggests that goal condition did not impact the amount of task related activities competed. A series of exploratory post-hoe analyses were conducted to examine relationships similar to those specified by Hypothesis 1. The central tenant of Hypothesis 1 was that high levels of motivation (suggested to come from challenging, specific goals) should have lead to higher levels of energy expended on task-relevant behaviors such as completing practice activities. While the predicted effect was not observed, perhaps the 62 effect was moderated by individual differences related to motivation. A series of analyses was conducted to examine whether challenging, specific goals impacted levels of practice activities differently for individuals with varying levels of (a) mastery goal orientation, (b) goal commitment, or (c) attention to assigned goals. In general, goal orientation can be thought of as the type of goal an individual pursues in achievement situations. People with a mastery goal orientation tend to focus on goals that deal with learning and developing competence (e.g., Dweck, 1986; Dweck & Leggett, 1988). In general, consistent positive relationships have been found between mastery goal orientation and intrinsic motivation (e. g. Ames & Archer, 1988; Duda & Nicholls, 1992) and the use of cognitive and metacognitive learning strategies (e. g. Pintrich, 1989; Pintrich & Garcia, 1991). However, the relationships between mastery goal orientation and learning outcomes such as declarative knowledge and classroom performance are somewhat ambiguous (e. g. Elliot & Church, 1997). Goal orientation was measured in this study using the Horvath, Scheu, and DeShon (2001) measure. A hierarchical linear regression analysis was conducted to examine the possibility that challenging, specific goals influenced the level of practice activities differently for people with different levels of mastery goal orientation (see Table 2). The results of this analysis revealed a positive main effect of mastery goal orientation on level of practice activities that approached statistical significance (,6: .79, p= .06, two-tailed). The interaction between mastery goal orientation and goal condition, however, was not significant (AR2= .00, p= n.s., two-tailed), suggesting that goal condition did not influence the level of practice activities differently for people with different levels of mastery goal orientation. 63 Goal commitment refers to the dedication of individuals to attaining goals that they set or are set for them. A number of studies suggest that in order for goals to affect performance, individuals must be committed to attaining them (e. g. Erez & Zidon, 1984). Furthermore, goal commitment is thought to be most important and relevant when goals are difficult (Klein, Wesson, Hollenbeck, & Alge, 1999). In this study, it was suspected that challenging, specific goals could have impacted task activity more for individuals who were more committed to attaining their goals. Goal commitment was measured in this study using the Hollenbeck, Klein, O’Leary, and Wright (1989) measure. A hierarchical linear regression analysis was conducted to examine the possibility that challenging, specific goals influenced the level of practice activities differently for people with different levels of goal commitment (see Table 3). The results of this analysis revealed no significant main effect of goal commitment on level of practice activities (6: .40, [F n.s., two-tailed). The interaction between goal commitment and type of assigned goal, also failed to reach statistical significance (AR2= .00, p= n.s., two- tailed), suggesting that goal condition did not influence the level of practice activity differently for people with different levels of goal commitment. Attention to assigned goals refers to how often individuals think about their assigned goal or worry about reaching their goal as they proceed through skill acquisition (Kanfer et al., 1994). One may expect that challenging, specific goals are more likely to influence task activity levels when people attend more to the goals they are assigned. Attention to assigned goals was measured using the Kanfer et al. (1994) measure. A hierarchical linear regression analysis was conducted to examine the possibility that challenging, specific goals influenced the level of practice activities differently for 64 people who paid more or less attention to their assigned goals as they progressed through the training (see Table 4). The results of this analysis revealed a significant positive main effect of attention to the assigned goal on level of practice activities (fl: .85, p< .05, two- tailed), suggesting that regardless of goal condition, people who were more focused on attending to their assigned goal completed more of the practice activities. The interaction between attention to one’s assigned goal and goal condition, however, was not significant (AR2= .00, p= n.s., two-tailed), suggesting that goal condition did not impact the number of practice activities completed differently for people with different levels of attention to their assigned goal. Hypothesis 2 Hypothesis 2 suggested the type of goal one was assigned would interact with the presence or absence of the metacognitive intervention to influence the amount of strategic activities performed. Two hierarchical multiple linear regressions were used to examine this hypothesis, as strategic activities were assessed in two ways: metacognitive activity and repeated page visits. For both regressions, step 1 regressed the amount of strategic activities (using the given operationalization) onto goal condition and a dummy coded variable indexing the presence (dummy coded 1) or absence (dummy coded 0) of the metacognitive intervention. These independent variables were entered simultaneously. In step 2, the amount of strategic activities was regressed onto the interaction term, which was the product of the dummy coded variable indexing presence or absence of the metacognitive intervention and goal condition. The results of the regression analysis for the first operationalization of strategic activities, metacognitive activity (see Table 5), suggested no interactive influence of 65 goals and the metacognitive intervention (AR): .00, p= n.s., two-tailed). However, a significant main effect for the metacognitive skills intervention was found (6: .30, p< .05 , one-tailed), although there was no main effect for goal condition (fl= .01, p= n.s., one-tailed). This suggests that the metacognitive intervention directly impacted the amount of metacognitive activity reported by participants. The results of the regression analysis for the second operationalization of strategic activities, number of repeated submodule page visits (see Table 6), also suggested no interactive influence of goal condition and the metacognitive intervention (AR2= .00, p= n.s., two-tailed). Furthermore, the results suggested that neither goal condition (B: 1.18, p= n.s., one-tailed) nor the presence or absence of the metacognitive intervention (,6: 1.92, p= n.s., one-tailed) had a significant main effect on the number of repeated page visits. The two operationalizations of strategic activities did not correlate with each other (see Table 1), thus suggesting that they were not assessing the same latent construct. Because the number repeated page visits was not affected by the metacognitive intervention and did not correlate with reported metacognitive activity levels, the construct validity of the measure was questionable. Because of this, the repeated page visits operationalization was not used for later analyses. A series of exploratory post-hoc analyses were conducted to examine relationships similar to those specified by Hypothesis 2. The basic premise of Hypothesis 2 was that people would need to be motivated to use metacognitive strategies during skill acquisition. Challenging, specific goals were suggested to provide some of that motivation, although it appears that these goals did not impact motivation to acquire 66 skills and perform well in this study. Because of this, the analyses presented here examined two variables that may have been distally related to motivation to perform, mastery goal orientation and computer experience. As stated earlier, mastery goal orientation refers to one’s focus on goals that deal with learning and developing competence (e.g., Dweck, 1986; Dweck & Leggett, 1988). If mastery orientation served as a motivational driver to acquire skills, it could have interacted with the presence or absence of the metacognitive intervention to influence strategic activities exhibited during the webpage training program. If so, participants that had higher levels of mastery orientation would have been more willing to accept and use the techniques provided by the metacognition training program than would participants that had low levels of mastery goal orientation. A hierarchical linear regression analysis was conducted to examine this possibility (see Table 7). The results of this analysis revealed a significant positive main effect of mastery goal orientation on level of metacognitive activity (,6: .25, p< .05, two-tailed) along with a significant positive main effect of the metacognitive training intervention (6: .34, p< .05 , two-tailed). Furthermore, the interaction between mastery goal orientation and the presence or absence of the metacognition intervention approached statistical significance (4R2: .024, p= .07, two-tailed), suggesting that the metacognitive intervention was better utilized or accepted by people that had higher levels of mastery goal orientation. Computer experience was also examined as a construct that may have been related to motivation to perform in this context. Specifically, one could have expected that individuals with higher levels of computer experience may have enjoy working with 67 computers more, thus they would have had higher levels of intrinsic motivation to work on the task and acquire skills in this domain. Because of this, they would have been more willing to engage in strategic activities and accept the metacognition training. A hierarchical linear regression analysis was conducted to examine these possibilities (see Table 8). The results of this analysis revealed a positive main effect of computer experience on level of metacognitive activity that approached statistical significance (,8= .14, p= .06, two-tailed) along with a significant positive main effect of the metacognitive training intervention (,6: .30, p< .05, two-tailed). The interaction between computer experience and the presence or absence of the metacognitive training intervention, however, was not statistically significant (AR2= .008, p= n.s., two-tailed). Individuals with high computer experience were not more receptive to the metacognition training intervention that those with low computer experience. Hypothesis 3 Hypothesis 3 suggested that strategic behavior levels and task activity levels would interact to influence levels of task performance. Hierarchical multiple linear regression was used to examine this hypothesis. In step 1 of the hierarchical linear regression, webpage score, measured as the number of objective completed on the webpage building task, was regressed onto self-reported metacognitive activity (strategic activities) and number of practice activities (task activities). In step 2 of the hierarchical linear regression, webpage score was regressed onto the interaction term, which was product of metacognitive activity level and number of practice activities completed. The results of this analysis indicate that there was a significant positive main effect of practice activities on performance (,B= 1.80, p< .05, one-tailed), although there 68 was no significant main effect of metacognitive activity (,6= 2.15, p= n.s., one-tailed). This analysis also indicated that there was no interactive effect of metacognitive activity and practice activities on webpage score (21122: .003, p= n.s., two-tailed). . Post-hoe analyses were conducted for potential relationships related to Hypothesis 3. The basic premise of Hypothesis 3 was that motivation (from challenging, specific goals) and skills or abilities (e. g. metacognitive strategies and abilities) would interact to influence task performance. This hypothesis parallels many that suggest performance is a function of ability (or skills) x motivation. As can be seen by examining the results pertaining to Hypothesis 1, it seemed that challenging, specific goals did little to impact motivation in this context as indexed by amount of practice activities. Mastery goal orientation, then, was examined as a possible indicator of motivation. A hierarchical linear regression analysis was conducted to examine the possibility that mastery goal orientation interacted with metacognitive activity to influence webpage score (see Table 10). This analysis revealed a significant positive main effect for mastery goal orientation (,6: 8.3, p< .05, two-tailed), although the main effect for metacognitive activity was not significant (fl= -1.05, p= n.s., two-tailed). The interaction also failed to reach statistical significance (AR2= .001, p= n.s., two-tailed). Hypothesis 4 Hypothesis 4 suggested that one’s level of declarative knowledge about the task would mediate the relationship between the interactive effects of strategic activities and task activities on overall levels of task performance. Baron and Kenny (1986) suggest that there are four steps that must be tested to establish mediation: (a) the initial independent variable must related to the dependent variable, (b) the initial independent 69 variable must be related to the mediator, (c) the mediator affects the dependent variable, and (d) the effect of the independent variable on the dependent variable controlling for the mediator is zero. Because the test of Hypothesis 3 established that there was not a significant relationship between the initial independent variable (the interaction term between practice activities and metacognitive activity) and the dependent variable (webpage score), mediation could not occur and no mediation test was conducted. Nevertheless, a hierarchical linear regression analysis similar to that used to test Hypothesis 3 was conducted with practice activities and metacognitive activity as independent variables and declarative knowledge as the dependent variable (see Table 11). The interaction between practice activities and metacognitive activity did not significantly predict declarative knowledge (AR2= .006, p= n.s., two-tailed). Furthermore, it should be noted that declarative knowledge levels were positively associated with levels of task performance (see Table 1). Using a rationale similar to that which guided the post-hoc analyses for Hypothesis 3, a hierarchical linear regression analysis was conducted to examine the possibility that motivation (in the form of mastery goal orientation) interacted with metacognitive activity levels to influence declarative knowledge levels (see Table 12). This analysis revealed a significant positive main effect for mastery goal orientation (,8: 1.29, p< .05, two-tailed), although the main effect for metacognitive activity was not significant (,6: .21, p= n.s., two-tailed). The interaction, however, approached statistical significance (AR2= .02, p= .12, two-tailed), suggesting that there may have been a small multiplicative effect. Interestingly, the interaction suggested that high levels of mastery 7O goal orientation were more strongly related to declarative knowledge levels when people exhibited lower levels of strategic activities. 71 Discussion The focus of this study was on examining two possible mechanisms responsible for successful learning and performance in complex tasks settings. Many approaches to performance and learning in complex settings take a self-regulation perspective in which goals are the central focus and strategies and tactics are devised and enacted to achieve them. Specifically, two areas of research that focus on aspects of self-regulation, goal setting and metacognition, were proposed to be particularly relevant in this study. Goal setting research deals primarily with goals as precursors to direct action. Goals are thought to mobilize energy, direct effort, and provide referent standards for performance. The mechanisms through which goals are thought to influence performance reflect this: effort, direction, persistence, and strategy use (Latham & Pinder, 2005; Locke et al., 1981; Locke & Latham 1990a; 1990c). Effort, direction, and persistence all deal with the mobilization of energy on behaviors related to achieving goals. Strategy use, on the other hand, deals with plans or tactics devised with respect to where energy will be allocated over time (Locke et al, 1981). Goal setting has been found to have robust effects on performance on simple tasks where performance is directly tied to how hard one works. The effect of goal setting on complex task performance, however, is less clear. At best, goals are thought to have a small positive impact on complex task performance (Wood et al., 1987). The decreasing effect of goal setting as tasks become more complex is thought to be primarily due to the notion that goals may not adequately facilitate proper strategy selection and generation for tasks where simple, effort-based strategies are no longer solely effective (Wood & Locke, 72 1990). This process of strategy selection, generation, and regulation, however, is not well explicated in the goal setting literature. Metacognition deals explicitly with strategy development, implementation, and revision and is a process by which individuals use prior knowledge to select strategies, make plans, monitor their performance or learning in real time, and evaluate the success of their strategies and plans after performing (Ertmer & Newby, 1996). Research on metacognition has been spurred on by the rather robust finding that adults often fail to monitor their thinking and behavior (Garner & Alexander, 1989). Metacognition research has focused primarily on how metacognitive control skills related to planning, monitoring, and evaluating can lead to higher levels of knowledge gain and leaming. Researchers examining metacognition suggest that there should be a reference point in place in order for individuals to monitor and evaluate the success of strategies and overall performance. This reference point is traditionally thought to come in the form of a goal. While the importance of having a goal or reference point is readily acknowledged, metacognition research does not typically deal explicitly with how these goals are devised or the impact they have on the perfonner’s motivation. Instead, this research focuses on a more “cold” cognitive process that is concerned with somewhat mechanical strategy selection and performance monitoring. There is some overlap between goal-setting theory and metacognition theory. Both deal with self-regulation and acknowledge the importance of both motivation and higher-level cognition (in terms of strategy development and monitoring). However, each literature tends to focus primarily on one part or the other, without explicitly detailing the importance of both motivation and higher-order cognition at the same time. 73‘ In an attempt to integrate these two literatures and gain a more complete understanding of performance and learning on complex tasks, this study was devised to examine the impact of goal setting and metacognition concurrently. The basic premise that drove this study was that in order for individuals to gain knowledge and perform well, it may be important that they have both specific, difficult goals to increase their level of mobilized energy and a way to use this energy systematically to devise strategies and plans in the manner suggested by metacognition theory. The present study examined a number of different hypotheses that dealt with the potential impact of challenging, specific goals and a metacognitive skills intervention on strategic and task-related activities. In turn, these activities were suggested to influence declarative knowledge gain and task performance (see Figure 2). In the main, these data failed to support the hypotheses. While the theoretical rationale for this study seemed reasonable, there was an overarching problem that greatly constrained the findings. Limitations The major overarching problem with this study centered on the distinction between performance and learning and how they were discussed and thought about during the design of this experiment. In skill acquisition settings, both learning and performance are thought to be important as declarative knowledge gain is considered to be a necessary precursor to successful task performance. In drawing from the literatures related to metacognition and goal setting, the argument made for this study was that goals would provide the motivation for studying and gaining knowledge while the metacognition intervention would provide the strategic benefits necessary to effectively direct that effort in pursuit of declarative knowledge gain. A disconnect exists here, 74 however, that was not observed when this study was designed. Specifically, constructs related to metacognition and goal setting deal with two different parts of the skill acquisition process. Constructs related to metacognition deal primarily with learning and knowledge gain. The study of metacognition has its roots in Educational and School psychology and has primarily focused on learning as the key criterion. Even though performance is explicitly considered in many metacognition studies, it is performance based on declarative knowledge gain as a manifestation of leaming. For example, studies by Pintrich & DeGroot (1990) and King (1990; 1991a; 1991b; 199lc) focus outcomes such as GPA, exam performance, lecture understanding and comprehension, and written problem solving ability, all of which are referred to as performance in their respective studies. While these types of outcomes certainly reflect performance of a certain type, they are qualitatively different from the types of performance outcomes often examined in goal setting studies. Goal setting constructs, on the other hand, typically focus more on performance once a task is routinized or well learned. Here, the motivation generated by goals can be directly tied to behaviors that facilitate performance. The types of strategies that goal- setting theory considers typically deal more with how to perform well on a given task once the basic requirements are well understood rather than on how to acquire knowledge about the task. While the process related to regulation as discussed by goal-setting theory and metacognition theory may be similar at an abstract level (e. g. making plans, monitoring the usefulness of strategies, etc. . .), they deal with different types of content and come into play at different times during the process of skill acquisition. Therefore, 75 these two literatures may be more difficult to reconcile with one another than I had originally suggested. This problem of not explicitly considering learning and performance as separate processes was particularly important in this study because of the task that was used. The webpage building task was aligned much more closely with those traditionally used by metacognition researchers than those used by goal-setting researchers. The selection and use of this type of task, and the subsequent disconnect between learning and performance, lead to a number of more specific problems or limitations related to the operationalization of key variables and the design of the study. These specific limitations will be discussed in turn, and their impact on the tests related to each of the hypotheses will be discussed in an upcoming section. Design and Operationalization Limitations The nature of the goals. The first of these limitations deals with the way that the goals operated in the study. In traditional studies examining challenging, specific performance goals and “do your best” performance goals, participants are able to track their progress and self-assess how close they are to achieving their goals. Such is the case when goals, performance, and performance outcomes are expressed in the same metrics. In this study, specific and difficult performance goals and “do your best” performance goals were set in relation to performance on the final webpage building task. These goals, however, were intended to influence behavior and motivation to learn during the training period. This posed something of a problem in terms of how participants were able to monitor their progress in relation to their goals. The way in which the goals were 76 framed likely made it quite difficult for people to link the goals to the types of behaviors in the training period that were necessary for goal accomplishment. During the training period, the criterion for performance (webpage building objectives completed) was not directly related to specific activities that could be done to learn or improve a given skill. While participants were informed that they would need to learn many or all of the different tasks or skills in the training system, there was little one-to-one correspondence between these skills as presented in the training and the objectives for the final webpage building task. Simply stated, if participants did not know the criterion on which their performance would be based, it would have been difficult for them to translate goals into motivation during the training as there would have been no easy way to determine exactly what tasks or skills they should devote their effort. Another limitation related to the goals in this study was the absence of objective performance feedback during the training period. It is well established that in order for goals to increase motivation and lead to higher levels of performance, people must have some sort of feedback available to them that lets them know how well they are performing (Locke & Latham, 1990a; 1990b). In this study, however, little or no explicit feedback existed. It was expected that those with higher levels of metacognition would be able to generate their own feedback in terms of how well they were performing on certain aspects of the tasks via self-questioning and online monitoring. It now appears that generating this feedback was probably difficult to do. There was also no feedback generated by the training system or by the experimenter that the participants could have used to evaluate whether or not they were learning how to do things correctly. 77 Metacognitive intervention. Another potential limitation of this study deals with the way metacognition was conceptualized and how the intervention was operationalized. The goals set for participants explicitly dealt with performance and skill exhibition and focused on learning only indirectly. The metacognition intervention, however, specifically focused on making plans for learning and monitoring knowledge levels. As discussed earlier, there was no objective feedback during the training period that could have been used to monitor behavior with respect to performing well On building a webpage. The lack of feedback may have made it difficult even for the people with high levels of metacognition to relate their levels of leaming or understanding to potential increases in performance. Without a clear performance standard and little to link training behaviors to performance, it is likely that people who were more metacognitively active regulated their behavior around judgments of leaming or “feelings of knowing” (Nelson & Narens, 1990), which may or may not have been accurate. Strategies. Another potential limitation deals with the types and variety of strategies that could have led to high levels of learning and/or performance. It was likely the case that the strategies were too straightforward and that the abilities to make better plans and monitor learning more successfully were not necessarily useful additional skills for this task. In terms of learning, strategies were probably straightforward. The webpage building training task was complex in the sense that there was high component complexity, which simply means that there was a lot of information (Wood, 1986). However, there were only small to moderate levels of coordinative complexity and no dynamic complexity of any sort. The tasks and information available in the training 78 system were not predominantly sequenced in ways that would make it difficult to learn correctly how to do one task without having the background knowledge associated with other tasks or areas of expertise. Furthermore, task demands did not change. A lack of both of these types of complexity may render the ability to select different learning strategies and estimate potential obstacles unhelpful with respect to influencing webpage construction performance. Without high levels of coordinative and dynamic complexity, the proper strategies for leaming basically boiled down to covering all of the material and making sure that judgments of learning and feelings of knowing were subjectively high. While people who were more metacognitively active may have been better at this, it might not have been enough of an advantage to meaningfully separate them from those who were not. In terms of performance strategies, it was likely the case that working harder and exerting more effort were the most appropriate strategies for performing well once knowledge was gained. Again, goals may not have helped here as it may have been difficult, if not impossible, to associate behaviors during the training with goal progress. Furthermore, the metacognition training did not deal with monitoring and regulating performance during the final training period; it only focused on leanring during the training period. Therefore, any regulatory skills gained from the metacognition training could not be expected to translate to regulation of performance. Cross-sectional design. Another limitation of this study was the cross-sectional nature of the design. In tasks where self-regulation is important, cycles of learning and performance continually occur where one can judge current levels of knowledge or performance against a referent standard. Indeed, in tasks of this nature one would expect 79 those who are able to. more accurately regulate to exhibit higher levels of learning and performance. This may be especially important in terms of the influence of metacognitive knowledge on leaming. This knowledge is gained from experience and can only influence future decisions if learning is cyclical in nature. In this study, however, the learning (training) and performance periods were entirely distinct. When selecting and designing the task, it was thought that people would be able to regulate their levels of learning and potential performance by making plans and monitoring their progress during the training period. However, as mentioned earlier, the feedback necessary for this type of cyclical regulation was not available to the participants. Because of this, the training period was probably more linear in nature making regulation less of a lever for success than is the case for learning over time. The single cycle nature of the experience created a discontinuity between the strategic or metacognitive knowledge that could have been gained timing the training and the final task experience confronted by the participants. Knowledge of process. The final design limitation is the lack of process variables assessed in this study. As noted above, this study intended to examine learning and performance in a cyclical manner. The hypotheses of the study were based on the assumption that goals and metacognitive training would affect mediating processes that would then influence the outcome variables of declarative task knowledge and webpage building performance. An attempt was made to measure some of these types of variables is this study (e. g. metacognitive activity, off-task thoughts, cognitive effort, etc. . .). Two of these measures, goal commitment (e. g. Hollenbeck & Klein, 1987; Klein & Wright, 1994; Tubbs, 1993) and attention to assigned goals (e. g. Kanfer et al, 1994), have been 80 shown to be particularly important in the goal setting process and were explicitly focused on as potential constraints. Upon examination, it appeared that participants had sufficiently high levels of both goal commitment and attention to their assigned goals (see Table 1), thus suggesting that low levels of either could not serve as valid explanations for why the hypotheses related to goal setting did not receive support. All of these process-related measures, however, were assessed after the training period was completed and before people worked on the webpage building task. This was primarily done to allow the webpage construction to go on without obtrusive measures. In hindsight, the availability of in situ process measures would have been more helpful in attempting to explain the general lack of effects. Measures of this nature would also have allowed for the possibility of within-person analyses exploring the potential existence of regulatory cycles of planning, monitoring, and evaluating during the training period. With the measures utilized in this study, however, I was relegated to examining mean levels of process-related activities estimated afler the fact. Execution Limitations Along with the limitations related to design and the perfonnance-leaming distinction discussed above, there were also limitations related to the execution of the study that may have impacted the findings. Statistical power. The first limitation related to the execution of the study is the amount of statistical power available. Statistical power can be thought of as the probability of correctly rejecting the null hypothesis given (a) the sample size, (b) the effect size, (c) alpha, and (d) the power of the statistical test (Cohen, 1969). Power analyses conducted before data collection suggested that a sample size between 96 and 81 180 participants would be necessary to find the hypothesized effects, based on prior findings and theory-based estimates of effect size. Because of missing data, technology problems, and general time constraints, this study only had an analyzable sample of between 103 and 114 cases, depending on the analysis conducted. While this was sufficient according to the liberal estimates of effect size, it was toward the low end of acceptable. Lack of statistical power renders it difficult to find significant relationships and potentially inflates Type 11 error rates. This is something of a concern for all of the hypotheses tested in this study. The majority of the hypothesis tests revealed effects in the proper direction, although they were quite small. Statistical power concerns may be most pertinent for tests of the interactions, though, which were prominent parts of the proposed model. Aguinis and colleagues (Aguinis, 1995; Aguinis, Boik, & Pierce, 2001; Aguinis & Stone-Romero, 1997) discuss the problem of low statistical power in moderated multiple regression at length. Furthermore, the tests of multiplicative effects, especially those in Hypotheses 3 and 4, dealt with small proposed main effects, which may have made it more difficult to find interactions effects (Rogers, 2002). If these factors truly impacted the findings of this study, a much larger sample (perhaps 50-60 participants per condition) may have been necessary to properly test the model. Collecting further data to in order to obtain significant effects is probably not a useful endeavor considering the small effect sizes that were obtained. None of the interaction terms, which would be most affected by lack of power, explained even one percent of the incremental variance over and above their respective main effects. F urthennore, these effects may not be practically significant even if enough power is 82 obtained to render them statistically significant. Also, because of the small effect sizes, collecting data from the number of extra participants necessary to make these effects statistically significant will likely not be worth the large amounts of additional time and effort needed. Sample issues. There were two possible concerns with the sample collected for this study. The sample consisted entirely of undergraduate college students who were participating in order to receive class extra credit. This may or may not have been problematic, as many of these students fi'equently learn new information and skills and, therefore, the type of task they were working on may not have been too foreign or unreasonable. Furthermore, it is reasonable to think that environments where this type of learning and skill acquisition may have to take place (organizations, for example) are populated with people who are educated to at least the high school and more likely the college level. Therefore, this sample may not have differed appreciably from the target population. The main concern with the participants was their motivation to perform well. No monetary incentives of any type were offered, and participants did not receive more or less credit based on how well they performed. While there were participants who seemed very motivated to do well and learn how to build the webpage, this type of intrinsic motivation was not uniform across the entire sample. Certainly, research examining goal setting and metacognitive interventions in field or lab settings that offer more potential for evaluation or rewards based on levels of skill acquisition may be necessary to more fully understand their effects. 83 The discussion above highlights a number of different limitations related to both the design and execution of this study. As suggested earlier, these limitations likely constrained the findings related to many or all of the hypotheses tested in this study. The following section provides a brief review of the rationale for each the hypotheses tested, the findings, and some of the limitations that may have impacted them. Findings and Related Limitations Hypothesis 1 Hypothesis 1 suggested that challenging and specific goals would lead to higher levels of task related activity and effort. This hypothesis was based on the notion that challenging, specific goals provide unambiguous performance targets and that people accepting these goals will not be satisfied until they reach them or get as close to them as possible (Locke et al., 1981; Locke & Latham, 1990a). High levels of energy expended, then, should have been exerted on task related activities, such as practice, in order to achieve these goals. Hypothesis 1 was not supported. The type of goal that participants were given (challenging and specific vs. “do your best”) did not have a significant effect on the number of practice activities that they completed. While the goal-setting-effort relationship is fairly well documented in a number of different studies (Locke & Latham, 1990b; Locke et al., 1981), there is one major limitation of this study that can help explain why this relationship did not hold true for these data. As described earlier, it was probably difficult for participants to link the type of behaviors necessary in the training period to actual performance levels on the webpage building task. Because of this, one would not expect task performance goals to be related 84 to the number of practice activities completed because participants would not know how practice behaviors would relate to actual performance. Hypothesis 2 Hypothesis 2 proposed that goal condition would interact with the presence or absence of the metacognitive intervention to influence the amount of strategic behaviors exhibited by participants. Specifically, it was thought that challenging and specific goals would lead to more energy expended, which in turn would be translated into more optimal strategy selection and monitoring processes when individuals received the metacognitive training. Hypothesis 2 was not supported. Although there was a direct, positive effect of the metacognitive intervention on metacognitive activity, goal condition and the presence or absence of the metacognitive intervention did not have the predicted significant multiplicative effect. The major limitation related to this hypothesis was that participants were probably not able to translate goals into motivation or effort-related behavior during training. This would offer a reasonable explanation for why the metacognitive intervention had a main effect but did not interact with the goal setting manipulation. Without the added motivation to perform well from the goals, participants may not have meaningfully differed in terms of their motivation to embrace the metacognitive intervention and, therefore, a multiplicative effect would not exist. Another limitation focuses on strategies for learning during the training period. As mention earlier, the strategies for learning were not as complex and multifaceted as was originally expected. Because of this, high levels of metacognitive activity may not 85 have been necessary on the task to determine the appropriate strategies for gaining knowledge about the task. Hypothesis 3 Hypothesis 3 proposed that strategic behavior levels and task activity levels would interact to influence skill-based performance. A number of different studies have examined the relationship between strategies or strategic activities and task performance (Clause et al., 2001; Barley, Connolly, & Ekegren, 1989; Barley, Connolly, & Lee, 1989; Barley et al., 1987). The results of these studies suggest fairly consistent positive relationships between various different operationalizations of strategy use and performance. Task related activities are thought to be tied directly to the goal- perforrnance mediating mechanisms of effort, direction, and persistence (Locke, 1968; Locke & Latham, 1990a), and are also proposed to influence performance. On tasks of low to moderate complexity, research shows that goals are translated into performance primarily through their influence on these mechanisms. Wood et al.’s (1987) meta- analysis suggests that the goal-performance relationship is not as strong on complex tasks as on more simple tasks, as effort cannot be directly translated into performance as easily. While both strategic activities and task activities were originally expected to lead to higher levels of performance, the primary focus of hypothesis 3 was on examining potential interactive effects. Specifically, individuals given specific, difficult goals and metacognitive training were expected to perform the best on the task due to both high motivation to work hard on the task itself and on making plans, monitoring and evaluating their effectiveness, and making changes when necessary. 86 Hypothesis 3 also failed to receive support. There were no multiplicative effects of task and strategic activities on overall performance, although there was a positive main effect of task activities on performance. There was also no significant main effect of strategic activities on performance. A number of the limitations discussed earlier may be useful in pointing out reasons why this hypothesis was not supported. The lack of a significant interactive effect is likely related to the lack of different strategies available for performing well on the task. As discussed earlier, there simply may not have been that many different strategies for performance other than exerting effort once knowledge was gained. Furthermore, in light of the previous discussion regarding the disconnect between learning and performance in this study, it was probably unreasonable to expect metacognition to directly relate to performance, even in an interactive way. Regulation of strategies for performance was not the focus of the metacognitive intervention. Other concerns related to floor, ceiling, or restriction of range effects for the variables included were also considered as possible limitations. These do not seem to be valid concems, however, as means were reasonably centered on the given scales for each of variables included in the analyses related to Hypothesis 3 and each variable also had reasonable amounts of variance (see table 1). Hypothesis 4 Hypothesis 4 proposed that declarative knowledge levels would mediate the relationship between the proposed strategic activity x task activity interaction and webpage building performance. It was suggested that in this setting, where the learner had control of the pace, sequence, and content of the training, strategic activities aimed at 87 progressing through the training in an optimal way would lead to higher levels of learning about the task (declarative task knowledge). It was also expected that task related activities would be related to higher levels of declarative knowledge gained about the task, as these types of activities would consist primarily of practicing how to do basic elements of task and acquiring some of the fundamental skills necessary to build a webpage. Consistent with learning theory, these increases in learning or comprehension were then expected to lead to increased levels of performance. A necessary condition for Hypothesis 4 was the observation of a strategic activity x task activity interaction that affected webpage building performance. Recall that that interaction was not significant. Therefore, declarative knowledge could not have mediated the relationship between the strategic activity x task activity interaction and webpage building performance. Analyses were conducted, however, to examine potential main effects and multiplicative effects of metacognitive activity and practice activities on declarative knowledge levels. This may have been the more apprOpriate analysis from the start in terms of examining the impact of the metacognitive intervention, as the leaming/performance disconnect in this study made it difficult to expect any effects of goals on learning or of metacognition on performance. None of these relationships were significant, however. Furthermore, there was ample variance on declarative knowledge quiz scores, suggesting that restriction of range, floor effects, and ceiling effects were not necessarily plausible concerns (see Table 1). Some of the aforementioned limitations of the study likely impacted the findings related to Hypothesis 4 as well. In particular, the lack of different strategies for learning may have made it difficult for the metaocgnitive intervention to influence levels of 88 declarative knowledge. Furthermore, the cross-sectional nature of the learning and performance periods would have made it difficult for monitoring and regulation of strategies and learning to be successful, even if different strategies did exist. Finally, the lack of appropriate process measures constrained any exploration of actual behaviors during the training period that may have shed light on how the metacognitive intervention operated. An Alternative Model One major problem with the proposed model in this study was the way the goals operated. It was difficult for participants to translate the goals into motivation to engage in relevant behaviors during the training. This was earlier attributed to the fact that the performance goals were not tied directly to behaviors during the training period and that performance feedback was not available during the training. Since goals did not serve as a reliable driver of behavior, I sought to examine an alternative form of motivation. Mastery goal orientation may serve as a potentially useful motivational construct in this type of setting. People with mastery goal orientations tend to focus on goals that deal with learning and developing competence rather than goals that focus directly on performance (Dweck, 1986; Dweck & Leggett, 1988; Elliot & Dweck, 1988). In a setting like this study where learning and declarative-knowledge driven performance are the primary outcomes of interest, self-generated leaming goals may lead to increased levels of motivation to exert effort and persist during learning. Because mastery oriented people focus on competence or skill acquisition rather than specifically on performing well on a given task, they may also be more willing use metacognitive strategies related to planning, monitoring, and evaluating their level of knowledge. Thus, the 89 metacognitive intervention may have been particularly well accepted by those with a mastery orientation. While metacognitive activity did not translate into learning in this study, presumably because of the task’s lack of strategic complexity, these relationships are still of interest for informing future work. Guided by theory and research related to goal orientation and metacognition, Schmidt and Ford (2003) developed a model of the interactive influences of goal orientation and a metacognitive skills intervention on metacognitive activity, declarative knowledge gain, skill based performance, and self-efficacy. Their theoretical model is shown in Figure 5. Schmidt and Ford’s (2003) results primarily suggested that the presence or absence of a metacognitive intervention interacted with goal orientation to influence their three learning outcomes. Those with low performance-avoid orientation were more willing to embrace the metacognitive intervention, and they subsequently exhibited higher levels of metacognitive activity, declarative knowledge gain, skill-based performance, and self-efficacy. Although Schmidt and Ford (2003) predicted that mastery goal orientation would also moderate the relationship between the presence or absence of the metacognitive intervention and metacognitive activity, that prediction was not supported. Using the Schmidt and Ford (2003) model as a guide, and based upon the post-- hoc analyses related to the original hypotheses tested in this study, a speculative alternative model of the impact of motivation and metacognitive activity on declarative knowledge gain and knowledge-driven task performance was constructed (see Figure 6). This model includes mastery goal orientation as the primary motivational construct. It is important to note that many of the general limitations discussed in relation to the original 90 model in this study (e. g. lack of feedback during training, inability to translate goals into behaviors, lack of strategic complexity, etc...) still apply. While this model is similar to the Schmidt and Ford (2003) model, there is one major difference. This model was an attempt to more formally specify the relationships between the outcomes of declarative knowledge gain and skill-based performance. While, Schmidt and Ford (2003) analyzed these as distinct outcomes, this model proposed that declarative knowledge gain should have at least partially mediated the relationships between task and strategic activity and performance. Again, this conceptual separation was important in order to gain a better understanding of the skill acquisition process in this type of setting. The propositions put forth by this model were supported in part by data from this study. The relationship between mastery goal orientation and task activities was supported by the post hoc analyses done in conjunction with Hypothesis 1, which suggested that mastery goal orientation was positively related to the amount of practice activities participants completed. This is consistent with the notion that mastery goal orientation leads to motivation to expend energy on task related activities via exploration and higher levels of intrinsic task interest. The post hoc analyses related to Hypothesis 2 revealed that mastery goal orientation and the metacognitive skills intervention had a multiplicative effect on the level of overall metacognitive activity exhibited by the participants. Over and above the positive main effects of the metacognitive skills intervention and mastery goal orientation, the two combined lead to the highest levels of strategic metacognitive activity. As one would expect, these findings suggested that mastery oriented people were more likely to accept and embrace the metacognitive skills intervention than were those with lower levels of mastery orientation. This supported the 91 notion that mastery orientation could serve as a driver of behavior, in this case metacognitive or regulatory behavior. These data and post hoc analyses, however, could not influence findings related to Hypotheses 3 and 4. By examining mastery goal orientation as an alternative motivational construct to goal setting, there is no way that this change can impact the relationships between metacognitive activity and practice activities with the outcomes of declarative knowledge and webpage building performance. The limitations regarding the design and execution of this study still apply. There are, however, enough empirical studies supporting these relationships to tentatively conclude that they exist. Obviously, more research needs to be done that focuses on relationships between metacognitive or other strategic activities, task related effort, and multiple skill acquisition-related outcomes such as declarative knowledge gain and task performance. It may be the case that moderators exist that need to be specified in order to better understand these relationships. Learning Points and Related Research Directions The introduction to this manuscript argued that the metacognition and goal-setting literatures were distinct in that they did not reference the importance of one another in terms of their benefits for skill acquisition. The theoretical model derived from these assertions, then, focused on the general importance of a motivation x strategic effectiveness interaction in driving learning and performance. After re-examining the theory that drove this study in light of the findings, it became clear that the study and the theoretical model suffered from some fairly substantial problems. The major overarching problem, as discussed above, is that the 92 distinction between learning and performance was not explicitly considered. Related to this was the notion that goals and metacognitive skills should exert simultaneous influences on the skill acquisition process. This notion now seems unreasonable. While both goals and metacognition may play important roles in ultimately determining how well one is able to acquire knowledge and perform with respect to a complex task, their influences will likely be the most significant at different times in the skill acquisition and performance cycle. The problem does not seem to be that the goal setting and metacognition literatures do not reference or concurrently consider one another as suggested earlier. Rather, the problem may be that goal setting and metacognition are not concurrently considered in larger, more cyclical models of skill acquisition and that their differential impacts have not been integrated to devise these types of models. Metacognition has been traditionally researched in terms of its influence on declarative knowledge gain. The majority of studies examining metacognitive skills and the effects of metacognitive interventions examine outcomes such as grade point average or classroom performance as the criteria of interest. On the other hand, goal-setting research primarily deals with simple tasks and findings suggest that people perform better when they have challenging and specific goals that can be directly tied to goal-relevant behaviors. Self-regulation here deals more with focusing attention and energy on the task rather than monitoring one’s levels of knowledge gain about the task. Consider this in terms of Anderson’s (1982; 1983) multi-phase view of skill acquisition. Briefly, Anderson (1982; 1983) suggests that skill acquisition can be broken down into three different phases: Declarative knowledge gain, knowledge compilation, and procedural knowledge gain. Declarative knowledge gain deals with gaining an 93 understanding of the elements, requirements, objectives, and fundamentals of the task. Knowledge compilation deals with the integration of sequences of requisite cognitive and/or motor processes that are learned during the declarative knowledge phase. Finally, procedural knowledge gain deals with acquiring complete knowledge about how to perform cognitive and/or motor activities and the automation of these skills. Metacognitive skills should be important in determining how well one is able to gain declarative knowledge about the task in the initial phase of skill acquisition. Indeed, the literature on metacognition and metacognitive interventions strongly supports this claim (e.g. Pintrich & DeGroot, 1990; King 1991a; 1991b). Challenging and specific goals, on the other hand, should serve as drivers of motivation, effort, and performance during the procedural knowledge phase of skill acquisition and during later periods of skill exhibition or use. The goal setting literature suggests that when skills or tasks are fairly straightforward, goals lead to higher levels of performance via effort, persistence, and directed attention (e. g. Locke & Latham, 19903). What does this mean for broad motivation x strategic effectiveness interactions, such as the one originally discussed in this study? It may actually be the case that both motivation and strategy-related activities of some sort need to be operating simultaneously throughout the skill acquisition and performance cycle. For example, the alternative model in this study suggested that the metacognitive intervention interacted with mastery goal orientation to influence levels of metacognitive activity. Furthermore, mastery goal orientation influenced levels of practice activities related to the task. This suggests that the intrinsic motivation associated with mastery goals may operate simultaneously with metacognitive activity to influence declarative knowledge 94 acquisition early in the skill acquisition cycle. In this case, strategic and metacognitive activities center on making plans, monitoring learning, and utilizing proper strategies for acquiring knowledge. They do not deal explicitly with task performance. As basic skills related to a given task become more routinized, the motivation x strategic effectiveness interaction may present itself in a different form. This is where the motivational effects of challenging and specific performance goals may have their effect on performance. However, if the task is difficult or complex, proper task strategies will need to be utilized in order to perform well. In this case, task or performance strategy assistance may help people perform better when operating in conjunction with performance goals (e. g. Barley, Connolly, & Lee, 1989; Kanfer et al, 1994). Here, self- regulation focuses primarily around performance and goals may provide the necessary energy for people to engage in these types of regulatory activities. This discussion points to two general directions for future research involving skill acquisition and performance. First of all, future research needs to focus on clarifying the existence of possible motivation x strategic effectiveness interactions at different points in the skill acquisition and performance cycle. This will certainly provide a better understanding of how motivation and self-regulatory behaviors can concurrently influence skill acquisition. Also, research of this nature can provide a more systematic approach to studying skill acquisition and performance in that it can determine what types of constructs or interventions exert more proximal or distal influences on knowledge gain or performance. F urtherrnore, this research can help to more clearly explicate different levers for action that potential interventions should focus on during different stages of skill acquisition. 95 A second, related direction for future research deals with approaching skill acquisition in a cyclical manner and studying the potential forward-reaching effects of early influences on declarative knowledge gain (e. g. metacognitive interventions and/or mastery goals) on later procedural knowledge gain and performance. For example, it may be the case that people who are able to gain declarative knowledge more successfully in early stages of skill acquisition may be more adept at compiling and proceduralizing this knowledge later on. From a Social-Cognitive Theory perspective (e. g. Bandura & Locke, 2003), early success in declarative knowledge gain may lead to higher levels of self-efficacy. In turn, higher self-efficacy may lead to higher levels of persistence and better reactions to negative feedback when attempting to ultimately perform well on the given task. While self-efficacy is not usually a criterion of interest for metacognition researchers, there is some evidence that metacognitive interventions (and subsequent declarative knowledge gains) are related to higher levels of self-efficacy (e. g. Schmidt & Ford, 2003). The previous discussion focuses on the importance of focusing on skill acquisition and performance as a cycle and clearly explicating what types of motivational and strategic factors exert influences at different points in time. Going one step further, future research should also focus on studying skill acquisition and performance over multiple different types of tasks or skills. As stated earlier, the nature of many types of jobs and activities are changing such that people must learn how to be adaptive and continually gain new skills over time (Ilgen & Pulakos, 1999; Smith, Ford, & Kozlowski, 1997). Indeed, being able to continuously acquire new skills and procedures is thought to 96 be an especially important dimension of adaptive performance (Pulakos, Arad, Donovan, & Plamondon, 2000). Metacognition, especially in the form of evaluation and metacognitive knowledge development, may be an important determinant of how quickly and how successfully people are able to acquire new skills. Recall that metacognitive knowledge is separated into three types: declarative, procedural, and conditional. Conditional knowledge of what types of strategies are the best for learning different types of material or tasks may be especially important when one has to continually acquire new skills. This knowledge is gained by evaluation and reflection, which occur after one completes the processes related to gaining knowledge about the task or skill at hand. To the extent that one can successfully evaluate what learning strategies worked better for different types of tasks or environments, conditional knowledge may be able to reach forward and influence the success of future ”skill acquisition. Future research should focus on examining how repeated experience with acquiring new skills can influence future skill gain in this type of manner. 97 APPENDD( A Goal Manipulations “Do your best” goal “Because the task of leanring of how to build a webpage is somewhat difficult and complex, I would like to provide a goal for you to work toward. The final task you will be working on today is actually building a webpage using what you learn during the training period. There are 83 different objectives that will need to be done to correctly build the webpage, and you will be given 35 minutes to work on it. Your goal is to do your best on this final task. Please try to keep this goal in mind as you are proceeding through the web-page training program.” Challenging and sgecific goal “Because the task of learning of how to build a webpage is somewhat difficult and complex, I would like to provide a goal for you to work toward. The final task you will be working on today is actually building a webpage using what you learn during the training period. There are 83 different objectives that will need to be done to correctly build the webpage, and you will be given 35 minutes to work on it. Your goal today is to complete at least 75 of the 83 objectives. Although this goal is difficult, it is also attainable. Please try to keep this goal in mind as you are proceeding through the web- page training program.” 98 APPENDIX B Metacognition Intervention — Overview and Description “I would like to provide you with a set of questions you can ask yourself before and during the training program to help you with your learning. These questions are derived from a theory called Metacognition, which deals with how peOple make plans, monitor their performance or learning, and evaluate how successful they are. Briefly, the theory of metacognition suggests that when people make plans for how to approach different tasks or problems, they are able to perform or learn better. These plans involve developing different strategies and coming up with back-up plans if things aren’t going well. Also, people who continuously monitor their levels of learning or performance tend to do better because they are able to better detect problems or concepts they do not understand. Because they are monitoring their performance or learning, they can then go back and address these problems in order to do better. Finally, people tend to learn more and perform better in the long run when they spend time evaluating their leaming or performance once they are finished with a task. This allows them to select better strategies for future performance and maximize the amount they learn from their experiences. The processes involved in metacognition are based on self-questioning. The questions I have provided you with here are examples of the types of questions people can ask themselves in order to make better plans, monitor their learning or performance more successfully, and evaluate how well their strategies and plans worked out. Before, during, and after you work on the training program, I will ask you to ask yourself some of these questions, and write down answers to them. You can also use the 99 questions on your own as you’re going through the training program if you feel that they will help you. This is designed to help you gain more from the training program.” 100 APPENDIX C List of Self-Questions Given in the Metacognitive Intervention Adapted from King (1991b) and Schraw (1998). Planning What am I trying to do here? What do I know about the task so far? What kind of information and strategies do I need? What is my plan? Is there another way to do this task? Monitoring Do I have a clear understanding of what I’m doing? Am I using my plan or strategy? Do I need to change my plan or strategy? How well am I learning? Evaluating What worked? What didn’t work? What would I do differently next time? 101 APPENDIX D Strategic Behaviors Scale Adapted from Schmidt & Ford (2003) During this training program, I made up questions to help focus on my leaming. During this training program, I asked myself questions to make sure I understood the things I had been trying to learn. During this training program, I tried to change the way I learned in order to fit the demands of the situation or topic. During this training program, I tried to think through each topic and decide what I was supposed to learn from it, rather than just jumping in without thinking. During this training program, I tried to determine which things I didn't understand well and adjusted my learning strategies accordingly. If I got confused during this training program, I made sure I sorted it out as soon as I could before moving on. During this training program, I thought about how well my tactics for leaming were working. During this training program, I thought carefully about how well I had learned material I had previously studied. During this training program, I thought about what skills needed the most practice. During this training program, I tried to monitor closely the areas where I needed the most improvement. During this training program, I thought about what things I needed to do to learn. During this training program, I carefully selected what to focus on to improve on weaknesses I identified. During this training program, I noticed where I made mistakes and focused on improving those areas. When I practiced a new skill in this training program, I monitored how well I was learning its requirements. Before I began studying, I thought about the things I would need to do. I asked myself questions about the training material before I began. I carefully planned my course of action on the training program I developed a plan for completing the training program. I clearly planned my course of action on the training program Response Options 1: strongly disagree 2= disagree 3= neutral 4= agree 5= strongly agree 102 APPENDIX E Declarative Knowledge Quiz Which is not a good reason to place a link on your webpage? a) To point to extra information on another webpage b) To point to another webpage related to content on your webpage c) To connect to another one of your own webpages (I) You can't think of anything else to put on your webpage How does a hyperlink work? a) It uses the address of another webpage b) It uses the title of another webpage c) It uses meta tags from another webpage d) It uses magic How do you place a link on your webpage? a) Modify, make link b) Insert, Link... 0) Text, Add a Hyperlink... (1) Right mouse button, Select Hyperlink If you are making a link to another web site, the "relative to" box should contain what option? a) w: b) Path: c) Document: (1) Site Root: How do you know a link is in place on your webpage? a) You can't tell b) It is underlined c) It is a different color (1) It is underlined and a different color How do you remove a link from your webpage? a) Edit, Hyperlink... b) Insert, Hyperlink... c) Right mouse click, Remove Link (I) View, Select Hyperlink What are the two attributes that specify the amount of space around and within the cells of a table? a) Borders and cell height b) Cell padding and cell spacing c) Cell padding and row height 103 d) Borders and cell spacing What name should you save your first webpage document under? a) user b) index c) any name you want (1) Your pilot login name In what folder should you save your webpages? a) web b) w c) snapshots d) any folder How many webpages can you have on the MSU server system? a) only one, and it must be named "indexhtml" b) several webpages; the one named "index.html" is your home page c) only two; "indexhtml" and one other webpage (I) only one, but it can have any name you want How are graphics files uploaded to the Internet? a) by saving each one separately b) by saving them as a group of files c) automatically when the htrnl file is saved (1) only by using FTP How do you open a new browser window to locate your webpage? a) New, Browser window... b) Tools, New window... c) Edit, New, Window (1) File, New, Window What do you need to know to find your webpage on the Internet? a) The page title b) The document name c) The address (1) Your pilot password Which of the following is the correct prefix for an MSU student's webpage address? a) http://www.msu.edu/user/ b) http://www.msu.edu/ c) http://www.msu.edu/pilot/ d) ftp://www.msu.edu/user/ Which statement about changing the background colors of a webpage is true? a) You can change the colors, but there are only a couple from which to choose 104 b) You can change the colors, and it is easy to do c) You can change the colors, but it is difficult to do (I) You can't change the colors Which menu is used in order to change the background colors on a webpage? a) Modify b) Insert c) Format (1) View Which sets the colors for all of the non-linking text that you write? a) Body text b) Normal text 0) Hyperlinks d) Followed hyperlinks How do you set the colors for text on a webpage? a) View, Font..., Effects b) Modify, Page Properties, Text c) Modify, Page Properties, Active Links (1) Format, Text colors... Why are tables useful on a webpage? a) Because all text on webpages must be in tables b) Because you can't use the tab key to create columns c) No particular reason; they're just fun to make (1) Tables are not useful; you can't put tables on a webpage How do you place a table on a webpage? a) Table, Insert table... b) Insert, Table... c) Insert, Object... (I) You can't Which of the following do you use the text menu for? a) Aligning text b) Inserting Graphics c) Choosing the color for links (1) Adding text to tables Where does the title for your webpage usually appear? a) At the top of the webpage b) At the top of the Internet browser c) In the Address field (I) You can't see it 105 Why should you have a title for your webpage? a) There is no reason to have a title b) A title is required for webpages c) A title makes your webpage look nicer (1) Search engines can find your webpage more easily What is it important for your webpage title to contain? a) Your name b) The Internet address for your webpage c) The most important aspect of the page's content (I) It doesn't really matter what is in the title What is the purpose of keywords? a) They allow you to link to specific sections of your webpage b) To keep the search engines away from your page c) To organize the information on your page (I) To help visitors find your webpage What can't you do with a horizontal line? a) Move it b) Hide it c) Delete it (1) Align it Which menu is used in order to place an image on a webpage? a) Edit b) Insert c) F orrnat d) Tools What can you do with an graphic once you have inserted it on a webpage? a) Resize it b) Change its color c) Rotate it (1) Nothing Which of the following is not an alignment option for horizontal lines and graphics? a) Left b) Center c) Right (1) None Which of the following is NOT true about table borders? a) you can change the color b) you can change the size c) tables always have to have borders 106 (1) tables without borders have dotted lines in design view 107 APPENDIX F Webpage Building Objectives There are a number of different tasks you must complete to build your final webpage. These tasks are listed below, in the same order with which the information was structured in the training program. Keep in mind that this is not necessarily the best way to proceed through the steps and complete the different tasks. There are 83 separate parts or tasks, each represented by a bullet point, and your performance will be judged based on how many of the tasks you can correctly complete in 35 minutes. ***Before you begin anything else, create a new page titled index.htm (given it an alternative name if you already have a page named index in your AF S space). Save this page in the folder you created on your desktop. Insert the following title: XXX MSU basketball page (XXX should be filled in with your code number) Insert the keyword: MSU Insert the keyword: Athletics Insert the keyword: Basketball Insert the Keyword: lzzo Insert an 11 row X 4 column table in the page that has the following properties: 0 Background color: Black 0 Border color: Gold or Yellow 0 Border thickness: 10 0 Text color: White 0 Table width: 50% of the page 0 Alignment: Center aligned 0 Cell padding: 10 0 Cell Spacing: 5 o This table will contain the following list of some of the members of the MSU men’s basketball team, along with their jersey number, position, and year in college Player Number Position Year Ager, Maurice 13 G Jr Anderson, Alan 15 G/F Sr Bograkos, Tim 30 G Sr Brown, Shannon 3 G So Davis, Paul 40 C Jr Harvey, Andy 43 F Sr 108 Hill, Chris 5 Naymick, Drew 34 Neitzel, Drew 12 Torbert, Kelvin 23 G Sr C So G Fr G Sr 0 Insert a text link to the webpage: http://www.msu.edu Insert a text link to the webpage: http://msuspartans.collegesports.com 0 Insert named anchors for 4 sections of the page: 0 Coach 0 Players 0 Mascot 0 Links - These anchor links should be located near the top of the page. Insert the green background image. The picture to use for this is found at http://www.msu.edu/user/jundtdus/background.htm. Save the picture to your desktOp folder and insert it from there. 0 Insert a horizontal line with the following characteristics between the sections anchors and Coach to separate them (see example sheet) 0 Alignment: Center aligned 0 Height: 5 0 Width: 80% of the page 0 Shading: Yes 0 Insert a horizontal line with the following characteristics between the sections Coach and Players to separate them (see example sheet) 0 Alignment: Center aligned 0 Height: 5 0 Width: 80% of the page D Shading: Yes 0 Insert a horizontal line with the following characteristics between the sections Players and Mascot to separate them (see example sheet) 0 Alignment: Center aligned 0 Height: 5 0 Width: 80% of the page 0 Shading: Yes 0 Insert a horizontal line with the following characteristics between the sections Mascot and Links to separate them (see example sheet) 0 Alignment: Center aligned 109 0 Height: 5 _ 0 Width: 80% of the page 0 Shading: Yes 0 Insert the picture of Sparty into the Mascot section of the page. This picture can be found at http://www.msu.edu/user/jundtdus/background.htm Save the picture to your desktop folder and insert it fi'om there. 0 Add alternative image (ALT) text to the Sparty image that says — “Sparty: 2 time mascot of the year!”. 0 Insert Large text at the top of the page that says: MSU basketball 0 Color: white 0 Alignment: Center 0 Font: Arial 0 Size: 7 Insert a text heading the second section of the page that says: Coach 0 Color: White Alignment: Left Size: 5 Style: Bold, Underlined Font: Arial Insert a text heading the third section of the page that says: Players Color: White 0 Alignment: Left 0 Size: 5 0 Style: Bold, Underlined 0 Font: Arial Insert a text heading the fourth section of the page that says: Mascot Color: White Alignment: Left Size: 5 Style: Bold, Underlined Font: Arial Insert a text heading the fifth section of the page that says: Links Color: White Alignment: Left Size: 5 Style: Bold, Underlined Font: Arial 110 0 Insert the following text for the first section (Coach): “The head coach of the MSU Men's basketball team is Iron Mountain, Michigan native Tom 1220. Now in his tenth year as head coach at MSU, 1220 has compiled an impressive list of accomplishments, including one NCAA National Championship, four regular- season Big Ten Championships, two Big Ten Tournament titles, three Final Four appearances, and three National Coach of the Year awards. 1220 has a career record of 207-90 and in 2003-04, he led the Spartans to a seventh straight NCAA Tournament appearance, marking the longest active streak among Big Ten teams.” 0 Color: White 0 Font: Arial 0 Alignment: Left 0 Size: Default (3) Section 2 is “Players”. This section contains only the table that is discussed earlier. 0 Section 3 is Mascot. Underneath the image of Sparty, enter the following text. “Sparty the Spartan is Michigan State University’s fearless and loveable mascot, a figure known throughout the state of Michigan and recognized across the nation as well. Sparty is the heart of Michigan State, forever supporting its teams, bringing smiles to young and old and continually uplifiing all who meet him. Sparty has received many honors, including being voted best mascot in the nation two years in a row and being named "Buffest" mascot. During Basketball season, Sparty can almost always be found charging up MSU fans in the "Izzone" and showing his school spirit.” Color: White Font: Arial Alignment: Left Size: Default (3) Section 4 is Links and should contain the text links to the other web-pages described earlier. 111 APPENDIX G Computer Experience Scale Adapted from Potosky & Bobko (1998) I frequently read computer magazines or other sources of information that describe new computer technology. I know what a LAN (Local Area Network) is. I know what an operating system is. I know how to write computer programs. I know how to install software on a personal computer. I know what email is. I know what a database is. I am computer literate. I regularly use a PC for word processing. 1 am good at using computers. Response Options 1= strongly disagree 2= disagree 3= neutral 4: agree 5= strongly agree 112 APPENDIX H Consent Form Investigating skill acquisition in a learner-controlled setting. The purpose of this study is to investigate different mechanisms that impact skill acquisition (e. g. learning and performance) in a learner-controlled setting. You will be asked to proceed through a web-based training program that contains information on how to build a web-page using the Macromedia Dreamweaver© software package. As this is a learner-controlled training program, you will have control over what content you view, the order you view it in, how long you spend viewing it, and how much you practice. Also, you will be asked to complete a number of different questionnaires throughout the study that assess different personal characteristics, demographics, and explanations of your behavior during the training program. Finally, you will be asked to take a brief “quiz” that assesses the knowledge you gained from the training program and you will be asked to build a web-page using this knowledge. When you are finished with the study, the experimenter will provide a thorough debriefing as to the exact hypotheses being investigated. This study will be conducted in one session that is expected to last approximately 150 minutes (2 ‘/2 hours). For participating in this study, you will receive class credit for the amount of time you spend as suggested by the Psychology Subject Pool guidelines: 1 credit per 30 minutes of participation. Participation is this study is totally voluntary. You may choose not to participate at all or you may refuse to participate in certain procedures or answer certain questions should you object to them. Furthermore, you may discontinue the experiment at any time without penalty or loss of benefits to which you are otherwise entitled (i.e. ability to participate in other studies through the subject pool). Should you decide not to participate in this study, research credits may also be obtained by participating in other studies listed on the Psychology Department Subject Pool website. Responses on all of these items on the questionnaires and your performance data will be completely confidential. The information gathered in this study will be combined with the data of all of the other participants in the study for any analyses so that your responses cannot be identified. You will be asked to provide your PID, however, so that you are able to receive extra credit for your participation, but once extra credit has been recorded your PID will be completely removed from our files an no longer associated with your responses. The data will only be accessible by the primary (Dr. Daniel Ilgen) and secondary (Dustin J undt) investigators in the study. Your privacy will be protected to the maximum extent allowable by law. If you have any questions about this study, please contact the investigator (Dustin Jundt, 348 Psychology building, (517) 432-7069, jundtdus@msu.edu). If you have questions or concerns regarding your rights as a study participant, or are dissatisfied at any time with any aspect of this study, you may contact — anonymously, if you wish — 113 Peter Vasilenko, Ph.D., Chair of the University Committee on Research Involving Human Subjects (UCRIHS) by phone: (517) 355-2180, fax: (517) 432-4503, e-mail: ucrihs@msu.edu, or regular mail: 202 Olds Hall, East Lansing, MI 48824. Your signature below indicates your voluntary agreement to participate in this study. Name: Date: PID: 114 APPENDIX I Debriefing Form The purpose of this experiment was to examine the impact of goal-setting and metacognition interventions on leaming and performance in a setting where individuals control the pace, sequence, and amount of content they view and the number of practice activities they complete. Participants are given either challenging, specific goals (i.e. your goal is to score 90% on the web-page building task) or goals that simply ask them to “do your best”. Also, half of the participants are given a metacognition intervention that provides them with strategies for making plans and monitoring their learning. The other half is not given this intervention. The primary hypotheses are as follows: (1) Individuals given challenging, specific goals will complete more practice activities than those given “do your best” goals. - This hypothesis is based on the notion that challenging, specific goals are more motivating than “do your best” goals. If this is true, individuals who are more motivated to perform well and learn should practice more. (2) Individuals given challenging, specific goals will engage in more strategically oriented activities (e.g. making plans, monitoring their learning) than those given “do your best” goals, but only when they also receive metacognition training. Metacognition training describes the benefits of strategic learning activities and provides participants with a set of questions they can ask themselves to help guide these activities. - As suggested before, challenging, specific goals are thought to lead to higher levels of motivation. This motivation, however, in not likely to be translated into higher levels of strategic activities unless an individual has guidance on how to perform them. The metacognition training is thought to provide this guidance. (3) More practice and strategic activities exhibited during training will lead to higher levels of performance on the web-page building task. - Getting better at doing certain tasks (practicing) and learning more strategically should both be related to better overall performance. (4) This relationship will be mediated by the amount declarative knowledge individuals’ gain via the training program. — The relationships between getting better at doing certain tasks (practicing) and learning more strategically and overall performance should occur because of the amount of knowledge people gain about how to do the task. If you would like to learn more about how goal-setting influences learning and performance, you can read the following articles: 115 Locke, E. A., Shaw, K. N., Saari, L. M., & Latham, G. P. (1981). Goal setting and task performance: 1969-1980. Psychological Bulletin, 90, 125-152. Wood, R. E., & Locke, E. A. (1990). Goal setting and strategy effects on complex tasks. Research in Organizational Behavior, 12, 73-109 If you would like to learn more about metacognition and how it may influence learning and performance, you can read the following articles: Ertmer, P. A. and T. J. Newby (1996). The expert learner: Strategic, self-regulated, and reflective. Instructional Science, 24, 1-24. King, A. (1991b). Effects of training in strategic questioning on children’s problem- solving performance. Journal of Educational Psychology, 83, 307-317. Nelson, T. O. (1996). Consciousness and metacognition. American Psychologist, 51, 102-116. 116' 563? 58:98 1835 m: 868%: b5: 2.: 3m 3:328 2% yfimpgfio 932329 3.5an .3 833%: 5:538 :33. . ._ concutas 02 u _ 48:30:53 :2 u 9 3:0 Mesa 8 €335 3.2% 8302 :2: 023:: a: means u _ as m»: n: as :58 age: so 3. v a u 2.83%: a: 23 E SBSES . ::.: a: ::.: ::.: a: 9...: 9.: ::.:- g: 3.: S: ::.: a: a: E 52368.: :2: 8: 3:. a: 3.: a: 2.: 2:. 2.: ::.:- 2.: S: :2 :3 E assemwaéamoaags 3.: 2:. 2.: :3 a: 2.: ::.: 2.: 2:. 2.: 3.: :3 :4: E ssseosaeoo: a: a: a. 8: 2.: 8:. :4: a: 3:. 2:- a: 2.: E asunegsaoasg2 a: 2.: a: 2.: 2:- :8 ::.: 3.: 8:. ::.: a: E 25588820.: as a: a. 8:- 2.: 2.: E. 3:. a: as E .8522: . a: 8:. 8: 2.: S: ::.: :2: 8:32 vastness: R90 8: 2:. v; 8.: Ba Sm 32 m: umvmaocxusfiflsoa . a: 2:. 2.: ::.: s: a: E emsssaeaei a: ::.:- :8 B: a: :2 E €35355:53:82s - 2:- ::.: a: c: :2 massages: . 2.: :n: s: E 553356883: . a: s: E 835820.: 2 2 2 S a w n o n v m N _ 9m. 3 z 823980 .395. 3: 5 agate; Eek SDEESWM backwash NBSE use .fiouumcbccmg unsung Begum. .983 H :38. a N52mmm< 117 Table 2 Eflects of Goal Condition and Mastery Goal Orientation On Practice Activities (n = 109) Practice Activities Step b R2 Total A R2 1. Goal Condition - 0.35 .036 .036 Mastery Goal Orientation 0.79 2 . Goal Condition X Mastery 0.09 .036 .000 Goal Condition Dummy Code: 0 = DYB Goal, 1 = Challenging and Specific Goal 118 Table 3 Eflects of Goal Condition and Goal Commitment On Practice Activities (n =1 09) Step Practice Activities R2 Total A R2 1. Goal Condition Goal Commitment 2. Goal Condition X Goal Commitment 0.34 .010 .010 0.40 0.15 .010 .000 Goal Condition Dummy Code: 0 = DYB Goal, 1 = Challenging and Specific Goal 119 Table 4 Effects of Goal Condition and Attention to Assigned Goals on Practice Activities (n=109) Practice Activities Step b R2 Total A RY 1. Goal Condition 0.52 .051 .051 Attention to Assigned Goals 085* 2. Goal Condition X -0.06 .051 .000 Attention to Assigned Goals * = p < .05 Goal Condition Dummy Code: 0 = DYB Goal, 1 = Challenging and Specific Goal 120 Table 5 Effects of Goal Condition and the Metacognitive Intervention on Metacognitive Activity (n =1 14) Metacognitive Activity Step b R2 Total A R2 1. Goal Condition 0.01 .082 .082* Metacognitive Intervention 0.30* 2. Goal Condition X 0.04 .082 .000 Metacognitive Intervention *=p<05 Goal Condition Dummy Code: 0 = DYB Goal, 1 = Challenging and Specific Goal MC Intervention Dummy Code: 0 = No MC Intervention, 1 = MC Intervention 121 Table 6 Effects of Goal Condition and the Metacognitive Intervention on Repeated Submodule Page Visits (n=114) Repeated Page Visits Step b R2 Total A R2 1 . Goal Condition 1 .1 8 .024 .024 Metacognitive Intervention 1.92 2. Goal Condition X -0.71 .025 .001 Metacognitive Intervention Goal Condition Dummy Code: 0 = DYB Goal, 1 = Challenging and Specific Goal MC Intervention Dummy Code: 0 = No MC Intervention, 1 = MC Intervention 122 Table 7 Effects of the Metacognitive Intervention and Mastery Goal Orientation on Metacognitive Activity (n =1 1 4) Step 1. Metacognitive Intervention Mastery Goal Orientation 2. Metacognitive Intervention X Mastery Metacognitive Activity b R2 Total A R2 0.34* .171 .171* 0.25* 0.26 .195 .024 *=p Activity » Performance 134 References Aguinis, H. (1995). Statistical power problems with moderated multiple regression in management research. Journal of Management, 21, 1141 -1 158. Aguinis, H., Boik, R. J ., & Pierce, C. A. (2001). A generalized solution for approximating the power to detect effects of categorical moderator variables using multiple regression. Organizational Research Methods, 4, 291-323. Aguinis, H., & Stone-Romero, E. F. (1997). Methodological artifacts in moderated multiple regression and their effects on statistical power. Journal of Applied Psychology, 82, 192-206. Ames, C. (1992). Achievement goals, motivational climate, and motivational processes. In G. Roberts (Ed.), Motivation in sports and exercise (pp. 161-176). Champaign, IL: Human Kinetics Books. Ames, C., & Archer, J. (1988). Achievement goals in the classroom: Students’ learning strategies and motivation processes. Journal of Educational Psychology, 80, 260- 276. Anderson, J. R. (1982). Acquisition of cognitive skill. Psychological Review, 89, 369- 406. Anderson, J. R. (1983). The architecture of cognition. Cambridge, MA: Harvard University Press. Ashkanasy, N. M., Wilderom, C. P. M. & Peterson M. F. (2000). Handbook of organizational culture and climate. Thousand Oaks, CA: Sage Publications. Austin, J. R. (2003). Transactive memory in organizational groups: The effects of content, consensus, Specialization, and accuracy on group performance. Journal of Applied Psychology, 88, 866-78 Baldwin, T. T., & Ford, J. K. (1988). Transfer of training: A review and directions for future research. Personnel Psycholog, 41 , 63-105. Bandura, A., & Locke, E. A. (2003). Negative self-efficacy and goal effects revisited. Journal of Applied Psychology, 88, 87-99 Bandura, A., & Schunk, D. H. (1981). Cultivating competence, self-efficacy, and intrinsic interest through proximal self-motivation. Journal of Personality & Social Psychology, 41, 586-598. 135 Baron, R., & Kenney, D. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic and statistical considerations. Journal of Personality and Social Psychology, 51 , 1173-1 182. Berardi-Coletta, B., Buyer, L. S., Dominowski, R. L., & Rellinger, E. R. (1995). Metacognition and problem solving: A process-oriented approach. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21, 205-223. Bereiter, C ., & Scardamalia, M. (1982). From conversation to composition: The role of instruction in a developmental process. In R. Glaser (Ed), Advances in Instructional Psychology: Vol. 2, (pp. 1-64). Hillsdale, NJ: Erlbaum. Bouffard-Bouchard, T, Parent, S., & Larivee, S. (1993). Self-regulation on a concept- formation task among average and gified students. Journal of Experimental Child Psychology, 56, 115-134. Brown, A. L. (1987 ). Metacognition, executive control, self-regulation, and other more mysterious mechanisms. In F. E. Weinert & R. H. Kluwe (Eds. ), Metacognition, motivation, and understanding (pp. 65-116). Hillsdale, New Jersey: Erlbaum. Brown, A. L., & Palincsar, A. S. (1982). Inducing strategic learning from texts by means of informed, self-control training. T 0pics in Learning and Learning disabilities, 2, 1-17. Brown, A. L., & Palincsar, A. S. (1985). Reciprocal teaching of comprehension strategies: A natural history of one program for enhancing learning. In Garner, R., & Alexander, P. A. (1989). Metacognition: Answered and unanswered questions. Educational Psychologist, 24, 143-158. Bunderson, J. S., & Sutcliffe, K. M. (2003). Management team learning orientation and business unit performance. Journal of Applied Psychology, 88, 552-560. Campbell, D. J. (1984). The effects of goal-contingent payment on the performance of a complex task. Personnel Psychology, 3 7, 23-40. Campbell, D. J. (1991). Goal levels, complex tasks, and strategy development: A review and analysis. Human Performance, 4, 1-31. Cervone, D., J iwani, N., & Wood, R. (1991). Goal setting and the differential influence of self-regulatory processes on complex decision-making performance. Journal of Personality and Social Psychology, 61, 257-266. Cervone, D. & Wood, R. (1995). Goals, feedback, and the differential influence of self- regulatory processes on cognitively complex performance. Cognitive Therapy and Research, 19, 519-545. 136 Clause, C. S., Delbridge, K., Schmitt, N., Chan, D., & Jennings, D. (2001). Test preparation activities and employment tests performance. Human Performance, 14, 149-167. Cohen, J. (1969). Statistical power analysis for the behavioral sciences: 1st edition. New York: Academic Press. Delclos, V. R., & Harrington, C. (1991). Effects of strategy monitoring and proactive instruction on children’s problem-solving performance. Journal of Educational Psychology, 83, 35-42. DeShon, R. P., & Alexander, R. A. (1996). Goal setting effects on implicit and explicit learning of complex tasks. Organizational Behavior and Human Decision Processes, 65, 18-36. DeShon, R. P., Kozlowski, S. W. J ., Schmidt, A. M., Milner, K. R., & Wiechmann, D. (2004). A multiple goal, multilevel model of feedback effects on the regulation of individual and team performance in training. Journal of Applied Psychology, 89, 1035-1056. Driver, M. (2002). The learning organization: Foucauldian gloom or Utopian sunshine? Human Relations, 55, 33-53. Duda, H. L., & Nicholls, J. G. (1992). Dimensions of achievement motivation in schoolwork and Sport. Journal of Educational Psychology, 84, 290-299. Dweck, C. S. (1986). Motivational processes affecting learning. American Psychologist, 41, 1040—1048. Dweck, C. S., & Leggett, E. L. (1988). A Social-Cognitive Approach to Motivation and Personality. Psychological Review, 95, 256-273 Earley, P. C. (1985). Influence of information, choice and task complexity upon goal acceptance, performance and personal goals. Journal of Applied Psychology, 70, 481-491. Earley, P. C., Connolly, T., & Ekegren, G. (1989). Goals, strategy development, and task performance: Some limits on the efficacy of goal setting. Journal of Applied Psychology, 74, 24-33. Barley, P. C., Connolly, T., & Lee, C. (1989). Task strategy interventions in goal setting: The importance of search and strategy development. Journal of Management, 15, 589-602. 137 Earley, P. C., Wojnaroski, P., & Prest, W. (1987). Task planning and energy expended: Exploration of how goals influence performance. Journal of Applied Psychology, 72, 107-114. Elliot, A. J ., & Church, M. A. (1997). A hierarchical model of approach and avoidance achievement motivation. Journal of Personality and Social Psychology, 72, 218- 232. Elliott, E. S., & Dweck, C. S. (1988). Goals: An approach to motivation and achievement. Journal of Personality and Social Psychology, 54, 5-12. Elliot, A. J ., & Harackiewicz, J. M (1994). Goal setting, achievement orientation, and intrinsic motivation: A mediational analysis. Journal of Personality and Social Psychologi, 66, 968-980. Erez, M., & Zidon, I. (1984). Effect of goal acceptance on the relationship of goal difficulty to performance. Journal of Applied Psychology, 69, 69-78. Ertmer, P. A., & Newby, T. J. (1996). The expert learner: Strategic, self-regulated, and reflective. Instructional Science, 24, 1-24. Fenigstein, A., Scheier, M. F., & Buss, A. H. (1975). Public and private self- consciousness: Assessment and theory. Journal of Counseling and Clinical Psychology, 43, 522—527. Flavell, J. H. (1976). Metacognitive aspects of problem solving. In L. B. Resnick (Ed.), The nature of intelligence (pp. 231-236). Hillsdale, NJ: Lawrence Erlbaum Associates. Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive- developmental inquiry. American Psychologist, 34, 906-911. Flavell, J. H., Friedrichs, A. G., & Hoyt, J. D. (1970). Developmental changes in memorization processes. Cognitive Psychology, 1, 324-340. Ford, J. K., Smith, E. M., Weissbein, D. A., Gully, S. M., & Salas, E. (1998). Relationships of goal orientation, metacognitive activity, and practice strategies with learning outcomes and transfer. Journal of Applied Psychology, 83, 218-233. Garner, R., & Alexander, P. A. (1989). Metacognition: Answered and unanswered questions. Educational Psychologist, 24, 143-158. Gist, M. E., & Stevens, C. K. (1998). Effects of practice conditions and supplemental training method on cognitive learning and interpersonal Skill generalization. Organizational Behavior and Human Decision Processes, 75, 142-169. 138 Guzzo, R. A., & Dickson, M. W. (1996). Teams in organizations: Recent research on performance and effectiveness. Annual Review of Psychology, 47, 307-338. Harackiewicz, J. M., & Elliot, A. J. (1993). Achievement goals and intrinsic motivation. Journal of Personality and Social Psychology, 65 , 904-915. Hinsz, V. B. (2004). Metacognition and mental models in groups: An illustration with metamemory of group recognition memory. In E. Salas & S. M. Fiore (Eds), Team cognition: Understanding the factors that drive processes and performance (pp. 33-58). Washington, DC: American Psychological Association. Hinsz, V. B., Tindale, R. S., & Vollrath, D. A. (1997). The emerging conceptualization of groups as information processors. Psychological Bulletin, 121, 43-64. Hollenbeck, J. R., & Klein, H. J. (1987). Goal commitment and the goal setting process: Problems, prospects and proposals for future research. Journal of Applied Psychology, 71, 212-220. Hollenbeck, J. R., Klein, H. J ., O'Leary, A. M. & Wright, P. M. (19893. Investigation of the construct validity of a self-report measure of goal commitment. Journal of Applied Psychology, 74, 951-956. Horvath, M., Scheu, C. R., & DeShon, R. P. (2001). The effects of domain specificity on the construct and predictive validity of goal orientation measures. Paper presented at the annual conference of the Society for Industrial and Organizational Psychology, San Diego, CA. Huber, V. L. (1985). Effects of task difficulty, goal setting, and strategy on performance of a heuristic task. Journal of Applied Psychology, 70, 492-504. Ilgen, D. R. (1999). Teams embedded in organizations: Some implications. American Psychologist, 54, 129-139. Ilgen, D. R., Hollenbeck, J. R., Johnson, M., & Jundt, D (2005). Teams in organizations: From I-P-O models to IMOI models. Annual Review of Psychology, 5 6, 517-543. Ilgen, D. R., & Pulakos, E. D. (Eds.). (1999). The changing nature of performance: Implications for staffing, motivation and development. San Francisco: J ossey- Bass. James, C. (2003). Designing learning organizations. Organizational Dynamics, 32, 46-61. Kanfer, R., & Ackerman, P. L. (1989). Motivation and cognitive abilities: An integrative/aptitude-treatment interaction approach to skill acquisition. Journal of Applied Psychology, 74, 657-690. 139 Kanfer, R., & Ackerman, P. L., Murtha, T. C., Dugdale, B., & Nelson, L. (1994). Goal setting, conditions of practice, and task performance: A resource allocation perspective. Journal of Applied Psychology, 79, 826-835. King, A. (1989). Effects of self-questioning training on college students’ comprehension of lectures. Contemporary Educational Psychology, 1 4, 366-381. King, A. (1990). Improving lecture comprehension: Effects of a metacognitive strategy. Applied Educational Psychology, 5, 331-346. King, A. (1991a). Facilitating elaborative learning through guided student-generated questioning. Educational Psychologist, 2, 111-126. King, A. (1991b). Effects of training in strategic questioning on children’s problem- solving performance. Journal of Educational Psychology, 83, 307-317. King, A. (1992). Comparison of self-questioning, summarizing, and notetaking-review as strategies for learning from lectures. American Educational Research Journal, 29, 303-325. King, A., & Rosenshine, B. (1993). Effects of guided cooperative-questioning on children’s knowledge construction. Journal of Experimental Education, 6, 127- 148. Klein, H. J ., Wesson, M. J ., Hollenbeck, J. R., &, Alge, B. J. (1999). Goal commitment and the goal setting process: Conceptual clarification and empirical synthesis. Journal of Applied Psychology, 84, 885-896 Klein, H. J ., & Wright, P. M. (1994). Antecedents of goal commitment: An empirical examination of personal and situational factors. Journal of Applied Social Psychology, 24, 95-114. Kozlowski, S. W. J ., Gully, S. M., Brown, K. G., Salas, E., Smith, E. M., & Nason, E. R. (2001). Effects of training goals and goal orientation traits on multidimensional training outcomes and performance adaptability. Organizational Behavior and Human Decision Processes, 85, 1-31. Kozlowski, S. W. J ., Gully, S. M., Nason, E. R., & Smith, E. M. (1999). Developing adaptive teams: A theory of compilation and performance across levels and time. In D. R. Ilgen & E. D. Pulakos (Eds), The changing nature of performance (pp. 240-292). San Fransisco: Jossey-Bass Kozlowski, S. W. J ., & Klein, K. J. (2000). A multilevel approach to theory and research in organizations: Contextual, temporal, and emergent processes. In K. J. Klein & S. W. J. Kozlowski (Eds), Multilevel theory, research, and methods in 140 organizations: Foundations, extensions, and new directions (pp. 3-90). San Francisco: Jossey—Bass. Kraiger, K., Ford J. K., & Salas, E. (1993). Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. Journal of Applied Psychology, 78, 31 1-328. Landine, J ., & Stewart, J. (1998). Relationship between metacognition, motivation, locus of control, self-efficacy, and academic achievement. Canadian journal of counseling, 32, 200-21 1. Latham, G. P., & Pinder, C. C. (2005). Work motivation theory and research at the dawn of the twenty-first century. Annual Review of Psychology, 56, 631-59. Latham, G. P., & Seijts, G. H. (1999). The effects of proximal and distal goals on performance on a moderately complex task. Journal of Organizational Behavior, 20, 421-429. Locke, E. A. (1968). Toward a theory of task motivation and incentives. Organizational Behavior and Human Performance, 3, 157-189. Locke, E. A., & Latham, G. P. (19903). A theory of goal setting and task performance. Englewood Cliffs, NJ: Prentice Hall. Locke, E. A., & Latham, G. P. (1990b). Work motivation and satisfaction: Light at the end of the tunnel. Psychological Science, 1, 240-246. Locke, E. A., & Latham, G. P. (1990c). Work motivation: The high performance cycle. In U. Kleinbeck, H. Quast, H. Thierry, & H. Hacker (Eds), Work Motivation (pp. 3-25). Hillsdale, NJ: Erlbaum. Locke, E. A., Shaw, K. N., Saari, L. M., & Latham, G. P. (1981). Goal setting and task performance: 1969-1980. Psychological Bulletin, 90, 125-152. McGrath, J. E., Arrow, H., & Berdahl, J. L. (2000). The study of groups: Past, present, and future. Personality and Social Psychology Review, 4, 95-105. Mento, A. J ., Steel, R. P. & Karren, R. J. (1987). A meta-analytic study of the effects of goal setting on task performance: 1966-1984. Organizational Behavior and Human Decision Processes, 39, 52-83. Milheim, W. D, & Martin, B. L. (1991). Theoretical bases for the use of learner control: Three different perspectives. Journal of Computer-Based instruction, 18, 99-105. 141 Minnaert, A., & J anssen, P. J. (1999). The additive effect of regulatory activities on top of intelligence in relation to academic performance in higher education. Learning and Instruction, 9, 77-91. Mohammed, S., & Dumville, B. (2001). Team Mental Models: Expanding Theory and Measurement Through Cross-Disciplinary Boundaries. Journal of Organizational Behavior, 22, 89-106. Mone, M. A., & Shalley, C. E. (1995). Effects of task complexity and goal specificity on change in strategy and performance over time. Human Performance, 8, 243-262. Morgan, B. B., & Bowers, C. A. (1995). Teamwork stress: Implications for team decision making. In R A. Guzzo, & E. Salas (Eds), Team eflectiveness and decision making in organizations (pp. 262-290). San Francisco: Jossey-Bass. Nelson, T. O. (1996). Consciousness and metacognition. American Psychologist, 51,- 102-116. Nelson, T. O., & Dunlosky, J. (1991). When people’s judgments of learning (JOLS) are extremely accurate at predicting subsequent recall: The “delayed-JOL effect”. Psychological Science, 2, 267-270. Nelson, T. O, & Narens, L. (1990). Metamemory: A theoretical framework and new findings. The Psychology of Learning and Motivation, 26, 125-141. Orasanu, J. M. (1993). Decision-making in the cockpit. In E. L. Wiener, B. G. Kanki, & R. l. Helmreich (Eds), Cockpit resource management (pp. 137-172). San Diego, CA: Academic Press. Osman, M. E., & Hannafin, M. J. (1992). Metacognition research and theory: Analysis and implications for instructional design. Educational technology research and development, 40, 83-99. Palincsar, A. S., & Brown, A. L. (1984). Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities. Cognition and Instruction, 2, 117-175. Pintrich, P. R. (1989). The dynamic interplay of student motivation and cognition in the classroom. In M. Maehr & P. Pintrich (Eds), Advances in motivation and achievemen: Motivation enhancing environments (vol. 6, pp 117-160). Greenwich, CT: J AI Press. Pintrich, P. R., & Garcia, T. (1991). Student goal orientation and self-regulation in the college classroom. In M. Maehr & P. Pintrich (Eds), Advances in motivation and achievement (vol. 7, pp 371-402). Greenwich, CT: JAI Press. 142 Pintrich, P. R., & DeGroot, E. V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82, 33-40. Pintnch, P. R, McKeachie, W. J., Smith, D. F, Doljanac, R, Yi-Guang, L. Naveh- Benjamin, M., Crooks, T., & Karabenick, S. (1988). Motivated strategies for learning questionnaire (MSLQ). National center for Research to Improve Postsecondary Teaching and Learning (N CRIPTAL), The University of Michigan. Pokay, P., & Blumenfeld, P. C. (1990). Predicting achievement early and late in the semester: The role of motivation and use of learning strategies. Journal of Educational Psychology, 82, 41-50. Potosky, D., & Bobko, P. (1998). The computer understanding and experience scale: A self-report measure of computer experience. Computers in Human Behavior, 14, 337-348. Pulakos, E. D., Arad, S., Donovan, M. A., & Plamondon, K. E. (2000). Adaptability in the Workplace: Development of a taxonomy of adaptive performance. Journal of Applied Psychology, 85, 612-624. Reeve, R. A., & Brown, A. L. (1985). Metacognition reconsidered: Implications for intervention research. Journal of Abnormal Child Psychology, 13, 343-356. Reeves, T. C. (1993). Pseudoscience in computer-based instruction: The case of learner control research. Journal of Computer-Based Instruction, 20, 39-46. Ridley, D. S., Schutz, P. A., Glanz, R. S., & Weinstein, C. E. (1989). Self-regulated learning: The interactive influence of metacognitive awareness and goal-setting. Journal of Experimental Education, 60, 293-306. Rogers, W. M. (2002). Theoretical and mathematical constraints of interactive regression models. Organizational Research Methods, 5, 231-254. Rosenshine, B., Meister, C., & Chapman, S. (1996). Teaching students to generate questions: A review of the intervention studies. Review of Educational Research, 66,181-221. Scardamalia, M. & Bereiter, C. (1982). Assimilative processes in composition planning. Educational Psychologist, 1 7, 165-171. Scardamalia, M., Bereiter, C., & Steinbach, R. (1984). Teachability of reflective processes in written composition. Cognitive Science. 8, 173-190. 143 Schmidt, A., & Ford, J. K. (2003). Learning within a learner control training environment: The interactive effects of goal orientation and metacognitive instruction on learning outcomes. Personnel Psychology, 5 6, 405-429 Schraw, G. (1998). Promoting general metacognitive awareness. Instructional Science, 26, 113-125. Schraw, G., & Moshman, D. (1995). Metacognitive theories. Educational Psychology Review, 7, 351-371. Seijts, G. H., & Latham, G. P. (2001). The effect of distal learning, outcome, and proximal goals on a moderately complex task. Journal of Organizational Behavior, 22, 291-307. Smith, E. M., Ford, J. K., & Kozlowski, S. K. J. (1997). Building adaptive expertise: Implications for training design. In M. A. Quinones & A. Dudda (Eds), Training for 21‘" century technology: Applications of psychological research (pp. 89-118). Washington, DC: APA Books. Speannan, C. (1923). The nature of intelligence and principle of cognition. London: MacMillan. Steele-Johnson, D., & Kanfer, R. (1992). Goal-performance relations: The effects of initial task complexity and task practice. Motivation and Emotion, 16, 117-141. Steinberg, E. R. (1997). Review of student control in computer-assisted instruction. Journal of Computer-Based Instruction, 3, 84-90. Steinberg, E. R. (1989). Cognition and learner control: A literature review, 1977-1988. Journal of Computer-Based Instruction, 16, 117-121. Swanson, H. L. (1990). Influence of metacognitive knowledge an aptitude on problem solving. Journal of Educational Psychology, 82, 306-314. Tarski, A. (1956). The concept of truth in formalized languages. In A. Tarski (Ed.), Logic, semantics, metamathematics (pp. 152-178). Oxford, England: Clarendon Press. Terborg, J. R. (1976). The motivational components of goal setting. Journal of Applied Psychology, 61, 613-621. Tubbs, M. E. (1993). Commitment as a moderator of the goal-performance relation: A case for clearer construct definition. Journal of Applied Psychology, 78, 86-97. VandeWalle, D. (1997). Development and validation of a work domain goal orientation instrument. Educational and Psychological Measurement, 8, 995-1015. 144 Williams, M. D. (1996). Learner-control and instructional technologies. In D.H. J onassen (Ed.), Handbook of research for educational communications and technology (pp. 957-982). New York: Simon & Schuster Macmillan. Winne, P. H. (1995). Inherent details in self-regulated learning. Educational Psychologist, 30, 173-187. Winne, P. H. (1996). A metacognitive view of individual differences in self-regulated learning. Learning and Individual Differences, 8, 327-353. Winters, D., & Latham, G. P. (1996). The effect of learning versus outcome goals on a simple versus a complex task. Group & Organization Management, 21 , 236-250. Wood, R. E. (1986). Task complexity: Definition of the construct. Organizational Behavior and Human Decision Processes, 3 7, 60-82. Wood, R. & Bandura, A. (1989). Social cognitive theory of organizational management. Academy of Management Review, 14, 361 -3 84. Wood, R., Bandura, A., & Bailey, T. (1990). Mechanisms governing organizational performance in complex decision-making environments. Organizational Behavior and Human Decision Processes, 46, 181-201. Wood, R. E., & Locke, E. A. (1990). Goal setting and strategy effects on complex tasks. Research in Organizational Behavior, 12, 73-109. Wood, R. E., Mento, A. J ., & Locke, E. A. (1987). Task complexity as a moderator of Goal Effects: A meta-analysis. Journal of Applied Psychology, 72, 416-425. 145 3 1293 027