 

In“

x l! O

This is to certify that the
thesis entitled

Self-regulation and Adaptation: A Process Approach

presented by

Brady M. Firth

has been accepted towards fulﬁllment
of the requirements for the

Master of degree in Psychology
Arts

 

 

WW4. '

V Major ﬁcfessotﬁignature
23 #45 50 m

Date

 

MSU is an Afﬁrmative Action/Equal Opportunity Employer

 

LIBRARY
Michigan State

University

PLACE IN RETURN BOX to remove this checkout from your record.
To AVOID FINES return on or before date due.
MAY BE RECALLED with earlier due date if requested.

 

DATE DUE

DATE DUE DATE DUE

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

5108 KIProj/Aoc8Pres/ClRC/DateDueJndd

 

 

SELF-REGULATION AND ADAPTATION: A PROCESS APPROACH
By

Brady M. Firth

A THESIS
Submitted to
Michigan State University
In partial fulﬁllment of the requirements
for the degree of
MASTER OF ARTS

Psychology

2010

ABSTRACT
SELF-REGULATION AND ADAPTATION: A PROCESS APPROACH

By
Brady M. Firth

Research on adaptation has only recently begun to rely on a self-regulatory,
process-oriented framework. This paper extends this framework by studying how
individuals engage in self-regulatory processes when overcoming changes in the
environment. Metacognitive and feedback manipulations are introduced during training
in order to investigate how training can enable individuals to better engage in self-
regulatory processes that enable adaptation to environmental changes. Results indicate
that studying adaptive processes over time is essential, as the effects of metacognitive
training were only manifest over time. Although many hypotheses were not supported,
this study indicates that multiple regulatory pathways and processes are involved in
adaptation. An alternate, dynamic model of adaptation is offered and future research

directions suggested.

This work is dedicated to Danielle, Liam, and Elsie.

ACKNOWLEDGEMENTS

I would like to offer a special acknowledgement to Steve Kozlowski for his
guidance throughout this project. He helped steer me in the right direction, assisted each

step of this project, and aided me through a lengthy and highly educational process.

I would also like to acknowledge Rick DeShon and Kevin Ford for their valuable
contributions. Rick has helped me to think about systems dynamically, and contributed
directly to the way I think about problems. Kevin has helped me to think in terms of

multiple frameworks, be optimistic about my work, and pursue interesting problems.

TABLE OF CONTENTS

LIST OF TABLES ............................................................................... iiiv
LIST OF FIGURES ............................................................................... ix
CHAPTER 1
INTRODUCTION
Introduction ................................................................................ 1
Adaptation Research ..................................................................... 4
ConceptualizingAdaptation 15
Adaptive Processes ....................................................................... 17
Training Interventions .................................................................... 20
CHAPTER 2
METHOD
Study 1 ...................................................................................... 32
Task ............................................................................... 33
Procedure ......................................................................... 33
Manipulations .................................................................... 33
Measures ........................................................................... 34
Results ............................................................................. 37
Study 2 ..................................................................................... 37
Sample ............................................................................. 37
Task ................................................................................. 38
Design and Procedure ........................................................... 38
Manipulations .................................................................... 39
Measures ........................................................................... 39
Analytic Strategy ................................................................. 43
CHAPTER 3
RESULTS
Results ...................................................................................... 46
CHAPTER 4
DISCUSSION
Discussion .................................................................................. 51
Contributions .............................................................................. 57
Limitation ................................................................................. 60
Future Directions and Alternate Model ................................................ 61
Conclusion ................................................................................. 7O
APPENDICES
Appendix A ................................................................................ 71

Appendix B ................................................................................. 93
REFERENCES ..................................................................................... 95

vi

LIST OF TABLES

Table 1. Correlation table, between-individuals ........................................... 71

Table 2. Correlation table, between- and within-individuals ........................... 73
Table 3. HLM results for H1 ................................................................. 74
Table 4. HLM results for H2 ................................................................ 74
Table 5. Results for H4 ....................................................................... 75
Table 6. Results for H5 ....................................................................... 75
Table 7. Results for H6 ....................................................................... 76
Table 8. Results for H6 ....................................................................... 78
Table 9. Results for H7 ...................................................................... 81
Table 10. Normative score values .......................................................... 82
Table 11. Equations ........................................................................... 83

vii

LIST OF FIGURES

Figure 1. Adaptive self-regulation heuristic ........................................... 19
Figure 2. Dynamic adaptation heuristic .................................................. 66
Figure 3. Metacognition ................................................................... 85
Figure 4. Self-efﬁcacy ..................................................................... 86
Figure 5.Goal level ......................................................................... 87
Figure 6. Effort .............................................................................. 88
Figure 7. Strategic effort .................................................................... 89
Figure 8. Performance ..................................................................... 90

viii

Chapter 1
Introduction

Adapting to changing environments is a central human endeavor. In the workplace,
technological developments, team-centered work, changes in organizational structures,
and an increasingly global economy require employees to be responsive to change. When
environmental pressures shift, individuals must adapt their behavior in order to maintain
or improve performance. The recent turmoil of the world economy has placed a new
premium on adaptability, as companies seek leaders and workers who cannot just survive
hard times, but proﬁt from them. From factory ﬂoor to executive suite, adaptation seems
to matter for both the bottom-line and the well-being of employees.

While researchers and managers have both become more interested in understanding
what enables adaptation, the nature of adaptability is still in question. Three primary
reasons explain why researchers have been unable to reach a consensus on how to best
deﬁne and conceptualize individual adaptation. First, adaptation is a multi-dimensional
phenomenon that involves multiple constructs. These multiple dimensions and
components will be described in detail below. However, this complexity precludes any
quick and easy solutions on how best to conceptualize adaptation. Second, there are
multiple methodological difficulties inherent in studying adaptation. For example, the
need to incorporate the element of time, a reliance on self-reported measurements of key
variables, how best to measure variables related to adaptation (e. g., knowledge structures
and metacognition) and the difﬁculties of generalizing ﬁndings beyond a local context all
complicate the study of adaptation (Kraiger, Ford, & Salas, 1993; LePine et al., 2000;

Smith et al., 1997). Third, researchers approaching any problem utilize alternative

perspectives and come to the table with differing assumptions. The study of adaptation
has been no different. These three difﬁculties have resulted in multiple deﬁnitions of
what adaptation is. Adaptation has been deﬁned as a latent variable (V erbruggen & Sels,
2008), behavior (Ashford, 1986; Pulakos et al., 2001), an outcome (Burke et al. 2006;
Kozlowski et al., 2001) and a process (Bell & Kozlowski, 2008; Chen, et al. 2005).
Furthermore, adaptation is closely related to other concepts such as innovation, learning,
creativity (Burke et al., 2006), and far transfer. Given such varied deﬁnitions,
operationalizations, and conceptualizations, researchers are yet to agree on what
adaptation “really” is.

These basic differences have resulted in recent adaptive performance research
progressing along three general frameworks. First, Pulakos and colleagues identiﬁed
eight task work dimensions of adaptation and found that individual differences predicted
performance on these dimensions (Pulakos et al., 2002). This research reﬁned our
understanding of adaptive performance dimensions and identiﬁed useful predictors of
such performance. However, it did little to inform what can be done to enable adaptation.
Second, many researchers have treated adaptation as a result of training. This research
typically identiﬁes individual differences and training elements which predict
performance on a transfer task (e.g., Ford et al., 1998; Kozlowski et a1. 2001 ). This
research does more to identify ways of improving adaptive performance for certain task
types, but falls short of explaining how individuals self- regulate learning and
performance while adapting. Other researchers suggest more directly that adaptation
occurs through self-regulatory processes that can be enacted during learning and task

performance. For example, learners who are able to regulate cognitive, motivational,

and/or affective processes during learning are better able to transfer knowledge, skills,
and abilities to novel or more complex tasks (Bell & Kozlowski, 2008; Chen et al., 2005).
This work begins to illuminate what the process of adaptation entails, but has not taken
full advantage of longitudinal data collection in order to measure the underlying
processes involved over time.

Delineating the roles that such regulatory processes play during transfer and future
performance is the primary aim of this study. I meet this aim and extend research on
individual adaptation to environmental change in three ways. First, I hope to extend and
develop process-oriented adaptation theory. Adaptation has only recently been construed
in the organizational literature as a process and, as noted previously, this
conceptualization is neither dominant nor pervasive. Previous process-oriented research
has used relatively simple and static methods of measurement, typically capturing
relevant processes by measuring training transfer at one or two time points. This design is
useful to answer whether individuals do adapt, but lacks the ability to answer how
individuals adapt (i.e., how regulatory pathways are enacted over time to enable
adaptation). In this paper I argue for a process-oriented theory of adaptation and use a
methodology suitable to such an argument. I measure self-regulatory processes over time
while subjects learn, practice, and perform a computerized radar task, and show that
individuals that enact such processes are better able to adapt to changing environmental
pressures.

Second, I seek to extend theory that entails multiple regulatory pathways. Bell &
Kozlowski (2008) hypothesized that three pathways (cognitive, motivational, .and

affective) are involved in adaptive self-regulation. They found that the cognitive and

 

motivational pathways are closely intertwined, and that the affective pathway did not
have as dominant a role. In this study, I seek to disentangle the ﬁrst two pathways by
investigating the relationship of several cognitive and motivational variables over time.
By measuring and describing the relationships between these variables across time in
more detail, I hope to come closer to identifying causal relationships within and across
these pathways.

Third, although adaptation is a result of learning, practice, and self-regulation, how
best to prepare individuals to adapt remains an unanswered question. Moving beyond
knowledge structures and individual differences, it is unclear what processes separate
hi gh-adapters from low-adapters. I implement metacognitive- and feedback-based
training approaches to investigate how self-regulatory processes are engaged during
training and what the effects of these processes during adaptation are. By introducing
metacognitive and feedback manipulations during training and then measuring their
effects during both learning and adaptive phases, it will be possible to gain a better
understanding of how adaptive processes are responsive to intervention. This ﬁrst step in
identifying meaningful interventions for leveraging adaptive processes will help identify
boundary conditions and practical utility of adaptation theory.

Adaptation Research

Recent research on adaptation can be described by two general frameworks. First,
adaptation has been conceptualized as the effect of individual differences on
performance. From this perspective, individuals possess particular characteristics that
enable them to perform well in particular contexts. Second, adaptation has been

considered from a process perspective. In this case, individuals with varying

characteristics engage in particular processes during learning or training which lead to
desirable performance outcomes in adaptive contexts. Within each of these general
frameworks, various deﬁnitions and approaches have been taken.

Adapation has been explained in terms of stable individual characteristics for a
number of years (Jackson, 1967; Morrison, 1977; Mumford, Baughman, Threlfall,
Uhlman, & Costanza, 1993; Murphy, 1989; Stemberg, 1997). Scholars have built from
this individual difference approach to offer more speciﬁc deﬁnitions of what adaptive
performance is, complete with differential predictors and speciﬁc job requirements.
LePine, Colquitt, and Erez (2000); Pulakos, Arat, Donovan, and Plamandon (2000); and
Pulakos et al. (2002) have done the brunt of this work.

LePine et al. (2000) investigated the effects of ability, conscientiousness, and
openness to experience on performance in a changing environment. These authors’
theoretical approach was based in a desire to identify personal characteristics that are
advantageous for selection efforts in adaptive environments. Identifying selectable traits
that permit adaptation, they argued, has the advantage over continually hiring new
employees that ﬁt the demands of the current environment or of offering continual
training to update employees’ capacities for changing demands because it is more
practical and less expensive.

LePine et al. (2000) deﬁned adaptive performance as the capability and willingness to
work in a changing environment. They operationalized adaptive performance as decision-
making performance within a computer simulation after rule changes occurred within the

task. Measuring adaptive performance in this way enabled the authors to offer empirical

support for conceptualizing adaptive performance as distinct from generic performance
by demonstrating that the two types of performance are predicted by different variables.

Participants engaged in a 75-trial computerized task in which they were required to
make decisions about how to process sets of enemy units that appeared on the game grid
and had different characteristics and processing requirements and priorities. Aﬁer trials
25 and 50, the processing and decision rules were, unknown to the participants, changed
such that participants were forced to adapt their behavior in a way that enabled them to
track these changes and learn and use the new rules. Prior to the lSt change, ability was
the only predictor of performance, with an absolute correlation of r = 0.23. Following the
1” change, ability was correlated more strongly with adaptive performance at an absolute
r =0. 43, and conscientiousness and openness predicted adaptive performance at r = 0.29
and r =-0. 35, respectively. The negative relationship between conscientiousness and
performance was driven by dependability facets rather than achievement facets. These
correlations were the same after the second change as they were after the ﬁrst change.

This study indicates that there are empirical differences in pre- and post-change
performance, at least in regards to valid predictors of each. The authors demonstrated that
adaptive performance is a useful idea, since it requires something more than generic
performance does from the individuals doing the performing. Furthermore, they
identiﬁed popular and widely used individual difference variables as predictors of
adaptive performance.

The primary limitations of the study in regards to the current paper is that the authors

overlooked any sort of processes involved in adaptive performance by focusing only on

the predictors of distinct performance periods. We are left with little idea about what’s

going on during or following the changes in the task that permits successful individuals to
bring their advantageous characteristics to bear. Some light was shed on this by the
supplemental analyses that indicated that individuals scoring higher on dependability,
order-seeking, and deliberation did worse in adapting than others. Contrary to
conventional wisdom in the ﬁeld, conscientiousness may not always be a good thing.
This suggests potential mechanisms that enable adaptive performance. For example, it is
possible that individuals focused on keeping things under control and seeking the status
quo are poor at adapting. However, the methodology precluded any investigation of
distinctions between those with characteristics predictive of adaptive performance. Are
these characteristic necessary or just sufficient for adaptive performance? Can individuals
enact particular processes that enable them to successfully perform regardless of their
personal characteristics? This study leaves these questions unanswered.

Contributing to the identiﬁcation of individual differences that enable adaptive
performance, Pulakos et al. (2000) developed a taxonomy of adaptive performance with
the intent that this taxonomy would in turn permit better understanding and prediction of
adaptive behavior. Their aim was to add adaptive performance to the eight dimensions of
performance put forward by Campbell et al. (1993). In pursuing this end, adaptation was
deﬁned in terms of observable and measurable behaviors across different contexts. By
deﬁning adaptive performance in terms of a set of speciﬁc behaviors, the authors hoped
to identify particular criteria that could then be used to produce a better index of relevant
predictors.

Pulakos et al. (2000) used a total of 9,462 critical incidents ﬁom 21 jobs across 11

organizations to assess the extent to which hypothesized adaptive performance

dimensions were involved across jobs. These critical incidents were coded and sorted into
eight dimensions of adaptive performance. The eight dimensions of adaptive performance
are handling emergencies or crisis situations; handling work stress; solving problems
creatively; dealing with uncertain and unpredictable work situations; learning work tasks,
technologies, and procedures; demonstrating interpersonal adaptability; demonstrating
cultural adaptability; and demonstrating physically oriented adaptability.

The critical incidents were used to create a 68-item Job Adaptability Instrument (JAI)
that assesses the extent to which adaptive performance dimensions are a) of importance
and b) require time on a given job. In a test of nine different jobs, the JAI demonstrated
high reliability (alpha M = .94, alpha SD = .20). The agreement of individuals within
these nine jobs was also high, with ICCs ranging from .72 to .98. Although EFAs and
CFAs indicate that eight-dimensional model is the best ﬁt, the dimensions correlate
anywhere from .30 to .69 with one another.

Pulakos et al. (2002) further tested this eight-dimensional model of adaptive
performance by investigating the degree to which certain individual differences predict
each of the eight dimensions. The authors developed and administered a biodata
instrument meant to sample adaptive behaviors, an interest instrument that measured
interest in working in situations that demand dimensions of adaptive performance, and a
task-speciﬁc self-efﬁcacy measure that captured how effective individuals feel they are at
adapting in ways consistent with the eight dimensional taxonomy of performance. These
three measures were aligned such that they measured their respective constructs in ways
that were as similar as possible across each of the eight dimensions of adaptive

performance. Openness to experience, emotional stability, achievement motivation, and

ability were also used to predict adaptive performance. Two adaptive performance
measures (i.e., a behaviorally based rating scale and supervisory ratings) were used to
provide criterion validity for the dimensions of adaptive performance. Overall, past
experience, interest, self-efﬁcacy, ability, and achievement motivation all predicted
adaptive performance. Only a single scale of the experience measure provided
incremental prediction of adaptive performance above and beyond ability and
achievement motivation.

Pulakos and colleagues demonstrated that adaptive performance can be described in
terms of multiple dimensions. As far as efforts to predict performance behaviors from
individual differences are concerned, this work is indispensable as it establishes relatively
concrete criteria that likely generalize across most jobs. Previous work regarding
individual differences and their impact on adaptive performance was largely fractured, as
performance was operationalized uniquely by each set of authors. While work by Pulakos
and colleagues did not completely obviate this problem, it did lay the groundwork for
systematizing any ﬁndings related to individual-difference predictors of adaptive
performance. With such a foundation in place, future efforts to identify additional
predictors of adaptive performance behaviors can simultaneously be more precise and
generalizable.

Despite these advances, the taxonomy of adaptive performance set forward by
Pulakos and colleagues and the other work on predicting adaptive performance reviewed
to this point rest on particular assumptions about the nature of adaptation that may not be

completely founded. The primary assumptions are that adaptation is an outcome

predictable by particular characteristics of the individual and contingent on the work
environment.

Adaptation can also be conceived in terms of processes and learning behaviors that
impact the way in which individuals process information and behave in adaptive
contexts. Similar to LePine et al. (2000), Kozlowski et al. (2001) investigated individual
differences as predictors of adaptive performance. However, in this case the focus was on
the inﬂuence that the process of training and learning had on adaptive performance.
Individual differences were used to predict numerous outcomes of training, and
adaptation was deﬁned as the generalization of what had been learned during training to a
more difficult and complex computer simulation task. It was found that goal orientation,
training goals, and ability predict training outcomes, which were operationalized as
knowledge structure development, training performance, self-efﬁcacy, and performance
in an adaptive transfer task. Regression results support the argument that the manner in
which people approach adaptive tasks impacts performance outcomes. Being leaming—
goal oriented, pursuing mastery goals, developing coherent knowledge structures, and
building self-efficacy all predict ﬁnal adaptive performance.

Ford, Smith, Weissbein, Gully, and Salas (1998) demonstrated that training
outcomes and ultimately transfer performance are inﬂuenced by the types of learning
strategies individuals engage in. Ford et al. (1998) measured learning strategies by
assessing levels of metacognitive activity during training, the extent to which tasks
similar to the transfer task were practiced, and the frequency of strategic target
classiﬁcation and prioritization during the computer simulation task. These learning

strategy measures predicted the learning outcomes of declarative knowledge, ﬁnal

10

training performance, and self-efﬁcacy. Of most relevance to the current study,
metacognitive activity predicted the learning outcomes of knowledge, ﬁnal training
performance, and self-efficacy, with standardized regression coefﬁcients of 0.22, 0.29,
and 0.32, respectively. These learning outcomes in turn each predicted transfer
performance, which was operationalized as the performance of a more complex version
of the trained task. Results indicated that the effect of at least one individual difference
variable (i.e., goal orientation) on transfer performance is inﬂuenced by the
metacognitive strategies engaged during learning.

The work of Kozlowski and Ford and their respective colleagues emphasized the
importance of individual difference variables on adaptive performance, but contributed
more to the literature by demonstrating that adaptive performance is enabled by more
than just the brute effect of individual characteristics. Training can act on these
characteristics and enable individuals to successfully navigate adaptive contexts.
Knowledge, training performance, and self-efficacy each can be developed through
manipulating the manner in which individuals pursue their goals and engage in learning
processes. Generally, these studies indicate that the activity of the individual during
learning and/or training can impact the likelihood that individuals successfully adapt.

Whereas the model tested by Ford et al. (1998) focused on learning strategies that
impacted learning outcomes; Chen, Thomas, and Wallace (2005) focused on post-training
self-regulatory processes that impacted adaptive performance. Adaptive performance was
operationalized as a novel and complex task on a computerized helicopter simulation
following training. The predictive training outcomes that were measured were declarative

knowledge, self-efﬁcacy, and task skill. It should be noted that these training outcomes

11

are similar to those measured by Ford et al. (1998) and Kozlowski et al. (2001). The
former measured efﬁcacy, training performance, and knowledge; the latter measured
declarative knowledge, self-efﬁcacy, training performance, and knowledge structure
coherence. Following Kanfer (1990), Chen et a1. (2005) bisected self-regulation into goal
choice and goal striving processes. Goal choice processes involve “deciding where and
how to allocate task-related effort”, and goal striving involves the allocation of resources
and expenditure of effort in the pursuit of speciﬁc goals (Chen et al., 2005, p 830).
Although the effects of knowledge and skill on adaptive performance were unmediated
by self-regulatory processes, self-efﬁcacy only impacted adaptive performance through
its inﬂuence on self-regulatory processes. Self-efﬁcacy predicted goal striving processes
more strongly than it did goal choice processes.

Chen and colleagues’ (2005) ﬁndings on the relationships between individual self-
regulatory processes, training outcomes, and adaptive performance are relevant to the
current study for two reasons. First, post- training self-regulation processes are predicted
by training outcomes. Although the reasons for this were not directly tested, it is
reasonable to suppose that whatever happened during training to improve skill and
knowledge also inﬂuences the way in which individuals engage in goal choice and goal
striving activities. Ford et al. (1996) and Kozlowski et al. (2001) also did not measure any
process changes that occurred during training. Even if it is not the case that training
inﬂuences goal processes, the speciﬁc outcomes of training directly impact goal choice
and goal striving activities. This suggests that the relationship between training and self-
regulatory processes is an important one. Second, self-regulatory processes predict

adaptive performance above and beyond training outcomes. The outcomes of training are

12

not of sole importance; the manner in which individuals select, prioritize, allocate
resources towards, and pursue strategic and task goals after training can impact the
degree to which they successfully adapt.

It is regretful that this study does not provide more insight in regards to what it was
about the training that improved training outcomes and enabled successful regulatory
processes. The study included no measurement of self-regulation during training. Rather,
goal choice and striving were respectively measured after a transfer planning and transfer
execution stage following training and prior to an adaptive stage. Potential predictors of
training outcomes were not within the scope of this study, perhaps largely in part because
such predictors had been successfully identiﬁed by previous work. However, the fact that
self-regulation was related to training outcomes and played an important role in post-
training performance raises the question of whether or not self-regulatory processes are
also important during training. Thus far we have seen that self-regulatory processes play
a role in adaptation, but it is yet unknown how these processes operate over time both
during and after training to inﬂuence training outcomes and adaptive performance. The
purpose of this study is to ﬁll this gap in the literature by investigating the role of self-
regulation during training, and how these processes consequently impact adaptive
performance.

Bell and Kozlowski (2008) offered a more expansive process-oriented perspective of
adaptation by suggesting that three general self-regulatory pathways are involved in
adaptive performance. This study examined three core training design elements central to
active learning interventions and measured their effects across cognitive, motivational,

and affective pathways. The ﬁnal outcomes of interest were analogical and adaptive

13

transfer; analogical transfer was performance on a post-manipulation, novel scenario of
equivalent difﬁculty, and adaptive performance was operationalized as performance on a
version of a computerized radar simulation task more complex than was faced during
training or previous trials. Bell and Kozlowski (2008) showed support for the implication
of cognitive, motivational, and affective pathways on adaptive performance, and found
that self-regulatory processes can be inﬂuenced by training design.

Using moderated structural equation modeling, the authors found general support for
their model. The three training designs involving exploratory learning and error framing
each brought about performance gains. The emotion control pathway impacted
performance through self-efficacy (i.e., the motivatonal pathway). The primary
regulatory processes that played a role in this model were metacognition, self-efﬁcacy,
state goal orientations, intrinsic motivation, and state anxiety. Without going into the
details of a rather complex model, all effects of self-regulatory processes on adaptive
transfer were mediated by either self-efﬁcacy or knowledge. Anxiety as well as state
prove and avoid goal orientations signiﬁcantly related to self-efﬁcacy, and supplemental
analyses indicated that metacognitive activity also signiﬁcantly related to self-efﬁcacy.
Metacognitive activity also predicted participation in self-evaluation activities and
intrinsic motivation.

In relation to the present study, Bell and Kozlowski (2008) offer three primary
contributions. First, they demonstrated that active learning interventions are effective.
Self-regulatory processes which in turn affect adaptive performance can be inﬂuenced by
interventions. Second, accounting for the interrelationships of regulatory processes when

predicting adaptive transfer is important. Of most interest are the links between pathways,

14

particularly the links connecting metacognition with intrinsic motivation and self-efﬁcacy
which had standardized coefﬁcients of .37 and .41 , respectively. These two relationships
were not originally hypothesized and were the result of supplemental analyses. However,
the relatively large coefﬁcients associated with these relationships indicate that future
work ought to more closely investigate how metacognition relates to intrinsic motivation
and self-efﬁcacy. Third, although training approaches and regulatory processes are
important predictors of adaptive performance, some individual differences predispose
individuals to be better at adaptive transfer than others. This is in line with previous
research (e.g., Ford et al., 1996; Kozlowski etal., 2001; LePine, 2000). This suggests that
in research focusing on processes of adaptation, individual differences must be controlled
or accounted for.
Conceptualizing Adaptation

Adaptation is the engagement over time of self-regulatory processes which enable
loss reduction, maintenance, or improvement of performance effectiveness in a changing
environment. This conceptualization builds most directly from Jundt (2008). He deﬁned
adaptation as “the ability to appropriately and functionally align behavior with task
and/or environment changes in a way that improves, maintains, or limits decrements in
performance effectiveness” (Jundt, 2008, p 15). In building from this conceptualization, I
highlight that the processes involved in adaptation necessarily unfold over time.

The mechanisms of adaptation are self-regulatory processes. Self-regulatory
processes are those that “enable an individual to guide goal-directed activities over time
and across contexts” and involve feedback loops and dynamic readjustrnents that are

contingent on environmental cues (Karoly, 1993, p 25). These processes and related

15

constructs will be discussed in detail in the following section, but consist of
metacognition, self-efficacy, goal processes, and effort. In the context of adaptation, these
processes will enable individuals to discover and employ new strategies. The
maintenance or improvement of performance serves as criteria for whether or not
adaptation has occurred. Because adaptation is a context dependent phenomenon, this
conceptualization allows for the preservation, maintenance, or improvement of
performance effectiveness; the decision of which to utilize as the ultimate criterion must
be made locally depending on the task. The loss reduction of performance effectiveness
means that decrements in performance are minimized. This may be a suitable criterion if
circumstances are such that decrements are unpreventable or unrealistic. The degree and
speed of environmental changes must also ultimately be decided at a local level. I suggest
that such environmental changes must signiﬁcantly degrade routine or automatized
performance levels and will likely be unanticipated (Eisenhardt & Tabrizi, 1995; Scott,
1987).

One type of adaptation is training transfer. Transfer is usually deﬁned as the ability to
perform a task in a more complex environment than was trained for. It is the
generalization of training to real-world problems. Transfer may also require adaptive
expertise (Kozlowski et al., 2001; Smith, Ford, & Kozlowski, 1997). Adaptive expertise
is the capability to meet novel, difﬁcult, and complex situations by modifying
knowledge, skills, and abilities gained through training (Schmidt and Bjork, 1992). The
development of knowledge structures and metacognitive skills during learning enable
individuals to both recognize opportunities to adapt and know what to do in order to

adapt during transfer. In other words, transfer is inseparable from the process of learning

16

(Gick and Holyoak, 1987). Unfortunately, transfer is not often studied in terms of
processes that are enacted over time (Burke & Hutchins, 2007). My conceptualization of
adaptation incorporates these relationships between learning, self-regulation via
metacognition, and performance in novel environments. This study investigates the role
of self-regulation during an adaptive phase of performance after training; this adaptive
phase is a form of training transfer.
Adaptive Processes

Adaptation requires individuals to alter behavior in strategic ways that require
self-regulation. When individuals are able to self-regulate in novel or changing
environments, they detect change, diagnose its cause, and then demonstrate adaptive
performance (Ivancic & Hesketh, 2000; Jundt, 2008). Once individuals detect and
diagnose a change, they must rebalance their efforts between exploitative versus
exploratory behavior as they adapt. Exploitative behavior is that which seeks to maximize
behavior based on what is known (e. g., engaging in proven strategies), while exploratory
behavior is that which seeks to maximize behavior that reduces uncertainty (e. g.,
discovering new strategies). In a stable environment over time, individuals tend to
balance exploration and exploitation in such a way as to optimize performance, which
often means that exploration occurs at a relatively low rate (March, 1991). When
environmental changes are introduced, this equilibrium must be rebalanced, which often
involves an initial increase in exploration. Individuals will likely be better at restoring
equilibrium between exploitative versus exploratory behavior when they effectively self-

regulate their efforts to navigate change. When individuals are unable to self-regulate in

17

such environments, they will likely maintain maladaptive behaviors or disengage from
the task (Bell & Kozlowski, 2008; Tsui & Ashford, 1994).

The current model which describes the variables and pathways expected to be
involved in adaptive performance over time is illustrated in Figure 1. Given the
responsiveness of self-regulatory processes to training interventions, two training
manipulations are introduced that are expected to create variation in the post-training
levels of self-regulatory processes across conditions. Beyond these initial post-training
differences, it is expected that individuals across conditions will demonstrate different
patterns of self-regulation over time as a result of these training interventions. I introduce
metacognitive and feedback interventions which are expected to inﬂuence metacognition
and self-efﬁcacy, respectively, during training. The effects of these interventions on
metacognition and self-efﬁcacy will then impact self-regulation during an adaptive phase.
Below, I describe the rationale behind these interventions and their expected effects

before turning to a description of how these variables affect patterns of self-regulation

over time.

18

Figure 1. Adaptive self-regulation heuristic

 

 

ZoSoomamé
Hana—um

 

 

 

ZaSoomamos

 

 

 

 

 

£38m? mad:

 

 

 

 

 

31.03286

 

 

 

 

 

manager .560

 

 

 

$3630»th

 

 

Dom—m llv mad:

 

 

 

 

 

 

19

Training Interventions

Metacognition is the knowledge of and control over one’s own cognitions (Ford,
Smith, Weissbein, Gully, & Salas, 1998; Flavell, 1979). Metacognition is also the
awareness of how information is acquired, and how information inﬂuences learning and
performance (Schmidt & Ford, 2003). However, metacognition extends beyond the
Socratic injunction to “know thyself” and the mere possession of knowledge. It consists
of processes that are engaged on-line during task performance (Kluwe, 1987).
Metacognition is often conceptualized as consisting of monitoring and control functions,
and also incorporates planning efforts (Brown, Bransford, Ferrara, & Campione, 1983;
Schraw & Moshman, 1995). Monitoring processes consist of the identiﬁcation of tasks to
receive effort and resources, awareness of progress of the task, and anticipation of future
outcomes of the current performance trajectory (Schmidt & Ford, 2003). Control
processes deal with actual engagement in the task. Resource allocation decisions, task
strategy and tactic selection, the selection of task pacing and level of effort to be exerted,
and prioritization of performance actions are all control processes (Schmidt & Ford,
2003). These processes aid planning efforts and assist individuals in learning how best to
allocate resources.

Metacognition permits the detection, diagnosis, and resolution of errors that occur
during learning (Ivancic & Hesketh, 2000). It enables deeper processing and enables
more complex and sophisticated strategies to be developed during training and transfer of
problem-solving tasks (Berardi-Coletta, Buyer, Dominowski, & Rellinger, 1995).
Metacognitive interventions have been successﬁrl at bringing about learning and

performance gains. Teaching students about cognition and providing strategic learning

20

plans have lead to successful strategy use and performance (Manning, 1992; Meloth,
1990; Short & Ryan, 1984). By providing knowledge of the importance and effects of
metacognition, knowledge of how to engage in metacognitive processes, self-questioning
strategies, and self-reﬂection prompts, Schmidt and Ford (2003) successfully induced
metacognitive activity that in turn impacted self-efﬁcacy, declarative knowledge, and
task performance. These ﬁndings suggest that metacognition can be effectively
inﬂuenced during training. Individuals that develop metacognitive processes during
training are likely to perform well during initial transfer tasks (Ford et al., 1998; Schmidt
& Ford, 2003). Therefore, it is hypothesized that:

H1: Metacognitive training will lead to higher initial levels of metacognition at
the beginning of the adaptive phase than no metacognitive intervention.

Feedback, or information regarding progress towards a goal, has long been
recognized as an important element of learning, self-regulation, and performance
(Ammons, 1956; Bogart, 1980; Carver & Scheier, 1982; Guzzo, Jette, & Katzell, 1985;
llgen, Fisher, & Taylor, 1979; Locke & Latharn, 1990). Feedback enables self-regulatory
efforts to be directional and sustainable over time. While feedback is implicitly
incorporated into most adaptive performance research, explicit consideration of general
feedback has been limited. Among the studies focusing on learning strategies, regulatory
processes, and their effects on adaptation, a small number of studies have. incorporated
feedback directly in their models (e.g., DeShon et al. 2004, Tolli & Schmidt, 2008). As
this prior work was not focused on individual adaptive performance, this paper ﬁlls a gap
in the literature by directly investigating the role of different types of feedback on

individual self-regulation and adaptive performance. More speciﬁcally, I investigate the

21

effect of self-referenced versus normative feedback during training on post-training self-
efﬁcacy during an adaptive performance episode.

Feedback interventions affect self-efﬁcacy beliefs (Tolli & Schmidt, 2008). When
positive feedback is given indicating that individuals have performed well, self-efﬁcacy
is likely to increase. Likewise, when negative feedback indicates that individuals have
performed poorly, self-efﬁcacy is likely to decrease. Aside from the direction of the
feedback, the referent of the feedback is also an important determinant of self-efﬁcacy.

Feedback Intervention Theory (FIT) suggests a general explanation as to how
feedback reference may impact self-efﬁcacy (Kluger & DeNisi; 1996). The basic
argument of FIT is that goals or standards are organized hierarchically. Since attention is
a limited resource, only feedback-standard gaps that receive attention actively affect
behavioral regulation. There are three general levels of attention: meta-task processes
involving the self, task-motivation processes involving the focal task, and task-learning
processes involving the details of the focal task. Attention is normally directed at
moderate levels of the goal hierarchy on task-motivation processes. Although attention is
probabilistically distributed across several feedback loops within a hierarchy, a feedback
intervention (F I) will shift attention across these loops up and down the hierarchy.

By conducting a meta-analysis, Kluger and DeNisi (1996) found partial support
for their propositions that F I-induced attention to self can attenuate performance and that
FI-induced attention to task-motivation or task-learning processes can augment
performance. Normative feedback is one mechanism that shifts attention away from task-
motivation processes towards meta-task processes. When individuals receive normative

feedback, they are likely to focus on the self-concept, which debilitates performance. FIT

22

also suggests that attention to the self is accentuated on non-dominant or cognitively
demanding tasks (Baumeister, Hutton, & Cairns, 1990; Mikulincer, Glaubman, Ben-
Artzi, & Grossman, 1991; Wicklund, 1975). When individuals must adapt while
performing a novel or changing task, the task is more likely to be non-dominant and
cognitively demanding.

Vancouver and Tischner (2004) investigated the role of normative feedback on
performance and ﬁ'amed their study in terms of FIT. Vancouver Tischner (2004) argued
that resource-sensitive tasks require more cognitive processing than tasks without
stringent resource limitations. These resources can be stolen by efforts to reduce self-
concept discrepancies. Efforts to reduce self-concept discrepancies, thus diverting
attention away from the task, were hypothesized to be driven by normative feedback
(Kluger & DeNisi, 1996). It was found that a downward spiral of decreasing performance
on a novel task can eventuate after individuals receive negative normative feedback if
they are not given the opportunity to self-afﬁrm. These ﬁndings support FIT’s claim that
normative feedback can draw attention to the meta-task level of the goal hierarchy and
impinge on task-motivation and task-learning processes.

Although meta-task processes, the self-concept, and self-efﬁcacy all share
commonalities, these three constructs are not isomorphic. Shih and Alexander (2000)
tested a more direct link between normative feedback and self-efﬁcacy. They
demonstrated that normative feedback led to lower student self-efﬁcacy, goal-setting, and
performance on a computer-based logic game than self-referenced feedback. This is
likely because normative feedback shifts attention away from task-motivation processes

and can lead to self-doubt and anxiety that hampers self-efﬁcacy. Self-referenced

23

feedback refers to an individual’s prior performance without comparison to others, and
can help maintain focus on task-motivational processes (Shih & Alexander, 2000). Self-
referenced feedback provides individuals with a more useful comparison in regards to
mastering the task, and can lead to a greater mastery orientation which enhances self-
efﬁcacy and performance (Chen & Mathieu, 2008; Fisher & Ford, 1998; Kluger &
DeNisi, 1996; Kozlowski et al., 2001). This can lead toboth an increased motivation to
learn and actual improvements in learning (Shih & Alexander, 2000). In line with Kluger
and DeNisi’s (1996) arguments, feedback that helps individuals maintain attention on
task-motivation processes ought to help maintain self-efﬁcacy. In accordance with these
ﬁndings, it is hypothesized that:

H2: Self-referenced feedback during training will lead to higher initial levels of
self-eﬂicacy at the beginning of the adaptive phase than normative feedback during
training.

It is expected that both interventions will be complimentary. The model predicts a
positive relationship over time between metacognition and self-efﬁcacy. Those receiving
both interventions will receive both the indirect beneﬁt to self-efﬁcacy through increased
metacognition (via metacognitive training) and the direct beneﬁt to self-efficacy (via self-
referenced feedback). Task-focused, strategic approaches should be better than self-
focused and/or non-strategic approaches. Given the expected positive effects of each of
these interventions on metacognition and self-efﬁcacy and the eventual effects these have
on other self-regulatory processes and adaptive performance, it is expected that those
receiving both interventions will out perform all other conditions. Thus, it is

hypothesized that:

24

H3: Those receiving both metacognitive training and self-referenced feedback
will demonstrate the highest levels of self-eﬂicacy and metacognition at the beginning of
the adaptive phase.

It is also expected that both the metacognitive and feedback interventions will
have an effect on self-regulation after the initial increases in post-training levels of
metacognition and self-efﬁcacy. It takes time for individuals to learn to implement
training in a novel environment (Baldwin, Ford & Blume, 2009). Individuals with higher
initial levels of metacognition and/or self—efﬁcacy are expected to engage more fully in
the task over time; they will be better at monitoring and controlling their efforts and at
persisting in the face of difﬁculty. The positive effects of the manipulations on
metacognition will improve over time as individuals learn to engage metacognitive
processes in more complex situations. The self-referenced feedback is expected to
condition individuals to focus on their own prior personal performance; this framing will
lead individuals to become more self-efficacious as they gain mastery over the changing
environment and task. Individuals with lower initial levels of metacognition and/or self-
efﬁcacy are expected to become overwhelmed or frustrated with the adaptive phase, and
rather than remaining engaged and receiving the practice gains that would otherwise be
experienced, they will sustain relatively low and ﬂat levels of self-regulation over time.
Beyond the effect of time, individuals with higher initial levels of metacognition and self-
effrcacy will be better able to respond to changes in the environment that occur during
adaptive episodes Therefore, it is hypothesized that:

H4: Metacognitive training will lead to a higher rate of increase in metacognition

during adaptation than no metacognitive training.

25

 

H5: Self-referenced feedback during training will lead to a higher rate of increase
in self-eﬂicacy during adaptation than normative feedback during training.
Self-regulation Processes Over Time

The general logic of the model (see Figure 1) is that the manipulations impact
metacognition and self-efﬁcacy, and the effects of these two processes on adaptive
performance are mediated by goals and effort. As hypothesized, training manipulations
are expected to ultimately result in greater increases in metacognition and self-efﬁcacy
over time. This mediated model of self-regulation and adaptive performance suggests that
initial values of metacognition and self-efficacy positively predict the initial values of
goals and effort. Furthermore, if the positive linkages between variables in the model
hold, and if metacognition and self-efﬁcacy increase over time, then it is expected that
goals, effort, and adaptive performance should also demonstrate change over time. These
relationships are all expected to be positive, such that an increase in one process leads to
increases in the consequent processes. Thus, those receiving no metacognitive training
and/or normative feedback during training will demonstrate lower levels of self-
regulation that increase slowly over time; those receiving metacognitive training and/or
self-referenced feedback during training will demonstrate higher levels of self regulation
that increase more rapidly over time. The remainder of this section provides further
rationale for these relationships as well as speciﬁc hypotheses.

Metacognition, self-efﬁcacy, goals, and effort are all closely related self-
regulatory processes, and the literature suggests that the linkage relationships posited in
the model will hold over time during adaptation. A positive relationship between

metacognition and self-efﬁcacy has been consistently replicated in a number of studies

26

 

(Bell & Kozlowski, 2008; Ford et al., 1998; Keith & Frese, 2005). The degree to which
individuals monitor and control the environment, their resources, and strategies will
likely impact the degree to which they recognize changes in the environment and enable
the formulation of viable alternative strategies. This, in turn, will lead to improved self-
efﬁcacy, as individuals will feel more conﬁdent in their ability to successfully engage in
the task and implement new strategies.

Self-efﬁcacy is the belief that “one can mobilize the motivation, cognitive
resources, and courses of action needed to exercise control over environmental events” in
order to accomplish a task (Bandura & J ourden, 1991, p 942). One’s efﬁcacy largely
determines the level of effort one will expend on a task, and how long they will persevere
when faced with obstacles (Bandura, 1986). Self-efﬁcacy can lead to higher learning and
performance, at least when between-person comparisons are made (Bandura, 1986, 1991;
Locke & Latham, 1990; Vancouver, 2006). Self-efficacy is also an important learning
outcome (Ford et al., 1998; Kozlowski et al., 2001). Although self-efﬁcacy may lose its
predictive power of future performance when prior performance is controlled for, it plays
an important role when the task or environment becomes more complex and/or difﬁcult
by enabling resilience and persistence (Gist & Mitchell, 1992; Kozlowski et al., 2001).
Through its inﬂuence both during and after learning, self-efﬁcacy positively relates to
adaptive transfer and performance, at least partially through its effects on goal processes
(Bell & Kozlowski, 2008; Chen et al., 2005).

The relationship between self-efﬁcacy and goal processes has been well-
documented. A number of studies indicate that individuals higher in self-efficacy set

higher goals (e. g., Bandura, 1991; Latham & Locke, 1991; Vancouver, Thompson, and

27

Williams, 2001). When efficacy is high, individuals are willing to attempt to accomplish
more. When efﬁcacy is low, individuals will lower their goals, expecting to accomplish
less. Tolli and Schmidt (2008) demonstrated that over time, self-efﬁcacy positively
relates to goal levels. In this case, self-efﬁcacy and goal levels were assessed at two time
points; a decrease in self-efficacy from Time 1 to Time 2 (M = 1.61 , 1.52, respectively)
was accompanied by a decrease in goal level from Time 1 to Time 2 (M = 65.41, 56.87,
respectively). During adaptive episodes, this relationship is believed to hold. When
individuals are faced with change, a belief that they can succeed despite these changes
will enable them to continue to set relatively high goals that will direct their efforts.
When self-efﬁcacy decreases, individuals are likely to set lower goals.

Goal level is the level of performance desired to be obtained (Locke, Chah,
Harrison, & Lustgarten, 1989). Goal level is consistently linked to performance increases
(Locke & Latham, 1990; DeShon et al., 1996). When individuals set higher goals, they
are better able to focus their efforts on tasks that will contribute to the realization of those
goals. However, research indicates that when individuals are performing complex tasks
or performing in novel environments, difficult goals can interfere with initial
performance (Earley, Connolly, & Ekegren, 1989; Locke et al., 1989). Despite these
ﬁndings, it is still unclear how goal level affects adaptive performance over time. Higher
goal levels inhibit initial adaptive performance as individuals struggle to explore and
develop new rules and responses. However, higher goal levels can act as an incentive to
strive even when the goal seems impossible (Locke, 1986), and may help individuals
persist in the face of challenges and push through initial performance decrements during

adaptive episodes. Furthermore, individuals may be more willing to set and pursue

28

difficult goals if they are self-set rather than assigned (Locke, 2001). In the present study,
individuals set their own goal levels. It is expected that as individuals set higher goals,
they will exert more effort on the task despite the changing environment.

Effort is the intensity and duration of work expended on a task (Weissbein, 1992).
Intensity is the rate of work done, and duration is the time spent doing the work. Effort is
a limited resource that can be applied differentially across multiple tasks at once.
Virtually all motivational theories incorporate effort; when individuals pursue goals, they
are exerting effort at particular tasks that enable the accomplishment of those goals
(Kanfer, 1990; Locke & Latham, 1991; Yeo & Neal, 2004). Effort has consistently been
shown to relate to performance on a wide number of settings (Brown & Leigh, 1996;
Rasch & Tosi, 1992; Terborg & Miller, 1978; Yeo & Neal, 2008). Even if individuals are
not discerning in how they expend effort, exerting more effort is more likely to lead to
adaptive performance gains than simply doing less (or nothing at all).

In order to capture the manner in which effort is exerted, I make a distinction
between effort and strategic effort. Strategic effort captures the idea of working smarter
rather than harder. When individuals exert strategic effort, they allocate their time and
resources towards those portions of the task that will bring the largest gains over time. As
discussed previously, metacognition enables the development of and use of strategy
(Berardi-Coletta, Buyer, Dominowski, & Rellinger, 1995; Manning, 1992; Meloth, 1990;
Short & Ryan, 1984). It is expected that as individuals engage in metacognitive
processes, they will develop viable alternative strategies. These strategies will be enacted
not simply by working harder, but by exerting effort in a speciﬁc, intelligent manner.

This strategic effort is expected to contribute directly to adaptive performance.

29

In line with the arguments above, it is expected that as individuals train and adapt
over time:

H6a: Metacognition will predict self—eﬁ‘icacy.

H6b: Self-eﬁ'icacy will predict goal levels.

H6c: Goal levels will predict eﬂort.

H6d: Eﬂort will predict adaptive performance.

H6c: Metacognition will predict strategic effort.

H6f: Strategic eﬂort will predict adaptive performance.

Due to the relationships between the self-regulatory processes and adaptive
performance described above, the change in one variable over time will likely impact the
way its consequent changes over time. It is predicted that the rates of change for each
self-regulation variables impact the rates of change of their respective consequents.

H7a: The rate of change in metacognition will predict the rate of change in self-
eﬂicacy; that is, if metacognition increases more rapidly, self-eﬂicacy will increase more
rapidly.

H7b: The rate of change in self-eﬂicacy will predict the rate of change in goal level.

H7c: The rate of change in goal level will predict the rate of change in eﬁ'ort.

H7d: The rate of change in ejfort will predict the rate of change in adaptive
performance.

H7c: The rate of change in metacognition will predict the rate of change in strategic
eﬂort.

H7f: The rate of change in strategic eﬂort will predict the rate of change in adaptive

performance.

30

Tying these hypotheses back to those made about the effect of the training
manipulations on metacognition and self-efﬁcacy, those in the metacognitive training
and/or self-referenced feedback during training conditions will demonstrate higher levels
of self-regulation that improve over time. If such training increases metacognition and
self-efﬁcacy, and if the relationships between self-regulatory variables are as
hypothesized, then it follows that for any self-regulatory variable, increased levels will
positively relate to higher rates of change over time.

H8: For all self-regulation variables, the initial levels of a self-regulatory variable at

the beginning of adaptation will positively relate to the rate of change of that variable

over time.

31

Chapter 2
Method
Study 1 : Pilot Study

The purpose of the pilot study was to gather normative performance data for use
in the normative feedback during training condition in the primary study. The same task,
measures, and procedure were implemented in the pilot study as in the primary study,
with the exception that no feedback manipulations were implemented. Furthermore, the
strength of the metacognitive training manipulation was piloted in order to ascertain if it
was of appropriate design and strength for the primary study.

Data was collected from 106 students at a large midwestem university.
Participants received credit in the psychology research pool for their participation.
Participants were removed from any analyses if they had large amounts of incomplete
data. This resulted in a ﬁnal sample size of 98.

Task

I used the computer-based radar-tracking simulation TANDEM for the
experimental task (Dwyer et al. 1992). TANDEM is a complex decision-making task that
requires participants to track and engage enemy contacts. To engage contacts,
participants “hook” them, gather information regarding 3 classes of characteristics, and
then use this information to decide if and how the contact should be processed (i.e., take
action or clear). Participants must also prevent contacts from crossing two perimeters on
the radar screen. To do this, participants must locate the perimeters, monitor the
perimeters, prioritize contacts, and process prioritized contacts. Points are gained for

correctly processing contacts, and are lost for permitting contacts to cross the perimeters.

32

Procedure

Before arriving at the lab, participants took an online survey which contains
demographic and individual difference items. This occurred when students signed up for
the study from one week to one day before arriving at the lab.

Initial training consisted of a 12-minute power point presentation that taught basic
operating procedures and rules of the game. Participants were then given 2.5 minutes to
review the game manual, and then played an initial 1 minute familiarization trial.

Next, participants completed four training trials while receiving their respective
experimental manipulation. Each trial consisted of ﬁve parts. Participants ﬁrst had the
opportunity to review the game manual, which contains all the information required to
perform well on the task. Next they set goals, performed a four-minute trial, received
feedback that details performance across a number of dimensions, and responded to a
small set of items. After the four training trials, participants took a ﬁve minute break.

Following the ﬁve-minute break, participants engaged in an eight-trial adaptive
phase. Each of these six trials consisted of the same ﬁve-step sequence as the training
trials. However, these six trials are more complex than the training trials (i.e., more
contacts, different point values for low- and hi gh-priority contacts, and harsher penalties
for perimeter-crossings by contacts). This increased complexity required participants to
change their strategy in order to be successful. Upon completion of the task, participants
took a series of brief exit measures.

Manipulations:
Metacognitive Training: Participants were given brief instruction on the

importance of metacognition to learning and performance, and were taught about the

33

basic elements of metacognition. Next, participants were taught how to engage in self-
questioning, which has been shown to increase comprehension and the elaboration of
strategy (Berthold, Nﬁckles, & Renkl, 2007; Schmidt & Ford, 2003). Participants were
prompted to respond to a number of questions following each round in an effort to
engage in metacognition. Finally, participants received a handout that summarized the
material taught about metacognition and lists the prompting questions. This sheet was
collected at the conclusion of training.

No-metacognitive Condition: Participants received inert information about radar
tracking in the military to ﬁll the time taken by the metacognitive instruction in the
experimental condition.

Measures

The measures were administered at three general time points. Most individual
difference variables were administered during participation sign-up prior to arriving at
the lab. Process variables were assessed after each performance trial, for a total of 11
times. Finally, once ﬁnished with the task participants completed a brief exit survey.

Individual Diﬂerences:

Goal Orientation (trait): A modiﬁed version of VandeWalle’s (1997) 13-item
measure was used, such that items were phrased to be domain-general rather than work
speciﬁc. Responses were made on a 6-point scale, ranging from 1 (strongly disagree) to 6
(strongly agree). The measure contains three scales. The Mastery Orientation scale
contains 5 items, the Performance-Avoid Orientation scale contains 4 items, and the
Performance-Prove Orientation scale contains 4 items. All items are in Appendix B. This

measure has demonstrated high internal consistency (a = .85 for learning, (1 =83 for

34

performance-avoid, and u =.84for performance-prove) and reasonably good test-retest
reliability (or = .66 for learning, 0. =60 for performance-avoid, and (1 =57 for
performance-prove). It has also demonstrated validity through correlations with related
measures (VandeWalle, 1997).

13mm The IPIP was used to assess personality on the dimensions of
neuroticism, extroversion, conscientiousness, openness, and agreeableness. This measure
consists of 50 items total, with 10 items for each dimension. Reliability coefﬁcients for
the neuroticism, extroversion, conscientiousness, openness, and agreeableness scales are
.86, .86, .81, .82, and .77, respectively (Goldberg, 1999).

AM Participants self-reported their highest ACT or SAT scores. Both the SAT
and ACT have demonstrated large general cognitive ability components (Frey &
Detterman, 2004) and reported internal consistency reliabilities have been high for both
(e.g., KR-20 = .96 for the ACT composite score; American College Testing Program,
1989). Self-report scores were utilized out of convenience, and should be acceptable
given that self-report scores have been shown to correlate strongly with actual scores
(i.e., between .88 and .92; Cassady, 2001; Gully et al., 2002).

Process Variables:

Metacognition: Metacognition was assessed by a 4-item measure. This measure is
designed to assess how well participants plan, monitor, and control their goal-oriented
behavior. All items were measured on a 5-point scale, from 1 (never) to 5 (constantly).

Self-efﬁcagvfsmte): A 2-item measure was used to assess participants’ self-
efﬁcacy. All items were measured on a 5-point scale, from 1 (strongly disagree) to 5

(strongly agree).

35

ML Goal level is the level of performance an individual seeks to obtain.
Participants were asked to record a speciﬁc performance goal, in terms of total points
expected to be gained in the upcoming round.

Eﬂgr; The total number of targets engaged during a trial was calculated and used
as the measure of effort.

Strategic Effort: Strategic effort was assessed behaviorally by summing the
number of times participants “zoom” in or out on the radar screen, engage high priority
contacts, and the time spent examining the manual before each trial. The nature of the
task makes it is difficult if not impossible to successfully adapt without a proportional
increase in strategic effort.

Dependent Variable:

Performance: Performance was operationalized as the total trial score, and was thus
assessed at the end of each trial. The total ﬁnal score of the adaptive performance trials
was used as a measure of adaptive performance. The adaptive performance trials differed
substantially from the training trials along the three dimensions of task complexity
identiﬁed by Wood (1986). In the adaptive performance trials, there were 22 contacts,
which is about 38% more than the 16 contacts in the training trials (component
complexity). For the adaptive trials, 16 contacts were present immediately at the start of
the trials, with an additional six ‘pop-up’ contacts appearing at 30 second increments
(coordinative complexity). There were also penalties imposed for perimeter-crossings by
contacts during the adaptive trials, such that 75 points was deducted for each crossing
(dynamic complexity). The nature of these changes was such that individuals must alter

their search patterns and prioritize how they process contacts.

36

Results

The primary purpose of this pilot study was to obtain baseline performance data
to use to calibrate the normative feedback for the primary study. Normative performance
scores were obtained from this pilot study and utilized as input to the normative feedback
section of the primary study. The mean score across the training trials for the pilot study
was 765, with a SD of 244. The mean score for trial 1 was 581, for trial 2 it was 784, for
trial 3 it was 815, and for trial 4 it was 882.

A secondary purpose of this study was to test the metacognitive training
manipulation and its effect on post-training metacognition levels. There was no main
effect of training on trial 5 metacognitive levels (F=.302, p = 0.75). It was expected that
there would be signiﬁcant results. In order to strengthen the metacognitive training
manipulation, changes were made to the training protocol and handouts received by the
subjects such that they were more clear, succinct, and straightforward. The training in the
primary study reﬂected these changes to training materials and content.

Study 2: Primary Study
Sample

Data was collected from 396 undergraduate students (approximately 100 for each
condition) at a large midwestem university. Participants received credit in the psychology
research pool for their participation. Participants were removed from any analyses if the
majority of their process measure data was missing or unusable. This removal resulted in
an approximate sample size of 378, with some analyses resulting in slightly different

sample sizes based on how minor missing data was treated and what speciﬁc variables

were included.

37

Task

The Tandem task used in this study was identical to that used in the pilot study,
with the exception that the placement of 6 contacts were varied from what was reported
above. These contacts were positions such that they crossed perimeters at different times.
Design and Procedure

The design of this study was a 2X2 between-subjects with repeated measures on
processes and outcomes. The ﬁrst between-subjects experimental manipulation was such
that one group received metacognitive training while the other received none. The second
between-subj ects experimental manipulation was such that one group received self-
referenced feedback during training, while the other received normative feedback.

Before arriving at the lab, participants took an online survey which contained
demographic and individual difference items. This occurred when students signed up for
the study from one week to one day before arriving at the lab. Upon arriving at the lab,
individuals were randomly assigned to one of the four experimental conditions. Next,
participants received training.

Initial training consisted of a 12-minute power point presentation that taught basic
operating procedures and rules of the game. Participants were then given 2.5 minutes to
review the game manual, and then played an initial 1 minute familiarization trial.

Next, participants completed four training trials while receiving their respective
experimental manipulation. Each trial consisted of ﬁve parts. Participants ﬁrst had the
opportunity to review the game manual, which contains all the information required to

perform well on the task. Next they set goals, performed a four-minute trial, received

38

feedback that details performance across a number of dimensions, and responded to a
small set of items. After the four training trials, participants took a ﬁve minute break.

Following the ﬁve-minute break, participants engaged in an eight-trial adaptive
phase. Each of these six trials consisted of the same ﬁve-step sequence as the training
trials. However, these six trials are more complex than the training trials The adaptive
performance trials differed substantially from the training trials along the three
dimensions of task complexity identiﬁed by Wood (1986). In the adaptive performance
trials, there were 22 contacts, which is about 38% more than the 16 contacts in the
training trials (component complexity). For the adaptive trials, 16 contacts were present
immediately at the start of the trials, with an additional six ‘pop-up’ contacts appearing at
30 second increments (coordinative complexity). There were also penalties imposed for
perimeter-crossings by contacts during the adaptive trials, such that 75 points was
deducted for each crossing (dynamic complexity). The nature of these changes was such
that individuals must alter their search patterns and prioritize how they process contacts.
Upon completion of the task, participants took a series of brief exit measures.
Manipulations

Self-referenced Feedback: Participants in this condition were told how their
overall trial performance compared to, their performance on the previous trial. This
comparison was expressed as a simple statement of both scores. In general terms,
feedback read, “Your overall score this round was X. Last round, you scored Y”.

Normative Feedback: Participants in this condition were told how their overall
trial performance compared to the performance of a normative sample on the previous

trial. This comparison was expressed as a value-free percentile ranking. Normative scores

39

were obtained from the pilot study, with scores for each training trial reported above.
Normative scores were obtained for each speciﬁc training trial and utilized in feedback
respectively. The mean score across all four training trials for the pilot study for those not
receiving metacognitive training was 765, with a SD of 244. The 40"% and 60% scores
were 676 and 828, respectively. Reported rankings were rounded up or down to the
nearest ﬁve percent, such that someone scoring at the 52% were told they scored at the
50th percentile. Thus, in the primary study, participants in the normative feedback
condition who scored 800 received feedback that read, “Your overall score this round
was in the 60% percentile compared to other participants’ scores this round”. A simple
deﬁnition of the concept of percentiles was provided to ensure subjects understood this
feedback. Normative performance scores used for feedback can be found in Appendix A.
Metacognitive Training: Participants were given brief instructions on the
importance of metacognition to learning and performance, and were taught about the
basic elements of metacognition. Next, participants were taught how to engage in self-
questioning, which has been shown to increase comprehension and the elaboration of
strategy (Berthold, Nﬁckles, & Renkl, 2007 ; Schmidt & Ford, 2003). Subjects received a
handout that summarized the material taught about metacognition and listed speciﬁc
prompting questions (see Appendix A). After each training trial, participants responded
to these same speciﬁc questions, which were geared towards helping them engage in self-
questioning for this speciﬁc simulation. For this, they were given two minutes to answer
as many of the questions as they could in any order, with the understanding that they
would not be graded, but that they were to use them to their advantage. Furthermore, they

were instructed to spend part of those two nrinutes to memorize the questions so they

40

could be utilized during the adaption trials. All training materials were collected at the
conclusion of training trials (i.e., after trial 4).

No—metacognitive Condition: Participants received inert information about radar
tracking in the military to ﬁll the time taken by the metacognitive instruction in the
experimental condition.

Measures

Goal orientation and ability items were administered during participation sign-up
prior to arriving at the lab. All other variables were assessed either during or after each
performance trial, for a total of 12 times each. Finally, once ﬁnished with the task
participants completed a brief exit survey.

Individual Diﬁkrences:

GoIal Orientation (trail); A modiﬁed version of VandeWalle’s (1997) l3-item
measure was used, such that items were phrased to be domain-general rather than work
speciﬁc. Responses were made on a 6-point scale, ranging from 1 (strongly disagree) to 6
(strongly agree). The measure contains three scales. The Mastery Orientation scale
contains 5 items, the Performance-Avoid Orientation scale contains 4 items, and the
Performance-Prove Orientation scale contains 4 items. All items are in Appendix B. This
measure has demonstrated high internal consistency (a = .85 for learning, (1 =83 for
performance-avoid, and a =.84for performance-prove) and reasonably good test-retest
reliability ((1 = .66 for learning, (1 =60 for performance-avoid, and 0. =57 for
performance-prove). It has also demonstrated validity through correlations with related

measures (VandeWalle, 1997).

41

A_biﬂt_y; Participants self-reported their highest ACT or SAT scores. Both the SAT
and ACT have demonstrated large general cognitive ability components (Frey &
Detterman, 2004) and reported internal consistency reliabilities have been high for both
(e. g., KR-20 = .96 for the ACT composite score; American College Testing Program,
1989). Self-report scores were utilized out of convenience, and should be acceptable
given that self-report scores have been shown to correlate strongly with actual scores
(i.e., between .88 and .92; Cassady, 2001; Gully et al., 2002).

Process Variables:

Music Motivation: Intrinsic motivation was assessed by a 7-item questionnaire
after each trial. All items were measured on a 5-point scale, from strongly disagree to
strongly agree.

Metacoggition: Metacognition was assessed by a 4-item measure. This measure is
designed to assess how well participants plan, monitor, and control their goal-oriented
behavior. All items were measured on a 5-point scale, from 1 (never) to 5 (constantly).

Self-efﬁcacy (state); A 2-item measure was used to assess participants’ self-
efﬁcacy. All items were measured on a 5-point scale, from 1 (strongly disagree) to 5
(strongly agree).

Goal Level: Goal level is the level of performance an individual seeks to obtain.
Participants were asked to record a speciﬁc performance goal, in terms of total points
expected to be gained in the upcoming round.

Em The total number of targets engaged during a trial was calculated and used

as the measure of effort.

42

Strategic Effort: Strategic effort was assessed behaviorally by summing the
number of times participants “zoom” in or out on the radar screen, engage high priority
contacts, and the time spent examining the manual before each trial. The nature of the
task makes it is difﬁcult if not impossible to successfully adapt without a proportional
increase in strategic effort.

Dependent Variable:

Performance: Performance was operationalized as the total trial score, and was
thus assessed at the end of each trial. The total ﬁnal score of the adaptive performance
trials was used as a measure of adaptive performance. The adaptive performance trials
differed signiﬁcantly from the training trials, as detailed above. The nature of these
changes was such that individuals must alter their search patterns and prioritize how they
process contacts.

The averaged performance across the four training trials was used as a control
variable, while the performance scores during the adaptation trials (i.e., trials 5-12) were
utilized to examine the relationships among the regulatory process variables of interest
Analytic Strategy

To test H1-H3, which address the main effects and interactions of the
metacognitive and feedback manipulations, I will use analysis of covariance (ANCOVA).
By using ANCOVA, individual difference variables will be treated as covariates and
partialled out from comparisons between conditions. H1-H3 will be supported if the
corresponding F -tests are signiﬁcant once covariates are accounted for.

To test H4 and H5, which address the changes over time of metacognition and

self-efﬁcacy between conditions, I will use Hierarchical Linear Modeling (HLM). By

43

utilizing HLM, I will be able to test whether or not the coefficients of change for
metacognition and self-efﬁcacy are signiﬁcantly different between groups. It is expected
that in the case of metacognition, the group receiving the metacognitive training will have
a signiﬁcantly larger slope parameter than the control group. Similarly, it is expected that
the self-referenced feedback group will have a signiﬁcantly larger slope parameter than
the normative feedback group.

To test H6, which addresses average relations between key variable pairings, I
will use hierarchical regression. This will enable control variables to be partialled out
before investigating the true relationships of interest.

To test H7, which addresses the degree to which the change in an antecedent over
time predicts the change in a consequent over time, I will use HLM. This analysis will
enable the relationships between each key variable pairing to be investigated overtime.
H8 will be supported if the appropriate slope parameters for each analysis is signiﬁcant.

Finally, to test H8 I will latent growth modeling (LGM) using AMOS 18.0. By
utilizing HLM to investigate this hypothesis, it is possible to obtain the desired
covariance parameter, which would also describes the desired relationship. However,
obtaining the statistical signiﬁcance of this parameter is difﬁcult by such means. By
examining the relationship between intercept and slope through LGM, one can ascertain
the magnitude of the relationship between the initial level of each process variable and
their corresponding rates of change. If the intercept is signiﬁcantly and positively related
to the slope parameter for each process variable, H8 will be supported.

Covariates were included in all analyses. The covariates included were all control

variables described above; that is, intrinsic motivation, goal orientation, ability, and

44

training performance. Intrinsic motivation was included as a Level-l covariate in HLM,

while the others were included as Level-2 Covariates.

45

Chapter 3
Results

Table 1 reports descriptive statistics and between-person correlations among all
variables. Table 2 reports on its lower diagonal the between-person correlations of the
key variables of interest, and on its upper diagonal the averaged within-person
correlations of the key variables. Analyses proceeded by examining only trials 5-12,
which were the adaptive trials. Training trials were not directly included in the analyses
except as a control variable, and training performance was averaged across the four
training trials.

To test H1 and H2, AN COVAs were run. H1, which predicted that metacognitive
training will lead to higher initial levels of metacognition at the beginning of the adaptive
phase than no metacognitive intervention, was not supported (F =.659, p = 0.73; see Table
3). Similarly, H2, which predicted that self-referenced feedback during training will lead
to higher initial levels of self-efﬁcacy at the beginning of the adaptive phase than
normative feedback during training, was not supported (F= 0.43 7, p=.901, see Table 4).
Regarding H1, these results indicate that at the end of training, there was no signiﬁcant
difference in self-reported metacognition scores between metacognitive training
conditions. Likewise for H2, these results indicate that at the end of training, there was no
signiﬁcant difference in self-reported self-efficacy scores between feedback conditions.

In line with these results, H3, which predicted that those receiving both
metacognitive training and self-referenced feedback will demonstrate the highest levels
of self-efficacy and metacognition at the beginning of the adaptive phase, was tested via

MANCOVA and was also not supported (F=.276, p=.843). Recall that participants in

46

condition 1 received metacognitive training with self-referenced feedback, condition 2
received metacognitive training with normative feedback, condition 3 received no
metacognitive training with self-referenced feedback, and condition 4 received no
metacognitive training with normative feedback. It was expected that participants in
condition 1 would demonstrate higher levels of both metacognition and self-efﬁcacy.
This was not the case. There were no differences in initial levels of metacognition or self-
efﬁcacy between any of these groups.

Both H4 and H5 predicted differences in slopes between groups for metacognition
and self-efﬁcacy during adaptation trials. Furthermore, these slopes were anticipated to
be positive. In reality, both process variables decreased over time (see Figures 3 and 4).
Thus, by default H4 and H5 were not supported. However, the thrust of these hypotheses
was to predict differences in slopes between groups for both process variables. HLM was
used to test whether or not there were differences despite the decreasing slopes. Reframed
in terms of metacognitive training leading to a slower rate of decrease in metacognition
than experienced by those without metacognitive training, H4 was supported (B=.16,
SE=.05, df==374, p < .001 , see Table 5). The resultant ICC value from running the
unconditional means model is ICC= 0.663. The results reported result from a model that
incorporates all covariates, includes metacognitive training as a level-2 predictor, with
random slopes and intercepts. Time is incorporated into this ﬁnal model as a level-1
predictor variable.

However, even when reframed in terms of decreasing slopes, H5 was not
supported (B=.03, SE=.02, df=374, p=.158, see Table 6). No signiﬁcant differences in the

slopes of self-efﬁcacy were found, indicating that feedback condition had no impact on

47

the rates of decrease in self-efﬁcacy during adaptation. Results are non-signiﬁcant
regardless of whether the random effect of intercept is included or not. Obtained from the
unconditional means model, the ICC=.597.

H6 predicted individual relationships between process variables in the general model
presented in Figure l averaging across time and conditions. Because the average
relationship, regardless of changes over time, was of interest, hierarchical linear
regression was used to test this block of hypotheses. For each process variable, the values
for trials 5-12 were averaged, creating an aggregate value of each variable for each
participant. Furthermore, all control variables were entered in ﬁrst, followed by the
relevant independent variable in a hierarchical fashion. Results for all H6 analyses are
reported in Table 7. H6a, which stated that metacognition would predict self-efﬁcacy,
was supported (B= .32, t=3.5, p<.001). The ICC for H6a is 0.586. H6b, which stated that
self-efﬁcacy would predict goal-level, was not supported (B=.09, t=2.057, p=.11); the
ICC for H6b is 0.623. H6c, which stated that goal level would predict effort, was
supported (B= .24, t=7.78, p<.001); the ICC for H6c is 0.568. H6d, which stated that
effort would predict performance, was supported (B=.75, t=26.9, p<.001); the ICC for
H6d is .526. H6e, which stated that metacognition would predict strategic effort, was not
supported (B=.084, t=1.26, p=.209), indicating that across conditions metacognition does
not predict strategic effort; the ICC for H6e is .285. To further investigate these null
effects, a supplementary analysis was run. An alternate model was tested in which control
variables were entered ﬁrst in block one, then metacognitive training was entered as a
predictor of strategic effort in block 2. Receiving metacognitive does predict strategic

effort (B=.131, t=2.50, p<.05), indicating that receiving metacognitive training positively

43

predicts strategic effort across trials. This ﬁnding indicates that self-reported
metacognitive levels may not adequately capture the beneﬁts received by metacognitive
training. H6f, which stated that strategic effort would predict adaptive performance, was
supported, but in the opposite direction anticipated. Results indicate that the average level
of strategic effort across all adaptive trials negatively predicts performance averaged
across all trials (B=-.23, t=-2.69, p<.05). This may be due to the speciﬁc features of the
simulation. The point values of targets were such that the penalty for not processing
contacts strategically were offset by the point values obtained by processing non-
strategic, easily processed targets. Thus, those individuals not demonstrating strategic
effort were able to process more targets and have higher scores as a result, even despite
penalties for not being strategic.

Having tested the average effects of each process variable averaged across time, I
sought to uncover the relationships between each variable pairing of interest over time
and across conditions. To do so, HLM was used because it enables one to obtain the
desired growth parameters by incorporating multiple measurements of key process
variables within individuals. Results for all H7 analyses are reported in Table 8. H7a,
which predicted that changes in metacognition would predict changes in self-efﬁcacy,
was supported (B=.039, SE=.01, t=3.18, df=375, p<.05). H7b was supported, such that
changes in self-efﬁcacy predict changes in goal-level (B=13.64, SE=6.38, t=2.04,
df=375, p<.05). H7e was supported, such that changes in goal-level predict changes in
effort (B=.002, SE=.00035, t=4.67, df=3 75, p<.000). H7d was also supported, indicating
that changes in effort predict changes in adaptive performance (B=73.7, SE=2.28, t=31.6,

df=375, p<.001). H7e was not supported (B=-0.13, SE=0.08, t=-l .34, df =375, p=.l8).

49

However, when adding metacognitive training as a level-2 predictor, H7e was supported
such that for those receiving metacognitive training, metacognition predicted adaptive
performance (B=-0.11, SE=.004, F218, dﬁ375, p<.05). The relationship is negative
because metacognition decreases over time. Further analyses (MAN COVA) indicate that
the levels of strategic effort are relatively stable over time, while metacognition decreases
over time, and that the effect of metacognitive training on strategic effort is signiﬁcant
(F=6.492 df=1 p<.05). H7f was supported (B= 6.39 SE= 1.24, t= 5.18, df=374, p=.000),
indicating that changes in strategic effort positively relate to changes in performance. The
general equation from which all H7 analyses were obtained is found in Equation 1.3.

To test H8, which stated that initial levels of a variable would predict the rate of
change of that variable, latent growth modeling was performed using Amos 18.0. For
each key variable, the general model used was as follows. For each variable of interest,
the intercept value was drawn so as to predict the slope value, and all covariates included
in prior analyses were included into the model. The desired covariance parameter relating
the intercept and slope was obtained. Running this model for each respective variable, the
desired parameters were only statistically signiﬁcant for self-efﬁcacy, effort, and adaptive
performance (-.047; -.034; and -.07, respectively; p<.05). Results for all H8 analyses are
reported in Table 9. Because each of these signiﬁcant results were both small and

negative, H8 was not supported.

50

Chapter 4
Discussion
As a whole, the results of this study were not as expected. The speciﬁc reasons for

why this may be so will be discussed in detail below. Generally, the failure of the
feedback intervention was the greatest disappointment, and likely contributed to the
inability to ﬁnd many of the expected relationships and trajectories. Also disappointing
was the lack of a major and immediate impact of metacognitive training. Despite the
setbacks experienced, the results of this study indicate that metacognitive training can
lead to changes in adaptive processes over time. Furthermore, the rates of change of
many important process variables predict the rates of change of their intended
consequents. Also demonstrated was that the general hypothesized relationships between
key process variables tend to hold when averaged over time.

It was found that receiving metacognitive training does lead to a slower rate of
decay of metacognitive processes enacted over time during adaptive periods. The rate of
change of strategic effort is predicted by the rate of change of metacognition for those
receiving metacognitive training, such that those with higher metacognitive scores over
time exert more strategic effort. Thus, higher levels of strategic effort are exerted during
adaptation trials by those receiving metacognitive training compared to those without.
Furthermore, the rates of change of self-efﬁcacy, goal level, and effort were all predictive
of the rates of change of their hypothesized consequents. In the cases of metacognition
and self-efﬁcacy, negative slope changes were found.

The metacognitive manipulation did drive the differences in slopes in

metacognition during the adaptation trials, but the fact that there were no differences in

51

intercepts for metacognition immediately following training indicates two points. First,
and least interesting, is that there are likely improvements that could be made to the
metacognitive training utilized, such that immediate differences following training could
be obtained. That is, if manipulation strength had been greater, it is possible that
differences in metacognition immediately following training could be created. However,
the metacognitive training utilized was far more intensive, lengthier, and ‘stronger’ than
other attempts to manipulate metacognition that the author is aware of. This leads to the
second point, which is that it appears very likely that the beneﬁts of metacognitive
training become more distinct over time. Because the metacognitive training
manipulation resulted in differences in slopes of metacognition but no differences in
intercepts at trial ﬁve, time was essential to uncovering the effects. The implications of
this will be discussed further below.

The feedback manipulation did not work at all as expected. Overall trends suggest
that there were slight differences in the slope of self-efﬁcacy by condition towards the
end of the adaptation trials, indicating that perhaps the effects of the feedback
manipulation may become apparent over time. If this were the case, it is possible that
adding additional trials to the experiment would have resulted in signiﬁcant effects.
Regardless, the desired differences in self-efﬁcacy at the end of training were not
obtained. An additional pilot experiment not reported above was run with n= 27 to test
general effects of the feedback manipulation. Although no signiﬁcant results were found,
there was a clear trend in the desired direction for both the intercepts and slopes of self-
efficacy after training. The non-signiﬁcant results were attributed to a small sample size,

and work progressed under the assumption that with a larger sample the desired effects

52

would precipitate. This was unfortunately not the case. As a result, because differences in
intercepts or slopes of self-efﬁcacy were not driven by the manipulation, no differences
in goal level, effort, strategic effort, and adaptive performance by condition were
obtained. Although these differences by condition in goal level, effort, strategic effort,
and adaptive performance were not formally hypothesized, it was expected that they
would exist and in turn enable other formal predictions to be conﬁrmed.

Additional analysis (an ANCOVA) revealed that training performance is
predicted by receiving self-referenced feedback during training when controlling for
other process variables and individual differences (F=3 .49, p=.06). Training performance,
in turn, was a signiﬁcant predictor of several process variables (see Tables 1, 7, and 8).
Training performance was correlated signiﬁcantly with metacognition, self-efficacy, goal
level, effort, strategic effort, and adaptive performance; each of these were positive
correlations, except for a negative correlation with metacognition. It was also a
signiﬁcant predictor of self-efficacy, goal level, effort, strategic effort, and adaptive
performance when averaged over time. Furthermore, training performance signiﬁcantly
predicted the rates of change of all key variables except metacognition and self-efﬁcacy.
Thus, although the speciﬁc results expected for the effect of the feedback manipulation
were not signiﬁcant, self-referenced feedback during training may affect adaptive
processes through its effect on training performance. I did not formally test for mediation,
as the crucial requirements outlined by Baron and Kenny (1986) that the predictor be
correlated with both the mediator and outcome were not satisﬁed. Although I do not posit
reasons or mechanisms for these potential relationships, future research may be fruitful in

this area. A feedback manipulation more directly intended to improve training

53

performance may result in the effects desired in this study (i.e., improved self-regulation
through the theorized linkages described in this study).

Although the declining slopes of metacognition and self-efﬁcacy were counter to
what was hypothesized, several explanations may describe why such ﬁndings occurred.
The most likely explanation for the decreasing scores for both self-efﬁcacy and
metacognition is survey response fatigue. Supporting this supposition is the fact that off-
task thoughts, an additional process measure not included in the analyses, increased over
time (F= 17.3, p< 0.001). Similarly, intrinsic motivation also decreased over time (F=
40.3, p< 0.001 ). One deterrent to performing longitudinal experiments is that they are
often difﬁcult to obtain enough measurement points. Even when care was taken to obtain
numerous measurement points in the present study, it appears that doing so worked
against ﬁnding desired results as participants tired of the task and surveys.

An additional, but unlikely, explanation for the decline in slopes for
metacognition is that engaging in metacognition required perceived effort, which declines
over time as a result of practice (Yeo & Neal, 2008). In the present study, metacognition
did require cognitive effort, and it is possible that engaging in metacognition became
easier over time as a function of practice. The slower rate of decline for those receiving
metacognitive training then would have resulted as individuals strove to sustain their
efforts more than others. Difﬁculties with this explanation are that the metacognitive
measure was not intended as a measure of subjective effort, actual measures of effort rose
over time, and an unreported measure of perceived task effort did not correlate well over

time with metacognition.

54

There was effectively no relationship found between self-efﬁcacy and
performance. Vancouver’s work anticipates a negative relationship (e. g., Vancouver &
Kendall, 2006), while Bandura would predict a positive relationship (Bandura, 1991). My
ﬁndings are out of line with the literature, as both between- and within-person
correlations were effectively zero. Self-efficacy was generally ﬂat in trajectory with a
slightly negative slope, while performance slowly improved. I argued earlier that self-
efﬁcacy may have declined over time as a result of measurement being inﬂuenced by
participant fatigue. The difﬁculty in such an explanation is that performance did not
decline, as would be expected if subjects were genuinely fatigued, but continued steadily
upwards with a plateauing trend during the last few trials. Goal-level was essentially ﬂat.
It is possible that fatigue only inﬂuenced the repetitive process questionnaires, and that
participants found the simulation itself more engaging so as not to evidence a similar
drop.

Averaged across trials, metacognition positively predicts self-efﬁcacy, goal level
positively predicts effort, effort positively predicts adaptive performance, metacognition
positively predicts strategic effort, and strategic effort negatively predicts performance.
The last ﬁnding regarding the negative relationship between strategic effort and
performance was counter to prediction, and on one hand is disheartening given that one
would hope investing effort in being strategic and careful about behavior would lead to
performance gains and a general advantage. A potentially positive upside of this ﬁnding
will be discussed below.

Generally, it was not found that initial levels of key process variables predicted

the rates of change of those variables, as was hypothesized. Although self-efﬁcacy, effort,

55

and adaptive performance had signiﬁcant parameters, the small coefficients and negative
signs are not in line with what was predicted. It is possible that these null results would
be obtained in other future studies. If this were the case, implications for self-regulation
and adaptation theories would be that the rates of change for self-regulatory processes are
independent of initial levels. This would be noteworthy, indicating that gains, or losses,
in these processes over time could happen for people with very different initial levels;
this would have large implications for training Unfortunately, such implications are
unlikely, given that the results obtained were likely due to the fact that there were no
initial differences to begin with (and furthermore, there were few if any differences in
slopes between conditions). Regarding the cases for which there were negative
correlations, the slope for self-efﬁcacy was opposite that of effort and adaptive
performance. These differences prevent any meaningful implications to be made
regarding a general relationship between intercepts and slopes across variables.

One interesting note is that some relationships did not work as expected when
measured over time, while simultaneously the cross-sectional analyses indicated that
signiﬁcant results were present. Additional analyses demonstrated that HLM results for
the effect of metacognitive training on self-efﬁcacy and strategic effort were not
signiﬁcant, although ANCOVA results show that averaged over trials, metacognitive
training predicts self-efﬁcacy and strategic effort (F =4.02; F =6.22, respectively; p < .05).
So although the trajectories of these process variables are not different across conditions,
general relationships do differ. This underscores points made by Ployhart and colleagues
that longitudinal data is critical for understanding processes, given that different answers

are obtained from cross-sectional approaches to the same general questions (Pitariu &

56

Ployhart, 2009; Ployhart & Vandenberg, 2010). Furthermore, these ﬁndings point out that
within-person ﬁndings may be different from between-person ﬁndings. The differences
between measuring at the between-person versus within-person level has been discussed
by others, and has recently been made apparent by the debate between Vancouver and
Bandura. The ﬁndings in the present study add emphasis to the point that theories of self-
regulation and adaptation ought to directly specify which level they are intended to apply.
For cases in which it is useful to examine average tendencies over time, it appears
reasonable that one must also examine the within-person tendencies over time to gain a
better understanding of how the processes are truly behaving.

Contributions

The main purpose of this study was to delineate the roles that regulatory processes
play during transfer and future performance. I intended to meet this aim and extend
research on individual adaptation to environmental change in three ways. First, this study
was intended to extend and develop process-oriented adaptation theory. Second, it was
meant to extend theory that entails multiple regulatory pathways. Third, it was meant to
shed light on how best to prepare individuals to adapt. Although not all these purposes
were fulﬁlled, this study still provides meaningful contributions along all three of these
lines.

This study supports adaptive process theory by demonstrating that if one treats
adaptation as an event captured via cross-sectional measurement, it is not likely that the
underlying phenomena will be accurately described. Also, it gives credence to the
approach of studying cognitive and motivational processes over time, particularly as they

are involved in adaptive processes. Although the design of the scenario unfortunately

57

precluded much necessity to adapt, the changes that occurred to metacognition and
strategic effort as a result of training indicate that the results of efforts to aid adaptation
must be captured over time. The results for H1 indicates that the effect of metacognitive
training was not evident immediately following training, but that its beneﬁt becomes
apparent overtime. The results for H2 and H3 indicate that there was no training effect of
feedback type, even over time. So although the trajectories of these process variables are
not different across conditions, general relationships over time do differ. One important
contribution is that this study demonstrated that regulatory processes do change and relate
to one another over time in signiﬁcant ways. Future research ought to continue to
investigate how regulatory processes change over time. However, care must be taken in
order to ensure that participants are not fatigued by the number of items or measurement
points.

This study also extended theory that relies on multiple regulatory pathways over
time. It was demonstrated that both cognitive and motivational pathways are important
for performance. Furthermore, a relationship between metacognition and self-efﬁcacy
indicates that both pathways are inter-related. Most of the failures in this study were
along the self-efﬁcacy/goals/effort pathway, but meaningful contributions can still be
gleaned. These processes did relate to one another, and the pathway served as an
important antecedent to performance. The results indicate that the trajectories of and
relationships between self-efﬁcacy, goal level, and effort are important to understanding
performance. Future work ought to continue to investigate both cognitive and
motivational pathways of adaptation, even given the limited support obtained here. Basic

relationships were present enough to merit inclusion of the key process variables included

58

in this study in future work. However, as evidenced by the signiﬁcance of some control
variables in the analyses such as intrinsic motivation and goal orientation, other variables
may require further inclusion into formal theories of adaptation.

Despite the failure of the feedback manipulation, meaningful insights can be
gained regarding how to best prepare individuals to adapt. The delayed effect of
metacognitive training on metacognition levels indicates that training does impact how
individuals engage in processes during transfer. Efforts to train individuals on
metacognition may not lead to immediate differences in that speciﬁc process, but that the
beneﬁts may accrue post-training. This is important for both those developing training
regimens and those measuring training effectiveness to remember. The relative weak and
either delayed or ineffective results of both training manipulations indicates that training
interventions for improving self-regulation ought to be pointed, deliberate, and very
strong. My original intent on manipulating self-efﬁcacy in the manner in which I did was
based on theoretical concern between self-referenced and normative feedback types.
Given that there was no effect discovered, it would have been more valuable to have used
another manipulation that would have just created mean differences in end-of-training
self-efﬁcacy. Future research could investigate how best to improve self-efﬁcacy during
training in a way that leads to improvements in efﬁcacy over time during transfer and
performance. Furthermore, future research could fruitﬁrlly investigate how to lead to

larger gains in metacognition over time, given the signiﬁcant results metacognition

demonstrated.

59

Limitations

Aside from the failure of the feedback manipulation discussed above, the next
largest limitation of this study is that the scenario design was such that potentially
meaningful results were made impossible. There was a relatively low number of
‘strategic’ contacts that attacked the outer perimeter of the radar screen and consequently
hurt the participants’ score. The scenario was designed such that metacognitive training
would cause participants to focus on processing these contacts. Although training resulted
in just such behavior, because these strategic contacts were more time intensive to
process, because the penalty for not processing them was relatively low, and because
there were relatively few of them, it turned out that overall performance was best
maximized by not processing the strategic contacts and just processing others. This
design ﬂaw was not a complete oversight, as care was taken to test a number of scenarios,
and performance data was obtained from a number of alternate scenarios through pilot
testing. However, because of time constraints, general trends were taken as workable
guidelines and the design was not reﬁned enough to ensure that performance scores
would align appropriately with the desired behaviors. If more contacts had attacked the
outer perimeter, each carrying a stiffer penalty for breaching the perimeter, it is likely that
the expected relationships between strategic effort and performance would have resulted.

However, the negative relationship between strategic effort and adaptive
performance is instructive in that it points to the cost that strategic effort implicitly
carries. Although scenario design alterations would likely have resulted in a positive
relationship between strategic effort and performance, these ﬁndings demonstrate that

strategic effort includes an opportunity cost. When individuals invested time into ﬁnding

60

and destroying high priority contacts, they gave up their options to process other contacts
that in this case would have been more valuable. The trade-off between exploration and
exploitation is well-known (March, 1991), and the present ﬁndings indicate that such
trade-offs ought to be accounted for in future theory development and measurement
efforts in work on self-regulation and adaptation.

Furthermore, the increased difﬁculty of a better designed scenario would have
likely introduced greater variance between conditions in self-efficacy, goal level, and
effort. Although not hypothesized formally, it was anticipated that performance would
decrease markedly during the beginning of the adaptation phase. In reality, performance
only ever improved. This indicates that the adaptation trials were not as difﬁcult as was
expected, and that participants were not being forced to adapt.

Future Directions and an Alternate Model

Given that the proposed heuristic of adaptation via cognitive and motivational
pathways was not supported as was expected, in this section I revisit the
conceptualization of adaptation and posit an alternate model. Although the task did not
require participants to alter their strategy or reallocate their efforts in order to maintain
performance increases, there were differences in behavior between conditions. I rely on
these ﬁndings and a dynamic process-oriented view of adaptation in order to suggest an
alternate model (Figure 2).

Adaptation consists of the engagement of self-regulatory processes that enable an
individual to detect change, diagnose what ought to be done as a result of that change,
and then to engage in new behavior that aligns with the diagnosis. Although ideally these

processes will enable performance beneﬁts, external forces may prevent such effects.

61

Thus, adaptation should not measured only through performance outcomes. For example,
random error or environmental events may act in such a manner as to prevent
performance gains for those that are engaging in adaptive processes. Alternatively,
random error or environmental events may enable individuals who do not engage in
adaptive processes to experience performance gains that are not, in fact, a direct result of
any adaptive processes engaged in by those individuals. Thus, focusing on performance
outcomes while overlooking actual adaptive behaviors may likely lead to erroneous
conclusions as to whether or not an individual is being ‘adaptive’, and will most likely
result in overlooking the processes that describe what individuals are actually doing.

In the event that there are no immediate performance beneﬁts resulting from
adaptive processes, there still remain suitable explanations for why adaptive processes
ought to be the focus of interest and study. First, it is possible that the underlying
adaptive processes prepare individuals to successfully cope with ﬁrture changes in the
same context, or enable them to generalize the same processes to new contexts which
would be accompanied with performance beneﬁts. Neither of these avenues have been
investigated, and may be fruitful targets for ﬁrture research. Second, social scientists are
interested in self-regulation in its own right; behavior is behavior whether or not there are
performance beneﬁts to it. Thus, even without any performance beneﬁts the regulatory
processes underlying individuals’ behavior are of direct interest.

Recent emphasis has been placed on two distinct adaptability types, that of
transition adaptation and reacquisition adaptation (Lang & Bliese, 2009). These. authors
describe transition adaptation as the reactions that take place that minimize performance

decrements resulting from changes in the task environment that cause previous routines

62

and procedures to become ineffective. Reacquisition adaptation is the process of recovery
that occurs once performance decrements are stopped. This subsequent increase in
performance is a result of “a systematic and analytical learning behavior needed in order
to understand and learn the new challenges of the task” (Lang & Bliese, 2009, p 415).

These authors provided valuable contributions by focusing on how to distinguish
adaptive performance from other aspects of performance, and on providing clearer
methodological and conceptual tools for differentiating between different modes of
adaptation. However, their singular focus on performance outcomes overlooks the
underlying processes that are actually involved in adaptation. They allude to the role such
underlying processes in their description of both transition and reacquisition adaptation;
namely, they indicate that detection and diagnostic processes are at play which include
recognizing that prior strategies are no longer effective, uncovering new strategies
through learning processes, and implementing alternate strategies. Furthermore, they note
that after controlling for cognitive ability, a considerable amount of variance in adaptive
performance remains. It is my contention that the regulatory processes described in this
paper, and detailed further below in my proposed alternate model, likely describe this
variance and ought to be used to deﬁne adaptation. The performance changes described
by Lang and Bliese (2009) may often result from the engagement of adaptive processes,
but such performance changes may not be necessary or sufﬁcient indicators of the
underlying processes.

Central to the rationale of the model proposed previously in this paper were the
issues of exploration and exploitation. The model was intended to capture, to some

degree, the efforts of individuals to explore their task space once changes were detected

63

in order to diagnose the nature of those changes and implement suitable strategies that
exploit resources in a way different from the status quo or pre-adaptive method. In the
current study, metacognition was conceptualized so as to capture both detection and
diagnostic processes. Although metacognitive training focused on asking questions to
both detect and diagnose change, the metacognitive process questions that were utilized
in the task did not explicitly measure the degree to which individuals detect changes.
Strategic effort was measured in such a way as to capture both explorative behaviors (i.e.,
zooms) and the exploitation of resources found and identiﬁed as a result of exploration
(i.e., high-priority contacts). Thus, the measure of strategic effort utilized in the study
confounded those two processes.

In order to more accurately represent and measure the processes of exploration
and exploitation, the alternative model distinguishes explicitly between metacognitive
processes directed towards detection and those directed towards diagnostics. Similarly, it
distinguishes between the exploration of the task domain space as well as the strategic
effort exerted towards utilizing resources in novel ways. These processes will be
described in detail below.

Breaking up exploration (i.e., search behaviors) and strategic effort in the model
emphasizes the point that individuals must both search (e. g., move to new locations,
inspect different options, learn new information, consider alternate solutions) and
experiment with alternative strategies. Exploration refers to the extent to which
individuals engage in search behavior, and strategic effort refers to the extent to which
effort is exerted towards exploiting resources that are discovered as a result of the search

behavior, and is likely either categorically different from the resources previously

64

exploited, or in a different pattern than before. In some cases, it may be difficult to
operationally distinguish between exploration and strategic effort and, depending on the
task, measurement of the two may need to be collapsed. In the current study, exploration
would have been measured by zooming behaviors, while strategic effort would have been

measured by the number of high-priority contacts processed.

65

 

 

 

 

maﬁa—43:8

 

 

gag—coming“ Deanne:

manganese: lv madam? mad: llv

 

 

 

 

 

r KaSoomaao? gem—Sam 1'

 

 

 

 

 

 

 

 

 

 

Figure 2. Dynamic adaptation heuristic

-----------------------’

 

 

 

 

Dom—m lllv muse—3820:” mason

if

 

 

moﬁbmwomow

 

 

 

 

 

 

 

 

mesa =58 Ema? momma/6 sue—Ema

 

 

 

66

In this revised model, strategic effort and exploitative effort are distinct. Exploitative
effort is intended to capture a ‘work harder’ strategy, while strategic effort is intended to
capture a ‘work smarter’ strategy. The dynamics of the model suggest a system in which
individuals exert effort towards exploiting resources in a somewhat mundane way until
performance decrements begin to accrue'. At this point, the performance decrements
would lead to the engagement of detection and diagnostic processes which, if successful,
would lead to the exertion of strategic effort.

A static distinction between strategic effort and exploitative effort is difficult to
make, given that after a time the behaviors used to deﬁne strategic effort may become the
new norm and thus the subject of exploitative effort. If the right environmental dynamics
are in place, it is likely that adaptive processes lead to the adoption of a new strategy,
exempliﬁed initially by strategic effort, which becomes routinized over time. Exploration
may, at different times, require learning or searching one’s repertoire of behaviors
(Kozlowski et al., 1999). Regardless, strategic effort during adaptation can always be
compared to the baseline levels of strategic effort prior to any changes, as well as to
levels of exploitative effort. In the case that strategic effort becomes the new norm, those
particular behaviors would be redeﬁned as exploitative effort, and strategic effort would
be redeﬁned in terms of potential alternate strategies that could be adopted as a result of
further exploration.

This ﬂuid conceptualization of strategic effort and exploitative effort has the
advantage of being able to contribute to a framework that explains the constant

rebalancing and refocusing of effort over time as individuals engage in adaptive

 

1 In the case of adaptive tasks, this performance decrement would be externally driven by changes in the
task and would not be directly attributable to decreases in self-efficacy or goals.

67

processes. Its disadvantage is that making speciﬁc recommendations on criteria for
redeﬁning strategic effort and exploitative effort over time is difﬁcult. One potential
option is that once the behaviors selected to represent strategic effort become the
predominant focus of effort they ought to be redeﬁned in terms of exploitative effort.
This would entail creating a ratio of strategic effort to total effort (i.e., strategic effort
plus exploitative effort) and using a ratio of 0.51 as a cutoff. A simple majority rule may
be too stringent, and a suitable ratio cutoff criterion ought to be made on a case by case
study as a result of both rational analysis and empirical frameworks, also likely informed
by subject matter experts. Alternatively, a simple description of the ratio between
strategic effort and total effort can be used to describe the switching that occurs between
the two pathways, without relying on a cutoff value. On the practical side, it is likely that
most studies of adaptive process in the near future will not need to redeﬁne strategic
effort as a result of time, given that the current paradigms in the literature have yet to
adequately address how adaptive processes are involved in simple changes over a few
time points. In the current study, the task was such that a redeﬁnition would have been
unnecessary.

The switching mechanism that controls the relative balance between exploitative
and explorative behavior is actually two-fold. First, the degree to which changes are
diagnosed and potential solutions identiﬁed is posited to negatively inﬂuence the rate of
exploitative effort and positively inﬂuence the rate of exploration. If changes are
diagnosed and potential solutions identiﬁed, individuals will be more likely to exert
effort exploring their task domain space in search of suitable alternate strategies. If

changes are not diagnosed and no potential solutions present themselves (which would

68

both likely be the result of low detection rates), individuals will be more likely to exert
effort towards the status quo behaviors. It is important to note that both the positive and
negative linkages designated are not intended to be on/off switches. Rather, the
relationship between diagnosis and both exploration and exploitation is one of degree.
That is, when diagnostic metacognitive process levels are relatively high, it is still likely
that a low base rate of exploitative effort is maintained even when more effort is exerted
towards exploration. In other words, while alternate strategies are searched for, some
effort may still be exerted towards the current status quo so as not to experience complete
performance loss.

The second aspect of this switching mechanism relates to the effect of
performance on detection levels. As performance improves, efforts to detect changes will
decrease, and relatively higher levels of exploitative effort will result. Since detection is
a distal predictor of explorative effort, as mentioned above it is expected that if detection
is low, diagnostic processes and explorative effort will subsequently be lower. It follows
that when performance is relatively stable, both exploration and exploitation will occur,
but at a rate that heavily favors exploitation. Also important to note is that although
performance acts as feedback to the system, performance is not required for the
enactment of the adaptive processes, nor is performance necessarily a direct reﬂection of
the extent to which adaptive processes are enacted. For example, low performance is
expected to drive detection, diagnostic, and exploration processes, and these processes
are more likely to continue the lower performance is. Similarly, as performance
improves, self-efficacy is expected to remain high and predict exploitative effort.

Although no speciﬁc exit mechanism is formally included in this model, it is to be

69

understood that persistent and extremely low levels of performance may cause
individuals to quit and exit the system.
Conclusion

This study attempted to demonstrate that training can lead individuals to engage
more in self-regulatory processes that enable adaptation. Unfortunately, the
manipulations did not work as intended, and not all hypotheses were conﬁrmed. Thus, it
is unclear whether training can lead to all the desired impacts on self-regulatory
processes. Furthermore, simulation design ﬂaws made it difficult to ascertain the extent
to which self-regulation enables adaptation. Despite these shortcomings, some results
indicate that self-regulatory variables do relate to performance, and that metacognitive
training impacts metacognition and strategic effort. These ﬁndings, although meager in
terms of what was hoped for, do provide support for future research into self—regulation
and its role in adaptation. Overall, the ﬁndings demonstrate the important of engaging in

such investigations using appropriate longitudinal methods.

70

Appendix A

Table 1. Correlation table, between-individuals

 

 

 

 

 

 

7]

Correlations

M SD 1 2 3 4 5
1 MC Training 0.49 0.5 1
2 Feedback Training 0.49 0.5 0.02 1
3 Metacognition 3.32 0.82 .12* 0.03 1
4 Strategic Effort 11 7.09 .15* 0.01 0.08 1
5 Self-efficacy 3.85 0.63 -0.04 0.02 .41** 0.03 1
6 Goal difficulty 688 290 0.03 -0.04 -0.02 0.01 .15M
7 Effort 12.9 2.95 —0.01 0.01 -0.09 0 0.05
8 Adaptive Perf. _ 1100 293 -0.01 0.01 -0.03 -0.06 0.06
9 Training Perf. 764 251 -0.01 0.06 -0.11* .22" .11* .
10 Learning 60 4.69 0.63 .19** 0.01 .20" 0.09 .20" j
11 Prove G0 4.22 0.76 0.03 -0.01 ‘0.01 -0.08 0.05 ,
12 Avoidance G0 3.44 0.88 -0.03 0 -0.05 -.12* -0.04 _
13 Ability 19.9 5 0.04 -0.03 0.02 0.02 0.06
14 Intrinsic Mot. 155 33 0.08 0.08 .455" -0.02 .35“

** p<.01; * p<.05 ‘

Table l (cont’d).

 

 

6 7 8 9 10 11 12 13 14
1
.46** 1
.476** .85“ 1
.351** .53** , .53** 1
0.06 0.03 0.05 .14** 1
0.02 -0.01 -0.03 -0.06 .26** 1
-0.04 0.07 0.06 -0.07 _-.27** .136** 1 _
0.02 0.07 0.06 0.08 0.07 -0.01 -0.03 1
.16" 0.09 .144** 0.07 0.1 -0.02 -0.03 0.01 1

 

72

 

Table 2. Correlation table, between- and within-individuals

if

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

f , . .. _ . M SD 1 2 3 4 5 6

1 Metacognition; 7 . p 3.32 g 0.82 1 -0.07 0.12 -0.06 -0.16 -0.03

2 Strategic Effort . 11 7.09 0.08 1 0.02 0 0.19 0.14

3 Self-efficacy ‘ 3.85 0.63 0.41" 0.03 1 0.13 0.01 0.06

f, 4. Goal difficulty 688 290 -0.02 0.01 .15" 1 0.114 0.04

5' .. Effort . .. _ 12.9. 2.95 -0.09 0 0.05 .46" 1 0.63

”6' _ Adaptive Perf. 1100 293 -0.03 -0.06 0.06 .48" .85" 1
.." p<.01 . f .. .
‘ Note- italic values are averaged within-person correlations

 

73

 

Table 3. HLM results for H1

 

 

 

 

 

 

 

DV: Trial 5
Metacognition

Source df F
Corrected model 7 1.03
Intercept 1 l .93 * *
Training Performance 1 6.16“
Learning goal
orientation 1 13.73“
Prove goal orientation 1 1.92
Avoidance goal
orientation 1 0.047
Ability 1 0.001
Intrinsic Motivation
(Train) 1 50.23”
Metacognitive Training 1 0.659
Error 355
Total 362
* p< .05

Table 4. HLM results for H2

DV: Trial 5 Self-
Efﬁcacy

Source df . F
Corrected model 7 4.7 **
Intercept 1 34.96M
Training Performance 1 4.34"
Learning goal
orientation 1 6.23“
Prove goal orientation 1 0.002
Avoidance goal
orientation 0.02
Ability 1.81
Intrinsic Motivation
(Train) 1 12.68"
Training Feedback Type 1 0.437
Error 355
Total 362
"‘ p< .05

 

74

 

 

Table 5. Results for H4

DV: Rate of change for Metacognition

 

 

 

Level-1 Predictors Parameter Coefﬁcient S.E. df
Intrinsic Motivation H2 .22“ 0.06 369

Level-2 Predictors
Metacognitive Training 1301 -0.18 0.38 369
Learning goal orientation 1302 1.62“ 0.32 369
Prove goal orientation [303 -0.36 0.27 369
Avoidance goal

orientation 304 0.13 0.2 369
Ability B05 -003 0.25 369
Training Performance 1306 0 0 369
Metacognitive Training 311 0.16” 0.05 374

Table 6. Results for H5

DV: Rate of change for Self-Efﬁcacy

Level-l Predictors Parameter Coefficient S.E. df
Intrinsic Motivation ”2 0.01 0.01 369

Level-2 Predictors

Training Feedback Type 501 -0.18 0.09 369

Learning goal orientation 1302 0.36“ 0.1 369
Prove goal orientation [303 0.01 0.08 369
Avoidance goal

orientation 1304 0 0.08 369
Ability 1305 0.02 0.01 369
Training Performance [306 0 0 369
Training Feedback Type [31 1 0.03 0.02 374

"p<.01

75

Table 7. Results for H6

 

 

At 2 2
Predictor/ Step Step Final R AR
DV: Self-Efﬁcacy
1 Training Performance 0.06 0.12
Learning goal orientation .16** 0.09
Prove goal orientation 0.02 0.04
Avoidance goal
orientation 0.02 0.01
Ability 0.04 0.04
Intrinsic Motivation .33“ 0.19“ 0.16"
2 Metacognition .32M 0.23" 0.07"
DV: Goal Level
1 Training Performance .32** .32"
Learning goal orientation -0.05 -0.07
Prove goal orientation 0.07 0.07
Avoidance goal
orientation -0.04 -0.04
Ability 0 0
Intrinsic Motivation .13** 0.1 .125"
2 Self-Efficacy 0.09 0.132 0.01
DV: Effort
1 Training Performance 0.55“ .47"
Learning goal orientation -0.08 -0.07
Prove goal orientation 0.07 0.05
Avoidance goal
orientation 0.09 .1 * "‘
Ability 0.04 0.04
Intrinsic Motivation 0.05 0.02 0.3"
2 Goal Level .24" 0.36" 0.06“
DV: Adaptive Performance
1 Training Performance .52” .12**
Learning goal orientation -0.04 0.01
Prove goal orientation -0.02 -0.02
Avoidance goal
orientation 0.09 0.03
Ability 0.02 0
Intrinsic Motivation 0.11 0.07 0.29”
2 Effort 0.75" 0.7" .41“

76

Table 7 (cont’d).

 

 

2 2
Predictor/ Step At Step Final R AR
DV: Strategic Effort
Training Performance .17 ** .2"
Learning goal orientation 0.04 0.02
Prove goal orientation -0.07 -0.06
Avoidance goal
orientation -0.1 0]
Ability 0 -0.01
Intrinsic Motivation -0.04 -0.1 .05"
Metacognition .14“ 0.07" .02"
DV: Adaptive Performance
Training Performance 0.52" .56"
Learning goal orientation -0.04 -0.03
Prove goal orientation 0.02 0.01
Avoidance goal
orientation 0.09 0.07
Ability 0.02 0.02
Intrinsic Motivation .1 IM .1* .29"
2 Strategic Effort -.225"'* .34" .05"
** p<.01; * p<.05
Supplementary
DV: Strategic Effort
1 Training Performance .17 ** .18”
Learning goal orientation 0.04 0.02
Prove goal orientation -0.07 -0.06
Avoidance goal orientation -0.1 -0.1
Ability -0.02 -0.01
Intrinsic Motivation -0.04 -0.1 .05“
2 Metacognitive Training .13" 0.07" .02M

77

Table 8. Results for H6
DV: Rate of change for Self-

 

 

 

 

Efficacy

Level-1 Predictors Parameter Coefﬁcient S.E. df
Intrinsic Motivation 112 0.02 0.01 369
Rate of Change for

Metacognition [310 0.04" 0 375

Level-2 Predictors 369
Learning goal orientation I301 0.39M 0.1 369
Prove goal orientation I302 0.01 0.09 369
Avoidance goal orientation 1303 0.005 0.07 369
Ability 1304 0.02 0.01 369
Training Performance 1305 0 0 369

DV: Rate of change for Goal Level

Level-l Predictors Parameter Coefﬁcient S.E. df
Intrinsic Motivation 112 -3.04 1.96 369
Rate of Change for Self-

Efficacy 1310 13.6** 6.38 375

Level-2 Predictors 369
Learning goal orientation 1301 -5.6 23.9 369
Prove goal orientation 502 21.38 18.6 369
Avoidance goal orientation [503 -10.5 15.8 369
Ability 1304 0.64 0.01 369
Training Performance 305 0.11" 0.02 369

 

78

Table 8 (cont’d).

DV: Rate of change for Effort
Level-1 Predictors

Intrinsic Motivation

Rate of Change for Goal Level
Level-2 Predictors

Learning goal orientation
Prove goal orientation
Avoidance goal orientation
Ability

Training Performance

DV: Rate of change for Adaptive
Performance

Level-1 Predictors
Intrinsic Motivation

Rate of Change for Effort
Level-2 Predictors

Learning goal orientation
Prove goal orientation
Avoidance goal orientation
Ability

Training Performance

 

 

 

 

Parameter Coefﬁcient S.E. df
II2 -0.08** 0.02 369
B10 0.002" 0 375
369
B01 015 0.19 369
B02 0.12 0.15 369
B03 0.24 0.14 369
B04 -002 0.03 369
B05 0.002" 0 369
Parameter Coefﬁcient S.E. (If
“2 5.01" 2.2 369
B10 73.7M 2.3 375
369
B01 -5.03 19.8 369
B02 0.02 15.3 369
B03 23.3 13.97 369
B04 -3.63 2.45 369
B05 0.17“ 0.01 369

79

Table 8 (cont’d).

DV: Rate of change for Strategic Effort

 

 

Level-1 Predictors Parameter Coefficient S.E. df
Intrinsic Motivation 112 -0.13** 0.06 369
Rate of Change for Metacognition [310 -0.13 0.08 375
Level-2 Predictors 369
Learning goal orientation 1301 0.66 0.52 369
Prove goal orientation I302 -0.57 0.53 369
Avoidance goal orientation 1303 -0.49 0.32 369
Ability B04 005 0.07 369
Training Performance 305 0.001" 0 369

DV: Rate of change for Adaptive

 

 

Performance

Level-l Predictors Parameter Coefﬁcient S.E. df
Intrinsic Motivation “2 0.23 2.1 369
Rate of Change for Strategic Effort 1310 7.4" 1.24 375

Level-2 Predictors 369
Learning goal orientation [301 1.97 19.9 369
Prove goal orientation 1302 -2.45 15.3 369
Avoidance goal orientation 503 22.7 13.8 369
Ability B04 -4.22 2.3 369
Training Performance B05 0.16" 0.01 369

80

Table 9

 

DV Coefficient S.E.
Metacognition -0.005 0.01
Strategic Effort -0.013 0.02
Self-efficacy -.047"‘* 0.01
Goal difficulty 0.003 0.002
Effort -0.034** 0.01
Adaptive
Performance -.07** 0.01

 

 

 

81

 

 

Figure 10. Normative score values

Normative Score Values

,. 1.0%-” 574
2.0%. p 591
30% 648
40%: V6.76.
.5096 . 765‘
50% ..... 828.
70%. ., 387
80% 77982
90% 1112

82

Table 11. Equations

Equation 1.1 for H4:

Y= 110 + 111(Trial) + 112(Intrinsic Motivation) + Eij

110= [300 + [301(Metacognitive Training) + [302(Learning G0) + Bo3(Prove G0) +
[304(Avoidance G0) + 1305(Ability) + [306(Training Performance) + COi

111= [310 + [31 1(Metacognitive Training) + Q1;

H2= B20
Equation 1.2 for H5:

Yij= I10 + 111(Trial) + 112(Intrinsic Motivation) + Sij

H01= B00 + B01(Training Feedback Type) + B02(Leaming GO) + B03(Prove GO) +
[304(Avoidance G0) + 005(Ability) + [306(Training Performance) + Cm

IT 1 i: [310 + [31 1(Training Feedback Type) + £11

H21= B20

83

Table 1 l (cont’d).

Equation 1.3 for a generic H7:

Yijz 110 + I11([Process Predictor]) + 112(Intrin8ic Motivation) + 8ij

110i= [300 + [301(Leaming G0) + [302(Prove G0) + [303(Avoidance GO) +
[304(Ability) + [305(Training Performance) + Cm

nli= B10 + £11

H2i= B20

84

Figure 3. Metacognition

 

 

condition

~—

#9319—

 

 

 

 

trials

 

 

 

85

Figure 4. Self-efﬁcacy

Estimated Marginal Means of self_efﬁcacy

 

8.2

"i

i

7.60-

Estirmted Marginal Means

ii

7.20”

 

 

SR_feedback

— Normative
— Self-referenced

 

 

N—

‘3...

trials

86

Figure 5. Goal level

Estimated Marginal Means of goa|_level

 

i

i

Estimated Marginal Means
i i

 

 

 

 

trials

87

condition

—' 1.00
— 2.00
‘ 3.00
—- 4.00

Figure 6. Effort

Estimated Marginal Means of effort

 

16.00‘

14.00-

12.00-

10.00-

Estimated Marginal Means

g

6.00-

 

 

 

 

”—1

trials

88

condition
*- 1.00
-- 2.00
.. 3.00
-— 4.00

Figure 7. Strategic effort

Estimated Marginal Means of strateglc_effort

 

condition
-—1.00
15.00- __2_00
73.00
a —4.00
1:
a
c 12.50-
E
E
.E
c»
E 10.00-
E
'u
i
=- 7.50-
a
111
5.00-

 

 

 

 

trials

89

Figure 8. Performance

Estimated Marginal Means of performance

 

 

1200.0 condition
——1.00
--—2.00
, 3.00
a —4.00
5 1000.00-
0
E
To
.5
9
w 800.00-
E
1:
i
'6' 600.00-
11.1
400.00-

 

 

 

llllllllllll
123456789101112

trials

90

Metacognitive Training Questions (answered after each training trial)
Please briefly respond to the following questions.
What do I need to learn about how to engage contacts?

Are target speeds related to scoring? If so, how?

What is hurting my score the most? How can I overcome that?

How can I ﬁnd out which contacts are most important to process?

How could I better use more of the game’s functions (e.g., zoom, manual, queries)?

What should I study in the manual to improve my skill and/or strategy?

How can I effectively use the feedback I receive to improve my performance?

How could I be more systematic about improving my performance?

91

Metacognition Cheat Sheet

Metacognition is the knowledge and control you have over your own thoughts. It
includes the awareness you have of how you learn and what you know. When you are
using metacognition, you are thinking about the processes of your thoughts, what you
know, and what you need to do be a better learner.

Metacognition has been shown to increase awareness, comprehension, self
conﬁdence, and the use of effective strategies in tasks like the one you are currently
participating in. Science has shown that is in your best interest to utilize the
metacognitive techniques you have been taught not just during this task (which is
important) but really for anything else you do in life.

Ask yourself questions about what you know
What can I do to more effectively engage contacts?
Are contact speeds related to scoring?
What is hurting my score the most? How can I overcome that?
Ask yourself questions about how you could better strategize and do things better
How can I ﬁnd out which contacts are most important to process?
How could I better use more of the game’s functions (e.g., zoom, manual,
queries)?
Think about your planning efforts and how you could improve them
What should I study in the manual to improve my skill and/or strategy?
How can I effectively use the feedback I receive to improve my
performance?
How could I be more systematic about improving my performance?

REMEMBER: Learning to ask these questions will help your performance in the game
after training is complete and you are faced with new challenges.

92

Appendix B

Goal Orientation Learning:
For each of the following statements, please indicate how true it is for you.
l) I am willing to take on challenged that I can learn a lot from.
2) I often look for opportunities to develop new skills and knowledge.
3) I enjoy challenging and difficult activities where I’ll learn new skills.
4) For me, development of my abilities is important enough to take risks.

Goal Orientation Prove:
For each of the following statements, please indicate how true it is for you.
I) I prefer to do things that require a high level of ability and talent.
2) I’m concerned with showing that I can perform better than my peers.
3) I try to figure out what it takes to prove my ability to others.
4) I enjoy it when others are aware of how well I am doing.
5) I prefer to participate in things where I can prove my ability to others.

Goal Orientation Avoidance:

For each of the following statements, please indicate how true it is for you.

1) I would avoid taking on a new task if there was a chance that I would appear rather
incompetent to others.

2) Avoiding a show of low ability is more important to me than learning a new skill.

3) I’m concerned about taking on a task if my performance would reveal that I had
low ability.

4) I prefer to avoid situations where I might perform poorly.

Ability:
1) What is your college GPA?
2) What was your ACT/SAT score?

Intrinsic Motivation:

For each of the following statements, please indicate how true it is for you.

1) I enjoyed doing this activity very much

2) This activity was fun to do.

3) I thought this was a boring activity.

4) This activity did not hold my attention at all.

5) I would describe this activity as very interesting.

6) I thought this activity was quite enjoyable.

7) While I was doing this activity, I was thinking about how much I enjoyed it.
Metacognition:

Please indicate the extent to which you agree with each of the following
statements regarding your thoughts and behaviors DURING THE LAST TRIAL ONLY.

1) I engaged in self-questioning techniques.

2) I asked myself questions about what I know.

3) I asked myself questions about how I can better strategize.

4) I asked myself questions about how I can improve my planning.

5) I tried to implement new strategies.

93

Self-efﬁcacy:

This set of questions asks you to describe how you feel about your capabilities for
performing on the next trial of this simulation.

1) I am conﬁdent in my ability to meet the challenges of this simulation.

2) I am certain that I can manage the requirements of this task.
Goal Level:

1) How many points do you plan to score during the next trial?

94

References

Ammons, R. B. (1956). Effects of knowledge of performance: A survey and tentative
theoretical formulation. Journal of General Psychology, 54, 279-299.

Ashford, J. (1986). F eedback-seeking in individual adaptation: A resource. Academy of
Management Journal, 29, 465-487.

Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory.
Englewood Cliffs, NJ: Prentice Hall.

Bandura, A. (1991). Social cognitive theory of self-regulation. Organizational Behavior
and Human Decision Processes, 50, 248—287.

Bandura, A. (1997). Self-eﬂicacy: The exercise of control. New York: Freeman.

Bandura, A., & Cervone, D. (1983). Self-evaluative and self-efﬁcacy mechanisms
governing the motivational effects of goal systems. Journal of Personality and
Social Psychology, 45, 1017-1028.

Bandura, A., & Cervone, D. (1986). Differential engagement of selfreactive inﬂuences in
cognitive motivation. Organizational Behavior and Human Decision Processes,
38, 92—113.

Bandura, A. & Jourden, F.J. (1991). Self-regulatory mechanisms governing the impact of
social comparison on complex decision making. Journal of Personality and
Social Psychology, 60, 941-951.

Baumeister, R. F, Hutton, D. G., & Cairns, K. J. (1990). Negative effects of praise on
skilled performance. Basic and Applied Social Psychology, 11, 131-148.

Bell, B.S. & Kozlowski, S.W.J. (2008). Active learning: Effects of core training design

elements on self-regulatory processes, learning, and adaptability. Journal of
Applied Psychology, 93, 296-316.

Berardi-Coletta, B., Dominowski, R.L., Buyer, L.S., & Rellinger, ER. (1995).
Metacognition and problem solving: A process-oriented approach. Journal of
Experimental Psychologz: Learning, Memory, and Cognition, 21, 205-223.

Bogart, DH. (1980). Feedback, feedforward, andfeedwithin: strategic information in
systems. Behavioral Science, 25, 237-249.

Brown, A. L., Bransford, J. D., Ferrara, R. A., & Campione, J. C. (1983). Learning,

remembering, and understanding. In J. H. Flavell & E. M. Markman (Eds),
Handbook of child psychology (Vol. 3, pp. 77—166). New York: Wiley.

95

Brown, S.P., Leigh, T.W. (1996). A new look at psychological climate and its
relationships to job involvement, effort, and performance. Journal of Applied
Psychology, 81, 358-368.

Burke, L.A., Hutchins, HM. (2007). Training transfer: An integrative literature review.
Human Resource Development Review, 6, 263-296.

Burke, C. S., Stag], K. C., Salas, E., Pierce, L., & Kendall, D. (2006). Understanding
team adaptation: A conceptual analysis and model. Journal of Applied
Psychology, 91, 1189-1207.

Campbell, J. P., McCon, R. A., Oppler, S. H., & Sager, C. E. (1993). A theory of
performance. In N. Schmitt & W. C. Borman (Eds), Personnel selection in
organizations (pp. 35-70). San Francisco: Jossey-Bass.

Carver, C. S., & Scheier, M. F. (1990). Origins and functions of positive and negative
affect: A control-process view. Psychological Review, 97,19-35.

Chen, G., & Mathieu, J .E. (2008) Goal orientation dispositions and performance
trajectories: The roles of supplementary and complementary situational

inducements. Organizational Behavior and Human Decision Processes, 106, 21-
38.

Chen, G., Thomas, B., & Wallace, J.C. (2005). A multilevel examination of the
relationships among training outcomes, mediating regulatory processes, and
adaptive performance. Journal of Applied Psychology, 90, 827-841.

DeShon, R.P. & Alexander, RA. (1996). Goal setting effects on implicit and explicit
learning of complex tasks. Organizational behavior and human decision
processes, 65, 18-36.

DeShon, R. P., Kozlowski, S. W. J ., Schmidt, A. M., Milner, K. R., & Weichmann, D.
(2004). A multiple goal, multilevel model of feedback effects on the regulation of

individual and team performance in training. Journal of Applied Psychology, 89,
103 5—1056.

Barley, P.C., Connolly, T. & Ekegren, G. (1989). Goals, strategy development, and task
performance: some limits on the efﬁcacy of goal setting. Journal of Applied
Psychology, 74, 24-33.

Eisenhardt, K.M., & Tabrizi, EN. (1995). Accelerating adaptive processes: Product

innovation in the global computer industry. Administrative Science Quarterly, 40,
84-110.

Fisher, S.L., & Ford, J .K. (1998). Differential effects of learner effort and goal
orientation on two learning outcomes. Personnel Psychology, 51, 397-420.

96

F lavell, J .H. (1979). Metacognition and cognitive monitoring: A new area of
cognitivedevelopmental inquiry. American Psychologist, 34, 906- 911.

Ford, J. K., Smith, E. M., Weissbein, D. A., Gully, S. M., & Salas, E. (1998).
Relationships of goal orientation, metacognitive activity, and practice strategies
with learning outcomes and transfer. Journal of Applied Psychology, 83, 218—233.

Frey, M. C., & Detterman, D. K. (2004). Scholastic assessment or g? The relationship
between the Scholastic Assessment Test and general cognitive ability.
Psychological Science, 15, 373—378.

Gick, M. L., & Holyoak, K J. (1987). The cognitive basis of knowledge transfer. In S.M.
Cormier & J. D. Hagman (Eds.), Transfer of training: Contemporary research
and applications (pp. 9-46). New York: Academic Press.

Gist, M. E., & Mitchell, T. R. (1992). Self-efficacy: A theoretical analysis of its
determinants and malleability. Academy of Management Review, 17, 183-21 1.

Goldberg, L. R. (1999). A broad-bandwidth, public-domain, personality inventory
measuring the lower-level facets of several ﬁve-factor models. In I. Mervielde, I.
Deary, F. De Fruyt, & F. Ostendorf (Eds.), Personality Psychology in Europe:
Vol. 7. (pp. 7-28). Tilburg, the NetherlandszTilburg University Press.

Gully, S. M., Payne, S. C., Koles, K. L. K., & Whiteman, J. K. (2002). The impact of
error training and individual differences on training outcomes: An attribute—
treatrnent interaction perspective. Journal of Applied Psychology, 87, 143—155.

Guzzo, R.A., Jette, R.D., Katzell, RA. (1985). The effects of psychologically based

intervention programs on worker productivity: A meta-analysis. Personnel
Psychology, 38, 275-291.

llgen, D. R., Fisher, C. D., & Taylor, M. S. (1979). Consequences of individual feedback
on behavior in organization. Journal of Applied Psychology, 64, 349-371.

Ivancic, K., & Hesketh, B. (2000). Learning from error in a driving simulation: Effects on
driving skill and self-conﬁdence. Ergonomics, 43,1966—1984.

J undt, D. (2008). Adaptive performance: Self-regulatory predictors and the development
of a process perspective. Doctoral dissertation proposal.

Karoly, P. (1993). Mechanisms of self-regulation: A systems view. Annual Review of
Psychology, 44, 23—52.

97

Keith, N., & Frese, M. (2005). Self—regulation in error management training: Emotion
control and metacognition as mediators of performance effects. Journal of
Applied Psychology, 90, 677—691.

Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on
performance: A historical review, a meta-analysis, and a preliminary feedback
intervention theory. Psychological Bulletin, 119, 254—284.

Kluwe RH. (1987). Executive decisions and regulation of problem solving behavior. In
Weinert FE, Kluwe RH (Eds.), Metacognition, motivation, and understanding
(pp. 31-64). Hillsdale, NJ: Erlbaum.

Kozlowski, S. W. J ., Gully, S. M., Brown, K. G., Salas, E., Smith, E. M., & Nason, E. R.
(2001). Effects of training goals and goal orientation traits on multidimensional
training outcomes and performance adaptability. Organizational Behavior and
Human Decision Processes, 85,1-31.

Kraiger, K., Ford, J. K., & Salas, E. (1993). Application of cognitive, skill-based, and
affective theories of learning outcomes to new methods of training evaluation.
Journal of Applied Psychology, 78, 311—328.

Latham, G. P., & Locke, E. A. (1991). Self-regulation through goal setting.
Organizational Behavior and Human Decision Processes, 50, 212-247.

LePine, J. A., Colquitt, J. A., & Erez, A. (2000). Adaptability to changing task contexts:
Effects of general cognitive ability, conscientiousness, and openness to
experience. Personnel Psychology, 53, 563—593.

Locke, E. A.,Chah, D., Harrison, 8., & Lustgarten, N. (1989). Separating the effects of
goal speciﬁcity from goal level. Organizational Behavior and Human
Performance, 43, 270-287.

Locke, E. A., & Latham, G. P. (1990). A theory of goal setting and task performance.
Englewood Cliffs, NJ: Prentice Hall.

Locke, E. & Latham, G. P. (2002). Building a practically useful theory of goal setting and
task motivation: A 35-year odyssey. American Psychologist, 5 7, 705-717.

March, J .G. (1991). Exploration and exploitation in organizational learning.
Organizational Science, 2, 71-87.

Mikulincer, M., Glaubman, H., Ben-Artzi, E., & Grossman, S. (1991). The cognitive

speciﬁcity of learned helplessness and depression deﬁcits: The role of self-
focused cognitions. Anxiety Research, 3, 273-290.

98

Pitariu, A.H. & Ployhart, RE. (2009). Explaining change: Theorizing and testing
dynamic mediated longitudinal relationships. Journal of Management, 36, 405-
429.

Ployhart, R.E. & Vandenberg, R. (2010). Longitudinal research: The theory, design, and
analysis of change. Journal of Management, 36, 94-120.

Pulakos, E. D., Arad, 8., Donovan, M. A., & Plamondon, K. E. (2000). Adaptability in
the workplace: Development of a taxonomy of adaptive performance. Journal of
Applied Psychology, 85, 612—624.

Pulakos, E. D., Schmitt, N., Dorsey, D. W., Arad, 8., Hedge, J. W., & Borman, W. C.
(2002). Predicting adaptive performance: Further tests of a model of adaptability.
Human Performance, 125, 299—3 23.

Schmidt, A. M., & Ford, J. K. (2003). Learning within a learner control training
environment: The interactive effects of goal orientation and metacognitive
instruction on learning outcomes. Personnel Psychology, 5 6, 405—429.

Schmidt, R. A., & Bjork, R. A. (1992). New conceptualizations of practice: Common

principles in three paradigms suggest new concepts for training. Psychological
Science, 3, 207—217.

Schraw, G. & Moshman, D. (1995). Metacognitive theories. Educational Psychology
Review, 7, 351-371.

Shih, S. & Alexander, J .M. (2000). Interacting effects of goal setting and self- or other-
referenced feedback on children’s development of self-efﬁcacy and cognitive skill

within the Taiwanese classroom. Journal of Educational Psychology, 92, 536-
543.

Short, E.S., Ryan EB. (1984). Metacognitive differences between skilled and less skilled
readers: Remediating deﬁcits through story grammar and attribution training.
Journal of Educational Psychology, 76,225-235.

Smith, E. M., Ford, J. K., & Kozlowski, S. W. J. (1997). Building adaptive expertise:
Implications for training design strategies. In M. A. Quinones & A. Ehrenstein
(Eds.), T raining for a rapidly changing workplacesApplications of psychological
research (pp. 89—118). Washington, DC: American Psychological Association.

Terborg, J .R., Miller, H.E. Motivation, behavior, and performance: A closer examination

of goal setting and monetary incentives. Journal of Applied Psychology, 63, 29-
39.

Tolli, A.P. & Schmidt, A.M. (2008). The role of feedback, causal attributions, and self-
efﬁcacy in goal revision, Journal of Applied Psychology, 93, 692- 701.

99

Tsui, A.S. & Ashford, SJ. (1994). Self-regulation: A process view of managerial
effectiveness. Journal of Management, 20, 93-121.

Vancouver, J. B., & Kendall, L. (2006). When self-efﬁcacy negatively relates to

motivation and performance in a learning context. Journal of Applied Psychology,
91, 1146—1153.

Vancouver, J. B., Thompson, C. M., Tischner, E. C., & Putka, D. J. (2002). Two studies
examining the negative effect of self-efficacy on performance. Journal of Applied
Psychology, 8 7, 506—516.

Vancouver, J. B., Thompson, C. M., & Williams, A. A. (2001). The changing signs in the
relationships among self-efﬁcacy, personal goals, and performance. Journal of
Applied Psychology, 86, 605—620.

Vancouver, J. B., & Tischner, E. C. (2004). The effect of feedback sign on task
performance depends on self-concept discrepancies. Journal of Applied
Psychology, 89, 1092—1098.

VandeWalle, D. (1997). Development and validation of a work domain goal orientation
instrument. Educational and Psychological Measurement, 5 7, 995—1015.

Verbruggen, M., Sels, L. (2008). Can career self-directedness be improved through
counseling? Journal of Vocational Behavior, 73, 318-327.

Weissbein, (1992). Development and validation of a work domain goal orientation
instrument. Educational and Psychological Measurement, 5 7, 995—1015.

Wicklund, R. A. (1975). Objective self-awareness. In L. Berkowitz (Ed.), Advances in

experimental social psychology (Vol. 8, pp. 233-237). New York: Academic
Press.

Yeo, G., & Neal, A. (2004). A multilevel analysis of the relationship between effort and
performance: Effects of ability, Conscientiousness and goal orientation. Journal
of Applied Psychology, 89(2), 231—247.

Yeo, G., & Neal, A. (2008). Subjective cognitive effort: A model of states, traits, and
time. Journal of Applied Psychology, 93, 617-631.

100