.13....
1:? v.1 .

in.“ n......?:

C
43!. u.

. .I' ‘ ,
. . . It: ‘ , .
a“? g. u.
.uw....‘.4§. , . fir
.3 “5%“ . .5. .; .. a
9535‘ . .r 3‘.
... i. V
. 6r \ . 1V3 1 .
(1..“ bk \ .
irks. 4?
. f? has n
"n :K.

9.1%

. u .
t

. ..

.. Lama: .2; . a

ring... ‘(a .YJHRLV

«L‘.3.$Wuufii

. x.
.22.

a. V
$.nh.m..%tmr In...“
hurt“... . aw...“
i. i
.1. km
x
a
95.12”“:

“M. ,v
5.32.?
It.
.1:

1.

r.
’ X
2
t.

a!
g

 

' "3‘5

\

 

x‘i“\ \
‘Q ‘
I)
W

\ ,\
. ._,

LIBRARY
Michi: :2 State
University

 

 

 

This is to certify that the
dissertation entitled

ADAPTABILITY FROM A PROCESS PERSPECTIVE:
EXAMINING THE EFFECTS OF TASK CHANGE TYPE
AND A METACOGNITIVE INTERVENTION ON
ADAPTIVE PERFORMANCE

presented by

DUSTIN K. JUNDT

has been accepted towards fulfillment
of the requirements for the

Ph. D. degree in PsychologL

 

 

 

Major ProfessorWnature
”171W Zap?

Date

MSU is an Affirmative Action/Equal Opportunity Employer

 

PLACE IN RETURN Box to remove this checkout from your record.
TO AVOID FINES return on or before date due.
MAY BE RECALLED with earlier due date if requested.

 

DATE DUE DATE DUE DATE DUE

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

5/08 K‘lProlecc&Pres/ClRC/DateDue Indd

 

ADAPTABILITY FROM A PROCESS PERSPECTIVE:
EXAMINING THE EFFECTS OF TASK CHANGE TYPE AND A METACOGNITIVE
INTERVENTION ON ADAPTIVE PERFORMANCE
By

Dustin K. Jundt

A DISSERTATION

Submitted to
Michigan State University
in partial fulfillment of the requirements
for the degree of

DOCTOR OF PHILOSOPHY
Psychology

2009

ABSTRACT
ADAPTABILITY F ROM A PROCESS PERSPECTIVE:
EXAMINING THE EFFECTS OF TASK CHANGE TYPE AND A METACOGNITIVE
INTERVENTION ON ADAPTIVE PERFORMANCE
By
Dustin K. J undt

While past work on adaptability and adaptive performance has developed and
matured primarily over the last two decades, a number of gaps exist in the literature.
Namely, existing work generally takes a static approach to examining adaptability and
fails to consider potentially important cognitive and behavioral processes that can
influence adaptation over time. Furthermore, this work pays little attention to the
different ways in which a task can change, which may have implications for how and
how well people are able to adapt. Finally, little work has focused on in situ self-
regulatory processes that may influence how well people are able to adapt to various
types of task change. To address these gaps, the current study introduces and builds from
a process model of adaptation by which it is hypothesized that in order to exhibit high
levels of adaptive performance effectiveness, people must detect changes in the task they
are facing, diagnose the nature of those changes, and finally develop and enact
appropriate task strategies. Furthermore, and based on Wood’s (1986) taxonomy of task
complexity types, this study suggests three different ways in which tasks can change and
examines the processes by which people are able to adapt to each of them. In addition, a
metacognitive intervention was designed that encouraged participants to devise plans,

monitor their performance effectiveness, and evaluate the effectiveness of their strategies

in order to investigate the impact of these in situ self-regulatory processes on adaptive

performance. Finally, a number of hypotheses were investigated regarding the potential
impact of individual differences factors related to cognitive ability, goal orientation, and
personality on adaptive performance effectiveness in relation to the different types of task
change. Using a laboratory-based radar simulation task, the findings provide support for
the efficacy of thinking about adaptability as a process rather than simply as performance
at a given point in time. Specifically, evidence suggests that change detection, relevant
knowledge acquisition (e. g. diagnosis), and appropriate strategy change were related to
higher levels of adaptive performance effectiveness, although these results were not

entirely consistent across the three types of task change that were investigated.

ACKNOWLEDGMENTS

I would like to thank Dr. Steve Kozlowski for all of the time, effort, and guidance
that he offered as my dissertation advisor. Thank you Steve for your ideas and
suggestions, for helping to provide clarity on many of the issues I faced while working on
this project, and for the resources and opportunities that you provided. I am especially
thankful for your patience. Also, I would like to thank Dr. Dan 11 gen, Dr. Kevin Ford,
and Dr. Remus Ilies for serving on my committee and for providing me with helpfiil

suggestions and future considerations regarding this work.

iv

TABLE OF CONTENTS

LIST OF TABLES ............................................................................................................. vii
LIST OF FIGURES ............................................................................................................ ix
INTRODUCTION ............................................................................................................... 1
A REVIEW OF ADAPTIVE PERFORMANCE ................................................................ 6
THE ADAPTATION PROCESS ....................................................................................... 18
Detection ................................................................................................................ 24
Diagnosis ............................................................................................................... 27
Adaptatron ................................. 3O
ADAPTABILITY TO DIFFERENT TYPES OF TASK CHANGE ................................. 33
METACOGNITION AND ADAPTIVE PERFORMANCE ............................................. 48
INDIVIDUAL DIFFERENCES AND ADAPTIVE PERFORMANCE ........................... 64
METHOD .......................................................................................................................... 74
Participants ............................................................................................................ 74
Design .................................................................................................................... 74

Task ........................................................................................................................ 74
Procedure ............................................................................................................... 75
Manipulations ........................................................................................................ 77

Task Change Type ...................................................................................... 77
Metacognition Intervention ........................................................................ 80

Measures ................................................................................................................ 81
Control Variables ........................................................................................ 81

Cognitive Ability ........................................................................................ 81

Learning Orientation .................................................................................. 81
Conscientiousness and Openness to Experience ........................................ 81

Detection .................................................................................................... 82

Declarative Task Knowledge Development .............................................. 82

Appropriate Strategy Use ........................................................................... 83

Adaptive Performance Effectiveness ......................................................... 84
Metacognition Intervention Manipulation Check ...................................... 84

RESULTS .......................................................................................................................... 85
Control Variables .................................................................................................... 85
Analysis Plan .......................................................................................................... 85

Hypothesis 1 ........................................................................................................... 87

Hypothesis 2 ........................................................................................................... 89 ‘
Hypothesis 3 ........................................................................................................... 90
Hypothesis 4 ........................................................................................................... 91
Hypothesis 5 ........................................................................................................... 92
Hypothesis 6 ........................................................................................................... 93
Hypothesis 7 ........................................................................................................... 94
Hypothesis 8 ........................................................................................................... 94
Hypothesis 9 ........................................................................................................... 95
DISCUSSION .................................................................................................................... 96
Findings and Specific Limitations .......................................................................... 98
Hypotheses 1, 2, and 3 ............................................................................... 99
Hypotheses 4 and 5 .................................................................................. 105
Hypotheses 6, 7, 8, and 9 ......................................................................... 108
Contributions ......................................................................................................... 1 12
General Limitations ............................................................................................... 114
Sample ...................................................................................................... 1 14
Research Setting ....................................................................................... 1 16
Statistical Power ....................................................................................... l 17
Consideration of time ............................................................................... 118
Future Research Directions ................................................................................... 119
APPENDICES
Appendix A: Definitions of Individual Adaptation and Adaptive Performance..125
Appendix B: Consent Form .................................................................................. 126
Appendix C: Debriefing Form .............................................................................. 129
Appendix D: Metacognitive Intervention Text ..................................................... 130
Appendix E: Metacognitive Intervention Probes .................................................. 131
Appendix F: Learning Orientation Scale Items .................................................... 132
Appendix G: Conscientiousness Scale Items ........................................................ 133
Appendix H: Openness to Experience Scale Items ............................................... 135
Appendix I: Declarative Knowledge Quiz Items .................................................. 137
Appendix J: Metacognitive Intervention Manipulation Check Items ................... 140
Appendix K: Tables .............................................................................................. 141
Appendix L: Figures ............................................................................................. 157
REFERENCES ................................................................................................................ 162

LIST OF TABLES

Table 1: Means, Standard Deviations, Intercorrclations, and Reliabilities for Key
Variables Across All Conditions (n = 283) ..................................................................... 141

Table 2: Means, Standard Deviations, Intercorrclations, and Reliabilities for Key
Variables in the Difficulty Task Change Condition (n = 95) .......................................... 142

Table 3: Means, Standard Deviations, Intercorrclations, and Reliabilities for Key
Variables in the Component Task Change Condition (11 = 93) ........................................ 143

Table 4: Means, Standard Deviations, Intercorrclations, and Reliabilities for Key
Variables in the Coordinative Task Change Condition (n = 95) ..................................... 144

Table 5: Interactive Effects of Task Effort and Task Change Type on Adaptive
Performance Effectiveness (n = 283) ............................................................................... 145

Table 6: Effects of the Use of New Information Cues on Component Change Adaptive
Performance Effectiveness (n = 93) ................................................................................. 145

Table 7: Effects of Behavioral Act Prioritization on Coordinative Change Adaptive
Performance Effectiveness (n = 95) ................................................................................. 146

Table 8: Effects of Relevant Declarative Knowledge Development on Difficulty Change
Task Effort (n = 95) ........................................................................................................ 146

Table 9: Effects of Relevant Declarative Knowledge Development on Component
Change Correct Use of New Information Cues (n = 93) ................................................. 147

Table 10: Effects of Relevant Declarative Knowledge Development on Coordinative
Change Behavioral Act Prioritization (n = 95) ...................................... . ......................... 147

Table 11: Effects of Change Detection on Relevant Declarative Knowledge Development
for Difficulty Change (11 = 95) ......................................................................................... 148

Table 12: Effects of Change Detection on Relevant Declarative Knowledge Development
for Component Change (n = 93) ...................................................................................... 148

Table 13: Effects of Change Detection on Relevant Declarative Knowledge Development
for Coordinative Change (n = 95) .................................................................................... 148

Table 14: Interactive Effects of the Metacognitive Intervention and Task Change Type on
Adaptive Performance Effectiveness (n = 283) ............................................................... 149

Vii

Table 15: Effects of the Metacognitive Intervention on Change Detection for Difficulty
Task Change (11 = 95) ....................................................................................................... 149

Table 16: Effects of the Metacognitive Intervention on Relevant Declarative Knowledge
Development for Difficulty Task Change (n = 95) ......................................................... 150

Table 17: Effects of the Metacognitive Intervention on Difficulty Task Change Effort
(11 = 95) ............................................................................................................................. 150

Table 18: Effects of the Metacognitive Intervention on Change Detection for Component
Task Change (n = 93) ....................................................................................................... 150

Table 19: Effects of the Metacognitive Intervention on Relevant Declarative Knowledge
Development for Component Task Change (11 = 93) ....................................................... 151

Table 20: Effects of the Metacognitive Intervention on Component Change Correct Use
of New Information Cues (n = 93) ................................................................................... 151

Table 21: Effects of the Metacognitive Intervention on Change Detection for
Coordinative Task Change (11 = 95) ................................................................................. 151

Table 22: Effects of the Metacognitive Intervention on Relevant Declarative Knowledge
Development for Coordinative Task Change (n = 95) .................................................... 152

Table 23: Effects of the Metacognitive Intervention on Coordinative Change Behavioral
Act Prioritization (n = 95) ................................................................................................ 152

Table 24: Interactive Effects of Cognitive Ability and Task Change Type on Adaptive
Performance Effectiveness (n = 283) ............................................................................... 153

Table 25: Interactive Effects of Learning Goal Orientation and Task Change Type on
Adaptive Performance Effectiveness (n = 283) ............................................................... 154

Table 26: Interactive Effects of Conscientiousness and Task Change Type on Adaptive
Performance Effectiveness (n = 283) ............................................................................... 155

Table 27: Interactive Effects of Openness to Experience and Task Change Type on
Adaptive Performance Effectiveness (n = 283) ............................................................... 156

viii

LIST OF FIGURES

Figure 1: A Control Theory-Based Heuristic of Self-Regulation .................................... 157
Figure 2: A Control Theory-Based Heuristic of Adaptation ........................................... 158
Figure 3: A General Model of the Adaptive Performance Process ................................. 159
Figure 4: A Task Change Specific Model of the Adaptive Performance Process ........... 160

Figure 5: The Interactive Effect of Task Related Effort and Task Change Type on Levels
of Adaptive Performance Effectiveness ........................................................................... 161

ix

Introduction

Jobs in current organizations are no longer characterized by static task
requirements and assembly line-like work. Rather, workers in many organizations today
possess jobs that are characterized by dynamic, ever-changing work environments (1] gen
& Pulakos, 1999; Smith, F 0rd, & Kozlowski, 1997). In fact, change is considered to be
omnipresent in organizations (e. g. Cascio, 2003), and it can come from a number of
sources. Changes in technology are rapid and pervasive in modern work environments.
Computers are everywhere, and the plethora of new software programs, versions, and
systems as well as the availability of the intemet for constant and immediate access to
data requires continuous adaptation on the part of employees (Hesketh & Neal, 1999;
Ployhart & Bliese, 2006). Mergers and restructuring are also common in organizations,
and along with them comes the requirement that employees learn new Skills for their
changed job or adopt new procedures to fit in with new ways of doing things (Pulakos,
Arad, Donovan, & Plamondon, 2000). Furthermore, the Shift in the US. to a primarily
information or knowledge based economy brings with it the demand for employees who
can continuously update their skills and expertise, as well people who can work with
other experts on project-based teams (Hesketh & Neal, 1999).

These changes have lead to a shift in what is considered to be important for job
performance. Organizational scholars and practitioners are increasingly coming to the
conclusion that employees with the ability to adapt to changes in their jobs are valuable
assets and may even be necessities. Indeed, adaptive performance has been suggested to
be a component of many different types of current jobs (Pulakos et a1, 2000) in addition

to traditional aspects of job performance (Campbell, 1990; Campbell, McCloy, Oppler, &

Sager, 1993) and citizenship or contextual performance (Borman & Motowidlo, 1993;
1997)

This increased focus on the importance of adaptability for employees is reflected
by a number of relatively recent studies in Industrial/Organizational psychology that
focus on examining or predicting adaptive performance. Recognizing the demand for
flexible and adaptive employees, one set of studies has taken a selection perspective and
focused on individual differences that are directly tied to, or in some cases proposed to be
the root of, adaptability (e. g. Cronshaw & J ethmalani, 2005; LePine, Colquitt, & Erez,
2000; Ployhart & Bliese, 2006). Another set of studies and theoretical contributions have
come from a training perspective, and they have primarily focused on identifying and/or
testing training techniques and methods that build critical task knowledge and skills as
well as adaptive skill or expertise (e. g. Bell & Kozlowski, 2002; 2008; Ford et al., 1998;
Ivancic & Hesketh, 2000; Keith & Frese, 2005; Kozlowski et al., 2001; Smith, Ford, &
Kozlowski, 1997).

While work on adaptive performance in the past 15 to 20 years has come along
way and is certainly impressive, the current state of the literature leaves a number of
avenues unexplored. First of all, the current work has focused primarily on selecting and
training for adaptive performance, but it has primarily treated the construct as a static
operationalization of performance after a task change occurs. Little attention has been
given to what people have to go through to adapt to a change in the nature of a learned
task. This lack of in situ consideration poses a problem when trying to understand the

nature of adaptive performance. Without knowing or thinking about what people are

doing to adapt, it is difficult to understand how or why individual differences, training
techniques, or the like should influence adaptive performance.

A second area that remains fairly unexplored pertains to different types of tasks
that people may have to adapt to. Primarily, current operationalizations of adaptive
performance have focused on performance on tasks that are novel, ill-defined, more
difficult, and/or more complex than previous tasks. While these tasks certainly require
adaptability, there is no existing framework for understanding or thinking about how
tasks can change. While Pulakos et a1. (2000; 2002) offer an empirically derived
taxonomy of different types of adaptive performance, they do not provide a discussion of
different components of these tasks that can change.

A third area of research on adaptive performance that has been given little
consideration deals with understanding the importance of self-regulatory processes
occurring during the process of adaptation. While recent research by Chen, Thomas, and
Wallace (2005) takes a step in this direction by examining self-regulatory processes that
are thought to translate existing task knowledge, skill, and efficacy into adaptive
performance, there is room for further exploration, especially when considering the
potential effects of regulatory interventions.

With these open areas of inquiry in mind, the first focus of this paper is to build a
conceptual model of adaptation as a process. This model is based on thinking about
adaptability to a changing task from a process perspective as a series of steps or activities
that one needs to go through. It is grounded in control theory models of self-regulated

behavior (e. g. Carver & Scheier, 2001; Klein, 1989) and attempts to highlight the

importance of detecting task changes, diagnosing the nature of those changes, and
devising and selecting strategies to adapt to those changes.

This model then serves as the foundation for a discussion and explication of
different types of task change and how people can adapt to adapt to them. Using Wood’s
(1986) taxonomy of task components, three distinct types of task change are proposed
that are suggested to cut across different types of adaptive performance. Using the
conceptual process model of adaptation as a guide, a series of hypotheses are developed
regarding the importance of different motivational and cognitive factors in predicting
adaptive performance effectiveness in response to these different types of task change.

The third part of this paper focuses on the potential importance of in situ self-
regulatory activities, primarily in the form of metacogrrition, for influencing adaptability
to different types of task change. Metacognition can be thought of as the regulation of
how well one is making plans, monitoring performance effectiveness, and evaluating the
success of one’s plans or strategies during task performance (Schraw & Moshman, 1995),
or in this case adaptive performance. While previous work has shown the importance of
metacognition as a predictor of learning, problem solving, and even adaptive
performance, potentially viable metacognitive interventions, especially those aimed at
increasing metacognition during actual adaptation, have received little attention. The
third section of this paper discusses and presents hypotheses regarding the potential
effects of a metacognitive intervention on adaptive performance effectiveness in response
to different types of task change.

Finally, a number of individual difference predictors of adaptive performance are

discussed in relation to their effects on predicting adaptive performance. This section

focuses on individual differences factors related to cognitive ability, personality, and goal
orientation. While these factors have been shown to be related to adaptive performance
in prior studies, this section uses the conceptual process model of adaptation as a guide
for developing hypotheses aimed at assessing potential differential influences of these
individual differences on adaptive performance in response to different types of task

change.

A Review of Adaptive Performance

While the I/O Psychology literature on adaptability and adaptive performance is
relatively young and certainly still emerging, it is characterized by different perspectives
and definitions of adaptability. Appendix A lists numerous definitions of individual
adaptability or adaptive performance that have been used in the I/O literature to discuss
the same general phenomenon. Reflective of this, Ployhart and Bliese (2006) identified
four streams of research that they saw as broadly related to current definitions of
adaptability: (1) adaptability as task performance, (2) adaptability as changes to cognitive
processing, (3) adaptability as coping, and (4) adaptability as responding to
organizational change. For the purpose of this study, I will primarily frame things in
terms of the most traditional [/0 perspective of adaptability as task performance. While
the research in each of these streams certainly has some relevance and conceptual
overlap, for the purpose of this study I believe that they are best thought of as
supplements and, in fact, some of this research will serve quite prominently in that
manner.

Within the [/0 literature the research can be further broken down into two
approaches: (1) adaptability fi'om an individual difference and selection perspective and
(2) adaptability from a training and outcome perspective. Relevant conceptual and
empirical work from these two perspectives will now be presented.

The individual difference or selection approach to adaptability primarily views
adaptability as a set of personal characteristics or skills that people posses which allow

them to perform successfully when a learned task changes. In this approach, adaptability

essentially resides within the person and, thus, people can be differentiated because of
their varying levels of inherent adaptability.

For example, Cronshaw and colleagues (Fine & Cronshaw, 1999; Cronshaw &
Jethmalani, 2005) discuss adaptability as “competencies that enable people to manage
themselves in relation to the demands of conformity and/or change in particular
situations” (Fine & Cronshaw, 1999, pg 40). Cronshaw and J ethmalani (2005) categorize
these skills as being (1) agentic or accommodative in directionality, (2) internally or
externally focused, and (3) activated in relation to things, data, or people. These skills
are thought to develop throughout one’s life and career and are integrated and manifested
as values, attitudes, and ways of reacting to physical, social, and/or environmental
changes. Cronshaw and colleagues suggest that people with more developed
competencies manage these changes better, and thus they exhibit more successful
adaptation.

Coming from a similar perspective, Ployhart & Bliese (2006, pg 13) define
adaptability as “an individuals' ability, skill, disposition, willingness, and/or motivation,
to change or fit different task, social, and environmental features”. Based on this
definition, they argue that adaptability resides completely within a given individual and is
driven solely by individual differences. Thus, similar to Cronshaw & Jethmalini’s (2005)
idea of agentic vs. accommodative skills, Ployhart & Bliese (2006) suggest that
adaptability does not occur only in response to a change in the given environment or task;
it can be both reactive and proactive.

Ployhart & Bliese (2006) also make a distinction between adaptability and

adaptive performance, suggesting that work from the training/outcome perspective (e. g.

Ford et al, 1998; Kozlowski et al, 2001) tends to focus on adaptive performance, but
individual adaptability is better conceptualized a multidimensional composite set of
knowledge, skills, and other abilities that form a higher-order adaptability factor. This
composite factor is thought to be influenced by sub-facet personality, cognitive ability,
physical ability, and value or interest factors, which are proposed to be invariant across
different situations, contexts, and environments (Ployhart & Bliese, 2006).

LePine, Colquitt, and Erez (2000) investigated the influence of ability and
personality factors on adaptive performance from a more general selection perspective in
which they operationalized adaptive performance as decision-making accuracy after
changes in proper decision making rules occurred. Using a decision-making simulation,
LePine et a1. examined the impact of general cognitive ability, conscientiousness, and
openness to experience on decision-making accuracy during an initial block of trials and
during two subsequent blocks of trials where decision-making rules changed. They
found that cognitive ability and openness to experience both became better positive
predictors of decision-making accuracy when people had to adapt to changes in the
simulation. Contrary to their hypotheses, however, they found conscientiousness was
increasingly negatively related to decision-making accuracy when people had to adapt to
changes in the simulation.

The training and outcome perspective primarily views adaptability as
performance effectiveness on a novel task. Adaptive performance here is often
considered to be a function or outcome of various types of training conditions and
factors. The outcomes of these training conditions generally focus on cognitive (e. g.

declarative and procedural knowledge, knowledge structure), behavioral (e. g. training

performance or skill), and/or affective (e. g. self-efficacy) factors, which stern fi'om the
Kraiger, Ford, & Salas (1993) taxonomy of training outcomes. These factors, in turn, are
thought to influence one’s ability to adapt to changes in the nature of the trained task. It
is important to note that individual differences such as cognitive ability, personality, and
goal orientation are often examined as predictors in these types of studies. Adaptability,
however, is Viewed as the individual’s effectiveness or rated/measured performance, not
the combination of these individual difference factors that people posses.

Srrritlr et al. (1997) provided an initial spur in thinking about adaptability from a
training perspective by suggesting that prior work in training in [/0 psychology had
focused primarily on training for “routine” rather than “adaptive” expertise. Smith et al.
suggested that simply being able to execute work-related tasks (routine expertise) is not
enough in turbulent work environments, rather, people need a more complete
understanding of the requirements and structure of work tasks in order to be able to
successfully adapt to changing task demands. As seen in Appendix A, Smith et al. (pp.
93) suggest “Adaptability or adaptive expertise is evidenced when the individual
responds successfully to changes in the nature of the trained task.”

In order to develop this type of adaptive expertise, Smith et al. (1997) proposed
that individuals must first develop detailed knowledge of the task domain. This involves
gaining well-developed procedural knowledge structures containing a wide range of
potential problems and related solutions that are tightly linked together. This knowledge,
however, may not be enough, as individuals must also have metacognitive skills related

to planning, monitoring, and evaluating their behavior. These skills are suggested to

allow individuals tO recognize change and successfully transfer relevant previous
knowledge to acquire new knowledge about novel tasks.

In concordance with some of the propositions made by Smith et al. (1997),
Kozlowski et al. (2001) examined the impact of mastery VS. performance training goals
and trait goal orientations (e. g. Dweck, 1986; Nicholls, 1984) on adaptive performance.
Utilizing a radar-tracking decision making simulation, adaptive performance was
operationalized as performance on a transfer trial that was more dynamic, difficult, and
complex than previous training trials. Kozlowski et al.’S results suggested that mastery
training goals were related to more complex knowledge structures and higher levels of
self-efficacy, which were both related to higher levels of adaptive task performance. In
addition, they found that both initial training performance and task declarative knowledge
levels were positively related to adaptive performance. Finally, they found that trait
learning orientation had a positive influence on adaptive performance through self-
efficacy.

Similarly, and based in part on a review of the training outcomes literature
conducted by Kraiger et al. (1993), Ford et al. (1998) examined the impact of knowledge,
skills, and affect on training performance and adaptive performance/training transfer in a
learner-controlled environment. Similar to Kozlowski et a1. (2001), Ford et al.’s
transfer/adaptability measure was operationalized as performance on a decision-making
task that was more difficult and complex than previous trials. Their results suggested that
declarative task knowledge levels, training performance, and self-efficacy were all related

to adaptive performance. Furthermore, these learning outcomes mediated various effects

10

of metacogrrition, task strategy practice levels, identical elements learning strategy use,
and goal orientations on transfer performance.

Bell & Kozlowski (2002) examined the influence of adaptive guidance training
techniques on adaptive performance, which they also operationalized as performance on a
radar-tracking Simulation scenario that was longer, more difficult, and more complex
than their given training scenarios. Unlike many feedback interventions or training
interventions, adaptive guidance training provides participants with progressively tailored
information about what areas trainees should focus and improve on as they engage in
future task iterations, first focusing on basic task knowledge and skills and later moving
toward more complex and strategic knowledge and skills. Bell & Kozlowski found that
adaptive guidance was related to more appropriate study and practice strategies, more on
task cognition, and higher self-efficacy. In turn, these factors influenced task knowledge
and skill, which ultimately positively predicted adaptive performance.

A number of studies focused on error-based training (cf. Frese & Altmann, 1989)
have also shown effects on adaptive performance. Error training encourages learners to
actively acquire information and skill by exploring the task, testing hypotheses, and using
trial-and-error methods. In a pair of studies utilizing a driving simulation, Ivancic &
Hesketh (2000) showed that those who received error training, as opposed to those
receiving training that discouraged making errors, were better able to adapt their skills
and perform on an adaptive transfer test that required a different strategy than was used in
training to avoid a collision. Keith & Frese (2005) found similar results using an
overhead slide creation training task, in which participants had to perform a more

difficult and complex slide creation task. Furthermore, Keith & Frese found that self-

11

regulatory activities related to emotion control (e.g. Kanfer & Ackerman, 1989; Kanfer,
Ackerman, & Heggestad, 1996) and metacognition (e. g. Flavell, 1979; Nelson, 1996)
mediated the effects of error training on adaptive transfer.

A series of works by Kozlowski and colleagues further discussed the importance
and development of adaptive performance with the primary focus being on developing
adaptive teams. Kozlowski, Gully, Salas, and Cannon-Bowers (1996a); Kozlowski,
Gully, McHugh, Salas, and Cannon-Bowers (1996b); Kozlowski (1998); and Kozlowski,
Gully, Nason, and Smith (1999) offer the perspective that team adaptability and
coordination are functions of team coherence (i.e. shared affect, cognition, and behaviors
in the team). While these works focus on the team level and, thus, are beyond the scope
of this review, it is important to note that Kozlowski and colleagues (1996a, 1996b)
explicitly discuss the importance of individual task and skill training and development
within dynamic, repeating cycles of task intensity over time and how this development is
- an important step in the development of adaptive teams (Kozlowski et al, 1999).

While the studies described above focus primarily on predicting adaptive
performance as an event, other work has focused more on defining different types of
adaptive performance that may be required on the job. Coming fi'om something of a
performance rating perspective, Pulakos et al. (2000) focused on building a taxonomy of
different types of adaptive performance at the individual level. Pulakos et al. worked
from a definition that adaptive performance constituted “situations in which individuals
modified their behavior to meet the demands of a new situation or event or a changed
environment”(p. 615). This view of adaptation is clearly situation-driven, and based on

an initial review of relevant literature and a subsequent analysis of a number of critical

l2

incidents that described occasions of actual adaptive performance on a job, Pulakos et al.
put forth eight dimensions of adaptive performance: (1) solving problems creatively, (2)
Dealing with uncertain or unpredictable work situations, (3) learning new tasks,
technologies, and procedures, (4) demonstrating interpersonal adaptability, (5)
demonstrating cultural adaptability , (6) demonstrating physically oriented adaptability ,
(7) handling work stress, and (8) handling emergency situations.

In a second study, Pulakos et al. (2000) empirically evaluated their eight-
dimension taxonomy of adaptive performance by constructing a performance-rating
instrument, the Job Adaptability Inventory (J AI), and adrrrinistering it to more than 3,000
employees in a number of different occupations. Both an exploratory factor analysis and
a subsequent confirmatory factor analysis provided evidence supporting the eight-
dimension model. Pulakos et al. (2000) also provided evidence that different occupations
had varying levels of adaptive performance requirements and suggested that not all
dimensions of adaptive performance should be relevant for all jobs.

In a follow up, Pulakos et al. (2002) further explored the structure of adaptive
performance behaviors. Confirmatory factor analyses in this study also supported the
eight-dimension taxonomy of adaptive performance using predictor measures of adaptive
performance based on past experience, interest, and self-efficacy. However, both
exploratory and confirmatory factor analyses yielded difficult to interpret solutions for
measures of adaptive performance completed via supervisor ratings, thus suggesting that
raters had difficulty separating the dimensions when rating adaptive performance.

While the existing literature reviewed here on adaptive performance identifies a

number of important predictors and generally Shows good support for the notion of

13

adaptive performance as a relevant construct (e. g. Pulakos et al, 2000), it is not without
issues.

First of all, there is considerable conceptual disagreement regarding what
adaptability or adaptive performance actually is. Researchers coming fiom the individual
differences perspective such as Ployhart & Bliese (2006) and Cronshaw & Jethmalani
(2005) argue that adaptability is something that resides within the individual, and
therefore it is the set of personality factors, values, cognitive abilities, and the like that
truly define adaptability. On the other hand, work coming from the training perspective
such as Smith et al. (1997) argues that adaptability is essentially task performance or
effectiveness that is evidenced when people perform well and generalize their skills to a
related, but novel task. In this perspective, essential cognitive, behavioral, and affective
KSAOs are perceived as being trainable, and it is the training itself that is essential as
adaptive performance flows from it. It is important here to highlight the fact that even
though these approaches are different, they do not completely contradict one another. In
both cases, KSAOs are seen as being important in determining adaptive performance.
However, their centrality to the notion of adaptability is debated.

Second, little attention has been paid to the types of task changes that people are
adapting to. Empirical work from both the individual difference/selection and
outcome/training perspectives on adaptation has been primarily done using an
operationalization of adaptive performance that focuses on effectiveness on a novel, more
difficult, and/or more complex version of a task that people are already familiar with.
While this approach is certainly reasonable in terms of operationalizing adaptive

performance as an outcome, it does little to help us understand if and how factors such as

14

training techniques, task knowledge, self-efficacy, personality, cognitive ability, and
others predict adaptability to different types of task changes. Initial conceptual work
done in this domain by Pulakos and colleague (Pulakos et al 2000; 2002) and Ployhart &
Bliese (2006) certainly provides some needed push in this direction, but to date useful
empirical work has not followed suit.

Third, and possibly most important, there has been little work focused on
understanding what people are actually doing to adapt to changes in their task
environment. If one takes the most common perspective of adaptability as consisting of,
or evidenced by, performance on a novel, more difficult, and/or more complex version of
a learned task, it is hard to ignore the parallels with models of traditional job
performance. Specifically, the work by Campbell and colleagues (Campbell, 1990;
Campbell et al. 1993) is relevant here as they make a sharp distinction between
performance, which is the actual behavior that is under one’s control, and effectiveness or
productivity, which is the evaluation or results of performance and may be influenced by
numerous other factors (for example, the availability of necessary resources). Behavior,
and thus performance, is seen as being ongoing, dynamic, and continuous. The current
work on adaptability tends to ignore this in search of individual differences or training
techniques that ultimately influence what Campbell and colleagues refer to as
effectiveness. While a number of studies have examined self-regulatory processes
related to behavior dming training or skill acquisition (Bell & Kozlowski, 2002; Ford et
al., 1998; Kozlowski et al., 2001), these have generally not focused specifically on
looking at in situ factors that may influence actual behavior during the time period where

people are attempting to adapt to a task change.

15

There are, however, a few exceptions. Although they strongly advocate an
individual differences approach to adaptability, Ployhart & Bliese (2006) do provide an
initial, albeit very general, discussion of how these individual differences may influence
people’s ability to identify situational cues that denote the need to change, select
appropriate strategies, and monitor and regulate their behavior when trying to adapt.
Furthermore, a multilevel empirical study conducted by Chen et al. (2005) examined
adaptive performance as transfer performance on a novel version of an attack helicopter
flight simulation scenario that was more dynamic and complex than the practice/training
scenarios. Chen et al proposed homologous models at the individual and team levels that
focused on task knowledge, efficacy, and task skill as predictors of performance through
self-regulatory activities related to goal choice and goal striving (e. g. Kanfer, 1990;
Mitchell & Daniels, 2003) during adaptation. At the individual level, their results
suggested that within-episode goal striving processes mediated the positive relationship
between between-episode goal choice activities and adaptive performance. Furthermore,
Chen et al. found that task-related role knowledge and individual task skill were directly
related to adaptive performance.

While this work is certainly a step in the right direction, it is my assertion that
further attention needs to given to each of these issues. Specifically, a model that views
adaptation or adaptive performance as a process, rather than just an operationalization or
outcome, can help to shed some light on these issues. In the next section, I will propose
just such a conceptual model with the intention of using it as a guide to firrther explore

issues related to in situ cognitive and behavioral self-regulatory processes occurring

l6

during adaptation as well as how the process of adapting to different types of task change

may differ.

17

The Adaptation Process
Viewing adaptability as a process essentially entails thinking about it as a

multitude of dynamic behaviors and cognitions that are engaged in with the purpose of
adapting to a change in one’s task environment. While existing definitions of
adaptability vary somewhat in focus and scope (see Appendix A), they all include this
element of success in responding to a task change. However, as suggested earlier, many
of these definitions focus on adaptability as residing within the individual or on
adaptability as generalization of trained skills. To think of adaptability as a process, a
different type of definition needs to be used, one that focuses on behavior as well as
effectiveness. This is consistent with the aforementioned View of Campbell and
colleagues (e. g. Campbell, 1990; Campbell et al. 1993) that performance is the actual
behavior that people engage in, while effectiveness is the outcome of that behavior.
From this perspective, the object of adaptability (i.e. adaptive performance) is to perform
effectively (i.e. improve, maintain, or limit decrements) in the new task environment,
while adaptability itself can be thought of as the behaviors and cognitions that people
engage in toward this end and the process of determining what these behaviors will be.
Given this, I will use the following definition of adaptability as the foundation for this
study.

Adaptability, or adaptive performance, is the process of using existing

knowledge and Skill to appropriately and functionally align behavior with task

and/or environment changes in a way that improves, maintains, or limits

decrements in performance effectiveness.

18

In my opinion, this type of dynamic approach to adaptability provides a
substantially more appropriate framework for thinking about how people actually adapt
to task changes, as opposed to previous work that focuses primarily on what factors
predict performance effectiveness in changing situations. In addition, this type of view
further opens the door for existing research on job performance and behavior to be
incorporated and integrated into theories of adaptive performance.

One such area of research that may be especially useful is related to self-
regulation of behavior. Indeed, Chen et al. (2005) provide some initial evidence that in
situ self-regulatory activities can be beneficial for adaptation. While encouraging, the
work by Chen et al. can best be seen as an initial, broad attempt at showing the
importance of self-regulatory factors as essential parts of an adaptability process. The
purpose of this section is to further this line of thinking and provide a way of tying
adaptation as a process more strongly to existing models of self-regulated behavior.

While a number of views and definitions of self-regulation exist (cf. Kanfer,
1990; Vancouver, 2000), one definition that is especially relevant here is provided by
Karoly (1993), who suggests that self-regulation can be thought of as processes “that
enable an individual to guide his/her goal-directed activities over time and across
changing circumstances” (p. 25). One can easily see the parallel that this perspective has
with the View of adaptability described above, as both focus on activities or behaviors
over time across changing circumstances or task demands. Furthermore, and consistent
with the definition of adaptability offered above, the idea of engaging in behaviors in
order to perform at a certain level or standard is considered to be possibly the most

ubiquitous and central element of theories of self-regulation (Kanfer, 1990; Vancouver,

19

2000). However, the self-regulation literature has not generally dealt with how people
respond to task or environment driven changes in performance-related goals or standards
over time, and thus, has not gained a great deal of traction as a conceptual foundation in
the adaptive performance literature.

One perspective on self-regulation that may be especially helpful for thinking
about adaptability is that based on control theory. Control theory encompasses a number
of related perspectives that tie their roots to mechanical control systems and cybemetics
in engineering (e. g. Weiner, 1948), and it was brought into psychology in early iterations
by works by Miller, Galanter, and Pribram (1960) and Powers (1973). While a number
of more modern variations of control theory exist (Hyland, 1988; Klein, 1989; Lord &
Levy, 1994; Carver & Scheier, 1981, 1982, 2001), they share many of the same common
features stemming from the initial conceptualizations.

A key feature that serves as the foundation for nearly every one of these models is
the negative feedback loop (Klein, 1991; Vancouver, 2000). The negative feedback loop
dictates that during goal-directed behavior, people are focused on reducing a discrepancy
between their current state and the goal or standard that they are trying to achieve. In
general, there are three common types of reactions that are considered to happen in
response to goal-performance discrepancies (Carver & Scheier, 1981; Kanfer, 1990).
The first would be retaining the same goal and increasing one’s effort or changing one’s
strategy, the second would be to lower one’s standard and maintain the level of effort
exerted or keep the same strategy, and the final would be to abandon the goal altogether,
thus ending the self-regulatory cycle. For example, if an employee had a goal to produce

one hundred widgets in an hour, but was only currently producing eighty widgets per

20

hour, the discrepancy would be twenty widgets. The person could work harder or change
their strategy to produce more widgets, lower the goal to 80 widgets and work just as
hard, or simply quit.

More specifically, control theory-based models of self-regulation suggest that
there are a series of specific steps one goes through in an attempt to identify and reduce
these goal-performance discrepancies. Figure 1 shows a general control theory-based
heuristic of self-regulated behavior. In this type of model, behavior is aimed at reaching
a goal or desired end-state, which is thought to serve as a driver of behavior (Kanfer,
1990; Locke, 1968; Locke & Latharn, 1990; Vancouver, 2000). As mentioned earlier, a
comparison function is used to determine the discrepancy between one’s current state or
level of performance and the goal or desired state. This discrepancy, then, drives action
aimed at reducing it. In the case of a discrepancy, plans are developed and behavior
consistent with these plans is enacted. This behavior, along with environmental factors
and constraints, results in one’s actual level of effectiveness. Subsequently, performance
feedback, which can come from a variety of sources, serves as the input that allows for
the next comparison between current state and desired goal or standard to be made, thus
re-initiating the cycle.

While control theory models of motivated behavior have received support, they
are not without their detractors (cf. Klein, 1991; Locke, 1991; Vancouver, 2000).
However, the use of control theory here is meant as a heuristic to help describe motivated
behavior, and as such is merely a surface level description of control theory models of
behavior. This discussion is not intended to fully represent more sophisticated and well-

developed control theory models aimed at predicting work motivation and behavior

21

(Klein, 1989) or motivated behavior in general (Carver & Scheier, 1981, 1982, 2001), but
rather to use the foundation of control theory as a platform for thinking about adaptive
performance in a dynamic, self-regulated manner and organizing measurement and
hypothesis generation.

AS suggested earlier, the type of dynamic cognitive and behavioral perspective
characterized by control theory models is consistent with a process approach to
adaptability. From this type of View, adaptation is not just performance effectiveness at a .
given point in time; it is an ongoing process of assessing the situation and examining the
appropriateness of one’s actions and levels of performance against a referent standard or
goal. However, one issue with relying on control theory in its heuristic form as described
above is that control theory considers these goal-directed regulatory activities as
something of a closed system, where it is often assumed (at least implicitly) that one’s
standard is fairly stable left on its own and that the methods or strategies for reaching that
standard, once discovered, should continue to remain viable. I believe that thinking about
adaptation in this manner demands more of an “open system” perspective, where the
situation or task demands can change, thus rendering previously appropriate strategies or
levels of effort insufficient. How then, can control theory be used as a referent for
thinking about adaptability? It is my assertion that an additional component must be
added that allows for goals or standards and the strategies necessary to achieve them to
change over time and, thus, incorporates or merges the process perspective of adaptability
into control theory as an overarching framework.

Figure 2 shows an attempt to highlight how adaptability can be thought of from a

process perspective, using control theory as a foundation. This heuristic is constructed

22

with the intention of providing a provision for goals or targets to change over time, as is
the case in situations that require adaptability. While much of the control theory heuristic
as described above remains intact, one critical component is added here. The notion of
source determination is based on the idea that when goal-performance discrepancies
exist, people will attempt to develop explanations and attributions regarding the source of
that discrepancy. Some work based in control theory has dealt with this notion of
attributions (Carver & Scheier, 1981; Klein, 1989), but they have primarily focused on
attributions aimed at factors related to ability, effort, luck, and task difficulty, in line with
the work by Weiner and colleagues (W einer, 1985: Weiner, Russell, & Lerman, 1979).

In regard to adaptability, another type of attribution would need to be considered, one that
attributes the cause of the discrepancy not to inadequacies in one’s existing ability levels,
effort levels, or strategies, but rather to an actual change that has occurred in the nature of
the task.

A task change is Often novel, and cues that the task is changing can come fi'om a
variety of different situations. For example, Louis and Sutton (1991) suggest that cues
for change can come from situations that are perceived or experienced as being novel or
unusual, unexpected discrepancies or failures, or deliberate requests calling for new
strategies or procedures. It is my belief that in order to be able to deal with these cues
and changes, people need to go through a series of processes in order to determine the
source of the task change and what the appropriate behaviors are to adapt to it.

These steps involve first detecting that a change in the task enviromnent has
occurred. Following that, a person must gather appropriate information in order to

diagnose the nature of the change in the task. Finally, the person must develop

23

appropriate strategies and act in accordance with them in order to perform well on the
task. These steps and their associated outcomes are represented in the process model
shown in Figure 3 and will be discussed in greater detail in the following sections.
Detection

When a discrepancy occurs on a task, people first need to notice the discrepancy
and then they need to determine its source. If that discrepancy results from a change in
the task environment, I assert that people need to be able to recognize that it is the task
itself that is the cause of the change, not their own failings, in order to successfully adapt.
Therefore, the first stage in the adaptation process is detection of a given change.

Detection or recognition of a change is fundamental to deterrrrining what kind of
change occurred and, ultimately, adapting to that change (Ployhart & Bliese, 2006), as
these steps cannot proceed without awareness that a change occurred. Consistent with
self-regulatory models of motivated behavior, self-monitoring and self-evaluation should
be critical processes for detecting a change. Self-monitoring refers to attention that one
gives to different aspects of his or her behavior (Kanfer, 1990; Kanfer & Ackerman,
1989). These processes are thought to provide an individual with information about the
consequences of his or her actions. Self-evaluation refers to the process by which
individuals compare their progress to their desired goal (Kanfer, 1990; Kanfer &
Ackerman, 1989). Feedback plays an important role here as well, as it is the information
people use to compare against their goal or plan(s). Monitoring and evaluation, then,
allow for goal-performance comparisons and are key processes in identifying

discrepancies.

24

However, I believe that the ability to successfully detect changes in a given task is
predicated on having basic knowledge of the given task and environment, which has been
shown to be related to adaptive performance (Chen et al, 2005; Ford et al, 1998;
Kozlowski et al., 2001). The skill acquisition literature discusses a number of different
types of knowledge that underlie skilled performance. Two types of knowledge that are
relevant here are declarative knowledge and procedural knowledge.

Declarative knowledge is the most basic level of knowledge about a task or skill.
It refers to knowledge of different facts, concepts, definitions, instructions, strategies, etc
about a given task (Anderson, 1982). This knowledge, however, is tied directly to the
task at hand and it is primarily based on surface level characteristics such as exposure,
recall, retention, and memorization rather than deep understanding of the structure of the
task. Declarative knowledge, however, does not directly deal with rules or processes that
direct behavior.

These rules are part of procedural knowledge (Anderson, 1982; 1993).

Procedural knowledge is concerned with links between stimulus-response contingencies
and forms the basis for performance of a given task. Procedural knowledge accumulates
with time and experience on a given task, and as it develops it allows for performance
that is quick and highly accurate.

Declarative and procedural knowledge form the basic understanding of how to
behave and perform in relation to a given task, without necessarily containing knowledge
of why given strategies, rules, or behaviors are appropriate. Since these types of
knowledge can be based primarily on memorization and rote repetition, they do not in

and of themselves signify a complex understanding of the key elements of the task and

25

their relationships. Because of this, declarative and procedural knowledge on the existing
task should allow people to detect, or notice, that task rules, elements, strategies, etc. that
change. When routine experts, i.e. people who posses large amounts of declarative and
procedural knowledge (Smith et al, 1997), notice goal-performance discrepancies on a
well-learned task, their task knowledge should allow them to attribute the discrepancies
to external (i.e. task change) rather than internal sources. However, the declarative and
procedural knowledge they possess should not provide any means for them to
successfully diagnose the nature of the problem since it is not necessarily based on a
well-developed understanding of the task.

A simple task, such as using a remote control to turn on a television, can be used
as an example. A person with declarative and procedural knowledge of the task knows
that when the objective is turn on the television, he or she can pick up the remote control,
point it at the television, and hit the power button, which should result in the television
turning on. However, a number of changes could occur such as a dead battery, a new
remote control with a different configuration, or something in the way of the television
that blocks the signal. Each of these would cause a problem and the end result would be
a goal-performance discrepancy in which the television does not tum on. In any case, the
person should detect a change and realize that since the previously successful strategy no
longer works, something must be wrong and it should be attributed to the task or the
environment itself. Therefore, a change is detected, and the person now needs to figure
out what exactly has changed and how to adapt to that change. The ability to detect this

change, however, is built upon both a foundation of declarative/procedural task

26

knowledge as well as active self-monitoring and self-evaluation that allows that
knowledge to be applied.
Diagnosis

Diagnosis is the second step in the process of adaptation. Briefly, diagnosis can
be defined as ascertaining the cause or nature of change in the given task. Whereas
detection deals primarily with answering the question of whether or not a change
occurred, diagnosis deals with understanding why and how that change occurred.
Diagnosis in this manner is an active, controlled cognitive process (e. g. Ackerman, 1992;
Louis, & Sutton, 1991; Schneider & Chein, 2003). When tasks (or components of tasks)
do not provide high levels of consistency in terms of the stimuli that occur and/or the
appropriate responses, as is the case when elements of a task change, upon detecting the
change and attributing it to the task people should need to engage in conscious, effortful
processing in order to determine the nature of the change. The process of diagnosis
should focus on just this, with the primary initial goal of developing or acquiring
additional declarative and procedural knowledge about the novel task that is being faced,
which can then be used to understand the nature of the change.

Research on active learning and training provides a bit of a parallel for thinking
about how people responding to change may go about learning and understanding the
elements of a novel task. In the training literature, active learning approaches give people
control over their own learning and skill development (Bell & Kozlowski, 2008; Salas &
Cannon-Bowers, 2001) by using techniques such as exploratory learning (Frese et al.,
1988; McDaniel & Schlager, 1990) where people are not provided with structured

training and instead are encouraged to explore, experiment, and even make errors in order

27

to learn from them (e. g. Frese & Altrnann, 1989; Ivancic & Hesketh, 2000; Keith &
Frese, 2005). These techniques have been shown to lead to higher levels of knowledge
gain and training performance (Frese et al., 1988), as well as adaptability (Ivancic &
Hesketh, 2000; Keith & Frese, 2005). Because training interventions do not exist during
adaptation, it is likely the case that people who are trying to adapt go about learning
about the novel task in a similar manner, by exploring and trying to gain knowledge
about the parts of the task that have changed.

However, one large difference exists when trying to adapt to a change as opposed
to learning or being trained to do a completely novel task. As the definition of
adaptability provided above states, the process of adaptation involves using existing
knowledge and skills to help in a novel, but not completely new, task. One approach that
may be particular useful for understanding how previous knowledge and skill may aid in
learning about the key elements of a novel task in an adaptive situation is provided by
Lovett and Schunn (1999). In discussing how base-rate and cue-predictiveness
information can be used to make decisions, Lovett and Schunn provide a discussion of
how people use previous experience and knowledge in a task domain to help learn about,
understand, and represent a task. Essentially, a task representation here refers to the
features of a given task or problem that people learn and use to gather and filter
information to understand the task as a whole. These features include things such as key
definitions, information, and contingencies that influence appropriate strategies,
decisions, or behaviors. Lovett and Schunn (1999) argue that an individual’s existing
knowledge regarding the task is used to help define what features of the task are most

important or influential and how new information about the task is attended to, gathered,

28

and organized to build understanding of the task itself. Thus, well-developed existing
task knowledge can be used to organize and direct the acquisition additional information,
which can then be used to diagnose the nature of a task change.

Work on recognition-primed decision making (Klein, 1993) also discusses how
previous knowledge and experience can guide processes related to learning about and
assessing a task change. Briefly, this work suggests that experts use their knowledge and
experience as foundations for consciously assessing a situation in order to understand the
goals that can be accomplished and narrow in on information that is potentially useful for
understanding the task, problem, or decision at hand.

Going back to the example of the turning on the television with a remote control,
in order to diagnose the nature of the change, the person would have to examine and
explore the elements of the task (television, remote, environment, etc.) and try to gain
additional knowledge about things that may have changed and how they changed. For
example, it may be the case that the person has a new television and the remote functions
differently. Using existing knowledge, the person may be able to guide their search for
information to find a manual and quickly learn new facts, definitions, and so forth
regarding the television and remote control and how they operate. This additional
knowledge, then, could provide the basis for future strategic and behavioral change.
However, as Lovett and Schunn (1999) point out, this would serve primarily as a
knowledge base and, therefore, would not necessarily provide direct prescriptions for
behavior. While this new task knowledge in and of itself is crucial, the process through
which that knowledge is applied in a given situation and used to direct action is vital, and

it is the core of strategic and behavioral adaptation.

29

Adaptation

As suggested earlier, adaptation, or adaptive performance, can be thought of as
the ability to use existing knowledge and skill to appropriately and functionally align
behavior with task and/or environment changes in a way that improves, maintains, or
limits decrements in performance effectiveness. One key to understanding the process of
adaptation is recognizing that the change that must be adapted to creates a task that
contains elements of novelty. By definition, a person performing on a task with novel
elements does not have a working pre-existing strategic contingency associated with the
task. Therefore, a potentially successful strategy must somehow be created, discovered,
or modified from an existing strategy. This process of identifying and implementing a
strategy is at the heart of the process of adaptation. Going back to the control theory
heuristic presented earlier in Figure 2 and the Campbell and colleagues distinction
between performance and effectiveness (Campbell, 1990; Campbell et al. 1993), this final
step determines the behavioral output, or performance, of the individual, which in turn
influences the effectiveness of adaptability.

While research on adaptability has focused on numerous antecedents to effective
adaptive performance such as individual differences, training techniques, and leader
behaviors, as well as on the importance of knowledge generalization in training (Kraiger
et al., 1993), little work has been done attempting to explicate the processes involved in
creating new strategies. Ployhart and Bliese (2006) do briefly touch on the importance of
proper strategy selection and implementation, but do little to describe how experience

and/or knowledge can influence this process. However, literature related to creativity,

3O

problem solving, and decision making provides some insight into how people use their
knowledge and experience to devise and select strategies for novel tasks.

Lovett and Schunn (1999) discuss the impact of previous knowledge and
experience on strategy development, selection, and implementation. Briefly, they
suggest that a person’s prior knowledge and experience with a given task will serve as a
constraint for the type of strategies that they can develop. Domain relevant knowledge
and Skills are thought to compose the complete set of response patterns or strategies that
an individual currently possesses. These strategies form the foundation or basis from
which new strategies or plans can be synthesized or generated when a novel task is
encountered. Some of the strategies or pathways to success will be more or less
common, well practiced, or more or less obvious than others and, thus, more available to
the individual. Furthermore, prior experience with sinrilar tasks can help guide strategy
selection and implementation, as some base rate information regarding success may
already be present.

A similar perspective is proposed in the creativity literature, in which a strategy or
behavior is considered to be creative to the extent that it is both novel and appropriate for
the task at hand (Amabile, 1983). One of the major aspects of creativity according to
Amabile is the ability of a person to break free from traditional strategies or ways of
thinking about a given task. This can take a number of forms. For example, breaking
cognitive set (N ewell, Shaw, & Simon, 1962) involves abandoning sets of old strategies
that are no longer successful, and breaking out of existing performance scripts (Schank &
Abelson, 1977) involves actively examining and, potentially, abandoning explicit sets of

steps for performing tasks (e. g. scripts or algorithms). The ability to successfully engage

31

in these types of behaviors, then, is predicated on the level of understanding one has
regarding the current task and its similarities and differences when compared to tasks
where the individual has previous experience.

Work on recognition-primed decision making (Klein, 1993) also provides
thoughts on linking existing task knowledge and skill with the ability to systematically
create, implement, and evaluate strategies. As described earlier, this approach suggests
that experts use their knowledge and experience to help assess and understand a new task
situation. Klein (1993) also suggests that existing knowledge and experience can be used
to form expectations and identify potential actions and strategies. Experience provides the
means for categorizing types of situations, and therefore guides strategy selection and
implementation based on the success of strategies used in similar situations.

Returning to the running example, knowledge gained in the diagnosis stage and
the person’s previous experience with Similar tasks can help guide strategy formulation
and implementation aimed at using the remote control to turn on the new television. In
this case, it may be that the television has to be taken off standby power before being
turned on. Information gained from reading the manual, or previous experience with
other items of technology with similar features could help lead the person to the
appropriate strategy. The enacted strategy and resulting behavior are then fed back into
the control cycle, which should lead to successful performance and removal of the
existing goal-performance discrepancy, or trigger another cycle of detection, diagnosis,

and adaptation should the discrepancy still exist.

32

Adaptability to Different Types of Task Change

The steps in the adaptability process discussed above are meant to generalize
across the different types of task change that people may face. However, it may be the
case that these steps take on different levels of importance for different types of task
change. Much of the empirical work conducted on adaptability or adaptive performance,
though, has taken something of a “catch-all” approach to manipulating task change and
operationalizing adaptive performance. For example, many of the studies using computer
simulation tasks (i.e. Bell & Kozlowski, 2008; Chen et al, 2005; Ford et al, 1998;
Kozlowski et al., 2001) manipulated change in similar ways, by including multiple
changes such as additional targets or task elements, changes in task sequencing or
prioritization rules, and/or longer performance trials. While these types of
operationalizations are usefirl as a starting point, they are likely insufficient for work
aimed at understand what people are actually doing to adapt to different types of changes
or how individual difference characteristics influence adaptability to different types of
changes.

One initial approach to categorizing different types of change has been put forth
by Pulakos and colleagues (Pulakos et al, 2000; 2002). Recall that Pulakos et al. (2000)
focused on constructing a taxonomy of different types of adaptive performance. Pulakos
et al. (2000; 2000) described eight dimensions of adaptive performance: (1) solving
problems creatively, (2) Dealing with uncertain or unpredictable work situations, (3)
learning new tasks, technologies, and procedures, (4) demonstrating interpersonal
adaptability, (5) demonstrating cultural adaptability, (6) demonstrating physically

oriented adaptability, (7) handling work stress, and (8) handling emergency situations.

33

This taxonomy serves as a foundation for theoretical work by Ployhart & Bliese (2006)
focusing on adaptability as a set of individual differences. Briefly, Ployhart & Bliese
suggest that individual differences related to cognitive ability, personality, interests, and
physical ability differentially predict adaptive performance based on the type of task
change one is facing.

While the work by Pulakos and colleagues (Pulakos et al, 2000; 2002) and
Ployhart and Bliese (2006) is certainly valuable, it focuses primarily on different types of
tasks that people have to respond to change in rather than the underlying structure of the
task and what parts of a given task are actually changing. This approach is something of
a “between task approach”, which is useful for differentiating among large classes of
tasks. This type of approach, though, does little in helping to understand how people
actually respond to different types of task change.

An alternative approach that focuses on breaking down the elements of a given
task and classifying change based upon them should be more useful to this end, as it
could potentially serve as a foundation for determining cognitive and motivational
processes related to adaptation and how they may differ when different parts or elements
of the task change. This would be more of a “within task approach”, focused on what
parts of a given task are actually changing over time. This type of approach would not
conflict with the existing fi'amework provide by Pulakos et al. (2000), but instead could
be used to further understand how people go about responding the different types of
change identified therein and what individual differences should be predictive of these

different types of change.

34

One categorization of task elements that could be useful in examining tasks fiom
this type of perspective is that proposed by Wood (1986). Wood (1986) provides an
approach to viewing tasks that focuses primarily on the behaviors required for performing
them. This type of viewpoint helps to isolate task characteristics from other factors such
as person characteristics, motivation, and environmental constraints that can also be
related to one’s ability to perform tasks and adapt to task changes. From a theoretical
standpoint, this is desirable as it helps to more clearly separate different tasks from one
another and articulate the different types of changes in a given task itself that necessitate
adaptation.

According to Wood’s (1986) framework, tasks can be broken down into three
core components that are thought to describe their basic form and structure. These core
components can be used to describe the parameters of a given task as long as the task
includes some sort of behavioral act(s) and has a readily identifiable product.

Products can be thought of as the observed and measurable results of acts (Wood,
1986). Products are the results of given behaviors and can be observed and measured
independently of the given behaviors. The product of an act is the actual effectiveness,
and would be the comparison point used for determining goal-performance discrepancies.
In order to break a task down into its core components, Wood suggests that the product of
a given task must first be identified. Once Specified and described, the attributes of a task
such as quantity, quality, timeliness, etc. provide the basis for identifying the other core
components of the task.

The second core component of a task, according to Wood (1986), is the required

set of behavioral acts. Acts can be thought of as the behavioral responses a person should

35

engage in during a given task in order to achieve the specified level and type of
performance effectiveness suggested by the product of the task. While behaviors can be
broken down into many different levels (neural level, individual movement, etc.), a
behavioral act in this framework is best thought of as a “pattern of behaviors with some
identifiable purpose or direction” (Wood, 1986, p. 5). The purpose is usually identifiable
as an entire behavior such as “lifting a box”, “scanning an item”, or “printing a
document”.

The final core component of tasks according to the Wood (1986) fiamework is the
informational cue. Informational cues are bits of information about the characteristics of
a task stimulus that an individual can use to make decisions and judgments during
performance on the given task. It is important to note that not all stimuli received during
task engagement can be classified as informational cues. Informational cues are only
those stimuli that can be used make discriminations between different potential behaviors
or courses of action.

In order to understand adaptability, it is essential to understand the nature of
task(s) being performed. The Wood (1986) task fiamework as described above can be
very useful in this pursuit. From a theoretical perspective, identification of the outcomes,
behavioral acts, and informational cues that define the given task enviromnent provide
the foundation for understanding not only the task(s) at hand, but it also helps to
understand and make sense of any potential changes that can happen in the task that will
necessitate adaptation. As suggested earlier, task change may take on different forms

based what parts of the task are actually changing. Understanding the products,

36

behavioral acts, and informational cues that make up the task, then, provides a starting
point for determining and classifying types of change.

To assess the type of change in the task, a framework for utilizing these task
elements to interpret and organize the types of change is needed. Wood (1986) provides
a starting point in this pursuit by using these elements to identify a number of different
types of task complexity, which should help to describe some different types of task
change and, subsequently, the type of adaptation they demand. Task complexity
describes the relationships between task inputs (behavioral acts and information cues)
and outputs (products or effectiveness), and it is suggested to be “an important
determinant of human performance through the demands it places on the knowledge,
skills, and resources of individual task performers” (Wood, 1986, p. 66). I assert that
changes to task characteristics related to different types of complexity should be directly
related to differences in the adaptation process required by those changes. Wood (1986)
discusses these types of changes in his description of dynamic complexity, which occurs
when the cause and effect relationships between different components of the task change
over time. It is the other two types of complexity Wood discusses, however, which may
be especially useful for categorizing types of task change.

The first of these is referred to as component complexity. Tasks with high levels
of component complexity have a large number of distinct acts to perform and information
cues to process. Thus, the higher the number of distinct acts that need to be performed
and information cues that need to be processed, the higher the component complexity of
the given task. Increases or changes in component complexity necessitate that people

learn how to complete new acts or properly process new cues in striving for the same

37

product or type/level of performance effectiveness. One example of component
complexity task changes are seen in work by Keith & F rese (2005) who required students
to perform tasks in a computer program that they had not previously completed.
Coordinative complexity refers to the types of relationships between different
aspects of the task and the timing or order they need to be performed in. Wood (1986)
suggests that the ordering and sequencing of acts in terms of timing, fiequency, and
location requirements form one type of component complexity. In this sense, more
complex tasks likely have longer sequences of tasks with higher levels of importance
regarding proper ordering or structuring of specific behavioral acts. Another type of
coordinative complexity has to do with the form and strength of relationships between
different types of informational cues, related behavioral acts, and related products. When
examining the form of the relationship between task inputs and products, tasks become
more complex as the degree of linearity changes. For example, tasks with direct linear
relationships between intensity or frequency of a behavioral act and performance have
low levels of coordinative complexity. On the other hand, tasks that require optimal
levels of a given behavioral act and/or coordination among different acts with different
optimal levels are more complex, as they require people to discover exactly what the
optimal levels of fiequency or intensity for a given act are. Changes in coordinative
complexity, then, would result in having to learn different sequences or configurations of
behaviors to reach the same product or outcome. Examples of this type of change in the
existing adaptability literature can be seen in Chen et al. (2005), whose operationalization

of adaptability required participants to use routes with steps that needed to be correctly

38

ordered (e. g. more “waypoints”) in order to properly execute the task and Kozlowski et
al. (2001) who included changes in task scoring rules related to certain behavioral acts.

While the two dimensions of complexity above are useful for thinking about types
of task change, one other important task factor that Wood (1986) does not explicitly
address is task difficulty. While increases in component or coordinative complexity can
certainly make a task more “difficult” or harder to perform well on, difficulty here can be
thought of as the total number of behavioral acts that one has to engage in to complete the
task, as well as the amount of time that one has to complete these acts. Changes in
difficulty in this sense can operate completely independently from component and
coordinative complexity changes. For example, one could have a task with low
sequencing requirements and relatively few distinct acts, but the person may be required
to complete many of the same acts in a short period of time. While not addressed in the
Wood (1986) framework, the notion of task difficulty changes as drivers of the need to
adapt is supported by numerous studies that include responses to task difficulty changes
in their operationalizations of adaptive performance (e. g. Bell & Kozlowski, 2008; Chen
et a1, 2005; Ford et al, 1998; Kieth & Frese, 2005; Kozlowski et al., 2001).

Considering the types of change discussed above results in a three factor
framework based primarily on the distinct task elements described by Wood (1986).
While Wood’s discussion of task complexity also serves as a foundation for identifying
types of change, it is certainly not complete; adding a task difficulty factor helps extend
and clarify the fi'arnework while remaining consistent with previous empirical work and
operationalizations of adaptive performance. These three factors certainly do not

represent every type of change that one could face, or the possible combinations of

39

multiple types of change in a given task. However, this type of simple framework can
serve as a first step in helping to gain additional understanding regarding how people
adapt to different types of task change, and that is one of the key foci of this study.
Based upon the previous discussion of the three-step adaptability process of
detection, diagnosis, and adaptation, Figure 4 presents a model of different “pathways” to
adaptation based on the different types of change identified above. Briefly, this model
attempts to lay out the proposed importance of cognitive and motivational factors in
influencing detection, diagnosis, and adaptation to difficulty, coordinative, and
component task changes. It is important to note that the aim of this study is not to
validate this process model of adaptation. The steps of detection, diagnosis, and
adaptation here are used primarily as a heuristic for thinking about adaptation as a
process and organizing the measurement of cognitive and motivational factors as
predictors of adaptive performance effectiveness. Generally, this heuristic suggests that
one needs to detect that a change in the task has occurred, gather and leverage relevant
knowledge about the change in order to figure out what has changed, and then devise and
implement appropriate strategies in order to perform effectively. This type of mediating
model is roughly analogous to much of the existing research on adaptability, which
primarily shares a conceptual fi'ame whereby individual differences (e. g. personality,
cognitive ability) influence mediating processes of some sort (e. g. goals, task
knowledge/expertise) which in turn influence how people perform in response to change
(Ployhart & Bliese, 2006). However, this model is not as interested in individual
difference factors (although they will be touched on later) and its primary focus is on in

situ mediating processes such as new knowledge development and strategy

40

development/selection, which have received little attention in the adaptability literature
other than a general investigation by Chen et al. (2005).

This model suggests that what is relevant or appropriate in terms of knowledge to
acquire and strategies to implement may vary substantially from one type of change to
another. This is one place where the breaking down tasks into their component parts and
classifying different types of change based upon them provides value. Even though the
process of adapting is generally the same for different types of change, the things that
peeple need to focus on are vastly different. This suggests that there are different
pathways to success for each type of change based on the knowledge and strategies
necessary for adaptation.

For task difficulty changes, the pathway to success should be primarily
motivational. As stated earlier, a change in task difficulty involves an increased number
of behavioral acts that one has to engage in or a decreased amount of time that one has to
complete the same number of acts. Detection of this type of change should be primarily
driven by monitoring goal-performance discrepancies and noticing that performance is no
longer appropriate. Upon detection of a change, diagnosis via additional relevant
knowledge acquisition should be limited to learning about the increase in the number of
required behavioral acts. Furthermore, strategy development and selection, which is
primarily a cognitive process, should be limited to ascertaining that one needs to apply
existing knowledge, skills, and strategies in a more efficient or speedy manner.
Therefore, someone who is proficient in the task to begin with, and thus has knowledge
and skill that can be adapted, should be able to continue to use their existing knowledge

and skill as the foundation for their behavior. However, to be successful this person will

41

need to work harder and exert additional effort to complete the additional behavioral acts
required by the change in task difficulty. Thus, adaptability to difficulty changes should
be most influenced by task effort, and the primary pathway to success should be
motivational in nature.

For component task changes, the pathway to success should be both motivational
and cognitive. A component change involves additional distinct information cues that
need to be processed and/or additional distinct acts that need to be performed. Detection
of this type of change, like detection of a difficulty change, should be primarily driven by
monitoring goal-performance discrepancies and noticing that performance is no longer
appropriate. Diagnosis, however, should be more cognitively demanding, as an
individual will need to learn new information about the task to allow him or her to
properly interpret the new informational cues or learn about the new acts that need to be
engaged in. However, barring corresponding coordinative changes, strategy development
and selection should not be overly cognitively taxing. Once new appropriate knowledge
about cues and acts is developed, the individual should be able to “plug it in” to his or her
existing strategy, as many of the contingencies, rules, etc. that were learned should still
apply. The key, however, is the use of newly understood cues and acts within these
strategies. Of course, task effort should be important here as well, as it should be a direct
determinant of adaptive performance once knowledge of the new cues and acts is
integrated and ready to be applied.

For coordinative task changes, the pathway to success should be both cognitive
and motivational, with cognitive processes being the dominant factor. A coordinative

change involves new sequencing of behavioral acts in terms of timing, frequency, and

42

location and/or different forms of relationships between informational cues and related
acts and products. As with the other types of change, detection should be driven by
monitoring goal-performance discrepancies. Similar to component change, diagnosis of a
coordinative change should be cognitively demanding, as the person will have to learn
about the new rules or contingencies that govern the relationships between information
cues and associated behavioral acts. However, strategy deveIOpment and selection
should be more cognitively demanding here than for difficulty or component changes, as
one must be able to successfully implement their new knowledge in order to allocate his
or her attentional resources and make correct sequencing and prioritization decisions
regarding cues and acts on the task at hand. This is a more inferential process than what
should be necessary for a component change, and it should be more cognitively complex
and demanding. Once again, task effort itself should be important here, but its primary
effect should be on strategy implementation and it should only lead to higher levels of
adaptive performance effectiveness if the correct strategy is in place.

Based on this discussion of the different pathways to success for different types of
task change as well as the general structure of the adaptation process outlined previously,
a number of hypotheses can be developed. These hypotheses will focus on the
importance of outcomes related to in situ processes of detection, diagnosis, and
adaptation as predictors of adaptive performance effectiveness and they will be presented
in a proximal to distal fashion, with each set of variables intended to mediate the effects
of the more distal predictors.

The process of adaptation, as discussed above, primarily deals with using existing

knowledge and skill to help develop, select, and implement appropriate task strategies.

43

While little work has focused on the importance of in situ strategy development,
selection, and implementation on adaptive performance, Chen et al. (2005) provide some
evidence that general goal striving was related to higher levels of adaptive performance
on a radar simulation task that combined elements of difficulty, component, and
coordinative change. Goal striving there was a factor that included both effort and
cognitive strategy components, and therefore has some parallel to the notion of
appropriate strategies here. Based on this, and the discussion above, Hypothesis 1
suggests that appropriate strategy change, a result of processes related to adaptation,
should positively predict adaptive performance effectiveness.

More specifically, condition specific sub-hypotheses can be generated based upon
the previous discussion regarding the importance of motivational and cognitive pathways
to success in adapting to different types of task change. For task difficulty changes the
pathway to success should be primarily motivational, and the appropriate strategy should
be to exert additional effort using what is already known about the task. However, while
still predictive, task effort alone should less directly predict success in adaptation to
component and coordinative task changes, as knowledge acquisition and strategy
development/use is more cognitively demanding in both. For component changes,
appropriate cognitive strategy use should involve incorporating new knowledge and skill
into existing strategies. Coordinative task changes should require the highest cognitive
load in terms of strategy selection, development, and appropriate use. In response to this
type of task change, appropriate cognitive strategy use should involve using newly
learned task rules and contingencies to make correct sequencing and prioritization

decisions regarding cues and acts on the given task.

44

Hypothesis 1 a: Task related effort will be most strongly positively related to
adaptive performance when people are facing changes in task difliculty.
Hypothesis 1 b: When facing a component task change, the correct use of new
information cues and/or acts will positively predict adaptive performance.
Hypothesis 1c: When facing a coordinative task change, the correct sequencing
and prioritization of behavioral acts will positively predict adaptive performance.
Processes related to diagnosis primarily focus on ascertaining the cause or nature
of change in the given task. It is suggested above that this is primarily done through the
use of previous task knowledge and experience to guide the additional development and
acquisition of task knowledge relevant to the novel aspects of the changed task. Work on
skill acquisition (e.g. Anderson, 1982; Kanfer & Ackerman, 1989) firmly focuses on the
importance of declarative and procedural task knowledge as foundations for the
development of task skill. Similarly, numerous studies on adaptability from the
training/outcome perspective have focused on the importance of task knowledge before
change as a predictor of adaptive performance (Chen et al., 2005; Ford et al., 1998;
Ivancic & Hesketh, 2000; Keith & Frese, 2005; Kozlowski et al., 2001). However, none
of these studies have focused on the role of new, appropriate knowledge development
during the process of adaptation as a predictor of adaptive performance.

In concordance with the discussion above on the importance of in situ diagnosis-
related knowledge development for identifying and implementing proper strategies, and
ultimately adaptive performance effectiveness, Hypothesis 2 suggests that task
appropriate in situ declarative knowledge development should positively predict

appropriate strategy selection and implementation. More specifically, for a task difficulty

45

change, gaining knowledge regarding the increased or changed number of behavioral acts
that one has to engage in to perform well should directly influence the amount of effort
that one exerts in trying to perform. For a component task change, gaining knowledge
and understanding of new information cues or necessary acts should form the foundation
for appropriately incorporating the new knowledge and skills into existing task strategies.
Finally, for a coordinative task change, knowledge gained regarding changes in task rules
or contingencies should provide the foundation for the proper strategy development and
execution in terms of correctly prioritizing and sequencing behaviors.

Hypothesis 2a: When facing a task difliculty change, declarative knowledge

development regarding increases in the number of required behavioral acts

should positively predict task related effort.

Hypothesis 2b: When facing a component task change, declarative knowledge

development regarding new information cues or necessary acts should positively

predict the correct strategic use of new information cues and/or acts.

Hypothesis 2c: When facing a coordinative task change, declarative knowledge

development regarding changes in task rules and contingencies should positively

predict the correct sequencing and prioritization of behavioral acts.

The process of detection is primarily focused on noticing goal-performance
discrepancies and attributing those discrepancies to a change in the task at hand.
Detection is thought to be a fundamental process to adaptation (Ployhart & Bliese, 2006)
as it is not possible to learn about a change and eventually adapt one’s strategies
appropriately if there is no knowledge that a change in the task occurred. To this point,

the empirical literature on adaptability has not focused on examining change detection as

46

an important part of adaptability. While research such as that done by LePine et a1.
(2000) has focused on how well people perform on tasks with unforeseen change that
necessitate adaptability, detection of the change itself has not been a variable of interest.
In this study, I intend to investigate the importance of change detection as a predictor of
adaptive performance effectiveness through its effects on processes related to diagnosis
and adaptation. Hypothesis 3 suggests that change detection will positively predict
appropriate declarative knowledge development for the given type of task change.
Hypothesis 3a: When facing a task difi‘iculty change, change detection will
positively predict declarative knowledge development regarding increases in the
number of required behavioral acts.
Hypothesis 3b: When facing a component task change, change detection will
positively predict declarative knowledge development regarding new information
cues or necessary acts.
Hypothesis 3c: When facing a coordinative task change, change detection will
positively predict declarative knowledge development regarding changes in task

rules and contingencies.

47

Metacognition and Adaptive Performance

While the model of adaptation discussed previously was seated within control
theory as a guiding heuristic, the hypotheses developed to this point deal directly with
factors related to how well people are able to engage in self-regulatory processes related
to critical factors such as monitoring goal-performance discrepancies and evaluating the
effectiveness of given strategies. These types of factors, though, are considered to be
critical for successful goal-directed behavior (Kanfer, 1990). However, past research has
found that adults often fail to monitor their thinking and behavior or to use their
knowledge and experience to direct future behavior toward successfully accomplish their
goals (Gamer & Alexander, 1989; Osman and Hannafin, 1992). Because this study is
predicated on thinking about adaptability as a self-regulated process, I believe that it is
essential for factors related to self-regulatory ability or skill to be examined.

One type of self-regulatory factor that has received attention in adaptability
literature is metacognition. Metacognition deals explicitly with strategy development,
implementation, monitoring, and revision and can be loosely defined as the knowledge
about one’s own cognitions and the monitoring and regulation of behavior based upon it
(F lavell, 1979; Brown, 1987). More specifically, metacognitive control processes are
thought to deal with the in situ regulation how well one is making plans, monitoring
performance effectiveness, and evaluating the success of one’s plans or strategies
(Schraw & Moshman, 1995; Winne, 1995; 1996).

Metacognition is a set of active processes that deal with the regulation and control
of one’s behavior or thinking (Ertmer & Newby; 1996). From a motivational

perspective, one can think of these processes as a central determinant of the direction of

4s

behavior. These processes are suggested to be important to both learning and
performance in this respect (Reeve & Brown, 198 5), and these effects likely carry over to
influence adaptability as well (Ford et al., 1998; Bell & Kozlowksi, 2008). There are
three primary dimensions of metacognitive control processes; they are commonly
referred to as planning, monitoring, and evaluating (Brown, Bransford, Ferrara, &
Campione, 1983; Ertrner & Newby, 1996).

Planning refers to strategic behaviors that are engaged in before actual task
performance behaviors begin. Ertmer and Newby (1996) identify three major tasks that
are essential to the planning process. The first is setting or accepting a goal. This goal
can come in the form of a self-set goal based on knowledge of the task at hand and what
is required for successful performance or in the form of an outside goal set by and
external stakeholder (e. g. supervisor, manager, etc.). Once an individual has a goal or
target in mind, his or her focus should shifi toward identifying possible strategies for
accomplishing that goal. This decision is thought to be based primarily on the declarative
knowledge possessed regarding task demands as well as past experience dealing with
how well their currently available strategies could match the demands of the task at hand.
The final step in the planning process involves identifying potential obstacles that may
stand in the way of successfully achieving one’s goal. This involves forethought in that
one must rely on previous experience or mental simulations to try to identify the types of
problems that may occur and how to deal with them.

Monitoring is the second dimension of metacognitive control and is suggested by
Schraw and Moshman (1995, p. 355) to be “one’s on-line awareness of comprehension

and task performance.” The process deals primarily with the identification of possible

49

discrepancies between current levels of performance and targeted levels as specified by
plans and goals. Like planning, there are three major steps or parts of the monitoring
process that people who are metacognitively active engage in while performing on a
given task. The first of step involves actively monitoring performance in order to
observe exactly what actions are being performed and how these actions are translating
into task performance effectiveness. The second step is the comparison of current
progress to desired progress. This is akin to looking for goal-performance discrepancies
and helps to determine the utility of one’s current strategy. If all is going well, the
strategy should be seen as appropriate and the individual can simply continue to monitor
his or her behavior. However, if a discrepancy is found the person will have to revert to a
pre-formed contingency strategy should it exist, or reformulate plans in order to increase
performing successfully. This re-planning is the third major step involved in monitoring
and is somewhat different from that involved in the planning stage in that it occurs in real
time while the performer is working on the task. It is also different in that the performer
now has more information to work with and may be better able to formulate strategies
based upon it.

Evaluating is the final dimension of metacognitive control, and it happens at the
conclusion of the goal striving process. This step involves assessing the strategies and
behaviors employed to attain one’s goals and the product achieved (Ertmer & Newby,
1996), how successful they were, and why they were or were not successful. This
process of reflection, then, is thought to be a key factor in helping people to learn from

their experiences and influence future planning and strategy selection for similar tasks.

50

Metacognition has been shown to be pOsitively related to numerous outcomes
related to learning, classroom performance, and problem solving. The primary theme
driving these studies is that individuals with greater metacognitive skills should be able to
learn or acquire skills well because they can quickly identify the problems they are
having and remedy them by implementing or devising different, more appropriate
strategies. For example, Pintrich and DeGroot (1990) found that high levels of
metacognition were correlated with higher GPA as well as with better performance on a
number of different aspects of classroom achievement such as exam performance and
proficiency in writing class reports. Landine and Stewart (1998) also found that high
levels of metacognition were positively related to GPA in high school students. In a
similar vein, Minnaert and J anssen (1999) found that college students with higher levels
of metacognition were more academically successful than were those who exhibited low
levels of metacognition.

For the most part, these types of studies have focused on broad or general learning
objectives, where there is likely a large amount of background knowledge and classroom
structure. The relationship between metacognition and learning, problem solving, or task
performance may become more variable as the task becomes more complex or more
novel, such as that which would be faced in an environment that requires adaptability.
Self-regulation theory grounded in a resource allocation perspective (e.g. Kanfer and
Ackerman, 1989) might suggest that regulatory activities early in adaptation may be
harmful because attention could be better spent on learning the basics of the new task.
However, I believe that enough existing knowledge of the task, or similar tasks, would

exist to buffer this type of effect. Furthermore, without engaging in processes related to

51

metacognition or self-regulation, people may not be able to detect that a change has
occurred in the first place. This is similar to thoughts by Barley, Connolly, and Ekegren
(1989), who suggest that in leaming a new task, high initial task demands might inhibit
the effectiveness of strategy search and selection. It is likely, then, that people with high
levels of metacognitive skill may be better able to perform well initially even when task
learning demands are high. I would argue that adaptive situations where the task changes
somewhat, but not completely, pose a similar type and level of novelty. Therefore, the
strategic benefits of using metacognition may help guide attention and result in more
targeted and adaptive strategy selection, thus influencing adaptive performance. This is
consistent with suggestions by Schmidt and Ford (2003) that metacognition should be
especially useful in situations that contain little external structure or guidance. Indeed, a
number of studies have investigated the effects of metacognition in these types of
situations. .

While not strictly focusing on adaptability, Clause, Delbridge, Schmitt, Chan, &
Jennings (2001) examined the influence of metacognitive control processes on the ability
of people applying for entry-level law enforcement positions to apply knowledge
regarding departmental procedures to a number of different simulated situations. Clause
et al. (2001) found that higher levels of metacognitive control during test preparation lead
to higher levels of effort, the use of more complex learning strategies, and through them
higher test performance.

As described earlier, Ford et al. (1998) examined the impact of metacognition on
task knowledge, skills, affect, and adaptive performance/training transfer. Ford et al.

found that levels of metacognition during training were positively related to transfer

52

performance through their effects on task knowledge and skill development as well as
levels of self-efficacy.

In a similar vein, Bell and Kozlowski (2008) examined the impact of various
types of active learning techniques on knowledge and skill development as well as
analogical and adaptive transfer. While their study focused on numerous cognitive,
motivational, and emotional mediating variables and processes, one key finding was that
levels of metacognition during active learning directly lead to higher levels of self-
evaluation, intrinsic motivation, and self-efficacy, which in turn had positive effects on
task knowledge development as well as analogical and adaptive transfer performance.

Taken together, these studies provide evidence for the general efficacy of
metacognition for influencing learning, problem solving, and task skill development.
Furthermore, the latter set of studies support the viewpoint of Barley et al. (1989) that
high levels of metacognitive or self-regulatory processes can benefit performance in
novel or difficult learning or problem solving situations and extend findings to show that
metacognitive activity during training can influence adaptability through the development
of task knowledge, skill, and self-efficacy. Because of these types of findings, and the
general findings proposed earlier that adults often do not engage in these types of
metacognitive activities on their own accord, it comes as no surprise that people have
focused on conducting interventions aimed at training people to more successfully use
metacognitive skills related to planning, monitoring, and evaluating one’s strategies and
behavior.

In the educational psychology literature there is a history of developing

metacognitive interventions with the primary purpose of examining their effectiveness on

53

learning and knowledge development. While not explicitly tested, it is also thought that
the techniques taught to individuals to help them regulate their learning can be transferred
to and successfully used on other tasks as well (Reeve & Brown, 1985). As Reeve and
Brown (1985, p. 345) state, “. . .the aim [of metacognitive training] is the generalization
of skills.” Unfortunately, researchers in educational psychology have not addressed the
ability of interventions to lead to strategies that generalize across of a number of tasks,
although many studies focusing on interventions have found promising results for their
given tasks.

These interventions have taken a number of different forms. In a set of studies
examining a reciprocal teaching technique, Brown and Palincsar (Brown & Palincsar,
1982; 1985; Palincsar & Brown, 1984) encouraged the students and teachers to alternate
in presenting passages and leading dialogue in order to get the student to engage self-
review, self-questioning, and clarifying processes, all of which were thought of as
metacognitive control processes. Their results showed students using these techniques
exhibited large gains on both classroom and standardized tests of reading comprehension.

King and colleagues (King, 1989; King, 1990; King 1991a; King, 1992; King &
Rosenshine, 1993) had students utilize generic question stems such as “How are and
similar” or “How does affect ...” to generate questions during learning that would
help them to monitor their comprehension as well as to integrate and recall previous
knowledge more successfully. These questioning strategies lead to greater increases in
understanding and reading comprehension across academic levels ranging from grade 5

to college.

54

Berardi-Coletta, Buyer, Dominowski, & Rellinger (1995) asked participants
questions aimed directly at assessing strategy use and monitoring during performance on
simple problem solving tasks. These questions such as “how do you know that this is a
good move?” were thought to focus participants on monitoring their success and planning
for upcoming behavior, and those who received them showed higher levels of problem
solving performance.

In a related study, King (1991b) required students to ask similar questions of
themselves and partners throughout learning pertaining to the metacognitive control
processes of planning (e. g. “what is our plan”), monitoring (e. g. “are we on the right
track”), and evaluating (e. g. “what would we do differently next time”). Students using
these questions outperformed students asking self- generated questions or students who
asked no questions of each other on tests of written problem solving ability and the
ability to solve a problem on a novel task.

Delclos and Harrington (1991) used a similar questioning technique, but paired it
with task and problem solving training. The self-questions that students used were
similar to those used by King (1991b) and aimed at helping them monitor their
performance at different stages of task completion. As expected, students engaging in
self-questioning were able to solve more difficult problems correctly than students who
received task training only or task and problem solving training.

While the study of metacognitive interventions primarily began in educational
psychology using relatively simple learning or problem-solving tasks, recent work in

Industrial/Organizational psychology has also shown some support for the efficacy of

55

these types of interventions using somewhat more complex and organizationally relevant
tasks.

Schmidt and Ford (2003) conducted a study in which they examined the impact of
metacognitive interventions on a number of different leanring and performance outcomes
using a self-guided webpage construction training task. Schmidt and Ford’s
metacognitive training intervention consisted of three parts that were aimed at increasing
elements of metacognitive control. The first utilized self-questioning strategies and
generic question stems similar to those used by King and colleagues (King, 1989; King,
1990; King 1991a; King, 1992; King & Rosenshine, 1993), the second instructed
individuals to delay self-questioning briefly after studying so that judgments of progress
were more accurate (e. g. Nelson & Dunlosky, 1991), and the third part involved
prompting trainees every 10 minutes throughout training to assess their current levels of
learning and understanding. Schmidt and Ford’s (2003) results showed that
metacognitive training interacted with individuals’ performance-avoidance goal
orientations (e. g. Elliot & Church, 1997; VandeWalle, 1997) to impact metacognitive
activity. More specifically, individuals not afraid of demonstrating incompetence were
more receptive of the metacognitive intervention. Metacognitive activity, then, was
related to higher levels of declarative knowledge regarding the webpage design task and
higher levels of performance on the webpage building task, although no direct effect of
the metacognitive intervention was found.

As a whole, this work shows that metacognitive interventions hold promise for
influencing metacognitive monitoring, learning, and skill acquisition. However, the tasks

used along with these interventions were primarily in the vein of classroom learning and

56

training. Based on work by Ford et al. (1998) and Bell and Kozlowski (2008), one would
expect that these interventions may also be useful in organizational settings, as they show
the efficacy of metacognitive activity there. Furthermore, Smith et al. (1997) tout the
potential of metacognitive training for influencing levels of adaptive expertise and
helping to aid in skill generalization across tasks. However, to this point only one study
has focused on examining the effects of a metacognitive intervention during training on
adaptive performance.

Keith and Frese (2005) examined the effects of a metacognition training
intervention combined with an error-based training intervention for predicting analogical
and adaptive transfer on an overhead slide creation task. The metacognitive intervention
used by Keith and F rese primarily involved self-questioning of the same nature used by
King (1991b) and Schmidt and Ford (2003). Briefly, their results suggested that
analogical transfer was the same'for groups receiving (a) error avoidant training, (b) error
management training, and (c) error management training coupled with a metacognitive
intervention. For adaptive transfer, both error management training groups outperformed
the error avoidant training group, but they did not differ from each other. HoWever,
Keith and F rese suggested that error management training itself spurs metacognitive
activity, and showed that the effects of error avoidant vs. error management training on
adaptive transfer were mediated by metacognitive activity engaged in during training.

Together, the work by Ford et al. (1998), Bell and Kozlowski (2008), and Keith
and Frese (2005) provide some evidence for the importance of metacognitive activity in
predicting factors related to adaptive performance as well as adaptive performance itself.

While this work in encouraging, it is far from comprehensive, especially when viewing

57

adaptability as a process or series of behavioral and cognitive events rather than just
performance at one point in time.

From this point of view, it makes sense to consider metacognitive activity during
adaptation, rather than just during training. Research and theory regarding metacognitive
interventions and their influence on transfer (e. g. Ivancic & Hesketh, 2000; Keith &
Frese, 2005) primarily rely on the transfer appropriate processing principle (Schmidt &
Bjork, 1992), which suggests that activities desired during transfer should be practiced
and engaged in during training. Because of this, it is argued that the effects of
metacognitive training should reach forward and influence metacognitive activity during
transfer or adaptive performance. While this may be the case, none of the studies
examining metacognition and adaptability have actually looked at metacognitive activity
during adaptive performance, presumably because of the static view of adaptive
performance that primarily focuses on effectiveness.

Unlike task knowledge and skill, though, metacognitive skills are general in
nature and are not restricted to a certain type of task. Therefore, it is reasonable that one
could prompt the use of these skills during the process of adaptation rather than simply
hoping or positing that they carry over from a training environment. This type of
prompting would be much less hands on and resource intensive than continuous task
training, and it should also be more robust in reaction to various types of task change as
the focus is on monitoring and evaluating the success of strategies and overall
performance regardless of the underlying content domain or type of change.

It is not difficult to see how metacognition could influence each step of the

adaptability process as described earlier. The key processes involved in detection of a

58

{35.x
[Ell
act
of g

ch

in
ma

shi

an:

5h.-

task change are evaluating goal-performance discrepancies and being able to attribute
newly existing discrepancies to changes in the task. A person who is metacogrritively
active by definition is monitoring his or her task environment and looking for these types
of goal-performance discrepancies, and thus should be more likely to detect them than
someone who is not. Diagnosis deals with attempting to understand the nature of the
change, and a key part of it should be using existing knowledge and experience to guide
knowledge development and acquisition regarding the novel elements of the task at hand.
Metacognitive planning processes primarily deal with using task knowledge to form
plans for upcoming goal behavior. While the actual strategy and plan development
involved in metacognitive planning falls more within the realm of the adaptation process
as discussed here, people who are metacogrritively active and trying to form plans should
realize that the understanding that they have of the task is not sufficient and should then
shift focus on acquiring additional information, knowledge, and understanding to help
diagnose the nature of the change so that these plans can be made. Finally, the adaptation
part of the process, which deals with the development, selection, and execution of plans
should be directly influenced by metacognitive activity related to planning, monitoring,
and evaluation, which should serve to help gauge strategic success and inform future
strategy development should one’s goal of high levels of adaptive performance not be
initially reached. This type of cyclical process, of course, ties back to the larger, more
general, control theory model of self-regulated behavior discussed earlier and should help
in understanding the potential broad ranging positive effects of metacognition during

adaptation.

59

This discussion argues for the potential efficacy of a metacognitive intervention
that focuses on getting people to actively engage in planning, monitoring, and evaluating
activities during adaptation. In fact, based on this discussion and the existing empirical
evidence showing the efficacy of metacognition during training as predictor of adaptive
performance (e.g. Bell & Kozlowski, 2008; Ford et al., 1998; Keith & Frese, 2005), as
well as the effectiveness of metacognitive interventions on promoting metacognitive
activity, learning, and problem solving performance (e. g. Berardi-Coletta et al.1995,
Brown & Palincsar, 1982; 1985; King, 1989; King, 1990; King 19913; King, 1991b;
King, 1992; King & Rosenshine, 1993, Palincsar & Brown, 1984; Schmidt & Ford,
2003), I expect that a metacognitive intervention conducted during adaptation will lead to
higher levels of adaptive performance effectiveness.

However, in consideration of the different types of task change discussed
previously and the motivational and cognitive demands of each, I think that this type of
general hypothesis regarding the effectiveness of an in situ metacognitive intervention
can be made more specific. Specifically, I believe that the overall load or importance of
cognitive factors in adapting to each type of change should be directly related to how
large of an influence a metacognitive intervention will have on adaptation to that type of
change.

For each type of change, detection of a task change is absolutely essential, as one
cannot adapt to a change that they don’t know is occurring. Therefore, monitoring the
task environment for changes and comparing current vs. desired performance is important
for each type of change, and a metacognitive intervention should positively influence this

process.

60

However, for task difficulty changes, as suggested earlier, the pathway to success
should be primarily motivational. Once someone detects a task difficulty change, the
only additional information they need to learn is that there are more acts that need to be
completed (or a lesser amount of time to complete the acts). The strategy that follows
should be very basic and straightforward: work harder. This is what Wood & Locke
(1990) refer to as a “Stored Universal Plan”, and they argue that they are so common and
familiar to people that they are essentially “built into goals” and, thus, serve to direct
goal-directed behavior. Because of the common nature of this type of strategy, people
should easily have access to it and, therefore, much of the influence that metacognition
could have on knowledge acquisition, strategy development, and evaluation may be
minimized.

For component task changes, processes related to diagnosis (i.e. acquiring
declarative knowledge) should take on higher importance once a task change is detected.
The correct plan should be to gain and leverage appropriate declarative knowledge about
the necessary new information cues and/or behavioral acts. This should require
additional regulation and cognitive effort, as one must not only determine what
knowledge is appropriate, but he or she must also regulate and monitor new knowledge
acquisition. Once knowledge is acquired, however, the cognitive load associated with
implementing and leveraging that knowledge (i.e. adaptation) should be fairly low. As
suggested earlier, people may be able to plug that new knowledge in to an existing
strategy of sorts. Therefore, along with monitoring process that should drive the

detection of goal-performance discrepancies, processes related to engaging in and

61

monitoring proper knowledge acquisition should make metacognition more important for
adapting to component task changes than task difficulty changes.

Finally, for coordinative task changes, it was suggested earlier that each step of
the adaptation process should be cognitively demanding, and therefore a metacognitive
intervention may have its largest effect adaptive performance effectiveness in response to
this type of change. Like the process for adapting to a component task change, both
detection and diagnosis-related processes should be quite cognitively complex and
taxing. However, processes related to strategy development and implementation should
carry the highest level of cognitive demand here, as one must successfully leverage newly
acquired task knowledge or experience into a new strategy that properly sequences and
prioritizes how information cues are interpreted and behavior acts are engaged in. In line
with this discussion, the following hypothesis is offered:

Hypothesis 4: A metacognitive intervention conducted during the adaptation

process will have the largest positive eflect an adaptive performance eflectiveness
in response to coordinative task changes and the lowest positive eflect on
adaptive performance eflectiveness in response to task difliculty changes.

Consistent with this rationale, this study will test for mediation of the effects of
the metacognition intervention on adaptive performance effectiveness. As suggested
above, the pathways to success when moving from difficulty to component to
coordinative task changes should become increasingly cognitive loaded, with detection
being the primary cognitive process of importance for difficulty task changes; detection
and diagnosis being primary cognitive processes of importance for component task

changes, and detection; and detection, diagnosis, and adaptation being primary cognitive

62

processes of importance for coordinative task changes. If this is the case, one would
expect that the primary cognitive processes of importance, given uniform or constant
levels of motivation, should mediate the relationship between the metacognitive
intervention and adaptive performance effectiveness. Based on this, the following
hypotheses are offered:
Hypothesis 5a: For difi‘iculty changes, change detection will fidly mediate the
relationship between metacognitive training and adaptive performance.
Hypothesis 5b: For component complexity changes, change detection and
relevant lmowledge gain will firlly mediate the relationship between metacognitive
training and adaptive performance.
Hypothesis 5c: For coordinative complexity changes, change detection, relevant
knowledge gain, and proper strategy use will fitlly mediate the relationship

between metacognitive training and adaptive performance.

63

Individual Differences and Adaptive Performance

While I earlier eschewed the individual differences or selection perspective to
adaptability, instead proposing a model and hypotheses that strongly focus on
adaptability from a process perspective, this does not mean that individual differences in
cognitive ability, personality, values, interests, and the like cannot function as usefirl
predictors of adaptability. While I strongly maintain my earlier assertion that adaptability
is best thought of as a set of cognitive and behavioral processes, I believe that it is also
important to attempt to understand different individual difference factors that can
influence processes related to detection, diagnosis, adaptation, and adaptive performance
effectiveness. However, LePine et al. (2000) argue that because the behavioral
requirements underlying performance in changing contexts have not been given
appropriate attention, one can expect ambiguity in research findings regarding predictors
of adaptability. Viewing adaptability from a process perspective, then, may help us to
comprehend some of these necessary behaviors and provide an elegant and effective way
to understand why certain individual differences influence adaptability and how their
effectiveness may differ across different types of task change. This would be consistent
with Ployhart & Bliese’s (2006) notion that individual difference factors have differential
weightings in predicting different types of adaptive performance, although clearly they
are dealing with different types of adaptive performance (i.e. Pualkos et al., 2000) rather
than task change and believe that adaptability resides within these individual differences.

The upcoming section focuses on describing and reviewing existing empirical
work that has focused on individual difference predictors of adaptability. From this, and

based on work above that focuses on adaptability from a process perspective, a number of

64

hypotheses regarding how key individual differences related to cognitive ability, goal
orientation, and personality differentially influence adaptability to different types of task
change will be proposed and investigated as a starting point for trying to understand some
of these effects. The factors reviewed here are in no way intended to represent the entire
domain of possible individuals that can influence adaptive performance. Rather, the
intent is to try to understand why individual differences examined in past work influence
adaptability and suggest why their effects may actually differ across different types of
task change.

One individual difference that has received considerable attention in work on
adaptability is cognitive ability. General cognitive ability, often referred to as g, can be
thought of as an individual difference related to one’s information processing ability,
reasoning ability, and capability to learn (Carrol, 1993; Hunter, 1986; LePine et al, 2000).
Cognitive ability is thought to be the best predictor of training performance or
effectiveness (Schmidt & Hunter, 1998) and has also been shown to predict job
performance across a large number of different jobs and contexts, with these effects being
more pronounced for tasks or jobs that are more complex or novel than in tasks can be
considered routine (Hunter & Hunter, 1984). Cognitive ability has also discussed as a
predictor of one’s ability to adapt existing knowledge and experience to novel tasks
(Pulakos et al., 2002; Snow & Lohman, 1984). In this vein, it is suggested that people
with higher levels of g should be able to use their experiences more effectively to gain
knowledge and influence future behavior and performance.

In line with these findings and beliefs, a handful of studies have focused on

examining the effects of cognitive ability on adaptive performance. As mentioned

65

earlier, LePine et al. (2000) examined the impact of general cognitive ability on decision-
making accuracy during an initial block of trials and during two subsequent blocks of
trials where decision-making rules changed. Based on the notion that participants faced
higher information processing demands when the decision making rules changed, they
hypothesized that the relationships between g and decision making performance would be
more pronounced during the blocks of trials where these rules changed. In line with this,
they found that cognitive ability was indeed a better positive predictor of decision-
making accuracy when people had to adapt to changes in decision making rules.
Similarly, Pulakos et al. (2002) found that cognitive ability was positively related to
supervisor ratings of adaptive performance, and Kozlowski et al. (2001) found that
cognitive ability was positively related to declarative knowledge levels acquired during
training, which in turn influenced adaptive performance both directly and through its
effects on training skill and knowledge structure development.

With this discussion in mind, I believe that the importance of cognitive ability as
a predictor of adaptive performance may vary across the different types of task change
suggested earlier. Considering LePine et al.’s (2000) rationale that cognitive ability
should be more strongly related to performance in situations with higher cognitive or
information processing demands, it makes sense to argue that as one goes from task
difficulty changes, to component task changes, to coordinative task changes, the
cognitive complexity of the change and requisite information processing demands
increase. This leads to the following hypothesis:

Hypothesis 6: Cognitive ability will differentially predict adaptive performance

efl'ectiveness in response to dijferent types of task change.

66

Another individual difference that has received attention in adaptability research
is goal orientation, specifically learning or mastery goal orientation. In general, goal
orientation can be thought of as the type of goals an individual tends to pursue across
achievement situations. People with a learning or mastery goal orientation tend to focus
on goals that deal with learning and competence development (e.g., Dweck, 1986; Dweck
& Leggett, 1988). Also, they are thought to be adaptive in response to situations that are
challenging or novel, as they gravitate toward and excel in these types of situations and
are resilient and persistent in the face of challenge (Kozlowski et al., 2001).
Furthermore, they espouse the belief that ability is malleable and that effort directed
toward a task leads to improvements in related outcomes (Ames, 1992; Dweck, 1986).
Recent meta-analytic evidence show that across a number of studies, learning orientation
was positively related to effective use of learning strategies, feedback seeking behavior,

3 and job performance (Payne, Youngcourt, & Beubien, 2007).

Learning orientation in the adaptability literature has usually been examined as a
predictor of various training outcomes, which in turn are thought to influence adaptive
performance. For example, Ford et al. (1998), Kozlwoski et a1. (2001), and Bell and
Kozlowksi (2008) all provide evidence that those with a learning goal orientation have
higher levels of self-efficacy at the end of training, and that self-efficacy reaches forward
to influence adaptive performance. Furthermore, both Ford et al. (1998) and Bell and
Kozlowski (2008) show that learning orientation positively influences metacognitive
activity, which in turn positively influences task knowledge (and in the case of Ford et
al., task skill and self-efficacy). This task knowledge, then, positively predicts adaptive

performance. While these findings do show the importance of considering learning goal

67

orientation during training, researchers have not considered the impact it may have
during processes related to adaptation.

If people with a learning orientation truly gravitate toward and excel in
challenging or novel situations, believe that they can work to improve their abilities, and
focus on learning and competence development, it would make sense that they would
adapt to task changes better than those without a learning orientation. This may be
especially true in regard to task changes that require new knowledge and strategy
development, as people with a learning orientation should enjoy and embrace these
challenges and persist in trying to develop competence on the newly novel elements of a
given task. If changes are primarily motivational in nature, such as task difficulty
changes, these sorts of competency and learning-based challenges likely don’t exist, and
one may not expect learning orientation to positively relate to adaptability. Based on this,
the following hypothesis is offered:

Hypothesis 7: Learning orientation will differentially predict adaptive

performance eflectiveness in response to diflerent types of task change.

Personality factors have also received attention as potential predictors of adaptive
performance. In general, the exploration of these factors has been limited to “Big-5”
factors of conscientiousness and openness to experience (e. g. Costa & McCrae, 1992).
While somewhat limited in scope in terms of the personality factors examined, this type
of research in important in that it keeps up with the field of personality research in HQ
psychology which has “moved beyond the search for significant correlations between Big

Five dimensions and general measures of job performance and is focused on

68

understanding in greater depth the nature of personality and job performance, and how
they are linked” (Johnson, 2003, p. 84-85).

A person who is conscientious is said to be self-disciplined, organized, reliable,
planful, responsible, systematic, and persevering, among other things (Costa & McCrae,
1992). Researchers propose that conscientiousness is comprised of two key elements,
one that is based on responsibility or dependability and another that centers around
volition (Barrick & Mount, 1991; Costa & McCrae, 1992). Characteristics such as
responsible, diligent, organized, careful, reliable, dependable, and many more (Costa &
McCrae, 1992) clearly highlight the role of responsibility. On the other hand,
characteristics such as persevering and hard working reflect the aspect of will or volition.
There is a history of meta-analytic research on conscientiousness that links it to a number
of desirable work outcomes such as job performance (Barrick & Mount, 1991); goal-
setting, self-efficacy, and expectancy motivation (Judge & Ilies, 2002); motivation to
learn (Colquitt & Simmering, 1998); and goal commitment (Gellatly, 1996). While this
type of research is interesting in that it highlights the potentially positive benefits of
having employees who are high on conscientiousness, there is still not much known with
regard to how conscientiousness may influence performance in rapidly changing
environments.

One study that directly examined the effects of conscientiousness on adaptive
performance was conducted by LePine et al. (2000). Due to the high levels of
commitment, motivation, and perseverance associated with high conscientiousness,
LePine et hypothesized that conscientiousness would be a be a better predictor of

decision making accuracy on a task once the rules change in an unforeseen manner than

69

before. Their results, however, ran counter to this expectation and showed that high
levels of conscientiousness were negatively related to decision making accuracy after a
change in decision rules. LePine et a1. (2000) attributed this finding to the fact that, when
broken apart, high levels on the dependability or responsibility aspects of
conscientiousness were negatively related to adaptive performance on their task.
However, scores on the volitional or achievement facets were not significantly related to
adaptive performance in either direction.

It is important to note here, however, that the type of task LePine et al. (2000)
used to examine adaptive performance essentially involved a coordinative complexity
change. Decision rules were changed in a manner that was not obvious to participants,
and it was up to them to discover the new rules and make correct decisions based upon
them. This type of change, as described above, should be quite cognitively taxing and
even though people with high levels of conscientiousness should be committed and
motivated to perform well on a task, the change may be too much for them deal with if
they rely on motivation alone. However, it may be the case that this type of comrrritrnent
and motivation is ideally suited for promoting high levels of adaptive performance for
other types of task change. Recall that adaptation to task difficulty changes is posited to
primarily occur through a motivation pathway in which extra effort and perseverance are
likely to lead to higher levels of adaptive performance, as proper strategy selection is
based on working harder and faster to apply existing knowledge and skills. With this in
mind, and based on prior arguments, as one goes from task difficulty changes, to
component task changes, to coordinative task changes, and the cognitive complexity of

the change and requisite information processing demands increase while the direct

7O

motivational impacts decrease, conscientiousness may become less strongly positively
related to, or even negatively related to adaptive performance. This leads to the
following hypothesis:

Hypothesis 8: Conscientiousness will differentially predict adaptive performance

effectiveness in response to different types of task change.

A second personality factor that has received attention as a predictor of adaptive
performance, also primarily by LePine et al. (2000), is openness to experience. Openness
to experience involves traits such as being broad-minded, curious, imaginative, and
original (Barrick & Mount, 1991; Costa & McCrae, 1992). People with high levels of
openness to experience are thought to be very intellectually curious, fond of finding new
ways of doing things, and interested in seeking out novel experiences (LePine et al.,
2000). Furthermore, open individual have been found to engage in higher levels of self-
monitoring (Blickle, 1996; Busato, Prins, Elshout, & Harnaker, 1999), which is suggested
to be important for detecting change, monitoring performance, and learning when task
elements change (LePine et al., 2000). Although openness has not generally been found
to have direct relationships with job performance across a wide variety of jobs (Barrick &
Mount, 1991), findings that those high in openness perform better in training contexts
(Banick & Mount, 1991) indicate that openness may be important for both knowledge
and skill acquisition in training, and potentially for new knowledge acquisition and
strategy development when task parameters change.

Using this type of rationale, LePine et al. (2000) investigated the effects of
openness to experience on decision making task. They hypothesized that because those

high in openness to experience tend to be more creative and receptive to change (Costa &

71

llcl

posi

£011!

in:

SITE

ll]

McCrae, 1992; King, Walker, & Broyles, 1996), openness to experience would be more
positively related to decision making performance after an unforeseen change in the task
context than before. Indeed, their results suggested that this was the case, with people
high in openness making more accurate decisions after task decision rule changes than
those low in openness. While not explicitly examining adaptive performance, a study by
Thoresen, Bradley, Bliese, & Thoreson (2004) found similar results in that openness to
experiences was positively related to job performance level and growth for salespeople
during job transitional periods, which were discussed as situations where methods and
strategies were undefined and employees had to potentially learn new skills (Murphy,
1989; Thoreson et al., 2004). .

If the mechanism for openness to influence adaptive performance truly involves
embracing novel experience, intellectual curiosity, and the development of new and novel
task strategies, then it may be the case the openness best predicts adaptive performance
effectiveness on task changes that require a great deal of strategy manipulation and
reformulation. Recall that LePine et al.’s (2000) task primarily involves coordinative
task changes, where decision rules are manipulated without notifying participants. In this
case, the “cognitive playfulness” of those with high openness to experience may help
them not only to discover, but also to embrace and react to changes in decision rules,
thereby spurring them to develop new strategies and eventually adapt more successfully
to task change. In situations where adaptive performance effectiveness is more
motivationally driven, however, this playfulness or curiosity may not have a chance to
manifest itself, and high levels of openness to experience may not necessarily predict

adaptive performance effectiveness. Based on this, the following hypothesis is offered:

72

Hypothesis 9: Openness to experience will difi’erentially predict adaptive

performance eflectiveness in response to dzflkrent types of task change.

73

Method

Participants

The participants for this study were 378 Michigan State University undergraduate
students recruited via the MSU Psychology student subject pool. Due to study piloting,
incomplete data, and technical problems, the final sample included 283 participants, with
a range of 43 to 50 participants per condition. Participants in the analyzed data ranged
from 18 to 30 years of age (M = 19.6 years) and were 54.8 % female. Participants were
volunteers and their participation was contingent upon informed consent (see Appendix B
for consent form). Each participant received course credit commensurate with the length
of his or her participation in the study.

Design
The foundation of the study is a 2 x 3 between subjects design in which both the

type of task change being faced (difficulty change, component change, or coordinative
change) and the provision of a metacognitive intervention were orthogonally
manipulated. In practice, this design operated much like three separate studies, as mean
comparisons across the difficulty change conditions were not of primary interest.
Participants were randomly assigned to conditions on the basis of the experimental
sessions that they attended.
Task

A version of the PC-based radar tracking simulation program TANDEM (Dwyer,
Hall, Volpe, Cannon-Bowers, & Salas, 1992) was used for the study. TANDEM serves
as a decision-making task in which participants must generally defend a geographic area
by gathering information about a number of different targets and using that information

to make engagement decisions regarding whether or not to attack each target. More

74

ppm

will

on i:
C163
rec:
dc:
‘- 1

PM

C01

OI

specifically, participants saw an emulated radar station console in which multiple targets
could initially be seen. Once the simulation started, participants selected specific targets
and gathered information cues regarding three different characteristics of each target:
Type, class, and intent. Using decision rules that could be acquired throughout the
training period, participants then used the information cues to classify each target based
on its type, class, and intent and finally decided whether to engage (i.e. shoot down) or
clear each target from the screen depending on the given classification. Participants
received points toward their overall score based on the success of these engagements
decisions, as well as other factors such as target prioritization, which will be discussed in
further detail below.

Procedure

Upon registering for the experiment, participants gave informed consent and
completed an online questionnaire that included measures of demographic information,
ACT /SAT scores, leanring goal orientation, and personality. Participation in the
experiment was contingent on completing this questionnaire. Once the questionnaire was
completed, participants were able to sign up and attend an experimental session.

When participants arrived at their scheduled session, they first received an initial
introduction to the TANDEM task. They were informed that they would progress
through twelve three-minute trials in which their goal was to do their best to perform well
on the task. Each task trial was preceded by a two-minute period in which the individuals
could study an online task manual that contained requisite declarative knowledge. Also,

each task trial was followed by a two-minute period where participants had an

75

opportunity to view feedback on numerous aspects of their performance and effectiveness
for the preceding trial.

After the first six trials (i.e. practice or training trials), the metacognition and task
change interventions were implemented for participants in the appropriate conditions.
Participants then completed a set of six adaptation trials in which their instructions for the
task were generally the same and the structure of the trials (i.e. view manual, perform on
the simulation, view feedback) was the same. Because change detection is a critical part
of the theoretical model for this study, participants were not informed that a task change
would occur. Rather, it was entirely up to them to discover this change and take
appropriate actions (i.e. study the manual, practice, view feedback) to adapt to the
changes.

It should also be noted that the task manual and feedback were changed at this
time to include new elements and information relevant to each of the different types of
task change that participants could face (i.e. difficulty, component, or coordinative).
Although direct comparison between conditions was not a primary concern of this study,
and therefore this was not strictly essential for the study design, this was done to more
closely resemble actual situations where multiple sources and stores of knowledge exist
that pe0ple can attempt to tap into in order to adapt to task change. If only the requisite
information for adapting to the given type of task change exists, the implications of
successful metacognition and self-regulation as well as the importance of processes
related to diagnosis and strategy adaptation may be minimized. However, when multiple
types of new information exist that can be discovered, learned, and leveraged into action,

the processes of proper change diagnosis and strategic adaptation, as well as the

76

implication of proper metacognition, likely take on a much more important, and realistic,
role.

Upon completion of the sixth and final adaptation trial, participants completed a
questionnaire that contained the task change detection probes, the declarative knowledge
quiz, and the metacognitive intervention manipulation check items. Following
completion of the questionnaire, participants were debriefed (see Appendix C for
debriefing form) and thanked for their participation in the study.

Manipulations

Task change type. As described earlier, the task change manipulation focused on
three different types of task change with the goal of orthogonally manipulating each
between-subjects. The first type of task change is task difliculty, which can be thought of
as the total number of behavioral acts that one has to engage in to complete a task in a
given amount of time. Recall that Wood (1986) discusses behavioral acts as patterns of
behavior with a particular purpose in mind such as “lifting a box” or “printing a
document”. In TANDEM, the primary behavioral act is “prosecuting a target”, which
deals with collecting information about targets and making decisions about their class,
type, and intent as well as whether or not to destroy them. To increase task difficulty in
TANDEM, then, one must increase the number of required behavioral acts, which can be
done most elegantly by simply increasing the number of targets that participants need to
prosecute. The task difficulty change manipulation in this study focused on doing exactly
that, by increasing the number of targets in the adaptation trials by 50% (from 12 to 18).

The second type of change, component task change, is rooted in Wood’s (1986)

notion of component task complexity, which can be thought of as number of distinct acts

77

and information cues that need to be dealt with in a particular task. To increase
component complexity for a given task, then, one would need to increase the number
distinct acts and/or cues that people need to engage in or process during the performance
of a task. For TANDEM, this increase could involve a number of different things. The
manipulation chosen here, however, focused specifically on new information cues that
individuals must process in order to correctly prosecute targets. Specifically, this
manipulation involved adding new informational cues that need to be used to help
ascertain the type, class, and intent of each target. Upon implementation of the
manipulation, these cues were completely unique and novel to participants, and thus
signified a change in component task complexity. The old information cues remained,
however, but they were given values that were “out of range” and did not conform to the
decision rules used to make class, type, and intent decisions, therefore rendering them
useless.

The final type of task change, coordinative task change, is based up on Wood’s
(1986) definition of coordinative complexity. As stated earlier, coordinative complexity
refers to the types of relationships between different aspects of the task and the timing or
order they need to be performed in. Based upon this, changes in coordinative complexity
required individuals to learn different sequences or configurations of behavioral acts to
reach the same levels of performance. Given that prosecuting a target constitutes the
primary behavioral act in TANDEM, a coordinative task change must involve altering the
sequence or configuration of prosecuting targets in some way. In this study, the
importance or “threat level” of each target was the same during the training periods.

Thus, the ordering or sequence of prosecuting targets took on no real importance for

78

determining one’s level of performance effectiveness. However, altering this level of
importance or threat for different targets should give higher levels of priority to certain
targets, and thus necessitate that they take precedent. In this study, the importance or
“threat level”, and thus the sequencing of target prosecution decisions, was altered in two
ways. First of all, a defensive perimeter was activated in the task that imposed a penalty
each time a target crossed it. Thus, targets closing in on the defensive perimeter took
higher precedent than others, as they harmed performance by taking away points from
one’s overall score if they crossed the defensive perimeter. Secondly, some of targets

. were changed from normal targets to “pop-up” targets. These targets did not initially
appear on the radar screen, but instead they “popped-up” close to the defensive perimeter
sometime during the trial. These targets should take immediate precedent over other
targets, as they moved toward the defensive perimeter immediately once they appeared
on the screen.

As stated earlier, these three manipulations were intended to be orthogonal. For
the task difficulty change, additional targets were added, but they involved only task cues
and behavioral acts that are already learned and did not impose additional sequencing
requirement, as they all carried the same level of importance. For the component task
change, the number of targets was kept constant, but novel informational cues needed to
be used to correctly prosecute the targets. This type of change also had no impact on how
behavioral acts are sequenced, as each target retained the same level of priority or threat.
Finally, for the coordinative task change, new sequencing requirements were imposed by

activating a defensive perimeter and changing some targets from the standard to the pop-

79

up variety. However, both the overall number of required behavioral acts or targets and
the informational cues needed to correctly prosecute the targets remained the same.

Metacognition intervention. The metacognitive intervention was designed to help
participants successfirlly use metacognitive control processes to regulate and control their
behavior and cognition during adaptation; it contained two parts. The first part of the
intervention (Appendix D), which was conducted prior to adaptation trials 1 and 4,
involved providing the participants with a brief overview of the important aspects of
metacognition and informing them of the potential usefulness of making plans,
monitoring their levels of knowledge and understanding, and evaluating the success of
the different strategies they may use while going through the web-page training program.
This was designed to aid participants in forming an accurate representation of the
potential usefulness of metacognitive control strategies

The second part of the intervention, which was conducted prior to adaptation
trials 2, 3, 5, and 6, involved having participants respond to a set of four open-ended
probe questions related to whether or not they were engaging in activities related to using
prior experience to make plans for upcoming learning and performance, monitoring
current levels of understanding and performance, and evaluating the success of
previously used strategies (see Appendix E for probes). The questions were similar in
nature to those found to be successful as drivers of metacognitive activity in previous
research (e.g. King, 1991b; Delclos and Harrington, 1991; Schraw, 1998). In addition,
these questions were presented after participants had an opportunity to review task-
related feedback for each trial, not directly after individuals performed the task. This

approach is consistent with past work that shows that delaying self-questioning can lead

80

to more accurate assessments of current levels of understanding and performance than
self—questioning done immediately after covering new information (Nelson & Dunlosky,
1991), and this should be especially true in this case given that participants had the
opportunity to view veridical, descriptive feedback regarding their previous performance
prior to responding to these questions.

Measures

Control variables. Participants were asked to provide demographic information
related to their gender. Also, computer and video game experience was assessed using a
single item, “How often do you play video games”, with responses given on a 5-point
scale ranging from “never” to “very frequently”.

Cognitive ability. Cognitive ability was assessed via self-reports of participants’
ACT or SAT college admission test scores. Twenty-one participants in the analyzed data
did not provide usable ACT or SAT scores, and in these cases mean substitution was
utilized. College admission test scores are generally considered to be valid indicators of
general cognitive ability (Schmidt, 1988; Schmidt & Ford, 2003). Furthermore, research
has shown that self-reports of test scores tend to correlate very strongly with actual test
scores (Gully, Payne, Kiechel, & Whiteman, 1999).

Learning orientation. Participants completed a five-item scale of trait learning
orientation adapted fiom VandeWalle (1997). Appendix F lists these items. The
responses to these items were made using a 5-point scale ranging from “strongly
disagree” to “strongly agree”.

Conscientiousness and openness to experience. Conscientiousness and openness

to experience were measured with scales from NEO—PI-R personality inventory (Costa &

81

McCrae, 1992). Each scale included forty-eight items that are thought to capture the
dimension as a whole as well as sub-facets underlying each dimension. Appendices G
and H list the conscientiousness and openness to experience items, respectively. The
responses to these items were made using a 5-point scale ranging from “strongly
disagree” to “strongly agree”.

Detection. Change detection was assessed by a self-report measure. Participants
in all conditions responded to a question aimed at ascertaining if a change in the task was
detected (i.e. “Did you notice a change in the TANDEM task you were working on?”).
This measure served as the primary measures of detection, and it was administered at the
conclusion of the study to avoid priming participants to look for task change.

Declarative task knowledge development. Relevant declarative task knowledge
development was suggested earlier to be a key outcome of processes related to diagnosis.
General declarative task knowledge development was assessed by a 16-item multiple-
choice quiz administered at the conclusion of the study (see Appendix I for items).

The relevance of task knowledge was thought to vary based on the type of task
change that each participant faced. For difficulty task changes, relevant declarative
knowledge development was operationalized as knowledge regarding the number of
targets that were in the simulation during the adaptation trials, and was assessed by one
multiple-choice item. For component task changes, relevant declarative knowledge
development was operationalized as knowledge regarding the values of the new
informational cues that they encountered as well as how to make class, type, and intent
decisions based upon them, and it was assessed by three multiple-choice items. Finally,

for coordinative task changes, relevant declarative knowledge development was

82

operationalized as knowledge regarding correct target prioritization such as where the
defensive perimeter was, changes in scoring rules or target threat levels based on
defensive perimeter crossings, and what pop-up targets were. This was be assessed by
four multiple-choice items.

Appropriate strategy use. As described above, the appropriate strategies for
adaptive performance were suggested to vary based upon the type of task change that
people faced. For task difficulty changes, the appropriate strategy was thought to be
primarily motivational or effort-based and should focus on engaging in the highest
number of behavioral acts possible. Thus, for difficulty changes, appropriate strategy use
(i.e. increased task effort) was operationalized as the overall number of targets that
participants prosecuted. For component and coordinative task changes, effort was also
thought to be important, but appropriate cognitive strategies are thought to take on higher
levels of importance. When participants faced a component task change, they had a
number of new information cues to learn about and master in order to make correct type,
class, and intent decisions. Thus, appropriate strategy use for component task changes
was operationalized as the nrunber of correct class, type, and intent decisions that were
made. When participants faced a coordinative task change, it was though that they would
need to learn about the importance and meaning of the defensive perimeter and how it
influenced target prioritization. Thus, appropriate strategy use for coordinative task
changes involved sequences of behavioral acts (i.e. target prosecutions) and was
operationalized as the number of high priority and pop-up targets that were engaged, as

these targets represent the highest levels of threat at any given point in time. The

83

measures of each of these operationalizations were taken directly from the simulation for
performance during the final adaptation trial.

Adaptive performance eflectiveness. In all cases, performance effectiveness was
assessed using performance composite scores generated by the TANDEM simulation.
These scores were based on correct target prosecution as well as defensive perimeter
intrusions when applicable. For the practice trials, each participant’s overall score was a
function of receiving 100 points for every correct target prosecution, whereby all four of
the decisions for the target (i.e. type, class, intent, and final engagement) were made
correctly, and losing 100 points for every incorrect target prosecution, whereby one or
more of the decisions for the target was incorrect. This same scoring scheme applied for
the adaptability trials in the difficulty and component task change conditions. For the
coordinative task change condition, the activation of the inner defensive perimeter had
implications for scoring. Participants still received 100 points for each correct target
prosecution and lost 100 points for each incorrect target prosecution. However,
participants also lost 100 points for each target that crossed the defensive perimeter
without being prosecuted. The measure of adaptive performance effectiveness used for
necessary hypothesis tests was taken fi'om the final adaptation trial for each condition.

Metacognition intervention manipulation check. After the final adaptation trial,
all participants were asked to respond to a series of four questions regarding whether or
not they were instructed to focus on metacognitive activities related to planning,
monitoring, and evaluating throughout the adaptation trials. Appendix J lists these items.
The responses to these items were made using a 5-point scale ranging from “strongly

disagree” to “strongly agree”.

84

Results

Means, standard deviations, intercorrelations, and alpha reliabilities (on diagonal
when applicable) for the variables of interest across the entire sample are shown in Table
1. Similarly, Means, standard deviations, and intercorrelations within task change
condition for difficulty, component, and coordinative task changes are reported in Tables
. 2, 3, and 4, respectively.

Control Variables

Gender, computer experience, and practice performance effectiveness (taken fiom
trial 6) were used as control variables for tests of each of the hypotheses. Furthermore,
task effort (number of targets engaged) was included as a control variable for tests of
hypotheses 1b and 1c.

Analysis Plan

There are three primary types of hypotheses in this study: (1) direct effect
hypotheses, (2) interaction or moderation hypotheses, and (3) mediation hypotheses. A
description of how each of these types of hypotheses was examined, as well as the
specific hypotheses that fall into each category, is provided below.

The direct effect hypotheses in this study primarily deal with the influence of
different parts of the proposed adaptation process on one another (i.e. detection,
diagnosis, adaptation). For example, hypothesis 1b suggests that when facing a
component task change, the correct use of new information cues and/or acts should
positively predict adaptive performance. Other hypotheses of this type include
hypotheses 1c, 2a, 2b, 2c, 3a, 3b, and 3c. Because these direct effect hypotheses are

concerned with linear relationships between two variables, a separate hierarchical linear

85

regression analysis was conducted to test each of these whereby the dependent variable of
interest was regressed onto the control variables at step one and the independent variable
of interest at step two. For each test, the given null hypothesis was rejected if the
unstandardized regression coefficient was statistically significant at a = .05, one-tailed.

The interaction or moderation hypotheses in this study deal with the
multiplicative or interactive effects of two independent variables on a given dependent
variable. Hypotheses 1a, 4, 6, 7, 8, and 9 all fall into this category, and were tested using
similar approaches. For each of these hypotheses, a separate hierarchical linear
regression analysis was conducted whereby the dependent variable of interest was
regressed onto the control variables at step one, the independent variables of interest at
step two (which were centered when possible), and the interaction terms (product of the
independent variable and moderator variables) at step three. Task change type, which
effectively served as the moderator in each of these hypotheses, was dummy coded into
two variables with the first indexing differences between the difficulty and component
change conditions and the second indexing differences between the difficulty and
coordinative change conditions. For each test, the given null hypothesis was rejected if
the A R2 from step two to step three is statistically significant at a = .05, two-tailed. For
any significant interactions, the slope differences depending on task change type were
examined in accordance with procedures outlined by Aiken and West (1991).

The mediation hypotheses, in this case hypotheses 5a, 5b, and So, were tested
using procedures outlined and discussed by James and Brett (1984) and Baron and Kenny
(1986). This involved a series of four analyses to show that (A) the independent variable

of interest is related to the outcome of interest, (B) the independent variable of interest is

86

related to proposed mediator(s), (C) the proposed mediator(s) are related to the outcome,
and (D) the effect of the independent variable of interest no longer exists when the effect
of the mediator(s) on the dependent variable are controlled for. For each of the mediation
hypotheses, hierarchical linear regression analyses was used to test for each of these
components, with the first step predictors always being the aforementioned control
variables.

For tests of the moderation hypothesis (1 a, 4, 6, 7, 8, and 9), variables related to
appropriate strategy use (i.e. task effort) and adaptive performance were standardized
within conditions for task change type for consistency of interpretation across conditions.
For all other hypothesis tests, which were done within one level of the task change
condition only, all variables were left in their original metrics for ease of interpretation.
Hypothesis 1

In general, hypothesis 1 suggested that appropriate strategy change should
positively predict adaptive performance effectiveness once a change in the task occurs.
Based upon this, three specific sub-hypothesis were generated in consideration of the
different types of task change that participants could have faced.

Hypothesis 1a suggested that task related effort, operationalized as the number of
targets prosecuted, would be most strongly positively related to adaptive performance
effectiveness when people were facing changes in task difficulty, as opposed to
component or coordinative task changes. Consistent with expectations, and as shown in
Table 5, the results of the analysis of hypothesis 1a showed a significant interactive effect
of task related effort and task change type on levels of adaptive performance (Step 3

AR2= .02, p < .05).

87

Figure 5 shows the plot of this significant interaction effect. To further probe this
interaction, the differences in slopes for the relationship between task related effort and
adaptive performance for different types of task change was investigated by examining
the unstandardized regression weights for the two interaction terms, which provide
explicit tests for the slope differences between the index condition (i.e. difficulty change)
and the other conditions (Aiken & West, 1991). As can be seen in Table 5, the strength
of the relationship between the number of targets prosecuted and adaptive performance
effectiveness is significantly different for the difficulty and coordinative change
conditions (b = -.34, p < .05). However, the strength of the relationship between the
number of targets prosecuted and adaptive performance effectiveness was not
significantly different for the difficulty and component change conditions (b = -.03, p =
.77). Based on these findings and the examination of Figure 6, it can be concluded that
task related effort was least strongly related to adaptive performance effectiveness when
people faced coordinative task changes, which provides partial support for Hypothesis 1a.

Hypothesis 1b predicted that when facing a component task change, the correct
use of new information cues and/or acts would positively predict levels of adaptive
performance. Results were consistent with this hypothesis. As shown in Table 6, the
number of correct type, class, and intent (TCI) decisions made positively predicted
adaptive performance effectiveness when people were facing a component task change (b
= 159.31, p < .05).

Finally, hypothesis 1c predicted that when facing a coordinative task change, the
correct sequencing and prioritization of behavioral acts would positively predict levels of

adaptive performance. As shown in Table 7, the findings did not support this hypothesis.

88

The number of high priority targets that were engaged did not significantly predict
adaptive performance effectiveness when people were facing a coordinative task change
(b = -52.07, p = .10).

Hypothesis 2

In general, hypothesis 2 suggested that-task appropriate in situ declarative
knowledge development would positively predict appropriate strategy selection and
implementation. Based upon this, three specific sub-hypothesis were generated in
consideration of the different types of task change that participants could have faced.

Hypothesis 2a focused on task difficulty changes and proposed that when facing a
task difficulty change, declarative knowledge development regarding increases in the
number of required behavioral acts would positively predict task related effort. The
results of the test of this hypothesis are shown in Table 8. Briefly, it can be seen that
increases in knowledge regarding the number of targets in the simulation did not
significantly predict the number of targets that individuals engaged when facing a task
difficulty change (b= .88, p = .14)

Hypothesis 2b suggested that when facing a component task change, declarative
knowledge development regarding new information cues or necessary acts would
positively predict the correct strategic use of new information cues and/or acts. As
shown in Table 9, the findings supported this hypothesis. More specifically, declarative
knowledge development regarding new informational cues positively predicted the
number of correct type, class, and intent decisions participants made when facing a

component task change (b = 11.45, p < .05).

89

dc:
hi1

C01

de

(l:

Hypothesis 2c predicted that when facing a coordinative task change, declarative
knowledge development regarding changes in task rules and contingencies would
positively predict the correct sequencing and prioritization of behavioral acts. As shown
in Table 10, the results were consistent with this hypothesis. When facing a coordinative
task change, levels of participant declarative knowledge development regarding changes
in task rules and contingencies positively predicted the number of high priority targets
that were engaged (b = .18, p < .05).

Hypothesis 3

Hypothesis 3 suggested that change detection would positively predict appropriate
declarative knowledge development for the given type of task change. Similar to
hypothesis 1 and hypothesis 2, three specific sub-hypothesis were generated in
consideration of the different types of task change that participants could have faced.

Hypothesis 3a suggested that when facing a task difficulty change, change
detection would positively predict declarative knowledge development regarding
increases in the number of required behavioral acts. This hypothesis was not supported.
As shown in Table 11, change detection did not significantly predict declarative
knowledge development regarding the increase in the number of targets in the simulation
(b = -0.l3, p = .14).

Hypothesis 3b predicted that when facing a component task change, change
detection would positively predict declarative knowledge development regarding new
information cues or necessary acts. The results of the test of this hypothesis are shown in

Table 12. Indeed, the results supported hypothesis 3b in that change detection

90

significantly and positively predicted the acquisition of declarative knowledge regarding
new informational cues (b = 0.28, p < .05).

Finally, hypothesis 3c predicted that when facing a coordinative task change,
change detection would positively predict declarative knowledge development regarding
changes in task rules and contingencies. As shown in Table 13, the results did not
support this hypothesis. Change detection did not significantly predict declarative
knowledge development regarding changes in task rules and contingencies (b = .41, p =
.16).

Hypothesis 4

Hypothesis 4 focused on the importance of a metacognitive intervention for
influencing adaptive performance effectiveness, presumably through its influences on key
in situ self-regulatory processes. It was suggested the influence of the intervention would
vary based on the level of cognitive “load” or demand that the type of change
necessitated, and thus, Hypothesis 4 predicted that a metacognitive intervention
conducted during the adaptation process would have the largest positive effect on
adaptive performance effectiveness in response to coordinative task changes and the
lowest positive effect on adaptive performance effectiveness in response to task difficulty
changes.

As stated earlier, hierarchical linear regression was used to test this hypothesis,
the results of which are shown in Table 14. As can be seen by the incremental variance
explained in step three of the regression, the interaction of the presence or absence of the
metacognitive intervention with the type of task change did not significantly influence

adaptive performance effectiveness (Step 3 AR2 = .00, p = .52). Furthermore, there was

91

no significant main effect of the metacognitive intervention on adaptive performance
effectiveness across the different types of task change (b = -.17, p = .11 two-tailed).
Therefore, it appears that the metacognitive intervention had little to no direct influence
on adaptive performance effectiveness in response to the different types of task change
investigated in this study.

While failure to support this hypothesis is certainly a disappointment, it is not a
complete surprise when information regarding the metacognitive intervention
manipulation check is considered. As Table 1 shows, the mean on this scale across all
conditions was 3.85, suggesting that on average, people responded that they believed they
were told to focus on activities related to planning, monitoring, evaluating, and adapting
their strategies. In addition, and even more challenging, is the lack of a significant (r =
.05) correlation between the presence or absence of the metacognitive intervention and
scores on the metacognitive intervention manipulation check. Obviously, this is
problematic as it suggests that people were confused about the implementation of the
intervention and, thus, their reactions in accordance with it are likely not what was
expected. Regardless, the intervention was implemented in a sound fashion during the
study.

Hypothesis 5

Broadly, hypothesis 5 suggested that some combination of change detection,
relevant knowledge gain, and proper strategy use would mediate the effects of the
metacognitive intervention on adaptive performance for each type of task change that was
studied. This hypothesis was not formally evaluated, however. As shown by the results

of the tests for hypothesis 4, the first step in the mediation analysis, that the independent

92

variable of interest is related to the outcome of interest, did not receive support.
Furthermore, separate tests conducted to examine potential main effects of the
metacognitive intervention on adaptive performance effectiveness independently for each
type of task change returned similar results and that the metacognitive intervention was
not found to significantly influence adaptive performance effectiveness in response to
any of the types of task change. Therefore, it can be concluded that there are no direct
effect to be mediated.

However, it may be the case that any effects of the metacognitive intervention
have indirect influences on adaptive performance effectiveness that manifest themselves
through their influences on change detection, relevant knowledge acquisition, and/or
proper strategy use. In this case, step one of the proposed sequence of tests for
investigating mediation would not necessarily need to be satisfied. To investigate this
possibility, a series of analysis was conducted examining potential effects of the
metacognitive intervention on change detection, relevant knowledge acquisition, and
proper strategy use separately for each task change condition. However, as shown in
Tables 15 — 23, the results of these analyses were not statistically significant in any case,
thus suggesting that the metacognitive intervention had little to no effect on adaptive
performance effectiveness either directly, as shown above, or indirectly through
influences on change detection, relevant declarative knowledge acquisition, or proper
strategy use.

Hypothesis 6
Hypothesis 6 predicted that the influence of cognitive ability on adaptive

performance effectiveness would be moderated by task change type such that cognitive

93

ability would more strongly predict adaptive performance effectiveness in response to
coordinative and component task changes than to task difficulty changes.

The results of the hierarchical linear regression used to test this hypothesis are
shown in Table 24. Contrary to expectations, cognitive ability and task change type did
not interact to influence adaptive performance effectiveness (Step 3 AR2 = .01 , p = .26).
However, there was a statistically significant positive main effect of cognitive ability on
adaptive performance effectiveness across all types of task change (b = .03, p < .05). In
any case, hypothesis 6 was not supported, as the proposed moderating effects of task type
were not found.
Hypothesis 7

Hypothesis 7 predicted that the influence of leaming goal orientation on adaptive
performance effectiveness would be moderated by task change type such that learning
goal orientation would more strongly predict adaptive performance effectiveness in
response to coordinative and component task changes than to task difficulty changes.

The results of the test of this hypothesis are shown in Table 25. As can be seen,
learning goal orientation levels and task change type did not interact to influence adaptive
performance effectiveness (Step 3 AR2 = .00, p = .60). Furthermore, there was not a
statistically significant positive main effect of learning goal orientation on adaptive
performance effectiveness across all types of task change (b = .14, p = .11). Thus,
hypothesis 7 was not supported.
Hypothesis 8

Hypothesis 8 predicted that task change type would moderate the influence of

conscientiousness on adaptive performance effectiveness such that the effects of

94

conscientiousness would be positive for task difficulty change, and null or even negative
in relation to component and coordinative task changes.

Hierarchical regression results for the test of this hypothesis are shown in Table
26. This hypothesis was not supported. As can be seen, the interaction of
conscientiousness and task type did not significantly predict adaptive performance
effectiveness (Step 3 AR2= .00, p = .90). Furthermore, but as anticipated, there was not a
significant main effect of conscientiousness on adaptive performance effectiveness across
task change types (b = .01, p = .93 two-tailed).

Hypothesis 9

Hypothesis 9 predicted that task change type would moderate the influence of
openness to experience on adaptive performance effectiveness such that the positive
effects of openness to experience would be largest for coordinative task change and
smallest for component task change.

The results of the test of this hypothesis are shown in Table 27. It was found that
task change type and openness to experience did not significantly interact to influence
adaptive performance effectiveness (Step 3 AR} = .00, p = .88). Furthermore, there was
not a statistically significant positive main effect openness to experience on adaptive
performance effectiveness across all types of task change (b = .15, p = .15). Based on

this, it can be concluded that hypothesis 9 was not supported.

95

Discussion

The focus of this study was on extending existing work on adaptive performance
effectiveness. The prevailing view of work in current organizations is that it now takes
place in dynamic work environments where existing skills and knowledge constantly
need to be built upon (Ilgen & Pulakos, 1999; Smith, Ford, & Kozlowski, 1997, Cascio,
2003). Accordingly, researchers and theorists have noted a shift in what is considered to
be important for job performance, and adaptive performance has been suggested to be an
important component of many different types of current jobs (Ployhart & Bliese, 2006;
Pulakos et al, 2000). While research on a multitude of predictors of adaptive
performance has followed, the current adaptive performance literature is characterized by
a number of areas that are unexplored.

One problem with the current adaptive performance literature is that it has
primarily treated the construct as a static operationalization of performance after some
sort of task change occurs. This is reflected in both training and selection perspectives
and approaches to influencing adaptive performance. However, very little attention has
been given to adaptive performance as a process by trying to explicate what sorts of
things people have to do or go through to adapt to a change in the nature of a learned
task. Without recognizing some of the potentially critical process-oriented aspects of
adaptive performance, it is difficult to predict or understand how or why individual
differences, training techniques, or the like should influence adaptive performance.
Based on heuristic models of self-regulation, one critical focus of this study was to
provide a foundation for thinking about some of these more process-oriented aspects of

adaptive performance by building a conceptual model of adaptation as a process and

96

examining the effects of change detection, new relevant knowledge development, and
strategy change on adaptive performance.

A second focus of this study was on explicitly considering the nature of task
change that people were trying to adapt to. The extant work primarily treats adaptive
performance as performance on a task that is novel, ill-defined, more difficult, and/or
more complex than previous tasks that people have experience with. While it is
reasonable to expect that all of these types of task change require adaptability in some
form, until now there have been no specific attempts to delineate different components of
tasks that can change. Accordingly, three different types of task change were identified
based on Wood’s (1986) taxonomy of task complexity. F urtherrnore, and in accordance
with the first focus of this study as specified above, a number of hypotheses were
generated and tested regarding the importance of various cognitive and motivational
factors related to change detection, new knowledge development, and strategy change on
adaptive performance with the expectation that some factors would be more or less ‘
important in response to different types of task change.

This study also indirectly focused on the importance of in situ self-regulatory
processes during the process of adaptation. While recent research has highlighted the
potential value of self-regulatory processes that are thought to translate existing task
knowledge, skill, and efficacy into adaptive performance (i.e. Chen etal., 2005), there
has been little discussion of how or if these self-regulatory processes can be influenced.
There is, however, a stream of work related to metacognition that deals explicitly with
some of these self-regulatory processes and how different types of interventions can

influence them. Based on this, the current study examined the potential effects of a

97

metacognitive intervention on adaptive performance effectiveness in response to the
different types of task change that were identified.

The final focus of this study was on examining the influences of a number of
different individual difference factors on adaptive performance. Individual differences
factors related to cognitive ability, personality, and goal orientation have been found to
influence adaptive performance in previous studies (e. g. Ford et al., 1998; Kozlowski et
al., 2001), but the mechanisms through which they have their effects have not received
much attention. As an initial step in addressing these types of issues, the conceptual
process model of adaptation developed herein was used as a guide for developing
hypothesis aimed at assessing potential differential influences of these individual
differences on adaptive performance in response to different types of task change.
Findings and Specific Limitations

The hypotheses investigated in this study can be grouped into three general
themes. In accordance with the conceptual model of adaptive performance proposed
earlier, hypotheses 1, 2, and 3 focused on examining the effects of change detection,
relevant knowledge acquisition, and strategy effectiveness on one another and adaptive
performance effectiveness in response to different types of task change. The second
theme was reflected by hypotheses 4 and 5, which focused on the influence of a
metacognitive intervention on adaptive performance effectiveness in response to different
types of task change. Finally, hypotheses 6, 7, ’8, and 9 consider the potential effects of
individual differences in cognitive ability, goal orientation, and personality on adaptive

performance effectiveness in response to different types of task change. The following

98

,
or,

WD

135.

Iii

WC

discussion of the research findings and specific limitations related to each will be
organized around these three themes.

Hypotheses I, 2, and 3. Hypothesis I predicted that appropriate strategy change
would positively predict adaptive performance effectiveness across the different types of
task change examined in this study. More specifically, it was predicted that when facing
task difficulty changes, the pathway to successful adaptive performance effectiveness
would be primarily motivational and the appropriate strategy would be to exert additional
effort using what is already known about the task. However task effort alone was not
expected to directly predict success in adaptation to component and coordinative task
changes as strongly as it did for difficulty changes. For these types of changes, cognitive
factors related to proper strategy use were also expected to play important roles in
predicting adaptive performance effectiveness. For component changes, appropriate
cognitive strategy use was suggested to involve incorporating new task knowledge into
existing strategies, whereas the use of newly learned task rules and contingencies to make
correct sequencing and prioritization decisions was suggested to be the appropriate
strategic activity for coordinative task changes.

Hypothesis 1 was generally supported. Hypothesis 1a, which suggested that the
influence of task effort on adaptive performance effectiveness would be the strongest for
task difficulty changes and the weakest for coordinative task changes received support.
Consistent with the notion that task effort would not be as strong of a predictor of
adaptive performance effectiveness in response to changes that were more cognitive in
nature, these results (see Figure 6) show that task effort had its smallest effect on adaptive

performance effectiveness in relation to a coordinative complexity change. Also in

99

support of hypothesis 1, the results suggested that effective strategy use was positively
related to adaptive performance effectiveness in response to component task changes.
However, hypothesis 1c was not supported in that the measure of effective strategy use
was not significantly related to adaptive performance effectiveness in response to
coordinative task changes.

Taken as a whole, these findings provide some support for the notion that
different types of task changes require different types of strategies or different
“pathways” to success and also support the importance of in situ strategy development,
selection, and implementation for adaptive performance effectiveness. For difficulty
changes, it appears that effort alone was a strong predictor of adaptive performance
effectiveness. While effort also predicted adaptive performance effectiveness in response
to more cognitively-based types of task change, proper strategy use also had an effect in
some cases. These findings support the earlier assertion that when thinking about
predictors of adaptive performance effectiveness, it is important to explicitly consider the
type of task change that individuals are facing, as the appropriate strategies (e. g. “work
har ” vs. “work smart”) may vary.

However, some caution is necessary when interpreting these results. An
examination of Tables 2, 3, and 4 show that the correlations (and thus the corresponding
R2 values) between the strategy variables and adaptive performance effectiveness were
very high for both the difficulty and component task change conditions (r = .93, .88,
respectively). While correlations this high are not generally observed, it is not
unreasonable in this context as the operationalizations of appropriate strategies in each of

these conditions were behaviors that were coupled almost directly to performance

100

effectiveness. For example, in the difficulty change condition the appropriate strategy
was to exert extra effort, which primarily consisted of using the same knowledge and
skills developed during the practice period, but more frequently. Since the practice
period was designed to build high levels of competence, it comes as no surprise that the
relationship between the total number of targets prosecuted (both correct and incorrect)
and adaptive performance effectiveness was so high, as there were likely very few errors
by participants after twelve trials. Of course, it is not always reasonable to expect this
tight of a coupling between strategy use and performance effectiveness, especially when
strategies are complex, as there are numerous environmental and task constraints such as
available resources, technology, etc. that may constrain adaptive performance
effectiveness even when people are able to devise appropriate strategies.

Furthermore, the lack of a relationship between the operationalization of proper
strategy use and adaptive performance effectiveness for the coordinative task change
condition is surprising, and somewhat suspect. Given that this was the condition where
cognitive factors and, presumably, proper strategy development would be most
important, this finding seems a bit off base. However, a closer examination of the
operationalization of proper strategy use for this condition reveals a potential issue that
could be responsible for the lack of significant findings. As stated earlier, this
operationalization focused on the overall number of high priority and pop-up targets that
were engaged, as these targets had the highest priority at any given point in time.
However, this measure did not specifically look at the sequence of high priority target
prosecution or the time it took to prosecute the high priority targets. Thus, the actual

sequencing of target prosecutions was not directly examined, and a “brute force”

101

approach of simply engaging targets as quickly as possible may have resulted in a similar
score on this variable as a more complex strategic approach that actually considered the
importance of sequencing engagements based on priority levels. To address this,
potential alternative operationalizations of proper strategy use for the coordinative task
change condition could be examined. For example, it is possible to examine the sequence
in which targets were prosecuted in order to ascertain whether the “pop-up” high priority
targets were given precedent once they appeared on the screen. Another alternative
operationalization involves examining not only how many of the high priority targets
crossed the defensive perimeter, but also how long these targets spent there before they
were prosecuted. These operationalizations could both help to provide richer information
regarding when the high priority targets were engaged, and thus may index how strategic
people were in prosecuting targets.

Hypothesis 2 predicted that task relevant in situ declarative knowledge
development would positively predict appropriate strategy selection and implementation.
While the necessary relevant knowledge was suggested to vary based on the type of task
change people faced, the general prediction was expected to hold across all different
types of task change.

For the most part, hypothesis 2 was supported. As predicted, relevant declarative
knowledge gain was positively related to proper in situ strategy use and development in
response to both component and coordinative task changes. This was not the case for
task difficulty changes, however, as declarative knowledge development was not related

to increased levels of task effort.

102

One potential explanation for the lack of a significant finding in the task difficulty
change condition has to do with the measure of knowledge acquisition. As shown in
Appendix I, the item that assessed relevant knowledge acquisition for the difficulty
condition focused on knowledge of the absolute number of targets in'the simulation
rather than the relative number. It could be the case that participants detected a change,
figured out that were more targets, and correspondingly increased their effort without
actually knowing the exact number of targets in the then-changed simulation scenario.
However, a follow-up test using an alternative operationalization of relevant knowledge
acquisition as knowledge that there are more targets (i.e. 18 or 25) rather than knowledge
of the absolute number of targets did not yield any differences in the pattern of results.

However, upon further examination it appears that the lack of findings related to
both operationalizations of relevant knowledge acquisition for the difficulty task change
may be explained by the relative lack of people who did not realize that there were more
targets. Indeed, 85% of people in the difficulty task change condition correctly
identified the absolute number of targets in the adaptation trials, with an additional 12%
(e. g. 97% total) identifying that there were more targets, even if they did not correctly
identify the absolute number. Furthermore, the original operationalization correlated
significantly with practice performance (r = .21 , p < .05), which in turn was a p6sitive
predictor of task effort levels. This suggests that even though there was a marginal level
. of variance regarding whether or not people correctly identified the absolute number of
targets in the difficulty change adaptability trials, the collinearity between this variable

and practice performance, as well as the fact that practice performance was entered first

103

in the regression model, may have made it difficult to find a statistically significant
relationship between this variable and task related effort.

Taken together, however, these findings provide some support for the importance
of new knowledge development for influencing proper strategy use and development
when trying to adapt to changes in the nature of one’s task. This new knowledge
development is thought to be a manifestation of processes related to diagnosis, which
focuses on ascertaining the cause or nature of change in the given task. When people are
aware that a task change of some sort has occurred, the model presented earlier specifies
that it is imperative that they attempt to understand the nature of the change before
successful adaptation can occur. As these findings suggest, the development of new
knowledge regarding additional necessary behavioral task elements or acts, informational
cues, and sequencing requirements can allow individuals to devise and act in accordance
with proper task strategies.

Hypothesis 3 suggested that change detection would positively predict appropriate
declarative knowledge development for the given type of task change. Change detection
has been suggested to be a key predictor of adaptive performance effectiveness (Ployhart
& Bliese, 2006), as one should not be able to learn about a change and adopt the
necessary strategies to perform well in response to it if there is no knowledge that a
change in the task actually occurred. Here, it was suggested that the effects of change
detection would manifest through diagnosis-related processes that lead to relevant task
knowledge acquisition. As with hypothesis 2, the type of relevant knowledge examined
here was expected to vary based on the type of task change faced, but the general

hypothesis was expected to hold across different types of task change.

104

Hypothesis 3 was only supported for the component task change condition. Here,
it was found that change detection was positively related to knowledge acquisition
regarding changes in the distinct information cues that needed to be used to prosecute
targets. Change detection was not related to knowledge acquisition regarding the
increased number of targets in the difficulty change condition or knowledge acquisition
regarding target prioritization and sequencing information in the coordinative change
condition.

While the results of hypothesis 3 are not generally supportive, there may be a
number of reasons why the relationships between change detection and relevant
knowledge acquisition were not significant for the difficulty and component change
conditions. First of all, there was little variance in whether or not participants reported
noticing a change in the task. In the difficulty, component, and coordinative conditions,
the percentages of participants who reported noticing a task change were 88%, 97%, and
92%, respectively. This comes as no surprise, as the nature of the task and the way the
study was constructed in the laboratory made change detection a relatively
straightforward process for all conditions. This lack of variance, then, makes finding
relationships difficult. However, a significant relationship between change detection and
relevant knowledge acquisition was found for the component change condition, so this
explanation is not completely sufficient.

Hypotheses 4 and 5. Hypothesis 4 focused on the importance of a metacognitive
intervention for influencing adaptive performance effectiveness. It was suggested that
the influence of the intervention would vary based on the level of motivational and

cognitive “load” or demand that the type of change necessitated. Therefore, it was

105

expected that the metacognitive intervention would have the largest positive effect on
adaptive performance effectiveness in response to a coordinative task change and the
smallest positive effect on adaptive performance effectiveness in response to a task
difficulty change.

The results of the study failed to support this hypothesis, though. The findings
suggested no differential relationships between the metacognitive intervention and
adaptive performance effectiveness across the different types of task change.
Furthermore, the metacognitive intervention did not have an omnibus positive main effect
on adaptive performance effectiveness, either, thus suggesting little influence of the
intervention.

Hypothesis 5 focused on the potential mediating effects of change detection,
relevant knowledge gain, and proper strategy use on the proposed relationships between
the metacognitive intervention and adaptive performance effectiveness as specified in
hypothesis 4. More specifically, it was suggested that since the pathways to success
when moving from difficulty to component to coordinative task changes should become
increasingly cognitive loaded, the relationship between the metacognitive intervention
and adaptive performance effectiveness would be fully mediated by change detection by
itself for difficulty task changes; change detection and relevant knowledge gain for
component task changes; and change detection, relevant knowledge gain, and effective
strategy change for coordinative task changes. However, since the initial conditions for
examining mediation tested in hypothesis 4 received no support, this hypothesis was not

formally evaluated.

106

The results of hypotheses 4 and 5 generally run counter to existing findings that
have shown the ability of metacognitive interventions to influence learning and problem
solving (e. g. Brown & Palincsar, 1982, 1985; Delclos & Harrington, 1991; King, 1991b).
However, they are not entirely inconsistent with work studying the influences of
metacognitive interventions on task performance or adaptive performance effectiveness.
For example, Schmidt and Ford (2003) found no direct effects of their metacognitive
intervention on training performance, although some indirect effects through
metacognitive activity and declarative knowledge development were found. Perhaps
more applicable to this study are the findings of Keith and Frese (2005). Recall that
Keith and Frese found no incremental effect of a metacognitive intervention on adaptive
transfer over a general error-based training intervention. Rather, their results suggested
that the error-based training in and of itself lead to higher levels of metacognition because
participants had the freedom to explore the task, make mistakes, and learn from any
errors they made.

There is a pair of limitations related to the use of the metacognitive intervention in
this study that could be responsible for the lack of findings, however. First of all, and
similar to the findings of Keith and Frese (2005), it may have been the case that other
elements of the task induced metacogritive activity and overruled the influence of the
intervention. The design of the task in this study involved providing the participants with
an opportunity to study the task manual before each trial and view feedback after each
trial. While the participants did not have to spend time doing these things, they still had
to close out of the manual and quickly page through the feedback to be able to advance

through the study. If adaptability truly involves an ongoing process of assessing the

107

situation, examining effectiveness, and trying to devise and implement appropriate
strategies as was suggested earlier, this ongoing cycle of looking at the manual and
viewing feedback after each trial in and of itself may have prompted individuals to think
more about their performance and strategies, thus mitigating some of the potential effects
of the metacognitive intervention.

A second limitation related to these hypotheses involves the design of the
metacognitive intervention. Recall that the intervention focused on providing
participants with an overview of the potential importance of metacognitive activity
related to planning, monitoring, and evaluating as well as having them respond to probe
questions regarding whether or not they were engaging in these sorts of activities. While
these aspects of the intervention were designed to prompt people into action, participants
may have simply saw the intervention as a suggestion of sorts and responded to the probe
questions in a descriptive manner instead of using them as an opportunity to evaluate
their levels of metacognitive activity. It may be the case that explicitly requiring
individuals to make plans for upcoming trials, monitor current levels of understanding
and performance through feedback, and evaluate and think about the success of their
strategies is necessary if one expects to create a successful metacognitive intervention.

Hypotheses 6, 7, 8, and 9. Hypothesis 6 focused on the importance of cognitive
ability as a predictor of adaptive performance across different types of task change.
Briefly, it was believed that cognitive ability would play a more important role in
predicting adaptive performance effectiveness in response to task changes that were more
cognitively complex in nature. This hypothesis was in line with past research findings as

well as LePine et al.’s (2000) speculation that cognitive ability should be more strongly

108

related to performance in situations with higher cognitive or information processing
demands.

The results of the test of this hypothesis, however, were not supportive. Cognitive
ability was not differentially related to adaptive performance effectiveness across the
different types of task change. However, the results did suggest a significant positive
omnibus main effect of cognitive ability on adaptive performance effectiveness. Based
on this, it appears that higher levels of cognitive ability provided at least some platform
for being able to better recognize and adapt to task changes. Of course, these findings
need to be interpreted cautiously as there was not a specific omnibus main effect
hypothesis offered and the effects of cognitive ability on outcomes related to change
detection, relevant knowledge acquisition, and proper strategy use were not explicitly
tested.

Hypothesis 7 suggested that learning goal orientation would differentially predict
adaptive performance effectiveness in response to different types of task change. This
hypothesis was predicated on the notion that those with higher levels of learning goal
orientation would excel in cognitively demanding, novel situations such as those
presented by component and, especially, coordinative task changes.

However, hypothesis 7 was not supported. Those with higher levels of learning
goal orientation did not exhibit higher levels of adaptive performance in general, and
furthermore no significant differential effects of learning goal orientation on adaptive
performance effectiveness across different types of task change were found. This is

somewhat surprising given recent meta-analytic evidence that learning orientation is

109

generally positively related to effective use of learning strategies, feedback seeking
behavior, and job performance (Payne et al., 2007).

Hypothesis 8 focused on the role of conscientiousness as a potential predictor of
adaptive performance effectiveness. This hypothesis suggested that conscientiousness
would differentially predict adaptive performance effectiveness in response to different
types of task change. More specifically, because conscientiousness is thought to breed
commitment and effort on a given task, hypothesis 8 surmised that it would be an
especially important predictor of adaptive performance effectiveness for changes that
were primarily motivationally-based. This hypothesis was in line with previous research
findings by LePine et al. (2000), who found that high levels of conscientiousness were
negatively related to adaptive performance effectiveness in response to what was
essentially a coordinative task change where cognitive strategy change was essential for
adaptive performance effectiveness.

However, the results of this study showed that conscientiousness was not
differentially related to adaptive performance effectiveness and, furthermore, there was
no omnibus effect of conscientiousness on adaptive performance effectiveness in either
direction. This finding is especially surprising in relation to the task difficulty condition,
where the high levels of commitment, motivation, and perseverance thought to be
associated with high conscientiousness seem particularly well suited to influencing
adaptive performance effectiveness.

Hypothesis 9 was also concerned with the effects of personality on adaptive
performance effectiveness, but its focus was on the potential effects of openness to

experience. People with high levels of openness to experience are thought to be

110

intellectually curious and fond of novel experiences, and because of this it was predicted
that high openness to experience would positively relate to adaptive performance
effectiveness. This was expected to be the case even more so when task changes were
more cognitive in nature and thus required a greater deal of strategy manipulation and
reformulation.

Similar to conscientiousness, openness to experience was not found to
significantly predict adaptive performance effectiveness differentially across task change
types or in an omnibus manner. Again, this is somewhat surprising due to the supposed
intellectual curiosity and fondness for novel experiences or change associated with high
levels of openness to experience. Furthermore, this contradicts past findings regarding
the positive effects of openness to experience on adaptive performance effectiveness in
response to a coordinative task change (LePine et al., 2000).

As a whole, the individual difference hypotheses in this study failed to receive
support. While this comes as something of a surprise, there are some limitations with the
study that could help to explain why these hypotheses were not supported. First of all,
this study focused on measuring adaptive performance effectiveness and its hypothesized
knowledge and strategy determinants six trials after the task given task change occurred.
While the approach taken here was reasonable as a starting point, it may be the case that
some of the hypothesized effects could have happened but did not manifest themselves at
the particular point in time that they were measured. While this limitation is likely
important for all of the hypotheses tested in this study, is may be especially relevant for
the individual difference predictors, as it is reasonable to argue that the effects related to .

them could manifest themselves within a few trials of the task change. Thus, over time

111

these effects could be “washed away” if simple exposure and practice on the newly-
changed task allowed those with differing levels of cognitive ability, learning orientation,
opermess to experience, and conscientiousness to “catch up”, essentially homogenizing
performance across different levels of these factors. This argument points to the
importance of explicitly considering the timeframe in which effects unfold, as well as the
level or magnitude of effects at a given point in time.

As described above, the findings related to personality effects on adaptive
performance effectiveness do not align with past findings (i.e. LePine et al., 2000).
Another potential reason for this may have to do with the generality of the personality
measures used here. Rather than looking at broad level personality factors, some scholars
have suggested that facet level predictors be examined as they are more behaviorally
specific than broad, overarching measures of personality like the ones used here (Ashton,
1998; Schneider, Hough, & Dunnette, 1996). Indeed, LePine et al. (2000) provide some
evidence for the efficacy of examining personality factors at the facet level in relation to
adaptive performance; their supplementary analyses suggested that their negative
relationship between conscientious and adaptive performance effectiveness could be
primarily attributed to dependability-related facets of conscientiousness. However,
LePine et al.’s findings regarding facets of openness to experience were less clear-cut, as
there were uniformly positive relationships between each of the facets and adaptive
performance effectiveness. These findings, however, do point to the importance of
explicitly considering the types of behaviors that are of interest in a study and more
specifically considering personality factors or facets as potential predictors.

Contributions

112

In general, the relationships proposed in hypotheses l, 2, and 3 were supported.
Based on these findings, a number of contributions to the extant literature on adaptive
performance can be identified. First of all, this study is the first to treat adaptive
performance as a process by trying to explicate what sorts of things people have to do or
go through to adapt to a change in the nature of a learned task. While research by Chen
et al. (2005) and theoretical work by Ployhart & Bliese (2006) provided an initial step in
this direction, this study is the first to systematically examine the effects of in situ change
detection, task knowledge development, and strategy use on adaptive performance
effectiveness.

By taking this type of approach, and providing support for the importance of these
types of in situ factors, this study should help to shift thinking about adaptive
performance away from more traditional, static views that primarily deal with
performance effectiveness toward more dynamic or process-oriented views. These types
of perspectives, then, could help to further address key questions regarding “what”,
“why”, and “when” people are engaging in activities related to adaptive performance
instead of simply focusing on questions pertaining to levels of adaptive performance
effectiveness.

In a similar vein, this study was the first to explicitly consider the nature of task
and the type of task change that people were attempting to adapt to. As noted earlier,
much of the existing empirical work conducted on adaptability or adaptive performance
has taken something of a “catch-all” approach to manipulating task change and
operationalizing adaptive performance by looking at changes that result in tasks that are

longer, more difficult, and/or more complex. By relying on the Wood (1986) taxonomy

113

of task complexity, a categorization of different types of task change was derived that
served as a foundation for examining the potential differential effects of various cognitive
and motivational factors on adaptive performance in response to different types of task
change.

While the results of the hypotheses tested herein did not generally find a
moderating effect of task change type, there are a few specific findings related to task
change type that deserve to be highlighted. As noted above, the results of hypotheses 1a
did suggest that the importance of a motivationally-based task strategy (i.e. increased
effort) varied depending on the type of task change that people facing such that effort was
more strongly related to adaptive performance effectiveness in response to task changes
that were less cognitively-based. Furthermore, it was found that relevant declarative
knowledge gain, which focused on different types of knowledge for each type of task
change, was important for predicting strategic variables that were thought to be important
predictors of adaptive performance effectiveness in response to component and
coordinative task changes. These findings, then, provide some initial evidence for the
importance of considering what is actually changing in a given task when trying to
predict and understand the process of adaptation as well as the end result, adaptive
performance effectiveness.

General Limitations

Sample. The sample for this study was composed entirely of undergraduate
college students who were participating in the study in exchange for course credit. There
were two possible concerns or limitations in this study regarding the sample. The use of

students in and of itself may be problematic from the standpoint of broad generalization.

114

One concern related to this is the lack of random selection and the relative homogeneity
of the participants in this sample. While this has the potential to be problematic, it is not
a challenge that is limited to this study or the laboratory setting in general. In support of
this, Colquitt (2008) suggests that samples in field studies typically have fairly
homogenous samples as well. Furthermore, since the participants in this study are likely
faced with situations where they have to learn new information and skills and potentially
adapt their knowledge and skill to new situations, the use of these students as a sample
does not seem overly problematic. In addition, it is reasonable to think that environments
where adaptive performance is a key concern (organizations, for example) are populated
with people who are educated to at least the high school and more likely the college level.
Therefore, this sample likely reflected the target population fairly well.

A second potential concern with use of student participants was their motivation
to perform on the task. As mentioned earlier, the primary reward for participation in this
study was class credit commensurate with the amount of time that the students spent
participating in the study. However, no monetary incentives of any type were offered,
and participants did not receive more or less credit based on how well they performed.
While it may seem like this would be detrimental in terms of the amount of effort that
students exerted, the data collected in the study does not reflect this. Indeed, it appears
that the participants worked quite hard on the task as the mean number of targets
prosecuted during the final trial, which was used as an index of effort in this study, was
11.5 out of 12 for component change condition, 11.6 out of 12 for the coordinative

change condition, and 13.2 out of 18 for the difficulty change condition. Taken together,

115

this suggests that participant effort due to lack of external, performance-based rewards
was not likely to be an appreciable problem.

Research setting. As with any psychological research conducted in a laboratory,
the setting itself may lead to questions regarding the external validity of any findings. In
addressing these types of questions, two points need to be made.

When dealing with questions regarding the relevance of external validity and the
“realism” of a given study, scholars such as Mook (1983) note the importance of
considering the nature of the research question. In discussing different types of realism
that can be achieved through experimental studies, Aronson & Carlsnrith (1968) and
Aronson, Wilson, and Akert (1994) broadly distinguish three ways in which an
experiment can be said to be realistic. Experimental realism comes about when the
situation is involving to the participants, they take it seriously, and it influences them in
some way. Mundane realism deals with the extent to which events occurring in the
research setting are likely to occur in the normal course of participants’ lives and how
well the research setting represents the “real world”. Finally, psychological realism
addresses the extent to which the psychological processes that occur or are necessary in
an experiment represent those that would occur if a similar situation was faced in
everyday life.

For the purpose of this study, the focus was squarely on creating a situation where
participants would be engaged by the nature of the study and the key psychological
processes related to adaptive performance effectiveness would be adequately represented
and measured. Thus, the objective was to create strong senses of both experimental and

psychological realism. As noted earlier, participants seemed to be involved and focused

116

on the study as they exerted high levels of effort, even toward the tail end of the study.
Furthermore, the design of the task necessitated that learning, effort, strategy change, and
the like would be important for adaptive performance effectiveness, and the results of the
study provided some support for these notions. Thus, it appears that the laboratory
design and setting was quite successful for the purposes and goals of this study, even
though the mundane realism was not necessarily strong.

Furthermore, while external validity is certainly a concern with any study

conducted in the laboratory or the field, the laboratory setting in and of itself may not be

 

as strong of a barrier to generalizability as many think, even with the apparent lack of ‘
mrmdane realism. First off, there are no boundary conditions or formal aspects of the

conceptual model from this study that I am aware of that would that influence things to

the extent that the hypothesized relationships in this study would not be supported if they

truly existed in the population of interest. Furthermore, Anderson, Lindsay, and

Bushman (1999) report that the correlations between the effect sizes found in research

using laboratory settings and field settings generally exceed .70. Thus, there is evidence

that hypothesis tests using field and laboratory settings can and likely do lead to similar

findings, and it appears likely that this would be the case given the apparent

psychological realism of the processes studied herein.

Statistical power. Another potential limitation in this study as whole is a lack of
statistical power. Statistical power can be thought of as the probability of correctly
rejecting the null hypothesis given (a) the sample size, (b) the effect size, (c) alpha, and
(d) the power of the statistical test (Cohen, 1969). Lack of statistical power, then, renders

it difficult to find significant relationships and potentially inflates Type 11 error rates.

117

3,1510 11
mle 1
its we
tr‘pt‘ttllfi

EIETECI

Stone-l
mitt:
mater
artist
predic
Elltt‘l
11018
Vain
titer

data

are

be

Due to incomplete data, technical problems, and general time constraints, the analyzable
sample for this study was 283 participants, or 43 to 50 per condition. While these sample
sizes were likely sufficient even by conservative estimates for tests of the main effect
hypotheses, they may have been toward the low end of acceptability for tests of the
interaction hypotheses.

Aguinis and colleagues (Aguinis, 1995; Aguinis, Boik, & Pierce, 2001; Aguinis &
Stone-Romero, 1997) keenly discuss the potential problem of low statistical power in
moderated multiple regression. However, based on the findings (or lack thereof) for the
moderation hypotheses in the current study, it is difficult to conclude that the available
statistical power was a hindrance. Specifically, the interaction proposed in hypothesis 1a
predicted approximately 2.3 percent of the incremental variance in adaptive performance
effectiveness and was found to be statistically significant. However, the interactions
proposed in hypotheses 4, 6, 7, 8, and 9 predicted one percent or less of the incremental
variance in adaptive performance effectiveness over and above any of the effects of the
given control variables and main effects. Given this, it seems that collecting additional
data in order to increase power to the point where these effects were statistically
significant would not be a useful pursuit. It would likely be the case that if enough power
was obtained to render any of these findings statistically significant, the practical
significance would be lacking.

Consideration of time. It has been argued throughout this paper that explicit
attention needs to be paid to thinking about adaptive performance from more of a process
perspective. Specifically, a conceptual model of adaptive performance was proposed

based on a heuristic control-theory model of self-regulation. This model was intended to

118

 

11mg

howl

stud
that
W1]

rim

help spur thinking about adaptability as a multitude of dynamic cognitions, behaviors,
and the like that people engage in to functionally align their behavior with given changes
in their task environment. This type of dynamic perspective inherently entails an explicit
consideration of time, and some of the key mechanisms proposed for the process of
adaptability are thought to involve cycles of change detection, source determination,
strategy change, performance, and feedback.

As it was mentioned earlier, this study incorporated time to a certain extent
through the causal and implied temporal relationships between change detection, relevant

knowledge development, strategy change, and adaptive performance effectiveness. This

 

is consistent with past empirical work conducted by Chen et al. (2005) as well as
theoretical work by Ployhart and Bliese (2006), both of which focused somewhat on
processes related to self-regulation in a dynamic manner as determinants of adaptive
performance effectiveness. However, it is still the case that the variables assessed in this
study were measured at similar points in time (i.e. during trail 12 and directly after it) and
that the dynamic nature of these processes was not investigated in a longitudinal manner.
While this is by no means a damning indictment of the findings, the fact remains that
time can and likely should be considered even more explicitly in future work on adaptive
performance.
Future Research Directions

While the current study attempted to address some gaps in the extant literature on
adaptive performance, a number of additional avenues for future research can be
identified that will help to further understanding about the nature of adaptive performance

and its antecedents.

119

W313 '

B we

{13.111

den

fiflt

0W

reg

in'

One valuable future direction for research on adaptive performance involves a
more explicit consideration of time. Mohammed, Hamilton, and Lim (2008) discuss four
different ways in which time can be considered or built into research. While Mohammed
et al. focus on the importance of considering time for research on team effectiveness, the
ways of considering time that they discuss are likely relevant for individual level research
as well. Two of these types of time considerations may be especially relevant for work
examining adaptive performance.

The first, and most obviously needed, is longitudinal research. The current
conceptualization of adaptive performance relies on a process-oriented perspective
whereby cycles of detection, diagnosis, and strategic adaptation are thought to influence
one’s ability to adapt to changes in a given task. However, existing research on adaptive
performance has focused almost exclusively on examining adaptive performance at a
single point in time. This study takes an initial step toward a more dynamic perspective
by examining an implied causal and temporal ordering between factors related to change
detection, relevant knowledge acquisition, strategy change, and adaptive performance
effectiveness, but more needs to be done. Adaptive performance should be examined
over multiple occasions in order to study the potential cyclical effects of different
regulatory processes and cycles and to more closely examine the shape, form, and
trajectory of adaptive performance within individuals.

A second type of time consideration identified by Mohammed et al. (2008)
involves the incorporation of time-referenced or time-oriented constructs into cross-
sectional research. Obviously, it is not always feasible to conduct longitudinal research,

although that limitation is more applicable to field than laboratory research, especially

120

nth It

mpfll

nllue

Chill

0% ad"

with regard to adaptive performance. Regardless, our understanding of potentially
important cognitive and behavioral processes and the way they unfold over time to
influence adaptive performance can potentially be bolstered by incorporating time
oriented constructs into research. For example, it was suggested earlier that a major part
of adapting to coordinative task changes involves appropriately sequencing and
prioritizing relevant behaviors. Building on Chen et al.’s (2005) findings that goal choice
processes have implications for adaptive performance effectiveness, time could be
incorporated into these goal choice and planning measures to assess not only what people
are planning on doing, but when they are planning to do different things and how their
behaviors will be prioritized. This type of consideration could clearly advance our
understanding of adaptive performance from a more dynamic perspective, even if
longitudinal data is not available.

Related to the notion of more carefully considering time is the need for more a
more focused examination of in situ regulatory processes that may influence adaptive
performance. The conceptual model of adaptation used as a foundation for this study was
based in control theory models of self-regulation, which deal directly with the way that
an individual guides “his/her goal-directed activities over time and across changing
circumstances” (Karoly, 1993; p. 25). These models discuss the importance of processes
related to performance monitoring, goal-performance discrepancy detection, plan
development, and strategy enactment as predictors of performance.

While the current study used these types of models to guide the measurement of
variables that were hypothesized to be important predictors of adaptive performance

effectiveness, it is important to point out that the actual regulatory processes in and of

121

 

 

gfiCO‘

MI

511

themselves were not directly manipulated or assessed. Rather, this study focused on the
outcomes of change detection, relevant knowledge development, and strategic behavior
that were suggested to be the results or manifestations of some of these key regulatory
processes and on a metacognitive intervention that was expected to prompt planning,
monitoring, and evaluating processes. Results from Chen et al. (2005) shed some initial
light on the importance of in situ self-regulatory processes related to goal choice and goal
striving, but a more specific investigation of these processes, especially from a
longitudinal perspective, could prove quite useful in increasing our understanding of
adaptive performance. This may be especially true if links between these regulatory
processes and the suggested outcomes discussed in this paper can be empirically
established. This would provide not only an additional understanding of how and why
people are able to adapt to task changes, but it could also serve as a platform for better
understanding the influences of numerous other factors that have been shown to predict
adaptive performance such as self-efficacy, training knowledge/skill, and task knowledge
structure (e. g. Ford et al., 1998; Kozlowski et al., 2001).

Another need for future research involves the further examination of individual
difference factors as predictors of adaptive performance effectiveness. While the
individual difference hypotheses in this study were not supported, previous research has
established that factors such as cognitive ability, personality, and goal orientation can
have effects (LePine et al., 2000; Pulakos et al., 2002). One potentially useful direction
for future research involves more critically examining these individual difference factors
in terms of what types of tasks change they may best predict adaptive performance in

response to as well as how they may have their effects. This type of approach is

122

consistent with recent theoretic work by Ployhart & Bliese (2006) who discuss the
potential importance of myriad individual difference factors for adaptability and how
their predictive strength may vary depending on different types of tasks that change. The
current study may provide some help in this vein by providing initial support for the
importance of change detection, relevant knowledge acquisition, and strategy change as
important predictors of adaptive performance effectiveness and offering different types of
task changes as a starting point for investigating the influences of individual differences
on adaptive performance.

In a similar vein, future research on adaptive performance could benefit fiom
considering the importance of additional individual difference factors. The factors
examined in this literature are traditionally cognitive and behavioral in nature, but certain
affective individual differences may also be important predictors. One such potentially
interesting individual difference is emotion control (e. g. Kanfer & Ackerman, 1989;
Kanfer et al., 1996). Emotion control is suggested to be a motivational skill that deals
with regulating one’s reactions to anxiety and negative-emotion inducing events during
skill acquisition or performance. These skills are suggested to protect essential on-task
attention and effort in the face of difficulty. Keith & F rese (2005) provide some initial
support for the importance of emotion control in predicting adaptive transfer
performance, but future work examining this and similar affective individual differences
as predictors of adaptive performance would likely be valuable.

Another possibly useful direction for future research centers on more critically
examining adaptive performance at the team level. While adaptive performance has not

received as much attention at the team level as it has at the individual level, this is not to

123

 

say the
glozlc
pill?
and p1
info:

there

pied
5'11]
ads

pro

for

say that work has not been done. Theoretical work by Kozlowski and colleagues
(Kozlowski, 1998; Kozlowski et al. 1996a, 1996b, 1999) and empirical work by LePine
(2003) and Chen et al. (2005) have began to discuss the importance of team composition
and process factors, among other things, as important determinants of team adaptive
performance. Similar to the individual level adaptive performance literature, however,
there is ample opportunity for future work investigating key in situ team processes that
may have an impact on team adaptive performance effectiveness.

Researchers such as Marks, Mathieu, and Zaccaro (2001) discuss a number of
team processes that are relevant for performance, and many of these may also be crucial
predictors of adaptive performance effectiveness. Chen et al. (2005) provide some initial
support for the importance of team transition and action phase processes as predictors of
adaptive performance effectiveness in teams, but more work can be done. These
processes, which include things such as mission analysis, goal specification, strategy
formulation, monitoring progress toward goals, and backing-up behaviors, deserve
further examination as predictors of adaptive performance. Furthermore, the findings of
the current study which suggest that change detection, relevant knowledge acquisition,
and strategy change may be important predictors of adaptive performance effectiveness at
the individual level may be of use as a platform for attempting to understand how or why

different team processes have effects on team adaptive performance.

124

APPENDIX A

Definitions of Individual Adaptation and Adaptive Performance

Authors Definition

The generalization of trained knowledge
Kozlowski, Toney, Mullins, Weissbein, and skills to new, more difficult, and more
Brown, & Bell (2001) complex situations.

An individual's ability, skill, disposition,
willingness, and/or motivation, to change or
fit different task, social, and environmental

Ployhart & Bliese (2006) features.

An individual's capacity to deal with
Joung, Hesketh, & Neal (2006); changing work requirements and novel or
Hesketh & Neal (1999) unusual situations.

Competencies that enable people to manage

themselves in relation to the demands of
Fine & Cronshaw (1999); Cronshaw & conformity and/or change in particular
Jethmalani (2005) situations.

The capability to modify knowledge, skill,

and other characteristics acquired during

training to effectively meet novel, difficult,
Chen, Thomas, & Wallace (2005) and complex situations.

Using one's existing knowledge base to
Ivancic & Hesketh (2000); Bell & change a learned procedure, or to generate a
Kozlowski (2008); Keith & Frese (2005) solution to a completely new problem.

Adaptability...is evidenced when the
individual responds successfully to changes

Smith, Ford, & Kozlowski (1997) in the nature of the trained task.

Situations in which individuals modified
Pulakos, Arad, Donovan, & their behavior to meet the demands of a new
Plamondon (2000) situation or event or a changed environment.

125

APPENDIX B
Consent Form
Research Participant Information and Consent

You are being asked to participate in a research project. Researchers are required to
proved a consent form to inform you about the study, to convey that participation is
voluntary, to explain the potential risks and benefits of participation, and to empower you
to make an informed decisions. You should feel free to ask the researchers any questions
that you may have.

Study Title:
Strategic Radar Simulation

Investigator Names:
Dustin J undt & Dr. Steve Kozlowski

Purpose of Research

You are being asked to participate in a research study examining how different personal
characteristics, thoughts, and beliefs influence learning, skill acquisition, and
performance on a complex radar simulation task. You have been selected as a possible
participant because of your membership in the psychology subject pool. In the entire
study, approximately 300 people are being asked to participate.

Description and Explanation of Procedure:

Your participation in this study involves completing an online questionnaire and
participating in a lab session. If you agree to participate, you will be asked to complete
an online questionnaire that involves demographic information and measures of a number
of different personal characteristics. This questionnaire should take approximately 30
minutes to complete. Once the questionnaire is completed, you will schedule a time to
participate in a 2.5 hour lab session where you will learn and work with a radar
simulation task. For this part of the study, you will go to the ADAPT LAB in room 204
Psychology Building. Upon arrival at your scheduled session, you will receive basic
training on the Strategic Radar Simulation task and will then complete a number of trials
on the simulation where you’ll have to opportunity to firrther learn about the simulation
and show your levels of skilled performance. Throughout these trials, you will be asked
questions about your reactions, thoughts, task knowledge, and behaviors. At the
conclusion of the lab session, you will be debriefed regarding the purpose of study and
how your participation contributed to it. Research findings will not be immediately
available upon your completion of the lab session but can be obtained by contacting the
lead researcher (Dustin Jundt) in the future.

Estimated time required and credits:

Online questionnaire: 30 Minutes (1 Psychology subject pool credit)
Laboratory session: 2.5 hours for the (5 Psychology subject pool credits)

126

Total: 3 hours (6 Psychology subject pool credits).

Potential Benefits:

There are a number of potential benefits to you for taking part in this study. Throughout
this study, you will gain experience with several standard, often used psychological
measures. In addition, you will gain experience with a computer-based simulation task
and training system similar to what is frequently used in organizations. Finally, you will
gain knowledge about the process of conducting and participating in psychological
research.

Potential Risks and Discomforts:

Although no foreseeable risks are expected from participating in this study, you may
experience some fatigue during the experiment. The information collected from you and
your privacy will be kept confidential to the maximum extent allowable by law, however
there is a very slight risk of loss of privacy regarding the data that is provided.

Researcher Contact Information:

You will be able to view your responses at a later date and be fully debriefed on their
meaning and implications if you so desire. The investigators will be available to answer
any questions you may have. If, at any time, you feel your questions have not been
adequately answered or you want to discuss the research, please contact the lead
experimenter, Dustin Jundt (jundtdus@rnsu.edu; 775-2196) or the principal investigator,
Dr. Steve Kozlowski (stevelgz@msu.edu; 353-8924).

[RB Contact Information:

If you have questions regarding your role and rights as a subject of research, would like
to obtain information or offer input, or would like to register a complain about this study,
you may contact, anonymously if you wish, Michigan State University’s Human research
Protection Program at 517-355-2180, fax: 517—432-4503, e-mail: irb@msu.edu, or
regular mail at 202 Olds Hall, Michigan State University, East Lansing, MI 48824.

Your Rights to Participate, Say N 0, or Withdraw.

Your agreement to participate in this research is completely voluntary; you have the right
to say no. You are free to withdraw consent and discontinue participation in this project
at any time without penalty. If you choose to withdraw from the study prior to its
completion, you will receive credit for the time you have spent in the study (1 credit per
30 minutes). Within one year of your participation, a computerized copy of this consent
form will be provided to you upon request.

Privacy and Confidentiality

The data collected from this study will be kept confidential. All data will be stored on
password-protected computers that can only be directly accessed by the two investigators
listed above. If you agree to participate in this study, you will be asked to report your
NAME, PID, and MSU email address at the beginning of the subsequent online
questionnaire. While this questionnaire is intemet-based, your IP addresses will not be
logged. The reason you are asked for your name and PID is to ensure that you receive

127

full credit for your participation in the study. Your identity will not be associated with
your responses, and it will be kept secure and confidential. Furthermore, once your credit
is assigned and registered all identifying information will be erased from our records and
any data published or presented in professional sources will be done at an aggregate
level; your responses will not be identifiable. Your privacy will be protected to the
maximum extent allowable by law.

Consent

If you voluntarily agree to participate in this study, you will be asked to check a box
below that indicates your consent.

128

APPENDIX C
Debriefing Form
Strategic Radar Simulation Study Debrief'mg Form

Thank you for your participation in this investigation. The study in which you
just participated was designed to examine the effects of an intervention aimed at
prompting metacognitive on the ability of people to perform well on the Tactical Action
Simulation (T AS) and adapt to a number of different types of task change. The
instructions that you received throughout the study were designed to influence how you
approached thinking about the task and monitoring the success of your learning,
strategies, and performance throughout the task, but even more so for the last six trials.
The questions that you answered at different points during the study will allow us to help
assess your personal characteristics, learning, beliefs, goals, and thoughts and how they
influenced your performance and adaptability to changes in the TAS task.

Most of the questions that you answered have no correct or incorrect answers and
have absolutely no significance other than to help us understand your performance and
adaptation on the TAS task. We have not recorded your identity with the data that we
collected, and we will keep this data confidential to the fullest extent possible. All of the
information that we have collected will be used to try to help us to more completely
understand how people are able to adapt to different types of task changes and if or how
having people in engage in metacognitive activities related to making specific plans and
strategies, monitoring progress toward goals, and evaluating the effectiveness of these
plans and strategies influences this adaptation and performance process.

If you have any questions about this study or would like to receive a copy of the
results when they are complete, please notify the investigator at this time. We have tried
to make your experience in this study interesting for you and we are open to suggestions
for improvement. If you think that we can improve this study in any way, please feel free
to talk to us now or in the future. If you have any additional questions about the study or
your involvement in it, please contact the Research Coordinator or Principal Investigator,
or you can speak with the investigator at this time. If you have questions or concerns
about your rights as a research participant, please contact UCRIHS.

Research Coordinator Principal Investigator UCRIHS

 

 

Dustin Jundt Dr. Steve Kozlowski ucrihs@msu.edu
Lundtdus@msu.edu stevekos@msu.edu 3 55-21 80
775-2196 353-8924

129

APPENDIX D
Metacognitive Intervention Text

Developing skill and performing well on a complex simulation like the TAS can be
challenging. However, decades of research show that focusing on being aware of and
managing your thoughts, strategies, and behaviors as you work on these types of tasks
can be extremely beneficial. Psychologists use the term “rneta-cogrritive awareness” to
describe this process of actively focusing on and managing your thoughts, strategies, and
behaviors while performing on a task. Metacognitive-awareness has 4 key features that
have repeatedly been shown to positively predict learning, skill development, and
performance on a variety of tasks, especially those like the TAS.

0 Planning what you will study in the manual and how you will use TAS simulation
practice opportunities to work toward developing skill, performing well on the task, and
achieving the goals that you set and commit to.

0 Monitoring your study and practice activities to see if you are focusing on information
and actions that help you perform well on TAS and achieve your goals.

0 Evaluating your TAS feedback to figure out which strategies or techniques worked well
in previous trials and which did not so that you can improve.

0 Adapting or changing your strategies and actions to improve future TAS performance
and success.

Throughout the next set of trials, you should focus on specifically planning for
performance and improvement, monitoring your performance levels and feedback,
evaluating the quality of your strategies and techniques, and adapting or changing those
strategies that are not successful.

130

APPENDDC E
Metacognitive Intervention Probes

To what extent are you actively planning (or thinking about) how you will achieve your
TAS goals through the use of study and practice opportunities?

To what extent are you actively monitoring your study and practice activities to see if you
are attending to information and actions that help you achieve your TAS goals?

To what extent are you evaluating and using your TAS feedback to figure out what
worked well and what did not in previous trials so that you can improve?

To what extent are you actively adapting or changing your strategy and actions in an
effort to improve your progress toward your TAS goals?

131

APPENDIX F
Learning Orientation Scale Items

Adapted from VandeWalle (1997)

I am willing to select a challenging assignment that I can learn a lot from.

I often look for opportunities to develop new skills and knowledge.

I enjoy challenging and difficult tasks where I'll learn new skills.
For me, development of my ability is important enough to take risks.
I prefer situations that require a high level of ability and talent.

Response Options

1: strongly disagree
2: disagree

3: neutral

4: agree

5: strongly agree

132

 

 

APPENDD( G
Conscientiousness Scale Items
From Costa & McCrae (1992)

I’m known for my prudence and common sense.

I don’t take civic duties like voting very seriously.

I keep myself informed and usually make intelligent decision.

I often come into situations without being fully prepared.

I pride myself on my sound judgment.

I don’t seem to be completely successful at anything.

I’m very competent person.

I am efficient and effective at my work.

I would rather keep my options open than plan everything rn advance.

I keep my belongings neat and clean.

I am not a very methodical person.

I like to keep everything in its place so I know just where it is.

I never seem to be able to get organized.

I tent to e somewhat fastidious or exacting.

I’m not compulsive about cleaning.

I spend a lot of time looking for things I’ve misplaced.

I try to perform all the tasks assigned to me conscientiously.

Sometimes I’m not as dependable or reliable as I should be.

I pay my debts promptly and in full.

Sometimes I cheat when I play solitaire.

When I make a commitment, I can always be counted on to follow through.
I adhere strictly to my ethical principles.

I try to do jobs carefully, so they won’t have to be done again.

I’d really have to be sick before I’d miss a day of work.

I am easy-going and lackadaisical.

I have a clear set of goals and work toward them in an orderly fashion.
When I start a self-improvement program, I usually let it slide after a few days.
I work hard to accomplish my goals.

I don’t feel like I’m driven to get ahead.

I strive to achieve all I can.

I strive for excellence in everything I do.

I’m something of a “workaholic.”

I’m pretty good about pacing myself so as to get things done on time.

I waste a lot of time before settling down to work.

I am a productive person who always gets the job done.

I have trouble making myself do what I should.

Once I start project, I almost always finish it.

When a project gets too difficult, I’m inclined to start a new one.

There are so many little jobs that need to be done that I sometimes just ignore them all.
I have a lot of self-discipline.

133

Over the years I’ve done some pretty stupid things.

I think things though before coming to a decision.
Occasionally I act first and think later.

I always consider the consequences before I take action.
I often do things on the spur of the moment.

I rarely make hasty decisions.

I plan ahead carefully when I go on a trip.

I think twice before I answer a question.

Response Options

1: strongly disagree
2: disagree

3: neutral

4: agree

5: strongly agree

134

APPENDIX H
Openness to Experience Scale Items
From Costa & McCrae (1992)

I have a very active imagination.

I try to keep all my thought directed along realistic lines and avoid flights of fancy.

I have an active fantasy life.

I don’t like to waste my time daydreaming.

I enjoy concentrating on a fantasy or daydream and exploring all its possibilities,
letting it grow and develop.

If I feel my mind starting to drift off into daydreams, I usually get busy and start
concentrating on some work or activity instead.

As a child I rarely enjoyed games of make-believe.

I would have difficulty just letting my mind wander without control or guidance.

Aesthetic and artistic concerns aren’t very important to me.

I am sometimes completely absorbed in music I am listening to.

Watching ballet or modern dance bores me.

I am intrigued by the patterns I find in art and nature.

Poetry has little or no effect on me.

Certain kinds of music have an endless fascination for me.

Sometimes when I am reading poetry or looking at a work of art, I feel a chill or
wave of excitement.

I enjoy reading poetry that emphasizes feelings and images more than story lines.

Without strong emotions, life would be uninteresting to me.

I rarely experience strong emotions.

How I fell about things is important to me.

I seldom pay much attention to my feelings of the moment.

1 experience a wide range of emotions or feelings.

I seldom notice the moods or feelings that different environments produce.

1 find it easy to empathize - to feel myself what others are feeling.

Odd things — like certain scents or the names of distant places —
can evoke strong moods in me.

I’m pretty set in my ways.

I think it’s interesting to learn and develop new hobbies.

Once I find the right way to do something, I stick to it.

I often try new and foreign foods.

I prefer to spend my time in familiar surroundings.

Sometimes I make changes around the house just to try something different.

On a vacation, I prefer going back to a tried and true spot.

I follow the same route when I go someplace.

I often enjoy playing with theories or abstract ideas.

I find philosophical arguments boring.

I enjoy solving problems or puzzles.

I sometimes lost interest when people talk about very abstract, theoretical matters.

135

I enjoy working on “mind-twister” - types puzzles.

I have little interest in speculating on the nature of the universe or the human condition.

I have a lot of intellectual curiosity.

I have a wide range of intellectual interests.

I believe letting students hear controversial speakers can only confuse and mislead them.

I believe that laws and social policies should change to reflect the needs of a changing
World.

I believe we should look to our religious authorities for decisions on moral issues.

I believe that the different ideas of right and wrong that people in other societies have
may be valid for them.

I believe that loyalty to one’s ideals and principles is more important than
“open-mindedness”.

I consider myself broad-minded and tolerant of other people’s lifestyles.

I think that if people don’t know what they believe in by the time they’re 25, there’s
something wrong with them.

I believe that the “new morality” of permissiveness is no morality at all.

Response Options

1: strongly disagree
2: disagree

3: neutral

4: agree

5: strongly agree

136

APPENDIX I
Declarative Knowledge Quiz Items

If you want to “hook” a target, what would you do?
a. Left-click on the target
b. Right-click on the target
c. Select “clear” from the final engagement menu
(1. Zoom in on the target

If a target’s Response is Given, what is the likely Intent of the target?
a. Military
b. Hostile
c. Civilian
d. Peaceful

Which type of target would travel at a Speed of 27 knots?
a. Air
b. Sub
c. Surface
d. Unknown

If a target’s Intelligence is Platform, what Class does this suggest for the target?
a. Civilian
b. Military
0. Surface
(1. Unknown

If a target’s Response = No Response, which of the following actions should you take?
a. Choose class is Civilian
b. Choose intent is Hostile
c. Choose intent is Peaceful
d. Choose type is AIR

Which is the proper order for making proper cue decisions?
a. Class, Type, Intent, Final Engagement
b. Intent, Class, Type, Final Engagement
c. Type, Class, Intent, Final Engagement
d. Type, Intent, Class, Final Engagement

A Final Engagement decision of “Shoot” should be made for a target with which of the
following characteristics?

a. Type = Air, Class = Military, Intent = Peaceful

b. Type = Sub, Class = Military, Intent = Unknown

c. Type = Surface, Class = Unknown, Intent = Hostile

(1. Type = Sub, Class = Civilian, Intent = Peaceful

137

How many points are awarded for making all four cue decisions correctly?
a. 10
b. 25
c. 50
d. 100

During the last set of trials, how many targets were in the simulation?
a. 8
b. 12
c. 18
d. 25

If a target’s Altitude/Depth is 30, which of the following actions should you take?
a. Choose class is Civilian
b. Choose intent is Hostile
c. Choose intent is Peaceful
(I. Choose type is AIR

What Direction of Origin would a Military Class target come from?
a. Blue Lagoon
b. Indian Ocean
c. Red Sea
(1. Unknown
A target with a Hostile Intent would engage in which of the following Countermeasures?
a. Attacking
b. Jamming
c. None
(1. Unknown

If you’ve just noticed three targets near your inner perimeter, which of the following is
the best thing to do next?

a. Engage the target closest to the inner perimeter

b. Engage the fastest target near the inner perimeter

c. Examine the area for “pop-up” targets

(1. Zoom-In to check how close targets are to the inner perimeter

What is a “pop-up” target?
a. A target that appears suddenly anywhere on your screen.
b. A target that appears suddenly near the inner defensive perimeter.
c. A target that appears suddenly near the outer defensive perimeter.
d. A target that appears suddenly near your ship.

138

than: is
r

b
c.
d

Where is the inner defensive perimeter located?
a. 4 NM
b. 10 NM
c. 20 NM
d. 40 NM

How many points are lost when a target crosses the inner defensive perimeter?
a. 10

b. 25
c. 50
d. 100

139

APPENDIX J
Metacognitive Intervention Manipulation Check Items
During the session, were you told and did you hear that you should focus on...

- Planning how to achieve your TAS goals through the use of study and practice
opportunities?

- Monitoring your study and practice activities to see if you are attending to information
and actions that help you achieve your TAS goals?

- Evaluating your TAS feedback to figure out what did or did not work well in previous
trials?

- Actively adapting or changing your learning strategy and actions to improve progress
toward your TAS goals?

Response Options

1: strongly disagree
2: disagree

3: neutral

4: agree

5: strongly agree

140

$383280 u g 5:855 n 8 N £800 098:0 vim-- e
3:289:00 n _ 5:885 n 8 _ 260 09820 vim-w o

85:838.: u — 6230225 02 n 8 cannot-85 0.2 a
A232 .1. N 688$ u C 828D a.

89m"...

 

 

 

.8. 8. 8. .2. 8.- :. 8. .1. .2. .8. .3. .2. 2: 8.8

- 8. 8. .2. 8.- S. 8.- 2. .8. .8. .3. 8. 83 8.8

- .8? .2. 8.- 8.- 8. 8.- .2- 8. 8. 8.- :8 3.8

- 8. 8. :. 8.- 8. .2. 8.- 8. 8. :8 8.8

8. 8. .2. .8. .2. 8. 2. 8.- .2; 8.8 88

- 8.- 8. 8.- 8.- 8. 8. 8.- one 8.8

8. 8. .8. .2. .2. 8. .8; 8.8 84m

8. .8. .3; 8. 8.- 8.- 8.8 2%

8. 8. .3. 8m. .2. 8.8 88

- .2. .3. :. 83 8.8

- 2. S. :88 2.5.2

- .8. 8.8 83

- 88 a:

2 : S 8 w a 8 m a. m N _ am 2
macaw—880

GoEw-accflfl amoeozuoomm

3588.88 gunner.

8888888 team 8.8

cm 88 08.5 new

0_ .80 .820 .88

38:0 seaming—2 0.2

cougar—85 0.2
venomoaxm 9 82825
8832520880

.06 8833

be? .3580
cognac-tom 838m

oocotoaxm 839:00

«BecoO

.2

.3
.2
.2

oi

Add-$66606

 

3%. u n» 8.2336200 Ex 8.33.. .8338: Amk Lokmflmmgemmm has .wnohefihooafifi .QSBBmQ haunts” 6:33

8 .38

858

M XHQmemxx

141

mo.V&H*

 

 

 

..ma. LN. ...mN. mo: ...mN. 3. ...MN. ...ON. ...Q. ...on. ..mN. vosam mm._mN_
- LN. ...VN. mo: oN. 3o. LN. ...wN. ...mm. .mm. ...VN. mom 3N.m_
- 00- mo: E. No.- S. :- LN. co. we. omd mwd
- 2.- 2. co.- mo: 2. ..bN. :- nor de mm;
- co. co. N _. _ _.- 3.- 3.- oo.- Omd Nwd
- we. ...cN. no. LN. w _. 5.- cmd 9.. m
- ...Nm. 2.- o_. 3.- 2.- and and
- N_. S. ...NN. LN. 9.6 5M
- .. _ N. .nm. ...hN. no...” wvéN
- ...NN. £- 2.6Nm 3.3::
- ...vo. no; EN
- and 3..—
_ _ o. m m N. o m 3 m N _ Om E
gown—chewy

mmocozuoo-tm
aggro-tom 03533.
“ho-cm x8..-
5:83:38.
uwcogozv— Ego—om
couooaoa

compaoEBE 0.2
oocotonxm

8 82825

8053283880
.06 8:53
8:3... 32:80
vocaEhO-tom 005095

oocotoaxm 825500

.5350

.N_

.2
.2

.m

(<06

Anim'éun‘xo'

 

«a n 5 2833230 mmSSU «.35 b~zo§Q 23 5 $33.88 83% xo\nm.:.<.53.:m- 3:3 .thsmtouxfis .EQEEQQ 3.53235 .33»:

N 033%

142

 

 

 

 

 

mc.vn.u...
iww. an. ...ov. 2. 2.- ac. co: no. we. 2. 2. no. 562.. 3.82 mmocoéootm .2
eunuch—atom o>zaav<
- ...wo. ...om. LN. Nor mo. co.. 3.- 2. 2. 2. we. mod hmém Season—0.2.8280 .2
- ...mm. LN. war 2. 3.- mo: 2. 2. we. 8. mm; v0: totmxmmh .2
- ...mm. 8.- we. 2.- S. 2. mo. 2. 2. 23 RN :oEm_:co< .2
032390— Ego—om
- no: 2. mo. 8.- 2. 8.- m _. no. 2 .o 8; 5:839 .m
- 2.- we. 2.- 2. 2. mo. 3. omd Ed 335385.02 .w
- 2.- .mm. 2. we. no. ...vm... $3 $6 oozotoaxm 9.0.8530 .n
- .3. w _ .. o_ .- 8. no. :3 and 30520356800 .c
- mo. o _. no. we. 3d 93” .0.0 wet—.83 .m
- 2. .. _ N. cm. a; 2.3 2.5 2,3:on .v
- no. no. 8.2%. 8.32 cop—agate; 838E .m
- __. 2 w. No; and oocotoaxm 65an0 .m
- an .o 92 3980 .—
2 Z S o w n c n v m m _ Om 2
80:22.80

 

«a n S 20.2.8989 mwtcfiu «85 wzmtomsob m5 :2 835.8; Auk SNmmstEEk 3:: ..302533385 ..ESBENQ E33235. .333
m 033.

143

mo.VQH§

 

2.2. ...mV. 2. no. co; 5. oo. 2. .mm. *3. 25.- vo. 2o.m2m 02.32: mmo=o>22outm2 .2
Seance-52 32:525-

 

 

 

- .8. .3. 8.- 8. :. 2.- 8.- 8.- 2.- 2.- .3..- ad :6 maoeowmwcm .2
$25 .29:

- 2. mo. oo. 3.. co. 2. we. no. we. no.- Nwd on. 2 2 to-tm xmwh .2 2
- 2o. 3. ..§. 8.- g. .8. 5.- 2.- .8- 82 2&- =2§=8< .2 4
03523052 255232 M

- no: co. no. .3”. 2 2. m 2. co. vo. wmd Na. 2 28220300 .9

- we. 22. 2.- aor S. no. 5. and de 22022220502222 0.32 .w

- v2. 3N. no. we. 02.- ...hm... emd 2m.m 32822095 8 3052220 .2.

- .. 2 v. mo.- mo. 2 o. 25.- wmd NV. m 30530225282200 .0

- mo. w 2. LVN. 0o. mmd 2w.m .0.0 wEFauA .m

- ...NN. mo... om; SN mcém b222n< 0322:on .v

- no. we. cod: oodwo2 accustom-Sm cocoa-2.2 .m

- .50. 022.2 wmd vocatoaxm 2322800 .m

- 0nd 3. 2 202200 . 2

N2 22 C2 a w 2. c w v m N 2 Om 2
22322.80

 

\3 n :2 2.625309 mwautb «.85 méBthooU m5 :2 $33.23; Auk xcxdflnmaaawz 33 5822523385 ..ESBENQ 3.62»ch 623:2

v Ens-2-

 

Table 5

Interactive Eflects of Task Eflort and Task Change Type on
Adaptive Performance Eflectiveness (n = 283)

 

 

 

 

DV Name
Step b R2 Total A R2
1. Gender .10 .19 .19*
Computer Experience .06
Trial 6 Performance .002*
2. Targets Prosecuted .66* .57 .39*
Task Change Code 1a .01
Task Change Code 2b -.03
3. Targets Prosecuted X -.03 .60 .02*
Task Change Code 1
Targets Prosecuted X -.34*
Task Change Code 2
* = p < .05

a Task Change Code 1 (O = Difficulty, 1 = Component)
b Task Change Code 2 (0 = Difficulty, 1 = Coordinative)

Table 6

Effects of the Use of New Information Cues on Component
Change Adaptive Performance Eflectiveness (n = 93 )

 

 

 

DV
Name
Step b R2 Total A R2
1. Gender 24.24 .63 .63*
Computer Experience 17.07
Trial 6 Performance .02
Targets Prosecuted 161 .75*
2. Correct TCI Decisions 159.31* .90 .26*

 

*=p<.05

 

145

Table 7
Eflects of Behavioral Act Prioritization on Coordinative
Change Adaptive Performance Eflectiveness (n = 95 )

 

 

 

DV Name
Step b R2 Total A R2
1. Gender 66.01 .41 .41*
Computer Experience -36.65
Trial 6 Performance .51 *
Targets Prosecuted 115.68*
2. High Priority Engagements -52.07 .42 .01
* = p < .05
Table 8

Effects of Relevant Declarative Knowledge Development on
Difficulty Change Task Effort (n = 95)

 

 

DV Name
Step b R2 Total A R2
1. Gender -.02 .36 .36*
Computer Experience .7 8*
Trial 6 Performance .01*
2. Relevant Declarative Knowledge .88 .37 .01

 

*=p<.05

146

 

Table 9
Ejfects of Relevant Declarative Knowledge Development on
Component Change Correct Use of New Information Cues (n = 93 )

 

 

 

DV Name
Step b R2 Total A R2
1. Gender -.31 .04 .04
Computer Experience .51
Trial 6 Performance .00
2. Relevant Declarative Knowledge 11.45* .35 .31*
* = p < .05
Table 10

Eflects of Relevant Declarative Knowledge Development on
Coordinative Change Behavioral Act Prioritization (n = 95)

 

 

DV Name
Step b R2 Total A R2
1. Gender -.27 .08 .08
Computer Experience -.03
Trial 6 Performance .00
Relevant Declarative
2 Knowledge .18* .17 .10*

 

*=p<.05

147

 

 

Table 11
Eflects of Change Detection on Relevant Declarative Knowledge
Development for Difficulty Change (n = 95)

 

 

 

 

DV Name

Step b R2 Total A R2
1. Gender .05 .05 .05

Computer Experience -.01

Trial 6 Performance .00 r: ..
2. Change Detection -.13 .06 .01
* = p < .05
Table 12 i;

Effects of Change Detection on Relevant Declarative Knowledge
Development for Component Change (n = 93 )

 

 

 

 

DV Name
Step b R2 Total A R2
1. Gender .04 .02 .02
Computer Experience .0]
Trial 6 Performance .00
2. Chang: Detection .28* .07 .05*
* = p < .05
Table 13

Ejfects of Change Detection on Relevant Declarative Knowledge
Development for Coordinative Change (12 = 95)

 

 

 

DV Name
Step b R2 Total A R2
1. Gender -.42 .04 .04
Computer Experience -.02
Trial 6 Performance .00
2. Change Detection .41 .05 .01

 

 

*=p<.05

148

Tab

lr-It'

Ta.)

(J)

Table 14
Interactive Eflects of the Metacognitive Intervention and Task
Change Me on Adaptive Performance Eflectiveness (n = 283)

 

 

 

DV Name
Step b R2 Total A R2
1. Gender .10 .19 .19*
Computer Experience .06
Trial 6 Performance .002*
2. M.C Intervention -.17 .20 .01
Task Change Code la .00
Task Change Code 2b -.08
3. M.C Intervention X -.24 .20 .00
Task Change Code 1
M.C Intervention X -.07
Task Change Code 2
* :p < .05

a Task Change Code 1 (0 = Difficulty, 1 = Component)
b Task Change Code 2 (O = Difficulty, 1 = Coordinative)

Table 15
Effects of the Metacognitive Intervention on Change Detection for
Difficulty Task Change in = 95)

 

 

DV Name
Step b R2 Total A R2
1. Gender -.16 .11 .11*
Computer Experience .06
Trial 6 Performance .00*
2. M.C Intervention -.O9 .13 .02

 

*=p<.05

149

* ll‘J
II .

Table 16
Efifzcts of the Metacognitive Intervention on Relevant Declarative
Knowledge Development for Difi‘iculty Task Change Jn = 95 )

 

 

 

DV Name
Step b R2 Total A R2
1. Gender .05 .05 .05
Computer Experience -.01
Trial 6 Performance .00
2. M.C Intervention -.02 .05 .00
* :p < .05
Table 17

Efifects of the Metacognitive Intervention on Difficulty Task
Change Eflort (n = 95)

 

 

 

 

DV Name
Step b R2 Total A R2
1. Gender -.02 .36 .36*
Computer Experience .78*
Trial 6 Performance .01*
2. M.C Intervention -.18 .36 .00
* = p < .05
Table 18

Effects of the Metacognitive Intervention on Change Detection for
Component Task Change (n = 93)

 

 

 

DV Name
Step b R2 Total A R2
1. Gender -.02 .04 .04
Computer Experience .04
Trial 6 Performance .00
2. M.C Intervention -.02 .04 .00

 

*=p<.05

150

Ste

«2... 2 ..9

Table 19
Ejfects of the Metacognitive Intervention on Relevant Declarative
Knowledge Development for Component Task Change (n = 93 )

 

 

 

 

DV Name
Step b R2 Total A R2
1. Gender .04 .02 .02
Computer Experience .0]
Trial 6 Performance .00
2. M.C Intervention -.02 .02 .00
* = p < .05
Table 20

Eflects of the Metacognitive Intervention on Component Change
Correct Use of New Information Cues (n = 93 )

 

 

 

 

DV Name
Step b R2 Total A R2
1. Gender -.31 .04 .04
Computer Experience .51
Trial 6 Performance .00
2. M.C Intervention -.48 .04 .00
* = p < .05
Table 21

Ejfects of the Metacognitive Intervention on Change Detection for

‘Coordinative Task Change (n = 95)

 

 

 

DV Name
itep b R2 Total A R2
1. Gender .01 .02 .02
Computer Experience .01
Trial 6 Performance .00
\2. M.C Intervention -.04 .02 .01

 

* =p<.05

151

*ll.)

Table 22
Efifects of the Metacognitive Intervention on Relevant Declarative
Knowledge Development for Coordinative Task Change (n = 99

 

 

 

 

DV Name
Step b R2 Total A R2
1. Gender -.42 .04 .04
Computer Experience -.02
Trial 6 Performance .00
2. M.C Intervention .32 .06 .02
* = p < .05
Table 23

EfléCtS of the Metacognitive Intervention on Coordinative Change
Behavioral Act Prioritization (n = 95)

 

 

 

DV Name
Step b R2 Total A R2
1. Gender -.27 .08 .08
Computer Experience -.03
Trial 6 Performance .00
2. M.C Intervention .11 .09 .01

 

*=p<.05

152

 

Table 34
Interactil‘l

arid-2pm

Slgp
1. Gem
Corr

Ilia

alat-l

Table 24
Interactive Eflects of Cognitive Ability and Task Change Type
on Adaptive Performance Eflectiveness (n = 283L

 

 

 

 

DV Name
Step b R2 Total A R2
1. Gender .10 .19 .19*
Computer Experience .06
Trial 6 Performance .002*
2. Cognitive Ability .03* .20 .01
Task Change Code 1‘31 -.03
Task Change Code 2b -.06
3. Cognitive Ability X -.03 .21 .01
Task Change Code 1
Cognitive Ability X .04
Task Change Code 2
* = p < .05

a Task Change Code 1 (0 = Difficulty, 1 = Component)
b Task Change Code 2 (O = Difficulty, 1 = Coordinative)

153

 

It‘ll:
lnlt‘r

m”
A:

$2)

'2’ J

Table 25
Interactive Eflects of Learning Goal Orientation and Task Change
Type on Adaptive Performance Eflectiveness Q1 = 283)

 

 

 

 

DV Name
Step b R2 Total A R2
1. Gender .10 .19 .19*
Computer Experience .06
Trial 6 Performance .002*
2. Learning G. O. .14 .19 .01
Task Change Code 18 .00
Task Change Code 2b -.07
3. Learning G. O. X -.29 .19 .00
Task Change Code 1
Learning G. O. X -.11
Task Change Code 2
* = p < .05

a Task Change Code 1 (0 = Difficulty, 1 = Component)
b Task Change Code 2 (0 = Difficulty, 1 = Coordinative)

154

2“- . A V

 

SJ

Table 26
Interactive Effects of Conscientiousness and Task Change Type
on Adaptive Performance Eflectiveness (n = 283)

 

 

 

DV Name
Step b R2 Total A R2
1. Gender .10 .19 .19*
Computer Experience .06
Trial 6 Performance .002*
2. Conscientiousness .01 .19 .00
Task Change Code la .00
Task Change Code 2b -.10
3. Conscientiousness X -. 10 .19 .00
Task Change Code 1
Conscientiousness X .06
Task Change Code 2
* = p < .05

a Task Change Code 1 (0 = Difficulty, 1 = Component)
b Task Change Code 2 (o = Difficulty, 1 = Coordinative)

155

Table I

intra-

lmt" (
44——

‘lfl

._Jr-\fl

Table 27
Interactive Eflects of Openness to Experience and Task Change

Type on Adaptive Performance Effectiveness (n = 283)

 

 

 

DV Name
Step b R2 Total A R2
1. Gender .10 .19 .19*
Computer Experience .06
Trial 6 Performance .002*
2. Openness to Experience .15 .19 .01
Task Change Code la -.02
Task Change Code 2b -.10
3. Openness to Experience X -.05 .19 .00
Task Change Code 1
Openness to Experience X -.18
Task Change Code 2
* = p < .05

a Task Change Code 1 (O = Difficulty, 1 = Component)
b Task Change Code 2 (0 = Difficulty, 1 = Coordinative)

156

Flare l

A Com

1

[~45
(D
m

APPENDIX J
Figures
Figure 1

A Control Theory-Based Heuristic of Self-Regulation

 

[Goal/Standard J

_______________ l

 

 

 

Comparison —> Discrepancy

 

 

 

 

 

 

/ l

 

 

 

 

 

 

 

 

 

Feedback Strategic/
Behavioral
Output
Behavior/ /
Performance

 

 

 

157

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

3§3§_

Roe-58$
3:0me

 

 

 

 

 

\

 

 

 

_m._o_>mcom

595

 

238m

 

\

 

 

sewage-.0200
850w

 

 

 

 

 

$052285
$2229..

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

sausage-<- mo 262.3022 2639.28222- 282280 <

N 2&5

158

 

 

8550th
9,822

 

 

 

 

om:

 

 

$6295 o2mtaoaa<

 

 

 

Ildlleoee o3.

 

eo_2_o_:oo<
382265..
2:96.61

 

 

 

5220630
omen-.220

 

 

 

Im2wloc.mm_ln_

 

cozoflmo

388m momma-8.2.8.2 e>22mo2o< 222 we 202522 28260 <

m 2:3

9
5
1|.

 

 

 

30:20 25:22:60 w 022222000 -
Sea 5:383. 68828.2 + 9288 u A .....
39.20 3.3055 -

 

 

eon. 8:383. Reese-.282 n A

 

 

 

mocmztoten.
o>22amn<

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

mocmco
term xmm... NV 2cm>o_mm
g Mammy-m

 

‘tllll

 

2.220900
69.95

 

 

2.0220200

380:2 858.5223 o>22n2e2o< 222 .3 20252 0226on 028220 23.2. <

a 8&5

160

Figure 5

The Interactive Effect of Task Related Effort and Task Change Type on Levels of
Adaptive Performance Effectiveness

1.200 ~——~f— 2 2 —~ -_ -

i

1.000 «f

 

0.800 J

0.600 J

|
0.400
l

 

l +Difficulty
—I- Component
+ Coordinativrfl

7
l

0.200 «5

 

Task Change Type

 

Adaptive Performance - 2 Score .

 

 

 

 

-0.600 ' _____- i .
- 1 SD +1 SD

Number of Targets Prosecuted

161

REFERENCES

Ackerman, P. L. (1992). Predicting individual differences in complex skill acquisition:
Dynamics of ability determinants. Journal of Applied Psychology, 77, 598-614.

Aguinis, H. (1995). Statistical power problems with moderated multiple regression in
management research. Journal of Management, 21, 1141-1 15 8.

Aguinis, H., Boik, R. J ., & Pierce, C. A. (2001). A generalized solution for
approximating the power to detect effects of categorical moderator variables using
multiple regression. Organizational Research Methods, 4, 291-323.

Aguinis, H., & Stone-Romero, E. F. (1997). Methodological artifacts in moderated
multiple regression and their effects on statistical power. Journal of Applied
Psychology, 82, 192-206.

Aiken, L. S., & West, S. G. (1991). Multiple regression: Testing and interpreting
interactions. Newbury Park, CA: Sage

Amabile, T. M. (1983). The social psychology of creativity: A componential
conceptualization. Journal of Personality and Social Psychology, 45, 357-3 76.

Ames, C. (1992). Classrooms: Goal, structures, and student motivation. Journal of
Educational Psychology, 84, 261-271 .

Anderson, C. A., Lindsay, A. J ., & Bushman, B. J. (1999). Research in the psychological
laboratory: Truth or triviality? Current Directions in Psychological Science, 8, 3-
9.

Anderson, J. R. (1982). Acquisition of cognitive skill. Psychological Review, 89, 369-
406.

Anderson, J. R. (1993). Problem solving and learning. American Psychologist, 48, 3544.

Aronson, E. & Carlsmith, J. M. (1968). Experimentation in social psychology. In G.
Lindzey & E. Aronson (Eds), The handbook of social psychology (V 01 2, pp. 1-
79). Reading, MA: Adddison—Wesley.

Aronson, 13., Wilson, T.D. & Akert, R. M. (1994). Social psychology (3“ ed.). New York:
Addison-Wesley

Ashton, M. C. (1998). Personality and job performance: the importance of narrow traits.
Journal of Organizational Behavior, 19, 289-303.

162

 

Baron, R. M, & Kenny, D. A. (1986). The moderator-mediator variable distinction in
social psychological research: Conceptual strategic, and statistical considerations.
Journal of Personality and Social Psychology, 51, 1173-1 182.

Barrick, M. R., & Mount, M. K. (1991). The big five personality dimensions and job
performance: A meta-analysis. Personnel Psychology, 44, 1-26.

Bell, B. S., & Kozlowski, S. W. J. (2002). Adaptive guidance: Enhancing self-regulation,
knowledge, and performance in technology-based training. Personnel Psychology,
55, 267-306.

Bell, B. S., & Kozlowski, S. W. J. (2008). Active learning: Effects of core training design
elements on self-regulatory processes, learning, and adaptability. Journal of
Applied Psychology, 93, 296-316.

Berardi-Coletta, 8., Buyer, L. S., Dominowski, R. L., & Rellinger, E. R. (1995).
Metacognition and problem solving: A process-oriented approach. Journal of
Experimental Psychology: Learning, Memory, and Cognition, 21, 205-223.

Blickle, G. (1986). Personality traits, learning strategies, and performance. European
Journal of Personality, 10, 337-3 52.

Borman, W. C., Motowidlo, S. J. (1997). Task performance and contextual performance:
The meaning for personnel selection research. Human Performance, 10, 99-109.

Brown, A. L. (1987). Metacognition, executive control, self-regulation, and other more
mysterious mechanisms. In F. E. Weinert & R. H. Kluwe (Eds), Metacognition,
motivation, and understanding (pp. 65-116). Hillsdale, New Jersey: Erlbaum.

Brown, A. L. Bransford, J. D., Ferrara, R. A., & Campione, J. C. (1983). Learning,
remembering, and understanding. In J. H. Flavell & E. M. Markrnan (Eds),
Handbook of child psychology (V 01. 3, pp. 77-166). New York: Wiley.

Brown, A. L., & Palincsar, A. S. (1982). Inducing strategic learning from texts by means
of informed, self-control training. Topics in Learning and Learning disabilities,
2, 1-1 7.

Brown, A. L., & Palincsar, A. S. (1985). Reciprocal teaching of comprehension
strategies: A natural history of one program for enhancing learning. In Garner,
R., & Alexander, P. A. (1989). Metacognition: Answered and unanswered
questions. Educational Psychologist, 24, 143-158.

Busato, C. C., Prins, F. J ., Elshout, J. J., & Harnaker, C. (1999). The relation between

learning styles, the Big Five personality traits, and achievement motivation in
higher education. Personality and Individual diflizrences, 26, 129-140.

163

 

Campbell, J. P. (1990). Modeling the performance prediction problem in industrial and
organizational psychology. In M. D. Dunnette & L. M. Hough (Eds), Handbook
of industrial and organizational psychology (V 01. 1., 2nd ed.; pp. 687-732. Pala
Alto, CA: Consulting Psychologists Press.

Campbell, J. P., McCloy, R. A., Oppler, S. H., & Sager, C. E. (1993). A theory of
performance. In N. Schmitt & W. Borman (Eds), Personnel selection in
organizations (pp. 35-70). San Francisco: Jossey-Bass.

Carroll, J. B. (1993). Human cognitive abilities: A survey of factor analytic studies. New
York: Cambridge University Press.

Carver, C. S, & Scheier, M. F. (1981). Attention and self-regulation: A control theory
approach to human behavior. New York: Springer-Verlag.

Carver, C. S, & Scheier, M. F. (1982). Control theory: A useful conceptual framework
for personality-social, clinical, and health psychology. Psychological Bulletin, 92,
l 1 1-135.

Carver, C. S, & Scheier, M. F. (2001). On the self-regulation of behavior. Cambridge,
England: Cambridge University Press.

Cascio, W. F. (2003). Change in workers, work, and organizations. In: R. J. Klimoski,
W. C. Borman, & D. R. Ilgen (Eds), Handbook of psychology (V 01. 12, pp. 401-
422). Wiley: Hoboken, NJ.

Chen, G., Thomas, B., & Wallace, J. C. (2005). A multilevel examination of the
relationships among training outcomes, mediating regulatory processes, and
adaptive performance. Journal of Applied Psychology, 90, 827-841.

Clause, C. S., Delbridge, K., Schmitt, N., Chan, D., & Jennings, D. (2001). Test
preparation activities and employment tests performance. Human Performance,
14, 149-167.

Cohen, J. (1969). Statistical power analysis for the behavioral sciences: Ist edition. New
York: Academic Press.

Colquitt, J. A. (2008). Publishing laboratory research in AMJ : A question of when, not if.
Academy of Management Journal, 51, 616-620.

Colquitt, J. A., & Simmering, M. J. (1998). Conscientiousness, goal orientation, and
motivation to learn during the learning process: A longitudinal study. Journal of
Applied Psychology, 83, 654-665.

164

Costa, P. T. Jr., & McCrae, R. R. (1992). Revised NEO Personality Inventory (NEO PI-
R) and NEO F ive-F actor Inventory (NEO-FFI) Professional Manual. Odessa,
FL: Psychological Assessment Resources, Inc.

Cronshaw, S. F., & J ethmalani, S (2005). The structure of workplace adaptive skill in a
career inexperienced group. Journal of Vocational Behavior, 66, 45-65.

Delclos, V. R., & Harrington, C. (1991). Effects of strategy monitoring and proactive
instruction on children’s problem-solving performance. Journal of Educational
Psychology, 83, 35-42.

Dweck, C. S. (1986). Mental processes affecting learning. American Psychologist, 41,
1040—1 048.

Dweck, C. S., & Leggett, E. (1988). A social-cognitive approach to motivation and
personality. Psychological Review, 95, 256-273.

Dwyer, D. J ., Hall, J. K., Volpe, C., Cannon-Bowers, J. A., & Salas, E. (1992,
September). A performance assessment task for examining tactical decision
making under stress (Spec. Rep. No. 92-002). Orlando, FL: Naval Training
Systems Center, Human Factors Division.

Earley, P. C., Connolly, T., & Ekegren, G. (1989). Goals, strategy development, and task
performance: Some limits on the efficacy of goal setting. Journal of Applied
Psychology, 74, 24-33.

Elliot, A. J ., & Church, M. A. (1997). A hierarchical model of approach and avoidance
achievement motivation. Journal of Personality and Social Psychology, 72, 218-
232.

Ertrner, P. A., & Newby, T. J. (1996). The expert learner: Strategic, self-regulated, and
reflective. Instructional Science, 24, 1-24.

Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-
developmental inquiry. American Psychologist, 34, 906-911.

Fine, S. A., & Cronshaw, S. F. (1999). Functional job analysis: A foundation for human
resources management. Mahwah, NJ: Erlbaum.

Ford, J. K., Smith, E. M., Weissbein, D. A., Gully, S. M., & Salas, E. (1998).
Relationships of goal orientation, metacognitive activity, and practice strategies
with learning outcomes and transfer. Journal of Applied Psychology, 83, 218-233.

Frese, M., Albrecht, K., Altrnann, A., Lang, J ., Papstein, P. V., Peyerl, R., et a1. (1988).
The effects of an active development of the mental model in the training process:

165

Experimental results in a work processing system. Behaviour and Information
Technology, 7, 295-304.

Frese, M. and Altrnann, A. (1989). The treatment of errors in learning and training. In L.
Bainbridge and S. A. R. Quintanilla (Eds), Developing skills with new technology
(pp. 65-86). Chichester: Wiley.

Garner, R., & Alexander, P. A. (1989). Metacognition: Answered and unanswered
questions. Educational Psychologist, 24, 143-158.

Gellatly, I. R. (1996). Conscientiousness and task performance: Test of cognitive process
model. Journal of Applied Psychology, 81, 474-482.

Gully, S. M., Payne, S. C., Kiechel, K. L., & Whiteman, J. K. (1999). Aflective reactions
and performance outcomes of error-based training. Paper presented at the 14th

annual conference of the Society for Industrial and Organizational Psychology,
Atlanta, GA.

Hesketh, B., & Neal, A. (1999). Technology and performance. In: D. R. Ilgen & E. D.
Pulakos (Eds), The changing nature of performance: implications for stafi‘ing,
motivation, and development (pp. 21-55). San Francisco, CA: Jossey-Bass.

Hunter, J. E. (1986). Cognitive ability, cognitive aptitudes, job knowledge, and job
performance. Journal of Vocational Behavior, 29, 340-362.

Hunter, J. E., & Hunter, R. F. (1984). Validity and utility of alternative predictors of
performance. Psychological Bulletin, 96, 72—98.

Hyland, M. E. (1988). Motivational control theory: An integrative fi'arnework. Journal of
Personality and Social Psychology, 55, 642-651.

Ilgen, D. R., & Pulakos, E. D. (Eds). (1999). The changing nature of performance:
Implications for stafling, motivation and development. San Francisco: J ossey-
Bass.

Ivancic, K. & Hesketh, B. (2000). Learning from errors in a driving simulation: Effects
on driving skill and self-confidence. Ergonomics, 43, 1966-1984.

James, L. R, & Brett, J. M. (1984). Mediators, moderators, and tests for mediation. 1
Journal of Applied Psychology, 69, 307-321.

Johnson, J. W. (2003). Toward a better understanding of the relationship between
personality and individual job performance. In Barrick, M. R. & A. M. Ryan
(Eds), Personality and work (pp. 82-120). San Francisco: J ossey-Bass.

Judge, T. A., & Ilies, R. (2002). Relationship of personality to performance motivation: A
meta-analytic review. Journal of Applied Psychology, 87, 797-807.

166

Kanfer, R. (1990). Motivation theory and Industrial/Organizational psychology. In M. D.
Dunnette and L. Hough (Eds), Handbook of industrial and organizational
psychology. Volume 1. Theory in industrial and organizational psychology
(pp.75-170). Palo Alto, CA: Consulting Psychologists Press.

Kanfer, R., & Ackerman, P. L. (1989). Motivation and cognitive abilities: An

integrative/aptitude-treatment interaction approach to skill acquisition. Journal of
Applied Psychologi, 74, 657-690.

Kanfer, R., Ackerman, P. L., & Heggestad, E. (1996). Motivational skills and self-
regulation for learning: A trait perspective. Learning and Individual Differences,
8, 185-209.

Karoly, P. (1993). Mechanisms of self-regulation: a systems view. Annual Review of
Psychology, 44, 23-52. J

 

Keith, N., & Frese, M. (2005). Self-regulation in error management training: Emotion
control and metacognition as mediators of performance effects. Journal of
Applied Psychology, 90, 677-691.

King, A. (1989). Effects of self-questioning training on college students’ comprehension
of lectures. Contemporary Educational Psychology, 14, 366-381.

King, A. (1990). Improving lecture comprehension: Effects of a metacognitive strategy.
Applied Educational Psychology, 5, 331-346.

King, A. (1991a). Facilitating elaborative learning through guided student-generated
questioning. Educational Psychologist, 2, 1 11-126.

King, A. (1991b). Effects of training in strategic questioning on children’s problern-
solving performance. Journal of Educational Psychology, 83 , 307-317.

King, A. (1992). Comparison of self-questioning, summarizing, and notetaking-review as
strategies for learning from lectures. American Educational Research Journal, 29,
303 -325.

King, A., & Rosenshine, B. (1993). Effects of guided cooperative-questioning on
children’s knowledge construction. Journal of Experimental Education, 6, 127-
148.

King, L. A, Walker, L. M., & Broyles, S. J. (1996). Creativity and the five factor model.
Journal of Research in Personality, 30, 189-203.

Klein, H. J. (1989). An integrated control theory model of work motivation. Academy of
Management Review, 1989, 1 50-1 72.

167

Klein, H. J. (1991). Control theory and understanding motivated behavior: A different
conclusion. Motivation and Emotion, 15, 29-4-4.

- Klein, G. A. (1993). A recognition-primed (RPD) model of rapid decision making. In G.
A. Klein, J. Orasanu, J. Calderwood, and D. MacGregor (Eds), Decision making
in action: Models and methods (pp. 138-147). Norwood, NJ: Ablex.

Kozlowski, S. W. J. (1998). Training and developing adaptive teams: Theory, principles,
and research. In J. A. Cannon-Bowers & E. Salas (Eds), Decision making under
stress: Implications for training and simulation (pp. 115-153). Washington, DC:
APA Books.

Kozlowski, S. W. J ., Gully, S. M., Brown, K. G., Salas, E., Smith, E. A., & Nason, E. R.
(2001). Effects of training goals and goal orientation traits on multi-dirnensional
training outcomes and performance adaptability. Organizational Behavior and
Human Decision Processes, 85, 1-31.

Kozlowski, S. W. J ., Gully, S. M., McHugh, P. P., Salas, E., & Cannon-Bowers, J. A.
(1996b). A dynamic theory of leadership and team effectiveness: Developmental
and task contingent leader roles IN G. R. Ferris (Ed.), Research in personnel and
human resource management (V 01. 14, pp. 253-305). Greenwich, CT: JAI Press.

Kozlowski, S. W. J ., Gully, S. M., Nason, E. R., & Smith, E. M. (1999). Developing
adaptive teams: A theory of compilation and performance across levels and time.
In D. R. Ilgen and E. D. Pulakos (Eds), The changing nature of performance:
Implications for stafiing, motivation and development (pp. 240-292). San
Francisco: Jossey-Bass.

Kozlowski, S. W. J ., Gully, S. M., Salas, E., & Cannon-Bowers, J. A. (1996a). Team
leadership and development: Theory, principles, and guidelines for training
leaders and teams. In M. Beyerlein, D. Johnson, & S. Beyerlein (Eds), Advances

in interdisciplinary studies of work teams: Team leadership. (Vol. 3, pp. 251-
289). Greenwich, CT: JAI Press.

Kraiger, K., Ford, J. K., & Salas, E. (1993). Application of cognitive, skill-based, and
affective theories of learning outcomes to new methods of training evaluation.
Journal of Applied Psychology, 78, 31 1-328.

Landine, J ., & Stewart, J. (1998). Relationship between metacognition, motivation, locus
of control, self-efficacy, and academic achievement. Canadian journal of
counseling, 32, 200-211.

LePine, J. A. (2003). Team adaptation and postchange performance: Effects of team
composition in terms of members’ cognitive ability and personality. Journal of
Applied Psychology, 88, 27-39.

168

LePine, J. A., Colquitt, J. A., & Erez, A. (2000). Adaptability to changing task contexts:
Effects of general cognitive ability, conscientiousness, and openness to
experience. Personnel Psychology, 53, 563-593.

Locke, E. A. (1968). Toward a theory of task motivation and incentives. Organizational
Behavior and Human Performance, 3, 157-189.

Locke, E. A. (1991). Goal theory vs. Control theory: Contrasting approaches to
understanding work motivation. Motivation and Emotion, 15, 9-28.

Locke, E. A., & Latham, G. P. (1990). A theory of goal setting and task performance.
Englewood Cliffs, NJ: Prentice Hall.

Lord, R. G., & Levy, P. E. (1994). Moving from cognition to action: A control theory
perspective. Applied Psychology: An International Review, 43, 335-367.

Louis, M. R., & Sutton, R. I. (1991). Switching cognitive gears: From habits of mind to
active thinking. Human Relations, 44, 55-76.

Lovett, M. C., & Schunn, C. D. (1999). Task representations, strategy variability, and
base-rate neglect. Journal of Experimental Psychology: General, 128, 107-130.

Marks, M. A., Mathieu, J. E., & Zaccaro, S. J. (2001). A temporally based framework
and taxonomy of team processes. Academy of Management Review, 26, 356-3 76.

McDaniel, M. A., & Schlager, M. S. (1990). Discovery learning and transfer of problem-
solving skills. Cognition and Instruction, 7, 129—159.

Miller, G. A., Galanter, E. & Pribaum, K. H. (1960). Plans and the structure of behavior.
New York, Holt, Rinehart, & Winston.

Minnaert, A., & J anssen, P. J. (1999). The additive effect of regulatory activities on top
of intelligence in relation to academic performance in higher education. Learning
and Instruction, 9, 77-91.

Mitchell T. R., & Daniels, D. (2003). Motivation. In W. C. Borman, D. R. Ilgen, & R. J.
Klirnoski (Eds) Handbook of psychology, vol. 12: Industrial organizational
psychology (pp. 225-254). New York: Wiley & Sons, Inc. .

Mohammed, S., Hamilton, K. & Lim, A. (2008). The incorpOration of time in team
research: Past, current, and future. In E. Salas, G. F. Goodwin, & C. S. Burke

(Eds), Team eflectiveness in complex organizations: Cross-disciplinary
perspectives and approaches (pp. 321-348). Mahwah, NJ: LEA.

Mook, D. G. (1983). In defense of external invalidity. American Psychologist, 38, 379-
387,

169

Murphy, K. R. (1989). Is the relationship between cognitive ability and job performance
stable over time? Human Performance, 2, 183—200.

Nelson, T. O. (1996). Consciousness and metacognition. American Psychologist, 51,
1 02-1 16.

Nelson, T. 0., & Dunlosky, J. (1991). When people’s judgments of learning (J OLs) are
extremely accurate at predicting subsequent recall: The “delayed-JOL effect”.
Psychological Science, 2, 267-270.

Newell, A., Shaw, J ., & Simon, H. (1962). The process of creative thinking. IN H.
Bruber, G. Terrell, & M. Wertheimer (Eds), Contemporary approaches to
creative thinking (pp. 63-119). New York: Atherton.

Nicholls, J. G. (1984). Achievement motivation: Conceptions of ability, subjective
experience, task choice, and performance. Psychological Review, 91, 328—346.

Osman, M. E., & Hannafin, M. J. (1992). Metacognition research and theory: Analysis
and implications for instructional design. Educational technology research and
development, 40, 83-99.

Palincsar, A. S., & Brown, A. L. (1984). Reciprocal teaching of comprehension-fostering
and comprehension-monitoring activities. Cognition and Instruction, 2, 117-175.

Payne, S. C., Youngcourt, S. S., & Beaubien, J. M. (2007). A meta-analytic examination
of the goal orientation nomological net. Journal of Applied Psychology, 92, 128-
1 50.

Pintrich, P. R., & DeGroot, E. V. (1990). Motivational and self-regulated learning
components of classroom academic performance. Journal of Educational
Psychology, 82, 33-40.

Ployhart, R. E., & Bliese, P. D. (2006). Individual adaptability (I-ADAPT) theory:
Conceptualizing the antecedents, consequences, and measurement of individual
differences in adaptability. In S. Burke, L. Pierce, & E. Salas (Eds),
Understanding adaptability: A prerequisite for effective performance within
complex environments, pp. 3-39. San Diego, CA: Elsevier.

Powers, W. T. (1973). Behavior: The control of perception. Chicago: Aldine.
Pulakos, E. D., Arad, S., Donovan, M. A., & Plamondon, K. E. (2000). Adaptability in

the workplace: Development of a taxonomy of adaptive performance. Journal of
Applied Psychology, 85, 612-624.

170

Pulakos, E. D., Schmitt, N., Dorsey, D. W., Arad, S., Hedge, J. W., & Borman, W. C.
(2002). Predicting adaptive performance: Further tests of a model of adaptability.
Human Performance, 15, 299-323.

Reeve, R. A., & Brown, A. L. (1985). Metacognition reconsidered: Implications for
intervention research. Journal of Abnormal Child Psychology, 13, 343-356.

Salas, E., & Cannon-Bowers, J. A. (2001). The science of training: A decade of progress.
Annual Review of Psychology, 52, 471-499.

Schank, R., & Abelson, R. (1977). Scripts, plans, goals, and understanding. Hillsdale,
NJ: Erlbaum.

Schmidt, F. L. (1988). The problem of group differences ability test scores in
employment selection. Journal of Vocational Behavior, 33, 272-292.

Schmidt, R. A., and Bjork, R. A. (1992) New conceptualizations of practice: common

principles in three paradigms suggest new concepts for training. Psychological
Science, 3, 207-217.

Schmidt, A., & Ford, J. K. (2003). Learning within a learner control training
environment: The interactive effects of goal orientation and metacognitive
instruction on learning outcomes. Personnel Psychology, 56, 405-429.

Schmidt, F., & Hunter, J. (1998). The validity and utility of selection methods in
personnel psychology: Practical and theoretical implications of 85 years of
research findings. Psychological Bulletin, 124, 262-274.

Schneider, R. J ., Hough, L. M., & Dunnette, M. D. (1996). Broadsided by broad traits:
How to sink science in five dimensions or less. Journal of Organizational
Behavior, 1 7, 639-655.

Schneider, W., & Chein, J. M. (2003). Controlled & automatic processing: Behavior,
theory, and biological mechanisms. Cognitive Science, 27, 525-559.

Schraw, G. (1998). Promoting general metacognitive awareness. Instructional Science,
26, 1 13-125.

Schraw, G., & Moshman, D. (1995). Metacognitive theories. Educational Psychology
Review, 7, 351-371.

Smith, E. M., Ford, J. K., & Kozlowski, S. J. (1997). Building Adaptive Expertise:
hnplications for Training Design Strategies. In M. Quinones (Ed), T raining for a
rapidly changing workplace (pp. 89-118). Washington, DC: American
Psychological Association Publications.

171

 

Snow, R. E., & Lohman, D. L. (1984). Toward a theory of cognitive aptitude for learning
from instruction. Journal of Educational Psychology, 76, 347—376.

Thoresen, C. J ., Bradley, J. C., Bliese, P. D., & Thoresen, J. D. (2004). The Big Five
personality traits and individual job performance growth trajectories in
maintenance and transitional job stages. Journal of Applied Psychology, 89, 83 5-
853.

Vancouver, J. B. (2000). Self-regulation in Industrial/Organizational Psychology: A tale
of two paradigms. In M. Boekaerts, P. R. Pintrich, & M. Zeidner, (Eds),
Handbook of Self-Regulation (pp. 303-341). San Diego, CA: Academic Press

VandeWalle, D. (1997). Development and validation of a work domain goal orientation
instrument. Educational and Psychological Measurement, 8, 995-1015.

Weiner, B. (1985). An attributional theory of achievement motivation and emotion.
Psychological Review, 4, 548-573.

Weiner, B., Russell, D., & Lerman, D. (1979). The cognition-ernotion process in

achievement-related contexts. Journal of Personality and Social Psychology, 3 7,
1 21 1 -l 226.

Weiner, N. (1948). Cybernetics: Control and communication in the animal and the
machine. Cambridge, MA: MIT Press.

Winne, P. H. (1995). Inherent details in self-regulated learning. Educational
Psychologist, 30, 173-187.

Winne, P. H. (1996). A metacognitive view of individual differences in self-regulated
learning. Learning and Individual Differences, 8, 327-353.

Wood, R. E. (1986). Task complexity: Definition of the construct. Organizational
Behavior and Human Decision Processes, 3 7, 60-82.

Wood, R. E., & Locke, E. A. (1990). Goal setting and strategy effects on complex tasks.
Research in Organizational Behavior, 12, 73-109.

172

 

M'lllllllllllllllll111111111111ltllllllllm
3 1293 03063 0127