I.
.f.\¢|

J. .\ N.

 

. .111...

(01...:

1a ..
“Mm“?
. «‘V. .

) 1

. i
£53.».

a. 1.3
. {$3

.3. A .1. ,
.i $33.5} .
.3 . a
H. 1

1» .
:1: ~bfl
XLf-I

zil.ﬁ

it
. . .1...

5“}:
ltI...

: 5!... in:
39.1%”.-

1
I:

 

.t.
Jrl

u.

‘t‘JII‘ \‘
1.}:
.zid‘IJ ‘
.cll.l|.|.. ,l\
‘IILV'. nil“
E.1.. I}-

‘u in
3L.

t: 13(91‘v
'si.!.4.1 .I'I X

RSITY LIBRANES

Illlllllllllllllllllllllllllllllll lllllll Ill

3 12930

 

 

 

 

 

This is to certify that the

thesis entitled

The Effects of Guidance on Learning and Training
Performance in a Complex Training Simulation

presented by

Bradford S. Bell

has been accepted towards fulﬁllment ‘1
of the requirements for L

l

l

l

M.A. degree in Psychology

 

WM;

Date ﬁﬂgz /777

0-7639 MS U is an Afﬁrmative Action/Equal Opportunity Institution

 

 

LIBRARY

Michigan State
University

 

 

PLACE IN RETURN BOX to remove this checkout from your record.
TO AVOID FINES return on or before date due.
MAY BE RECALLED with earlier due date if requested.

 

DATE DUE

DATE DUE

DATE DUE

 

 

 

 

 

 

 

 

 

 

 

 

 

 

11m chiRC/DateDmpGS—p.“

 

THE EFFECTS OF GUIDANCE ON LEARNING AND TRAINING PERFORMANCE
IN A COMPLEX TRAINING SIMULATION
By

Bradford S. Bell

A THESIS

Submitted to
Michigan State University
in partial fulﬁllment of the requirements
for the degree of

MASTER OF ARTS
Department of Psychology

1999

ABSTRACT
THE EFFECTS OF GUIDANCE ON LEARNING AND TRAINING PERFORMANCE
IN A COMPLEX TRAINING SIMULATION
By

Bradford S. Bell

Over the past two decades, a considerable amount Of research has examined the effects of
giving trainees control (i.e., sequence, content, pace) over their learning (Steinberg, 1977,
1989; Williams, 1993). The most consistent ﬁnding of this research has been that
trainees are not good judges of what or how much they need to practice and they do not
make good use of the control they are given. The current study examined the effects Of
two forms of guidance, behavioral\cognitive and affective, on learning and performance
in a complex training environment (Kozlowski, Weissbein, Brown, Toney, & Mullins,
1997). Overall, it was found that behavioral\cognitive guidance had a substantial effect
on trainees’ Study and practice sequence and also several self-regulation variables.
Affective guidance had a direct effect on the amount of effort trainees put forth on the
task. The process by which guidance inﬂuenced various training outcomes is discussed

as are the implications and limitations of the present study.

Dedicated to Robert Bell and Edward Harvey

One Taught Others and One Taught Himself
But Both Showed Me that there are Many Different Paths to Knowledge

iii

ACKNOWLEDGMENTS

There are many people who have, in one-way or another, helped me to reach this
point, and I would like to recognize them here. First, I would like to thank my chair,
Steve Kozlowski. I truly appreciate all the time and effort you have put into this project.
Thank you for providing the direction and support that have helped to keep this project
moving forward. Most of all, thank you for having the conﬁdence and trust to allow me
to learn so much along the way. You provide an excellent example Of what guidance
should be.

I would also like to thank my committee members, Rick DeShon and Kevin Ford,
for all their help with this project. Thank you for pushing me to more fully develop the
idea of affective guidance; it was a very educational exercise. Also, thank you for your
extremely helpful advice concerning publication and future research directions.

While the individuals listed above contributed directly to this project, there are
several people who had more of an indirect effect on this paper. I would like to thank
Aleks, Brad, Darin, and Kevin for their support, help, and advice throughout the last two
years. Graduate school would deﬁnitely be much duller without each of you. I would
also like to express a great deal of gratitude to my family for their love and support. I
would especially like to thank my parents for the interest they have taken in my education
since the very beginning. My educational success over the years is due in a large part to
your support and encouragement.

Last, but certainly not least, I would like to thank my wife Kelly for all her love

and support. Thank you for putting up with the long hours and listening to my

iv

complaints and discontents. There is no doubt in my mind that I would not be where I
am today if I did not have you by my Side. The future that I imagine and desire for us is
one my greatest sources Of motivation. Looking forward to seeing you each evening also
makes it a little easier to get my work done each day. You encourage me to work hard
and be the best that I can be, while at the same time helping me to keep it all in

perspective. I love you and thank you for everything.

TABLE OF CONTENTS

LIST OF TABLES .............................................................................................................. ix
LIST OF FIGURES ............................................................................................................ xi
INTRODUCTION ............................................................................................................... 1
REVIEW OF THE LITERATURE ..................................................................................... 4
Learner Control ........................................................................................................ 4
Learner control vs. program control ............................................................ 4

Adaptive Advisement ............................................................................................... 8
Guidance ................................................................................................................ 10
Independent Variable Constructs ........................................................................... 12
Behavioral and cognitive guidance ............................................................ 12

Affective guidance ..................................................................................... 13

Process Constructs ................................................................................................. 13
Self-efﬁcacy ............................................................................................... l3
Self-satisfaction .......................................................................................... 15

Attributions ................................................................................................ 1 6

On-task cognition ....................................................................................... 17

Effort Constructs .................................................................................................... 19

Time spent studying ................................................................................... 19

Time spent on feedback ............................................................................. 19

Relevance of Effort Constructs .............................................................................. 19
Practice sequence ...................................................................................... 19

Study sequence and content ....................................................................... 20

Outcome Constructs ............................................................................................... 20

Basic and strategic performance ................................................................ 20

Basic and strategic task knowledge ........................................................... 21
Generalization performance ....................................................................... 21
HYPOTHESES .................................................................................................................. 23
METHOD .......................................................................................................................... 33
Design .................................................................................................................... 33
Overview .................................................................................................... 33

Simulation .................................................................................................. 33

Skill components ........................................................................................ 34

Procedure ............................................................................................................... 36
Participants ................................................................................................. 36

Informed consent ....................................................................................... 37

vi

Experimental training ................................................................................. 37

Practice ....................................................................................................... 38
Generalization ............................................................................................ 39
Incentives ................................................................................................... 39
Manipulations ........................................................................................................ 40
Behavioral\cognitive guidance ................................................................... 40
Affective guidance ..................................................................................... 43
Measures ................................................................................................................ 47
Control variables ........................................................................................ 47

Basic and strategic task knowledge ........................................................... 48
Self-efﬁcacy ............................................................................................... 49
Self-satisfaction .......................................................................................... 50
On—task cognition ....................................................................................... 50
Attributions ................................................................................................ 50
Practice sequence ....................................................................................... 51

Study sequence ........................................................................................... 51

Study time .................................................................................................. 51

Time spent on feedback ............................................................................. 52

Basic and strategic performance ................................................................ 52
RESULTS .......................................................................................................................... 54
RM-MANCOVA ................................................................................................... 54
Relevance of Effort Variables ................................................................................ 55
Hypothesis 1 ............................................................................................... 55
Hypothesis 2 ............................................................................................... 56
Additional analyses .................................................................................... 56
Summary .................................................................................................... 56
Process Variables ................................................................................................... 57
Hypothesis 3a ............................................................................................. 57
Hypothesis 4 ............................................................................................... 57
Hypothesis 6 ............................................................................................... 58
Hypothesis 7a ............................................................................................. 58
Hypothesis 8a ............................................................................................. 58
Hypotheses 9a & 9b ................................................................................... 59
Additional analyses .................................................................................... 59
Summary .................................................................................................... 60

Effort Variables ...................................................................................................... 60
Hypothesis 3b ............................................................................................. 61
Hypothesis 5 ............................................................................................... 61
Hypothesis 7b ............................................................................................. 61
Hypothesis 8b ............................................................................................. 62
Additional analyses .................................................................................... 62
Summary .................................................................................................... 62
Outcome Variables — Practice ................................................................................ 63
Hypothesis 10a ........................................................................................... 64
Hypothesis 10b ........................................................................................... 64

vii

Hypothesis 10c ........................................................................................... 64

Hypothesis 10d ........................................................................................... 65
Additional analyses .................................................................................... 65
Summary .................................................................................................... 67
Outcome Variables - Generalization ..................................................................... 68
Hypothesis 1] ............................................................................................. 68
Additional analyses .................................................................................... 69
Summary .................................................................................................... 69
Overall Model Fit ................................................................................................... 70
DISCUSSION .................................................................................................................... 73
Behavioral\Cognitive Guidance ............................................................................. 74
Affective Guidance ................................................................................................ 79
Process Variables ................................................................................................... 80
Relevance of Effort, Level of Effort, and Outcome Variables .............................. 81
Generalization Performance ................................................................................... 82
Overall Model Fit ................................................................................................... 83
Limitations ............................................................................................................. 85
Research and Practical Implications ...................................................................... 87
Conclusion ............................................................................................................. 88
FOOTNOTES .................................................................................................................... 9O
APPENDICES
APPENDD( A: Informed Consent ..................................................................................... 91
APPENDD( B: Debrieﬁng Form ....................................................................................... 92
APPENDIX C: Tag Topics Sheet ...................................................................................... 93
APPENDIX D: Tag Topics Sheet ...................................................................................... 94
APPENDIX E: Measures ................................................................................................... 95
APPENDIX F: Incentive Instructions .............................................................................. 106
APPENDD( G: Example of Behavioral and Cognitive Guidance ................................... 107
APPENDD( H: Affective Guidance ................................................................................ 109
APPENDIX 1: Example of Feedback .............................................................................. 111
APPENDD( J: Practice and Study Sequence and Content Measure ................................ 112
APPENDIX K: Hypotheses and Results .......................................................................... 113
APPENDD< L: Tables ...................................................................................................... 116
APPENDIX M: Figures ................................................................................................... 141
REFERENCES ................................................................................................................ 157

viii

LIST OF TABLES

Table 1 — Means, Standard Deviations, and Intercorrelations for Variables Included
in Analyses ...................................................................................................... 116
Table 2 — RM-MANCOVA for Affective Guidance and Behavioral and Cognitive
Guidance on Dependent Measures During Training ....................................... 122
Table 3 - Between-subjects Effects from RM-MANCOVA for Affective Guidance ..... 123
Table 4 — Between-subjects Effects from RM-MANCOVA for Behavioral and
Cognitive Guidance ......................................................................................... 124
Table 5 — Summary Of Hierarchical Regression Analysis for Variables Predicting
On-task Cognition at Time 1 (N = 276) ........................................................... 125
Table 6 — Summary of Hierarchical Regression Analysis for Variables Predicting
On-task Cognition at Time 2 (N = 276) ........................................................... 126
Table 7 — Summary Of Hierarchical Regression Analysis for Variables Predicting
Total Time Spent Studying (N = 276) ............................................................ 127
Table 8 -— Summary of Hierarchical Regression Analysis for Variables Predicting
Time Spent Reviewing Feedback (N = 276) .................................................. 128
Table 9 -— Summary of Hierarchical Regression Analysis for Variables Predicting
Basic Knowledge at Time 1 (N = 276) ........................................................... 129
Table 10 — Summary of Hierarchical Regression Analysis for Variables Predicting
Strategic Knowledge at Time 2 (N = 276) ..................................................... 130
Table 11 - Summary of Hierarchical Regression Analysis for Variables Predicting

Basic Performance at Time 1 (N = 276) ........................................................ 131

ix

Table 12 -— Summary Of Hierarchical Regression Analysis for Variables Predicting
Strategic Performance at Time 2 (N = 276) ................................................... 132
Table 13 — Summary Of Hierarchical Regression Analysis for Variables Predicting
Basic Knowledge at Time 1 (N = 276) .......................................................... 133
Table 14 — Summary of Hierarchical Regression Analysis for Process Variables
Predicting Strategic Knowledge at Time 2 (N = 276) ................................... 134
Table 15 — Summary of Hierarchical Regression Analysis for Process Variables
Predicting Basic Performance at Time 1 (N = 276) ....................................... 135
Table 16 - Summary Of Hierarchical Regression Analysis for Process Variables
Predicting Strategic Performance at Time 2 (N = 276) ................................. 136
Table 17 - Summary of Hierarchical Regression Analysis for Practice Variables
Predicting Basic Performance During Generalization (N = 276) .................. 137
Table 18 — Summary of Hierarchical Regression Analysis for Practice Variables
Predicting Strategic Performance During Generalization (N = 276) ............. 138
Table 19 — MANCOVA for Affective Guidance and Behavioral and Cognitive
Guidance on Basic and Strategic Performance During Generalization ......... 139
Table 20 — Between-subjects Effects from Generalization Trial MANCOVA for

Behavioral and Cognitive Guidance .............................................................. 140

LIST OF FIGURES

Figure 1 — Overall model ................................................................................................. 141
Figure 2 — Appropriate sequence Of trainee’s practice and study .................................... 142

Figure 3 — Path Model used to test the factor structure of basic and strategic

knowledge scales ............................................................................................ 143
Figure 4 — Effect of behavioral\cognitive guidance on practice sequence ...................... 144
Figure 5 - Effect of behavioral\cognitive guidance on study sequence .......................... 145
Figure 6 - Effect of behavioral\cognitive guidance on self-satisfaction ......................... 146

Figure 7 — Relationship between behavioral\cognitive guidance and self-efﬁcacy ........ 147

Figure 8 - Effect Of on-task cognition on total study time .............................................. 148
Figure 9 — Effect of affective guidance on total study time ............................................. 149
Figure 10 - Relationship between practice sequence and basic knowledge .................... 150
Figure 11 — Relationship between practice sequence and strategic knowledge .............. 151
Figure 12 — Relationship between practice sequence and strategic performance ............ 152

Figure 13 — Relationship between basic performance during practice and basic
performance during generalization ............................................................... 153

Figure 14 — Relationship between strategic performance during practice and

strategic performance during generalization ................................................. 154
Figure 15 - Path diagram used to test hypothesized model ............................................. 155
Figure 16 — Path diagram used to test revised model ...................................................... 156

xi

INTRODUCTION

AS technology becomes an increasingly more important part Of today’s work
environment, the nature of work in today’s organizations is changing (T umage, 1990).
Technology has taken over more of the mundane vigilance and information processing
tasks, Often requiring fewer people on the job (Kozlowski et al., 1997). In addition,
technology has created jobs that are more cognitively complex and demanding (Smith,
Ford, & Kozlowski, 1997). The work itself is increasingly dynamic, requiring that
employees adapt their knowledge and skills to new situations and problems (Kozlowski,
in press; Smith et al., 1997). As a result of these changes, the skills of individuals will
become more, rather than less, important, and training will play a critical role in
providing individuals with these Skills (Kozlowski et. a1, 1997).

Although training requirements have increased, the time and resources allocated
to training, in many instances, have not increased proportionately. This has led
researchers to search for new ways to train people on complex tasks. The result has been
research in areas such as computer-based training (CBT), simulations, and distance
learning. These are all relatively new methods of using today’s advanced technology in
an attempt to more effectively and efﬁciently train individuals. These training methods
have been Shown to lead to several positive training outcomes, including improved test
performance and attitudes (Duncan, 1991; Goolkasian, 1989; Marcoulides, 1990;
Worthington, Welsh, Archer, Mindes, & Forsyth, 1996). However, the greatest beneﬁt Of

these new methods is Often their cost. Using CBT or Simulations, the overall training

time and student-instructor contact time can usually be lessened considerably with little
or no performance decrements (Dossett & Hulvershom, 1984; Wexley, 1984).

Another advantage of computer—based training and instruction is that these
applications often allow the trainee to have some control (e.g., Speed, content, sequence)
over his or her learning process. Over the past two decades, a considerable amount of
research has examined the effects of giving individuals control over their learning (for
reviews see Steinberg, 1977, 1989; Williams, 1993). Although there has been several
problems with this research, ranging from mixed and equivocal ﬁndings (Kinzie,
Sullivan, & Berdel, 1988; Steinberg, 1989) to theoretical, definitional, methodological,
and analytical deﬁciencies (Reeves, 1993), one of the most consistent ﬁndings is that
individuals are not good judges of what or how much they need to practice (Tennyson &
Rothen, 1979; Tennyson, 1980; Williams, 1993). When given control over their learning,
most individuals will either exit instruction before they have mastered the task
(Tennyson, Tennyson, & Rothen, 1980; ROSS & Rakow, 1981) or practice well beyond
the point at which concepts have been acquired (Tennyson 1980, 1981). These ﬁndings
led researchers to conclude that it is necessary to supplement learner control with some
form of advisement or guidance which will provide information needed for making
learning decisions, such as how much to practice (Tennyson, 1980, 1981; Tennyson &
Buttrey, 1980; Tennyson & Rothen, 1979).

The interest in combining learner control with advisement has led to a great deal
of research on computer aided instruction (CAI), computer based instruction (CBI), and
intelligent tutors. These computer systems and other adaptive algorithms attempt to

monitor trainee development and guide the trainee on what he or She Should be Studying

or practicing. Although learner control combined with advisement has generally proved
to be more effective than learner control alone (e. g., Tennyson, 1980; Steinberg, 1989),
this research has been limited by its focus on simple tasks. In his comprehensive review
of the learner control literature, Williams (1993) noted that most of the research on
learner control has been conducted using drill and practice or tutorial programs with little
research being conducted with simulation, hypermedia, or tool software.

In this study, I focused on an advisement strategy referred to as guidance. A great
deal of research has focused on feedback as a means of providing trainees with
information about their performance (for a review see Kluger & DeNiSi, 1996); however,
feedback focuses on past behavior and is not diagnostic about what trainees should do,
think, or feel next. Whereas feedback is past oriented, guidance is future oriented and
provides an interpretation of “what next” (Kozlowski et al., 1997).

In this study, I examined two forms of guidance, behavioral\cognitive and
affective, and how they operated in a complex training situation. Behavioral guidance
directs the trainee about what actions to do next. Cognitive guidance directs the trainee
about what to think, such as concentrating on particular practice strategies. Affective
guidance directs the trainee to exercise emotional control with respect to his or her
reactions to practice feedback and provides strategies to help the trainee control his or her
emotions. These three forms of guidance should help the trainee to better judge his or her
progress and determine the cognitive, behavioral, and affective Strategies that aid
improvement (Kozlowski et al., 1997). Thus, guidance should act to enhance self-

regulation, learning, and performance.

REVIEW OF THE LITERATURE

Learner Control

The idea of providing individuals with control over their learning was borne out
of research designed to improve the quality of education as well as to ﬁnd methods to
educate a broader segment of society (Steinberg, 1977). The application of computers to
education made individualization even more feasible, and led to the development of
computer-managed (CMI) and computer-assisted (CAI) instruction. Computers enabled
researchers to go beyond programmed instruction and to individualize instruction
(Bunderson, 1970). The result has been a great deal of research on the effects of leaving
decisions on instructional management, either in part or in entirety, to the student
(Steinberg, 1977).

garner control vs. program control. In one of the first studies on learner control,
Judd, Bunderson, and Bessent (1970) examined the effects of four levels of student
control over course flow in a remedial mathematics course for college students. The four
levels of student control were (1) total computer management; (2) student control over
the sequence of topics from a table of contents; (3) additional student control of the
amount of practice; and (4) total learner control (Judd et al., 1970). The researchers
found that students under learner control did worse than those under computer control
and that learner control did not improve student attitudes. Also, contrary to expectations,
students were good judges of how much to practice. However, they didn’t practice the
appropriate material. In other words, they displayed ineffective learning strategies (Judd

et al., 1970).

Additional studies examining the differences between learner control and program
control were conducted using two major CAI systems, TICCIT (Faust, 1974) and PLATO
(Alpert & Bitzer, 1970). Whereas a basic premise of the TICCIT system was that
students should control course ﬂow, the PLATO system gave lesson authors complete
control over educational decisions. Studies conducted using these two systems provide
useful information about learner control.

The TICCIT system was developed Speciﬁcally to teach concepts and principles.
The student chose his or her own learning strategy, using Special keys on the terminal that
accessed examples, rules, practice, help, advice, and difﬁculty level. Using the TICCIT
system, Bunderson (1976) conducted a study in a college mathematics course. Overall,
students using the TICCIT system did no worse in achievement than regular classroom
students although they had not completed all the lessons. However, those students who
did not do well exhibited inefﬁcient learning Strategies. Bunderson (1976) concluded
that Students needed advice on management of time and review strategies.

In a Navy setting, Lahey, Crawford, and Hurlock (1975) used the PLATO CAI
system as a means for applying the TICCIT instructional paradigm. Control students
used Standard booklets, while experimental students used PLATO. Both groups were
self-paced, but the PLATO students also controlled their own instructional Strategies.
There were no signiﬁcant post-test performance differences between the groups, but the
PLATO students took longer to complete the lessons. The authors attributed the time
difference to overleaming on the part of the students using PLATO (Lahey et al., 1975).

Since these ﬁrst studies comparing learner control to program control, a

signiﬁcant amount of research has continued to explore the differences between these two

instructional strategies (for reviews see Steinberg, 1977, 1989; Williams, 1993). Much of
this research has examined the effects of giving the learner control over different aspects
of his or her instruction, such as sequence, content, or pace. This research has resulted in
not only mixed and inconsistent ﬁndings but also methodological criticisms (see Reeves,
1993, for a thorough critique). For example, much of the past research on learner control
has failed to control for various individual difference variables, such as cognitive ability,
that may influence training outcomes. The result is that it is difﬁcult to determine the
actual effects of giving individuals control over their learning. In the next section, I
provide a brief review of these mixed ﬁndings and discuss the implications of this
research (see Williams, 1993, for a complete review).

A few studies have supported the use of learner control of at least some
instructional events (e.g., Avner, Moore, & Smith, 1980; Kinzie, Sullivan, & Berdel,
1988; Shyu & Brown, 1992). Most of these studies support Hannaﬁn’s (1984)
suggestion that learner control results in a deeper or more lon g-lastin g effect on memory.
For example, Mayer ( 1976) found that more complex outcomes were learned better when
learners had control over the order of presentation, but Simple outcomes were learned
better under experimenter-controlled instruction.

In contrast, numerous studies (e. g., Morrison, Ross, & Baldwin, 1992; Tennyson
& Buttrey, 1980; Tennyson, Park, & Christiansen, 1985; Tennyson, Tennyson, & Rothen,
1980) have found various types or degrees of program control superior to learner control
over the same instructional elements for posttest achievement. Most of these researchers
have concluded that learners do not have or do not know how to utilize appropriate

strategies when they are left to themselves to manage their own learning. In a meta-

analysis of 10 years of interactive video instruction, McNeil and Nelson (1991)
concluded generally that program-controlled conditions are superior to learner—controlled
conditions. They suggest, however, that partial or “guided” learner control over review
and practice activities might be better for learning than program control over these
activities (McNeil & Nelson, 1991).

Most studies, however, have found no overall differences between learner-
controlled and program—controlled treatments (e. g., Balson, Manning, Ebner, & Brooks,
1984/1985; Carrier, Davidson, & Williams, 1985; Goetzfried & Hannaﬁn, 1985; Gray,
1987; Judd et al., 1970; Lahey et al., 1976; Pridemore & Klein, 1990). Various
conclusions have been drawn from these “no difference” ﬁndings. Some researchers
have used this ﬁnding to support the use of learner control, stating that programming the
computer to handle “branching” tasks is far too difﬁcult. They believe, therefore, that it
is easier to just allow students to handle their own branching (Williams, 1993). Other
researchers use these ﬁndings to support the use of program control, stating that other
effects of learner control, such as increased time due to overleaming, make program
control more efficient.

It is obvious that the above ﬁndings do not clearly support or reject the use of
learner control as an instructional approach. In his review, Williams (1993) concludes
that there are good reasons to believe that learner control is a desirable instructional
approach, but also notes that the majority of studies have found that students left on their
own do not make good use of strategies or time. In general, it has been found that many
students are just not capable of making good use of the control they are given (Williams,

1993). This ﬁnding led researchers to examine whether supplementing learner control

with some form of adaptive advisement would be more beneﬁcial than total learner
control. The ﬁndings of this research are the focus of the next section.
Adaptive Advisement

Learner control with advisement is an attempt to take advantage of the best
features of both adaptive computer control and learner control. Adaptive control
individualizes instruction by using a mathematical or statistical algorithm to determine a
learner’s understanding and to prescribe a sequence of instruction. It has been Shown that
students do not perform as well under learner control as under adaptive computer control
(e. g., Tennyson, Tennyson, & Rothen, 1980). This is due in part to the fact that when
given control of their learning students often exit the lesson too soon. Under learner
control, students view significantly fewer examples and complete the lesson in
Si gnificantly less time (Tennyson, Tennyson, & Rothen, 1980; Ross & Rakow, 1981).

Although adaptive control has advantages over learner control, it has some
disadvantages too (Park & Tennyson, 1983). Most important is that adaptive control
assumes all the responsibility for learning, rather than giving it, or at least some of it, to
the student (Tennyson & Buttrey, 1980). This led researchers to suggest that if students
are to be responsible for their learning, they need information for making learning
decisions, such as what and how much to practice (Tennyson & Rothen, 1979).

To examine this idea, Tennyson (1980) conducted a study to test the hypothesis
that learner control with adaptive advisement is more effective than either adaptive
computer control or learner control without advisement. In the adaptive control
condition, all decisions about sequence and amount of instruction were made by the

adaptive instructional program. Under adaptive learner control, students received

diagnostic and prescriptive information but controlled the decision to make use of it. In
the third condition, students were under the learner control condition but received no
advisement. Performance was about the same in the adaptive computer and adaptive
learner control conditions, but both conditions were better than learner control without
advisement. Tennyson (1980, p. 531) concluded that advisement was “highly Significant
in providing Students in the learner-adaptive control condition with meaningful
information with which to make appropriate decisions about acquisition of the coordinate
concepts.” Students in the learner control condition took less time to complete the lesson,
indicating that they exited instruction before they had mastered the task. Similar results
were obtained by Tennyson and Buttrey (1980) in a study on high school seniors’
acquisition of concepts in psychology.

In a more recent study, Santiago and Okey (1992) compared three types of
advisement. One group received adaptive advisement, which was advice on current
learning needs in terms of amount and sequence of instruction needed for the task at
hand. Another group received evaluative advisement, which provided learners with
information about their current states of knowledge. A third group received a
combination of these two forms of advisement. Although there were no signiﬁcant
differences between the achievement of students in the adaptive and combined groups
and in the evaluative and combined groups, adaptive advisement resulted in signiﬁcantly
higher post-test scores than the evaluative advisement (Santiago & Okey, 1992). In
addition, students in the adaptive advisement group found it much easier to make
decisions about their instruction than did students in the evaluative advisement group.

The authors concluded that “These ﬁndings indicate that providing speciﬁc information

adapted to learners’ needs results in better performance than information on learners’
current states of knowledge” (Santiago & Okey, 1992, p. 50).

These ﬁndings form the basis for the training strategy, referred to as guidance,
which was examined in the present study. However, the current study built on past
research in several ways. First, most of the research on both learner control and
advisement has focused on one main dependent variable, posttest performance (i.e.,
declarative knowledge). Little attention has been focused on process variables, such as
self-efﬁcacy, motivation, and attributions, that might mediate the effects of guiding
individuals through the learning process. The present Study examined how guidance
effected these process variables, and how, in turn, different levels of these variables
inﬂuenced performance. Second, research on advisement has not examined the use of
this strategy for influencing individuals’ affect. In the present study, one form of
guidance, affective, was aimed at controlling trainees’ reactions to their feedback.
Finally, most of the research on learner control and advisement has been conducted using
Simple learning tasks. In the present Study, guidance was applied to a complex and
difﬁcult training simulation.

Guidance

Guidance involves giving a trainee information regarding future directions that
should be taken for improvement. Guidance can be thought of as “feedforward,” more
than “feedback” (Kozlowski et al., 1998). Guided information can describe the behaviors
a trainee Should next engage in (Earley, Connolly, & Ekegren, 1990), what a trainee
should think about and how to think about it (e. g., metacognitive strategies; Nelson,

Dunlosky, Graf, & N arens, 1994), and what emotions the trainees might next encounter

10

and how to handle that affect in a constructive way. Thus, guidance Should be considered
future oriented, and can be deﬁned as any information that directs future cognition,
behavior, and/or affect (Kozlowski et al., 1997, 1998). Guidance provides an
interpretation of what next.

By enhancing trainee self-regulation, guidance can help the trainee to better
calibrate progress and determine the cognitive, behavioral, and affective strategies that
aid improvement (Kozlowski et al., 1997). Guidance helps the trainee to recognize areas
of needed improvement, and directs the trainee on what can be done to achieve desired
levels of performance. In addition, guidance acts to sequence the learning of trainees, so
that they build fundamental skills before learning more strategic aspects of the domain.

The model shown in Figure 1 depicts the processes by which both
behavioral\cognitive guidance and affective guidance were predicted to affect learning
and training performance. There are ﬁve distinct sets of variables that make up the
model: (1) independent variables; (2) process variables; (3) effort variables; (4) relevance
of effort variables; and (5) outcome variables. There are two independent variables,
behavioral\cognitive guidance and affective guidance. Behavioral\cognitive guidance
and affective guidance were both hypothesized to effect several process variables.
Speciﬁcally, behavioral\cognitive guidance was predicted to influence trainees’ self-
efﬁcacy and self-satisfaction. Affective guidance was predicted to effect trainees’ self-
efﬁcacy, on-task cognition, and attributions. In addition, behavioral\cognitive guidance
was hypothesized to have a direct effect on two relevance of effort variables, practice
sequence and content and study sequence and content. The process variables were

predicted to mediate the effect of the two forms of guidance on two main effort variables,

11

time Spent studying and time spent on feedback. These effort variables and the sequence
and content of trainees’ study and practice were expected to affect two main outcome
variables, trainees’ knowledge and performance on the task, although to differing
degrees. Finally, trainees’ knowledge and performance on the task during practice was
hypothesized to inﬂuence performance on the ﬁnal, more complex generalization trial. I
will now present each of these constructs, followed by Speciﬁc hypotheses.
Independent Vangble Constructs

Behavioral_and cognitive guidance. Although behavioral and cognitive guidance
are two separate constructs, they are complementary and work together. It is often
difﬁcult to provide one without the other, and thus were combined as an independent
variable in the present Study. Behavioral guidance directs trainees on what they should
be doing next (Kozlowski et. al, 1997, 1998). AS its name suggests, behavioral guidance
focuses on the actions and behaviors that are necessary to reach a desired level of
performance or knowledge on a task. For example, behavioral guidance may direct a
trainee on what Skills or Strategies he or she needs to practice. Cognitive guidance directs
trainees on what they should be thinking about or learning (Kozlowski et al., 1997, 1998).
This form of guidance focuses on what trainees need to learn to achieve desired levels of
performance. For example, cognitive guidance may instruct a trainee to focus on
studying Speciﬁc information which their past performance indicates they have not yet
learned. Both behavioral and cognitive guidance provide the trainee with information
necessary for judging his or her performance on speciﬁc aspects of a task (Kozlowski et
al., 1997). These two forms of guidance Should focus trainee self-regulation on task-

relevant material and facilitate appropriate learning and practice Strategies, resulting in

12

increased knowledge and improved performance. In the present Study, the guidance
given focused on behavioral and cognitive aspects of learning, and was based on the
trainee’s own past practice sessions. In this way, the behavioral and cognitive guidance
was adaptive to trainee progress.

Affective guidance. Affective guidance directs trainees to exercise emotional
control with respect to their feedback (Kozlowski et al., 1997). It focuses on the
emotions trainees are likely to encounter and provides trainees with information and
strategies to help them handle their affect (Kozlowski et al., 1998). Affective guidance
may, for example, instruct a trainee to expect poor performance in the beginning of a task
because the task is difﬁcult and takes time to learn. The affective guidance may also
provide trainees with Strategies and techniques, such as positive self—talk, imagery, and
deep breathing, that they can use to control their emotions. In the present study, the
affective guidance was not adaptive to individual trainee performance. However, the
affective guidance was scripted so as to provide unique and novel guidance on each
presentation and was designed to coincide with how most trainees should feel about their
performance relative to where they are in the training session.

Process Constructs

Self-efficacy. Self-efﬁcacy is an individual’s belief about whether he or she can
perform a task or behavior. In this way, self-efﬁcacy can be thought of as competency
beliefs one holds about oneself. It iS partly on the basis of self-perceptions of efﬁcacy
that individuals choose what challenges to undertake, how much effort to expend in an
endeavor, and how long to persevere in the face of difﬁculties (Bandura, 1982, 1986a).

Self-efﬁcacy is most often directly relevant to a particular task, but can also be

13

generalized to other tasks and Situations (Bandura, 1977). Past performance on a task is
considered to be the best indicator of future performance (Gist & Mitchell, 1992; Silver,
Mitchell, & Gist, 1995). Successful experiences typically lead to increases in self-
efﬁcacy; failures undermine it.

Whether perceived discrepancies between personal standards and attainments are
motivating or discouraging is likely determined by the strength of people’s perceived
capabilities to attain the standards they are pursuing (Bandura & Cervone, 1983, 1986a).
Those whose distrust their capabilities are easily discouraged by failure, whereas those
who are highly assured of their efficacy for goal attainment will increase their efforts
when their performances fall short and persevere until they succeed (Bandura & Cervone,
1983, 1986a; Brown & Inouye, 1978; Cervone & Peake, 1986; Schunk, 1984; Weinberg,
Gould, & Jackson, 1979). Inherent within this conceptualization of self-efficacy iS the
idea that increased effort will lead to increased performance. Many of the Studies that
have examined self-efficacy (e. g., Bandura & Cervone, 1983, 1986a) have employed a
physical task, for which the belief that one can complete the task translates quite easily
into an intention, and subsequently, into successful performance. There is a relatively
direct relationship between effort and performance. However, for complex tasks, like the
one used in the present study, it was more appropriate to conceptualize self-efficacy as a
more complex psychological construct that captures a broader self-perception of
competence. Here the relationship between self-efﬁcacy, effort, and performance is not

as direct, it is a relationship mediated by numerous process variables.

14

Self-satisfaction. Self-satisfaction is an affective reaction to behavior that
contributes to motivation. Satisfaction with oneself is a direct result of comparing one’s
performance with one’s goal for that performance or some set standard of performance
(Bandura & Cervone, 1983). When a goal is attained or surpassed, a person will
experience the positive affect of satisfaction. When a goal is not attained, a person will
experience the negative affect of dissatisfaction.

Findings have shown that the higher the self-dissatisfaction with substandard
attainments, the greater is the subsequent intensiﬁcation of effort (Bandura & Cervone,
1983; Locke, Cartledge, & Kerr, 1970). Research has also Shown that when there is a
simple and direct relation between effort and performance, self-dissatisfaction that
enhances effort will positively affect performance (Bandura & Cervone, 1983, 1986b).
However, on complex tasks, the negative affect due to self—evaluations can undermine
performance by interfering with cognitive processes critical to success (Cervone, 1993).
In a Study employing a complex decision-making task, Cervone, J iwani, and Wood
(1991) found that individuals who became dissatisﬁed with their performance exerted a
great deal of effort in an attempt to improve; however, in doing so, they applied a number
of conﬂicting strategies which inhibited their performance. These ﬁndings suggest that
individuals learning complex tasks must be given proper guidance on what they should
do to improve their performance and eliminate self-dissatisfaction, or their effort may be
misguided or counterproductive.

Self-satisfaction or self-dissatisfaction can be either motivating or demotivating
(Bandura, 1991; Carver & Scheier, 1990). Performance that falls considerably short of

standards is likely to give rise to discouragement and goal abandonment (Bandura &

15

Cervone, 1983). Moderately discrepant performances, which make the standard appear
attainable (Atkinson, 1964; Locke, 1968), are likely to activate self-dissatisfactions that
spur efforts to bring performance in line with valued standards. Attainments that meet or
surpass personal standards create self-satisfactions that serve as motivation for further
pursuits (Bandura & Cervone, 1983).

Attributions. Attributions are the causes to which behavior is ascribed. The
attributed causes may or may not be the actual cause of the behavior, but rather are what
the individual believes caused the behavior. Research indicates that trainee attributions
of causality can have impacts on their affective states, future expectations, and the
behaviors they engage in during training (Forsterling, 11985; Curtis, 1992). Attributions
are generally classiﬁed as one of three types: locus, stability, and controllability (Weiner,
1985). Stability refers to whether the attributed cause is Stable or unstable. For example,
a difﬁcult task may be considered a stable cause for performance, but the effort one puts
into the task is unstable and changing. Controllability refers to whether the trainee
perceives the attributed cause as being under his or her control. Luck, for example, is
likely to be perceived as an uncontrollable cause of behavior, whereas effort would likely
be perceived as a controllable cause. The locus of attributions refers to whether the cause
is perceived as being a factor within the person (internal), or a factor within the
environment (external).

In the present study, I focused on the internal/external locus of attributions.
Typically, when good performance is attributed to one’s self, motivation, learning, and
performance increase over time. When bad performance is attributed to one’s self, the

effects are negative (Kozlowski et al., 1997). Typically, in order to protect the self, bad

16

performance is more often attributed to external factors, justifying withdraw from the
task. Attributional interpretations occur naturally. Because complex tasks often involve
many trainee errors, particularly during early skill acquisition, uncontrolled attributional
processes have the potential to undermine self-regulation, learning, and performance
(Kozlowski et al., 1997). If trainees attribute their performance to external causes they
will be more likely to feel detached from the task and less likely to put forth effort to
improve their performance. Thus, it is important, especially in the beginning of training,
to control the locus of trainee’s attributions. Attributions should be internally focused in
order to give trainees a sense of control over their learning (Kozlowski et al., 1997).

On-task cognition. In most training situations there exists some sort of positive
relation between persistence and performance success, and most people believe that
greater effort will result in improved performance (Sandelands, Brockner, & Glynn,
1988). Although persistence is usually adaptive, a number of different factors, including
the nature of the task and available opportunities for success, can inﬂuence individuals’
perceptions of this persistence-performance relationship, and subsequently their decisions
to put forth effort on a task. For example, during the early stages of learning a complex
task, performance often involves many errors and the majority of feedback may be
negative. In these situations, individuals may perceive future effort to be futile or may
withdraw effort in an attempt to protect their “competence image” (Jones, 1989).

The general idea behind theories of persistence is that individuals will be more
likely to persist at a task if they believe they can achieve desired outcomes. However,
research has also uncovered numerous individual and situational determinants of task

persistence. Henry and Sniezek (1993) found that task persistence is increased when

17

individuals perceive a high degree of internal control and when rewards are contingent on
performance. Research has also found that feedback focusing on positive outcomes leads
to greater persistence than feedback focusing on negative outcomes (Roney, Higgins, &
Shah, 1995). Sandelands and colleagues (1988) found that task persistence was greater
when individuals were informed that persistence was a wise Strategy for task completion
(e. g., persistence-performance relationship is continuous) than when individuals were
told that the nature of the task was such that persistence was a less prudent strategy (e. g.,
persistence-performance relationship is discrete). Furthermore, Cervone (1989) found
that individuals’ self-efﬁcacy judgments were positively related to their task persistence.

Many of the studies in this line of research (e.g., Cervone, 1989; Kroll, 1991;
Roney et al., 1995; Sandelands et al., 1988) have measured task persistence by having
participants work on unsolvable problems, usually anagrams. Task persistence is then
measured by calculating the amount of time individuals spend on the unsolvable
problems before quitting. In many Situations, however, individuals may withdraw
mentally from a task long before they actually quit physically performing the task or the
nature of the task may prohibit such objective measures of time on task. This is
especially true in training Situations where individuals are required to Spend a certain
amount of time on a task. Under these circumstances, it is more appropriate to measure
task persistence by assessing individuals’ on and off-task cognition (e.g., Kanfer,

Ackerrnan, Murtha, Dugdale, & Nelson, 1994).

18

Effort Constructs

Time sp_ent studying. The present Study focused on two variables to represent the
effort construct. The ﬁrst was the amount of time the trainee spent studying. In
situations where learning is an important outcome, separate from performance, effort
toward learning Should be measured. For complex, difﬁcult, and novel tasks, it is
important for trainees to study task information and materials prior to opportunities to
practice or perform the task. In these Situations, many skills and strategies Simply cannot
be learned through repeated practice, rather trainees must study task materials, such as
manuals, to acquire the task knowledge necessary for developing Skills and strategies.
Thus, the amount of time learners spend studying these task materials constitutes an
indication of their effort to learn the task.

Time Spent on feedbaglg. Feedback allows trainees to measure their performance
against either some personal or set standard and/or their previous performance (Kluger &
DeNisi, 1996). In addition, feedback allows trainees to judge their performance and
progress and make appropriate decisions about future behaviors and levels of effort.
Thus, the amount of time a trainee Spends reviewing and interpreting feedback is an
indication of their effort to achieve some standard of performance. The more time a
trainee spends on feedback the more effort they are putting forth in an attempt to learn
and perform well on the task.

Relevance of Effort Constructs

Practice sequence. Effort alone is often not enough to perform well on difﬁcult

and complex tasks. Individuals must also focus their effort on relevant aspects of the

task. In the present study, the relevance of a trainee’s practice was measured by the

19

sequence in which they practiced the relevant training material. Trainees must ﬁrst learn
the fundamental skills of the task. Once these skills are developed, the trainee can then
proceed to learn the more strategic aspects of the task, which are critical to performing
well on the task. Without the fundamental Skills, a trainee will be unable to correctly
learn or utilize the Strategic Skills and will not perform well at the task. If a trainee is not
proﬁcient at performing particular operations or applying certain strategies, he or she will
not perform well at the task.

Study sequencgand content. The relevance of a trainee’s study can also be
determined by the order in which they study the relevant training material. Trainees must
Study the training material in a particular sequence to completely and appropriately learn
the Skills and Strategies of the simulation. AS with practice, trainees must ﬁrst
concentrate on learning the fundamental aspects of the task. Once these basic skills have
been learned, trainees can then progress to learning the strategic aspects of the
simulation. Both forms of knowledge are necessary to perform well at the task. The
appropriate sequence of trainees’ practice and Study is presented in Figure 2.

Outcome Constructs

Basic and strategigerformance. The outcome variable which is most salient in
training studies is training performance. Training performance can be construed as a
skill, or what the trainee does with respect to the task. The present study focused on two
types of performance, basic and strategic. Basic performance refers to trainees’ ability to
perform fundamental task features and operations, which must be learned in order for
participants to develop more advanced skills. Strategic performance refers to a trainee’s

ability to perform more complex and difficult operations, which are based on an

20

understanding of the deeper elements of the task. Both of these outcome constructs are
important measures of how performance varies as a result of manipulating guidance and
the associated changes in other process variables.

Basic and strategic task knowledge. An outcome variable often forgotten in
training Studies is learning (Kozlowski et al., 1998). In Situations involving complex and
difﬁcult tasks, learning can often be broken down into two components. The ﬁrst is basic
task knowledge. Basic task knowledge refers to the extent to which a trainee has learned
the fundamental principles of a task. The second component is strategic task knowledge.
Strategic task knowledge refers to the extent to which a trainee has learned the underlying
or deeper elements of a task. Both types of task knowledge are appropriate outcome
variables for measuring the effects of the guidance manipulation and the related changes
in process variables on learning.

Generalization performance. Researchers are beginning to realize that the
environments in which trainees apply their knowledge and Skills are dynamic and
changing rather than static and constant. As a result, research is beginning to examine
how different training strategies effect performance in not only the training situation but
also in more complex and difﬁcult Situations (Kozlowski et al., 1997). The goal is to
measure the extent to which trainees can adapt what they have learned to a variety of task
situations, often involving an increased workload and new challenges. In the present
study, trainees’ ability to adapt what they had learned was measured by their performance
on a more complex and challenging generalization trial. Performance on this

generalization trial is an indication of the degree to which trainees have learned both

21

fundamental task features and, more importantly, the Strategic and deeper elements of the

task.

22

HYPOTHESES

The hypotheses which were examined in the present study are outlined in Figure
1. The behavioral and cognitive guidance was designed to assess each trainee’s
performance, recognize deﬁciencies, and provide recommendations on what the trainee
Should be doing and thinking. Focusing on the behavioral component, this form of
guidance directs trainees on what material they should be practicing and the order in
which they should practice the material (Kozlowski et al., 1997). AS a result, trainees
receiving behavioral and cognitive guidance should be more likely to not only practice all
the relevant TAG procedures and skills but also to follow the suggested sequence of
practice (basic --> strategic). All trainees Should Spend a signiﬁcant amount of time
practicing the basic aspects of the game; however, trainees receiving behavioral and
cognitive guidance should spend signiﬁcantly more time practicing the strategic
components of the game than trainees not receiving this guidance.

Hypothesis 1: Trainees receiving behavioral and guidance will be more likely to

follow the suggested sequence of practice, basic to strategic, than trainees not

receiving this guidance.

In addition to recommending future courses of practice, behavioral and cognitive
guidance also directs trainees on what they Should be studying (Kozlowski et al., 1997).
The behavioral and cognitive guidance assesses trainees’ past performance, recognizes
areas of deﬁciency, and recommends the material trainees should be studying in order to
overcome these deﬁciencies. In addition, the guidance sequences trainees’ learning so

that they focus on the more basic aspects of the task in the beginning of training, and

23

progress to learning Strategic skills later in the task. AS a result, trainees receiving
behavioral and cognitive guidance should not only study more of the relevant TAG
procedures and strategies but also should study the material in the proper sequence. All
trainees should spend a Signiﬁcant amount of time studying the basic aspects of the game;
however, trainees receiving behavioral and cognitive guidance Should spend signiﬁcantly
more time studying the strategic components of the game than subjects not receiving this
guidance.

Hypothesis 2: Trainees receiving behavioral and cognitive guidance will display

a more appropriate sequencing of study, basic to strategic, than trainees not

receiving this guidance.

For individuals to develop feelings of self-satisfaction or self-dissatisfaction, they
need to be able to compare their past performance to some standard of performance
(Bandura & Cervone, 1983). Behavioral and cognitive guidance provides trainees with
speciﬁc information about how their performance on particular skills or strategies
compares to a set standard. In this way, the behavioral and cognitive guidance allows
trainees to compare their performance to a reference point or desired level of
performance. As trainees attempt to learn a new skill or strategy, their performance will
initially fall below the set Standard of performance. This failure to meet the Standard
Should result in increased self-dissatisfaction (Bandura & Cervone, 1983).

Hypothesis 3a: Behavioral and cognitive guidance provides trainees with an

opportunity to compare their performance to a clear set standard of performance,

resulting in increased self-dissatisfaction for trainees not achieving the standard.

24

When a person fails to achieve a personal or set standard of performance, they
experience self—dissatisfaction (Bandura & Cervone, 1983, 1986a). As long as the
standard appears attainable, self-dissatisfaction leads to an intensiﬁcation of effort to
achieve the Standard (Bandura & Cervone, 1983, 1986a). Thus, increased self-
dissatisfaction with performance should lead to increased effort on the task.

Hypothesis 3b: Higher levels of self-dissatisfaction will lead to higher levels of

eﬂ'ort on the task, such as more time spent studying and more time spent

reviewing feedback.

As discussed earlier, whether perceived discrepancies between personal standards
and attainments are motivating or discouraging is determined in part by the degree to
which people believe they can attain the standards they are pursuing (Bandura &
Cervone, 1983, 1986a). Individuals who receive behavioral and cognitive guidance are
told that they will be guided through the training process. They also receive guidance,
which, if followed, will help them succeed on the task. They are given information about
their performance deﬁciencies and are told exactly what they need to do in order to meet
the set performance standards. Trainees receiving behavioral and cognitive guidance
should feel as though they are being given an excellent opportunity to succeed at the task,
and that they can overcome performance decrements and perform well. As a result,
trainees receiving behavioral and cognitive guidance should display higher levels of self-
efﬁcacy than individuals not receiving this guidance.

Hypothesis 4: Trainees who receive behavioral and cognitive guidance will

exhibit higher levels of self-eﬂicacy than trainees not receiving this guidance.

25

An individual’s self-efﬁcacy effects his or her decisions concerning what
challenges to undertake, how much effort to expend on a task, and how long to persevere
in the face of difﬁculties (Bandura, 1982, 1986a). Individuals with low self-efficacy are
easily discouraged by failure, whereas those with high self-efﬁcacy will increase their
effort when their performance falls Short and persevere until they succeed (Bandura &
Cervone, 1983, 1986a; Brown & Inouye, 1978; Cervone & Peake, 1986; Schunk, 1984;
Weinberg, Gould, & Jackson, 1979). Thus, trainees with high self-efﬁcacy should
expend more effort on the task than trainees with lower levels of self-efﬁcacy.

Hypothesis 5: The higher a trainee ’s self-efﬁcacy, the more effort, in terms of

amount of time spent studying and reviewing feedback, he or she will expend on

the task.

Affective guidance is designed to help trainees control their reactions to their
feedback. This guidance tells trainees that the task is difﬁcult and that poor performance,
especially in the beginning of training, is to be expected. It makes trainees aware of the
emotions they may face during training, such an anxiety or anger, and provides strategies
and techniques for helping them handle these emotions. Receiving negative feedback or
being told often that past performance is below a set standard can have a negative affect
on trainees’ motivation. However, affective guidance Should act to control trainees’
negative affect and convince them that they can attain the desired level of performance.
AS a result, trainees receiving affective guidance should possess higher levels of self-
efﬁcacy than trainees not receiving this guidance.

Hypothesis 6: Trainees receiving aﬂective guidance should display higher levels

of self-eﬁicacy than trainees not receiving aﬁective guidance.

26

Sandelands and colleagues (1988) showed that individuals persisted longer on a
task when they were advised that persistence was a wise Strategy for task completion. In
the present study, affective guidance tells trainees that although they may do poorly at
times, if they continue to work hard they will do well. This guidance is telling trainees
that the relationship between persistence and performance is continuous, or that success
is the result of gradual, continuous effort devoted to the task (Sandelands et al., 1988).
Trainees who receive affective guidance should be more apt to believe that continued
effort will result in success, and thus be less likely to withdraw from the task. These
trainees should display more on-task thoughts and less off-task thoughts than trainees not
receiving the affective guidance.

Hypothesis 7a: Trainees who receive aﬁ‘ective guidance will be less likely to

withdraw from the task and will display higher levels of on-task cognition than

trainees not receiving affective guidance.

Trainees may withdraw from a task for several different reasons, including
protecting their self-concept and believing that future effort is futile (Jones, 1989).
Whatever the reason, when trainees withdraw from a task, or plan to withdraw, they
display less effort than trainees who persist at the task. Thus, there Should be a direct
relationship between the degree to which trainees have withdrawn from the task and the
amount of effort they put forth on the task.

Hypothesis 7b: Trainees with higher levels of on-task cognition will display more

effort, in terms of time spent studying and reviewing feedback, than trainees who

indicate they have withdrawn from the task.

27

Affective guidance also focuses on controlling trainees’ attributions. Poor
performance and a high rate of errors are to be expected during early skill acquisition on
complex and difﬁcult tasks. However, negative feedback can have detrimental effects on
trainee’s motivation, especially if they perceive their failure as resulting solely from
external causes. During the early stages of training the affective guidance is designed to
inform the trainees that the task is difﬁcult and poor performance is to be expected.
However, it also provides trainees with techniques and strategies to help them handle
their reactions to feedback, thus creating an internal locus of control. Trainees receiving
affective guidance, therefore, should display a more internal locus of attributions
throughout the course of training.

Hypothesis 8a: Trainees receiving affective guidance should display a more

internal locus of attributions than other trainees.

Uncontrolled attributions have the potential to undermine trainee motivation, self-
regulation, performance, and learning (Kozlowski et al., 1997). It iS important to
encourage trainees to focus their attributions on internal causes so that they develop a
sense of control over their learning (Kozlowski et al., 1997). This internal focus will help
trainees to persist in the face of difﬁculties and will help to boost motivation and effort
(Kozlowski et al., 1997).

Hypothesis 8b: An internal locus of attributions during training will result in

increased trainee effort, in terms of amount of time spent studying and reviewing

feedback.

An individual’s self-efﬁcacy judgements should inﬂuence his or her task

persistence. Self-efﬁcacy theory proposes that when individuals are faced with obstacles

28

or difﬁculties, efﬁcacy judgements inﬂuence the persistence of effort on an activity
(Bandura 1986a; 1988). As Sherman and colleagues (1981, p. 145) stated, “expectations
of success or failure are likely to affect the amount of effort and concentration exerted.”
In an experiment designed to test this proposition, Cervone (1989) found that individuals’
perceived self-efﬁcacy was positively related to task persistence. Thus, in this study, we
would expect a positive correlation between trainees’ self-efﬁcacy judgments and their
task persistence.

Hypothesis 9a: Higher levels of self-efficacy will result in greater on-task

cognition and less off-task cognition.

Trainee attributions should also inﬂuence task persistence. Henry and Sniezek
(1993) found that task persistence was increased when people perceived a higher degree
of internal control. It makes sense that if trainees feel a sense of control over their
learning, they will be more likely to expend effort on the task and less likely to withdraw
effort when faced with poor performance or difﬁculties. Trainees who develop internal
attributions Should feel increased control over their learning and display higher levels of
task persistence.

Hypothesis 9b: Internal attributions throughout training will result in greater

on-task cognition and less off-task cognition.

The effort trainees expend on the task, in terms of time spent studying and
reviewing feedback, and the relevance of this effort, in terms of the sequence and content
of practice and study, will effect several dependent variables. The ﬁrst is basic task
knowledge. Trainees who Spend considerable time studying task materials and reviewing

feedback Should learn more about the basic aspects of the task than individuals who do

29

not spend as much time performing these activities. In addition, trainees who study and
practice the relevant task Skills and strategies, and do so in the appropriate sequence,
should display greater basic task knowledge than other trainees. It iS important to note
that basic task knowledge will be acquired by all trainees through repeated practice and
exposure to the task; therefore, overall differences expected between groups on this
variable are minimal. However, individuals who follow the appropriate plan of study and
practice will probably acquire this knowledge quicker than other trainees. Thus, the
greatest differences on basic task knowledge Should be found early in training.

Hypothesis 10a: The effort trainees expend on the task and the relevance of this

effort will be signiﬁcantly and positively correlated to basic task knowledge.

The effort trainees put forth on the task and the relevance of this effort will also
affect strategic task knowledge. Trainees who Spend more time studying task materials
and reviewing practice feedback will develop a better understanding of the strategic
aspects of the game than those trainees who do not Spend as much time at these activities.
More importantly, the sequence and content of trainees’ study and practice will have a
large effect on their Strategic task knowledge. Trainees who follow the desired plan of
study and practice Should be more likely than other trainees to Spend time learning the
deeper elements of the game. The greatest differences in strategic knowledge Should be
found during the later stages of training when trainees have had an opportunity to learn
the strategic elements of the game and when the guidance directs trainees’ attention to

these aspects of the task.

30

Hypothesis 10b: Trainee effort on the task and the relevance of this effort to
important aspects of the task will be signiﬁcantly and positively correlated with
strategic task knowledge.

All trainees will achieve a minimum level of basic performance through repeated
exposure to and practice on the task. However, trainees who spend more time studying
task materials and reviewing practice feedback will perform better at the basic aspects of
the task than trainees who Spend less time performing these activities. In addition,
trainees who study and practice relevant task content and do so in the desired sequence
will display higher levels of basic performance than other trainees. Trainees who put
forth more effort and focus this effort on relevant aspects of the task should be able to
perform well at the basic components of the task quicker than other trainees. Thus, the
largest differences on this variable Should be found during the early stages of training.

Hypothesis 10c: Basic performance will be signiﬁcantly and positively correlated

to the amount of effort trainees expend on the task and the relevance of this effort.

Strategic performance is an important aspect of complex and difﬁcult training
situations. In the present study, a trainee’s strategic performance will be affected by both
the effort he or She expends on the task and the relevance of this effort to important
aspects of the task. Trainees who spend more time studying task materials and reviewing
practice feedback should be better equipped with the information necessary to perform
well on the strategic aspects of the task. In addition, studying and practicing appropriate
task content and doing so in the proper sequence will enable trainees to develop the

strategic skills necessary to perform well at the task. The greatest differences on strategic

31

performance Should be found in the later stages of training when trainees have had an
opportunity to develop these Skills.

Hypothesis 10d: Strategic performance will be signiﬁcantly and positively

correlated with the amount of effort trainees expend on the task and the relevance

of this eﬁfort to important aspects of the task.

Trainees who perform well on the basic and strategic aspects of the task and
display high levels of basic and strategic knowledge Should perform well on the
generalization trial. Although each of these variables will be positively correlated to
performance on the generalization trial, the Size of their respective effects will not be
equal. In the generalization trial, the workload is increased dramatically and the rules of
the game are modified. To handle this complex and difﬁcult Situation, trainees must be
able to effectively and efficiently apply the strategic knowledge and Skills they have
learned during practice. Basic Skills and knowledge are Still critical in the generalization
trial, but they alone will not allow the trainee to perform well. As a result, trainees’
performance on and knowledge of the strategic aspects of the task will have the greatest
impact on performance in the generalization trial.

Hypothesis 1]: Performance on the generalization trial will be signiﬁcantly and

positively correlated with trainees’ basic and strategic performance on the task

during practice and their basic and strategic knowledge.

32

METHOD

DiSigﬂ

Overview. The present Study employed a 2 (behavioral\cognitive guidance vs. no
behavioral\cognitive guidance) by 2 (affective guidance vs. no affective guidance) fully
crossed between-subjects design. The experiment took place in a single session
consisting of three blocks, each block consisting of three trials, for a total of nine training
trials. Blocks (3) and trials (9) both represent within subjects factors. Following the
training blocks, trainees participated in a generalization trial in which the workload (e. g.,
more targets, length of the trial was doubled) and complexity (e. g., the principles and
rules underlying task proﬁciency were modiﬁed) were increased. The purpose of the
generalization trial was to measure the extent to which trainees are able to adapt their
acquired skills.

Simulation. The PC-based decision making simulation TEAMS/TANDEM (see
Kozlowski & Gully, 1996; Weaver, Bowers, Salas, & Cannon-Bowers, 1995; Weaver,
Morgan, Hall, & Compton, 1993) was used as the experimental platform. This platform
is a low physical fidelity Simulation of a naval radar tracking task which provides a high
psychological ﬁdelity for complex and difﬁcult decision making and information
processing tasks. The task was originally developed by the Naval Air Warfare Center
Training Systems Division (NAWCTSD) and the University of Central Florida. The
original task has undergone extensive design modiﬁcations to create the current version
of TEAMS/TANDEM (version 8.10, which provides a dynamic, self-contained, and

completely novel task environment. The current version of TEAMS/TANDEM allows

33

researchers to examine the process of training individuals on a complex task and
developing adaptive expertise. The Simulation was designed to include events that unfold
in real time that can be scripted by the researcher. In addition, the researcher has control
over what performance information trainees are given, what decisions they need to make,
and how the performance of these actions are scored.

The current version of TEAMS/TANDEM, which was used in this study, is
referred to as the Tactical Action Game (TAG). During TAG, each participant is seated
at a Simulated radar console upon which multiple contacts are presented. The trainee
must gather information about each contact by “hooking” the target, gathering
information about its characteristic “cue values,” and making a ﬁnal decision about the
target’s disposition based on information gathered. Using this information, the trainee
then makes a ﬁnal decision about what action to take against the target (shoot or clear).
Three cues are available for each of the component decisions (nine cues overall), which
must be made in the following order: (1) Type (air, surface, or submarine); (2) Class
(civilian or military); (3) Intent (peaceful or hostile); and (4) Engage (Shoot or clear).

Skill components. Throughout training, all participants had access to a
comprehensive training manual. During a two minute period before each trial, the
manual was presented on-line so as to provide the capability of tracking the content and
duration of studied material. Three sets of skills were documented in the comprehensive
manual. These sets included basic functionality, declarative knowledge, and strategic
knowledge.

Basic functionality on the task involves learning those features of the program, in

terms of both hardware and software, which make it possible to perform the task. Task

34

features which must be learned in order for a participant to perform well on TAG include
hooking targets, accessing cue menus, and “zooming” to alter the display radius. In
addition, individuals need to understand several basic functions of a PC, such as using a
mouse and keyboard.

Declarative knowledge is also required to process information and render target
sub-decisions. The cue information is accessed from pull-down menus, with 3 cues
available for each of the three component decisions. After making the three component
decisions regarding the Type (air, surface, submarine), Class (civilian or military), and
Intent (peaceful or hostile) of a target, participants are required to either shoot targets (if
hostile) or clear them from the radar screen (if peaceful). For individuals to successfully
progress to learning the Strategic aspects of the game, it is important that they ﬁrst
become proficient at making correct and efﬁcient component and ﬁnal engagement
decisions. These skills, which are a foundation for later strategic skills, are measured by
indicators of basic performance, such as number of correct sub-decisions and ﬁnal
engagement decisions.

Strategic knowledge involves the participant’s ability to understand the deeper
elements of the simulation and to develop two critical strategic Skills: Situational
assessment and target prioritization. Three elements of the task (using the zoom function,
identifying defensive perimeters, and locating and utilizing marker targets) are relevant to
situational assessment. The zoom function allows participants to either expand (zoom
out) or constrict (zoom in) the radius of their radar screen. Participants need to “zoom
out” to determine the overall Situation in which they are operating. This action is critical

because of the presence of two “defensive perimeters” within the task. The inner

35

defensive perimeter, located at lONM, is clearly marked and easy for participants to
identify. However, the outer perimeter, located at 256 NM, is beyond the initial viewing
range of 32 NM, and is not clearly marked. Thus, participants must ﬁrst “zoom out” and
then locate “marker targets” that serve to identify the outer boundary.

Target prioritization requires the trainee to utilize a separate set of strategic skills.
Decisions about which targets constitute the greatest threats to the participant’s defensive
perimeters are critical, as the scenarios are designed so there is often multiple targets
approaching each perimeter, some of which are more threatening than others. The order
in which targets Should be prosecuted should be based on two pieces of information,
Speed and distance from the perimeter. Faster targets are of higher priority than Slower
targets because they will penetrate the defensive perimeters more quickly. Also, targets
close to either of the defensive perimeters are of higher priority than those which are
farther away. Thus, the highest priority targets are those that are both (a) moving quickly
and (b) near a defensive perimeter. Trainees also need to “trade off” targets approaching
their inner and outer perimeters. Strategic decisions related to such trade-offs revolve
around the number of targets at each perimeter, their priority, and their “cost” if they
penetrate.

Procedure

Participants. A total of 292 undergraduate college students from a large
Midwestern university participated in the present Study. A total of 15 participants were
dropped from the sample for various reasons, resulting in a ﬁnal sample of 277
participants.1 All participants were volunteers who received course credit for their

participation in the three hour experiment. Fifty-six percent of the participants were

36

female, and most (86.3 percent) of the participants were between 18 and 21 years old.
The experimental sessions were conducted with groups consisting of from one to 12
participants. These groups were randomly assigned to one of the four experimental
conditions, resulting in a relatively even distribution of participants across the four
experimental conditions. Using Cohen’s (1988) sample size tables for analysis of
covariance (ANCOVA), it was found that a sample size of 277 participants
(approximately 70 participants per cell) resulted in approximately 25 percent power to
detect small (f = .10) effect sizes, 95 percent power to detect medium (f = .25) effect
sizes, and over 99 percent power to detect large (f = .40) effect Sizes.

Informed consent. Before beginning the experiment, all subjects were required to
read, Sign, and date a consent form which provided a general description of the
experiment and its risks and beneﬁts. After completing the study, all participants were
debriefed. The consent form is presented in Appendix A, and the debrieﬁng sheet is
shown in Appendix B.

Experimental training. After reading a Short introduction to the experiment, the
experimenter presented participants with a brief demonstration of the task, outlining its
features and decision sequence. During this demonstration, the experimenter brieﬂy
displayed how to hook targets, use the pull down menus, use the zoom function, and
make decisions in the proper order. The experimenter also demonstrated how to use the
on-line manual and how to scroll through the feedback and guidance. Participants were
informed that they would have an opportunity to practice the TAG game in a one-minute
“familiarization” trial. The purpose of this familiarization trial was to provide a basis for

presenting the ﬁrst block of guidance. Trainees were told that they would progress

37

through three blocks of study, practice, feedback, and guidance (if appropriate) cycles,
followed by an opportunity to demonstrate how much they had learned on a more
difﬁcult and complex version of the task.

Practice. The experimenter ﬁrst instructed the participants to begin the
“familiarization trial.” Participants were told that the purpose of this initial trial was to
allow them to get familiar with the task and to obtain a baseline measure of their
performance, which was used to provide the ﬁrst presentation of guidance. Following
this initial trial, participants in the behavioral\cognitive guidance condition and
participants in the affective and behavioral\cognitive guidance condition were given the
first presentation of behavioral\cognitive guidance, which was the same for all
participants. This guidance instructed the participants on what topics they Should focus
on during the upcoming block of practice trials. In addition, all participants were given a
list of TAG topics and told that they may want to focus on these areas during their
training. For participants in the behavioral and cognitive guidance conditions, the list of
TAG tOpics was presented in the appropriate sequence (basic to strategic). This list is
presented in Appendix C. Participants in the other two conditions, however, received a
random list of the same TAG topics. This list is presented in Appendix D. After
reviewing the guidance, participants then began the regular practice cycle, consisting of a
two minute study period, a ﬁve minute practice trial, and a two minute feedback and
guidance session. Trainees were asked to complete the Wonderlic Personnel Test and a
short demographic questionnaire before beginning the experiment (see Appendix E). In
addition, other measures, including dependent variables, were collected at the end of

block 1 and the end of block 3.

38

Generalization. The generalization task was designed to substantially increase
both the workload and complexity of the task. Effective performance on the
generalization trial required that trainees adapt the knowledge and skills they had
acquired throughout training to a new and more complex Situation. Following the nine
trials, trainees received instructions describing important task differences they would
encounter in the ﬁnal session.

To increase the workload, the number of targets was increased from 22 to 60, a
172% increase. Task complexity was increased by (1) including more “pop-up” targets,
which appeared suddenly on the screen; (2) changing the “rules of engagement” so that a
greater number of points were deducted when targets crossed the visible inner perimeter
( 175 points) and the invisible outer defensive perimeter (125 points); (3) creating more
defensive perimeter intrusions; (4) creating “pop-up” targets that appeared very close to
defensive perimeters; and (5) differentially distributing boundary intrusions to create a
situation in which many targets threatened the outer perimeter, while fewer targets
threatened the inner perimeter, requiring strategic trade-offs on the part of trainees. The
generalization task represented a much more demanding, difﬁcult, and complex task than
that faced in training, requiring the adaptation of trainee’s Strategies and the
generalization of their Skills.

Incentives. Trainees were informed about awards for (a) questions answered
during the training trials (knowledge) and (b) performance on the ﬁnal, more difﬁcult
generalization trial. The instructions at the beginning of training indicated that trainees
would be asked to answer questions throughout the training program, emphasizing that

they needed to “...think carefully about your answers to the questions.” They were told

39

that awards would be given to the four players who answered these questions the best,
and that four additional awards would be given to the players who did the best on the
ﬁnal (generalization) trial. Written instructions regarding incentives were provided to
trainees at the beginning of the experiment, and they were reminded by the experimenter
just prior to the start of the generalization trial. These instructions are presented in
Appendix F.

Manipulations

Behavioral\cognitive guidance. Due to the difﬁculty in distinguishing between
behavioral and cognitive guidance, the two manipulations were combined throughout the
experiment. The initial presentation of behavioral and cognitive guidance followed the
“familiarization” trial and was the same for all participants. This initial behavioral and
cognitive guidance instructed trainees that they should learn how to hook targets and
make correct Type, Class, Intent, and Final Engagement decisions, and that they should
Study their manual to help them learn these skills. Following this initial presentation, all
subsequent behavioral and cognitive guidance was adaptive and was based on the
trainee’s performance on the preceding practice trial.

Adaptive behavioral and cognitive guidance was based on three levels of
performance. The ﬁrst level was represented by a failure to reach a minimum level of
performance on a particular skill or Strategy. This is basically an inability to perform the
required Skill or strategy. The second level of performance was represented by the ability
to achieve minimum performance on a Skill or strategy, but a failure to achieve the
desired level of performance. This was indicative of trainees who showed they could

perform a certain action, but were not yet proﬁcient. The third level of performance was

40

representative of those individuals who had achieved the desired level of performance on
a skill or strategy. To create cutoff scores for these three levels of performance, prior
experimental data employing the TAG simulation was analyzed.

Based on that data, cutoff scores for each skill and strategy were set at the 50th
and 85th percentiles. It was determined that these cutoffs allowed for adequate
discrimination between scores representing low, medium, and high performance. In
addition, these scores make minimum performance quite easily achievable, and
maximum performance difﬁcult to achieve. The behavioral and cognitive guidance for
individuals scoring within each of the three ranges of performance on a particular skill or
strategy was designed as follows:

Individuals scorirg below the 50th percentile: The behavioral and cognitive

guidance for individuals scoring within this range informed the person that they

had not yet learned how to perform the necessary skill or strategy and instructed
the individual on what they should be practicing and studying to achieve the
desired level of performance.

Individuals scoring between the 50th and 85th parcentile: The behavioral and

cognitive guidance for individuals scoring within this range informed the

individual that they had reached a level of minimal performance, but needed to
become more proﬁcient at the Skill or strategy. In addition, the guidance
instructed the trainee on what he or She Should be practicing and studying to
achieve the desired level of performance.

Individuals scoring above the 85th percentile: The behavioral and cognitive

guidance for individuals falling within this range informed the person that they

41

had mastered the speciﬁc skill or Strategy, and that they should concentrate on

improving in areas in which they were still deﬁcient.

The behavioral aspect of the guidance directed trainees on what skills and
strategies they should be practicing based on their previous performance. For example, if
a person hadn’t learned how to hook targets, the behavioral guidance suggested to the
person that this is an area in which he or she needs additional practice. The cognitive
aspect of guidance directed trainees on what material they Should be studying based on
their previous performance. For example, if a trainee was allowing too many outer
penalty circle intrusions, the cognitive guidance suggested that the trainee should review
the material in the manual on “zooming out” and identifying the outer penalty circle. The
behavioral and cognitive guidance were designed to complement one another, and were
framed in a manner which recommends or suggests certain actions or behaviors, rather
than telling participants what to do. See Appendix G for more examples of behavioral
and cognitive guidance.

In addition to helping trainees identify areas of needed improvement, the
behavioral and cognitive guidance was also designed to sequence the learning and
practice of trainees. This sequencing is important because it allows trainees to develop
the fundamental Skills of TAG before proceeding to the more strategic aspects of the
simulation, where the fundamental Skills will be required. Before each training block, the
behavioral and cognitive guidance recommended general areas they Should be covering
during the upcoming practice and study sessions. Then, following each practice session,
the behavioral and cognitive guidance gave trainees Speciﬁc information about what they

needed to practice and Study to improve their performance in these general areas.

42

Affective gaidance. Affective guidance directs the trainee to exercise emotional
control with respect to his or her reactions to feedback (Kozlowski et al., 1997). The
main goal of the affective guidance was to maintain trainees’ effort and lessen the
chances that they would withdraw from the task. The affective guidance was designed to
provide trainees with techniques to help them handle the emotions, thoughts, and feelings
they may encounter during their training. For example, trainees who become extremely
frustrated or upset with how they are performing at the task are more likely to withdraw
from training. Thus, the affective guidance might provide techniques that help the
trainees to relax and control their frustrations.

The ﬁrst step in designing the affective guidance was to determine the affective
states that trainees were most likely to experience at different points during the three-hour
training session. To determine the emotional states of trainees while playing the TAG
Simulation, thirty individuals participated in pilot sessions of the experiment. These
sessions were run using the no guidance or control version of the simulation. To allow
the experimenter to interact and question the participants, the sessions were run with
groups of three to four participants. Following each block of three practice trials, the
experimenter questioned the participants about the emotions and thoughts they
experienced during the preceding practice block. These discussions with participants
yielded a relatively consistent pattern of emotions experienced by trainees across the
training blocks, which I present in the following section.

During the ﬁrst block of practice trials, trainees typically became discouraged,
frustrated, anxious, or upset due to the fact that they were not performing as well as they

would like. This is not surprising considering the fact that it takes some time to become

43

proﬁcient at the TAG simulation. The ﬁrst few practice trials were characterized by a
great deal of trainee error and typically poor performance. During the second block of
practice trials, however, trainees began to see some improvement in their performance.
At this point in the training, most individuals had become fairly proﬁcient at the basic
TAG Skills. This resulted in slightly lower levels of frustration and anxiety during the
beginning of the second practice block. However, trainees who began to work towards
learning some of the more difﬁcult, strategic TAG skills once again experienced
frustration and anxiety due to an initial period of poor performance. Thus, it was
necessary to determine methods of reducing trainee frustration and anxiety during the
ﬁrst and second practice blocks. By the beginning of the third trial block, most trainees
had become fairly proﬁcient at the TAG simulation. They had mastered the basic skills
and were improving on the strategic skills. This increased performance on the task
combined with over two hours of steady study and practice made motivation a major
issue in the ﬁnal trial. Some trainees began to withdraw from the task due to fatigue,
while others became less motivated due to what they perceived as high levels of
performance. As a result, it was necessary to devise methods for keeping trainees
engaged on the task to the end of training.

After determining the affective states experienced by individuals during the TAG
training, it was then necessary to devise techniques to help trainees handle these
emotions. In the ﬁrst and second practice blocks, the affective guidance provided
trainees with several skills or techniques they could use to reduce their anxiety and
frustrations and maintain their focus on the task. One such technique was guided mental

imagery. A guided mental imagery script can involve several components, including

relaxation, thought stopping, and covert reinforcement scenes (Wheatley, Maddox, and
Anthony, 1989). Guided mental imagery can help an individual change the way he or
She communicates intrapersonally, and in doing so can help to reduce feelings of anxiety
and frustration and can help to eliminate self-defeating thoughts (e.g., Carter & Kelly,
1997; Mace & Carroll, 1985; Murphy & Woolfolk, 1987; Sapp, 1994). To replace
negative thoughts with more positive thoughts (a combination of thought stopping and
reinforcement), trainees who became frustrated were instructed to imagine themselves
performing extremely well at the TAG simulation. Another technique used to reduce
trainee frustration and anxiety was relaxation therapy. Some relaxation therapy
techniques are fairly involved, such as Bernstein and Borkovec’s (1973) progressive
muscle relaxation in which individuals tense and relax 16 different muscle groups during
a 90 minute session. Due to time constraints, more simple relaxation techniques were
used in the present study, such as breathing. It has been suggested that controlled
breathing is an important part of nearly all relaxation techniques (e. g., Foreyt, 1990;
Lichtenstein, 1988). A ﬁnal technique used to help trainees handle their emotions in the
ﬁrst two training blocks was functional self—talk. A major premise of Ellis’ (1962)
rational-emotive theory is that irrational self-statements produce psychological distress
and rational self-statements decrease it. Consistent with this theory, research has shown
that reciting rational statements, such as “If I am not performing as well I would like it is
not terrible,” or positive statements can help to reduce individuals’ anxiety (Crarner &
Kupshik, 1993; Rosin & Nelson, 1983), increase their self-efﬁcacy (Milliman & Latham,

1996), and improve their performance (Van Raalte et al., 1995).

45

In the ﬁnal block of practice trials, the main goal of the affective guidance was to
provide trainees with techniques that would help them maintain motivation and stay
focused on the task. This was done by providing trainees with, for lack of a better term,
“psyching-up” techniques. Research in sports psychology has shown that “psyching-up”
techniques can be effective for increasing the energizing capacity of the body and
focusing an athlete’s attention (Caudill, Weinberg, & Jackson, 1983; Genov, 1970;
Weinberg, Gould, & Jackson, 1980). Research has also shown that there are several
different effective “psyching strategies” (Shelton & Mahoney, 1978; Weinberg, Gould, &
Jackson, 1980). In the present study, the affective guidance provided individuals with
several psyching strategies, such as imagery, control of attention, and self-statements, in
an attempt to maintain trainee motivation.

The affective guidance was not adaptive; however, it was designed so that each
presentation was unique and novel. The affective guidance was presented after the
practice feedback, immediately following the behavioral and cognitive guidance (if
applicable). The affective guidance presented to participants in each trial is presented in
Appendix H.

In the affective guidance only conditions, trainees were given a random list of
objectives or topics that they should learn throughout the training session. This list
contained the same topics given to individuals receiving behavioral\cognitive guidance;
however, the information was not sequenced as it was for those trainees in the

behavioral\cognitive guidance conditions.

46

Measures

Control variables. Previous studies conducted using the TEAMS/TANDEM
Simulation have Shown cognitive ability to be a Signiﬁcant predictor of performance
(Kozlowski et al., 1995, 1996). As a result, all participants were administered the
Wonderlic Personnel Test at the beginning of the experiment. The Wonderlic Personnel
Test is a well-known and widely used index of cognitive ability. The user’s manual for
the Wonderlic ( 1992) offers predictive validities as high as .63, with reliability estimates
from .73 to .95, depending on the type of reliability estimated.

Veridical feedback on the task was provided immediately following the
completion of each training trial. Although the speciﬁc nature of the feedback was based
on each trainee’s performance, all trainees received the same type of information. The
feedback provided information relevant to the full range of training objectives. One piece
of information given in the feedback was a score. Task score was computed by adding
100 points to the trainee’s score every time a target was identiﬁed and prosecuted
correctly, and subtracting 100 points from the trainee’s score every time a target was
misidentified or prosecuted incorrectly. In addition, 10 points were deducted from the
trainee’s score for each target that crossed a defensive perimeter. In addition to score, the
feedback provided information concerning the trainee’s performance on both basic and
strategic TAG skills, such as making correct cue decisions and “hooking” marker targets.
An example of the feedback is presented in Appendix I.

The no guidance condition acted as the control condition in this experiment.
Individuals in this condition received the same feedback as other trainees, but did not

receive any of the guidance information. They did, however, receive the random list of

47

TAG topics. The control condition served as a learner control condition. As discussed
earlier, individuals can be given control over several different aspects of the learning
process, including pace, sequence, and content. In the present study, all individuals were
given control over what they wanted to study and practice (content) and the order in
which they wanted to study and practice the material (sequence). Trainees were also
given some control over the pace of their learning, such as being able to exit the
computerized manual early if they chose; however, for practical reasons it was necessary
to impose maximum time limits on the study and practice sessions. All trainees had
access to the same training materials (e. g., training manual) and all had the same control
over the sequence, content, and pace of the learning process. However, because the
experimental conditions contained one or both forms of advisement or guidance, only the
control condition was a “pure” learner control situation.

Basic and strategic task knowledge. Basic task knowledge was assessed
following the completion of the first and third practice blocks (after trial 3 and trial 9).
Thirteen multiple-choice items focusing on the extent to which declarative knowledge
about the task had been acquired were utilized, focusing on cue values and basic
operating features of the task. This measure is presented in Appendix E. Strategic task
knowledge was also assessed following the completion of the ﬁrst and third training
blocks. Fourteen multiple-choice items focusing on the extent to which strategic
knowledge about the task had been acquired were utilized. This measure focused on
aspects of the game such as locating the outer defensive perimeter and prioritizing

targets. The measure of strategic task knowledge is presented in Appendix E.

48

To ensure that the basic and strategic knowledge scales were measuring different
components of knowledge, I performed a conﬁrmatory factor analysis (CFA) to assess
the factor structure of the two scales. Because the knowledge items were binary in
nature, I created continuously scaled indicators by summing two or three knowledge
variables. This resulted in six or seven indicators for each of the two knowledge scales.
At each time period, I then assessed the ﬁt of the data to the path model in which each
component of knowledge was linked to its respective indicators. This path model is
presented in Figure 3. The path model was tested using AMOS. The ﬁt indices at Time
1 were as follows: x2(64, N = 277) = 69.40, p = .30; GFI = .96; AGFI = .95; CFI = .98;
and RMSEA = .02. The ﬁt indices at Time 2 were as follows: x2(64, N = 277) = 94.25, p
< .01; GFI = .95; AGFI = .93; CFI = .94; and RMSEA = .04. The results at both time
periods indicated an excellent ﬁt of data to the hypothesized model, conﬁrming the a
priori factor structures of the two scales.

Self-efﬁcacy. Following the ﬁrst and third training blocks, self-efﬁcacy was
assessed using an 8-item self-report measure developed for use in this research paradigm
(see Kozlowski, Gully, Smith, Brown, Mullins, & Williams, 1996). This measure
assesses self-efﬁcacy with a Likert-type scale rather than with ratings of conﬁdence about
particular aspects of the task (Hysong & Quinones, 1997; Lee & Bobko, 1994). The
construct of self-efﬁcacy is operationalized as a “task-focused, self-perception with item
content Speciﬁcally focused on the capability to cope and to develop methods to
effectively deal with the information, decisions, and challenges of the simulation”
(Kozlowski et al., 1996, p. 18). This is consistent with conceptualizations of efﬁcacy as

central to self-regulatory processes (Gist & Mitchell, 1992). Response options for this

49

scale range from “strongly disagree” (1) to “strongly agree” (5). Coefﬁcient alpha for
this scale was .91 at Time 1 and .94 at Time 2. The self-efﬁcacy measure is presented in
Appendix E.

Self-satisfaction. Following the first and third trial blocks, trainees completed a
5-item measure of self-satisfaction. This scale is similar to one used by Podsakoff &
Fahr (1989), but was modiﬁed to measure satisfaction rather than dissatisfaction. Likert-
type response options have also been reduced from 13 to 5, ranging from “strongly
disagree” (1) to “strongly agree” (5). Coefﬁcient alpha for this scale was .89 at Time 1
and .86 at Time 2. The self-satisfaction scale is presented in Appendix E.

On-task cognition. Following the ﬁrst and third trial blocks, trainees completed
an 8-item measure designed to assess their on-task cognition. This measure was adapted
from Kanfer and colleagues ( 1994) and is designed to measure the frequency of on and
off-task thoughts. Response options are on a ﬁve-point Likert-type scale ranging from
“never” (1) to constantly (5). Higher scores on this measure indicate more on-task
cognition and greater persistence on the task. Coefﬁcient alpha for the on-task cognition
scale was .75 at Time 1 and .81 at Time 2. The on-task cognition measure is presented in
Appendix E.

Attributions. Following the ﬁrst and third trial blocks, trainees completed a six
item questionnaire designed to measure the causes to which they attribute their
performance. One item was dropped from the scale to increase its internal consistency,
resulting in a ﬁnal scale of ﬁve items. The attribution items were restricted to internal

versus external locus of control. Response options are on a ﬁve-point Likert-type scale

50

ranging from “strongly disagree” (1) to “strongly agree” (5). Coefﬁcient alpha for this
scale was .57 at Time 1 and .67 at Time 2. This measure is presented in Appendix E.

Mice Sequence. The sequence of trainees’ practice was measured by having
trainees complete a self-report measure after each trial block which asked them to record
the three main concepts or skills they practiced during the preceding three trials. The
self-report measures were scored by determining how many of the relevant TAG skills
the trainee practiced during the relevant trial block.2 A trainee could receive a score
ranging from zero to three on each questionnaire depending on how many of the
appropriate TAG Skills (presented in Figure 2) the trainee indicated practicing during
each training block. This measure is presented in Appendix J.

Study sequence. The data collected by the computerized TAG manual provided
information concerning what each participant studied, and when during the training they
studied the material. For each training block, the amount of time a trainee Spent studying
the relevant pages of the manual was calculated and used to determine the degree to
which the trainee was following the appropriate sequence of Study (presented in Figure
2).

Study time. The amount of time each participant spent Studying the TAG material
during the study sessions was measured using the computerized TAG manual. Trainees
had a maximum of two minutes before each practice trial during which they could review
the computerized manual. The total amount of time Spent studying the manual during
each training block was calculated and used as a measure of study time. Although
trainees were instructed to exit the manual if they ﬁnished studying early, some trainees

would instead leave the table of contents page on the screen and wait for the manual to

51

exit automatically after two minutes. AS a result, time Spent on the table of contents page
was not included in the total time Spent studying. Only time spent studying pages of the
manual that contained TAG content was included in total time spent studying.

Time spent on feedbac_k. The FASTBACK feedback program measured the
amount of time trainees spent on feedback. As trainee’s scrolled through the feedback,
FASTBACK recorded the amount of time (in seconds) that the trainee spent on each
screen. The time spent on each screen of feedback was summed within each training
block to determine the total time spent on feedback.

Basic and strategic performance. Within the task itself, data were collected that
allowed assessments to be made of trainees’ basic performance and also the extent to
which strategies were being utilized in the domains of Situational assessment and target
prioritization. Indicators of a trainee’s basic performance were the number of points a
trainee gained for correct decisions and the number of points the trainee lost for incorrect
decisions. These were the two main components of a trainee’s score. Score during the
training trials was calculated by adding 100 points for each target correctly identified and
prosecuted, and subtracting 100 points for each target misidentiﬁed or prosecuted
incorrectly. Ten points were also subtracted for each target that crossed a defensive
perimeter over the course of the training. In terms of strategic performance, the number
of times participants zoomed out and the number of marker targets hooked served as
indicators of situational assessment. Target prioritization was assessed by the number of
high priority targets (those targets that are both moving fast and near a perimeter)
prosecuted. Time one indicators are based on performance in the third training trial, and

time two indicators are based on performance in the ﬁnal or ninth training trial.

52

To examine whether separate basic and strategic composite variables could be
created by combining the relevant performance indicators, I conducted an exploratory
factor analysis. This factor analysis was conducted on the indicators as measured after
the ﬁnal training trial to allow time for the factor structure to stabilize. Principal
components analysis using varimax rotation was utilized. Following the Kaiser
normalization criterion guideline of selecting components with eigenvalues greater than
one, two components were rotated. The eigenvalues for the ﬁrst and second component
were 2.10 and 1.39, respectively. The ﬁrst component accounted for 42.00% of the
variance and the second component accounted for 27.73% of the variance. Thus, the two
components collectively explained nearly 70% of the variance. The ﬁrst component
consisted of the strategic performance indicators and the second component consisted of
the basic performance indicators. Loadings of the variables on their respective
components were all greater than 1.70, and cross-loadings were all less than 1.20. Based
on these results, it was deemed appropriate to create basic and strategic performance
composite variables. To do so, the performance measures were standardized within
training block and then summed using unit weights. The basic performance composite
was created by combining points correct and points incorrect. The strategic performance
composite was created by combining the total number of times the trainee zoomed out,
the number of marker targets hooked, and the number of high priority targets prosecuted.
All indicators were given a positive weight except for points incorrect which was given a

negative weight Similar to how it is used to calculate the overall score value.

53

RESULTS

The means, standard deviations, and intercorrelations among all variables
included in the analyses described below are provided in Table 1. Below I ﬁrst present
the results of a RM-MANCOVA that was used to test the overall and direct effects of the
two manipulations. I then provide the results of regression analyses used to test the
hypotheses outlined earlier and discuss the results of additional analyses that were
performed to test the model in more detail. The results of the regression analyses are
organized by the different subsets of variables outlined in Figure 1. Appendix K presents
a summary of the tests of the hypotheses.

RM-MANCOVA.

A repeated measures MANCOVA was performed to examine the overall and
direct effects that the behavioral\cognitive guidance and affective guidance had on the
entire set of dependent measures over time, excluding the generalization outcomes.
Cognitive ability, as measured by the Wonderlic Personnel test, was treated as a covariate
in these and all subsequent analyses. Using Wilk’s Lambda, the MANCOVA
demonstrated a signiﬁcant overall effect for cognitive ability, 13(12, 261) = 9.25, p< .001,

n2 = .30. These results suggest that the use of cognitive ability as a covariate is
appropriate. The MAN COVA also demonstrated a Si gniﬁcant overall effect for affective
guidance, 13(12, 261) = 2.33, p_< .01, n2 = .10, and behavioral\cognitive guidance, E(12,
261) = 18.98, p_< .001, n2 = .47. In addition, behavioral\cognitive guidance was found to
have a Signiﬁcant effect over time, E(2, 261) = 8.93, p_< .001 , n2 = .29. Signiﬁcant

overall effects suggest that regression analyses exploring each of the proposed

54

hypotheses are appropriate. The repeated measures MANCOVA described here will also
be used to test the direct effects of the manipulations on the different sets of dependent
variables. The results of the overall RM-MANCOVA are presented in Table 2. The
between-subjects tests from the RM-MANCOVA for affective guidance are presented in
Table 3, and the between-subjects tests from the RM-MANCOVA for behavioral and
cognitive guidance are presented in Table 4.
Relevance of Effort Varialples

The relevance of trainees’ effort was measured by the sequence of their practice
and study. The direct effect of the behavioral and cognitive guidance on the relevance of
effort variables was analyzed using the between-subject effects from the repeated
measures MANCOVA described above (see Tables 3 and 4). The results for Speciﬁc
hypotheses are described below.

Hypothesis 1. The ﬁrst hypothesis suggested that trainees receiving behavioral
and cognitive guidance would display a more appropriate practice sequence. In other
words, trainees receiving behavioral and cognitive guidance would be more likely to
practice the TAG Skills and Strategies in the appropriate order. It was found that
behavioral and cognitive guidance had a Si gniﬁcant overall effect on trainees’ practice
sequence, E(1, 272) = 131.35, p_< .001, 112 = .33. An analysis of the means revealed that
trainees receiving behavioral and cognitive guidance practiced signiﬁcantly more of the
TAG Skills in the appropriate sequence. Thus, hypothesis 1 was supported. The effect of

the behavioral\cognitive guidance on practice sequence is presented in Figure 4.

55

Hypothesis 2. The second hypothesis predicted that trainees receiving behavioral
and cognitive guidance would display a more appropriate study sequence. In other
words, trainees receiving behavioral and cognitive guidance would be more likely to
study the TAG Skills and Strategies in the appropriate order. The repeated measures
MANCOVA revealed that the behavioral and cognitive guidance had a Signiﬁcant overall
effect on study sequence, E0, 272) = 79.87, p_< .001, n2 = .23. Consistent with
hypothesis 2, an examination of the means showed that trainees receiving behavioral and
cognitive guidance spent Significantly more time Studying relevant TAG skills and
strategies during the appropriate training blocks. The effect of the behavioral\cognitive
guidance on study sequence is presented in Figure 5.

Additional analyses. Post—hoe analyses were performed to examine whether
affective guidance had a direct effect on the relevance of effort variables. The repeated
measures MANCOVA indicated that affective guidance had a Signiﬁcant overall effect
on both practice sequence, E( 1, 272) = 3.23, p< .05, n2 = .01 , and study sequence, E(1,
272) = 10.57, p_< .01, n2 = .04. An examination of the means revealed that individuals

receiving affective guidance actually practiced Signiﬁcantly less of the TAG skills and
strategies in the appropriate sequence, but they did study Signiﬁcantly more of the TAG
skills and strategies in the appropriate sequence.

Summa_ry. As expected, trainees who received behavioral\cognitive guidance
displayed a more appropriate sequence of practice and Study. They practiced and studied
more of the relevant TAG skills and strategies during the appropriate training blocks.
Post-hoe analyses revealed that affective guidance also had Signiﬁcant effects on practice

and study sequence. Trainees who received affective guidance practiced signiﬁcantly

56

fewer of the TAG Skills and strategies in the appropriate sequence; however, they studied
signiﬁcantly more of the TAG skills and strategies during the relevant training blocks.
Process Variables

The model presented in Figure 1 hypothesizes relationships between the guidance
manipulations and several process variables. The process variables examined in the
present study were self-satisfaction, self-efﬁcacy, on-task cognition, and the locus of
trainees’ attributions. The hypothesized relationships between the manipulations and the
process variables were tested using the between-subject effects from the repeated
measures MANCOVA described earlier (see Tables 3 and 4). The results of the analyses
for specific hypotheses are presented below.

Hypothesis 3a. This hypothesis predicted that behavioral and cognitive guidance
would lead to lower levels of self—satisfaction by means of providing individuals with
sometimes negative evaluations of their performance. The MANCOVA revealed that
behavioral and cognitive guidance had a Si gniﬁcant overall effect on self-satisfaction,

E( 1, 272) = 8.79, p_< .01, n2 = .03. Contrary to expectations, however, it was found that
individuals receiving behavioral and cognitive guidance displayed higher levels of self-
satisfaction. Thus, hypothesis 3a was not supported, although there was a signiﬁcant
relationship between behavioral and cognitive guidance and self-satisfaction. This
relationship is presented in Figure 6.

Hypothesis 4. The fourth hypothesis suggested that individuals receiving
behavioral and cognitive guidance would display higher levels of self-efﬁcacy. The
MANCOVA revealed instead that behavioral and cognitive guidance did not have a

Signiﬁcant overall effect on self-efﬁcacy. However, the MANCOVA did reveal a

57

Si gnificant interaction between time and behavioral\cognitive guidance on self-efficacy,
E(1, 272) = 16.36, p_< 001,112 = .06. Follow-up univariate tests conducted at each time
period revealed that behavioral and cognitive guidance led to signiﬁcantly higher levels
of self-efﬁcacy at Time 1, E0, 274) = 4.34, a< 05,112 = .02, but signiﬁcantly lower
levels of self-efﬁcacy at Time 2, E(l, 274) = 3.89, p = .050, n2 = .01. Although the
effects at Time 1 were consistent with the hypothesized relationship between behavioral
and cognitive guidance and self-efﬁcacy, the overall effects of behavioral and cognitive
guidance on self-efﬁcacy did not support hypothesis 4. The effect of the
behavioral\cognitive guidance on self-efﬁcacy is presented in Figure 7.

Hypothesis 6. The sixth hypothesis predicted that affective guidance would lead
to higher levels of self-efficacy. The repeated measures MANCOVA, however, revealed
that affective guidance did not have a signiﬁcant overall effect on self-efﬁcacy. Thus,
hypothesis 6 was not supported.

Hypothesis 7:; This hypothesis suggested that individuals receiving affective
guidance would display greater on-task cognition than individuals not receiving affective
guidance. Contrary to expectations, however, affective guidance did not have a
signiﬁcant effect on trainees’ on-task cognition. Thus, hypothesis 7a was not supported.

Hypothesis 8a It was predicted that trainees receiving affective guidance would
display a more internal locus of attributions. In other words, trainees receiving affective
guidance would attribute their performance to internal rather than external causes.
However, the MANCOVA revealed that affective guidance did not have a signiﬁcant

effect on the locus of trainees’ attributions. Thus, hypothesis 8a was not supported.

58

Hypotheses 9a & 9b. It was predicted that both higher levels of self-efﬁcacy and
a more internal locus of attributions would lead to greater on-task cognition. To test
these hypotheses, a two-step hierarchical regression analysis using on-task cognition as
the dependent measure was performed at each time period. Cognitive ability was entered
in the ﬁrst step and treated as a covariate. Self-efﬁcacy and attributions were entered in
the second step. At Time 1, it was found that the step containing self-efﬁcacy and
attributions explained a Signiﬁcant portion of the variance (ﬂ = .113, p < .001) after
controlling for ability. Both self-efﬁcacy (ﬂ = .284, p < .001) and attributions (ﬂ = .130,
p < .05) had Signiﬁcant and positive effects on on-task cognition. The results for Time 1
are presented in Table 5. The results at Time 2 were similar. Once again, the step
containing self-efﬁcacy and attributions explained a signiﬁcant portion of the variance

(4133 = .199, p < .001) after controlling for ability. Both self-efﬁcacy (ﬂ = .209, p < .01)
and attributions (ﬂ = .318, p < .001) had signiﬁcant and positive effects on on-task

cognition. The results for Time 2 are presented in Table 6. Thus, hypotheses 9a and 9b
were supported at both Time 1 and Time 2.

Additional analyses. Post-hoc analyses were performed to examine whether
behavioral and cognitive guidance inﬂuenced on-task cognition and the locus of trainees’
attributions and whether affective guidance inﬂuenced trainees’ self-satisfaction. The
MANCOVA revealed that behavioral and cognitive guidance did not have a signiﬁcant
overall effect on on-task cognition or attributions and that affective guidance did not have
a Si gniﬁcant overall effect on self-satisfaction. However, the MANCOVA did reveal a
Si gniﬁcant interaction between time and behavioral\cognitive guidance on attributions,

E( l, 272) = 9.88, p_< .01, n2 = .04. Follow-up univariate tests conducted within each

59

time period revealed that behavioral and cognitive guidance led to a signiﬁcantly more
internal locus of attributions at Time 1, Ij(1, 276) = 7.61 , p_< .01, n2 = .03, but did not
signiﬁcantly effect the locus of trainees’ attributions at Time 2.

Summag. Contrary to expectations, it was found that behavioral\cognitive
guidance led to Signiﬁcantly higher levels of self-satisfaction. Individuals who received
behavioral\cognitive guidance also displayed Signiﬁcantly higher levels of self-efﬁcacy at
the beginning of training, but signiﬁcantly lower levels of self-efﬁcacy at the end of
training. Affective guidance, however, did not have a signiﬁcant effect on self-efﬁcacy,
on-task cognition, or the locus of trainees’ attributions. AS expected, individuals with
higher levels of self-efficacy and more internal attributions displayed Signiﬁcantly higher
levels of on-task cognition. Post-hoc analyses revealed that behavioral\cognitive
guidance did not have a signiﬁcant effect on on-task cognition and affective guidance did
not have a Si gniﬁcant effect on self-satisfaction; however, behavioral\cognitive guidance
did lead to signiﬁcantly more internal attributions among trainees early in training.

Effort Variables

The amount of effort put forth by participants during training was measured by
the total amount of time spent studying and the total amount of time Spent on feedback.
These effort variables were measured both at Time 1 and Time 2. However, it was
expected that the greatest differences on these variables would be observed at Time 2
when some trainees are withdrawing or have already withdrawn from the task. Thus, the
effects of the process variables on the effort variables were tested at Time 2 using two-
step hierarchical regression analyses. Cognitive ability was entered in the ﬁrst step and

treated as a covariate and the process variables were entered in the second step. Two

60

separate hierarchical regression analyses were performed. In one analysis, the dependent
measure was total study time. The results of this analysis are presented in Table 7. In the
second analysis, the dependent measure was total time spent on feedback. The results of
this analysis are presented in Table 8. The results of the regression analyses for Speciﬁc
hypotheses are presented below.

Hypothesis 3b. This hypothesis suggested that individuals who were more
dissatisﬁed with their performance would display greater effort on the task. Speciﬁcally,
individuals with lower levels of self-satisfaction would spend more time studying and
reviewing feedback. Contrary to expectations, however, self-satisfaction did not
Si gniﬁcantly effect total study time or time spent reviewing feedback. Thus, hypothesis
3b was not supported.

Hypothesis 5. The ﬁfth hypothesis predicted that higher levels of self-efﬁcacy
would lead to more time spent studying and reviewing feedback. The regression analyses
revealed, however, that self-efficacy did not signiﬁcantly effect total study time or time
spent reviewing feedback. Thus, hypothesis 5 was not supported.

Hypothesis 7b. This hypothesis predicted that individuals who had higher levels
of on—task cognition would display greater effort. Speciﬁcally, individuals who displayed
more on-task cognition would spend more time studying and reviewing feedback. It was
found that on-task cognition had signiﬁcant and positive effects on both total study time
(Q = .199, p < .01) and time spent on feedback (13 = .198, p < .01) after controlling for
ability. Thus, hypothesis 7b was supported. The effect of on-task cognition on total time

Spent Studying is presented in Figure 8.

61

Hypothesis 8b. It was predicted that individuals with a more internal locus of
attributions would spend more time Studying and reviewing feedback. However, the
regression analyses revealed that attributions did not Signiﬁcantly effect time spent
studying or reviewing feedback. Thus, hypothesis 8b was not supported.

Additional Analyses. The between-subject effects from the repeated measures
MANCOVA conducted earlier were examined to determine whether the manipulations
had direct effects on either total time Spent studying or time spent reviewing feedback.
The MANCOVA revealed that affective guidance had a Si gniﬁcant overall effect on total

time spent studying, E(l , 272) = 14.97, p< .001, n2 = .05, but did not have a signiﬁcant

overall effect on time Spent reviewing feedback. An examination of the means revealed
that individuals who received affective guidance Spent signiﬁcantly more time studying.
The effect of the affective guidance on total Study time is presented in Figure 9. The
MANCOVA also revealed that behavioral and cognitive guidance had a marginally
significant effect on total time spent studying, E(1, 272) = 3.40, p = .066, n2 = .01, and a
Signiﬁcant effect on time spent reviewing feedback, E(1, 272) = 8.77, p_< .01, n2 = .03.
An analysis of the means revealed that individuals who received behavioral and cognitive
guidance spent signiﬁcantly more time studying but signiﬁcantly less time reviewing
feedback.

Summa_ry. Contrary to expectations, it was found that neither self-satisfaction,
self-efﬁcacy, nor locus of attributions had a Signiﬁcant effect on trainee effort. As
predicted, however, higher levels of on-task cognition led to signiﬁcantly higher levels of
effort among trainees. Although affective guidance did not have a signiﬁcant effect on

time spent reviewing feedback, trainees who received affective guidance did spend

62

signiﬁcantly more time studying the computerized training manual. Finally, individuals
who received behavioral\cognitive guidance spent signiﬁcantly more time studying than
other trainees, but also Spent signiﬁcantly less time reviewing their feedback.
Outcome Variables — Practice

The sequence of trainees’ study and practice and the amount of time trainees
spent studying and reviewing their feedback were hypothesized to inﬂuence a number of
outcome variables. The outcome variables measured in the present study were basic
knowledge, Strategic knowledge, basic performance, and strategic performance. All of
these outcome variables were measured at both Time 1 and Time 2. However, the basic
outcome measures were analyzed at Time 1 because the guidance directs trainees to focus
on practicing and studying basic TAG components early in training. In contrast, the
strategic outcomes were analyzed at Time 2 because the guidance directs trainees to
focus on practicing and studying strategic TAG components later in training. Once
again, two-step hierarchical regression analyses were used to test Speciﬁc hypotheses.
Ability was entered in the ﬁrst step and treated as a covariate and the relevance of effort
and effort variables were entered in the second step. Four separate hierarchical
regression analyses were performed on the outcome variables. The ﬁrst analysis was
conducted with basic knowledge at Time 1 as the dependent variable. The results of this
analysis are presented in Table 9. The second analysis was conducted with strategic
knowledge at Time 2 as the dependent variable. The results of this analysis are presented
in Table 10. The third analysis was conducted with basic performance at Time 1 as the
dependent variable. The results of this analysis are presented in Table 11. The ﬁnal

analysis was conducted with strategic performance at Time 2 as the dependent variable.

63

The results of this analysis are presented in Table 12. The results for speciﬁc hypotheses
are presented below.

Hypothesis 10a. This hypothesis predicted that individuals who displayed a more
appropriate practice and study sequence and who spent more time studying and reviewing
feedback would exhibit higher levels of basic knowledge. The results of the hierarchical
regression analysis indicated that only practice sequence had a Signiﬁcant effect on basic
knowledge (0 = .109, p = .05) after controlling for ability. AS expected, individuals with
a more appropriate practice sequence exhibited higher levels of basic knowledge. Thus,
hypothesis 10a was partially supported. The effect of practice sequence on trainees’
basic knowledge is shown is Figure 10.

Hypothesis 10b. This hypothesis suggested that individuals who displayed a more
appropriate practice and study sequence and who Spent more time studying and reviewing
feedback would exhibit higher levels of strategic knowledge. The regression analysis
revealed that both practice (13 = .264, p < .001) and study sequence (11 = .259, p < .001)
had signiﬁcant effects on strategic knowledge after controlling for ability; however,
neither total study time nor time spent on feedback had a signiﬁcant effect on strategic
knowledge. Thus, hypothesis 10b was partially supported. The effect of practice
sequence on trainees strategic knowledge is presented in Figure 11.

Hypothesis 10c. This hypothesis predicted that individuals with a more
appropriate practice and study sequence and who Spent more time studying and reviewing
feedback would display higher levels of basic performance. Contrary to expectations,
however, none of the relevance of effort or effort variables signiﬁcantly affected basic

performance. Thus, hypothesis 10c was not supported.

Hypothesis 10d. This hypothesis suggested that individuals who displayed a more
appropriate practice and Study sequence and who spent more time Studying and reviewing
feedback would exhibit higher levels of strategic performance. The regression analysis
revealed that practice sequence (0 = .268, p < .001), study sequence (ﬂ = .186, p < .01),
and total study time (Q = -.213, p < .001) had signiﬁcant effects on strategic performance
after controlling for ability. AS expected, individuals with a more appropriate practice
and study sequence had signiﬁcantly higher levels of Strategic performance. Contrary to
expectations, however, more time Spent studying resulted in signiﬁcantly lower levels of
strategic performance. Thus, hypothesis 10d was partially supported. The effect of
practice sequence on strategic knowledge is presented in Figure 12.

Additional Analyses. The repeated measures MANCOVA was used to examine
the direct effects of the manipulations on the outcome variables. The MANCOVA
revealed that affective guidance did not have a direct overall effect on any of the outcome
variables. However, behavioral and cognitive guidance had an overall effect on strategic
knowledge, E(1, 272) = 9.90, p < .01, n2 = .04, and strategic performance, Ij(1, 272) =
15.76, p < .001, n2 = .06. Individuals who received behavioral and cognitive guidance
displayed significantly higher levels of both Strategic knowledge and performance.
Although behavioral and cognitive guidance did not have an overall signiﬁcant effect on
basic knowledge and performance, there was a Signiﬁcant interaction between time and

behavioral and cognitive guidance on basic knowledge, E(1, 272) = 5.60, p < .05, n2 =
.02, and basic performance, E(1, 272) = 21.20, p < .001, n2 = .10. AS would be expected,

follow—up univariate tests conducted within each time period revealed that behavioral and

cognitive guidance had a significant effect on both basic performance, E(1, 276) = 13.12,

65

p < 001,112 = .05, and basic knowledge, Ed. 276) = 7.93, 2 < .01, n2 = .03, at Time 1
but not Time 2. Individuals who received behavioral and cognitive guidance displayed
higher levels of both basic knowledge and basic performance at Time 1.

Another series of post-hoc analyses were performed to examine whether the
process variables had direct effects on the outcome variables. Once again, two-step
hierarchical regression analyses were performed. In the ﬁrst step, ability was entered and
treated as a covariate. In the second Step, the process variables were entered. As before,
four separate hierarchical regression analyses were performed. In the ﬁrst analysis, basic
knowledge at Time 1 was the dependent variable. The results of this analysis are
presented in Table 13. In the second analysis, strategic knowledge at Time 2 was the
dependent variable. The results of this analysis are presented in Table 14. In the third
analysis, basic performance at Time 1 served as the dependent variable. The results of
this analysis are presented in Table 15. In the ﬁnal analysis, strategic performance at
Time 2 was the dependent variable. The results of this analysis are presented in Table 16.
A summary of the findings from these analyses is provided below.

The ﬁrst hierarchical regression analysis demonstrated that both self-efﬁcacy (ﬂ =
.159, p < .05) and attributions (ﬂ = .156, p < .010) had signiﬁcant and positive effects on
basic knowledge after controlling for ability. The second hierarchical regression revealed
that self-efﬁcacy (ﬂ = -. 191 , p < .05) and on-task cognition (ﬂ = .166, p < .05) had

Si gniﬁcant effects on strategic knowledge after controlling for ability. While higher
levels of self—efﬁcacy resulted in Si gniﬁcantly lower levels of strategic knowledge, higher
levels of on-task cognition resulted in signiﬁcantly more strategic knowledge. The third

hierarchical regression analysis indicated that self-satisfaction (ﬂ = .354, p < .001) and

66

internal locus of attributions (ﬂ = .212, p < .001) had signiﬁcant and positive effects on
basic performance after controlling for ability. The ﬁnal hierarchical regression
demonstrated that self-satisfaction (ﬂ = .142, p < .05) and on-task cognition (B = .171, p <
.01) both had Si gniﬁcant and positive effects on strategic performance after controlling
for ability.

Summg. Analyses revealed that individuals with a more appropriate study
sequence had Signiﬁcantly higher levels of strategic knowledge and performed
signiﬁcantly better than other trainees on the Strategic aspects of the task. Individuals
with a more appropriate practice sequence had signiﬁcantly higher levels of basic and
strategic knowledge and also performed signiﬁcantly better than other trainees on the
Strategic aspects of the task. In addition, individuals who Spent more time studying
performed signiﬁcantly better on the strategic portions of the task. Post-hoe analyses
revealed that while affective guidance did not have a signiﬁcant effect on any of the
outcome variables, behavioral\cognitive guidance led to signiﬁcantly higher overall
levels of Strategic knowledge and performance and also led to higher levels of basic
knowledge and performance early in training. Additional post-hoc analyses revealed that
higher levels of self-efﬁcacy led to Signiﬁcantly higher levels of basic and strategic
knowledge. A more internal locus of attributions led to signiﬁcantly higher levels of
basic knowledge, while more on-task cognition led to signiﬁcantly higher levels of
strategic knowledge and performance. Finally, higher levels of self-satisfaction led to

signiﬁcantly better basic and strategic performance.

67

Outcome Variables — Generalization

 

Basic and strategic performance were also measured in the more difﬁcult and
complex generalization trial. The model presented in Figure 1 hypothesizes relationships
between the outcome variables measured during training and the outcome variables
measured during generalization. To test these relationships, the outcome variables
measured during the generalization trial were regressed onto the outcome variables
measured at Time 2 during training. A hierarchical regression was performed in which
cognitive ability was entered in the ﬁrst step and the training outcome variables at Time 2
were entered in the second step. Two separate hierarchical regression analyses were
performed. In the ﬁrst analysis, basic performance during generalization served as the
dependent variable. The results of this analysis are presented in Table 17. In the second
analysis, strategic performance during generalization was the dependent variable. The
results of this analysis are presented in Table 18.

Hypothesis 1 1. This hypothesis predicted that individuals who performed better
during practice and who displayed higher levels of knowledge during practice would
exhibit higher levels of basic and strategic performance during the generalization trial.

The regression analyses revealed that basic knowledge during practice ([1 = .193, p <
.001), basic performance during practice (ﬂ = .552, p < .001), and strategic performance
during practice (I) = .153, p < .01) each had a Signiﬁcant and positive effect on basic

performance during generalization after controlling for ability. The relationship between
basic performance during practice and basic performance during generalization is Shown
in Figure 13. Strategic knowledge at the end of training did not predict basic

performance during transfer. However, Strategic knowledge during training (13 = .200, p

68

< .001) and strategic performance during training (13 = .648, p < .001) each had a
signiﬁcant and positive effect on Strategic performance during generalization after
controlling for ability. Thus, hypothesis 11 was supported. The effect of strategic
performance during practice on Strategic performance during generalization is shown in
Figure 14.

Additional analyses. A MANCOVA was performed to examine the direct effects
of the manipulations on the outcome variables during generalization. Cognitive ability
served as a covariate and basic and strategic performance during generalization served as
the dependent variables. The MANCOVA revealed that affective guidance did not have
a Si gnificant overall effect on the combined dependent variables. Behavioral and
cognitive guidance, on the other hand, did have a signiﬁcant overall effect on the
generalization variables, EU, 271) = 16.35, p < .001, n2 = .11. The overall results are
presented in Table 19. Follow-up between subjects analyses revealed that behavioral and
cognitive guidance did not have a signiﬁcant effect on basic performance during
generalization, but it did have a signiﬁcant effect on strategic performance during
generalization, Ij(1, 271) = 29.86, p < 001,1]2 = .10. An examination of the means
revealed that individuals who received behavioral and cognitive guidance displayed
signiﬁcantly higher levels of strategic performance in the more difﬁcult and complex
generalization trial. The between-subject effects for behavioral and cognitive guidance
on the generalization variables are presented in Table 20.

Summgy. AS expected, individuals with higher levels of basic knowledge, basic
performance, and strategic performance during practice displayed signiﬁcantly higher

levels of basic performance during generalization. In addition, higher levels of strategic

69

knowledge and strategic performance led to signiﬁcantly higher levels of Strategic
performance on the generalization trial. Post-hoc analyses revealed that while affective
guidance did not have an effect on performance during the generalization trial,
behavioral\cognitive guidance led to signiﬁcantly higher levels of strategic performance.
Owl] Model Fit

After testing the individual hypotheses, I performed an overall test of the
hypothesized model using Structural equation modeling. The model was tested using
AMOS, and the path diagram that was analyzed is presented in Figure 15. The ﬁt indices

for the hypothesized model were as follows: x2(88, N = 277) = 550.78, p < .001; GFI =

.79; AGFI = .64; CFI = .67; and RMSEA = .138. Values of around .90 or higher for the
GFI, AGFI, and CFI indicate good ﬁt. Values between 0 and .05 for the RMSEA
indicate very good ﬁt, and values between .05 and .10 indicate good ﬁt. Thus, the
observed fit statistics indicate that the hypothesized model did not ﬁt the data very well.

Because the hypothesized model did not provide a good ﬁt to the data, a post-hoe
or revised model was constructed and tested using AMOS. Although guided somewhat
by earlier analyses, this post-hoc model was conceptual in nature. The path diagram used
to test the revised model is presented in Figure 16. Both the hypothesized and revised
model contained mostly the same variables; however, an attempt was made to create a
more parsimonious model. As a result, total time spent on feedback and self-satisfaction
were not included in the revised model. Total time spent on feedback was eliminated due
to restricted variance. Trainees only Spent an average of 1 minute 41 seconds reviewing
the feedback during the third training block, which limited the degree of variability

among trainees. Self-satisfaction was eliminated because it was highly correlated with

70

both self-efficacy (_r_= .481) and attributions (g = .347). In addition to eliminating these
two variables, several other changes were made to the hypothesized model.

As can be seen in Figure 16, the revised model included a link between
attributions and self-efﬁcacy. It was hypothesized that more internal attributions would
lead to higher levels of self-efﬁcacy. In addition, it was predicted that self-efﬁcacy and
attributions early in training would be signiﬁcantly and positively related to level of on-
task cognition at the end of training. This relationship was examined across time because
it was hypothesized that the behavioral\cognitive guidance would have its effects on self-
efficacy and attributions early in training, but that differences in on-task cognition would
be observed later in training as individuals begin to withdraw from the task. On-task
cognition, in turn, was hypothesized to directly affect all four of the outcome variables:
basic knowledge, basic performance, strategic knowledge, and strategic performance.
Greater focus on the task should result in improved performance. Relationships were
also hypothesized between total study time and study sequence and study sequence and
practice sequence. I predicted that the more time an individual Spent Studying the more
likely they were to Study the relevant training material. Also, there should be a positive
and Si gniﬁcant relationship between Study and practice sequence because trainees will
practice the skills and strategies they learn while studying. The remainder of the revised
model is very Similar to the original, hypothesized model. In the original model, it was
hypothesized that each of the relevance of effort and effort variables would have an effect
on all of the outcome variables. In the revised model, however, speciﬁc relationships
were specified in an attempt to create a more parsimonious ﬁt. Thus, it was predicted

that total time Spent studying would have Significant and positive effects only on basic

71

and strategic knowledge. The effect of total study time on performance would exist
through these knowledge variables. Study sequence at the end of training Should have
Signiﬁcant and positive effects only on strategic knowledge because this is the material
that the guidance is directing trainees to Study. Thus, those individuals following the
proper Study sequence Should display higher levels of strategic but not basic knowledge.
For similar reasons, practice sequence at the end of training should only have positive
effects on strategic knowledge and Strategic performance. It was also hypothesized that
basic performance during generalization would be affected only by basic performance
and knowledge during practice. Likewise, strategic performance during generalization
should only be affected by strategic performance and knowledge during practice.

The revised model resulted in a much better ﬁt to the data. . The ﬁt indices for the

hypothesized model were as follows: x203, N = 277) = 228.08, p < .001; GFI = .90;

AGFI = .84; CFI = .88; and RMSEA = .088. The revised model resulted in a moderately
good ﬁt to the data and a much better and more parsimonious ﬁt than the original,

hypothesized model.

72

DISCUSSION

Computers have been used to train individuals Since computer-managed and
computer-assisted instruction ﬁrst emerged in the early 1970’s. These initial computer-
based training programs were designed to accomplish two main goals: (1) reduce total
training time and the amount of student-instructor interaction; and (2) provide
individualized instruction by giving people control over different aspects of the learning
process, such as sequence, content, and pace. The idea was that individuals could learn
more and more quickly if given control over the learning process, and could do so with
little or no student-instructor interaction. Although much more complex, many of today’s
computer-based training techniques, such as web-based training and distance learning,
incorporate these same principles (Brown, 1999).

A considerable amount of research over the last three decades has focused on the
advantages and disadvantages of different aspects of computer-based training, including
the effects of learner control. One of the most consistent ﬁndings associated with learner
control has been that individuals do not make good use of the control they are given
(Williams, 1993). Typically, when given control over their learning, most individuals
will either exit instruction before they have mastered the task (Tennyson, Tennyson, &
Rothen, 1980; Ross & Rakow, 1981) or practice well beyond the point at which concepts
have been acquired (Tennyson 1980, 1981). Despite this drawback, learner control has
been shown to offer several beneﬁts, such as reduced Student-instructor time with little or
no performance decrement. Thus, a major challenge is to ﬁnd a way to eliminate the

negative aspects of learner control while maintaining its positive contributions. One

73

proposed means of achieving this goal is to supplement learner control with some form of
advisement or guidance that helps to direct trainees’ learning.

Although there hasn’t been much research conducted on advisement, it appears
that advising individuals on their learning needs and providing them with information for
making learning decisions is an effective instructional approach (Tennyson, 1980;
Tennyson & Buttrey, 1980, Santiago & Okey, 1992). The present study was designed to
build on the past research on advisement in several ways. Most of the research on both
learner control and advisement has focused on only one main dependent variable, post-
test performance (i.e., declarative knowledge). In addition, past research has failed to
examine the use of guidance for inﬂuencing individuals’ affect. In the present Study,
however, I examined the effect of two forms of guidance, behavioral\cognitive and
affective, on several different behavioral, cognitive, and affective variables, and
investigated how different levels of these variables subsequently inﬂuenced performance.
Another limitation of past research on learner control and advisement is that it has been
conducted using Simple learning tasks. In the present study, guidance was applied to a
complex and difficult training simulation.

Behavioral\Cognitive Guidance

One problem with giving individuals control over their learning is that they often
spend too little time studying and practicing important training content (Tennyson,
Tennyson, & Rothen, 1980; Ross & Rakow, 1981). For example, individuals may spend
too much time on material that is easy to learn and too little time on that material which is
difficult to grasp. Even if individuals do spend enough time studying and practicing

relevant training material, they may fail to do so in the appropriate sequence. Learning is

74

typically a gradual process in which the acquisition of difﬁcult and complex skills and
knowledge depends on the degree to which one has previously learned more fundamental
or basic Skills and knowledge. Thus, poor sequencing of learning can inhibit the
acquisition of critical skills and knowledge. This is especially true when learning
complex tasks, like the one used in the present study.

In the present study, one of the major goals of the behavioral\cognitive guidance
was to direct individuals’ learning so that they not only studied and practiced relevant
training material, but also did so in the appropriate sequence. AS trainees mastered basic
knowledge and skills, the guidance adapted to their level of acquisition and instructed
them to focus attention and effort on increasingly advanced knowledge and Skills. The
behavioral\cognitive guidance was not aimed at getting trainees to practice and study
more, but rather to practice and study better. It appears from the results that the
manipulation was extremely successful in this regard. Individuals who received
behavioral\cognitive guidance practiced almost twice as many of the relevant training
topics as those trainees who did not receive the guidance (see Figure 4). In addition, they
spent over 25% more time studying the relevant training topics than other trainees (see
Figure 5). These findings not only illustrate the effectiveness of the guidance for
directing trainees’ study and practice but also support past research that has shown that
individuals do not make good use of the control they are given over the sequence and
content of their Study and practice.

While the behavioral\cognitive guidance was primarily aimed at inﬂuencing the
sequence of trainees’ study and practice, it was also expected to have an effect on several

affective variables. One such variable was self-satisfaction. Past research has found that

75

individuals need to be able to compare their past performance to some standard of
performance if they are going to develop feelings of self-satisfaction or self-
dissatisfaction (Bandura & Cervone, 1983). In the present study, the behavioral\cognitive
guidance provided trainees with speciﬁc information about how their performance on
particular skills or Strategies compared to a desired level of performance. Because it
would be difﬁcult for trainees to reach the desired level of performance, it was
hypothesized that the behavioral\cognitive guidance would lead to higher levels of self-
dissatisfaction. Contrary to expectations, however, it was found that the
behavioral\cognitive guidance actually resulted in signiﬁcantly higher overall levels of
self-satisfaction. As can be seen in Figure 6, the greatest difference in self-satisfaction
between trainees who received behavioral\cognitive guidance and those who did not is
found early in training. Most likely, trainees who received behavioral\cognitive guidance
displayed higher levels of self-satisfaction because they learned the basic skills more
quickly and experienced more rapid progress in their performance early in training.
Behavioral\cognitive guidance was also expected to inﬂuence trainees’ self-
efficacy. Whether perceived discrepancies between personal Standards and attainments
are motivating or discouraging depends in part on the degree to which people believe
they can achieve the standards they are pursuing (Bandura & Cervone, 1983, 1986a). In
the present Study, individuals who received behavioral\cognitive guidance were told that
they would be guided through training. In addition, they were given information about
their performance deﬁciencies and told exactly what they could do to meet the set
performance standards. Thus, it was hypothesized that the behavioral\cognitive guidance

would increase self-efﬁcacy. In the beginning of training, individuals who received

76

behavioral\cognitive guidance did display Signiﬁcantly higher levels of self-efficacy.
This is extremely important considering that early in training is when trainees received
the most negative feedback and were thus most likely to develop feelings of low self-
efﬁcacy. In contrast, however, behavioral\cognitive guidance actually led to signiﬁcantly
lower levels of self-efﬁcacy at the end of training. As can be seen in Figure 7, trainees
who received behavioral\cognitive guidance experienced a very small increase in self-
efﬁcacy from early in training to the end of training. Trainees who did not receive
behavioral\cognitive guidance, on the other hand, displayed a much more dramatic
increase in self-efﬁcacy across time. This difference is most likely caused by the fact that
trainees who did not receive behavioral\cognitive guidance were focusing mostly on the
basic aspects of the task and thus had unrealistically high perceptions of their progress
and overall performance. Trainees who received behavioral\cognitive guidance, on the
other hand, were focusing not only on the basic aspects but also the more difﬁcult,
strategic aspects of the task and thus had a more realistic perception of their progress and
overall performance. Just as low self-efﬁcacy can lead trainees to withdraw from a task,
high levels of self-efﬁcacy can result in overconﬁdence and the same outcome. Thus, the
moderately high level of self-efﬁcacy maintained by trainees who received
behavioral\cognitive guidance is probably more realistic for effective performance in the
long run.

The behavioral\cognitive guidance was expected to have an affect on trainee
performance and knowledge through the variables discussed above. However, the results
for these relationships, which will be discussed in more detail below, were mixed. One

possible explanation is that the behavioral\cognitive guidance had an effect on individual

77

performance and knowledge that was not explained by the mediating variables. Thus, we
chose to perform post-hoc analyses to examine the direct effect of the
behavioral\cognitive guidance on trainees’ performance and knowledge during practice.
These analyses showed that the behavioral\cognitive guidance had signiﬁcant overall
effects on both strategic knowledge and performance. As expected, the
behavioral\cognitive guidance led to Signiﬁcantly higher levels of both Strategic
knowledge and performance. The behavioral\cognitive guidance also had an effect on
the basic outcome indicators; however, as expected, these effects were only found early
in training. Trainees who received behavioral\cognitive guidance displayed higher levels
of basic performance and knowledge early in training. By the end of training, however,
most trainees had acquired the basic skills and knowledge and there was no difference
between groups on these basic outcome variables.

In summary, the behavioral\cognitive guidance had a large impact on a number of
behavioral, cognitive, and affective variables. The major goal of the behavioral\cognitive
guidance was not to get trainees to practice and study more, but to get them to practice
and study better. The behavioral\cognitive guidance was able to achieve this goal by
getting trainees to follow the appropriate sequence of study and practice, and, as we will
discuss below, the sequence of traineeS’ Study and practice had substantial effects on how
they performed on the task. Although not the major focus of the behavioral\cognitive
guidance, the manipulation did have an effect on two affective variables, self-satisfaction
and self-efﬁcacy, that were later shown to effect trainees’ level of on-task cognition.
Finally, the guidance had signiﬁcant and positive direct effects on basic and strategic

knowledge and performance variables.

78

 

Affective Guidance

One of the primary challenges of training design for complex tasks is that trainees
often perform poorly during the initial Stages of skill acquisition. Performance that fails
to meet the trainees’ — often unrealistic — expectations undermines self-evaluations, and
can lead to significant reductions in motivation to learn (Bandura, 1991; Kanfer &
Ackerman, 1989). As a result, a second form of guidance, affective, was created in an
attempt to help trainees exercise emotional control and to bolster self-evaluations in the
face of early task difﬁculties. The affective guidance was not an instructional
intervention per se, but rather a support that was intended to work in concert with the
behavioral\cognitive guidance.

The affective guidance was hypothesized to inﬂuence a number of process
variables, such as self-efficacy and on-task cognition, that would in turn inﬂuence the
amount of effort trainees put forth on the task. Contrary to expectations, however, the
affective guidance did not have a signiﬁcant effect on any of the process variables.
Instead, post-hoe analyses revealed that the affective guidance had a signiﬁcant direct
effect on the amount of effort trainees put forth on the task. As we would expect,
individuals who received affective guidance Spent signiﬁcantly more time than other
trainees studying the on-line training manual. It appears, therefore, that although the
affective guidance did not inﬂuence the process variables as we expected, it led to the
desired outcome — more on-task effort.

Additional post-hoc analyses also revealed that trainees who received affective
guidance were more likely than other trainees to follow the desired study sequence. This

was most likely a by-product of their increased time Spent studying. Trainees who

79

studied more were probably just more likely to study the appropriate material. This
conclusion is supported by the fact that trainees who received affective guidance were
actually less likely than other trainees to follow the appropriate sequence of practice.
Regardless, the affective guidance was successful in inﬂuencing two variables that had
signiﬁcant and positive impacts on trainees’ performance and degree of knowledge.
Process Lariables

One goal of this study was to explain the process by which both
behavioral\cognitive and affective guidance inﬂuenced different training outcomes, such
as performance and knowledge. It was expected that a major determinate of these
outcomes would be the degree of effort trainees put forth on the task. As discussed
above, the behavioral\cognitive and affective guidance were expected to inﬂuence a
number of process variables that would, in turn, inﬂuence the amount of effort trainees
put forth on the task. Unfortunately, however, the process variables did not have much of
an impact on trainee effort. On-task cognition was the only process variable that had a
signiﬁcant effect on the amount of effort trainees put forth on the task. As expected,
trainees with higher levels of on-task cognition spent Signiﬁcantly more time Studying the
on-line training manual. Although on-task cognition was the only process variable to
have a direct effect on trainee effort, both self-efﬁcacy and attributions were Signiﬁcantly
related to on-task cognition. As hypothesized, higher levels of self-efﬁcacy and more
internal attributions led to Si gniﬁcantly higher levels of on-task cognition. It appears,
therefore, that self-efﬁcacy and attributions both affected trainee effort by keeping

trainees focused on the task.

80

Relevance of Effort, Level of Effort, m Outcome Vamlis

In the present study, the variables expected to inﬂuence trainee performance and
knowledge were divided into two categories. One category was the relevance of trainees’
effort, which included the sequence of trainees’ study and practice. As mentioned earlier,
these variables measured the extent to which the guidance was able to get trainees to
work better or smarter. The second category was the level of trainee effort, and included
the amount of time trainees spent studying and the amount of time they spent reviewing
feedback. These variables measured the extent to which the guidance was able to get
trainees to work more.

As expected, the relevance of trainees’ effort had a large impact on a number of
training outcome variables. Trainees who followed a more appropriate sequence of study
displayed significantly higher levels of strategic knowledge and performance. Similarly,
trainees who followed a more appropriate practice sequence displayed Si gniﬁcantly
higher levels of both basic and strategic knowledge and higher levels of strategic
performance. In terms of trainee effort, it was found that individuals who studied more
performed significantly better on the strategic aspects of the task. It is important to note
that both the relevance and amount of trainee effort affected mostly the strategic, or more
difﬁcult and complex, training outcomes. This may explain why past research on learner
control and advisement, which has used mostly Simple tasks, has failed to ﬁnd consistent
effects. Most trainees are able to acquire the basic skills and knowledge; however, the
more complex Skills are not as easy to learn. Thus, the guidance had its largest impact by
way of directing individuals’ learning of these more difﬁcult and complex aspects of the

task.

81

Generalization Performance

In the present study, trainee performance was measured not only during practice
but also in a ﬁnal generalization trial. The generalization trial was designed to assess the
degree to which trainees were able to adapt their basic and strategic skills and knowledge
to a more difﬁcult and challenging environment.

As expected, trainees’ performance and level of knowledge during practice were
the best predictors of their performance during the generalization trial. Both basic
knowledge and basic performance during practice had signiﬁcant and positive effects on
basic performance during generalization. In addition, higher levels of strategic
performance during practice led to signiﬁcantly higher levels of basic performance
during generalization. AS hypothesized, strategic knowledge and strategic performance
during practice had positive and significant effects on strategic performance during
generalization.

While the best predictor of trainees’ performance during generalization was their
performance during practice, it is important to note that behavioral\cognitive guidance
also had a Si gniﬁcant and positive effect on trainees’ level of strategic performance
during generalization. This ﬁnding has two important implications. First, it once again
shows that the guidance had its largest impact on how trainees performed on the more
difﬁcult and complex aspects of the task. Second, it suggests that trainees who received
behavioral\cognitive guidance were not only better able to learn the strategic elements of
the task, as illustrated by the practice results, but they were also better able to adapt those

Skills to the more difﬁcult and challenging generalization task.

82

Overall Model Fit

After testing the individual hypotheses, an overall test of the hypothesized model
was performed using structural equation modeling. AS described earlier, the
hypothesized model did not provide a good ﬁt to the data. As a result, we modiﬁed the
hypothesized model on both a conceptual and empirical basis in an attempt to better
explain the process by which the behavioral\cognitive and affective guidance had their
effects.

In many ways, the revised model was quite similar to the original, hypothesized
model. Major changes, however, were made in the area of the process variables.
Whereas the process variables were originally hypothesized to have an impact on trainee
effort, tests of the original model led us to believe that on-task cognition might itself be a
measure of trainee effort. Trainee effort was originally measured by the amount of time
trainees spent studying and reviewing their feedback. However, we felt that the degree to
which trainees displayed on-task thoughts, or metacognition, might also serve as a
measure of their level of effort, and therefore may directly affect training outcomes, such
as knowledge and performance. Fisher & Ford (1998) found, for example, that on-task
cognition and time on task were only minimally related; however, both measures of
trainee effort signiﬁcantly predicted trainee performance. It was also expected that the
other process variables, such as self-efﬁcacy and attributions, may inﬂuence trainee
performance and knowledge by means of on-task cognition. This relationship was
partially speciﬁed in the original model. These revised relationships are displayed in

Figure 16.

83

In addition to the changes made in the area of the process variables, several
conceptual links were added to the model. For example, links were inserted between
basic knowledge and basic performance and strategic knowledge and strategic
performance. These links are also shown in Figure 16. The ﬁnal change to the original
model was made by eliminating several of the links between the effort, relevance of
effort, and practice outcome variables. The original model did not specify speciﬁc
relationships between these variables, and as a result each outcome variable was linked to
each effort and relevance of effort variable. In an attempt to create a more parsimonious
model, speciﬁc relationships were speciﬁed. For example, total study time was expected
to only inﬂuence the knowledge variables. In addition, Since the guidance late in training
directed individuals to study and practice the strategic aspects of the task, study and
practice sequence measured at the end of training were expected to only effect strategic
knowledge and performance.

The revised model produced a much better fit to the data. The ﬁt statistics
indicated that the revised model ﬁt the data moderately well and provided a more
parsimonious explanation of the data than the original model. AS mentioned above, the
revised model was quite Similar to the original, hypothesized model. Sections of the
original model ﬁt the data quite well. However, major changes were needed in the area
of the process variables in order to better explain the effects of the behavioral\cognitive
and affective guidance. As we expected, on-task cognition had signiﬁcant and positive
effects on all of the basic and strategic outcome variables. This supports the use of this

measure as an indication of the amount of effort trainees put forth on the task.

84

Limitations

The results of the present study clearly demonstrate that the behavioral\cognitive
guidance was a much more effective training intervention than the affective guidance.
There are a few possible reasons for this ﬁnding. First, while the behavioral\cognitive
guidance was adaptive to trainees’ performance, it was not feasible to individualize the
affective guidance. Although pilot tests were performed to determine the most likely
pattern of trainee emotions across the course of training, there was almost certainly a
great deal of variability in how trainees reacted to their performance. As a result, the
affective guidance may have been helpful for some trainees, but ineffective or possibly
detrimental for other trainees. Another factor that most likely limited the effect of the
affective guidance was the rather short duration of the present experiment. The main
goal of the affective guidance was to maintain trainees’ effort and to keep them from
withdrawing from the task. It is possible, however, that the present study was not long
enough to induce factors, such as fatigue, that increase the likelihood of trainee
withdrawal. The affective guidance would probably prove much more effective in
training paradigms in which individuals are trained over the course of days or weeks.

One goal of the present study was to explain the process by which
behavioral\cognitive and affective guidance inﬂuence a number of different training
outcomes, such as knowledge and performance. I was only able, however, to partially
capture this process. One reason why I may have been unable to fully explain this
process is its complexity. I examined the effect of the behavioral\cognitive and affective
guidance on a number of different variables; however, there were surely variables that

were not measured in the present study but nonetheless inﬂuenced the process. I am

85

currently collecting additional data from individuals who participated in the experiment
to examine one such variable, goal orientation. Fisher & Ford (1998) found that goal
orientation had a signiﬁcant effect on both the amount and type of trainee effort, two
major constructs examined in the present study. Similarly, I am interested in whether
guidance and goal orientation may interact to inﬂuence the amount and nature of trainee
effort

Although the behavioral\cognitive guidance had rather large and widespread
effects in the present study, it is important to note that it was a limited form of
advisement. It was not feasible to design the guidance to adapt to very small differences
in trainees’ performance. Instead, the behavioral\cognitive guidance was designed to
adapt to three major levels of performance. More detailed and speciﬁc forms of
behavioral\cognitive guidance would probably demonstrate even larger effects on
training outcomes.

A final limitation that Should be noted is the generalizability of the results of the
present study. The use of a Student sample and a synthetic task may somewhat limit the
degree to which the results of the present study can be generalized to other instructional
paradigms. Although many of the findings will probably generalize to other situations,
caution should be used in this process. That said, however, it is important to note that
this is one of the ﬁrst studies to examine the effects of advisement in a complex training
Situation. Thus, the results of the present study are probably our best current indication
of how guidance can be used to direct the training of individuals on complex and difﬁcult

tasks.

86

Reseapchﬁand Pﬁractical Implications

Computers give us the ability to provide trainees with not only a great deal of
control over their learning but also a great deal of individualized and adaptive
performance information. The results of the present study suggest that combining these
two beneﬁts of computer-based training in an appropriate fashion may help to overcome
some of the disadvantages associated with learner control. The present study supported
past research by Showing that individuals who did not receive advisement failed to make
good use of the control they were given. Trainees who were guided through the learning
process, however, displayed a more effective use of the control they were given over the
sequence and content of their learning.

One of the major contributions of the present study is that it illustrates the effects
of guidance in a complex training situation. This appears to be especially critical
considering that the guidance had its most consistent effects on the more complex and
strategic outcomes. This finding may help to explain why previous learner control and
advisement research, which has used, almost exclusively, simple tasks, has produced
mixed and inconsistent ﬁndings. It is not surprising that the guidance had its largest
effects on those aspects of the task which were difﬁcult and complex, for these are the
areas in which the qualitative aspects of learning, such as sequence, are most critical.
Future research Should expand on the present research by examining the effects of
guidance on training individuals on other complex and difﬁcult tasks. In addition, more
advanced forms of guidance Should be designed and tested in complex experimental

situations.

87

Another contribution of the present research was the partial explanation of the
process by which guidance inﬂuences different training outcomes. While, the effects of
the guidance on trainees’ study and practice sequence and the effects of these variables
on the training outcomes were clear, the process by which the guidance affected trainees’
effort was only partially explained. Future research should examine this process in more
detail. Variables that appeared promising in the present study, such as on—task cognition,
should be examined further, and additional variables not measured in the present study,
such as goal orientation, Should be included and tested.

Conclusion

The present study was designed to examine the effects of two forms of guidance,
behavioral\cognitive and affective, in a complex training environment. It was found that
the behavioral\cognitive guidance had pervasive and substantial impacts on self-
regulation process indicators and on the sequence of trainees’ study and practice.
Behavioral\cognitive guidance yielded signiﬁcant improvements in the acquisition of
basic knowledge and performance capabilities early in training, Signiﬁcant improvements
in the acquisition of strategic knowledge and strategic performance skills later in training,
and significant improvements in the capability to retain and adapt skills in a more
difficult and complex generalization Situation. Despite the fact that the affective
guidance did not inﬂuence the self-regulation process indicators as expected, it did have a
signiﬁcant and positive effect on the amount of trainee effort. Although the process by
which the two forms of guidance operated was only partially explained, the results of the
present study suggest that both behavioral\cognitive and affective guidance have high

potential as effective training strategies. Future research should expand on these ﬁndings

88

in an effort to capitalize on the beneﬁts and eliminate the drawbacks of computer-based

training.

89

FOOTNOTES

1 A total of 15 participants were dropped from the sample for various reasons.
Three participants were dropped due to extremely low scores on a motivation scale
administered at the end of the experiment. The motivation scale included three items
measured on a ﬁve point Likert scale ranging from strongly disaggee (l) to Strongly aggee
(5). A sample item was, “I put forth effort to answer questions accurately and honestly.”
The three individuals described above were dropped from the sample for a mean score
below 2.0 on this scale. The 12 remaining participants were dropped for reasons such as
missing or incomplete data, not following experimenter instructions, and not attending to
the experimental manipulations.

2 The self-report practice measures were scored by both the author and an
assistant. The author provided the assistant with a written scoring guide, explained the
scoring system, and trained the assistant on a random sample of 50 questionnaires. The
author and assistant then independently scored each questionnaire. Inter-rater reliability
was .96 for the session one practice questionnaire, .99 for the session two practice
questionnaire, and .96 for the session three practice questionnaire. Any disagreements in
ratings were resolved through discussion.

90

APPENDIX A

Date: Subject #:

Informed Consent
Tactical Action Game (TAG)

The study in which you are about to participate investigates your learning and
performance on the Tactical Action Game (TAG). TAG is a computer-simulated, radar
tracking task that you will be trained to use, and then you will practice. Using the
computer-mouse, you will assess the attributes of contacts that appear on your screen and
decided what action should be taken for each contact. You will also be asked to answer
questions which will help us understand your task performance and learning.

Your participation in this Study requires three and one-half (3.5) hours of your time.
Your participation is completely voluntary and you are free to discontinue the study at
any time for any reason without penalty. No risks or discomforts are anticipated as a
result of this Study.

You will be given a subject number at the beginning of the experiment. The purpose of
this subject number is to keep track of the various materials you will complete throughout
the study. Your name will not be associated with your responses and will be kept
confidential.

Awards available for this Study are explained in detail in a separate handout. Winners
will be determined at the conclusion of the study. If you win, you will be contacted at the
address and phone number you indicate below within 3 weeks of the study’S conclusion.
Instructions for claiming the award will be provided when you are contacted.

At the end of your involvement in this study, you will be provided with feedback
explaining the purpose of the research in more detail. You may ask about the results of
the study when it is complete by contacting the investigator. The investigator’s name and
phone number is on the debrieﬁng form you will receive at the conclusion of your
involvement. If you have any question about your participation in this study, ask the
investigator before you indicate your consent to voluntarily participate by Si gning below.

Consent: I have been fully informed of the above-described study and its possible risks.
I give permission for my participation in this study. I know that the investigator will be
available to answer any questions I may have. I understand that I am free to withdraw
this consent and discontinue my participation in this study at any time without penalty.

Print Name Signature

 

Address Phone

 

91

APPENDIX B

Debrieﬁng Form
Tactical Action Game (TAG)

The study in which you just participated was designed to examine the effects of guidance
and training on learning and decision-making processes. During this Study, you operated
the TAG radar Simulation. TAG simulates the complex physical performance,
information processing, and decision-makin g demands of performing fast-paced, critical
tasks. To perform the TAG simulation, you needed to learn how to operate the task and
develop strategies for effective task performance. TAG required you to gather
information about the objects on the screen, make decisions, and take actions based on
the information you gathered. We will use the information gathered during the study to
link your performance on the task to your knowledge of the task.

If you have any questions about this study or would like to receive a copy of the results
when they are complete, please notify the investigator now. If, in the future, you have
any questions about the study or would like to receive the results when they are complete,
please call the investigator listed below. Finally, thank you for participating in this study.
If you have any other questions or comments please do not hesitate from contacting the
experimenter.

Investigator: Brad Bell 353-9166

 

92

APPENDIX C

TAG TOPICS SHEET

Listed below are a number of topics that you may want to focus on during your
training. However, these are just suggested topics and you can focus on learning
anything you choose.

0 Exploring the game and getting familiar with your equipment.
0 Making correct Type decisions.

0 Making correct Class decisions.

0 Making correct Intent decisions.

0 Making correct Final Engagement Decisions.

0 Zooming-out to see the big picture.

0 Recognizing “pop-up” targets.

0 Defending your inner and outer defensive perimeters.

- Making “trade-offs” between targets approaching your inner and outer defensive
perimeters.

o Prioritization strategies.

93

APPENDIX D

TAG TOPICS SHEET

Listed below are a number of topics that you may want to focus on during your
training. However, these are just suggested topics and you can focus on learning
anything you choose.

0 Exploring the game and getting familiar with your equipment.

0 Recognizing “pop-up” targets.

0 Making “trade-offs” between targets approaching your inner and outer defensive
perimeters.

0 Making correct Intent decisions.

- Zooming-out to see the big picture.

0 Making correct Type decisions.

0 Prioritization strategies.

0 Making correct Final Engagement Decisions.

0 Defending your inner and outer defensive perimeters.

0 Making correct Class decisions.

94

APPENDD( E
Measures

INSTRUCTIONS FOR ANSWERING UESTIONS ON THE SCANTR N
Please write your last name in the area marked YOUR LAST NAME. Please write in
your ﬁrst name and middle initials in the areas marked YOUR FIRST NAME and MI.
DO NOT bubble the letters Since we will not be using your name for data, and your
responses will be kept anonymous.
Please Do ﬁll in your PID and bubble the corresmnding numbers. After ﬁlling in
your name and PID, please answer the following questions by ﬁlling in the appropriate

bubble, beginning with question number 1.

After ﬁlling in your name and subject number, please answer the following questions by
ﬁlling in the appropriate bubble, beginning with question number 1.

1. What is your sex?

(1) Male (2) Female
2. What is your age?

(1)<18 yrs (2) 18- 19 yrs (3) 20 - 21 yrs (4) 22-23 yrs (5) > 23 yrs
3. What is your overall grade point average?

(1) 0- 1.0 (2) 1.1-2.0 (3) 2.1 -3.0 (4) 3.1 -4.0 (5) >40
4. Have you ever been to this particular lab before?

(1) Yes (2) No
5. Are left or right handed?

(1) Left (2) Right
6. Do you play with video games?

(1) Never (2) Rarely (3) Sometimes (4) Frequently (5) Always

95

Self-Efﬁcacy Scale

This set of questions asks you to describe how you feel about your capabilities for
playing TAG. Please use the scale shown below to make your ratings.

 

Strongly Strongly
Disagree Disagree Neutral Agree Agree
< l 1 l l l >
(1) (2) (3) (4) (5)

7. I can meet the challenges of this simulation.
8. I am conﬁdent in my understanding of how information cues are related to decisions.
9. I can deal with decisions under ambiguous conditions.

10. I am certain that I can manage the requirements of this task.

1 l. I believe that I will fare well in this task if the workload is increased.

12. I am confident that I can cope with this Simulation if it becomes more complex.
13. I believe I can develop methods to handle changing aspects of this task.

14. I am certain I can cope with task components competing for my time.

96

Self-Satisfaction Scale

This set of questions asks you to describe how you feel about your performance on the
last TAG practice period. Please use the scale shown below to make your ratings.

 

Strongly Strongly
Disagree Disagree Neutral Agree Agree
< l l l l l >
(1) (2) (3) (4) (5)

15. I am satisﬁed with my overall performance on this task.
16. I am pleased with how I am doing.
17. My current performance satisﬁes me.

1

oo

. I am happy with my performance at this point.

19. I would be happier if I were performing better than I am now.

97

On-Task Cogpition

This set of questions asks you to describe your thoughts during the last TAG practice
period. Please use the scale shown below to make your ratings.

Never Rarely Sometimes Frequently Constantly
< ' ' 1 l l >

(1) (2) (3) (4) (5)

 

20. I paid close attention to the kind of errors I was making.

21. I focused my attention on learning a speciﬁc rule.

22. I thought about new strategies for improving my performance.

23. I thought ahead to what I would do next to improve my performance.
24. I took “mental breaks” during the task.

25. Idaydreamed while doing the task.

26. I lost interest in the task for Short periods.

27. I thought about other things that I have to do.

98

Attribution Soak

This set of questions asks you to describe how you feel about your performance on the
last TAG practice period. Please use the scale shown below to make your ratings.

 

Strongly Strongly
Disagree Disagree Neutral Agree Agree
< 1 1 1 1 1 >
(1) (2) (3) (4) (5)

28. The cause of my performance involved something I did.

29. The cause of my performance was po_t related to what I did.

30. My performance was mostly caused by the amount of effort I put into the task.
31. My performance was mostly caused by how good I was at the task.

32. My performance was mostly caused by luck.

33. My performance was mostly caused by how difficult the task was.

99

B_asic Knowledge Test

The following is a knowledge test about TAG. Please use the Scantron Sheet to answer
the following questions. Bubble in the correct leiar for each question, making sure the
question numbers match.

Check to make sure you are beginning on question number 7.

34. If a Response is Given, what is the likely Intent of the target?

a. Military
b. Hostile
c. Civilian
d. Peaceful

35. If a target’s Speed is 25 knots, its Communication Time is 85 seconds, and its
Altitude/Depth is 0 feet, what does this suggest about the target’s Type?

a. The target is a Surface Vessel
b. The target is a Submarine

c. The target is Civilian

d. The target is Military

36. A submarine may have which of the following characteristics?

. Speed 30 knots, Altitude/Depth - 20, Communication time 85 seconds.
. Speed 30 knots, Altitude/Depth 0, Communication time 30 seconds.

. Speed 20 knots, Altitude/Depth 0, Communication time 80 seconds.

. Speed 20 knots, Altitude/Depth -20, Communication time 90 seconds.

0.00”!»

37. A Maneuvering Pattern of Code Delta indicates the target is which of the following?

a. Air

b. Military

c. Surface

d. Civilian

38. A Blue Lagoon Direction of Origin indicates the target is which of the following?
a. Unknown
b. Sub

c. Civilian
(1. Military

100

39.

40.

41.

42.

43.

If a target’s Altitude/Depth is 10 feet, what is the Type of the target?

a. Air

b. Surface
c. Submarine
d. Unknown

If a target’s Intelligence is Unavailable, what Class does this suggest for the target?

a. Air

b. Civilian
c. Military
(1. Unknown

If a target’s characteristics are Communication Time = 20 seconds and Speed = 50
knots, which of the following actions should you take?

a. Choose Intent is Peaceful

b. Choose Type is Surface

c. Get another piece of information
(1. Choose Type is Air

A Communication Time of 52 seconds indicates that the target is likely:

a. Air

b. Surface
c. Submarine
d. Unknown

If a target’s characteristics are Intelligence in Private and Maneuverin g Pattern is
Code Foxtrot, which of the following actions Should you take?

a. Choose Class is Military
b. Choose Intent is Peaceful
c. Choose Class is Civilian
d. Choose Intent is Unknown

. If a target’s Maneuvering Pattern is Code Echo, this suggests that the target falls into

which category?

a. Class in unknown.
b. Class is Military
c. Class is Hostile

d. Class is Peaceful

101

45. If a target’s characteristics are Response is Inaudible, Threat Level = 3, and
Countermeasures are Jamming, which of the following actions Should you take?

a. Choose Intent is Unknown
b. Choose Intent is Peaceful
c. Choose Intent is Hostile

(1. Choose Intent is Military

46. If a target’s Speed is 40 knots, what does this suggest about the target?

a. The target is Air.

b. The target is Surface.
c. The target is Civilian
d. The target is Military.

102

Strategic Knowledge Test

47. Your Outer Defensive Perimeter is located at:

48.

49.

50.

51.

a. 64 nm

b. 128 nm
c. 256 nm
d. 512 run

If a target is outside the current radius of your console, you can view it by doing
what?

a. There is nothing you can do.
b. Wait for the target to enter
c. Zoom out

(1. Zoom in

If you’ve just noticed three targets near your inner perimeter, which of the following
should you do next?

. Engage the target nearest the inner perimeter.

. Engage the fastest target near the inner perimeter.

. Zoom-Out to check the outer perimeter.

. Zoom-In to check how close targets are to the inner perimeter.

D-OO‘QD

If three Targets are about 20 miles outside your inner defensive perimeter, which of
the following Should you do to prioritize the Targets?

a. Engage the closest Target
b. Engage the hostile Target
c. Engage the fastest Target
(1. It makes no difference in what order you engage the Targets.

If you Zoom-Out to ﬁnd three targets clustered around your Outer Perimeter, how
would you determine which target is the marker target?

3. Check to see which target is closest to the outer perimeter.
b. Check the speeds of the targets

c. Check to see which target is Civilian

(1. Check to see which target is Hostile

103

52.

53.

54.

55.

56.

Which of the following targets would be the lowest priority?

a. A target which has a Speed of 15 knots.

b. A target which has just crossed your inner defensive perimeter.
c. A target which is Peaceful.

d. A target which is Civilian.

What is the purpose of marker targets?

a. To determine which Targets are Hostile and which are Peaceful.
b. To locate your Inner Defensive Perimeter.
c. To quickly determine the speeds of targets near your perimeters.
d. To locate your Outer Defensive Perimeter.

Which of the following functions is most useful for identifying marker targets?

. Zoom-in

. Right-button feedback.
. Engage Shoot or Clear
. Zoom-out

0-0719

If three Targets are about 10 miles outside your outer defensive perimeter, which of
the following Should you do to prioritize the Targets?

a. Engage the fastest Target
b. Engage the hostile Target
c. Engage the closest Target
(1. It makes no difference in what order you engage the Targets.

On the average, approximately how many Targets pop-up during each practice trial?

999'!”
cost».—

104

57. Which of the following would be the most effective strategy for defending your outer
defensive perimeter?

58.

59.

a.

b.

Zoom-out to 128 nm, locate the Marker Targets, and check the Speed of targets
near the outer perimeter.
Zoom-out to 256 nm, locate the Marker Targets, and check the Speed of targets
near the outer perimeter.

. Zoom-out to 128 nm, locate a Hostile Air Target, and check the speed of targets

near that target.
Zoom-out to 256 nm, locate a Hostile Air Target, and check the Speed of targets
near that target.

If all penalty intrusions are cost -100 points, which would be the most effective
strategy?

a.

b.

Do not allow any Targets to enter your Inner Defensive perimeter, even if it
means allowing targets to cross your Outer Defensive perimeter.
Do not allow any Targets to enter your Outer Defensive perimeter, even if it
means allowing targets to cross your Inner Defensive perimeter.
Defend both your Inner and Outer Defensive perimeters equally.

. None of these are effective strategies.

It is important to make trade-offs between targets:

9-9 9'!”

That are Hostile and those that are Peaceful.

Approaching your Inner and Outer perimeters.

That are Civilian and those that are Military.

That have already crossed your Inner Defensive perimeter, and those that are
approaching your Outer Defensive perimeter.

105

APPENDIX F

Incentive Instructions

TAG:
TACTICAL ACTION GAME

We will award a total of eight cash prizes to the best TAG players. Two categories
of proficiency will be recognized, and four awards will be offered in each category:

Two 18. place prizes at $50 each
Two 2ml place prizes at $40 each
Two 3rd place prizes at $30 each
Two 4‘” place prizes at $20 each

Each set of four awards will be made for different types of game proﬁciency:

One pair of awards will be made for the players who understand the game the best
during the three practice sessions. You will answer several questionnaires about
the game during and after the practice sessions. You need to think carefully about
your answers to the questions. The top players who answer these questions the
best will be awarded 1", 2'”, 3rd, and 4th place prizes.

The second pair of awards will be made for the players who do the best on the
final TAG session that will take place at the end of the experiment. The top
players who do the best will be awarded 1“, 2“, 3'“, and 4th place prizes. 1 will let
you know when the ﬁnal session is about to begin.

The Prizes are independent of each other. That means its possible to be awarded prizes in

both categories of proﬁciency. Your changes of getting an award are pretty good - it’s
all up to you.

106

APPENDD( G
Example of Behavioral and Cognitive Guidance
Practice Trial Seven
High Priority Targets:

Minimum Performance — In the last practice trial, you only engaged # high priority
targets. This means that you haven’t learned how to prioritize targets. You should study
the material in your TAG manual on prioritization strategies and practice engaging high
priority targets in the next practice trial.

Medium Performance — In the last practice trial, you engaged # high priority targets.
This is good but you are still missing some of the high priority targets. You should study
the material in your TAG manual on prioritization strategies and practice engaging high
priority targets in the next practice trial.

Maximum Performance — In the last practice trial, you engaged # high priority targets.
Nice work. This shows that you have learned how to prioritize targets. You should
concentrate now on other areas in which you need to improve, but make sure you
continue to apply your prioritization strategies.

Making Trade-Offs Between Targets Approaching the Inner and Outer Perimeters:

Minimum Performance — In the last practice trial, you didn ’t check the speed of any of
the targets near your inner defensive perimeter. This means that you have not learned
how to make trade-offs between targets approaching your inner and outer defensive
perimeters. You should study the material in your TAG manual on making trade-offs
between targets and you should practice this skill in the next practice session.

Medium Performance -— In the last practice trial, you checked the speed of # targets near
your inner defensive perimeter, which is good. But, to effectively make trade-offs
between targets approaching your inner and outer defensive perimeters, you should study
the material in your TAG manual on making trade-offs and practice this skill in the next
practice section.

Maximum Performance — In the last practice trial, you checked the speed of # targets
near your inner defensive perimeter. Hopeﬁtlly you are using this information to make
trade-offs between targets approaching your inner and outer defensive perimeters. You
should concentrate now on other areas in which you need to improve your performance,
but make sure you continue to make trade-offs between targets.

107

APPENDIX G (continued)
Making Trade-Offs Between Targets Approaching the Inner and Outer Perimeters

Minimum Performance — In the last practice trial, you didn ’t check the speed of any of
the targets near you outer defensive perimeter. This means that you have not learned
how to make trade—offs between targets approaching your inner and outer defensive
perimeters. You should study the material in your TAG manual on making trade-offs
between targets and you should practice this skill in the next practice session.

Medium Performance — In the last practice trial, you checked the speed of # targets near
your outer defensive perimeter, which is good. But, to effectively make trade-offs
between targets approaching your inner and outer defensive perimeters, you should study
the material in your TAG manual on making trade-offs and practice this skill in the next
practice trial.

Maximum Performance — In the last practice trial, you checked the speed of # targets
near you outer defensive perimeter. Hopeﬁrlly you are using this information to make
trade-offs between targets approaching your inner and outer defensive perimeters. You
should concentrate now on other areas in which you need to improve your performance,
but make sure you continue to make trade-offs between targets.

108

APPENDIX H
Affective Guidance
Practice Trial One

The TAG simulation is difficult and complex. It takes time to become good at the TAG
simulation and there is a lot to learn. Sometimes people become frustrated or
discouraged if they are not performing as well as they would like. If you become
frustrated while performing the task, sit back, breathe in and then out, and say to
yourself; “I am calm. ” Do this whenever you feel yourself getting frustrated, it will help
you relax.

Practice Trial Two

You are still in the beginning of your training and it takes time to learn all the necessary
TAG skills and strategies. Sometimes people become worried if they are not performing
as well as they would like. If you feel yourself getting worried, sit back, relax, and
imagine yourself performing extremely well during the TAG simulation. This will help
you to relax and will build your conﬁdence.

Practice Trial Three

You just ﬁnished the ﬁrst block of practice trials. Good work. You should look back on
these practice trails and say to yourself, “I have learned a great deal in a short period of
time. This task is difficult and I have made some mistakes, but it is alright to make
mistakes because I am learning.” Keep up the good work and remember that you can
perform well if you want, it ’s all up to you.

Practice Trial Four

You are now in the second block of practice trials and should be beginning to see some
improvement in your performance. Since the task is difﬁcult and complex, you may have
a tendency to get frustrated or anxious if you are not performing as well and you would
like. If this happens, imagine yourself being more relaxed, less tense, but with a
moderate level of anxiety and a great deal of motivation to do well on the task.

Practice Trial Five

Although the TAG skills are diﬂicult and complex, you have every opportunity to master
the simulation. You have come this far, and you are doing a great job. If you feel
yourself becoming anxious or are worried about your performance, it may be helpful to
sit back and concentrate on feeling conﬁdent and focused. Do whatever you need to do
to relax your mind. You may want to close your eyes and breathe in and out or count
backwards from 20. This will help you relax and prepare you for the next practice trial.

109

Practice Trial Six

You have just ﬁnished the second block of practice trials. Good work. Just think about
how much you have learned in such a short period of time. There are three practice
trials remaining in your training. You should prepare yourself for these trials. You may
want to say to yourself, “I can do well at this task. If I am not performing as well as I
would like, it is not terrible. I have learned a great deal and I am going to maintain a
positive outlook.

Practice Trial Seven

Over the last seven practice trials you have been asked to learn a great deal and handle a
lot of information. You may begin to feel mentally tired and fatigued. If this happens,
take a deep breath, relax, and clear your mind. Then do whatever it takes to charge
yourself up and return to the simulation. For example, you may imagine yourself
performing the task in an actual battle situation. This will help you focus your attention
on the task at hand.

Practice Trial Eight

You now have only one practice trial remaining. Since you have been training for a
while now, it may be easy to let yourself get distracted from the task. If you feel yourself
getting distracted or feel like giving up, say to yourself “I have come this far. It makes
no sense to give-up now. I am going to stay focused on the simulation and concentrate
on becoming the best TAG player I can. "

Practice Trial Nine

You are now ﬁnished with the practice trials. Great work. Soon you will be asked to
show how much you have learned the ﬁnal and more challenging TAG trial. If you feel
tired or fatigued, it may be helpful for you to emotionally charge-up before the ﬁnal trial.
In essence, psych yourself up for maximum performance. In addition, if you get
frustrated or anxious during the ﬁnal trial, remember all the techniques you have learned
to handle these emotions.

110

APPENDIX I

Example of Feedback

For this practice period:
Number of targets engaged: #
Number of targets engaged correctly: #
Number of targets engaged incorrectly: #
Your total score: ###
Targets that popped-up: 6
Number of pop-up targets you engaged: #

Percentage of pop-up targets you engaged: #%

Press any key to continue . . .

lll

APPENDD( J

Practice and Study Sequence and Content Measure

In the blanks provided below, please list the three main concepts, skills, or strategies
you plan to PRACTICE during the upcoming three practice trials. If you cannot
think of three items or are unsure of what you are going to practice, you may leave one or
more of the lines blank.

 

 

 

112

 

escape

3350.:em

352:0 o>£=woo£au£>mnom

$353.23 be £32 353:
cam—no.6 53 352% O>E=woo
was 3.5323 9:332
moo—HEP ”3“ £352.»:

 

683.3%

903.30"— wE3£>om
e5 macaw Edam 25.:
term

cozoﬂmzawmﬁﬁom

.o—mﬂ
05 co tote mo £32 35w... 2
v3. :5 :ocoﬂmcamﬂnéom .3
£26— Eocwmm ”amt manioaxm

 

escape

cocosaeeaeeeom

352:0 O>E=woo/_Eo_>a;om

dozoﬂmsammﬁ

$3. .333: 333.:

:3» 3.33am 32.—woo

Ea Eo_>a:om an £353»:

 

vetoamam

cocoa—3m beam

352:0 o>E=woo/_Eo_>anom

.3533 35% “323%
on. Bo=£ 2 b8:— EOE

on EB 352:» o>£cwoo
was $8323 9:332
3058p. ”at £35093.—

 

votomnsm

oocoscom 3535

352:0 o>£:woo/_So_>mcem

.3:an

.3 3:33» Baruch—am
05 25:8 2 box: 20E
on EB 352% 3253
can 12222.3 9:332
3058p. n; £353»:

 

 

=33—

 

??Zatasw .5653:

 

Amvo_aata> Eco-.036:—

 

nausea»:

 

 

8.33— was 3352;:

M EQZmnE/x

113

 

360me .m> 382::

.38 05
so 220m 8 300—: 0.88 03 EB
32:53.53 «0 302 3:32 :3

 

 

 

 

 

 

 

 

 

 

 

502396 =23:on 38-:0 323282 30028.3. no 303 53B 30023; 5% 3050me
6308223 38 330%

2 :32 EB 330350.203 20

3:25am =22:on 38-:0 30350.23 232 20:33 3% 3053wa
.38

20350023 w2B2>0m 05 :0 283 205 :28 En EB

23 $2355 30% 02.5 A3E0§m .m> 38025 323283 ho 3:02 3522 .3

30200.n0m totm 323582 33:33. «.0 3:00..— 53B 30:32.3. ”awn 30533.3
3.223283 30 302

3.282 208 3 >835 EB

A3E0§m .m> 35025 332% 03800393 w2332

c0800H0m 323282 30023:. .6 3003 332:0 0330b< 30:32.3. ”amt 30525.3
.B3553B

30353.3 w2B2>0m 05B 035 .35 28.30 205

23 $2325 Sam 025 >895 EB 38 05 co 220m

52.5935 totm :22:moU 38-:0 02B 30283. but 3052;:
.38 05

:82 B355? 8 32.: 30— 3

EB 3:32am 0330.33 0332

503233 =22:on 3820 332:0 03303.2 onB 300:3; 35 305223
$0350.28

30 230— 20.32 3325 2:03

3.32% 0380033 w2332

50800.n0m 50350.23 3.32:0 038022 30:32.“. ”on 3052;:
0.035000m w2B2>0M .38 05 co toto

23 m2~€8m Ham 083.5 0.55 8 22 EB 3350.203.

503233 cotm 3350.23 go 230. .035 ”wt 305223

 

 

114

 

5...... 5050:8250

030.305.
0.w05:m :5 0.05m. :5

00:08.. w....:.: 0055.88“.

0&2305. 0.855 :5

0.5.. 0.0.: :5 00:08.. mats:
0055.802. 0.855 :5 0.5..
.0025: .:.3 85.280 2.

.23 5:... 852.8050 2: :0

 

 

 

 

 

 

 

 

8225.6 2: E 005800.00. 030505 :5 0.25 00558.03. H. I. 0.02:8»:
.005:...ot0m

0:02.00 :5 00:2..8m 8:085. 0.5. 0.855 0.2: 00:03:...

:5 32m. ﬂatm. .0 005>0.0~. =3. tot0 0.0.: h.0 005>0.0.. 0.:

:5 3.03:8". w=.30.>0~. :5 :5 0.5: 0.: :0 2.09.0 0025:

8:895 3.0.95. 005:...ot0n. 0.m05..m w:.>:2m 2.8m 2...... totm. tot0 2. H ”:0; 0.02:8»:
0055.802.

3:02.00 :5 00=0=v0m 00:55 0.5: 0.5.. :2: 00:03:...

:5 .389 totm. .0 0058.0”. :5 tot0 0.2: .0 005>0.0.. 2:

:5 3.03:8... 9.5033. :5 :5 0.5. 2: :0 9.2.0.0 0.00.5:

:200.~0~. 005.5800; 0.5m weigﬁ E25 08.... team «5:0 2. H “0...... $0052.»:
080.305.

3:02.00 :5 00:0:c0m 00:55 0.5. 0.855 :2: 00:02....

33m. term. .0 0058.3. :3. 9.0....0 0.2: .0 005>0.0. 2:

. . - 203:8". w:.30.>0~. :5 :5 0.5: 2: :0 2.09.0 0.02.5:

:0tommsm 3....th 030.305. 0.05... 0.85.5 w:.>:3m 20mm 0.5.5 totm €0.00 2. .. ”no; 0.00:8»...
030.305.

35:50 :5 00:03.00m 00:55 05. 0.5.. :2: 00:02:...

:5 52m. totm. .o 005>0.0~. =3. tot0 0.0.: .0 0058.0. 2:

:5 203:8”. w=.30.>0~. :5 :5 0.5: 0.: :0 2.2.0.0 0025:

8:02.05 35:03 080.305. 0.5... 0.5m. w:.x:3m E2.m 0E..C totm. tot0 2. H 50.: $0052.»...

 

 

115

005:...0 8...:on :5 5.0.32.3. n 85:...0 onE0m .m:=000m :5 8.00.8

:. 05 00....H ..:0.5.-N. .0>0. .cd 5 .5055... m. 00.5.2000 i. ..:0..5.-m. .0>0. mod 0.: 5 .5055... m. =0.5.0...0U * .02.:

 

00.800. .0 :00 0.: 5 :0252: 53 0.5.5., 0.: 8.000: .m. .000 00.3-00- .0 :..0 0.: 5 :22508 53 0.5.5> 2: 5.: 00.000: 2. ”0.02

 

 

:5. NS. 08.- 3... :o. :80 5.. .3... E... .2 ....08_...o-.._om .o.
22.... :o. 8... 08.- m 5.- 1%.. ~3- 3... 2.... mm... 5.8.30.3...2 a
- .35.... 8... 3.8.. :02. *9... imam. 08. a... mom ...=o..8.w..a-..om .w
- .02. 1.0:. .32. .8- 15... .02. .o”. 3. $8835 .25 s
-- m8. *3. 5...- 1.3.. a... mu... m...- ...8§..om 520 .0
- 18m. S... 3.20 5...- om... m... €85..an 8.68.. .m
-- 0w... :8... 8. .- 8.. 85 . . .8550 8.02.. .-
-- 8o. 8... 8.- MSN 0...?- .m
-- ox..- -- -- 852:0 0002.00 .N
i l -- 02.—«EDD o>500t< .—
w n 0 m .- m N . 00 .2 22...;

 

 

0035:... :. :0:...0:. m0.05..5> .0. 80:20:00.2... :5 500.5309 :5:5.m £50.).

-. X.QZm.a.n.<

. 2%...

116

030.305. 0.05m u .305. 0.93 0.50:8”.
00 .025 0.0.... .30... u 00....- .:00... 00... .w0.>::.m .00am 0E...- .30... u 00....- >::.m .0... 00...:on .0200 u .on 0.08-00 ”0.07.

 

 

 

35mm. coo. imam. “.90.. 3.90.. twan- *mm.. o3. OWN am: 23505. 0.5m .ON
03.- one. m3: NNov m 5.. *mm.. .33 . .- 3o. mmuo :0”. .3000...- .:00.... go..- .0.
.8. one. *mn..- .mo. v.0. go. 1.5..- 03.. ovuc ovnw CV08...- .:00n. 40.. .w.
Z..- .Ihmm. .52. 0:. So.- 303 .- 8.. *oom. .m. cm”: .808... >:5m .0..- .0.
m3. coo. 10$. mmo. over *0. .. amo. wmo. NYC 9. ”m 2.00.... ~€05 .00... .0.
.1. .NN. mmor wmor wmor mmor .Imom. mmor mmo. Rd mwd Amvm00..:n.....< .n.
13%. who. o. .. *0. .. awo. thaw. iwm .. woo. vwd wed 2500.30.32 .v.
08. wmo. 08. *OM .. awe. immw. moor .8. 00.0 wad ANonU 0.5.00 .m.
3.0m .. 08. mm... vac. CS. .550.- mvo. emo. mmd o9.V Svaou 0.0300 .N.
.102. .o. .- Co. omor .vor **ow.. o. ..- 9o. mud £6 Amvxuaoc.0...0m .. .
w h o w v m N . Om S. 0.00.5>

 

 

 

000.5040. 0. :0:=.00. 00.0d.5> .0. 0025.00.50.20. :04... 500.5309 :5:5.m 0.50.).

2.8. . 2.5

117

005008.00 0.w05..m u ...0n. 0.w05..m 005008.05.

 

0.5m u ...0n. 0.05m. .0w:0.300v. 0.w05..m u .305. 0.w05..m 00.300 00.50.....000w w0..0: 3.00000. 0.00.0; 0.0.000: a. “0.02

 

 

igu- wvm. *hN. .- m5. mbo. ._.....wN. ......mom. moor w0.~ oo.o ADV-ton. 0.w05..m .om
......w.m. vmo. mNo. .mo. ovo. ......omm. §.- ...m~.. MN... oo.o 20.000. 0.95 .wm
3.0m. iomm. v0or 3.0mm. *3“.- ...._.oom. ...._.ovm. w0o.- .mN oo.o 2......0a. 0.w05..m Hm

mwo. mo. .- 1.00. .- wmo. ooo. Law. 05.. w.o..- ow. oo.o 2.....0n. 0.w05..m .0N
5.00.. vmor who. omor moo.- 3.0mm. mwor 1.2. mm... oo.o Amvﬁon. 0.5m. .mm
1.80. *mN.. .32. n. .. who. .23.-N. *...0oN. mNo. NE. oo.o 2.....0n. 0.05m. .VN
imam. ......omm. 0No. 3.00m. mo.. .1. .3”. .253. 0.o.- and mod 23.305. 0&05..m .mm

o... ....- $.08.- who. vmo. 3.5. omor mo. .- ZEN N..0 2:305. 0.w05..m .NN
......o.m. 0.o. owo. coo. woo. 3.0. m. 0.o. oo.- . ..m vwo. €3.55. 0.5m. ..N

w 0 0 w v m N . Om .2 0.00.5>

 

 

 

0091.3... 0. :0:=.00. w0.0m.5> .0. 000.015.008.20. :5J000.5.>0Q :5:5.w .50.).

2:8. . 2.5

118

 

 

000.- .00..- :30. 1.0.0. 1.000. 1.0.0. 1.00.. .00.. 1.0.0. 00.. 5.305. 2000.00
.2000. E... 000.- .00.. 0.0. 3.000. .00.. .10.. 000.- .00.. avg-0.0800000.
- 000. 00..- 000. .00. 3.. 00.. 000. 000.- 000. €2.00 .08... .80 .0.
- .00.. .00.- 000.- 000. ~00. 000.. 1.00..- 000.- .325 0020 00.0 .0.
-- 00.. 0. .. $00.. .00. 0.0.. 000. 10.0.. 2.050102000020.
-- $000. :00... 1.000. 1.000. 1.0.00. 1:... 50:00:03.... .0.
-- 1.0.0. 1.0.0. 5.000. 1.000. .00.. 50803.5... .3
-- 3.000. 3.000. 1.000. 3.000. @000 V.0000 .0.
- :03. 1.9.0. 000. 5.000 00.2.00 .0.
-- :00... 1.000. €580.33 .. .
-- 1:0. 0:82-09:00 .0.
0. 0. 0. 0. 3 0. m. .. 0. 0 200...;

 

 

000bg< 0. 00.020... 00.00..0> .0. 000.00.250.00... .05-0 00050309 0.0.0006 .0000:

:08. _ 200.0

119

 

 

.00.- 00. .- 3.0. 1.0NN. 1.00 .. 1.0..” 1.3m. .....N.. 1.00m. .00.. AD....0A. 0.w0.0..0 .mm
000. 000.- 1.00.. 1.09.. ......Nm~. 1.0.00. 1.00m. 1.1.000. 1.0.0. 1.000. $0.50. 0.00m. .wm
000.- 1.00. .- 000. 1.0NN. 1.00.. 1.00m. 1.00m. 1.00.. 1.000. 1.. .N. €5.00. 030.000 SN
000. ...N0..- 000. ...00.. .00. 000. 1.3.. 1.00.. 000. ......N0.. 2.....00. 0.00.3.0 .0m
.00.. 1.00..- 1.00.. 1. . 0.». 1.00.. 1.000. ... . 0.. 1.0.0. 1.0.0. 1.000. ANY-.80. 0.00m. .0N
000. 30... 1.0.... 1.0%. 1.0Nv. 1.0-0... 1&3. 1.00.. 1.000. as. 2....00. 0.00m. .vm
.00. .00. 1.00.. 1.0.N. 1.00m. 1.9%. 1.00.. 000. 1.00m. 1...... @0305. 0.w0.0..0 .mm
00.. .0..- 000. 1.008 ......N.N. 1.00.. 1.00.. 1.0.... 1.00.. 1.00.. 2.2.05. 0.3.8.0 .NN
0.0.- wmor 1.00.. 1.030. 1. .2. 1.000. 1...... 1.00m. 1.0mm. 1.00m. 30.305. 0.00m. ..N
w. h. 0. 0. v. m. N. .. 0. 0 0.00..0>

 

 

0000.001. 0. 0000.00. 00.00.05 .0. 000.000.80.20. .000- .000..0.>0n. 0.00080 0000.).

0.000. . 0.00..-

120

 

 

-- 3.000. .10.. 1.00. .18.. .300. ..0. 0. 3.0.0. ...00. 1.000. 0.0. 6...... 0.02.00 .00
-- .300. ..000. 200.. 103.. ...00. .000. ..000. £000. 000. 60...... 0.000 .00
-- .000. ..000. .000. :00... $000. ..000. .000. 000. .0005. 0.02.00 .00
-- ..0.. 0.0. ..00.. .000. .0... 000. 000. 0.0.0.. 0.02000 .00
-- 103.. ..00.. .0.... 3000. 30.0. .00. .00.... 0.0.0 .00
- i... 0. 1.0:. .000. :03. 0.0. 00...... 0.0.0 .0
-- ...00. ..0.0. 3.000. 00.. 00.305. 0.02.00 .00
-- ..000. ..00. 000. 00.305. 0.02.00 .00
-- :00... 000. .0035. 0.0.0 ..0
-- .00.- ....305. 0.0.0 .00
-- .0020; 08,-. .50 .0.
00 00 00 00 00 .0 00 00 .0 00 0. 0.0.0..>

 

 

0000.00... 0. 000200. 00.0.0.0.» .0. 000..0.0..00.0.0. 00m .000..0.1>0n. 0.00080 0000.).

.008. _ 0.00..-

121

Table 2

RM-MANCOVA for Affective Guidimce and Behavioral and Cognitive Guidance on

Dependent Measures During Training

 

 

Effect Df F

Covariate

Ability 12, 261 9.25**
Independent Variables

Affective Guidance (AG) 12, 261 2.33**

Beh.\Cog. Guidance (BCG) 12, 261 18.98**

Time 12, 261 15.81**
Interactions

AG X BCG 12, 261 0.51

Time X AG 12, 26] 1.45

Time X BCG 12, 261 8.93**

Time X AG X BCG 12, 261 1.29

 

.298

.097

.466

.421

.023

.062

.291

.056

 

*p<.05. **p<.01.

122

Table 3

Between-subjects Effects from RM-MANCOVA for Affective Guidance

 

 

Effect Df F 1]
Practice Sequence 1, 272 3.90* .014
Study Sequence 1, 272 10.57“ .037
Self-satisfaction 1, 272 0.70 .003
Self-efﬁcacy 1, 272 0.17 .001
On-task Cognition 1, 272 0.00 .000
Attributions 1, 272 0.45 .002
Total Study Time 1, 272 14.97** .052
Total Feedback Time 1, 272 0.41 .002
Basic Knowledge 1, 272 1.07 .004
Strategic Knowledge 1, 272 2.64 .010
Basic Performance 1, 272 1.54 .006
Strategic Performance 1, 272 1.58 .006

 

*p<.05. **p<.01.

123

Table 4

Between-subjects Effects from RM-MANCOVA for Behavioraiand Cognitive Guidaﬁ

 

 

Effect df F 11
Practice Sequence 1, 272 131 .35** .326
Study Sequence 1, 272 79.87** .227
Self-satisfaction 1, 272 8.79** .031
Self-efficacy 1, 272 0.00 .000
On—task Cognition 1, 272 0.12 .000
Attributions 1, 272 1.10 .004
Total Study Time 1, 272 340" .012
Total Feedback Time 1, 272 8.77** .031
Basic Knowledge 1, 272 3.79». .014
Strategic Knowledge 1, 272 9.90** .035
Basic Performance 1, 272 1.57 .006
Strategic Performance 1, 272 15.76** .055

 

+p<.10. *p<.05. **p<.01.

Table 5
Summary of Hierarchical Regression Analysis for Variables Predicting On-task

Cognition at Time 1 (N = 276)

 

 

Variable ﬂ 9&2
Step 1 .028**
Cognitive Ability .072
Step 2 .113**
Self-efficacy( 1) .284**
Attributions( 1) . 130*

 

Note: [3 is the standardized regression coefﬁcient from the full regression equation with
all predictor variables. Increments for variables entered at the AR2 signiﬁcance levels are
based on F tests for that step. * p < .05. ** p < .01.

125

Table 6

Summm of Hierarchical Regression Analysis for Variables Predicting On-task

Cognition at Time 2 (N = 276)

 

 

Variable 1: £2
Step 1 .057**
Cognitive Ability .l 18*
Step 2 .199**
Self-efficacy(2) .209*
Attributions(2) .318**

 

Note: [3 is the standardized regression coefﬁcient from the full regression equation with
all predictor variables. Increments for variables entered at the AR2 signiﬁcance levels are
based on F tests for that step. * p < .05. ** p < .01.

126

Table 7

Summary of Hierarchical Regression Analysis for Variables Predicting Total Time Spgnt

 

Studying (N = 276)

 

 

Variable [5 E
Step 1 .025**
Cognitive Ability -.174**
Step 2 .033f
Self-satisfaction(2) -.010
Self-efficacy(2) -.096
On-task Cognition(2) .199**
Attributions(2) -.052

 

Note: B is the standardized regression coefﬁcient from the full regression equation with
all predictor variables. Increments for variables entered at the AR2 signiﬁcance levels are

based on F tests for that step. * p < .05. ** p < .01. f p < .10.

127

Table 8
Summary of Hierarchical Regression Analysis for Variables Predicting Time Spent

Reviewing Feedbacjgﬂ = 276)

 

 

Variable 15 ARE
Step 1 .015*
Cognitive Ability .062
Step 2 .049**
Self-satisfaction(2) .061
Self-efﬁcacy(2) -.022
On-task Cognition(2) .198**
Attributions(2) .032

 

Note: [3 is the standardized regression coefﬁcient from the full regression equation with
all predictor variables. Increments for variables entered at the AR2 signiﬁcance levels are

based on F tests for that step. * p < .05. ** p < .01.

128

Table 9

Summg of Hierarchical Regression Analysis for Variables Predicting Basic Knowledge

at Time 1 (N = 276)

 

 

Variable B_ Ag
Step 1 .155**
Cognitive Ability .374**
Step 2 .067**
Practice Sequence( 1) .109*
Study Sequence( 1) .1 11
Total Study Time(l) -.127
Total Feedback Time( 1) -.064

 

Note: B is the standardized regression coefﬁcient from the full regression equation with

all predictor variables. Increments for variables entered at the AR2 signiﬁcance levels are

based on F tests for that step. * p = .05. ** p < .01.

129

Table 10

Summary of Hierarchical Regression Analysis for Variables Predicting Strategic

 

Knowledge at Time 2 (N = 276)

 

 

Variable b E
Step l .116**
Cognitive Ability .323**
Step 2 .193**
Practice Sequence(2) .264**
Study Sequence(2) .259**
Total Study Time(2) -.057
Total Feedback Time(2) .061

 

Note: B is the standardized regression coefﬁcient from the full regression equation with
all predictor variables. Increments for variables entered at the AR2 signiﬁcance levels are

based on F tests for that step. *p < .05. ** p < .01.

130

Table 11
Summag of Hierarchical Regression Analysis for Vaﬂbles Predicting Basic

Performance at Time 1 (N = 276)

 

 

Variable ﬂ _A_R13
Step 1 .074**
Cognitive Ability .264**
Step 2 .168
Practice Sequence( 1) .047
Study Sequence( 1) .097
Total Study Time( 1) .054
Total Feedback Time( 1) .017

 

Note: B is the standardized regression coefﬁcient from the full regression equation with
all predictor variables. Increments for variables entered at the AR2 signiﬁcance levels are

based on F tests for that step. *p < .05. ** p < .01.

131

Table 12
Summaryof Hierﬁarchical Regression Analysis for Vﬂ'ables Predicting Strategic

Performance at Time 2 (N = 276)

 

 

Variable a A_R_2
Step 1 .095**
Cognitive Ability .258**
Step 2 .155**
Practice Sequence(2) .268**
Study Sequence(2) .186**
Total Study Time(2) -.213**
Total Feedback Time(2) .093

 

Note: B is the standardized regression coefﬁcient from the full regression equation with
all predictor variables. Increments for variables entered at the AR2 signiﬁcance levels are

based on F tests for that step. *p < .05. ** p < .01.

132

Table 13

Summary of Hierarchical Regression Analysis for Process Variables Predicting Basic

Knowledge at Time 1 (N = 276)

 

 

Variable B ABE
Step 1 .155**
Cognitive Ability .303**
Step 2 .085**
Self-satisfaction( 1) .076
Self-efﬁcacy( 1) . 159*
On-task Cognition(l) .004
Attributions( l) . 156*

 

Note: B is the standardized regression coefﬁcient from the full regression equation with

all predictor variables. Increments for variables entered at the AR2 signiﬁcance levels are

based on F tests for that step. *p < .05. ** p < .01.

133

Table 14

Summary of Hierarchical Regression Analysis for Process Variables Predicting Strategic

Knowledge at Time 2 (N = 276)

 

 

Variable I; _A_R_2
Step 1 .116**
Cognitive Ability .288**
Step 2 .051 **
Self-satisfaction(2) .120
Self-efﬁcacy(2) -.191*
On-task Cognition(2) .166*
Attributions(2) .107

 

Note: B is the standardized regression coefﬁcient from the full regression equation with

all predictor variables. Increments for variables entered at the AR2 signiﬁcance levels are

based on F tests for that step. *p < .05. ** p < .01.

134

Table 15
Summary of Hierarchical Regression Analysis for Process Variables Predicting Basic

Pedomce at Time 1 (N = 276)

 

 

Variable ﬂ .AE
Step 1 .074**
Cognitive Ability .126*
Step 2 .286**
Self-satisfaction(1) .354* *
Self-efficacy( 1) .106
On-task Cognition( 1) .067
Attributions( 1) .212**

 

Note: B is the standardized regression coefﬁcient from the full regression equation with
all predictor variables. Increments for variables entered at the AR2 signiﬁcance levels are

based on F tests for that step. *p < .05. ** p < .01.

135

Table 16
Summary of Hierarchical Regression Analysis for Process Variables Predicting Strategic

Performance at Time 2 (N = 276)

 

 

Variable 3 .433
Step 1 .095**
Cognitive Ability .243**
Step 2 .056**
Self-satisfaction(2) . 142*
Self-efficacy(2) -.065
On-task Cognition(2) .l71**
Attributions(2) .052

 

Note: B is the standardized regression coefﬁcient from the full regression equation with
all predictor variables. Increments for variables entered at the AR2 signiﬁcance levels are

based on F tests for that step. *p < .05. ** p < .01.

136

Table 17

Summal of Hierarchical Regession Arab/sis for Practice Variables Predicting Basic

Performance During Generalization (N = 276)

 

 

Variable B AR:
Step 1 .152**
Cognitive Ability .097*
Step 2 .445**
Basic Knowledge(2) .193**
Strategic Knowledge(2) -.006
Basic Performance(2) .552**
Strategic Performance(2) .153**

 

Note: B is the standardized regression coefﬁcient from the full regression equation with
all predictor variables. Increments for variables entered at the AR2 signiﬁcance levels are

based on F tests for that step. *p < .05. ** p < .01.

137

Table 18

Summary of Hierarchical Regression Analysis for Practice Variables Predicting Strategic

 

Performance During Generalization (N = 276)

 

 

Variable B £2
Step 1 .079**
Cognitive Ability -.001
Step 2 .520**
Basic Knowledge(2) .028
Strategic Knowledge(2) .200**
Basic Performance(2) .016
Strategic Performance(2) .648**

 

Note: B is the standardized regression coefﬁcient from the full regression equation with
all predictor variables. Increments for variables entered at the AR2 signiﬁcance levels are

based on F tests for that step. *p < .05. ** p < .01.

138

Table 19

MANCOVA for Affective Guidance and Behavioral and Cogaitive Guidance on Basic

and Stgitegic Performance During Generalization

 

Effect df F n2

 

Covariate

Ability 2, 271 29.00** .176

Independent Variables

Affective Guidance (AG) 2, 271 1.80 .013

Beh.\Cog. Guidance (BCG) 2, 271 16.35** .108
Interactions

AG X BCG 2, 271 0.73 .005

 

*p<.05. **p<.01.

139

Table 20
Between-subjects Effects from Generalization Trial MANCOVA for Behavioral and

Cognitive Guidance

 

 

Effect df F n2
Basic Performance 1, 272 0.00 .000
Strategic Performance 1, 272 29.86** .099

 

*p<.05. **p<.01.

140

 

 

 

 

 

 

 

 

 

:=.

_l

 

 

 

 

 

 

$23.5.» 2.8.5

.38.: =Eo>O gum

 

 

 

 

 

 

 

 

 

 

 

 

 

r _

835.5, tes— ._e got»—

_>_ VA—QZmaEaﬂ

 

 

 

 

 

 

 

 

 

 

 

 

835...; 808...—

 

_

 

Fans—aw.” v
3.8%

 

 

 

 

8.3.3
2:35

@253.

 

 

 

_
azure, .5338...

_

141

 

35% use 85an 98:8: Co oozes—ova 9393534 .m occur..—

 

aaxm 9339.5 A

mag—m ammun—

 

 

 

 

couoEtom 2 857.5 .
85:0 3on .

mto- owed. .
,8th 3on .
sews ESE E: .

ﬂowed. wcictorm .

 

 

 

82:. coamom

303$. Era—om.
meouoEtom
850 can 8:5 $.65th .
ﬂowar—l coo—SE .
couoEcom EEO wccmoog .

So-wEEooN .

 

 

255600
EoEowawcm 35m .
E25.
$20.
on»?
8:23 26 .

Bowman. was—com .

 

 

 

oar—r :Emmom

0:0 cosmom

142

  

.moﬁom ego—305— Bwoﬁbm can 2me Co 832:3 583 2: 32 2 com: .25.: 5mm .m 95w?—

mmthsocv. 2mmm

  

\(T/i/r

P r
,er 0er arm: mun: We bfmb owe .vmv Mme

@900 30:! 2mm mbm
an- E aﬁ- E
P

P

F
arms

.@

143

.oocosvom 8:an :o 852% o>Ecwoo/_Eo_>m:on Co 80am .v .2me

0305 wEEEH

N 08E. ﬂ 2:5.

 

O

 

 

V?
o

 

 

 

 

 

8:830 9m 02 I-
8:330 van.

 

 

 

 

 

 

 

 

 

 

 

I

 

M

paonomd
sotdol Bututml IIIBAQIQH

144

 

 

853.0 05 OZ I

 

8:880 06 D

 

.8538 3:3 so 852% 8258530328 .8 Hook—m .m EsmE

_.

82m maﬁa;

N 2:5.

lll
_.I

 

 

 

 

ﬂ oar—r

 

-: 85

.. Q;

., 43

.- cmnm

 

was.

 

., cone

1ua1uo:)
omndorddv But/(pmg mods will

145

 

 

8:85 9m 02 I

 

 

detoﬂmuamuzom so 852% o>E=moo¢82>asun be 885 .o .8me

#85 mag

NQEH #9:;

 

 

 

 

 

 

 

 

 

 

IIODOBJSQBS-JIQS

146

.zomocmoiom 85 852% o>E=woo/_Eo_>mnon c8258 98:23.3. 3

:8: mass;

NQEH #8:?

 

O

 

l
v—t

 

 

 

(\l

Wows-Hes

8:85 gm oz I
885.0 U/m D

 

 

M

 

 

 

 

 

 

q-

 

147

.wEbu—zm 50% 0E: .88 :o cost—woo ﬁﬂéo mo 80th .w PSwE

SE8 8

 

 

 

 

 

 

 

 

 

 

8 955% 8 E g 8
CC V 6’5 N 1— O
911111. ﬁlms 19101,

148

 

 

 

9:82 02 I

0888 9:8? n.

 

@883

 

#
W

.85 82m :38 :o 852% 8:88“ :0 Subm— d oSmE

82: mag;

NmEF

 

 

 

P

 

PmEF

1 85
8|! «3

IT EN

 

 

0:0

149

Bug/(pms weds 911111

98% 29:20:88 >283 82: 2 88258:
:08 mm; 2me ”802 638—305. £me 85“ 8:258 85an :838 Enmcozm—om d— Emmi

8888: 8th wEESH “883%

N F o

 

 

'0.
co

 

\

 

 

‘9.
on

 

 

O
,—

eﬁpemouy 318123

150

.8088 88.2588 bane—o EOE 9 822.58.
:08 8; 28m “802 682305— 3883 8am 8:268 8:an c858 Ecmcoza—um .: 8

 

8888: 8:55 maﬁa; 8:663:

m, N r o

 

p.
~

 

co

 

O)

 

 

O
‘—

 

 

F
F

aﬁpamoux 3189112113

151

98:0 885888 3:83 29: 2 88:52
:83 8: 28m 8qu 6058888 2838 8am 8:088 888: 8858 :Emcocﬂom N— 2me

88:5 8&0 H mg; 8:829:

m N P o

 

 

 

 

 

 

 

 

goueuuoped 3189112115

152

 

 

n
a u
- n
a - an
ann_-n n n
umn- on a
E: and:I ﬁnial: n
Mug" D D
mamm‘auu an
. . u u

 

V N O

I‘D

u

yo

 

(D

(anso dtuoo pozipmpums) uormz nmauag aoueunopad 31823

153

Basic P erformance T rial 9 (standardized composite)
Figure 13. Relationship between basic performance during practice and basic performance during generalization.

.cosaﬁzuocow wctsc oocwscotom 3888 was 858.5 wctsc 8.8888: £885 c838: :Emcosﬂom .3 Emmi

8:89:80 88888.:88 a :8 P 85:28.8: oﬁoumhm

m 0 ¢ N N- V- m-
.- 8 -

 

 

 

0..

(oirso dtuoo poztpmpums) uotiez “mango aoueuuopod 31391ng

154

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

155

Figgre 15. Path diagram used to test hypothesized model.

 

9 Cl 5 i
l ‘ l
: Bum Smpc
Performance Gene

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

\\ ‘ T
\ ability I
\
\ .m. e! l /
‘ l I
e9 . Y
S
Buic mm; 2
1 Performance 2 '
‘ Basic 1 Strange
Knowledge 2 Performance 2
@I o =5 .3
i 1 I I ' I
. Oank
Total Study , Cognition
Time 2 I .

 

 

 

 

 

 

 

 

 

 

 

I i
? Af‘CCDVC Bch & Cog '
! Guidance

Guidance i
\

 

Figure 16. Path diagram used to test revised rhodel.

156

REFERENCES

Alpert, D., & Bitzer, D. L. (1970). Advances in computer-based education.
Sciencgé67, 1582-1590.

Atkinson, J. W. (1964). An introduction to motivation. Princeton, NJ: Van
Nostrand.

Avner, A., Moore, C., & Smith, S. (1980). Active external control: A basis for
superiority of CBI. Journal of Computer-Based Instruction, 6, 115-118.

Balson, P. M., Manning, D. T., Ebner, D. G., & Brooks, F. R. (1984/1985).
Instructor-controlled versus student-controlled training in a videodisc-based paramedical
program. Journlal of Educational Technology Systems, 13, 123-130.

Bandura, A. (1977). Self-efﬁcacy: Toward a unifying theory of behavioral
change. Psychological Review, 84, 191-215.

Bandura, A. (1982). The self and mechanisms of agency. In J. Suls (Ed),
Psychological perspectives on the self (Vol. 1). Hillsdale, NJ: Erlbaum.

Bandura, A. (1986). Social foundations of thought and action: A social cognitive
theory. Englewood Cliffs, NJ: Prentice-Hall.

Bandura, A. (1988). Reﬂections on nonability determinants of competence. In J.
Kolligan, Jr. & R. J. Stemberg (Eds), Commtence considered: Perceptions of
competence and incompetence across the lifespa_n_. New Haven, CT: Yale University
Press.

Bandura, A. (1991). Social cognitive theory of self-regulation. Organizational
Behavior and hump Decision Processes. 50, 248-287.

Bandura, A., & Cervone, D. (1983). Self-evaluative and self-efﬁcacy
mechanisms governing the motivational effects of goal systems. Journal of Personality

and Social Psychology, 45,1017-1028.

Bandura, A., & Cervone, D. (1986a). Differential engagement of self-reactive
inﬂuences in cognitive motivation. Organizational Behavior 3nd Human Decision
Processes. 38, 92-113.

Bandura, A. & Cervone, D. (1986b). Social foundations of thought and action: A
socizi cognitive theory. Englewood Cliffs, NJ: Prentice-Hall.

Bernstein, D. A., & Borkovec, T. D. (1973). Progressive relaxation training: A
manual for the helping professions. Champaign, IL: Research Press.

157

Brown, 1., Jr., & Inounye, D. K. (1978). Learned helplessness through modeling:
The role of perceived similarity in competence. Journal of Personality and Social

Psychology, 36, 900-908.

Brown, K. G. (1999). Individual differences in choice during learning: The
inﬂuence of learner goals and attitudes in web-based training. Unpublished Doctoral

Dissertation. Michigan State University, East Lansing, MI.

Bunderson, C. V. (1970). The computer and instructional design. In W. H.

Holtzman (Ed), Computer-assisted instruction, testing, and guidance. New York: Harper
& Row.

Bunderson, C. V. (1976, April). TICCIT courseware development report. Paper
presented at the meeting of the American Educational Research Association, San
Francisco, CA.

Carrier, C. A., Davidson, 6., & Williams, M. (1985). Selection of instructional
options in a computer-based coordinate concept lesson. Educatiorﬂ Communigtions
and Technology Journal, 33, 199-212.

Carter, J. E., & Kelly, A. E. (1997). Using traditional and paradoxical imagery
interventions with reactant intramural athletes. The Sports Psychologist, 11, 175-189.

Carver, C. S., & Scheier, M. F. (1990). Origins and functions of positive and
negative affect: A control-process view. Psychological Bulletin. 97, 19-35.

Caudill, D., Weinberg, R., & Jackson, A. (1983). Psyching-up and track athletes:
A preliminary investigation. Journal of Sports Psychology, 5, 231-235.

Cervone, D. (1989). Effects of envisioning future activities on self-efﬁcacy
judgments and motivation: An availability heuristic interpretation. Cogpitive Therapy
and Research, 13, 247-261.

Cervone, D. (1993). The role of self-referent cognitions in goal setting,
motivation, and performance. In R. Mitchell (Ed), Cognitive science foundations of
instruction (pp. 57-95). Hillsdale, NJ: Lawrence Erlbaum Associates.

Cervone, D., J iwani, N., & Wood, R. (1991). Goal-setting and the differential
inﬂuence of self-regulatory processes on complex decision-making performance. Journal
of Personality jand Social Psychology, 61, 257-266.

Cervone, D., & Peake, P. K. (1986). Anchoring, efﬁcacy, and action: The

inﬂuence of judgmental heuristics on self-efﬁcacy judgments and behavior. Journal of
Personalityand Social chhologv, 50. 492-501.

158

Cohen, J. (1988). Statistical ppwer analysis for the behavioral sciences (2Ind Ed).
Hillsdale, NJ: Lawrence Erlbaum Associates.

Cramer, D., & Kupshik, G. (1993). Effect of rational and irrational statements on
intensity and “inappropriateness” of emotional distress an irrational beliefs in
psychotherapy patients. British Journal of Clinical Psychology, 32, 319-325.

Dossett, D. L., & Hulvershom, P. (1984). Increasing technical training

efficiency: Peer training via computer-assisted instruction. Journal of Applied
Psycholggy, 68, 552-558.

Duncan, N. C. (1991 ). CAI-enhanced exam performance in a research design
course. Behav_ior Research Methods, Instruments, & Computers, 23, 324-327.

Earley, P. C., Connolly, T., & Ekegren, G. (1989). Goals, strategy development,
and task performance: Some limits on the efﬁcacy of goal setting. Journal of Applied
Psychology, 74, 24-33.

Ellis, A. (1962). Reason and Emotion in chhotherapx New York: Lyle Stuart.

Faust, G. W. (1974). Design strategy and the TICCIT system. Viewpoints. 50,
91-101.

Fisher, S. L., & Ford, J. K. Differential effects of learner effort and goal
orientation on two learning outcomes. Personnel Psychology, 51, 397-420.

Foreyt, J. P. (1990). Behavioral medicine. In C. M. Franks, G. T. Wilson, P. T.
Kendall, & J. P. Foreyt (Eds), Review of behayior therapy: Theory and
Practice, Vol. 12 (pp. 637-644). New York: Guilford Press.

Genov, F. (1970). The nature of the mobilization readiness of the sportsman and
the inﬂuence of different factors upon its formation. In G. S. Kenyone (Ed),
Contemporary sport psychology, Chicago: The Athletic Institute.

Gist, M. E., & Mitchell, T. R. (1992). Self-efﬁcacy: A theoretical analysis of its
determinants and malleability. Academy of Management Review, 17, 183-211.

Goetzfried, L., & Hannaﬂn, M. J. (1985). The effect of locus of CA1 control
strategies on the learning of mathematics rules. American Educational Research Journal
_2_2, 273-278.

 

Goolkasian, P. (1989). Computerized laboratories for psychology instruction:
How successful are they? Behavior Research Methods, Instruments, & Computers, 21,
148-150.

159

Gray, S. H. (1987). The effect of sequence control on computer assisted
learning. Journal of Computer-Based Instruction. 14, 54-56.

Hannaﬂn, M. J. (1984). Guidelines for using locus of instructional control in the
design of computer-assisted instruction. Journal of Instructional Development, 7, 6-10.

Henry, R. A., & Sniezek, J. A. (1993). Situational factors affecting judgments of

future performance. Organizational Behavior and Human Decision Processes, S‘L 104-
132.

Hysong, S. J ., & Quinones, M. A. (1997). The relationship between self-efficacy
and performance: A meta-analysis. Paper presented at the Twelfth Annual Conference of
the Society for Industrial and Organizational Psychology, St. Louis, MO.

Jones, E. E. (1989). The framing of competence. Personality and Social
Psychology Bulletin, 15, 477-492.

Judd, W. A., Bunderson, C. V., & Bessent, E. W. (1970). An investigation of the
effects of learner control in computer-assisted instruction prergguisite mayhematics
(MATHS). Austin, TX: University of Texas.

Kanfer, R., & Ackerman, P. L. (1989). Motivation and cognitive abilities: An
integrative aptitude-treatment interaction approach to skill acquisition [Monograph].
Journal of Applied Psychology, 74, 657-690.

Kanfer, R., Ackerman, P. L., Murtha, T. C., Dugdale, B., & Nelson, L. (1994).
Goal setting, conditions of practice, and task performance: A resource allocation

perspective. Journal of Applied Psychology, 79, 826-835.

Kinzie, M. B., & Sullivan, H. J. (1989). Continuing motivation, learner control,
and CAI. Educational Technology Research and Development, 37, 5-14.

Kinzie, M. B., Sullivan, H. J ., & Berdel, R. L. (1988). Learner control and
achievement in science computer-assisted instruction. Jourryal of Educational
Psychology, 804 299-303.

Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on
performance: A historical review, a meta-analysis, and a preliminary feedback
intervention theory. Psychological Bulletin, 119, 254-284.

Kozlowski, S. W. J. (1998). Training and developing adaptive teams: Theory,
principles, and research. In J. A. Cannon-Bowers & E. Salas (Eds), Decision making

under stress: Implications for training and simulation. Washington, DC: APA Books.

160

Kozlowski, S. W. J ., Gully, S. M., Smith, E. M., Brown, K. G., Mullins, M. E., &
Williams, A. E. (1996, April). Sequenced mastery goals and advance organizers:
Enhancing the effects of practice. In K. Smith-Jentsch (Chair), When. how, and why
does praLtice ma_ke perfect? Symposium conducted at the 11th Annual Conference of the
Society for Industrial and Organizational Psychology, San Diego, CA.

Kozlowski, S. W. J ., Gully, S. M., Smith, A. E., Nason, E. R., & Brown, K. G.
(1995, May). Sequenced mastery training and advance organizers: Effects on learning,
self-efﬁcacy, performance, and generalization. In R. J. Klimoski (Chair), Thinking and
feeling while doing: Understanding the learner in the learning process. Symposium
conducted at the 10‘h Annual Conference of the Society for Industrial and Organizational
Psychology, Orlando, FL.

Kozlowski, S. W. J ., Toney, R. J ., Weissbein, D. A., Mullins, M. E., Brown, K.
G., & Bell, B. S. (1998, April). Training adaptive performance. In S. W. J. Kozlowski,
S. K. Parker, & M. Frese (Chairs), Beyond task pﬁerformance: Proactivity and learning.
Symposium conducted at the 13th Annual Conference of the Society for Industrial and
Organizational Psychology, Dallas, TX.

Kozlowski, S. W. J ., Weissbein, D. A., Brown, K. G., Toney, R. J ., & Mullins, M.
E. (1997). Guiding the development of deployable shipboard training systems:
Enhancing skill acquisition, adaptability, and effectiveness (Final Report 1.]; Contract
No. N61339-96-K-0005). Orlando, FL: Naval Air Warfare Center Training Systems
Division.

Kroll, M. D. (1991). Persistence following differential failure. Journal of
Psychology, 125, 241-251.

Lahey, G. F., Crawford, A. M., & Hurlock, R. E. (1976). I_.e_arner control of
lesson strategy: a model for PLATO IV system lessons. San Diego, CA: Naval Personnel
Research and Development Center.

Lee, C., & Bobko, P. (1994). Self-efﬁcacy beliefs: Comparison of ﬁve measures.
Journal of Applied Psychology, 79, 364-369.

Lichstein, K. L. (1988). Clinical relaxation strategies. New York: J. Wiley.

Locke, E. A. (1968). Toward a theory of task motivation and incentives.
Organizational Behavior and Human Performance, 3a 157-189.

Locke, E. A., Cartledge, N., & Knerr, C. S. (1970). Studies of the relationship

between satisfaction, goal setting, and performance. Organizational Behavior and
Human Performamce, 5. 135-158.

161

Mace, R., & Carroll, D. (1985). The control of anxiety in sport: Stress

inoculation training prior to abseiling. International Journal of Smrts Psychology, 16,
165-175.

Marcoulides, G. A. (1990). Improving learner performance with computer based
programs. Jouml of Educational Computing Research, 6, 147-155.

Mayer, R. E. (1976). Some conditions of meaningful learning for computer
programming: Advance organizers and subject control of frame order. Journal of

Educational Psychology, 68, 143-150.

McNeil, B. J ., & Nelson, K. R. (1991). Meta-analysis of interactive video
instruction: A 10-year review of achievement effects. Journal of Computer-Based
Instruction, 18. 1-6.

Milliman, Z., & Latham, G. P. (1996, August). Increasing re-employment

through training in verbal self-gpidance. Academy of Management Best Paper
Proceedings. Cincinnati, OH.

Morrison, G. R., Ross, S. M., & Baldwin, W. (1992). Learner control of context
and instructional support in learning elementary school mathematics. Educational
Technology Research and Developmentg40, 5-13.

Mullins, M. E., Kozlowski, S. W. J., Toney, R. J., Bell, B. S., Brown, K. G., &

Weissbein, D. A. (1999, April). Adaptive performance: Masteg versus parformance
goals and feedbagt consistency. Paper presented at the 14th Annual Conference of the

Society for Industrial and Organizational Psychology, Atlanta, GA.

Murphy, S. M., & Woolfolk, R. L. (1987). The effects of cognitive interventions
on competitive anxiety and performance on a ﬁne motor skill accuracy task.

International Journal of Sport Psychology, 18, 152-166.

Nelson, T. O., Dunlosky, J ., Graf, A., & Narens, L. (1994). Utilization of
metacognitive judgments in the allocation of study during multitrial learning.

Psychological Sciences, 5, 207-213.

Park, 0., & Tennyson, R. D. (1983). Computer-based instructional systems for
adaptive education: A review. Contemmrm Educational Review, 2, 121-135.

Podsakoff, P. M., & Fahr, J. (1989). Effects of feedback sign and credibility on
goal setting and task performance. Organizational Behavior apd Human Decision
Processes, 44a 45-67.

Pridemore, D. R., & Klein, J. D. (1991). Control of feedback in computer-
assisted instruction. Educational Technology Research and Development, 39, 27-32.

162

Reeves, T. C. (1993). Pseudoscience in computer-based instruction: The case of
learner control research. Journal of Computer-Based Instruction, 20, 39-46.

Roney, C. J. R., Higgins, E. T., & Shah, J. (1995). Goals and framing: How
outcome focus inﬂuences motivation and emotion. Personality and Social Psychology
Bulletin, 21,1151-1160.

Rosin, L., & Nelso, W. M., III. (1983). The effects of rational and irrational self-
verbalizations on performance efﬁciency and levels of anxiety. Journal of Clinical

Psychology, 39, 208-213.

Ross, S. M., & Rakow, E. A. (1981). Learner control versus program control as
adaptive strategies for selection of instructional support on math rules. Journal of

Educational Psychology, 73, 745-753.

Sandelands, L. E., Brockner, J., & Glynn, M. A. (1988). If at ﬁrst you don’t
succeed, try, try again: Effects of persistence-performance contingencies, ego

involvement, and self-esteem on task persistence. Journal of Applied Psychology, 73,
208-216.

Santiago, R. S., & Okey, J. R. (1992). The effects of advisement and locus of
control on achievement in leamer-controlled instruction. Journal of Computer-Based
Instruction, 19, 47-53.

Sapp, M. (1994). The effects of guided imagery on reducing the worry and
emotionality components of test anxiety. Journal of Mental Imagepy, 18, 165-180.

Schunk, D. H. (1984). Self-efﬁcacy perspective on achievement behavior.
Educational Psychologist, 19, 48-58.

Shelton, T. O., & Mahoney, M. J. (1978). The content and effect of “psyching-
up” strategies in weight lifters. Cognitive Therapy and Research, 2, 275-284.

Sherman, S. J ., Skov, R. B., Hervitz, E. F., & Stock, C. B. (1981). The effects of
explaining hypothetical future events: From possibility to probability to actuality and
beyond. Journal of Experimental Social Psychology, 17, 142-158.

Shyu, H. Y., & Brown, S. W. (1992). Learner control versus program control in
interactive videodisc instruction: What are the effects in procedural learning?
Intemational Journal of Instructional Media. 19, 85-96.

Silver, W. S., Mitchell, T. R., & Gist, M. E. (1995). Responses to successful and
unsuccessful performance: The moderating effect of self-efﬁcacy on the relationship
between performance and attributions. Organizatimal Behavior and Human Decision
ProcessesL6; 286-299.

163

Smith, E. M., Ford, J. K., & Kozlowski, S. W. J. (1997). Building adaptive
expertise: Implications for training design. In M. A. Quinones & A. Ehrenstein (Eds),

Training for a rapidly changing workplace: Applications of psychological research (pp.
89-118). Washington, DC: APA Books.

Steinberg, E. R. (1977). Review of student control in computer-assisted
instruction. Journal of Computer-Based Instruction, 3. 84-90.

Steinberg, E. R. (1989). Cognition and learner control: A literature review, 1977-
1988. Journal of Computer-Based Instruction. 16, 117-121.

Tennyson, R. D. (1980). Instructional control strategies and content structure as
design variables in concept acquisition using computer-based instruction. Journal of
Educational Psychology, 72. 525-532.

Tennyson, R. D. (1981). Use of adaptive information for advisement in learning
concepts and rules using computer assisted instruction. American Educational Research
Journal 18 425-438.

 

Tennyson, R. D., & Buttrey, T. (1980). Advisement and management strategies
as design variables in computer-assisted instruction. Educational Communication and
Technolcgy Journal. 28, 169-176. Reprinted in D. F. Walker & R. D. Hess (Eds).
(1984). Instructional software: Principles and perspectives for design and use. Belmont,
CA: Wadsworth.

Tennyson, R. D., Park, O. C., & Christensen, D. L. (1985). Adaptive control of
learning time and content sequence in concept learning using computer-based instruction.
Journal of Educzgional Psychology, 77, 481-491.

Tennyson, R. D., & Rothen, W. (1979). Management of computer-based
instruction: Design of an adaptive control strategy. Joumal of Computer-Ba_s_eg
Instruction, 5, 126-134.

Tennyson, C. L., Tennyson, R. D., & Rothen, W. (1980). Content structure and
instructional control strategies as design variables in concept acquisition. Journal of

Educational Psychology, 72, 499-505.

Tumage, J. J. (1990). The challenge of new workplace technology for
psychology. American Psychologist, 45, 171-178.

Van Raalte, J. L., Brewer, B. W., Lewis, B. P., Linder, D. E., Wildman, G., &
Kozimor, J. (1995). Cork! The effects of positive and negative self-talk on dart
throwing performance. Journal of Smrt Behavior, 18, 50-57.

164

Weaver, J. L., Bowers, C. A., Salas, E., & Cannon-Bowers, J. A. (1995).
Networked simulations: New paradigms for team performance research. Behavioral

Research Methods, Instruments, & Computers, 27, 12-24.

Weaver, J. L., Morgan, B. B., Jr., Hall, J ., & Compton, D. (1993). Team decision

making in the command information center: Development of a low-ﬁdelity team decision
making task for assessing the effects of teamwork stressors. Orlando, FL: Naval Training

Systems Center, Human Factors Division.

Weinberg, R. S., Gould, D., & Jackson, A. (1979). Expectations and
performance: An empirical test of Bandura’s self-efﬁcacy theory. Journal of Smrt

Psychology, 1, 320-331.

Weinberg, R. S., Gould, D., & Jackson, A. (1980). Cognition and motor
performance: Effect of psyching-up strategies on three motor tasks. Cogaitive Therapy
and Research 4 239-245.

 

Wexley, K. N. (1984). Personnel Training. Annual Review of Psychology, 35,
519-551.

Wheatley, W. J ., Maddox, E. W., & Anthony, W. P. (1989). Enhancing guided
mental imagery scripts utilized in the classroom. Education 110 33-39.

 

Williams, M. D. (1993). A comprehensive review of learner-control: The role of

learner characteristics. In M. R. Simonson (Ed), Proceedings of the Annual Conference
of the Assoﬂtion for Educational Communications and Technology (pp. 1083-1114).

New Orleans, LA: Association for Educational Communications and Technology.

Worthington, E. L., Jr., Welsh, J. A., Archer, C. R., Mindes, E. J ., & Forsyth, D.
R. (1996). Computer-assisted instruction as a supplement to lectures in an introductory

psychology class. Teaching of Psychology, 23, 175-181.

165

HICHIGRN STRTE UNIV. LIBRQRIES
1111111111111|11111111||1||1|1|111|11|H1111
312930206041 16