 

.‘J

,
l
' 3f:
. * "wry
0.qu

.‘o

 

as- --mJ-‘.-‘n

:
M...

 

 

..
...__
,2.“
‘.
‘

u

.

v

o

 

 

 

x. z..t.f.~L....a.... ,At.l¢.rl. ,.u.l. ‘v. b.3211 .v y t . . r 1.. ,. . . , I . O . . ‘

 

THESlS

ICHIGAN STATE UNIVERSITY LIBRARIE

Illlllllllll Ill I”! .l illillillllllllll

, 3 1293 01561 9061

 

Ill

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

F LIBRARY

‘ Michigan State
University

 

 

This is to certify that the

thesis entitled

MOTIVATIONAL AND INFORMATIONAL CONSEQUENCES OF ERRORS
IN EARLY SKILL ACQUISITION: THE EFFECTS OF INDIVIDUAL
DIFFERENCES AND TRAINING STRATEGY ON
PERCEPTIONS OF NEGATIVE FEEDBACK

presented by

KENNETH GUY BROWN

has been accepted towards fulfillment
of the requirements for

MA degree in PsyChOIOQY

 

 

WQ/Qﬂ/ ~

M ' rprofess

Steve W.J. Koz ski
Date August 19, 1996

0-7639 MS U is an Afﬁrmative Action/Equal Opportunity Institution

PLACE II RETURN BOX to roman this checkout from your record.
TO AVOID FINES mum on or baton dd- duo.

DATE DUE DATE DUE DATE DUE

        
   

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

_ L]? A f E

MSU I. An mum ActionlEqml Opporturmy Imam
Wan-OJ

 

 

MOTIVATIONAL AND INFORMATIONAL CONSEQUENCES OF ERRORS IN
EARLY SKILL ACQUISITION: THE EFFECTS OF INDIVIDUAL DIFFERENCES
AND TRAINING STRATEGY ON PERCEPTIONS OF NEGATIVE FEEDBACK

By

Kenneth Guy Brown

A THESIS

Submitted to
Michigan State University
in partial fulﬁllment of the requirements
for the degree of

MASTER OF ARTS

Department of Psychology

1996

ABSTRACT

MOTIVATIONAL AND INFORMATIONAL CONSEQUENCES OF ERRORS IN
EARLY SKILL ACQUISITION: THE EFFECTS OF INDIVIDUAL DIFFERENCES
AND TRAINING STRATEGY ON PERCEPTIONS OF NEGATIVE FEEDBACK

By

Kenneth Guy Brown

A laboratory experiment in which 110 undergraduates learned a difficult computer
simulation is reported. Perceptions of the “evaluative” and “prescriptive” characteristics
of feedback were found to have a number of signiﬁcant effects on learning, the strongest
for evaluative feedback magnitude judgments that involve internal, stable attributions for
poor performance. Individuals who reported higher attributional magnitude judgments
were more likely to have lower self-efﬁcacy, lower motivation to learn, and lower
knowledge test scores, regardless of their actual performance level during training.
Individual difference variables signiﬁcantly predicted attributional magnitude judgments,
with individuals low in cognitive ability, high in performance goal orientation, and low in
learning goal orientation most “at-risk” for these negative outcomes. Training strategies
used in this study were generally useful, yet unable to ameliorate the inﬂuence of these

attributional judgments on the learning process and learning outcomes.

To all the teachers in the world, formal and otherwise, who help others to become

something better than they were before...

iii

ACKNOWLEDGMENTS

Research is not a natural process. While curiosity and the desire to explore are,
the rigor and persistence demanded by social science research do not easily follow from
these more natural tendencies. Many people provided the emotional, intellectual, and lest
we not forget, ﬁnancial support necessary to sustain me through this difficult process.

First, I want to thank my thesis advisor, Steve Kozlowski, for all of the advice,
assistance, and support he has provided over the last 3 years. He has been a tremendous
inﬂuence on me. I also want to thank the other members of my committee, Kevin Ford
and Rick DeShon. Each has inﬂuenced me and, whether or not they choose to admit it,
created in some part my identity as a thinker and researcher. All of my classmates also
deserve thanks for putting up with me over the last few years. Special thanks go to Stan,
Earl, and Eleanor for providing me with an amazing introduction to teamwork at MSU.
And to Dan and David, thanks go for their friendship and support, including many late
night snacks, which helps make the hard work worthwhile.

Of course none of this would be possible without the love (and prodding) of an
amazing family. I have been lucky to have parents who are fantastic role models, ﬁlling
each and every role they hold with dedication and humor. I love them both dearly. My
brother and sisters are equally amazing people, each setting the bar a little higher in their
own way. And ﬁnally, to my best friend and wife, Amy. It is for her that I strive each

day to better myself and the world around me. Thank you.

ACKNOWLEDGMENTS

Research is not a natural process. While curiosity and the desire to explore are,
the rigor and persistence demanded by social science research do not easily follow from
these more natural tendencies. Many people provided the emotional, intellectual, and lest
we not forget, ﬁnancial support necessary to sustain me through this difﬁcult process.

First, I want to thank my thesis advisor, Steve Kozlowski, for all of the advice,
assistance, and support he has provided over the last 3 years. He has been a tremendous
inﬂuence on me. I also want to thank the other members of my committee, Kevin Ford
and Rick DeShon. Each has inﬂuenced me and, whether or not they choose to admit it,
created in some part my identity as a thinker and researcher. All of my classmates also
deserve thanks for putting up with me over the last few years. Special thanks go to Stan,
Earl, and Eleanor for providing me with an amazing introduction to teamwork at MSU.
And to Dan and David, thanks go for their friendship and support, including many late
night snacks, which helps make the hard work worthwhile.

Of course none of this would be possible without the love (and prodding) of an
amazing family. I have been lucky to have parents who are fantastic role models, ﬁlling
each and every role they hold with dedication and humor. I love them both dearly. My
brother and sisters are equally amazing people, each setting the bar a little higher in their
own way. And ﬁnally, to my best friend and wife, Amy. It is for her that I strive each

day to better myself and the world around me. Thank you.

TABLE OF CONTENTS

LIST OF TABLES ............................................................................................................. vii
LIST OF FIGURES ............................................................................................................ ix
INTRODUCTION ............................................................................................................... 1
LITERATURE REVIEW .................................................................................................... 7
Errors in Early Skill Acquisition ............................................................................. 7
Learning from Errors ............................................................................................... 8
Empirical Evidence for Errors ................................................................... l3

Empirical Evidence against Errors ............................................................. 15

Learning from Feedback ........................................................................................ 18
Sender ........................................................................................................ 2O

Receiver ..................................................................................................... 20

Message ..................................................................................................... 24

Attentional Focus in Training ................................................................................ 29
Motivation in Training ........................................................................................... 33
Training Outcomes ................................................................................................. 38
INTEGRATION AND HYPOTHESES ............................................................................ 42
METHOD AND MEASURES .......................................................................................... 55
Participants ............................................................................................................. 55

Task Selection and Description ............................................................................. 56
Design .................................................................................................................... 60
Training Strategy Manipulations ........................................................................... 61
Mastery versus Performance ...................................................................... 61

Sequenced versus Unsequenced ................................................................. 62

Procedure ............................................................................................................... 64
Measures ................................................................................................................ 67
Learning Outcomes .................................................................................... 67

Feedback Constructs .................................................................................. 68

Mediating Constructs ................................................................................. 69

Individual Differences ................................................................................ 7O

RESULTS .......................................................................................................................... 72

Reliability Analyses ............................................................................................... 72
Manipulation Checks ............................................................................................. 79
Training Strategy Effects ....................................................................................... 85
Individual Difference Effects ................................................................................. 92
Process Model ...................................................................................................... 102
Attentional Focus ..................................................................................... 104

Self-Efﬁcacy ............................................................................................ 105

Motivation to Learn ................................................................................. 111

Training Outcomes ............................................................................................... 116
Knowledge ............................................................................................... 116

Skill .......................................................................................................... 117

Attitudes ................................................................................................... 117

Mediation Analyses ............................................................................................. 123
Individual Differences-> Feedback Perceptions->Learning Process ....... 123

Feedback Perceptions-> Learning Process-> Training Outcomes ........... 125

Individual Differences» All Intervening-> Training Outcomes ............. 126

Summary of Results and Overall Model Fit ........................................................ 127
DISCUSSION .................................................................................................................. 133
Process Model ...................................................................................................... 134
Training Strategies ............................................................................................... 138
Limitations and Future Research Directions ........................................................ 140
APPENDIX A: Training Strategy Manipulations ........................................................... 146
APPENDIX B: Learning Outcome Questionnaire .......................................................... 149
APPENDIX C: Feedback and Mediating Construct Questionnaire ................................ 154
APPENDIX D: Individual Differences Questionnaire ................................................... 163
APPENDIX E: Hypotheses, Analyses, and Summary of Results .................................. 166
LIST OF REFERENCES ................................................................................................. 167

vi

LIST OF TABLES

Table 1 - Outline of Proposed Procedure ........................................................................... 65
Table 2 - Factor Solution for Individual Difference Scales ............................................... 74
Table 3 - Factor Solution for Feedback Constructs ........................................................... 75
Table 4 - Factor Solution for Diagnosticity Item Couplets ............................................... 77
Table 5 - Factor Solution for Mediating Constructs .......................................................... 78
Table 6 - Factor Solution for Attitude Training Outcomes ................................................ 80
Table 7 - Descriptive Statistics and Intercorrelations ................................................... 81-82

Table 8 - Repeated Measures Analysis of Variance of Training Strategy
on Descriptive Magnitude of Negative Feedback ...................................... 86

Table 9 - Repeated Measures Analysis of Variance of Training Strategy
on Attributional Magnitude of Negative Feedback .................................... 88

Table 10 - Repeated Measures Analysis of Variance of Training Strategy
on Diagnosticity of Negative Feedback ..................................................... 89

Table 11 - Individual Differences Regression Equation for Diagnosticity
of Feedback ................................................................................................ 94

Table 12 - Individual Differences Regression Equation for First Change
in Diagnosticity of Feedback ..................................................................... 95

Table 13 - Individual Differences Regression Equation for Second Change _
in Diagnosticity of Feedback ..................................................................... 96

Table 14 - Individual Differences Regression Equation for Knowledge
Test ............................................................................................................ 97

Table 15 - Individual Differences Regression Equation for Final
Performance Score ..................................................................................... 98

vii

LIST OF TABLES

Table 1 - Outline of Proposed Procedure ........................................................................... 65
Table 2 - Factor Solution for Individual Difference Scales ............................................... 74
Table 3 - Factor Solution for Feedback Constructs ........................................................... 75
Table 4 - Factor Solution for Diagnosticity Item Couplets ............................................... 77
Table 5 - Factor Solution for Mediating Constructs .......................................................... 78
Table 6 - Factor Solution for Attitude Training Outcomes ................................................ 80
Table 7 - Descriptive Statistics and Intercorrelations ................................................... 81-82

Table 8 - Repeated Measures Analysis of Variance of Training Strategy
on Descriptive Magnitude of Negative Feedback ...................................... 86

Table 9 - Repeated Measures Analysis of Variance of Training Strategy
on Attributional Magnitude of Negative Feedback .................................... 88

Table 10 - Repeated Measures Analysis of Variance of Training Strategy
on Diagnosticity of Negative Feedback ..................................................... 89

Table 11 - Individual Differences Regression Equation for Diagnosticity
of Feedback ................................................................................................ 94

Table 12 - Individual Differences Regression Equation for First Change
in Diagnosticity of Feedback ..................................................................... 95

Table 13 - Individual Differences Regression Equation for Second Change
in Diagnosticity of Feedback ..................................................................... 96

Table 14 - Individual Differences Regression Equation for Knowledge
Test ............................................................................................................ 97

Table 15 - Individual Differences Regression Equation for Final
Performance Score ..................................................................................... 98

vii

LIST OF TABLES

Table 1 - Outline of Proposed Procedure ........................................................................... 65
Table 2 - Factor Solution for Individual Difference Scales ............................................... 74
Table 3 - Factor Solution for Feedback Constructs ........................................................... 75
Table 4 - Factor Solution for Diagnosticity Item Couplets ............................................... 77
Table 5 - Factor Solution for Mediating Constructs .......................................................... 78
Table 6 - Factor Solution for Attitude Training Outcomes ................................................ 80
Table 7 - Descriptive Statistics and Intercorrelations ................................................... 81-82

Table 8 - Repeated Measures Analysis of Variance of Training Strategy
on Descriptive Magnitude of Negative Feedback ...................................... 86

Table 9 - Repeated Measures Analysis of Variance of Training Strategy
on Attributional Magnitude of Negative Feedback .................................... 88

Table 10 - Repeated Measures Analysis of Variance of Training Strategy
on Diagnosticity of Negative Feedback ..................................................... 89

Table 11 - Individual Differences Regression Equation for Diagnosticity
of Feedback ................................................................................................ 94

Table 12 - Individual Differences Regression Equation for First Change
in Diagnosticity of Feedback ..................................................................... 95

Table 13 - Individual Differences Regression Equation for Second Change
in Diagnosticity of Feedback ..................................................................... 96

Table 14 - Individual Differences Regression Equation for Knowledge
Test ............................................................................................................ 97

Table 15 - Individual Differences Regression Equation for Final
Performance Score ..................................................................................... 98

vii

Table 16 - Individual Differences Regression Equation for Initial

Attributional Magnitude of Negative Feedback ......................................... 99
Table 17 - Individual Differences Regression Equation for First

Change in Attributional Magnitude of Negative Feedback ..................... 100
Table 18 - Individual Differences Regression Equation for Second

Change in Attributional Magnitude of Negative Feedback ..................... 101
Table 19 - Individual Differences Regression Equation for Initial

Descriptive Magnitude of Negative Feedback ......................................... 102
Table 20 - Individual Differences Regression Equation for First

Change in Descriptive Magnitude of Negative Feedback ....................... 103
Table 21 - Individual Differences Regression Equation for Second

Change in Descriptive Magnitude of Negative Feedback ....................... 104
Table 22 - Regression Equation for Initial Attention Focus (Self-Report) ...................... 106
Table 23 - Regression Equation for Change in Attention Focus

(Self-Report) ............................................................................................ 1 07
Table 24 - Regression Equation for Initial Attention Focus (Time) ................................ 108

Table 25 - Regression Equation for First Change in Attention Focus

(Time) ...................................................................................................... 109
Table 26 - Regression Equation for Second Change in Attention Focus

(Time) ...................................................................................................... 110
Table 27 - Regression Equation for Initial Self-Efficacy ................................................ 112
Table 28 - Regression Equation for Change in Self-Efﬁcacy .......................................... 113
Table 29 - Regression Equation for Initial Motivation to Learn ...................................... 114
Table 30 - Regression Equation for Change in Motivation to Learn ............................... 115
Table 31 - Overall Test for Knowledge Outcome of Test Score ..................................... 118
Table 32 - Overall Test for Skill Outcome of Final Simulation Score ............................ 120
Table 33 - Overall Test for Affective Training Outcome of Task Satisfaction ............... 122

viii

LIST OF FIGURES
Figure l - Manipulation Research Model .......................................................................... 43
Figure 2 - Individual Differences Research Model ............................................................ 44

Figure 3 - Hypothesized Magnitude Perception Interaction by Training
Strategy Condition ..................................................................................... 49

Figure 4 - Hypothesized Diagnosticity Perception Interaction by Training
Strategy Condition ..................................................................................... 49

Figure 5 - Descriptive Magnitude of Negative Feedback by Goal
Condition ................................................................................................... 87

Figure 6 - Descriptive Magnitude of Negative Feedback by
Sequencing Condition ............................................................................... 87

Figure 7 - Self-Report Attention and Motivation to Learn Interaction

Predicting Knowledge Test Score ............................................................ 119
Figure 8 - Self-Report Attention and Motivation to Learn Interaction

Predicting Final Performance Score ........................................................ 121
Figure 9 - Path Diagram for Learning from Negative Feedback ..................................... 129
Figure 10 - Summary of Signiﬁcant Results .................................................................... 130

INTRODUCTION

The increased rate of social and technological change in today’s workplace
demands that workers constantly learn new skills (Bridges, 1994; Goldstein & Gilliam,
1990; Howard, 1995). While this dynamic complexity of modern work is most obvious
with computer technology, change now pervades most industries and almost all aspects of
work (Howard, 1995). Employers recognize the resulting need for employees to engage
in constant learning and skill updating. In a national survey of employers, the ability and
desire to learn was rated among the most desirable characteristics for employees
(Carnevale, Gainer, & Meltzer, 1988). In short, today’s workplace will increasingly
require employees to learn new, complex skills. A result of this trend is the increased
need for research investigating: (1) How people react to and learn from challenging new
tasks, and (2) the effectiveness of practical training strategies that can be employed to
facilitate learning in these situations.

It is clear that learning new skills can be a difﬁcult and frustrating process.
Complex skills such as operating production machinery, navigating computer programs,
or troubleshooting electrical equipment all require an extensive knowledge base before
successful performance can be achieved. Early attempts at these skills, particularly when
a trainee’s knowledge-base is inadequate, are error-prone. Disappointment and confusion
often result, with trainees sometimes feeling that they are not capable of gaining the
necessary knowledge. Although these feelings may subside as skill increases, early

stages of skill acquisition can be critical. During initial interactions with a task, trainees
1

2
develop attitudes regarding their task-speciﬁc capability, the task itself, and the training

enterprise. Each of these attitudes may inﬂuence subsequent motivation and behavior.

Historical approaches to training complex skills suggest that negative
consequences of errors can be avoided by creating a careful program of learning where
trainees are taken step-by-step through the learning process (e.g., Biehler, 1978).
Dominated by behaviorism, early learning research suggested errors interfere with the
acquisition of skills by distracting the trainee from proper behaviors while strengthening
improper behaviors (e.g., Skinner, 1956). More recently, both the ACT“ (Anderson,
1983) and QUEST (White & F rederiksen, 1986) programs of instructional research
suggest that minimization of errors is the most effective manner to build expertise
(Glaser, 1990; Ohlsson, 1986).

Based less on a behavioral and more on a cognitive perspective of learning, a
growing literature recognizes the value of errors in promoting concept formation, rule
induction, and skill acquisition (e.g., Ivancic & Hesketh, 1995). The notion that errors
can aid learning is not new as it is reﬂected in the “discovery” side of the traditional
debate over discovery learning versus programmed learning (e.g., Hermann, 1969).
Recent work, perhaps driven in part by the current need for constant learning, has revived
this interest in the usefulness of errors in the training context (Frese & Altrnann, 1989;
Frese, Brodbeck, Heinbokel, Mooser, Schleiffenbaum, & Theilrnann, 1991; Ivancic &
Hesketh, 1995; Nordstrom, Wendland, & Williams, 1995).

The recent research in this area has been driven largely by one speciﬁc approach.
This approach, termed Error Management Training (EMT), suggests that training

programs should explicitly include errors at certain stages of the learning process (e. g.,

2
develop attitudes regarding their task-speciﬁc capability, the task itself, and the training

enterprise. Each of these attitudes may inﬂuence subsequent motivation and behavior.

Historical approaches to training complex skills suggest that negative
consequences of errors can be avoided by creating a careful program of learning where
trainees are taken step-by-step through the learning process (e. g., Biehler, 1978).
Dominated by behaviorism, early learning research suggested errors interfere with the
acquisition of skills by distracting the trainee from proper behaviors while strengthening
improper behaviors (e.g., Skinner, 1956). More recently, both the ACT“ (Anderson,
1983) and QUEST (White & F rederiksen, 1986) programs of instructional research
suggest that minimization of errors is the most effective manner to build expertise
(Glaser, 1990; Ohlsson, 1986).

Based less on a behavioral and more on a cognitive perspective of learning, a
growing literature recognizes the value of errors in promoting concept formation, rule
induction, and skill acquisition (e.g., Ivancic & Hesketh, 1995). The notion that errors
can aid learning is not new as it is reﬂected in the “discovery” side of the traditional
debate over discovery learning versus programmed learning (e. g., Hermann, 1969).
Recent work, perhaps driven in part by the current need for constant learning, has revived
this interest in the usefulness of errors in the training context (F rese & Altrnann, 1989;
Frese, Brodbeck, Heinbokel, Mooser, Schleiffenbaum, & Theilmann, 1991; Ivancic &
Hesketh, 1995; Nordstrom, Wendland, & Williams, 1995).

The recent research in this area has been driven largely by one speciﬁc approach.
This approach, termed Error Management Training (EMT), suggests that training

programs should explicitly include errors at certain stages of the learning process (e.g.,

3
F rese & Altmann, 1989). While studies based on EMT have demonstrated that this

particular training method is effective, these studies do not address the more basic issue
of how individuals learn from the difficult, early stages of skill acquisition. Furthermore,
the training method suggested in these studies is impractical. EMT requires the addition
of an entire training sequence. Further, this sequence is not appropriate during the early
stages of skill acquisition. Thus, the question regarding what can be done to aid trainees
early on in training remains.

While EMT research has pushed our understanding of how to use errors in the
learning process, research is still needed on the basic process of how individuals react and
learn from error-ﬁlled learning experiences. Furthermore, research should try to identify
and evaluate training strategies that can be easily adopted to early training experiences.
More speciﬁcally, strategies should be investigated that affect perceptions of errors so as
to decrease the frustration and increase the learning that occurs as a result of their
commission.

The growing literature on errors in training asserts that errors have both an
informational and motivational role in learning (Ivancic & Hesketh, 1995). Errors
provide information to trainees about incorrect states, procedures, and actions. This
information can provide knowledge and skills that are integral to successﬁrl task
performance. However, errors may lower motivation and lead trainees to disengage

either mentally or physically from the task]. While it may be difficult to completely

 

' Errors may also increase motivation by presenting a challenge to the trainee. While this is acknowledged,
the focus of this paper is on early skill acquisition where errors are frequent and argued to generate
frustration and anxiety rather than increased motivation. The possibility of errors increasing motivation is
addressed in greater detail in the motivation section.

4
disentangle motivational and informational effects, a great deal remains to be learned

about the how motivation and information interact in the learning process. In the past,
different training paradigms have targeted either the informational or motivational role of
errors. For example, intelligent tutoring research focuses on the information from errors,
including the diagnosis of why an error occurred and how that information can be used to
correct trainee’s knowledge (Ohlsson, 1986). Meanwhile, the motivational implications
of errors are generally ignored or assumed constant across individuals and training
environments. Conversely, recent training research incorporates motivational strategies
aimed at decreasing the negative effects of errors while ignoring those training strategies
that might enhance the information errors provide (e.g., Boyle & Klimoski, 1995; Kanfer,
1996). To my knowledge no study has investigated how training strategies inﬂuence
both informational and motivational consequences of veridical feedback.

One perspective that may prove useful in an in-depth investigation of errors is an
explicit recognition of errors as feedback. If an error is recognized by a trainee as having
occurred, negative feedback is conveyed. Attention to this fact may provide a means to
dissect the psychological experience of difﬁcult training situations, and in turn provide a
greater understanding of how errors inﬂuence the learning process, and ultimately
learning outcomes.

The feedback literature (e.g., Ilgen, Fisher, & Taylor, 1979) clearly indicates that
feedback confounds evaluative information (i.e., this is how well you did) with
prescriptive (i.e., this is what you should have done) information. The dual meaning
conveyed by feedback parallels the motivational and informational effects discussed in

the errors in training literature. Thus, while an error can provide information in the form

5
of prescriptive information, it also provides evaluative information that the trainee is

doing something wrong. The prescriptive information can beneﬁt trainees by helping
them focus their attention on important, unlearned aspects of the task. The evaluative
information, however, can be either beneﬁcial or harmful, depending on whether it
stimulates the recipient to work harder or to withdrawal from the task. When evaluative
information leads the trainee to withdrawal, the beneﬁcial effects of prescriptive
information are ignored and learning will not occur. What leads individuals to
withdrawal and ignore the attentional advantages that negative feedback from errors can
provide?

The purpose of this thesis is to develop and validate a theoretical model that
answers this question within the context of error-ﬁlled early skill acquisition. To distill
key constructs in the learning process, a literature review was conducted. Based on this
review, a process model was developed that explicates the psychological processes that
result from each message -- evaluative and prescriptive -- conveyed by negative feedback.
The model suggests two perceptions of feedback -- magnitude and diagnosticity -- are
predominant inﬂuences on motivational (evaluative) and informational (prescriptive)
learning “tracks,” respectively. Although these tracks ultimately interact to determine
many training outcomes, perceptions of feedback provide a means to partially disentangle
the effects of the evaluative and prescriptive messages that are conveyed when errors are
committed. Two easily implemented training strategies are suggested as useful ways to
research this model, as they provide for differential effects on the suggested mechanisms

and may be useful for improving learning outcomes from the difﬁcult, early stages of

6
training. An experiment conducted to evaluate the training strategies and the model

underlying their development is reported.

6
training. An experiment conducted to evaluate the training strategies and the model

underlying their development is reported.

LITERATURE REVIEW

In order to build a process model of learning during the difﬁcult, early stages of
skill acquisition, literature on errors in training and on learning from feedback is
reviewed. Further, to investigate the possible informational effects of errors, literature on
attention in training and training sequencing is reviewed brieﬂy. Then, as a way to
investigate motivational issues in training, research that has explicitly incorporated
motivational variables in training is reviewed, including research on mastery versus
performance goals. Next, to clarify the criterion domain, a narrow survey of training
evaluation research is presented, with an emphasis on the multi-dimensional perspective
on training outcomes.

An error is deﬁned by F rese and Altmann (1989) as the result of a potentially
avoidable action that causes the violation of an objective rule and the non-attainment of a
trainee’s goal. This deﬁnition emphasizes that errors occur only if they are avoidable by
alternative actions or behaviors. Further, to be considered an error, an action must
prevent or delay the accomplishment of a desired goal state. Because goals are situation-
speciﬁc, errors are often discussed as the result of an interaction between the situation and
the individual (Fuller, 1990).

In the context of early skill acquisition, errors often occur because knowledge
about a system is incorrect or inadequate (Frese & Altmann, 1989). When trainees are

given free reign in unfamiliar situations or with unfamiliar systems, errors are a natural
7

8
result of the inadequacy of their knowledge base. The alternative to errorful training is

error-free training, in which the trainee’s behaviors are guided and controlled in order to
avoid incorrect actions. While there are other forms of errors and more advanced forms
of error-ﬁlled training, the focus of this research is on errors that result from the
interaction between the trainee as a novice and the situation of early skill acquisition.

There is little research to identify how errors in training may facilitate or inhibit
the learning process. Most literature on errors emphasizes the explanation or prediction
of human error (e.g., Norman, 1984; Rasmussen, 1990). However, in the training
literature, there are two theoretical papers that directly address the potential usefulness of
errors in training -- Frese and Altmann (1989) and Ivancic and Hesketh (1995). The work
of these authors is reviewed for their perspective on the positive and negative effects of
errors in training. Then empirical work regarding these ideas is reviewed, with a note on
limitations and avenues for future research.
LeaminafmmEum

Frese and Altmann (1989) present arguments for and against the usefulness of
errors in training. According to these authors, researchers have identiﬁed three potential
negative consequences of errors for learning. First, Skinner and other behaviorists argue
that errors are akin to punishment. While an individual may learn to avoid the particular
behavior or action that led to the error (or punishment), the individual does not learn the
correct behavior. In order for the trainee to learn the correct sequence of actions, positive
reinforcement must be provided to the trainee upon commission of the correct behaviors,
a process often called “shaping” (Skinner, 1956, 1968). Thus, errors are merely aversive

stimuli with no beneﬁt to the instructional process. Second, there are also behaviorist

9
arguments that when an incorrect action is performed, it is learned to some degree. That

is, the very commission of a behavior makes that behavior more likely in the future, so
incorrect actions should never be allowed to occur in the ﬁrst place. Third, a more
humanistic perspective dictates that errors arouse anxiety and ﬁ'ustration. If it is possible
to avoid these negative states, a trainer or teacher should help do so (Frese & Altmann,
1989)

On the positive side, Frese and colleagues take a more cognitive perspective and
suggest errors increase knowledge about a system. F rese et al. (1991) note that “mental
models” of a system are enhanced when a person makes an error, both by indicating that
an area is not known well and by encompassing pitfalls and error-prone areas into the
trainee’s mental model. Although the term mental [model has been differentiated from
the general term knowledge (e.g., Rouse & Morris, 1986), Frese and colleagues use the
term loosely to indicate all forms of knowledge about a system (e.g., F rese & Altmann,
1989,p.66)

Frese et al. (1991) also note that trainers restrict the kinds of strategies used by
trainees if they try to eliminate errors from training. Errors can spur exploration and
creative strategies that would not have been employed in a more restricted training
environment. There is growing evidence that allowing the use of exploration strategies
can improve learning, particularly with adult learners (Carroll & Mack, 1984; Frese et al.,
1988)

The notion of exploration as beneﬁcial to learning marks the theme of a number
of studies on discovery learning (Andrews, 1984; Hermann, 1969; McDaniel & Schlager,

1990; Prather, 1971; Singer & Pease, 1974). Each of these studies suggests that active

10
exploration of a novel system provides advantages over more programmed or static

learning environments. Most of this research, however, is concerned with the issue of
transfer to tasks that differ in signiﬁcant ways from the trained task (i.e., McDaniel &
Schlager, 1990; Singer & Pease, 1976). Furthermore, these studies do not focus on errors
per se, but on exploration and hypothesis-testing that may involve errors. For the
purposes of this study, the research on discovery learning provides ambiguous
information about how individuals react or learn from errors. The discovery research
investigates instructional techniques that involve errors without ever equating subjects on
the degree or amount of exploration and comparing the perceptions, reactions, and
consequences of error commission.

Each of the positives of error commission discussed above involves increases in
knowledge that may come as a result of abehayiomueagtionjmm, not necessarily as
a result of error commission itself. In other words, error commission does not
necessarily provide new knowledge merely by its commission. Error commission can
provide a signal to the learner that something should be thought about, paid attention to,
or studied more. It is this “signal” provided by errors that initiates a learning process. It
is the behavioral manifestations of this process, or the trainee’s reaction to the error, that
ultimately cause changes in knowledge. In the discovery learning studies, it is often the
commission of an error that leads to exploration of a system and its capabilities. Again,
however, these studies investigate this phenomena by encouraging or limiting exploration
following error commission, not by observing and measuring reactions to errors. To
investigate the reactions and consequences of error commission during early skill

acquisition, exploration and other behavioral reactions to errors should be measured, not

ll
manipulated. Note that this suggests a somewhat different research approach than that

employed by traditional research on errors in training.

The Frese and Altmann (1989) work implies three important preconditions for
learning to occur from errors: (1) Errors must be noticed, (2) errors must be interpreted
correctly, and (3) errors must be used to guide attention in order for learning effects to
occur. Some of these pre-conditions are made explicit by Frese and Altmann (1989). For
example, they emphasize the importance of the interpretation of feedback generated from
errors in the following quote: “Errors provide feedback to the person. However, feedback
is only useful when the trainee is able to perceive and interpret the feedback...” (Frese &
Altmann, 1989, p. 76). Unfortunately, their research does not look at perceptions of
errors. Nor do they consider explicitly the role of the behavioral outcomes of these
perceptions, other than ultimate learning outcomes. Instead, the focus of their work is
explicating Error Management Training which combines error commission, emotional
and informational strategies to deal with the errors, and active exploration. As a result,
the process of learning from errors is hard to disentangle from their empirical results. An
interesting future direction for their research would be to investigate how trainees use
errors to guide their learning efforts during skill acquisition. This research would provide
a clearer indication of which aspects of EMT are integral to generating positive learning
effects.

Ivancic and Hesketh (1995) also discuss the positive and negative effects of errors
in training. These authors note three beneﬁts errors have in promoting learning. First,
the commission of errors provokes controlled or conscious processing of information.

When trainees become more conscious of their activities, a task is actively thought about

12
and practiced longer than it would be if the trainee judged the task well-learned and

focused on new activities. This statement provides the emphasis on attention and
behavioral reaction that seems missing in the work by Frese and Altmann (1989). The
important statement derived from this argument is that the longer something is thought
about or practiced, the better learned it is likely to become (e.g., Ericsson, Krampe, &
Tesch-Romer, 1993).

Second, echoing the comments of Frese and colleagues, Ivancic and Hesketh
(1995) suggest that errors help trainees develop accurate mental models. This is
accomplished by initiating hypothesis testing and problem solving activities which can
provide greater information about a system and even reveal faulty assumptions.
Similarly, an error provides information that a particular strategy or activity is
inappropriate and must be adapted or modiﬁed. So errors can narrow the range of
possible alternatives for achieving a goal. Again, Ivancic and Hesketh (1995) are noting
the importance of behavioral reactions to errors in order for learning to occur.

Third, errors or failures can be useful as reminders of analogous problems. In
essence, an error can serve as a way to cue up memories of successful and unsuccessﬁrl
strategies. A particular error may help an individual recall relevant examples of past
performance. As the preceding paragraphs indicate, the positive effects of errors are
generally informational in that they can provide knowledge about the target activity.

Ivancic and Hesketh (1995) suggest that motivational effects of errors are mostly
negative. The authors note that errors can induce learned helpless reactions. Learned
helplessness is a well-researched phenomena that involves motivational deﬁcits following

repeated exposure to failure (e.g., Dweck, 1986; Mikulincer, 1989). Learned helpless-

13
like reactions involve reducing effort and/or withdrawing from a task. This reduction in

effort can undermine training efforts in error-ﬁlled programs. If a trainee reduces effort,
withdrawing in some way ﬁ'om the training exercises, it is unlikely that the trainee will
learn.

Empmcalﬁxrdent‘efonﬁms. Over the last 5 years there have been a few studies
that tested the effects of errors in the training context. Frese, Brodbeck, Heinbokel,
Mooser, Schleiffenbaum, and Thiemann (1991) conducted one of the most
straightforward empirical tests of the ideas discussed above. With 24 German volunteers,
the experimenters tested two different training approaches for learning word processing.
One group received error-avoidant training where they were guided through word
processing tasks during the six hour training phase. The training materials in this
condition outlined the precise steps necessary to complete each task. When subjects in
this condition committed errors, the experimenter corrected the problem immediately
with no explanation.

The error training group was not provided with speciﬁc lists of operations and
was allowed to make errors. Subjects were encouraged to correct their own errors, and
were allowed three minutes to do so before having the situation corrected by the
experimenter without explanation. Trainees in this condition were told that errors are
opportunities and that they should try to learn from them. A list of learning “heuristics”
were displayed using an overhead projector urging trainees to pay attention to the screen
and keep trying to learn ﬁom confusing or difficult situations. This type of training has

been labeled error management training (EMT).

13
like reactions involve reducing effort and/or withdrawing from a task. This reduction in

effort can undermine training efforts in error-ﬁlled programs. If a trainee reduces effort,
withdrawing in some way from the training exercises, it is unlikely that the trainee will
learn.

EmpmcﬂﬁmdencefoLErmzs. Over the last 5 years there have been a few studies
that tested the effects of errors in the training context. F rese, Brodbeck, Heinbokel,
Mooser, Schleiffenbaum, and Thiemann (1991) conducted one of the most
straightforward empirical tests of the ideas discussed above. With 24 German volunteers,
the experimenters tested two different training approaches for learning word processing.
One group received error-avoidant training where they were guided through word
processing tasks during the six hour training phase. The training materials in this
condition outlined the precise steps necessary to complete each task. When subjects in
this condition committed errors, the experimenter corrected the problem immediately
with no explanation.

The error training group was not provided with speciﬁc lists of operations and
was allowed to make errors. Subjects were encouraged to correct their own errors, and
were allowed three minutes to do so before having the situation corrected by the
experimenter without explanation. Trainees in this condition were told that errors are
opportunities and that they should try to learn from them. A list of learning “heuristics”
were displayed using an overhead projector urging trainees to pay attention to the screen
and keep trying to learn from confusing or difﬁcult situations. This type of training has

been labeled error management training (EMT).

14
Testing conducted aﬁer all training was complete indicated that the EMT subjects

performed better on a free recall test and better at difﬁcult competence tests (although
error-free did just as well on easy and moderate competence tests). There were no
differences in transfer to non-trained tasks. While this study offers preliminary evidence
for the usefulness of errors, it provides a complex explanation for why the EMT subjects
outperformed the error-avoidant subjects.

First, subjects in the EMT condition were exhorted to think of errors as beneﬁcial
to the learning process. EMT subjects were told that errors and mistakes help the training
situation. This ﬁrst aspect of EMT should reduce the detrimental motivational effects of
errors, attenuating frustration, anxiety, and thoughts of task withdrawal. The study offers
some evidence in support of this explanation. Second, subjects were told to try to learn
from the errors they committed. EMT subjects were told to pay attention to the screen
and actively seek a way to correct an error situation. This second aspect of EMT should
increase the attention of trainees on task, particularly when errors occur. With increased
attention to the error states, the actions that lead to them, and the strategies to resolve
them, EMT subjects should gain more information from a given error situation than error-
avoidant trainees. Third, EMT trainees were forced to explore to a greater degree than
control subjects, because they were not provided with a list of how tasks should be
completed. While EMT was beneﬁcial overall, this study offers no hint as to whether the
difference in learning outcomes over the error-avoidant group was due to changes in
motivation, information, or a combination of these effects. A ﬁrst step in explaining the

effects of EMT would involve a more elaborate theoretical model of the psychological

1 5
and behavioral process factors that are triggered by errors. As of yet, such a model has

not been presented.

Further limitations of the Frese et al. (1991) study involve the sample, which
comprises 24 German citizens. This sample size was small and the results may differ
from a United States sample because German culture has a stronger emphasis on
perfection. The commission of errors and mistakes may be interpreted more severely by
Germans (Nordstrom, Wendland, & Williams, 1995).

In part to address the limitations in the sample, a more recent error training study
was conducted by Nordstrom and colleagues (1995). These experimenters evaluated
EMT training with a larger American sample. The results were similar to the Frese et al.
(1991) study because EMT improved task knowledge. However, as with the Frese et al.
(1990) study, there was little indication as to whether the EMT training helped maintain
motivation, improved the acquisition of important information, or both of these. In
addition, the explicit reactions of trainees to errors in these situations was not assessed,
making it difﬁcult to ascertain whether the training affected error perceptions or some
other psychological variable. More research is clearly needed to determine how trainees
react to errors and how these errors affect motivation and information acquisition. In
accomplishing this task, researchers must ﬁrst build theory by identifying how errors
affect learning. Then training strategies can be developed that directly inﬂuence this
process.

Empmcalﬁyidenchgamstﬁnnzs, While the studies reviewed above support the
use of error training, there is some literature that indicates that limiting the commission of

errors can improve learning. In particular, Carroll and Carrithers (1984) used a “training-

j 16
wheels” system of a commercial word processing program. The term “training wheels” is

used to indicate that the word processor was modiﬁed so that certain commands were
shut off from the user. Commands that often create errors for novices were blocked out
of the menus in order to avoid having the new users make errors. The rationale for
creating this system was that introductory users often end up ignoring manuals and
exploring irrelevant parts of the program.

In an experiment to test the training wheels system, 12 temporary workers with
little or no computer experience were split into training wheels and complete system
conditions. For both conditions, the task was to become familiar with the word
processing program, then type in and print a letter. After two hours on task, only 2
complete system subjects had typed in the letter, neither had printed it out. In the training
wheels system, all subjects had typed the letter and 4 were able to print it out. Trainees in
the training wheels task actually spent less time on task (overall) but generated
signiﬁcantly higher comprehension and task satisfaction scores. Error rates were not
different across conditions, but error recovery times were different, with training wheels
subjects spending less time recovering from errors.

One of the major conclusions from this paper is that training wheels designs
provide useful limits on user options. While hypothesis testing and discovery can be a
signiﬁcant means to improve learning, drastically unfamiliar situations can make errors
so distracting that is precludes learning. Delimiting the learning situation makes the
situation manageable for the learner, so she or he can engage in effective hypothesis
testing. This is the same goal of discovery learning studies that provide some guidance or

structure to the exploratory behavior of trainees.

17
An issue that was not considered by Carroll and Carrithers (1984), but is raised by

Frese and Altmann (1989) is the issue of avoiding the negative motivational
consequences of error commission. While subjects in both conditions of the Carroll and
Carrithers study had similar error rates, the errors committed by complete systems
subjects took longer to handle. It is likely that these subjects experienced greater distress
over their errors, resulting in decreased motivation to engage in the task. No data is
available from the Carroll and Carrithers study, however, to indicate how trainees viewed
their errors. Future research would beneﬁt from looking at the perceptions of errors, so
that researchers can estimate the motivational impact of error commission.

In summary, the Carroll and Carrithers (1984) work suggests that limiting the
focus of trainees may be useful for lowering the overwhelming amount of information
presented during early stages of learning a new skill. However, the use of a modiﬁed task
makes the “training wheels” approach unfeasible in many situations. There may be less
costly and less time consuming ways to restrict the trainees’ attention in order to avoid an
overload of information.

The research studies discussed above involve attempts to improve learning
outcomes in situations that involve many errors. While Frese and colleagues and Carroll
and Carrithers arrived at different solutions, each group of researchers was attempting to
avoid the potentially detrimental motivational effects of errors. However, neither group
provides for a full theoretical account for why their respective training strategies improve
learning. In all of the studies above, motivational and informational effects of errors are
combined in both the creation and evaluation of their training manipulations. In order to

disentangle these issues, the next section considers the role of errors as feedback. A

18
substantial literature exists regarding learning from feedback. To the extent that errors

provide feedback, this literature may provide a means to specify the motivational and
informational effects of errors.
Leaminaﬁcmﬁcedhack

Errors inﬂuence learning via the feedback that they convey to a trainee. This is
the informational result of errors that can provide for learning advantages over trainees
that do not receive or pay attention to error feedback. Unfortunately, the literature on
feedback is divided across disciplines and paradigms. There have been a number of
different reviews covering research in these different areas. Balzer, Doherty, and
O’Connor (1989) have reviewed feedback research from the multiple-cue probability
learning (MCPL) paradigm, an inductive reasoning task based on the Brunswik Lens
Model. Ilgen, Fisher, and Taylor (197 9) and Taylor, Fisher, and Ilgen (1984) reviewed
literature on feedback from a performance appraisal perspective. Kluger and DeNisi
(1996) recently reviewed literature on feedback as augmentation or intervention. In other
words, Kluger and DeNisi excluded those forms of feedback that are derived solely from
the task itself, which is the form of feedback of most interest in this thesis. While none
of these reviews deal explicitly with the issue of learning from errors, they do offer
information about the characteristics of feedback that may inﬂuence individual reactions
to it.

First, however, its important to note that these reviews identiﬁed a series of
studies that show people do not learn from outcome feedback on complex tasks (Azuma
& Cronbach, 1966; Hammond & Summer, 1972; Schmitt, Coyle, & Saari, 1977; for a

recent review see Balzer et al., 1989). Similarly, Jacoby, Mazursky, Troutman, and Kuss

l 9
(1984) found that seeking outcome feedback was negatively associated with performance.

However, this work has been completed under the MCPL paradigm where learning is a
completely inductive process. Much learning today is both inductive and deductive. For
example, learning a word processor involves generating, testing, and verifying
hypotheses. However, this process can be done either by trial-and-error QI by reference to
an information source. Hypotheses regarding task rules do not have to be tested solely by
interacting with the environment in an iterative trial-and-error learning process. Rather
than being implicit or hidden within the structure of the task, rules that govern systems
are often available in manuals, on-line documentation, or from expert advice.

In environments where learning can be both inductive and deductive, feedback
may serve to cue the trainee about when and where attention should be focused at
reference materials. In the words of Ivancic and Hesketh (1995), errors focus attention.
For example, if someone always has trouble with changing margins on their word
processor (an event that is salient to the author right now), that person can look up
margins in the software manual. More generally, if the screen on the computer always
becomes incomprehensible after a particular command is executed, the user has been
provided with information necessary to refer to a manual or expert regarding the use of
that command. Ordinarily, individuals do not have to rely on testing hypotheses over and
over again to uncover “latent” rules as they do in the MCPL paradigm. The rules are, at
least in part, laid out in manuals or by reference experts; these resources can be sought
when error feedback is perceived. This discussion serves to indicate that error feedback
on complex tasks can be useful for learning. The remainder of this section explores some

of the issues involved in translating error feedback into action that improves learning

20
outcomes. However, the idea that feedback can generate action requires a more in-depth

discussion of the characteristics of feedback.

Ilgen et al. (1979) present a conceptualization of feedback as a special case of the
general communication process where a sender conveys a message to a Leeipitmt. While
this is the deﬁnition adopted in this thesis, there are a number of boundary conditions
around the type of feedback under investigation here. These boundary conditions are
noted as each of the three components of feedback are discussed.

Senden First, while characteristics of the sender can affect individual reactions,
these are not the focus of the current study. In the training context, the sender is often
either the trainer/instructor or the task itself. When feedback is provided by an external
agent like a trainer or supervisor, characteristics of the source such as power over
rewards, credibility (Ilgen et al., 1979), and trust (Earley, 1988) become important
variables to consider in determining an individual’s reaction to feedback. Furthermore,
there is evidence that feedback presented by external agents has little effect on
performance, and can even have negative effects (Kluger & DeNisi, in press).

The focus of this review is on objective feedback conveyed directly from the task.
As a result, the sender is constant across subjects and is assumed to provide potentially
useful and trustworthy feedback. This has been shown to be the case for self-generated or
computer-generated feedback (Earley, 1988). The questions of usefulness and trust
would become more important if feedback was “yoked” or false, neither of which is the
case in the current study.

Reeeiyen Second, characteristics of the receiver provide important clues in

determining the effects of a feedback message. Three characteristics seem important to

21
isolate: Goal orientation, negative affectivity, and general cognitive ability. First, recent

research in educational psychology has indicated that learning orientation, or the degree
to which an individual values challenge and learning, signiﬁcantly affects how
individuals approach difﬁcult tasks (Bouffard, Boisvert, Vezeau, & Larouche, 1995;
Dweck, 1986, 1989; Elliot & Dweck, 1988). Similarly, performance orientation, or the
degree to which an individual values performance and achievement, signiﬁcantly affects
how individuals react to failure in achievement situations. While learning oriented
individuals view errors as challenge and show increased effort and persistence in the face
of adversity, performance oriented individuals are focused on demonstrating competence
and disengage from activities that are difﬁcult and hard to learn (Dweck, 1986, 1989).
This form of disengagement is similar to the well-researched phenomena of learned
helplessness, in which individuals withdraw a task after repeated negative feedback
(Dweck, 1986; Mikulincer, 1994).

Applying the notion of goal orientation to adult workplace leaming has been the
topic of a number of recent studies. Boyle and Klimoski (1995) and Kozlowski, Gully,
Smith, Nason, and Brown (1995) have conducted empirical tests to demonstrate the
usefulness of the goal orientation constructs in predicting training outcomes. Boyle and
Klimoski (1995), in a study on learning from a computer tutorial, demonstrated that
learning orientation was correlated positively with learning outcomes. Similarly,
Kozlowski and colleagues (1995) found that learning orientation facilitated the
acquisition of knowledge structure and self-efﬁcacy. High levels of performance

orientation were found to inhibit these learning outcomes. Similar results were found by

22
Ford et al. (1995) who reported that performance orientation was negatively related to

self-efficacy in training.

Although this perspective of goal orientation is dispositional, situational
inﬂuences have also been proposed. Kozlowski et al. (1995) used sequenced mastery
goal interventions to improve knowledge structure and self-efﬁcacy. This intervention
improved knowledge structure and self-efﬁcacy to a greater extent than performance
goals, even though performance goals appeared to facilitate basic task performance.
These situational manipulation effects were found independent of effects for individual
differences. Having measured learning and performance orientation as dispositions prior
to training, Kozlowski and colleagues demonstrated that traits were indeed relevant.
Furthermore, training with performance or mastery goals had independent, additive
effects beyond the effect of natural trait tendencies (Kozlowski et al., 1995).

However, the studies reviewed above do not provide direct evidence that the
adoption of learning or performance goals involves a different perspective on or
interpretation of the errors committed. Particularly in the study by Kozlowski and
colleagues, it is difficult to ascertain whether it was the mastery focus created by the
mastery goals, the sequence of goals, or the combination of the two that initiated the
differences in learning between the mastery and performance subjects. Disentangling the
motivational focus issue from the sequencing manipulation would provide a clearer
picture of how individuals react to error-ﬁlled training experiences.

Furthermore, direct evidence should be collected as to whether goal orientation
directly affects the perception of errors, as suggested by Dweck’s work. If this is indeed

the case, then it is clear that the process by which mastery or learning goals affect

22
Ford et al. (1995) who reported that performance orientation was negatively related to

self-efﬁcacy in training.

Although this perspective of goal orientation is dispositional, situational
inﬂuences have also been proposed. Kozlowski et al. (1995) used sequenced mastery
goal interventions to improve knowledge structure and self-efﬁcacy. This intervention
improved knowledge structure and self-efﬁcacy to a greater extent than performance
goals, even though performance goals appeared to facilitate basic task performance.
These situational manipulation effects were found independent of effects for individual
differences. Having measured learning and performance orientation as dispositions prior
to training, Kozlowski and colleagues demonstrated that traits were indeed relevant.
Furthermore, training with performance or mastery goals had independent, additive
effects beyond the effect of natural trait tendencies (Kozlowski et al., 1995).

However, the studies reviewed above do not provide direct evidence that the
adoption of learning or performance goals involves a different perspective on or
interpretation of the errors committed. Particularly in the study by Kozlowski and
colleagues, it is difficult to ascertain whether it was the mastery focus created by the
mastery goals, the sequence of goals, or the combination of the two that initiated the
differences in learning between the mastery and performance subjects. Disentangling the
motivational focus issue from the sequencing manipulation would provide a clearer
picture of how individuals react to error-ﬁlled training experiences.

Furthermore, direct evidence should be collected as to whether goal orientation
directly affects the perception of errors, as suggested by Dweck’s work. If this is indeed

the case, then it is clear that the process by which mastery or learning goals affect

23
learning outcomes involves feedback perception and interpretation. Ivancic and Hesketh

(1995) note, “Research in applied contexts is needed to determine if manipulating goal
orientation is an effective strategy for managing errors” (p. 144).

In short, both disposition and situational inﬂuences should be considered in future
research. The prediction from Dweck’s work would be that negative feedback for
learning oriented individuals is less demotivating. The opposite is expected for
performance oriented individuals. Yet, future research is needed both to manipulate goal
orientation without confounding other variables, and to measure the perceptions of errors
directly in order to ascertain whether this receiver characteristic affects the learning
process.

The second characteristic of the recipient that should be considered is the
propensity for negative thoughts and anxiety. Research has demonstrated that a primary
reason errors cause learning and performance decrements is through the creation of
anxiety and ﬁ'ustration (e.g., Ivancic & Hesketh, 1995; Mikulincer, 1989). Certain
individuals have been shown to experience events as more negative, and consequently
exhibit greater anxiety from them. These individuals have been labeled as having high
negative affectivity (NA, Watson, Clark, & Tellegen, 198 8). Individuals high in NA have
been shown to view the world more negatively and to experience greater levels of anxiety
and stress following negative feedback. As a result, these individuals may respond
differently to errors than individuals low in negative affectivity. This characteristic of the
recipient should also be measured in identifying the effects of errors in early skill

acquisition.

24
The third characteristic of the trainee that may affect feedback perceptions is

general cognitive ability. One would expect that cognitive ability would have an effect
on the ability of trainees to use information. Cognitive ability has been demonstrated as
an important inﬂuence on training outcomes (e.g., Ree & Earles, 1991, 1992) and one
cause for this may be the ability to utilize feedback during the learning process. There is
evidence that the effects of general cognitive ability on work samples at the end of
training occur through the acquisition of j ob knowledge (Ree, Carretta, & Teachout,
1995). Thus, individuals who have lower levels of general cognitive ability may have
more difﬁculty interpreting feedback and consequently gain less information from
feedback. This process may at least partially explain the differences in acquisition of job
knowledge noted by Ree and colleagues (1995). Thus it seems likely that individuals
high in general cognitive ability would perceive feedback as more important and more
useful.

Message. Finally, the message of the feedback is an important determinant of an
individual’s reaction to it. Ilgen et a1. (1979) mention three critical characteristics of
message: timing, frequency, and sign. In many training environments, the timing and
frequency of feedback is constant. All individuals receive their feedback around the same
time throughout the training. Consequently, the current focus is not on timing and
frequency. The notion of feedback sign, however, is germane to the topic of errors.
While one might assume that all errors convey only negative feedback, it is important to
consider that the sign of feedback depends on both the message and the receiver (Ilgen et
al., 1979; Kluger & DeNisi, in press). Two characteristics of feedback message are noted

below for their potential usefulness in studying the effects of feedback on learning. Brief

25
comments about how the individual differences reviewed above may affect perceptions of

these feedback characteristics are also provided below.

One of the criticisms leveled against current theories of feedback by Kluger and
DeNisi (in press) is the lack of an a priori means to identify the sign of feedback. The
reason for this is quite simple. The message conveyed by feedback about the “goodness”
or “badness” of performance depends on the goal of the recipient. In the current work,
the goal of the recipient is assumed to be this -- to lower the number of errors committed
on the task. In early skill acquisition, errors provide a clear indication that the trainee has
not learned the task yet. The goal of any trainee then is to demonstrate that they are
learning and performing well by lowering the number of errors they commit. Even given
this assumption, Kluger and DeNisi raise an important issue about the relativity of any
feedback message. As discussed below, this relativity can be assessed by directly
measuring perceived characteristics of the feedback.

The ﬁrst perception of the feedback message that is relevant here is the extent to
which a feedback message is positive or negative. The “magnitude” of the feedback
depends on the number of errors, certainly, but also on the characteristics of the recipient
as noted earlier. Unfortunately, little research has attempted to directly ascertain how
different individuals gauge or scale feedback in terms of positivity or negativity. In other
words, researchers have not stopped to ask learners with different characteristics, “What
did you think about that feedback? How good/bad do you think you did based on the
feedback you just received.”

There are at least two beneﬁts to this direct measurement approach for the

research process. First, instead of looking at feedback as dichotomous, positive or

26
negative, feedback is viewed along a continuum. Second, the psychological

interpretation of feedback is not left to a priori theoretical conjecture. By asking about
the perception of the magnitude of the feedback in terms of how good or how bad, the
researcher gains access to a construct that can be used model both the antecedents and
consequences of perceived feedback. In a leaming situation it is the psychological
interpretation of the feedback message that dictates subsequent behaviors. In a sense, the
assessment of positivity provides a direct psychological assessment of goal--performance
discrepancy, which has been shown to have important consequences in learning and
performance environments (Thomas & Mathieu, 1994). In summary, perceptions of the
feedback message can and should be measured in order to model the effects of feedback
on behavior. In the previous discussion, the “magnitude” or extent to which error
feedback is seen as negative is an important characteristic of the feedback message.
Characteristics of the recipient that may inﬂuence the judgment of negativity are
learning and performance orientation. As noted earlier, individuals who are more
learning oriented are hypothesized to experience negative feedback as less demotivating
and more challenging. Although learning and performance orientations tend to be
uncorrelated (e. g., Kozlowski et al., 1995), a similar result is expected for individuals
with a low performance orientation. One explanation for this ﬁnding would be that high
learning oriented individuals and low performance oriented individuals view a given
episode of negative feedback as less negative others. As a result of this possibility, the
effects of goal orientation should be considered when measuring perceptions of feedback
magnitude. Furthermore, training should manipulate goal orientation in order to reduce

the perceived negativity of error feedback.

27
Another characteristic of the recipient that may inﬂuence the interpretation of

feedback negativity is negative affectivity. Individuals high in NA have been shown to
experience greater levels of anxiety and rumination following exposure to negative
feedback (Watson, Clark, & Tellegen, 1988). One explanation for this ﬁnding is that
individuals high in NA are more likely to view a given episode of feedback as more
negative than individuals low in NA. Given the likelihood of such differences, it would
be worthwhile to consider differences in NA when measuring perceptions of the
magnitude of negative feedback.

Another important characteristic of the feedback message is its information value
(Ilgen et al., 1979). Ilgen et al. (1979) note that feedback is only information. The
usefulness of the information depends on both the nature of the feedback and on the
recipient. If all trainees are given a similar form and similar quantity of feedback, the
meaning of that feedback is ascertained solely by the trainee. A trainee’s ability and past
experience with the task inﬂuences how useful the trainee judges the feedback.

General cognitive ability, as noted earlier, should inﬂuence the extent to which
feedback is judged as useful. Individuals with higher levels of cognitive ability have
been shown to learn more from given training episodes, and at least one reason for this
may be their perception of the importance of feedback. Having judged feedback as
useful, higher cognitive ability should allow a trainee to assimilate more information
from a given feedback message (Ackerman, 198 8).

Similarly, the more a trainee’s attention is focused, the less likely she is to
become overloaded with information. That is, individuals who only focus on a single

facet of a task, and the feedback relevant to that facet, should be able to interpret that

28
feedback more easily. Within a limited time frame, one individual can only perceive and

interpret a ﬁnite amount of information -- this is the basic argument of attentional
resource capacity (e.g., Kanfer & Ackerrnan, 1989). Similarly, Goldstein (1993) and
Ilgen et al. (1979) both note that too much feedback can be confusing. One way
instructors or trainers prevent information from becoming overwhelming is to ask
trainees to attend to only portions of the task information that is available. This is the
approach advocated by training wheels strategies (Carroll & Carrithers, 1984) and
discovery learning advocates who include sequencing or guidance (e.g., Singer & Pease,
1976). To study the effectiveness of these strategies, though, cognitive ability should be
assessed as it likely affects the usefulness of feedback for self-diagnosis of learning and
performance.

As with the magnitude, the perception of the usefulness of feedback can and
should be measured in order to ascertain how each individual perceives the feedback
received. In the current study, feedback that is useful is termed “diagnostic” because that
term captures the notion that feedback can be used to diagnosis problems in learning and
direct attention to ﬁx those problems. Thus diagnosticity, when trainees are given similar
amounts of information as feedback, depends largely on the ability and current focus of
the trainee. As a result, perceptions of diagnosticity may provide clues into whether or
not feedback messages are attended to and used to guide effort and attention during the
training process.

This discussion still leaves unanswered how negative feedback inﬂuences
learning. From an informational perspective, trainees must allocate attention to those

aspects of the task that have not been well-learned. This means perceiving, interpreting,

28
feedback more easily. Within a limited time frame, one individual can only perceive and

interpret a ﬁnite amount of information -- this is the basic argument of attentional
resource capacity (e.g., Kanfer & Ackerrnan, 1989). Similarly, Goldstein (1993) and
Ilgen et al. (1979) both note that too much feedback can be confusing. One way
instructors or trainers prevent information from becoming overwhelming is to ask
trainees to attend to only portions of the task information that is available. This is the
approach advocated by training wheels strategies (Carroll & Carrithers, 1984) and
discovery learning advocates who include sequencing or guidance (e.g., Singer & Pease,
1976). To study the effectiveness of these strategies, though, cognitive ability should be
assessed as it likely affects the usefulness of feedback for self-diagnosis of learning and
performance.

As with the magnitude, the perception of the usefulness of feedback can and
should be measured in order to ascertain how each individual perceives the feedback
received. In the current study, feedback that is useful is termed “diagnostic” because that
term captures the notion that feedback can be used to diagnosis problems in learning and
direct attention to ﬁx those problems. Thus diagnosticity, when trainees are given similar
amounts of information as feedback, depends largely on the ability and current focus of
the trainee. As a result, perceptions of diagnosticity may provide clues into whether or
not feedback messages are attended to and used to guide effort and attention during the
training process.

This discussion still leaves unanswered how negative feedback inﬂuences
learning. From an informational perspective, trainees must allocate attention to those

aspects of the task that have not been well-learned. This means perceiving, interpreting,

29
and utilizing error feedback to drive the attentional focus of learning efforts. The

perception of diagnosticity should be an important antecedent to this type of attentional
focus. When trainees judge feedback as diagnostic, they are more likely to use that
feedback to drive their attention.

From a motivational perspective, people who receive negative feedback know that
they are not performing the task well. Thus, these individuals have to decide whether to
exert more or less effort as a reaction to this feedback. The perception of feedback
magnitude would seem to be an important antecedent to the decision of whether to
engage in effortful thinking about the task. In general, it would seem that greater levels
of negative feedback (i.e., higher magnitude of negative feedback) would lead trainees to
disengage from their learning attempts and apply less effort. Each of these processes are
discussed in greater detail in the next sessions.

The notion of attention is not well deﬁned in the training literature. The most
common notion of attention is that presented by Gagne et al. (1992). In their 9 events of
instruction, Gagne and colleagues list “gaining attention” as the ﬁrst event. Gaining
attention involves appealing to the leamer’s interests or using rapidly changing stimuli.
The notion of attention, however, goes beyond a one-time event of attracting the trainee’s
mental focus to the task at hand.

Attentional focus is simply the information about which trainees are currently
thinking. A rough, but useful, distinction made in much of the literature is between on-
task and off-task cognitions (e.g., Fisher, 1995; Kanfer & Ackerman, 1989). On-task

cognitions involves attentional focus on material that is directly relevant to the to-be-

3O
learned material. Off-task cognitions are thoughts and concerns that have no immediate

relevance to the task at hand. In general, one would expect that focusing attention on-
task should provide for the greatest learning beneﬁts. This has been demonstrated in a
number of studies (Fisher, 1995; Kanfer & Ackerman, 1989).

The purpose of training sequencing is to narrow the attention of the trainee more
speciﬁcally. Sequencing involves narrowing trainees’ attention on speciﬁc portions or
facets of task relevant information. Generally deﬁned, sequencing is the organization of
subunits in a training program (Gagne et al., 1992). Reviews of instructional sequencing
(Reiguluth & Curtis, 1987; Reiguluth & Stein, 1983) have indicated that almost all
sequencing involves a movement of trainee focus from simple to more complex material.

Different traditions provide different dimensions on which to judge task
complexity. For example Anderson’s ACT" model suggests that verbal or declarative
knowledge should (and must) proceed the presentation (and acquisition) of procedural or
action-based knowledge (Anderson, 1983). Simple facts, terms, and ideas must be
understood before an individual can use those ideas to generate action. Elaboration
theory (Reiguluth & Curtis, 1987) suggests that the most basic unit of action or
knowledge should be presented ﬁrst. From this core, elaborations or different
permutations are presented in succession. In other words, elaboration theory suggests
that a simple foundation should be built before exceptions, counter-examples, or
differentiating complexities are explored. In general then, sequencing is simply a means
of focusing attention to different areas of study with the ultimate goal of accomplishing

the training objective.

 

 

3 1
In order to accomplish the ﬁnal training objectives, pre-requisite capabilities must

be built through-out the course of instruction. That is, the goal of sequencing is to build
the basic skills (i.e., simple skills) necessary to move on to the more complex skills for
which training is attempting to build proﬁciency. For example, for a person to learn how
to understand and solve a calculus problem, that person must both understand the
principles of calculus and be able to utilize basic algebra skills to solve simple equations.
Learning how to solve simple algebra equations such as 3x + 5 = 14 requires previously
acquired skills in basic math functions like subtraction and division. Pre-requisite
knowledge must be built by focusing trainee attention on simple math skills before
moving to more complex skills like how derive an integral.

Sequencing, then, is the traditional training solution to the same problem
addressed by Carroll and Canithers (1988) “training wheels” system. Carroll and
Carrithers were trying to limit the focus of trainee’s attention in order to prevent
information overload and consequent withdrawal from the learning attempt. They
modiﬁed a system so trainees could only focus on those aspects of the task that were
most relevant to the to-be-learned task. As noted earlier, however, modifying the task is a
costly solution, one that involves time and money. Furthermore, there is the possible
danger of the trainee missing out on important information or skills because the task has
been modiﬁed.

This focusing of attention via sequencing should directly affect the perception of
feedback received. As noted earlier, limiting the attention of trainees should allow for
unambiguous interpretation of the feedback. If trainee’s attention is focused on all

aspects of the task, there will simply be too much feedback presented for it to be

 

3 1
In order to accomplish the ﬁnal training objectives, pre-requisite capabilities must

be built through-out the course of instruction. That is, the goal of sequencing is to build
the basic skills (i.e., simple skills) necessary to move on to the more complex skills for
which training is attempting to build proﬁciency. For example, for a person to learn how
to understand and solve a calculus problem, that person must both understand the
principles of calculus and be able to utilize basic algebra skills to solve simple equations.
Learning how to solve simple algebra equations such as 3x + 5 = 14 requires previously
acquired skills in basic math functions like subtraction and division. Pre-requisite
knowledge must be built by focusing trainee attention on simple math skills before
moving to more complex skills like how derive an integral.

Sequencing, then, is the traditional training solution to the same problem
addressed by Carroll and Carrithers (1988) “training wheels” system. Carroll and
Carrithers were trying to limit the focus of trainee’s attention in order to prevent
information overload and consequent withdrawal from the learning attempt. They
modiﬁed a system so trainees could only focus on those aspects of the task that were
most relevant to the to-be-learned task. As noted earlier, however, modifying the task is a
costly solution, one that involves time and money. Furthermore, there is the possible
danger of the trainee missing out on important information or skills because the task has
been modiﬁed.

This focusing of attention via sequencing should directly affect the perception of
feedback received. As noted earlier, limiting the attention of trainees should allow for
unambiguous interpretation of the feedback. If trainee’s attention is focused on all

aspects of the task, there will simply be too much feedback presented for it to be

32
meaningful. Thus sequencing should directly improve the diagnosticity of feedback by

pushing the trainee to limit their efforts on the task.

While sequencing should help trainees judge feedback as more diagnostic, it does
not necessarily have any effect on the level of on- and off-task attention. Sequencing is
designed to move a trainee from subject x to subject y and ﬁnally to subject 2. What this
movement does not demand is greater on—task attention than someone who is provided
with training that is not sequenced. Simply because a trainee is told to focus on different
information does not suggest that the focus will last longer or will be less susceptible to
interference from off-task cognitions. However, sequencing may improve on—task
attention indirectly.

Feedback that is highly diagnostic provides trainees with unambiguous
information about their skills. This information can be used to focus their attention on
improving the shortcomings with their skills. So trainees who perceive their error
feedback as diagnostic are more likely to use that information to attend to on—task
information about how to remedy the problem identiﬁed by the feedback. On the other
hand, trainees presented with information that is perceived to be useless or ambiguous
should have no speciﬁc focus for their follow-up study efforts. These trainees
consequently spend less time using information resources in follow-up study time. As a
result of these different reactions to diagnosticity of feedback, research should ﬁnd that
sequencing affects feedback perceptions directly, but only affects on-task attentional
focus indirectly, via those feedback perceptions.

While speciﬁc methods of sequencing are not compared here, a basic simple to

complex sequencing is used to test whether the above feedback effect occurs. Although

32
meaningful. Thus sequencing should directly improve the diagnosticity of feedback by

pushing the trainee to limit their efforts on the task.

While sequencing should help trainees judge feedback as more diagnostic, it does
not necessarily have any effect on the level of on- and off-task attention. Sequencing is
designed to move a trainee from subject 2; to subject y and ﬁnally to subject 2. What this
movement does not demand is greater on-task attention than someone who is provided
with training that is not sequenced. Simply because a trainee is told to focus on different
information does not suggest that the focus will last longer or will be less susceptible to
interference from off-task cognitions. However, sequencing may improve on-task
attention indirectly.

Feedback that is highly diagnostic provides trainees with unambiguous
information about their skills. This information can be used to focus their attention on
improving the shortcomings with their skills. So trainees who perceive their error
feedback as diagnostic are more likely to use that information to attend to on—task
information about how to remedy the problem identified by the feedback. On the other
hand, trainees presented with information that is perceived to be useless or ambiguous
should have no speciﬁc focus for their follow-up study efforts. These trainees
consequently spend less time using information resources in follow-up study time. As a
result of these different reactions to diagnosticity of feedback, research should ﬁnd that
sequencing affects feedback perceptions directly, but only affects on-task attentional
focus indirectly, via those feedback perceptions.

While speciﬁc methods of sequencing are not compared here, a basic simple to

complex sequencing is used to test whether the above feedback effect occurs. Although

 

33
there is considerable evidence that sequencing makes learning easier (e.g., Gagne et al.,

1992), it is unclear how sequencing affects perceptions of errors and the feedback
inherent in them. More speciﬁcally, the aim of this research is to ascertain whether
sequencing affects the diagnosticity of feedback, which seems likely to inﬂuence the
attentional focus of trainees during skill acquisition.

I I . . . I . .

Some research suggests that individuals who receive negative feedback are likely
to exert more effort than those who receive positive feedback because positive feedback
indicates that the status quo can be maintained or that further effort is unnecessary
(Anderson & Rodin, 1989; Podsakoff & F arh, 1989; Waldersee & Luthans, 1994). Other
research suggests that exposure to negative feedback can result in a reduction of
motivation and on-task effort (Mikulincer, 1994). How can these contradictory results be
remedied in order to render a prediction in the current paradigm?

Kluger and DeNisi (1996) suggest four strategies that people use to address
negative feedback, based on a cybernetic model of discrepancy reduction: 1) Increase
effort, 2) abandon the standard, 3) change the standard, 4) reject the feedback message.
From an operational perspective, the last three strategies manifest themselves with very
similar behaviors. Individuals who abandon, change, or reject the standard and/or the
feedback generally decrease their effort on the task in question. The similarity of the
behavioral manifestation of these three outcomes makes it difﬁcult to evaluate the
conceptual distinctiveness of each of these strategies. Thus, the key theoretical question,
and practical issue for trainers, is whether individuals work harder after being provided

negative feedback.

 

 

33
there is considerable evidence that sequencing makes learning easier (e.g., Gagne et al.,

1992), it is unclear how sequencing affects perceptions of errors and the feedback
inherent in them. More speciﬁcally, the aim of this research is to ascertain whether
sequencing affects the diagnosticity of feedback, which seems likely to inﬂuence the
attentional focus of trainees during skill acquisition.

Some research suggests that individuals who receive negative feedback are likely
to exert more effort than those who receive positive feedback because positive feedback
indicates that the status quo can be maintained or that further effort is unnecessary
(Anderson & Rodin, 1989; Podsakoff & Farh, 1989; Waldersee & Luthans, 1994). Other
research suggests that exposure to negative feedback can result in a reduction of
motivation and on-task effort (Mikulincer, 1994). How can these contradictory results be
remedied in order to render a prediction in the current paradigm?

Kluger and DeNisi (1996) suggest four strategies that people use to address
negative feedback, based on a cybernetic model of discrepancy reduction: 1) Increase
effort, 2) abandon the standard, 3) change the standard, 4) reject the feedback message.
From an operational perspective, the last three strategies manifest themselves with very
similar behaviors. Individuals who abandon, change, or reject the standard and/or the
feedback generally decrease their effort on the task in question. The similarity of the
behavioral manifestation of these three outcomes makes it difficult to evaluate the
conceptual distinctiveness of each of these strategies. Thus, the key theoretical question,
and practical issue for trainers, is whether individuals work harder after being provided

negative feedback.

 

33
there is considerable evidence that sequencing makes learning easier (e. g., Gagne et al.,

1992), it is unclear how sequencing affects perceptions of errors and the feedback
inherent in them. More speciﬁcally, the aim of this research is to ascertain whether
sequencing affects the diagnosticity of feedback, which seems likely to inﬂuence the
attentional focus of trainees during skill acquisition.

I I . . . I . .

Some research suggests that individuals who receive negative feedback are likely
to exert more effort than those who receive positive feedback because positive feedback
indicates that the status quo can be maintained or that further effort is unnecessary
(Anderson & Rodin, 1989; Podsakoﬁ & F arh, 1989; Waldersee & Luthans, 1994). Other
research suggests that exposure to negative feedback can result in a reduction of
motivation and on-task effort (Mikulincer, 1994). How can these contradictory results be
remedied in order to render a prediction in the current paradigm?

Kluger and DeNisi (1996) suggest four strategies that people use to address
negative feedback, based on a cybernetic model of discrepancy reduction: 1) Increase
effort, 2) abandon the standard, 3) change the standard, 4) reject the feedback message.
From an operational perspective, the last three strategies manifest themselves with very
similar behaviors. Individuals who abandon, change, or reject the standard and/or the
feedback generally decrease their effort on the task in question. The similarity of the
behavioral manifestation of these three outcomes makes it difficult to evaluate the
conceptual distinctiveness of each of these strategies. Thus, the key theoretical question,
and practical issue for trainers, is whether individuals work harder after being provided

negative feedback.

 

34
In the current paradigm, the most common reaction is likely to be to a reduction

of effort via one of the three mechanisms above. At the early stages of skill acquisition,
individuals are faced with a novel, difﬁcult task. The feedback received is almost entirely
negative, indicating serious problems with the individuals capability to learn and perform
the task. Because the individual has not had prior exposure to this task, early time-on-
task provides one of the best means to judge capability.

If low levels of perceived capability result from error feedback, then individuals
see little beneﬁt in continuing efforts to learn the task. The question for an individual
then becomes whether the negative feedback is judged as something the individual feels
he or she is capable of transcending. A traditional approach to understanding this issue is
to measure attributions of causation, controllability, or stability (e.g., Thomas & Mathieu,
1994; Weiner, 1985, 1986). An alternative approach involves capturing how negative the
perception of feedback appears. Individuals who perceive high levels of negative
feedback are more likely to judge that feedback is indicating lower levels of capability.

However, because of the evidence supporting attribution theory, an attempt should
be made to ascertain whether attributional processes are affecting how individuals
interpret their feedback. This can be accomplished within this theoretical framework by
construing magnitude of negative feedback in two distinct ways. First, magnitude can be
purely descriptive, as suggested above. This would indicate the extent to which feedback
is highly positive or highly negative. Second, magnitude can be assessed from an
attributional perspective. This would indicate the extent to which feedback is highly
positive or highly negative and reﬂects stable, internal factors. Although this perspective

combines across dimensions of attribution noted by Weiner (1986), it provides a simple

 

34
In the current paradigm, the most common reaction is likely to be to a reduction

of effort via one of the three mechanisms above. At the early stages of skill acquisition,
individuals are faced with a novel, difﬁth task. The feedback received is almost entirely
negative, indicating serious problems with the individuals capability to learn and perform
the task. Because the individual has not had prior exposure to this task, early time-on-
task provides one of the best means to judge capability.

If low levels of perceived capability result ﬁ'om error feedback, then individuals
see little beneﬁt in continuing efforts to learn the task. The question for an individual
then becomes whether the negative feedback is judged as something the individual feels
he or she is capable of transcending. A traditional approach to understanding this issue is
to measure attributions of causation, controllability, or stability (e.g., Thomas & Mathieu,
1994; Weiner, 1985, 1986). An alternative approach involves capturing how negative the
perception of feedback appears. Individuals who perceive high levels of negative
feedback are more likely to judge that feedback is indicating lower levels of capability.

However, because of the evidence supporting attribution theory, an attempt should
be made to ascertain whether attributional processes are affecting how individuals
interpret their feedback. This can be accomplished within this theoretical framework by
construing magnitude of negative feedback in two distinct ways. First, magnitude can be
purely descriptive, as suggested above. This would indicate the extent to which feedback
is highly positive or highly negative. Second, magnitude can be assessed from an
attributional perspective. This would indicate the extent to which feedback is highly
positive or highly negative and reﬂects stable, internal factors. Although this perspective

combines across dimensions of attribution noted by Weiner (1986), it provides a simple

 

35
means of assessing whether causal judgments affect perceptions of magnitude. Further

the combination of intemality and stability provides an indicator of whether trainees feel
that current performance could improve with time and effort. Thus, this study
distinguishes between attributional magnitude of negative feedback and descriptive
magnitude of negative feedback. Both of these magnitude constructs are expected to
inﬂuence perceptions of task capability.

Perceived task capability has been measured in a number of ways, but most
recently the focus has been on self-efﬁcacy judgments. Self-efficacy is the overall
assessment an individual makes about his or her task-relevant capability (Bandura, 1986,
1991; Gist & Mitchell, 1992). As one form of expectancy (e.g., Schunk, 1991), self-
efficacy serves as an indication of whether the individual feels capable of performing the
task. Perceptions of capability in the form of self-efﬁcacy have been linked to
persistence, effort, and strategy use (Bandura, 1977, 1986, 1991; Latham & Locke, 1991;
Schunk, 1991). Thus, high levels of self-efﬁcacy would be beneﬁcial in eliciting and
maintaining high levels of motivation during training. Very high levels of negative
feedback, such as those encountered during initial attempts at learning a new task, should
lead to low perceptions of self-efficacy. A goal of training interventions in early stages of
skill acquisition should then be to lower the perceived magnitude of negative feedback in
order to avoid lowering self-efﬁcacy. Self-efficacy, in turn should affect the overall
motivation level of trainees. More speciﬁcally, higher levels of capability on a task may
inﬂuence the extent to which individuals value learning about that task.

Interest in the effects of motivation has become very popular in the training

literature in recent years (Campbell, 1988; 1989; Goldstein, 1993). Although a number

35
means of assessing whether causal judgments affect perceptions of magnitude. Further

the combination of intemality and stability provides an indicator of whether trainees feel
that current performance could improve with time and effort. Thus, this study
distinguishes between attributional magnitude of negative feedback and descriptive
magnitude of negative feedback. Both of these magnitude constructs are expected to
inﬂuence perceptions of task capability.

Perceived task capability has been measured in a number of ways, but most
recently the focus has been on self-efficacy judgments. Self-efficacy is the overall
assessment an individual makes about his or her task-relevant capability (Bandura, 1986,
1991; Gist & Mitchell, 1992). As one form of expectancy (e.g., Schunk, 1991), self-
efﬁcacy serves as an indication of whether the individual feels capable of performing the
task. Perceptions of capability in the form of self-efﬁcacy have been linked to
persistence, effort, and strategy use (Bandura, 1977, 1986, 1991; Latham & Locke, 1991;
Schunk, 1991). Thus, high levels of self-efﬁcacy would be beneﬁcial in eliciting and
maintaining high levels of motivation during training. Very high levels of negative
feedback, such as those encountered during initial attempts at learning a new task, should
lead to low perceptions of self-efﬁcacy. A goal of training interventions in early stages of
skill acquisition should then be to lower the perceived magnitude of negative feedback in
order to avoid lowering self-efﬁcacy. Self-efficacy, in turn should affect the overall
motivation level of trainees. More speciﬁcally, higher levels of capability on a task may
inﬂuence the extent to which individuals value learning about that task.

Interest in the effects of motivation has become very popular in the training

literature in recent years (Campbell, 1988; 1989; Goldstein, 1993). Although a number

 

36
of different techniques for conceptualizing and measuring motivation are available, recent

work has concentrated on the construct of motivation to learn. Early deﬁnitions of
motivation to learn, posed by Noe (1986) and Noe and Schmitt (1986), suggest that it is
the degree to which an individual desires to learn from training. The deﬁnition used here
reﬂects this early tradition, with motivation to learn deﬁned as the extent to which an
individual wants to learn about the task. In the terms of expectancy theory, motivation to
learn reﬂects the value placed on learning the task in question. This deﬁnition suggests
that individuals who are high in motivation to learn are more likely to spend time
attending to those activities that lead to changes in internal knowledge states as opposed
to activities that satisfy external task demands. This concept is distinct from on- and off-
task attention as it implies greater effort for a similar level of on-task attention and the use
of learning strategies such as mnemonics and rehearsal that are commonly used to learn
new material.

A number of recent studies have studied the effects of motivation on learning
outcomes. Hicks and Klimoski (1987) used overall measures of motivation but found
little effect for motivation on a ﬁnal role play and test performance. Their study,
however, showed few signiﬁcant predictors of these criteria, indicating possible
contamination problems. Similarly, Tannenbaum, Mathieu, Salas, and Cannon-Bowers
(1991) tested the effects of various training characteristics on attitudinal outcomes of
training. While it was not the focus of their study, they did ﬁnd signiﬁcant relationships
between training motivation and 3 of 5 training outcome variables. One of these was in
negative direction, opposite what one might predict. The variables that were not

predicted well were honors and demerits, outcomes that likely have signiﬁcant inﬂuences

 

 

37
from sources external to the individual. Motivation should not be expected to have a

signiﬁcant effect on these types of criteria. The Tannenbaum et al. (1991) study did ﬁnd
that test performance, a variable that is more likely to be inﬂuenced by motivational
differences, was signiﬁcantly related (in the predicted direction) to training motivation.
As a result of these frndings, future research on training motivation would beneﬁt from
more careful attention to the criteria used. For example, in a carefully designed study of
educational administrators, Noe and Schmitt (1986) found a small but signiﬁcant effect
for pro-training motivation on learning outcomes.

Mathieu and Martineau (in press) note that different criteria might be predicted
differently by different measures of motivation. In particular, they note that some
motivational variables have direct effects on learning outcomes while others will have
indirect effects. More speciﬁcally, these authors note that self-efﬁcacy measures are
likely to affect pre-training motivation measures, while expectancy based motivation
measures, similar to the deﬁnition espoused here, will predict work outcomes. In other
words, self-efficacy predicts overall motivation level and it is this motivating force that
predicts performance.

Martocchio (1994) suggests this relationship exists in the computer training when
he states, “Self-efﬁcacy beliefs are expected to directly inﬂuence motivation...” (p.820).
His results suggest this effect, although motivation is not a measured variable.
Martocchio found differences in declarative knowledge test performance were related to
efficacy levels. Other studies, however, have indicated a non-signiﬁcant effect of self-

effrcacy on learning (Thomas & Mathieu, 1994).

 

37
from sources external to the individual. Motivation should not be expected to have a

signiﬁcant effect on these types of criteria. The Tannenbaum et al. (1991) study did ﬁnd
that test performance, a variable that is more likely to be inﬂuenced by motivational
differences, was signiﬁcantly related (in the predicted direction) to training motivation.
As a result of these findings, future research on training motivation would beneﬁt from
more careful attention to the criteria used. For example, in a carefully designed study of
educational administrators, Noe and Schmitt (1986) found a small but signiﬁcant effect
for pre-training motivation on leaming outcomes.

Mathieu and Martineau (in press) note that different criteria might be predicted
differently by different measures of motivation. In particular, they note that some
motivational variables have direct effects on learning outcomes while others will have
indirect effects. More speciﬁcally, these authors note that self-efficacy measures are
likely to affect pre-training motivation measures, while expectancy based motivation
measures, similar to the deﬁnition espoused here, will predict work outcomes. In other
words, self-efﬁcacy predicts overall motivation level and it is this motivating force that
predicts performance.

Martocchio (1994) suggests this relationship exists in the computer training when
he states, “Self-efﬁcacy beliefs are expected to directly inﬂuence motivation...” (p.820).
His results suggest this effect, although motivation is not a measured variable.
Martocchio found differences in declarative knowledge test performance were related to
efficacy levels. Other studies, however, have indicated a non-signiﬁcant effect of self-

efﬁcacy on learning (Thomas & Mathieu, 1994).

 

 

37
from sources external to the individual. Motivation should not be expected to have a

signiﬁcant effect on these types of criteria. The Tannenbaum et al. (1991) study did ﬁnd
that test performance, a variable that is more likely to be inﬂuenced by motivational
differences, was signiﬁcantly related (in the predicted direction) to training motivation.
As a result of these ﬁndings, future research on training motivation would beneﬁt from
more careful attention to the criteria used. For example, in a carefully designed study of
educational administrators, Noe and Schmitt (1986) found a small but signiﬁcant effect
for pre-training motivation on learning outcomes.

Mathieu and Martineau (in press) note that different criteria might be predicted
differently by different measures of motivation. In particular, they note that some
motivational variables have direct effects on learning outcomes while others will have
indirect effects. More speciﬁcally, these authors note that self-efﬁcacy measures are
likely to affect pre-training motivation measures, while expectancy based motivation
measures, similar to the deﬁnition espoused here, will predict work outcomes. In other
words, self-efficacy predicts overall motivation level and it is this motivating force that
predicts performance.

Martocchio (1994) suggests this relationship exists in the computer training when
he states, “Self-efficacy beliefs are expected to directly inﬂuence motivation...” (p.820).
His results suggest this effect, although motivation is not a measured variable.
Martocchio found differences in declarative knowledge test performance were related to
efficacy levels. Other studies, however, have indicated a non-signiﬁcant effect of self-

efﬁcacy on learning (Thomas & Mathieu, 1994).

 

38
The contradiction in these ﬁndings seems to be due to a difference in the variables

included in the studies. Studies that do not show direct effects for self-efﬁcacy control
for self-set or personal goals (e.g., Thomas & Mathieu, 1994). Personal goals, like
motivation to learn, provide a measure of motivation for subsequent on-task behavior.
Thus these studies suggest that self-efﬁcacy inﬂuences are mediated by the motivation
level (whether ascertained as self-set goal or as motivation level) which itself is the
primary determinant of learning outcomes. Martocchio (1994) found a direct effect for
self-efficacy because, while he controlled for expectancies, he did not control for a global
assessment of motivation. Motivation captures the desire to mobilize the capability
assessment inherent in self-efficacy judgments. Why should an individual who is capable
always perform better than an individual who is less capable? Capability may result in a
desire to learn, but it does not directly imply a change in learning outcomes.

Motivation to learn, in turn, should inﬂuence the cognitive and skill-based
learning outcomes through an interaction with attentional focus. The combination of high
levels of motivation to engage in learning activities and high levels of attention on task
should increase learning over individuals who possess neither. Motivation to learn
should directly inﬂuence affective learning outcomes. If a trainee is motivated to learn,
task satisfaction and task withdrawal should be directly affected by this attitude. While
changes in knowledge and skill require the mobilization of effort via a motivating force, a
change in attitudes can result directly from a desire to do so.

I . . D
On the criterion side, recent work has emphasized that learning outcomes are

multi-dimensional. Most recently, Kraiger, Ford, and Salas (1993) suggest three primary

38
The contradiction in these ﬁndings seems to be due to a difference in the variables

included in the studies. Studies that do not show direct effects for self-efficacy control
for self-set or personal goals (e.g., Thomas & Mathieu, 1994). Personal goals, like
motivation to learn, provide a measure of motivation for subsequent on—task behavior.
Thus these studies suggest that self-efficacy inﬂuences are mediated by the motivation
level (whether ascertained as self-set goal or as motivation level) which itself is the
primary determinant of learning outcomes. Martocchio (1994) found a direct effect for
self-efﬁcacy because, while he controlled for expectancies, he did not control for a global
assessment of motivation. Motivation captures the desire to mobilize the capability
assessment inherent in self-efﬁcacy judgments. Why should an individual who is capable
always perform better than an individual who is less capable? Capability may result in a
desire to learn, but it does not directly imply a change in leaming outcomes.

Motivation to learn, in turn, should inﬂuence the cognitive and skill-based
learning outcomes through an interaction with attentional focus. The combination of high
levels of motivation to engage in learning activities and high levels of attention on task
should increase learning over individuals who possess neither. Motivation to learn
should directly inﬂuence affective learning outcomes. If a trainee is motivated to learn,
task satisfaction and task withdrawal should be directly affected by this attitude. While
changes in knowledge and skill require the mobilization of effort via a motivating force, a
change in attitudes can result directly from a desire to do so.

I . . D
On the criterion side, recent work has emphasized that learning outcomes are

multi-dirnensional. Most recently, Kraiger, Ford, and Salas (1993) suggest three primary

39
categories for training evaluation: Cognitive, skill-based, and affectively-based

outcomes.

Cognitive learning outcomes refer to variables that represent the quantity and type
of knowledge available to the trainee. The traditional tests of cognitive learning are
achievement tests of verbal knowledge. While tests of verbal knowledge have been
criticized for being unable to discriminate among learners at higher levels of
development, they are appropriate for early stages of skill acquisition (Kraiger et al.,
1993). Measures of verbal knowledge can be taken using traditional pencil-and-paper
questions. Other cognitive outcomes include knowledge organization and cognitive
strategies. Both of these outcomes, however, are more likely to develop in later stages of
the skill acquisition process.

Skill-based learning outcomes also tend to reﬂect later stages of learning.
According to Kraiger et al. (1993) the two major components of skill-based outcomes are
compilation and automization. Compilation involves the combination of discrete
behaviors into domain-speciﬁc routines that are relatively fast and efﬁcient. During this
stage errors are reduced, verbal rehearsal is eliminated, and behavior is more task-
focused. During early stages of skill acquisition this may take place and be ascertained
by observations of performance. Similarly, task outputs of products or performance can
be examined for evidence that compilation is beginning to occur -- fewer errors and faster
production or reaction time.

Automaticity, on the other hand, involves an even greater level of skill.
Automaticity implies that tasks or portions of task can be handled without conscious

monitoring. The lack of monitoring frees cognitive resources to engage in other

39
categories for training evaluation: Cognitive, skill-based, and affectively-based

outcomes.

Cognitive learning outcomes refer to variables that represent the quantity and type
of knowledge available to the trainee. The traditional tests of cognitive learning are
achievement tests of verbal knowledge. While tests of verbal knowledge have been
criticized for being unable to discriminate among learners at higher levels of
development, they are appropriate for early stages of skill acquisition (Kraiger et al.,
1993). Measures of verbal knowledge can be taken using traditional pencil-and-paper
questions. Other cognitive outcomes include knowledge organization and cognitive
strategies. Both of these outcomes, however, are more likely to develop in later stages of
the skill acquisition process.

Skill-based learning outcomes also tend to reﬂect later stages of learning.
According to Kraiger et al. (1993) the two major components of skill-based outcomes are
compilation and automization. Compilation involves the combination of discrete
behaviors into domain-speciﬁc routines that are relatively fast and efficient. During this
stage errors are reduced, verbal rehearsal is eliminated, and behavior is more task-
focused. During early stages of skill acquisition this may take place and be ascertained
by observations of performance. Similarly, task outputs of products or performance can
be examined for evidence that compilation is beginning to occur -- fewer errors and faster
production or reaction time.

Automaticity, on the other hand, involves an even greater level of skill.
Automaticity implies that tasks or portions of task can be handled without conscious

monitoring. The lack of monitoring frees cognitive resources to engage in other

40
activities. No subjects are expected to engage in this level of skill during initial studies of

learning.

The third learning outcome suggested by Kraiger and colleagues are affectively-
based. These outcomes reﬂect a class of variables such as attitudes and motivation that
are relevant to the purpose of training. Attitudinal outcomes include outcomes that are
speciﬁc to the training, such as appreciation for diversity in a diversity training course. In
the context of early skill acquisition, there are several affectively-based outcomes that
should be considered. The early stages of training provide the ﬁrst exposure of trainees
to a task that require many hours of practice to master. The attitude trainees hold toward
the task and toward the training are important outcomes of training (Tannenbaum et al.,
1991). It is possible that trainees who feel negatively about their early experiences,
regardless of their ability or skill, will disengage from later training efforts. As a result,
attitudes such as task satisfaction and task withdrawal should be considered as important
outcomes of early training experiences. These are attitudinal outcomes that can be
affected by training and are potentially important indicators of future performance.

Motivational outcomes are somewhat distinct from attitudinal outcomes, but they
nonetheless represent important outcomes of the training process. In particular, self-
efﬁcacy has received a great deal of research attention for its impact on individual’s
willingness to use newly acquired skills in new situations. Baldwin and Ford (1988), in
their review of transfer of training, note the importance of self-efficacy in determining
whether learned skills are employed back on the job. As a result, self-efﬁcacy is

important as both a process and as an outcome variable of any training effort.

41
The study by Kozlowski et al. (1995) noted earlier measured verbal knowledge,

metacognitive knowledge structure, self-efficacy, and ﬁnal training performance as a skill
indicator. All four of these variables were found to affect the generalization of learning.
Therefore each represents an important outcome of training. Although that study did not
investigate task speciﬁc attitudes like satisfaction and withdrawal, those constructs were
noted as potential future research directions. Particularly in the context of early stages of
skill acquisition, these attitudes may affect future effort in the training.

To conclude, to evaluate the effects of errors and feedback on learning, it is
important that a number of different learning outcomes are considered. Cognitive, skill-
based, and affectively-based outcomes are all important outcomes of training, as each
contributes to the use of acquired knowledge at a later time. Furthermore, in the context
of early skill acquisition, variables from each category are likely to have signiﬁcant
effects on subsequent behavior and performance in the training environment. If trainees
disengage from the task after the ﬁrst hour of a week long program, the effects of that

ﬁrst hour affect the remaining training.

INTEGRATION AND HYPOTHESES

To understand the effects of errors on learning during early skill acquisition, an
integrative model is needed that identiﬁes the major psychological mechanisms through
which negative feedback affects learning outcomes. Thus far, little research has been
conducted to identify the psychological processes that result from feedback. Theoretical
and practical progress in the area of training design also calls for an understanding of how
errors during early stages of learning a novel, complex task may affect learning. To aid
in the design of training programs, instructional techniques should be investigated that
directly affect the theoretically relevant mechanisms. The literature reviewed above is
integrated in Figures 1 and 2.

As noted earlier, the literature on negative feedback indicates that both negative
feedback and errors have informational and motivational consequences. The perceived
descriptive and attributional magnitude and diagnosticity of feedback were noted as
important dimensions of feedback that may affect the learning process. Each perception
can be manipulated via training strategies, has individual differences that likely affect it,
and is causally antecedent to psychological variables that may facilitate or impair
learning. Thus each of these perceptions is modeled as both a dependent variable of
training strategies and individual differences and as an independent variable inﬂuencing
the learning process.

The basic processes involved in early stages of learning include beth motivation

variables like self-efficacy and motivation to learn and information variables such as
42

.338 5388 noun—aging Adam

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

43

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

22m goon ﬁmcouaot< woodeoomab
a.
035—30qu . 4/ EMMMHMMQ \ voocoawom
584 8 33023 4.... ..
cougsog .
0956on \.
o—omnuncom
moﬁamﬁdw / \ VO>MOOHOA~ / ooggowhom
.m>
chaﬁng
58533
38830 380.5 magma mommowmbm

wag. 6A wEEEH

.3on £0382 Somehow? 36362 .N 2:me

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

44

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

+ 38E??? has...
:3. \ geawoo
om
€84 8 33683 ~838qu
cowazuouz warhead
2:53ng \
xoonwoom
mowamﬂ< / \ U0>m00hvnm
@583.
xomommmﬁom 039302
380850 338m wEanA mooaocoma

waged 3:239:

45
attentional focus. The motivational variables directly determine attitudinal or affective

outcomes, as those outcomes do not require substantial attention or practice in order to
change. Attentional focus interacts with motivation to determine knowledge and skill
outcomes. Feedback perceptions initiate these two learning processes.

During the early stages of skill acquisition in error-ﬁlled training, all trainees
receive large quantities of negative feedback. Each of the training strategies affects
feedback perceptions in a way that ultimately improve learning. Sequencing can provide
a means to increase the usefulness or diagnosticity of feedback, pushing trainees to focus
their attention on task. Compared to performance focused training, mastery focus
decreases the magnitude of negative feedback so that self-efficacy is not so damaged by
the error-ﬁlled environment. As noted earlier, past research has tied goal focus and
sequencing together (Kozlowski et al., 1995). To disentangle the effects of these
strategies and examine the process by which they affect learning in error-ﬁlled training,
each strategy is portrayed separately in the model. The effects of these two training
strategies are explained in more detail below.

The ﬁrst strategy is mastery focus, a training strategy that calls for trainers and
training materials to emphasize the learning and mastery of content over the performance
of training content in comparison to others. Performance or mastery focus is expected to
have a direct effect on the perceived magnitude of negative feedback. Trainees who de-
emphasize performance and consider progress more important should consider errors to
be less negative than trainees who seek high levels of performance. Trainees who
consider performance of paramount importance should construe errors as conveying more

negative information. As noted earlier, there are two different ways to construe feedback

 

46
magnitude, as purely descriptive or as attributional. Both of these perceptions should be

affected by mastery framing.

Hypothesis #1: Trainees in the mastery condition will perceive lower

attributional and descriptive magnitude of negative feedback than trainees in the

performance condition (controlling for the actual performance feedback).

The second training strategy, sequencing, involves changing the focus of trainees’
attention over time. Sequencing focuses attention on particular aspects of the task at
different times. In terms of the feedback constructs, sequencing should directly affect the
perceived diagnosticity of different feedback information by limiting the amount of
information to which the trainee attends. Trainees who are asked to focus on particular
aspects of the task attend more closely to the feedback about those task aspects and
consequently judge that feedback as useful for diagnosing their performance. Over time,
the focus on speciﬁc aspects of the task provides the trainee with the necessary skills to
interpret and use the feedback relevant to that task dimension.

Hypothesis #2a: Trainees in the sequenced condition will perceive greater

diagnosticity of feedback than trainees in the no sequence condition (controlling

for the actual performance feedback).
As all trainees become more familiar with the task, there should also be a general trend
for feedback to be more diagnostic. Trainees who have greater knowledge about the task

should be better able to interpret feedback messages.

47
Hypothesis #2b: Over time all trainees will perceive greater diagnosticity of

feedback.
In addition to this global effect, differences in diagnosticity of speciﬁc forms of feedback
should result from the sequencing. When a trainee is asked to focus on a particular
subtask, then feedback regarding that aspect of the task should be more diagnostic than
for other topics. This effect should occur because the individual is paying closer attention
to behavior relevant to that topic, because the individual is gaining experience with that
topic relative to other topics, and because the individual is concentrating on the meaning
of that feedback. The result is greater diagnosticity of sequenced materials over material
that has not been sequenced. In addition, subjects who are not in the sequenced condition
should have lower diagnosticity for that speciﬁc topic compared to subjects in the
sequenced condition. The latter hypothesis, #2d below, is similar to #2a but is concerned
with speciﬁc topic area feedback, not with overall feedback.

Hypothesis #2a: Trainees in the sequenced condition should perceive greater

diagnosticity for feedback relevant to the material that is currently the focus of the

sequence than for material that has yet to be brought to focus by the sequence.

Hypothesis #2d: Trainees in the sequenced condition should perceive greater

diagnosticity of feedback relevant to the material that is currently the focus of

the sequence than trainees that are in the unsequenced condition.

In addition to the main training strategy and developmental effects, each training
strategy is expected to have secondary effects on the remaining feedback construct.
These secondary effects result in interactions between mastery and sequencing to

determine the feedback perceptions. Figures 3 and 4 portray these interactions.

48
The first interaction involves sequencing and mastery effects on perceived

magnitude of negative feedback. Sequencing is predicted to affect perceived magnitude
directly by lowering the magnitude of negative feedback. In the mastery condition,
however, this effect should only be minor as trainees already are inﬂuenced to view the
feedback less negatively. In other words, both mastery and sequencing affect magnitude,
but mastery has a more primary effect that makes the sequencing effect redundant.

Hypothesis #3 3: There will be an interaction between sequencing and goal frame

such that trainees in the sequenced condition will perceive lower descriptive and

attributional magnitude of negative feedback in the performance condition, but
there will be no difference in descriptive and attributional magnitude between

the sequenced and non-sequenced condition in the mastery condition.

The second interaction involves sequencing and mastery effects on perceived
diagnosticity. Mastery is predicted to affect perceived diagnosticity directly by raising
the trainee’s ability to attend to the negative feedback. In the sequenced condition,
however, this effect should only be minor because diagnosticity should have already
approached high levels. In other words, both mastery and sequencing affect
diagnosticity, but sequencing has a more primary effect that makes the mastery effect
almost redundant. These interactions account for the usual statement that it is difﬁcult to
disentangle motivational from informational consequences of feedback (F rese &

Altmann, 1989; Goldstein, 1993).

49

 

 

 

 

 

 

 

 

 

 

0

.2

‘5

3‘

Z - .

H h

3 .2. ‘ ~ . . . . + Sequenced
g E ' ‘ - . . _ Trammg
'E' g . ' ' - - D- - ~Unsequenced
an . .

g in Training
'3

.2

GD

0

3

a.

Performance Mastery

 

 

 

Figure 3. Hypothesized magnitude perception interaction by training strategy.

 

 

 

 

 

 

 

—O—Mastery
Frame
--<>--Performance
Frame

 

 

 

Perceived Diagnosticity of Feedback

Unsequenced Sequenced

 

 

 

Figure 4. Hypothesized diagnosticity perception interaction by training
strategy condition.

50
Hypothesis #3b: There will be an interaction between sequencing and goal frame

such that trainees in the mastery condition will perceived greater

diagnosticity of feedback than trainees in the no sequence condition,

but there will be no difference in diagnosticity between mastery and performance

conditions in the sequence condition.

A number of individual differences also affect feedback perceptions. Figure 2
portrays these variables and their effects. First, cognitive ability should increase the
diagnosticity of feedback. The more an individual can rely on working memory and
processing speed, two variables associated with general cognitive ability, the more
quickly the individual can learn the task. The ability to learn quickly does not imply that
attitudinal and motivational variables are directly affected. Affective and cognitive
learning outcomes, while related, are inﬂuenced by different mechanisms (Kraiger et al.,
1993). However, the more a trainee knows, the easier the interpretation and utilization of
feedback. As a result, diagnosticity of feedback may be affected by cognitive ability.
Thus there are two hypotheses about cognitive ability, about both ﬁnal learning outcomes
and perceived diagnosticity.

Hypothesis #4a: Trainees with higher levels of cognitive ability will perceive

higher levels of feedback diagnosticity than trainees with lower levels of cognitive

ability.

Hypothesis #4b: Trainees with higher levels of cognitive ability will have

perform better on cognitive and skill learning outcomes than trainees with lower

levels of cognitive ability.

51
In addition to cognitive ability, negative affectivity was mentioned as a

dispositional personality variable that inﬂuences feedback perceptions. Individuals high
in NA tend to experience more anxiety and depression that individuals low in NA. This
tendency results in more negative interpretations of even neutral stimuli. The effects of
NA on negative feedback should be more pronounced.

Hypothesis #4c: Trainees high in negative affectivity will perceive higher

magnitudes of negative feedback than trainees with low levels of negative

affectivity.
Learning-oriented individuals view negative feedback as informative or challenging,
without viewing it as a threat to their sense of skill or ability. Conversely, performance-
oriented individuals should view negative feedback as more threatening. As a result,
learning-oriented individuals are likely to judge their feedback as less negative than
individuals who are low on this trait while performance-oriented individuals should
display the opposite pattern. Research by Dweck and colleagues also suggests that these
individual differences are related to attributional style (Eliott & Dweck, 1988). Thus,
these hypotheses are expected to hold for both descriptive and attributional magnitude
descriptions.

Hypothesis #4d: Trainees high in learning orientation will perceive lower

attributional and descriptive magnitudes of negative feedback than trainees with

low levels of learning orientation.

52
Hypothesis #4c: Trainees high in performance orientation will perceive higher

attributional and descriptive magnitudes of negative feedback than trainees with

low levels of performance orientation.

Each of the feedback constructs is predicted to affect the leaming activities of
trainees. The effects of each feedback dimension is discussed separately below. The ﬁrst
feedback perception discussed is the perceived diagnosticity of feedback. Diagnosticity
is predicted to affect the study efforts of the trainee on later trials. In essence, negative
feedback, if interpreted as diagnostic, signals to the trainee that a particular aspect of the
task is not well learned and should be studied (Ivancic & Hesketh, 1995). As with the
general notion of usefulness, the diagnosticity of feedback is an interaction of the
feedback stimulus (e.g., how much information does it convey, how clearly presented is
it) and the person perceiving it (e.g., how much do they know about the task, what past
experience do they have with the task). Even with this complexity it is possible to
identify, from the trainee’s perspective, the diagnosticity of a feedback message and to
measure the consequences that perception. Feedback that is judged as diagnostic should
push trainees to focus on the task in order to rectify the problems the feedback has
indicated. In other words, diagnostic feedback helps trainees focus their attention to
speciﬁc aspects of the task, decreasing off-task attention.

Hypothesis #5: Trainees who judge their feedback as more diagnostic will have

greater attentional focus to relevant study material than trainees who judge their

feedback as less diagnostic.

The second feedback dimension involves the perceived magnitude of negative

feedback. Both forms of this perception should affect the self-efficacy of trainees. In

 

53
essence, people who judge their performance to be very negative lack conﬁdence in their

ability to perform the task. Although the model does not depict expectations or goals
directly, the presumption is that all trainees are attempting to minimize if not eliminate
errors from their performance. With number of errors noted explicitly in the feedback,
trainees should be dissatisﬁed with error-ﬁlled performance and lower their perceptions
of capability. The extent of this lowering of self-efﬁcacy varies across individuals,
however. In order to ensure that differences across individuals are perceptual and not a
result of actual differences in the nature of the feedback received, the actual amount of
feedback should be statistically controlled.

Hypothesis #6: Trainees who perceive the attributional and descriptive

magnitudes of negative feedback to be greater will produce lower self-efficacy

judgments than trainees who perceive the magnitude of negative feedback to be

smaller (controlling for the actual level of negative feedback).

Self-efficacy judgments have been shown to affect trainees’ motivation level in
the form of self-set goals (Latham & Locke, 1991). The perspective of this study is that a
global assessment of motivation, rather than self-set goals only, results from self-efficacy.
After-all, self-set goals are only one means by which highly motivated individuals direct
their efforts. Trainees who judge themselves as more capable on the task should be more
willing to invest energy into the task, as measured by the more global measure of
motivation to learn. In training situations where a lot of negative feedback is conveyed,
low levels of self-efficacy seem likely. These low levels should lower self-assessed

motivation.

 

54
Hypothesis #7: Trainees with lower levels of self-efficacy will be less motivated

to learn the task.

Motivation to learn, as an indicator of the effort trainees are willing to exert
during training, is expected to moderate the relationship between attentional focus and
learning outcomes. In essence, this variable serves as an indicator of the quality of the
energy focused on learning the simulation. In addition, individuals who are motivated to
learn likely develop more positive attitudes toward the task. As a result, motivation to
learn should also have a direct effect on affective learning outcome measures, such as
attitudes towards to the task and training.

Hypothesis #8a: Trainees with high motivation to learn will learn more from their

attentional focus than trainees with low motivation to learn.

Hypothesis #8b: Trainees with high motivation to learn will have more positive

affective learning outcomes (attitudes toward the task) than trainees with low

motivation to learn.

Overall tests of the model are also conducted. Of particular interest is the extent
to which two different tracks can be identiﬁed -- an informational and a motivational --
as the pathways through which errors inﬂuence the learning process. Ultimately these
paths intersect to determine many outcomes, but whether they can be distinguished is a

question fundamental to the issue of learning from errors during early skill acquisition.

METHOD AND MEASURES
Radium

There were 115 participants in this study drawn from introductory and advanced
summer courses in psychology, communication, and business at Michigan State
University. From this sample, 4 participants did not follow directions and their data was
discarded. One additional student asked to terminate the experimental session, and was
dismissed without penalty. The ﬁnal sample size was 110. There is no missing data.

Nine subjects were left handed (8.2%). All subjects who asked to modify the
position of the computer mouse were allowed to do so. Eighty-two subjects (75%) were
female. Sixty-six subjects (60%) were aged 22 or older, while only 3 subjects (2.7%)
were age 19 years or younger.

Although participation in this particular study was voluntary, most participants
fulﬁlled class requirements by participating. A few students participated for extra credit
and two people who were not taking summer classes volunteered to participate with no
direct form of compensation.

All participants were informed of possible awards for highest performance and
learning scores. Six small monetary awards were offered, three for each different
outcome. The ﬁrst three awards were noted as ﬁnal performance awards, with the three
individuals who perform the best on the ﬁnal scenario receiving the awards. The second
three awards were noted as ﬁnal knowledge awards, with the three individuals who

perform the best of the ﬁnal knowledge test receiving the awards. The award information
55

 

56
was contained on the consent form, was mentioned at the beginning of training, and was

contained in the ﬁnal instructions provide just prior to taking the knowledge test and
conducting the ﬁnal scenario.

The task for this experiment was selected based on a number of criteria. First, the
experimental task should be relatively complex such that individuals can make progress
but not master it during a 2 hour laboratory session. Second, the task should be governed
by a ﬁnite number of deterministic rules. These rules should be summarized and
explained in written material that trainees can reference. Third, the task should be
relatively engaging so trainees are at least initially interested in the activity. Fourth, the
task should allow for standardization, control, and unambiguous evaluation of
performance. Generally speaking, tactical and strategic computer simulations meet all of
these criteria. They offer controlled performance environments for which ﬁnite numbers
of veridical rules must be learned before effective performance can be achieved.

Recent research has demonstrated that the TANDEM computer simulation is an
effective means to test the effects of training strategies on learning (Ford et al., 1995;
Kozlowski et al., 1995; Kozlowski et al., 1996; Nason, Brown, Gully, & Kozlowski,
1995). Further, this simulation offers an interesting game-like task with rules that can be
modiﬁed to vary the complexity of the task. As a result of these characteristics, the
experimental task chosen for this study is a recently modiﬁed version of the TANDEM
simulation initially programmed by the Naval Air Warfare Center, Human Factors
Division. Under the guidance of a research team at Michigan State, the program was

signiﬁcantly modiﬁed. The latest collaborative version, TANDEM 8.1f, has been

 

57
renamed TEAMS/TANDEM in recognition of the signiﬁcant differences in experimenter

control provided by the latest version.

TEAMS/TANDEM is a computer simulation that can be run on personal
computers. The simulation graphically emulates a simple radar terminal, with the
computer screen portraying a series of blips or “contacts” that the user classiﬁes using
pull-down menus. On-screen displays include time remaining in the current scenario, the
current range of sensors, and a number identifying the current contact that is highlighted
or “hooked” by the user. Hooking contacts involves clicking on a contact on the screen;
in this way users indicate to the simulation that they wish to collect information about
that particular contact. Information is collected by pulling down menus using the mouse.
As the current experiment involves a number of changes from the basic setup used in
these earlier studies, I explain how the task is designed below. As a result of the
difference between this conﬁguration and past conﬁgurations, I refer to this experimental
conﬁguration as RADAROP.

RADAROP has three relatively independent performance dimensions: Labeling,
engaging, and prioritizing. Trainees start the game by clicking on a menu and then
selecting or “booking” a particular contact. Once selected, information regarding the
nature of the vessel can be obtained.

The ﬁrst performance dimension is labeling. Three pieces of information must be
considered prior to making a label decision. Altitude and communication time
information is collected from menus to determine whether the contact is an air or surface
vessel. While the altitude operates as a simple classiﬁcation rule, communication time

forms an exception rule, with values within a certain range completely determining the air

 

58
or surface nature of the vessel while values outside of that range have no impact on the

label decision. The third piece of information is maneuvering pattern and this provides a
contact’s civilian or military status. Based on this information, the trainee must render a
label decision by selecting the proper identifying label (i.e., air civilian, air military,
surface civilian, or surface military) from a decision menu. Trainees are informed that
contacts can appear as any of these combinations, and an equal number of each appear in
the training scenarios.

Once this decision is made, subjects can begin the second performance dimension
of the task, engaging. Engaging involves making a decision about what should be done
about the contact. This decision involves collecting three pieces of information: Threat
level, response intent, and missile lock. Correct engage decisions depend on an
interaction of the level of threat level and response intent, rendering this decision more
complex than the label decision. As with the label decision, the third cue provides an
exception rule. There are 4 possible engage decisions, including clear, monitor, warn,
and shoot.

The third performance dimension is somewhat different from the two above.
Prioritizing involves deciding which contacts to process using the above procedure. In
short, it involves making executive decisions about whether to clear a contact from the
screen using the procedure above. Prioritization decisions are based on features of the
game that include defensive perimeters, speed, course, and range of contacts on the
screen, so this decision is even more complex than the previous two. Although
individuals can choose to focus on some decisions over others, effective overall

performance involves operating all three components of the task in rapid succession.

 

 

 

59
Feedback automatically appears on the computer screen after a trial is completed.

This feedback is actually created by an add-on program entitled “FASTBACK” designed
and developed at Michigan State University. This program reads the data ﬁles from
RADAROP and displays performance information on the computer screen. In this
experiment, four pieces of feedback information are presented. The ﬁrst piece of
information is a global indicator of performance, called score. Score is generated by the
following formula:

SCORE = [(50 * a) - (50 * (A - a))] + [(50 * b) - (50 * (B - b))] + 100 * C
where A is total label actions, a is total labels correct, B is total engage actions, b is total
engages correct, and C is the total number of prioritization errors. This provides 50
points to trainees for performing each label and engage action correctly, for a total of 100
points for correctly handling a contact. This formula subtracts 50 points for every
incorrect decision. Thus, if one of those decisions is made incorrectly, the sum score is
zero; if both decisions are rendered incorrectly the sum score is -100. The total of 100
also reﬂects the penalty for making prioritization errors. This formula places a heavy
emphasis on avoiding errors, in line with the study focus.

The numbers used to generate the score were based on pilot work which
demonstrated that most trainees continue to receive negative scores throughout most of
the experiment. Furthermore, the task was designed such that no individual could attain
error-ﬂee performance levels in the time allotted, although error-free performance is
attainable given further practice. It is important to note that, while scores were intended
to be negative, no comparative information was provided to trainees. Thus, feedback was

suggested to be negative by the score, but trainees had to rely on their personal

60
interpretations of the meaning of negative scores and errors to decide how well they were

performing.

The next three pieces of feedback information presented are the labeling,
engaging, and prioritizing statistics used to compute the overall score. As the focus of the
current study is on negative feedback, the data presented on the screen provides feedback
about the number of errors committed along each dimension of the task. A sample
feedback screen for labeling appears as follows:

5 ERRORS on 6 LABEL actions!
Press any key to continue
This would represent feedback for the labeling 6 contacts, but only labeling one correctly.

As RADAROP requires a good deal of task knowledge, past experiments have
used paper manuals to explain the characteristics of the task and the rules of engagement.
In the current experiment the manual is computerized. Computerization allows the
manual to accurately reﬂect on-line documentation, an increasingly popular means of
providing information to computer system users as well as to provide general instruction.
Further, computerization allows for data collection about what information the user has
displayed on the screen. The time spent on each page is automatically recorded by the
manual program and written to a separate data ﬁle. This data allows the experimenter to
ascertain the exact amount of time the trainee spends looking at different types of
material (represented by the content of pages).

Design
Based on the theoretical model described earlier, two training manipulations are

tested for their independent and interactive effects on learning process and outcomes. As

60
interpretations of the meaning of negative scores and errors to decide how well they were

performing.

The next three pieces of feedback information presented are the labeling,
engaging, and prioritizing statistics used to compute the overall score. As the focus of the
current study is on negative feedback, the data presented on the screen provides feedback
about the number of errors committed along each dimension of the task. A sample
feedback screen for labeling appears as follows:

5 ERRORS on 6 LABEL actions!
Press any key to continue
This would represent feedback for the labeling 6 contacts, but only labeling one correctly.

As RADAROP requires a good deal of task knowledge, past experiments have
used paper manuals to explain the characteristics of the task and the rules of engagement.
In the current experiment the manual is computerized. Computerization allows the
manual to accurately reﬂect on-line documentation, an increasingly popular means of
providing information to computer system users as well as to provide general instruction.
Further, computerization allows for data collection about what information the user has
displayed on the screen. The time spent on each page is automatically recorded by the
manual program and written to a separate data ﬁle. This data allows the experimenter to
ascertain the exact amount of time the trainee spends looking at different types of
material (represented by the content of pages).

Design
Based on the theoretical model described earlier, two training manipulations are

tested for their independent and interactive effects on learning process and outcomes. As

60
interpretations of the meaning of negative scores and errors to decide how well they were

performing.

The next three pieces of feedback information presented are the labeling,
engaging, and prioritizing statistics used to compute the overall score. As the focus of the
current study is on negative feedback, the data presented on the screen provides feedback
about the number of errors committed along each dimension of the task. A sample
feedback screen for labeling appears as follows:

5 ERRORS on 6 LABEL actions!
Press any key to continue
This would represent feedback for the labeling 6 contacts, but only labeling one correctly.

As RADAROP requires a good deal of task knowledge, past experiments have
used paper manuals to explain the characteristics of the task and the rules of engagement.
In the current experiment the manual is computerized. Computerization allows the
manual to accurately reﬂect on-line documentation, an increasingly popular means of
providing information to computer system users as well as to provide general instruction.
Further, computerization allows for data collection about what information the user has
displayed on the screen. The time spent on each page is automatically recorded by the
manual program and written to a separate data ﬁle. This data allows the experimenter to
ascertain the exact amount of time the trainee spends looking at different types of
material (represented by the content of pages).

Design
Based on the theoretical model described earlier, two training manipulations are

tested for their independent and interactive effects on learning process and outcomes. As

61
a result, the experiment employs a 2 (mastery goal versus performance goal) x 2

(sequencing versus control) between-subjects design. There is also a block within-subj ect
factor consisting of 3 different performance segments. The total on task time is 30
minutes, constant across all conditions. This time is divided into 3 blocks of trials, with
trials 1 to 3 composing block 1, trials 4 to 6 composing block 2, and trials 7 to 9
composing block 3. The ﬁnal on-task trial, trial 10, is used as a ﬁnal measure of skill.
Study time lasts 21 minutes, divided into 7 sessions of 3 minutes between on—task trials.
I . . S I I . l .

As each manipulation involves a different approach to structuring and running the
ﬁrst stage of a training program, each involves different instructions throughout the
experiment. These instructions were handed out to trainees while the instructor read them
aloud. The experiment was conducted by one experimenter, the principal investigator,
and the manipulations and all interaction with trainees were tightly scripted. The
manipulations are presented in Appendix A and summarized below.

Mastery yeisns Berfennanee. As noted earlier, a mastery frame involves
presenting the training in terms of learning and progress. Conversely, performance
presents the goal of training as high scores (few errors) and end-states. The instructions
provided to trainees about the purpose of the experiment, the emphasis of the training,
and the way to interpret their progress, vary across each condition. To provide a quick
summary, mastery subjects receive the following training instructions:

Your purpose during this study is to learn and understand RADAR-OP. You will

go through 3 blocks of trials to give you and opportunity to learn, practice, and

explore the game. Pursuing knowledge about the game is the key to gaining skill
with this simulation.

61
a result, the experiment employs a 2 (mastery goal versus performance goal) x 2

(sequencing versus control) between-subjects design. There is also a block within-subject
factor consisting of 3 different performance segments. The total on task time is 30
minutes, constant across all conditions. This time is divided into 3 blocks of trials, with
trials 1 to 3 composing block 1, trials 4 to 6 composing block 2, and trials 7 to 9
composing block 3. The ﬁnal on-task trial, trial 10, is used as a ﬁnal measure of skill.
Study time lasts 21 minutes, divided into 7 sessions of 3 minutes between on-task trials.
As each manipulation involves a different approach to structuring and running the
ﬁrst stage of a training program, each involves different instructions throughout the
experiment. These instructions were handed out to trainees while the instructor read them
aloud. The experiment was conducted by one experimenter, the principal investigator,
and the manipulations and all interaction with trainees were tightly scripted. The
manipulations are presented in Appendix A and summarized below.
Masteryyemﬂerfennanee. As noted earlier, a mastery frame involves
presenting the training in terms of learning and progress. Conversely, performance
presents the goal of training as high scores (few errors) and end-states. The instructions
provided to trainees about the purpose of the experiment, the emphasis of the training,
and the way to interpret their progress, vary across each condition. To provide a quick
summary, mastery subjects receive the following training instructions:
Your purpose during this study is to learn and understand RADAR-OP. You will
go through 3 blocks of trials to give you and opportunity to learn, practice, and

explore the game. Pursuing knowledge about the game is the key to gaining skill
with this simulation.

62
In contrast, the performance goal instructions are as follows:

Your purpose during this study is to perform at your maximum on RADAR-OP.

You will go through 3 blocks of trials to give you an opportunity to maximize

your performance. Your level of performance, as measured by the overall score,

is critical to demonstrating your skill at this game.

In summary, in the mastery condition the purpose of the experiment is presented
as trying to discover how people go about learning and mastering skills like those
contained in this task. In the performance condition, the purpose is stated as testing to
determine who can perform the task best and how quickly they can perform at the highest
levels on the task. Performance conditions emphasize the score and the number of errors
as indicators of how well the trainee can perform while the mastery condition emphasizes
this feedback as essential to learning. Mastery condition subjects are encouraged to use
the feedback to learn more about the task, as feedback provides valuable information.
Manipulation reminders are provided once during each block and at the beginning of the
remaining blocks.

WW. Sequencing is initiated via instructions to the
trainee just prior to beginning the task for the ﬁrst time. The purpose of the sequencing is
to focus trainees attention on particular aspects of the task in an order that helps subjects
learn parts of the task separately. As there are three primary dimensions of performance
on the RADAROP task, each is be covered separately, in a part-task approach to
sequencing. Following the ACT* model (Anderson, 1982), sequencing begins with the

declarative aspects of task performance. As labeling requires the most memorizing and

involves the simplest aspect of task performance, subjects are asked to focus on this

63
aspect of the task for the ﬁrst block of trials. This focus is mentioned before the block

and repeated before every trial within the block, as follows:

Our suggestion is that you limit your attention to certain parts of the simulation

for each block. Instead of working on all parts of the program, the best way to

approach this task is to concentrate your effort. So, before each block of trials we

will give you a set of skills to focus on during that block. When you are ’

practicing and studying RADAR-OP during that block, you should focus on the

part of the task that we suggest.

For this ﬁrst block, we suggest that you focus on LABELING. Although there is

more to the simulation than just this one skill, LABELING is integral to all

subsequent skills and actions with the task. So concentrate your efforts of

LABELING. . ..
Trainees still have control over what they choose to focus on; however, these instructions
are paraphrased once during each block.

For the second block, trainees are asked to focus on engaging contacts. Engaging
also involves memorization of information about contact characteristics, but it entails a
more complex relationship between the information provided and the decision to be
made. Finally, trainees are told to focus on prioritizing contacts for the third block of
trials. Prioritizing is a superordinate skill, requiring performance of the preceding two
skill components. Following Anderson’s model, it involve a heavier procedural
component than the previous two performance dimension, in addition to a few additional
declarative requirements. In the unsequenced condition, no mention of sequencing is
made. Subjects are told to engage the task with no speciﬁc instructions on how to focus
their attention.

Manipulation checks were conducted with 10 questions for the

mastery/performance induction and 2 questions for the sequencing induction. The

mastery/performance manipulation check involves two sets of 5 questions ascertaining

64
the level of mastery and performance state goal orientation of trainees, based on scales

developed by Boyle (1995). The sequencing manipulation check consists of two items
assessing whether trainees felt that the training suggested they focus their attention on
speciﬁc aspects of the simulation over others. These questions are presented to trainees
just after the manipulations, but before the trainee begins the ﬁrst block of trials.
Preceding

Table 1 outlines the procedure for experiment. The experiment is basically
divided into four sections. In the ﬁrst section the experiment is explained and informed
consent is obtained. In the second, individual differences measures are collected. This is
done before the manipulations are enacted, in order to avoid contaminating the measures.
Third, a demonstration and training is provided. This part of the experiment begins with
a guided tour of the RADAROP simulation, the feedback screen, and the manual
program. After this demonstration, trainees are given 3 minutes to become familiar with
the manual program. This allows trainees time to get comfortable with how to collect

relevant information from the manual and time to read the ﬁrst few introductory pages

65

Table 1. mmmmmmamm

Individual Differences Measures and Introductory Training

+5 Min
+10 Min
+30 Min
+35 Min

Block 1

+40 Min
+44 Min
+47 Min
+52 Min
+55 Min
+58 Min
+61 Min
+64 Min
+70 Min

Block 2
+74 Min
+75 Min
+78 Min
+83 Min
+86 Min
+89 Min
+92 Min
+95 Min
+100 Min

Block 3

+104 Min
+105 Min
+108 Min
+114 Min
+117 Min
+120 Min
+123 Min
+132 Min
+145 Min
+150 Min

Explanation/Purpose/Consent

Individual differences measures
Introduction training with demonstration
Introduction read on manual (3 minutes)

Manipulations-- frame (mast or pert), sequenced focus labeling (or none)
On-task + feedback (3 minutes)

Measure--feedback perceptions

Study (3 minutes)--measure attentional focus

Manipulation reminder + on-task + feedback (3 minutes)

Study (3 minutes)--measure attentional focus

On-task + feedback (3 minutes)

Measure--mediating constructs

Break (3-4 minutes)

Manipulation reminder with sequenced focus on engaging (or none)
On-task + feedback (3 minutes)

Measure--feedback perceptions

Study (3 minutes)--measure attentional focus

Manipulation reminder + on-task + feedback (3 minutes)

Study (3 minutes)--measure attentional focus

On-task + feedback (3 minutes)

Measure--mediating constructs

Break (3-4 minutes)

Manipulation reminder with sequenced focus on prioritizing (or none)
On-task + feedback (3 minutes)

Measure--feedback perceptions

Study (3 minutes)--measure attentional focus

Manipulation reminder + on-task + feedback (3 minutes)

Study (3 minutes)--measure attentional focus

On-task + feedback (3 minutes)

Measure--mediating constructs and learning (knowledge, attitudes)
Final block on-task (3 minutes)--performance measure

End of experiment, debrief and dismiss

66
without becoming familiar with the major rules that govern the simulation. This ensures

that all trainees begin the ﬁrst task trial with little substantive knowledge about the
simulation, but the ability to collect such information in the future.

The fourth part is the core of the experiment when subjects actually receive the
manipulations. Following the reading of the manipulations, trainees answer manipulation
check questions. Then the main part of the experiment begins with subjects practicing
the task for 1 trial of 3 minutes and receiving feedback. Following that the program
automatically asks the trainees to answer feedback perception questions on bubble sheets.
Following these questions, trainees are allowed 3 minutes to study the manual. After the
3 minutes study period is up, subjects are taken back into the task to perform another time
3 minutes trial. Trainees then study and practice again, for a total of 3 on-task trials and 2
study sessions per block. At the end of the block trainees answer the mediating construct
questions. This is one block of the experiment. Two more blocks follow this one, each
providing a different focus in the sequencing condition or simply more opportunity to
perform in the control condition. Between blocks subjects are given a 3 to 5 minute
break. After the second block no mediating construct questions are asked, simply to
prevent trainees from becoming bored with answering too many questions. As a result,
analyses for mediating constructs only have 2 levels of the within-subjects factor.

When all three blocks are complete, subjects complete the cognitive and affective
learning outcome measures. Then, a ﬁnal task scenario is played to provide skill learning

outcomes measures. Finally, subjects are debriefed and provided with class credit slips.

67
Measures

Measures employed in this study are presented in Appendices B, C, and D. These
are presented as they appeared to trainees, with one exception. Item codes are presented
after each item so readers can reference item numbers to the later analyses.

Leamingeiiteemes. To capture the multi-dimensional nature of learning, three
different learning outcomes are measured, each measuring a different component of
complex cognitive skill. Questionnaire items, when employed, are presented in
Appendix B as they appeared to trainees. As noted earlier, cognitive, skill-based, and
affective learning outcomes are all possible outcomes of training (Kraiger et al., 1993).
Each category of training outcome is measured in a number of ways. Knowledge or
cognitive outcomes are assessed via an 2l-item multiple choice paper-and-pencil
measure. Knowledge items were developed to tap into all three of the primary
dimensions of task performance--labeling, engaging, and prioritizing. Seven items were
written for each content area, so each performance dimension is weighted evenly. Items
were written to reﬂect both declarative knowledge, such as what does altitude of 10
nautical miles mean, and the utilization of that knowledge in standard performance
settings. An example of this latter question type is, “Given a target with altitude of 10
N.M., a maneuvering pattern of Code_Foxtrot, and a communication time of 30 seconds,
which is the correct label action?” All questions are presented with 5 multiple choice
alternatives. Total number of correct answers, ranging from a possible low of 0 to a high
of 21, serves as the indicator of knowledge.

Skill is operationalized as behaviors performed correctly to complete portions of

the task. This measure is taken from a ﬁnal training scenario for which all subjects (i.e.,

68
regardless of condition) are given performance instructions. That is, all trainees are told

to perform at their maximum for the frnal scenario. There are a number of different
measures of skill, including indicators for each of the primary dimensions of task
performance and a global score.

For labeling, the number of label actions and the number of label errors is used.
For engaging, the number of engage actions and the number of engage errors is used. For
prioritizing, the number of contacts that cross defensive perimeters is used. A global
score is also used as a dependent variable, based on the scoring algorithm presented to
trainees. Although there are no speciﬁc hypotheses about different skill outcomes, results
may differ depending on the criterion used. Exploratory analyses determine whether the
manipulations and subsequent learning process differentially affect these training
outcomes.

Affect is operationalized with two self-report questionnaires. Task withdrawal
and task satisfaction are measured using paper-and-pencil surveys. Task withdrawal is
assessed using a 6-item scale used by Gilliland (1992). Task satisfaction is measured
using a 4-item scale developed by Carsten (1987). All but one of these questions is
answered with respondents indicating their agreement to statements using a 5-point Likert
scale with “Strongly Agree” to “Strongly Disagree” anchors. The ﬁnal question of the
withdrawal scale using a different format, with individuals selecting the alternative
statement that most reﬂects their attitude. As these scales are expected to be highly
related, a composite index is formed to reﬂect the overall affective reaction to the task.

EeedhaelLCenstniets. All feedback and mediating construct questionnaire items

are presented in Appendix C, exactly as they appeared to trainees. The questions are

69
statements for which respondents indicate their agreement using a 5-point Likert scale

with “Strongly Agree” to “Strongly Disagree” anchors.

Descriptive magnitude of negative feedback, the ﬁrst feedback construct, is a
descriptive judgment by the trainee of the degree to which feedback received is negative.
The 4 items tapping this construct ask how poorly the trainee feels s/he did based on this
feedback. The second feedback construct, attributional magnitude of negative feedback,
is also measured using a 4-item scale. This combines a description of magnitude with the
assertion that this magnitude is caused by internal, stable factors. While this is a non-
traditional way of measuring attributions, it offers a way to capture magnitude judgments
that are viewed as more serious in their implications.

The third feedback construct is the perceived diagnosticity of feedback. This
construct taps the individual’s perception of the utility of feedback for improving future
learning and performance on the task. A 4-item diagnosticity scale provides for the
global assessment of utility. Furthermore, to obtain measures of diagnosticity for speciﬁc
feedback on labeling, engaging, and prioritizing, three pairs of items are used. Each pair
asks about diagnosticity speciﬁcally relating to a single dimension of performance. All
feedback constructs are measured 3 times throughout the experiment.

Mediatingﬂqmtniets. The following scales are all measured using 5-point Likert
scales with “Strongly Agree” to “Strongly Disagree” anchors. These scales are answered
twice during the course of the experiment. I

Self-efficacy is measured using a 5-item shortened version of the questionnaire

employed by Kozlowski et al. (1995; 1996). These items were developed for a similar

 

70
experimental task using the same simulation and has demonstrated sufficient reliability.

Motivation to learn is measured using a S-item, modiﬁed version of the scale used by
Noe and Schmitt (1986) and Quiﬁones (1995). Noe and Schmitt used a longer scale
composite and found alpha reliabilities of .98 for pre-training motivation and .95 for post-
training motivation. Quiﬁones (1995) used a 10-item scale more similar to the one
employed in this study and reported an alpha of .93.

Attentional focus is measured in two ways. First, a shortened version of the scale
developed by Fisher (1995) is used to assess how much attention was spent on-task. The
scale has demonstrated sufficient reliability and validity in predicting learning outcomes.
Six items are used with a few wording changes to reﬂect differences in experimental
tasks. Second, a time measure is calculated based on time spent studying content pages
in the manual. The sum of time spent reading pages that contain information relevant to
simulation performance is used for this measure. Of nineteen total pages in the manual, 3
pages provide procedural information about the 3 performance components of the task.
For each dimension, two more pages provide speciﬁc information about the rules which
govern the task. Thus, a total of 9 pages provide the core information to successfully run
the simulation. The other 9 pages provide either: (1) Information that is redundant with
the introductory training, or (2) information that is irrelevant to task performance.

W5. Individual difference questions are located in Appendix
D, reproduced in the form used in the experiment. To test for individual differences in
learning and performance trait orientation, scales from Button, Mathieu, and Zajac (in

press) are employed to measure learning and performance orientation. These measures

71
are 5-point Likert scale statements divided into two 8-item scales. Both scales were

employed in 3 different validation studies reported by Mathieu et al. (in press). Reported
alpha reliabilities were .79, .85, .82 and .76, .77, .81 for learning and performance trait
orientation, respectively. Boyle and Klimoski (1995) and Kozlowski et al. (1995; 1996)
used the same scales and reported similar reliabilities.

To ascertain whether propensity for negative thoughts and anxiety have an effect
of perceptions of feedback, negative affectivity (NA) is measured. The 10-item measure
of negative affectivity from the PANAS scale, developed by Watson, Clark, and Tellegen
(1988), is used. In order to tap the overall propensity to experience negative mood states,
the items are worded to general or average feelings, the traditional way the scale is used
to tap stable individual differences. Watson et al. (1988) reported alpha reliabilities of
.87 for this scale and 8 week test-retest reliabilities of .71. Watson et al. (1988) note that
this latter reliability suggests this scale is stable enough to be used to measure traits.

Cognitive ability is also assessed, as noted above, using the 12 minute Wonderlic
Personnel Test. Test-retest reliability for this test has ranged ﬁom .82 to .94 and
correlations of odd and even items range ﬁom .88 to .94 (Wendeilieﬁersennellestand
SehelastieLexeLExamﬂsemMannal, 1992). For the purpose of this experiment, the
reliability of the test is assumed approximately equal to .88, the lower bound for typical
internal consistency reliabilities. Two different forms were used during the experiment,

and adjustments were made for form difficulty according to the User’s Manual.

RESULTS

Before testing the hypotheses, the quality of the self-report data was investigated.
First, factor analyses were run on all self-report data collected at the same point in time.
These analyses help determine whether trainees were able to effectively discriminate
between different constructs measured at the same time. Method or response bias may
result in a single factor representing the general response tendency. Recovering the latent
structure provides some evidence against the response bias explanation for observed
correlations between self-reported constructs. Further, this analysis can be used to assess
the homogeneity of scales. Recognizing that the solutions obtained may vary depending
on technique (e.g., Ford, MacCallum, & Tait, 1986), the following analyses were
computed with both principal component and principal axis factor analytic techniques.
As the solutions were nearly identical, all reports are for principal components solutions
only. Varimax rotation is employed, as it can aid the psychological meaningfulness,
reliability, and reproducibility of factor solutions (Ford et al., 1986).

Exploratory factor analyses of the individual differences data results in 9 factors
using the Kaiser eigenvalue criterion. Eigenvalues for these factors were 6.7, 5.3, 3.3,
1.9, 1.8, 1.3, 1.2, 1.1, 1.0. However, the scree plot of these eigenvalues indicates a clear
break occurs between the ﬁfth and sixth factors. Tucker, Koopman, and Linn (1969)
have noted that the scree criterion, while somewhat subjective, generally performs more

effectively than the “eigenvalue over one” rule. The ﬁve factor solution from the scree
72

73
test results in performance orientation, learning orientation, positive affectivityz, and 2

negative affectivity factors. Given the extensive amount of research on negative
affectivity and the limited number of hypotheses employing it in this study, it was
determined that there was little conceptual clarity to be gained from dividing the scale
into two subscales. Forcing a four factor solution results in a more parsimonious
description of the data and a description that follows theoretical assertions regarding the
structure of affect.

Table 2 presents the forced 4 factor solution for the individual differences data.
Aside from one positive affectivity item loading on the learning orientation scale, the
latent factor structure of the data was recovered. As a whole the positive affectivity and
learning orientation scales had a number of items with moderate cross-loadings, implying
that these latent constructs are correlated. Given the consistency of loadings both within
and across scales, it is concluded that these scales are sufficiently homogeneous.
Table 3 presents the exploratory factor analysis solution for the two magnitude and one
diagnosticity scale collected at the beginning of each block. Eigenvalues for the three
factors were 4.5, 2.6, and 1.5 for block 1; 6.8, 1.6, 1.3 for block 2; 6.3, 1.7, 1.4 for block
3. Once again, the latent structure of the data was successfully recovered, aside from one
attributional magnitude item which loaded evenly on the descriptive and attributional
scales. However, this only occurred one of the three times data was collected. As a

result, this item was maintained in future analyses.

 

2 The results for positive affectivity are not explored in the thesis, but the factor analysis results are
presented with this scale because the data was collected at the same point in time.

74
Table 2. EactcLsnlntrcanLmdmdrmmﬁ'ercnccsscales.

___Ea£IQr_l_Ea£tm_2_Ea£.th_3_Ea&mL_4

NA2 .76080 -.08952 -.06355 . 15546
NA6 .74714 .19920 -.2] 5 15 .06205
NA9 .70786 .09099 -.24891 .08213
NAS .68560 .16480 -.22537 .01475
NA7 .67129 -.05439 .1 1801 .05560
NA8 .6524] -.O 1304 .09547 .0655]
NA4 .63536 -.24334 .16165 .] 1927
NA3 .62938 -.06 l 60 . 10752 .01828
NAIO .62895 .06759 -.06436 -.06980
NA] .58894 .08593 -.08215 .18378
LS -.05469 .68460 .23645 .031 l 1
L8 -. 10963 .67725 .25379 -.O3454
L6 .12242 .65533 .25978 .] 1631
L4 .06038 .63434 .2742] -. 17382
L3 .08353 .61828 .33826 .0671 l
L] -.02594 .61823 .09052 -.05930
L7 .00763 .60934 .091 14 .09897
L2 .09223 .51474 .13125 -.O4172
PA2 -.02202 .48559 .38183 -.00794
PA3 .00613 .42509 .67530 .02402
PAS -.04293 .29475 .67297 -.06504
PA8 -.27297 .0090] .66083 .00148
PA9 -.09986 .06302 .64774 .05195
PA] .03792 .22154 .63 346 -.05031
PA7 -.09365 .2546] .61275 -.0721 1
PAIO -.O3289 .37452 .59100 -.05506
PA4 . 16356 .20573 .57057 .00966
PA6 .0066] .22006 .4578] .0235]
P7 .09570 -. 16687 .0] 1 17 .79610
P4 .08260 -. 17198 .00004 .77130
P5 .1146] .15027 -.012]] .67156
P3 .03415 .22026 -.09437 .66262
P] -.00579 -.22681 . 14526 .64280
P8 .09219 -.036 12 -. 12300 .63207
P2 -.02659 .0369] .0876] .59294
P6 .25716 .17063 -.l 1929 .58718

Nete. L = learning orientation, P = performance orientation, NA = negative affectivity,
and PA = positive affectivity. Numbers represent item numbers. Solution obtained
forcing 4 factors with a principal components analysis with Varimax rotation.

Table 3. Eactcnsclmicnfcnfeedhackmnstmcts.

Wand

AMAG4 .88085 .08210 -.2371 l
AMAG2 .86840 .08173 -.21420
AMAG3 .8497] .09510 -.22527
AMAG] .69547 .27519 -.05995
DMAG2 . 12985 .9084] -.04072
DMAG] .16053 .89680 -.OO694
DMAG3 .11567 .76553 -.12441
DMAG4 .04127 .70708 .05309
DIAG3 -.07177 -.05081 .90525
DIAG2 -.27748 .04302 .8250]
DIAG4 -. 10676 -. 15294 .78964
DIAG] -.39948 .0877] .69182

WW3

DMAG2 .86994 -.121 19 .28405
DMAG] .86676 -.21922 .22178
DMAG4 .8146] -.22092 .33258
DMAG3 .79008 -.37564 .09782
AMAG] .5836] -.34406 .56080
DIAG3 -. 19507 .8575] -.09342
DIAG2 -.214 10 .84549 -. 19610
DIAG] -. 17000 .83507 -.21702
DIAG4 -.331 13 .69504 -.31843
AMAG4 . 19880 -. 18992 .90390
AMAG3 .28133 -. 18205 .88779
AMAG2 .23948 -.22633 .80700
W
AMAG4 .90785 .18623 -. 14502
AMAG3 .90014 .28204 -.15936
AMAG2 .8801 l .2450] -.223 83
AMAG] .77746 .33039 -.28693
DMAG2 .26954 .87907 -. 12604
DMAG] .391 13 .82854 -. 14929
DMAG3 .0171] .76183 -.32816
DMAG4 .40073 .73650 -.1 1210
DIAG2 -.01515 -.20157 .83089
DIAG3 -. 13724 -. 19969 .81745
DIAG] -.30587 -. 15893 .79734
DIAG4 -.41 l 12 -.05518 .6425]

Nete, AMAG = attributional magnitude, DMAG = descriptive magnitude, and DIAG =
diagnosticity. Numbers represent item numbers. Solution obtained with an exploratory
principal components factor analysis using the Kaiser criterion for extraction.

Table 3. Eactctscluticnfcnfcedhackmnzucts.

WW

AMAG4 .88085 .08210 -.2371 l
AMAG2 .86840 .08173 -.21420
AMAG3 .8497] .09510 -.22527
AMAG] .69547 .27519 -.05995
DMAG2 . 12985 .9084] -.04072
DMAG] . 16053 .89680 -.00694
DMAG3 .11567 .76553 -.12441
DMAG4 .04127 .70708 .05309
DIAG3 -.O7177 -.05081 .90525
DIAG2 -.27748 .04302 .8250]
DIAG4 -. 10676 -. 15294 .78964
DIAG] -.39948 .0877] .69182
W
DMAG2 .86994 -.121 19 .28405
DMAG] .86676 -.21922 .22178
DMAG4 .8146] -.22092 .33258
DMAG3 .79008 -.37564 .09782
AMAG] .5836] -.34406 .56080
DIAG3 -.19507 .85751 -.09342
DIAG2 -.21410 .84549 -. 19610
DIAG] -.17000 .83 507 -.21702
DIAG4 -.33113 .69504 -.3 1843
AMAG4 . 19880 -. 18992 .90390
AMAG3 .28133 -.l8205 .88779
AMAG2 .23948 -.22633 .80700
W
AMAG4 .90785 . 18623 -.14502
AMAG3 .90014 .28204 -. 15936
AMAG2 .8801 l .2450] -.22383
AMAG] .77746 .33039 -.28693
DMAG2 .26954 .87907 -. 12604
DMAG] .39113 .82854 -. 14929
DMAG3 .0171] .76183 -.32816
DMAG4 .40073 .73650 -.1 1210
DIAG2 -.01515 -.20157 .83089
DIAG3 -. 13724 -.l9969 .81745
DIAG] -.305 87 -. 15893 .79734
DIAG4 -.4l 1 12 -.05518 .6425]

Nete. AMAG = attributional magnitude, DMAG = descriptive magnitude, and DIAG =
diagnosticity. Numbers represent item numbers. Solution obtained with an exploratory
principal components factor analysis using the Kaiser criterion for extraction.

Table 3. Eactcnscluticnﬂfeedbackmnstmcts.

WM

AMAG4 .88085 .08210 -.2371 1
AMAG2 .86840 .08173 -.21420
AMAG3 .8497] .09510 -.22527
AMAG] .69547 .27519 -.05995
DMAG2 . 12985 .9084] -.04072
DMAG] .16053 .89680 -.00694
DMAG3 .1 1567 .76553 -. 12441
DMAG4 .04127 .70708 .05309
DIAG3 -.07 l 77 -.05081 .90525
DIAG2 -.27748 .04302 .8250]
DIAG4 -. 10676 -. 15294 .78964
DIAG] -.39948 .0877] .69182
W
DMAG2 .86994 -.121 19 .2 8405
DMAG] .86676 -.21922 .22178
DMAG4 .8146] -.22092 .33258
DMAG3 .79008 -.3 7564 .09782
AMAG] .5836] -.34406 .56080
DIAG3 -. 19507 .8575] -.O9342
DIAG2 -.21410 .84549 -. 19610
DIAG] -. 17000 .83507 -.21702
DIAG4 -.331 13 .69504 -.31843
AMAG4 . 19880 -. 18992 .90390
AMAG3 .28133 -. l 8205 .88779
AMAG2 .23948 -.22633 .80700

W

AMAG4 .90785 .18623 -.14502
AMAG3 .90014 .28204 -.15936
AMAG2 .8801 l .2450] -.22383
AMAG] .77746 .33039 -.28693
DMAG2 .26954 .87907 -. 12604
DMAG] .391 13 .82854 -.14929
DMAG3 .0171] .76183 -.32816
DMAG4 .40073 .73650 -.1 1210
DIAG2 -.0]515 -.20] 57 .83089
DIAG3 -.13724 -.19969 .81745
DIAG] -.30587 -. 15893 .79734
DIAG4 -.41112 -.05518 .6425]

Note. AMAG = attributional magnitude, DMAG = descriptive magnitude, and DIAG =
diagnosticity. Numbers represent item numbers. Solution obtained with an exploratory
principal components factor analysis using the Kaiser criterion for extraction.

76
The diagnosticity couplets targeted towards speciﬁc performance dimensions

were run through a separate series of factor analyses. For data collected at both block 1
and 3, only 1 factor (eigenvalues of 4.0 and 4.3, respectively) was recovered using the
Kaiser criterion. At time 2, two factors were recovered (eigenvalues of 3.5 and 1.1). As
the perceptions of diagnosticity were expected to change over time, the factor structures
and intercorrelations of the latent constructs represented in these items may change over
time. The third block data is used to check item quality, as trainees have the most
knowledge about the simulation and the feedback at this point. Even though only one
factor was extracted in the exploratory analyses, a forced three factor solution was tested
to ascertain whether the latent structure could be recovered. Table 4 indicates that the
latent structure of this data was well-recovered except for one item. The second item of
the labeling scale has a larger loading on the engaging scale than on the labeling scale.
To maintain homogeneity of the labeling and engaging scales, this item was removed
from ﬁrrther analysis. Moderate cross-loadings suggest that the latent constructs are
correlated; this accounts for the difficulty in recovering the three factor solution in
exploratory analyses.

Table 5 presents exploratory factor analysis results for the mediating constructs of
motivation to learn, attentional focus, and self-efficacy. Eigenvalues for these factors
were 8.1, 2.3, and 1.1 at the end of block 1, and 7.7, 3.0, and 1.1 at the end ofblock 2.
These analyses support the proposed latent structure. At each time, a self-efficacy item
cross-loaded with the motivation to learn factor, but the pattern of loadings between these
factors suggests the cross-loading reﬂects a moderate latent correlation between scales.

All other items loaded on the correct factors. Thus, no items were discarded.

Table 4. Eactcnsclutimrfmdiagncsticrtﬂtemmplets.

77

W

DIPRII .89589 .03086 .24490
DIENGZ .80893 .3 8484 .15598
DIPR12 .79833 .32932 .25805
DIENG 1 .66059 .597 73 . 10657
DILABZ . 19729 .84 149 .39099
DILAB] .27565 .30813 .89436
Wand—Faun};
DIENGl .83568 .30218 .22322
DIENGZ .82228 .28593 .24040
DILABZ .78758 .17540 .43252
DIPRII . 14508 .93722 .15063
DIPR12 .42788 .82122 -.01177
DILAB] .36957 .07979 .91480
IIME_L_Eactcr_L_Eactcr_2_£actnr_3
DIENGZ .86752 .27889 .2350]
DILAB2 .83776 .29563 .29419
DIENG] .80205 .30515 .36866
DIPRII .21373 .92776 .17853
DIPR12 .3 8240 .85908 . 17719
DILABl .4635] .2508] .84682

Note. DILAB = diagnosticity for labeling, DIENG = diagnosticity for engaging, and
DIPRI = diagnosticity for prioritizing. Numbers represent item numbers. Solutions
obtained forcing 3 factors with a principal components analysis and Varimax rotation.

 

Exploratory factor analysis of the task attitudes of satisfaction and withdrawal
result in one factor with eigenvalue of 5.5 using the Kaiser criterion. A forced two-factor
solution, reported in Table 6, further supports the earlier assertion that these two
constructs are highly related. Both the task satisfaction and withdrawal scales split, with
items loading on two factors which do not represent one factor or the other. These two

factors appear to be correlated, as indicated by the moderate cross-loadings of most items.

Table 4. Eammlmicnfctdiagncsticimitemmnlm

77

WW

DIPRII .89589 .03086 .24490
DIENGZ .80893 .3 8484 . 15598
DIPR12 .79833 .32932 .25805
DIENG] .66059 .59773 .10657
DILABZ . 19729 .84149 .39099
DILAB] .27565 .30813 .89436
W
DIENG] .83568 .30218 .22322
DIENG2 .82228 .28593 .24040
DILABZ .78758 .17540 .43252
DIPRI] . 14508 .93 722 . 15063
DIPR12 .42788 .82122 -.01 177
DILAB] .36957 .07979 .91480
W
DIENGZ .86752 .27889 .23501
DILABZ .83776 .29563 .29419
DIENG] .80205 .30515 .36866
DIPRII .21373 .92776 .17853
DIPR12 .3 8240 .85908 .17719
DILAB] .4635] .2508] .84682

Nete, DILAB = diagnosticity for labeling, DIENG = diagnosticity for engaging, and
DIPRI = diagnosticity for prioritizing. Numbers represent item numbers. Solutions
obtained forcing 3 factors with a principal components analysis and Varimax rotation.

 

Exploratory factor analysis of the task attitudes of satisfaction and withdrawal
result in one factor with eigenvalue of 5.5 using the Kaiser criterion. A forced two-factor
solution, reported in Table 6, further supports the earlier assertion that these two
constructs are highly related. Both the task satisfaction and withdrawal scales split, with
items loading on two factors which do not represent one factor or the other. These two

factors appear to be correlated, as indicated by the moderate cross-loadings of most items.

78

Table 5. EactcLscluticanLmediatinucnsmrcts.

W3

MOTLN5 .83024 -. 15222 . 13088
MOTLN] .77913 -.3261 1 .24277
MOTLN4 .76823 -.29392 .32029
MOTLN2 .743 85 -.20525 . 17587
MOTLN3 .66752 -.29934 .2657]
SELF EF3 .57280 -. 12467 .50063
ATTNTNS -.]3121 .89612 -.Oll94
ATTNTN] -.09 l 75 .80168 -.O6938
ATTNTN4 -.34394 .75326 . 12643
ATTNTN2 -.3 1962 .74736 -.21 178
ATTNTN3 -.43464 .70084 -.04496
ATTNTN6 -.05993 .58738 -.34933
SELFEF 4 .00204 -.02445 .76734
SELFEFS .45 546 .04202 .67078
SELFEF 1 .36844 -. 15537 .66622
SELF EF2 .34710 -. 13685 .6068]

W3

ATTNTN] .92785 -. 13099 -.02 133
ATTNTN3 .89716 -.21206 -.O7578
ATTNTNS .88187 -.26418 -.02435
ATTNTN4 .85073 -.23969 -. 10017
ATTNTN2 .801 16 -.05628 -. 12063
ATTNTN6 .74177 -.28005 -.0953 l
MOTLN2 -. 16901 .83923 . 17588
MOTLN] -.30030 .77803 .243 87
MOTLN5 -.34 164 .77025 .27998
MOTLN4 -.32443 .763 80 .29769
MOTLN3 -.46449 .67168 .25772
SELFEF4 . 18324 .5 1038 .49922
SELFEF3 -.01083 .3 8489 .79535
SELFEF 1 -. 16780 .24954 .79249
SELF EFS -. 12968 .40006 .73914
SELFEF2 -.09813 .02304 .73058

Note. ATTNTN = attentional focus, MOTLN = motivation to learn, and SELFEF = self-
efficacy. Numbers represent item numbers. Solution obtained with an exploratory
principal components factor analysis using the Kaiser criterion.

79
Based on these ﬁndings, one scale is created, with all items reﬂected to represent task

satisfaction.

The second way in which the quality of the self-report data was investigated was
by generating internal consistency reliabilities. Having established scale homogeneity,
Cronbach’s alpha provides a meaningful estimate of internal consistency reliability.
Reliabilities are reported in the diagonal of Table 7, along with means and standard
deviations of the variables used in the majority of analyses. Reliability for self-report
scales was generally quite good, ranging from .92 to .77.

Cronbach’s alpha is not a meaningful indicator of internal consistency reliability
for the multi-dimensional task knowledge test. The items in this test were designed to
systematically assess knowledge of different task rules. As a result, split-half reliability
based on odd and even items offers a more reasonable indicator of internal consistency.
Because of the organization of test items, this process results in two subtests with parallel
content. The correlation between odd and even test scores is .68. Corrected to the ﬁll]-
length of the test using the Spearman-Brown prophecy formula, the reliability of this test
is .81. Item difﬁculties of test items range from .92 to .15 (M = .56, SD = .27), indicating
the test items generally provide good discrimination.

I I . 1 . 1 l

The ﬁrst manipulation check was for the effectiveness of the sequencing
manipulation. As noted earlier, a two-item scale was employed to assess whether trainees
recognized that their training was designed to focus attention on certain aspects of the

task. The correlation between these items was .40, resulting in an alpha of .57. Trainees

80
Table 6. EactcLschrtianQLattimdettainingmnccmes.

__Eacts2r_1_liact9r_2
TSAT3 -.83308 -.28291

WTHD6 .78038 .05750
TSAT2 -.74885 -.26979

WTHD3 .68278 .41613
WTHD2 .6595] .45445
WTHDS .64704 .4352]
WTHD4 .62619 .57773

TSAT] -. 14565 -.77702
WTHD] .211 17 .74754
TSAT4 -.40530 -.61486

Nate, TSAT = task satisfaction and WTHD = task withdrawal. Numbers represent item
numbers. Solution obtained forcing 2 factors with a principal components analysis and
Varimax rotation.

 

in the sequenced group (M = 4.10, SD = .72) reported their training was signiﬁcantly
more focused than trainees in the unsequenced group (M = 3.70, SD = .74), t(108) = 2.93,
p < .01, d = .54. No signiﬁcant difference between goal framing groups was found on
this manipulation check, t(108) = .95, n.s., d = .17.

To further verify that the sequencing training strategy was effective, the relative
time trainees spent reviewing various pages of the manual was investigated. In the ﬁrst
block, sequenced trainees were told to focus on labeling. Trainees in the sequenced
group (M = 131.50, SD = 82.01) spent signiﬁcantly greater time in the ﬁrst block

studying material relevant to labeling than trainees in the unsequenced condition (M =

.3. v a 8 250E288 0.8 88on .8 2. mecca—0200 dam

 

9m.- mm.- 3: Se.- mmr mo. 2.- 0o: if me. man Von cor mm wwm a mum- 233m 3

2“.- ST mNr cm.- 3..- 2. E .- war 2 .- 3. No . mo. 8.- cc. 3m mods-N- macaw-om
mm: EV.- mmr war act 3. v6.- 8. 5.- av. no. mo. vor 36mm 3.89 vouoomdm
act 3 .- 3 .- mac 3 .- Z”.- 3 .- we. met vm. m _. :- wo. 26$ bmsmc- 3.85.3
omr mm: 3“.- hm.- hm.- No. Em.- 8. 3 .- oo- vor vor 8.- Nh.m n5. 2 gas—SN
Gr not 5....- mm.- mm: co. m 2 .- cm. 8. mm. wot no. 8.- 05. SN ammo—maﬁa
Not me: we; cm: 2“.- vo. w _ .- em. vor em. no. I. No. 3.. 34m QEEmdm

mm: Sr 3..- 3».- Gr 8: co: hm. me. am. 2. m _. 8. mp. aﬂm Ce_um.vm
om: em: 3...- ~m.- mm: we. 3.- am. 2. S. S. 3. co. 8; vmd Nam—82.3

8.- G.- s..- 8.- 8.- 8. 2.- 2.- 8. 8. 2. 8. 8. S. 88 282.8
2.- :- :.- 8.- 8. 8.- 8.- 2. 8.- 2.- :- 8. :. 5.8 8.8 nee-52.8
8.- 8.- 2.- z.- 8- 8.- 2.- 2. 8.- 8. 2. 8. 8.- 8.8 8.88 m§-§<.8
8.- 8.- 8.- 8.- 8.- 8. 8.- 8.- 8.- 3.. 8.- 8. 8.- 8.8 8.88 25-8-62.2
8.- 8.- 8.- 2.- S. 8. 8.- 8. 8.- 8.- 8. 2. 2. 8. 38 Nam-52.2
3..- :..- 2.- 8.- 8.- 2. 2.- 2. 2.- 8. 2. 8. 8. 8. 88 8.52.:
8.- 8.- 8.- 8.- 8.- 8.- 2.- 2. S. 2. 8. 8. 8. 8. 88 85.2
2..- 8- 8.- 8.- 8.- 8.- 2.- 8. 8. 8. 2. 2. 8. S2 88 885.2
8.- 8.- 2..- 2.- 8.- 8.- 3.- 8. 2. 2. 8.- 8. 8.- 8.. 8.8 85.:
83 8. 8. G. 8. 8. 8. 8.- 8. 8.- 8.- 2.- 8.- 82 :8 85.2
:8. 8. 8. 8. 8. 8. 8.- 2. 2..- 8.- 2.- 8.- 8. 2.8 882.2
E: 8. 8. 8. 8. 8.- :. 8.- 8.- 2.- 2.- 8. 3.8 285.:

88 8. 8. 2. 2.- 2. 8.- 8.- 2.- 8.- 8. 2.8 M8822

8.8 8. 8. 8.- 2. 8.- z.- 2.- 2.- 82 8.... 8258

$3 2. 8.- 8. 8. 2.- 8.- 3.- 8. 2.2. 2889.8

88 8.- 8. 2- 2.- 8.- 2.- 8. 2mm 88.58

88 8. 8. 8.- 8. 8.- 8. 8+ 2052.8

ES 2.- 8. 8.- S. G. 8.2 2802.8

38 2.- 8.- 8.- 8.8 :8 £83

-- 8. 8. 2.. 8. 382808

- 8. 8. 2.. 888

- 8. 8. 2802

 

9 NH 2 OH m m N o m w m N H um aqua-lg

.moﬁmtg 358 90 82322020885 28 move-35m oﬁﬁtomon $.2an

82

 

 

- 8. 8. 2. 2.. 8. 8. 8. 2. 2. 8.- 2. 8. 8.- 2. 8. 8. 22828
-- 8. 2. 8. 2.. 2.. 2.. 8. 8. 2.- 8. 8. 8. 8. 2.. 8. 888.8
- 8. 8. 2.. 2.. 8. 8. 2.. 2.- 8. 8. 8.- 8. 8. 2.. 8.828

-- 2. 8. :. 2. 8. 8. 8.- 2.- 8. :.- 8.- z. 2. 888.8

:8 8. 8. :.. 8. 8. 8. 8. 2.. 2. 8. 8. 8. 5.3828

:8 8. 8. 8. 8. 2. 2. 8. 2.. 8. 8. 8. 883.8

28.8 8. 8. 8. 8. 2. 3. 8. 8. 8. 8. 288.8

28.8 8. 8. 2. 8. 8. 8. 8. 8. 8. 8.28.8

A88 8. 8. 8. 8. 8. 8. 8. 8. 2582.8

83 8. 2. 8. 8. 8. 2.. 8. 8.82.8

- 8. 8. 2.. 8. 2. 2. 85-528

- 8. 8. 2. 8. 8. 85-528

-- 8.- 8. 8. 8. _§-§<.2

A88 8. 8. 8. 2252.2

28.8 2.. 8. :.-.-§<.2

8.8 8. 888.2

88 888.2

1m 08 8 mm 8 on 8 8N mm mm 8 cm 2 8 E 2 2 03.85

A33

83
92.62, SD = 55.93), 1(93) = 2.893, p < .01. Following the sequence directions, trainees in

the sequence condition (M = 184.74, SD = 95.05) spent more time in the second block
reviewing engage information than trainees in the unsequenced condition (M = 75.61 , SD
= 48.75),1(78) = 7.544, p < .01. Finally, following sequencing instructions in the third
block, trainees in sequence condition (M = 245.78, SD = 108.89) reviewed prioritization
information to a greater extent than trainees in the unsequenced condition (M = 120.79,
SD = 92.74), t(108) = 6.49, p < .01.

The second manipulation check was for the effectiveness of the mastery and
performance manipulations. Five items were used to assess each state goal orientation.
Cronbach’s alpha for mastery and performance state orientation scales were .61 and .78,
respectively. An independent groups t-test indicates that trainees in mastery-framed
training (M = 3.56, SD = .49) did not report higher levels of mastery state orientation
than trainees in the performance-framed training (M = 3.59 , SD = .47), t(108) = -.36,
n.s., d = -.07. No signiﬁcant difference in mastery state emerged between the sequenced
and unsequenced conditions, t(108) = .69, n.s., d = .12.

The performance manipulation did create a signiﬁcant difference in performance
state orientation for performance trainees (M = 3.64, SD = .61) compared to the mastery
trainees (M = 3.28, SD = .70), t(108) = 2.95, p < .01, d = .54). No signiﬁcant differences

in performance state emerged between the sequenced and unsequenced conditions, 1(1 08)

 

3 The homogeneity of variance assumption was violated for this test, as indicated by a signiﬁcant Levene’s
F-test. While t-tests performed on samples with different variances can be misleading, this effect is minor
when samples are large and of equivalent size (Hays, 1988, p. 304). None-the-less, to provide a
conservative test, the I value reported here is based on degrees of freedom adjusted for the use of two
variance estimates in calculating Las opposed to the pooled estimate that is ordinarily employed.

‘ As above, the reported 1 value is based on corrected degrees of freedom.

84
= -.16, n.s., d = .03. These results suggest that the performance manipulation was

successful in eliciting a performance-oriented state, but the mastery manipulation was not
successful in eliciting a mastery-oriented state.

Although the current study utilized random assignment of subjects, it is possible
that differences in dispositional goal orientation are the underlying cause for the
manipulation effects reported above. Research has indicated that dispositional and state
orientation are related (Boyle & Klimoski, 1995). In this study, the zero-order correlation
between performance orientation and performance state was .40 (p < .05), between
learning orientation and mastery state was .46 (p < .05). To investigate the possibility
that randomization was not effective, independent group t-tests were run on goal framing
groups for both learning, t(108) = .23, n.s., d = .04, and performance orientation, t(108) =
1.55, n.s., d = .29. Neither of these analyses indicated signiﬁcant differences in goal
orientation by condition.

As a ﬁnal step in preparing the data for analysis, the relationship between the two
attentional focus measures was investigated. While the correlation between self-report
measures for block 1 and block 3 was .64 (p < .01), the correlation between self-report
measures and time-based measures was generally much lower. The most relevant
relationships are those between measures that correspond to similar time periods. The
correlation between block 1 self-report and block 1 study time was .24 (p < .01). The
correlation between block 3 self-report and block 3 study time was .45 (p < .01). Because
these correlations do not suggest these measures are tapping the same construct, both
measures are tested. Thus, analyses with attentional focus will be tested twice, once with

the self-report measure and once with the time-based measure.

84
= -.16, n.s., d = .03. These results suggest that the performance manipulation was

successful in eliciting a performance-oriented state, but the mastery manipulation was not
successful in eliciting a mastery-oriented state.

Although the current study utilized random assignment of subjects, it is possible
that differences in dispositional goal orientation are the underlying cause for the
manipulation effects reported above. Research has indicated that dispositional and state
orientation are related (Boyle & Klimoski, 1995). In this study, the zero-order correlation
between performance orientation and performance state was .40 (p < .05), between
learning orientation and mastery state was .46 (p < .05). To investigate the possibility
that randomization was not effective, independent group t—tests were run on goal framing
groups for both learning, t(108) = .23, n.s., d = .04, and performance orientation, t(108) =
1.55, n.s., d = .29. Neither of these analyses indicated signiﬁcant differences in goal
orientation by condition.

As a ﬁnal step in preparing the data for analysis, the relationship between the two
attentional focus measures was investigated. While the correlation between self-report
measures for block 1 and block 3 was .64 (p < .01), the correlation between self-report
measures and time-based measures was generally much lower. The most relevant
relationships are those between measures that correspond to similar time periods. The
correlation between block 1 self-report and block 1 study time was .24 (p < .01). The
correlation between block 3 self-report and block 3 study time was .45 (p < .01). Because
these correlations do not suggest these measures are tapping the same construct, both
measures are tested. Thus, analyses with attentional focus will be tested twice, once with

the self-report measure and once with the time-based measure.

85
1.. S Eﬁﬂ

Hypothesis 1 suggests both descriptive and attributional measures of negative
feedback are lower in the mastery goal condition than in the performance goal condition.
Similarly, hypothesis 3a suggests that sequencing and goal framing should interact in
determining magnitude judgments, controlling for actual performance. To control for
performance, the scores obtained at the ﬁrst trial in each block are entered as a varying
covariate. This procedure tests how training strategies affect perceptions of negative
feedback independent of their effect on actual performance.

Table 8 presents the RM-AN OVA for the manipulation effects on descriptive
magnitude of negative feedback. The results show signiﬁcant between and within
subjects covariate effects, indicating that, as expected, score values had an effect on the
magnitude judgments. The goal effect was marginal, E(1,105) = 3.15, M3,; = .82, p < .10,
112: .03), providing some support for hypothesis 1. Trainees in the mastery framed
training tended to perceive their feedback as less negative regardless of the actual score
values. This effect is depicted in Figure 5. The graph indicates that while mastery
training lowered perceptions of descriptive magnitude for blocks 1 and 2, this effect
disappeared for block 3.

While the sequencing main effect noted in Table 8 was signiﬁcant, the interaction
between goal and sequencing was not. These results do not support the interaction

hypothesis presented in 3a. The main effect for sequencing on descriptive magnitude was

86

 

 

Table8. {‘I‘q‘c u‘g. ,‘ «it... ' 0 no.1 ‘RU-ak. ; 0 Mt“ lo." M
l . . . l E . ﬂ 1] 1
Effect df E 112
Between Subjects
Covariatesa l 3683* .26
Sequencing l 5.72”“ .05
Goal 1 3.15‘1L .03
Seq. by Goal 1 .10 .00
Within-group Error 105 (.82)
Within Subjects
Covariatesa 1 143.63* .40
Block 2 .77 .01
Seq. by Block 2 1.57 .02
Goal by Block 2 .93 .01
Seq. by Goal by Block 2 .69 .01
Within-Group Error 211 (.45)

 

aCovan'ates include performance scores for trials 1, 4, and 7.

*p<.05
'l'p<.10

 

somewhat unexpected, but it does provide general support for the secondary effects (i.e.,
sequencing to magnitude; goal frame to diagnosticity perceptions) that provide the
foundation for hypothesis 3a. This effect is depicted in Figure 6. As the graph indicates,
compared to trainees in the unsequenced condition, trainees in the sequenced condition
reported lower descriptive magnitudes of negative feedback throughout the experiment.
Table 9 presents the RM-AN OVA results for the attributional magnitude of

negative feedback. While this analysis indicates signiﬁcant effects for score covariates

87

 

 

 

Descriptive Magnitude

 

 

 

 

 

—O— Mastery

- - <> - - Performance

 

 

 

 

 

2 3

Block Number

 

Figure 5. Descriptive magnitude of negative feedback by goal condition.

 

 

 

3

:3

5‘

3

'5

a.

t

g 2.9 j~
2.7 «

 

2.5

 

 

 

+ Sequenced

 

- - Cl- - - Unsequenced

 

 

 

 

 

2 3

Block Number

 

Figure 6. Descriptive magnitude of negative feedback by sequencing condition.

 

88

 

 

Table9. {‘O‘a‘cu‘e. -..r-. ° 0 111. ‘ {kl-at. ,5 0 !-..ru° l:‘% H
'l . l . l E . ﬁ 1] 1

Effect df F. ‘12

Between Subjects
Covariatesa 1 2594* .20
Sequencing 1 1 .07 .01
Goal 1 .48 .00
Seq. by Goal 1 .42 .00
Within-group Error 105 (1.96)

Within Subjects
Covariatesa 1 21.33* .09
Block 2 5.46* .05
Seq. by Block 2 .11 .00
Goal by Block 2 .98 .01
Seq by Goal by Block 2 1.12 .01
Within-Group Error 211 (.25)

 

aCovariates include performance scores for trials 1, 4, and 7.

*p<.05

 

and block, the effect for goal framing is not signiﬁcant, E( 1,105) = .48, M3}: = 1.96, n.s.,
n2= .00). Similarly, the sequencing main and interaction effects are not signiﬁcant.
Thus, for attributional magnitude of perceptions, neither hypothesis 1 nor 3a were
supported.

Hypothesis 2a suggests that the diagnosticity judgments should be enhanced by
the sequencing manipulation. Hypothesis 3b predicts that goal frame and sequencing
interact to affect diagnosticity judgments. Table 10 presents the RM-AN OVA results for

diagnosticity of feedback. This table indicates that, while covariates had a signiﬁcant

89

 

 

TablelO. {‘o‘e‘r u‘q. 20.2. ' I 1111‘ {U-ak. ; I mum Iq" H
l' . . [E I] I
Effect df E 112
Between Subjects
Covariatesa 1 3 1.51 * .23
Sequencing 1 .07 .00
Goal 1 .29 .00
Seq. by Goal 1 .13 .00
Within-group Error 105 (1.56)
Within Subjects
Covariatesa 1 16.32“ .07
Block 2 2.511' .02
Seq. by Block 2 .23 .00
Goal by Block 2 1.94 .01
Seq. by Goal by Block 2 .13 .00
Within-Group Error 211 (.44)

 

aCovariates include performance scores for trials 1, 4, and 7.

*p<.05
fp<J0

 

effect, none of the manipulations or interactions had a signiﬁcant effect on these
judgments. Thus, no support was obtained for hypothesis 2a or 3b.

Hypothesis 2b suggests that over time diagnosticity judgments should increase.
The block effect obtained in the within-subj ects analysis provides marginal support for
this hypothesis, E(2,211) = 2.51, M3,; = .44, p < .10, n2= .03). The trend for diagnosticity
indicates that diagnosticity judgments rose ﬁom the ﬁrst to the second block, but fell

again in the third (M = 2.85, SD = 1.00 for block 1; M = 3.20, SD = 1.01 for block 2; M =

90
3.03, SD = .94). This trend may reﬂect the difﬁculty of the more complex aspects of the

task which become the focus in later trials.

Hypothesis 20 suggests that trainees in the sequenced condition perceive greater
diagnosticity of feedback for material that is currently the focus of sequencing than for
material that has yet to be focused. This hypothesis suggests that diagnosticity for
labeling, which is the focus of the ﬁrst block in the sequence condition, should be greater
than diagnosticity for both engaging and prioritizing in block 1. In block 2, this
hypothesis suggests that engaging diagnosticity should be greater than prioritization
diagnosticity. Paired sample t-tests run on only the trainees in the sequenced condition
indicate that the diagnosticity for labeling (M = 3.13, SD = 1.03) is greater than both the
diagnosticity for engaging (M = 2.70, SD = .98), 1(53) = 3.14, p < .01, r = .51, and the
diagnosticity for prioritizing (M = 2.74, SD = .90), t(53) = 2.80, p < .01, r = .45 for the
ﬁrst trial block. For the second block, the diagnosticity for engaging (M = 3.38, SD =
1.04) is greater than the diagnosticity for prioritizing (M = 2.60, SD = .96), 1(53) = 5.62, p
< .01, r = .48. These analyses support hypothesis 2c. However, post-hoc analyses
indicate that some of these diagnosticity trends are also obtained for trainees in the
unsequenced condition.

For trainees in the unsequenced condition, diagnosticity of labeling (M = 2.73, SD
= 1.15) is not signiﬁcantly different than diagnosticity of engaging in the ﬁrst block (M =
2.54, SD = 1.04), 1(55) = 1.47, n.s., r = .56. Diagnosticity of labeling is, however, greater
than diagnosticity of prioritizing feedback (M = 2.37, SD = .98), 1(5) = 2.70, p < .01, r =
.54 in the ﬁrst block. Further, engaging diagnosticity (M = 2.92, SD = 1.09) is greater

than diagnosticity of prioritizing feedback (M = 2.46, SD = .90) in the second block, 1(55)

91
= 4.24, p < .01, r = .58. Therefore, prioritization diagnosticity seems to have the lowest

diagnosticity for all trainees, regardless of sequencing condition. This effect likely results
from the difficulty of that task component. And, because sequencing involves only one
ordering, the manipulation effect cannot be disentagled from effects of task component
difﬁculty. The sequencing manipulation was designed to start with the most simple task
component and move towards the more complex. Thus, while hypothesis 2c is

supported, this effect may reﬂect in part the nature of the task and the design of the
sequence manipulation.

Hypothesis 2d requires a direct comparison of sequenced and unsequenced
trainees. This hypothesis predicts that for a particular block, trainees in the sequenced
condition should judge feedback relevant to the sequenced topic more diagnostic than
trainees who are not in the sequenced condition. Thus, to test this hypothesis,
diagnosticity should be compared between sequenced and unsequenced groups for
labeling in block 1, engaging at block 2, and prioritizing at block 3. These comparisons
follow the focus of sequencing. Although this analysis procedure conducts 3 separate
tests where one omnibus test could be used, these tests should be considered “protected”
from type I error inﬂation because they directly reﬂect the a priori hypothesis.

For the ﬁrst block, trainees in the sequenced condition (M = 3.13, SD = 1.03)
reported labeling diagnosticity that was marginally higher than trainees in the
unsequenced condition (M = 2.73, SD = 1.15), t(108) = 1.91, p < .10, d = .36. For the
second block, trainees in the sequenced condition (M = 3.38, SD = 1.04) reported
signiﬁcantly higher diagnosticity for engaging feedback than trainees in the unsequenced

condition (M = 2.92, SD = 1.09), t(108) = 2.26, p < .05, d = .46. For the third block,

92
trainees in the sequenced condition (M = 2.72, SD = 1.08) and the unsequenced condition

(M = 2.44, SD = 1.07) reported similar levels of prioritization diagnosticity, t(108) =
1.38, n.s., d = .26. These analyses provide partial support for hypothesis 2d. More
speciﬁcally, it appears that while the sequencing training strategy enhanced diagnosticity
for labeling and engaging, the diagnosticity of prioritization was unaffected. This ﬁnding
may again reﬂect the difﬁculty of the prioritization activity and resulting ambiguity of
feedback for that dimension of performance. Again, however, because the sequencing
involves only one ordering, the sequencing effect cannot be disentagled from effects of
task component difﬁculty.

In order to provide a complete test of individual differences effects, learning
orientation, performance orientation, negative affectivity, and cognitive ability are
entered simultaneously into a regression predicting the dependent variables suggested by
each hypothesis. As these variables are relatively uncorrelated, simultaneous entry
should not alter the derived beta-weights estimates and, consequently, the conclusions
drawn. For this analysis, individual difference variables were assessed using a
hierarchical regression strategy, controlling for manipulation effects and actual
performance on the trial just prior to the collection of the relevant dependent variable.
Actual differences in performance may result from these factors, but performance
differences are controlled in order to test how different individuals react to similar
performance levels.

First, hypothesis 4a suggests that cognitive ability raises diagnosticity judgments.

Table 11 displays the regression results relevant to this hypothesis. The only signiﬁcant

93
predictors of this dependent variable are actual score and learning orientation. The effect

for learning orientation was not hypothesized, but it reﬂects a trend for individuals high
in learning orientation to view their feedback as more diagnostic.

As diagnosticity data was collected at three points in time, change analyses were
conducted to determine if individual differences affected the change in diagnosticity over
time. These tests were conducted by controlling for previous diagnosticity judgments in
a hierarchical regression. Table 12 indicates that initial diagnosticity judgment and score
are signiﬁcant predictors of diagnosticity judgment made at the beginning of the second
block. Learning orientation had a marginal effect on this change, with individuals high in
learning orientation exhibiting more positive change in diagnosticity. Table 13 displays
this analysis for the change from block 2 to block 3. As the table depicts, early
diagnosticity judgments and performance level predict the change in diagnosticity, but
individual differences do not. More speciﬁcally, cognitive ability did not predict initial
diagnosticity judgments or the change in diagnosticity judgments over time. These
results do not support hypothesis 4a.

Cognitive ability was also hypothesized to inﬂuence ﬁnal knowledge and skill
outcomes, as stated in hypothesis 4b. Tables 14 and 15 contain the regression results for

each of these dependent variables. In both tables, the effects of cognitive ability are

94

 

 

Tablell. l'l!,.-. Ii"I ' ‘u‘ II‘I.--..II I 'IH-I'uII 'I I “II...
Step: Variable(s) R2 df AR2 Adf [3a
1: Training Strategy .00 2 -.- --

Goal -.06

Seq. .03
2: Strategy Interaction .00 3 .00 1

Goal x Seq. .04
3: Performance .06 4 .06* 1

Score (T1) .24*
4: Individual Differences .16* 8 .10* 4

Cognitive Ab. .09

Learning Om. .24*

Perform. Om. -. 14

Negative Aff. .14

 

a The [3’s refer to standardized regression coefﬁcients associated with each step of the
hierarchical regression.

*p<.05

 

positive and signiﬁcant, indicating that individuals with higher levels of cognitive ability
learned more and performed better at the end of the experiment. These results support
hypothesis 4b. Table 14 also indicates that performance orientation is a signiﬁcant
predictor of knowledge test scores. The sign of the beta-weight indicates that ﬁnal

knowledge was signiﬁcantly lower for individuals with high performance orientations.

Tablell. II. °I --.. Ii' ‘

 

 

Step: Variable(s) R2 df ARZ Adf a“
1: Training Strategy .00 2 - - --
Goal -.06
Seq. .03
2: Strategy Interaction .00 3 .00 1
Goal x Seq. .04
3: Performance .06 4 .06* 1
Score (T1) .24*
4: Individual Differences .16* 8 .10* 4
Cognitive Ab. .09
Learning Om. .24*
Perform. Om. -. 14
Negative Aff. .14

 

3‘ The B’s refer to standardized regression coefﬁcients associated with each step of the

hierarchical regression.

*p<.05

 

positive and signiﬁcant, indicating that individuals with higher levels of cognitive ability

learned more and performed better at the end of the experiment. These results support

hypothesis 4b. Table 14 also indicates that performance orientation is a signiﬁcant

predictor of knowledge test scores. The sign of the beta-weight indicates that ﬁnal

knowledge was signiﬁcantly lower for individuals with high performance orientations.

 

Tablell. II 'I 'i' -

 

 

 

Step: Variable(s) R2 df AR2 Adf Ba
1: Training Strategy .00 2 - - --
Goal -.06
Seq. .03
2: Strategy Interaction .00 3 .00 1
Goal x Seq. .04
3: Performance .06 4 .06* 1
Score (T1) .24*
4: Individual Differences .16* 8 .10* 4
Cognitive Ab. .09
Learning Om. .24*
Perform. Om. -. 14
Negative Aff. .14
a The 13’s refer to standardized regression coefficients associated with each step of the
hierarchical regression.
* p < .05

 

positive and signiﬁcant, indicating that individuals with higher levels of cognitive ability

learned more and performed better at the end of the experiment. These results support

hypothesis 4b. Table 14 also indicates that performance orientation is a signiﬁcant

predictor of knowledge test scores. The sign of the beta-weight indicates that ﬁnal

knowledge was signiﬁcantly lower for individuals with high performance orientations.

 

 

Table12. II I.--.. Ii“I ' 'H‘ II I II I -.I° I'JI. ..II
feedback.
Step: Variable(s) R2 df AR2 Adf pa
1: Training Strategy .02 2 - - --
Goal .08
Seq. .10
2: Strategy Interaction .02 3 .00 1
Goal x Seq. .08
3: Previous Perception .21 * 4 .19* 1
Diagnosticity (B1) .43*
4: Performance .34* 5 .14* 1
Score (T4) .39*
5: Individual Differences .37* 9 .02* 4
Cognitive Ab. -.01
Learning Om. .141’
Perform. Om. -.06
Negative Aff. -.00

 

a The B’s refer to standardized regression coefﬁcients associated with each step of the

hierarchical regression.

*p<05
Tp<J0

 

As noted in the discussion of task characteristics, the frnal performance score is

composed of a number of skill indicators including label actions, label errors, engage

actions, engage errors, and prioritization errors. The regression results predicting these

components of ﬁnal score, not reported in tabular form here, suggest that no individual

differences signiﬁcantly predict label actions, label errors, and engage actions. Cognitive

 

 

Tablel3. II I..-.. Ii"I ‘ '°‘ 00"..10l0 ' III. lt-,l§‘l!e°|0 ..I I
feedback.
Step: Variable(s) R2 df ARZ Adf p“
1: Training Strategy .01 2 -.- --
Goal .03
Seq. .14
2: Strategy Interaction .01 3 .00 1
Goal x Seq. .03
3: Previous Perception .60* 4 .59* l
Diagnosticity (B2) .77*
4: Performance .63* 5 .04* 1
Score (T7) .21*
5: Individual Differences .64* 9 .01 4
Cognitive Ab. -.08
Learning Om. .00
Perform. Om. -.02
Negative Aff. .02

 

a The 13’s refer to standardized regression coefficients associated with each step of the
hierarchical regression.

*p<.05

 

ability does, however, predict the number of engagement ([3 = -.27, p < .01) and
prioritization errors ([3 = -.46, p < .01) in the ﬁnal trial, controlling for training strategy
manipulations and the other individual differences variables. The effects of cognitive
ability on these two performance components suggests that the overall effect of cognitive

ability on performance is a result of a reduction in engagement and prioritization errors.

Table 14.

 

 

 

Step: Variable(s) R2 (if AR2 Adf Ba
1: Training Strategy .01 2 -.- --
Goal -.07
Seq- -.04
2: Strategy Interaction .01 3 .00 1
Goal x Seq. .07
3: Individual Differences .45* 7 .44* 4
Cognitive Ab. ,57*
Learning Om. -.05
Perform. Om. -.28"'
Negative Aff. -.01

 

a The 13’s refer to standardized regression coefﬁcients associated with each step of the
hierarchical regression.

*p<.05

 

As the rules for engaging and prioritizing are the most complex, this ﬁnding is not
surprising.

Next, individual differences variables were tested for their inﬂuence on magnitude
judgments. Hypothesis 4c suggests individuals higher in negative affectivity perceive
higher levels of negative feedback. Tables 16 to 21 present the regression of attributional
and descriptive magnitude on individual differences. These analyses indicate that the
negative affectivity hypothesis was not supported either for initial magnitude estimates or

for change in these estimates.

98

 

 

TablelS. IIAI.e.Ii“‘I ‘ 'u' OI‘Q.-IOO 11-,I'U.IIOI.,O ‘ I‘
Step: Variable(s) R2 df AR2 Adf Ba
1: Training Strategy .00 2 - - --
Goal -.06
Seq. -.04
2: Strategy Interaction .01 3 .00 1
Goal x Seq. -.08
3: Individual Differences .20* 7 .20* 4
Cognitive Ab. .42*
Learning Om. -.09
Perform. Om. -.04
Negative Aff. -.08

 

a The B’s refer to standardized regression coefﬁcients associated with each step of the

hierarchical regression.

*p<.05

 

Hypothesis 4d and 4e predict that learning and performance orientation traits

inﬂuence magnitudes judgments. Table 16 indicates that initial attributional magnitude

judgments support both of these hypotheses. The effect for cognitive ability, learning and

performance orientation are all signiﬁcant in predicting attributional magnitude. The

effect for individuals with higher levels of cognitive ability to note lower levels of

attributional magnitude was unexpected.

-l
C
o
O
-
O
O
-
O
.

Table16. to 9.2 I'

It... Ila. lla’OLQ‘ I

 

 

neganxekedhaek.
Step: Variable(s) R2 df AR2 Adf [3“
1: Training Strategy .02 2 -.- --
Goal -.12
Seq. -.10
2: Strategy Interaction .03 3 .01 1
Goal x Seq. .16
3: Performance .05 4 .02 1
Score (T1) -.13
4: Individual Differences .23* 8 .18* 4
Cognitive Ab. -.18*
Learning Om. -.27*
Perform. Om. .23*
Negative Aff. .04

 

a The B’s refer to standardized regression coefﬁcients associated with each step of the
hierarchical regression.

*p<.05

 

Tables 17 and 18 present change analyses for attributional magnitude judgments.
For the second judgment the only signiﬁcant predictor was the judgment in the ﬁrst
block. In the third block both the previous judgment and performance orientation were
positively associated with the attributional magnitude judgment. Thus, while
attributional magnitude is initially determined by performance level, cognitive ability and
goal orientation, this judgment remains relatively stable throughout the training. The

only signiﬁcant predictor of change was performance goal orientation.

Table 16. II I..-. Ii‘ ‘

II'I.--_ II I I ... IlII|_OII. IItuIII‘I

 

 

negatmefeedhaek.
Step: Variable(s) R2 df AR2 Adf a“
1: Training Strategy .02 2 - - --
Goal -.12
Seq. -.10
2: Strategy Interaction .03 3 .01 1
Goal x Seq. .16
3: Performance .05 4 .02 1
Score (T1) -.13
4: Individual Differences .23* 8 .18* 4
Cognitive Ab. -.18*
Learning Om. -.27"‘
Perform. Om. .23*
Negative Aff. .04

 

a The B’s refer to standardized regression coefﬁcients associated with each step of the

hierarchical regression.

*p<.05

 

Tables 17 and 18 present change analyses for attributional magnitude judgments.

For the second judgment the only signiﬁcant predictor was the judgment in the ﬁrst

block. In the third block both the previous judgment and performance orientation were

positively associated with the attributional magnitude judgment. Thus, while

attributional magnitude is initially determined by performance level, cognitive ability and

goal orientation, this judgment remains relatively stable throughout the training. The

only signiﬁcant predictor of change was performance goal orientation.

Table 17. .. .t.. -
magnimdehfnegatixefeedhaek.

100

0.1"...4.

 

 

 

Step: Variable(s) R2 (if ARZ Adf [3“
1: Training Strategy .02 2 -.- --

Goal -.02

Seq. -.13
2: Strategy Interaction .02 3 .00 1

Goal x Seq. .01
3: Previous Perception .48* 4 .47* 1

Att. Magnitude (B1) .70*
4: Performance .59* 5 .10* 1

Score (T4) -.33*
5: Individual Differences .61* 9 .02 4

Cognitive Ab. -.11

Learning Om. -.08

Perform. Om. .02

Negative Aff. .07
a The 13’s refer to standardized regression coefficients associated with each step of the
hierarchical regression.
* p < .05

 

For descriptive magnitude judgments the only signiﬁcant predictors were

sequencing and score. Table 19 indicates that none of the individual differences affected

initial descriptive judgments; Tables 20 and 21 indicate that these variables also did not

inﬂuence changes in these judgments over time. Thus, the hypotheses regarding negative

affectivity and goal orientation are not supported for descriptive judgments. In summary,

 

 

 

101

TablelB. I'l'..e Ii"I ' ‘u' II‘I..-.II I ‘ III I-._I9""I..I.I.II-.

. l E . ﬂ 1] I
Step: Variable(s) R2 df AR2 Adf l3“
1: Training Strategy .01 2 -.- --

Goal -.02

Seq. a12
2: Strategy Interaction .01 3 .00 1

Goal x Seq. .00
3: Previous Perception .73* 4 .71 * 1

Att. Magnitude (B2) .85*
4: Performance .73* 5 .01 1

Score (T7) -.09
5: Individual Differences .75* 9 .02 4

Cognitive Ab. .04

Learning Om. -.03

Perform. Om. . 10*

Negative Aff. .06

 

a The 13’s refer to standardized regression coefficients associated with each step of the
hierarchical regression.

*p<.05

 

goal orientation traits predict attributional descriptions of feedback magnitude, but not

descriptive descriptions. These results provide partial support for hypotheses 4d and 4e.

102

 

 

Table19. II 'I --_. ' II I II I I II. ‘ II-.°II.I‘I
negativefeedhaek.
Step: Variable(s) R2 df AR2 Adf a“
1: Training Strategy .08* 2 - - --
Goal -.14
Seq. -.25*
2: Strategy Interaction .08* 3 .00 1
Goal x Seq. .10
3: Performance .16* 4 .08* 1
Score (T1) -.28"‘
4: Individual Differences .19* 8 .03 4
Cognitive Ab. .14
Learning Om. -.03
Perform. Om. .12
Negative Aff. .05

 

a The B’s refer to standardized regression coefficients associated with each step of the

hierarchical regression.

*p<.05

 

BreeeSSMedel

In order to examine the learning process, a series of hierarchical regressions are

conducted on the mediating variables of self-efﬁcacy, motivation to learn, and attentional

focus. Initial levels of these constructs are regressed on training strategy manipulations,

individual differences, performance in the ﬁrst trial, and the feedback perceptions.

Feedback perceptions are entered separately on the last step in order to determine if these

103

 

 

TableZO. 01...: Ii“I ‘ ‘°' 00"..-2'! I . II._J"IQ‘, II"
magnimdenfnegatixefeedhaek.
Step: Variable(s) R2 df AR2 Adf Ba
1: Training Strategy .03 2 -.- --
Goal -.10
Seq. -.12
2: Strategy Interaction .03 3 .00 1
Goal x Seq. -.04
3: Previous Perception .03 4 .00 1
Des. Magnitude (B1) .01
4: Performance .52* 5 .49* 1
Score (T4) -.71*
5: Individual Differences .54* 9 .02 4
Cognitive Ab. .05
Learning Om. -.08
Perform. Om. -.02
Negative Aff. .12

 

a The 13’s refer to standardized regression coefﬁcients associated with each step of the
hierarchical regression.

*p<.05

 

perceptions inﬂuence the learning process above and beyond the inﬂuence of individual
differences and past performance. Analyses for independent steps in the regression
equation are noted with small letters a, b, and c on the table. Effects for individual
differences on mediating processes were not hypothesized, so these are not discussed

here. Those effects are discussed later in the results section.

104

 

 

Tab1e21. II I---. Ii“I ‘ ‘u' II'I.--.II I ' III I-._I°‘II‘ I...
. 1 E . E 1] 1
Step: Variable(s) R2 df ARZ Adf pa
1: Training Strategy .02 2 -.- --
Goal -.02
Seq. -. 14
2: Strategy Interaction .02 3 .00 1
Goal x Seq. .01
3: Previous Perception .22* 4 .20* 1
Des. Magnitude (B2) .45*
4: Performance .38* 5 .15* 1
Score (T7) -.45*
5: Individual Differences .40* 9 .02 4
Cognitive Ab. .04
Learning Om. -.12
Perform. Om. .03
Negative Aff. .08

 

a The B’s refer to standardized regression coefficients associated with each step of the
hierarchical regression.

*p<.05

 

Anemignalﬁogis. Hypothesis 5 suggests that individuals who judge their
feedback to be more diagnostic have greater attentional focus on the study material. This
hypothesis can be tested for block 1 and block 3 for self-report measures, and for blocks

1, 2, and 3 for time-based attentional measures.

105
Regression results presented in Tables 22 and 23 indicate that diagnosticity does

not predict self-reported attentional focus in block 1, B = -.05, n.s., or the change in
attentional focus from block 1 to block 3, [3 = .08, n.s.. Tables 24 to 26 present the
regression results for the time based measures. For time-based attention measures,
diagnosticity does not account for signiﬁcant variance in attention for the initial measure
or for the second change measure. However, diagnosticity does predict the change in
attentional focus from block 1 to block 3, [3 = .24, p < .05. Thus, partial support was
obtained for the inﬂuence of diagnosticity on attention.

An unexpected but very interesting ﬁnding is the positive effect of descriptive
magnitude on initial attentional focus, [5 = .20, p < .05. This trend suggests that the
higher the descriptive magnitude, the greater the on-task attention. The direction of this
effect is opposite what might be hypothesized, and indicates a positive motivational effect
for negative feedback. Error feedback may have resulted in a motivating force to
improve performance, which is attempted by increasing on-task attention. In other
words, the positive inﬂuence of descriptive magnitude may reﬂect a self-set goal or
challenge effect whereby individuals who feel they are not doing well focus on the
material in order to so.

Self-Emcacy, Hypothesis 6 suggests that magnitude feedback perceptions affect
self-efficacy over and above the effects of actual performance. There are a number of
possible tests of this hypothesis, including tests for block 1 and block 3, and for

attributional and descriptive magnitude.

105
Regression results presented in Tables 22 and 23 indicate that diagnosticity does

not predict self-reported attentional focus in block 1, B = -.05, n.s., or the change in
attentional focus from block 1 to block 3, B = .08, n.s.. Tables 24 to 26 present the
regression results for the time based measures. For time-based attention measures,
diagnosticity does not account for signiﬁcant variance in attention for the initial measure
or for the second change measure. However, diagnosticity does predict the change in
attentional focus from block 1 to block 3, B = .24, p < .05. Thus, partial support was
obtained for the inﬂuence of diagnosticity on attention.

An unexpected but very interesting ﬁnding is the positive effect of descriptive
magnitude on initial attentional focus, [3 = .20, p < .05. This trend suggests that the
higher the descriptive magnitude, the greater the on-task attention. The direction of this
effect is opposite what might be hypothesized, and indicates a positive motivational effect
for negative feedback. Error feedback may have resulted in a motivating force to
improve performance, which is attempted by increasing on—task attention. In other
words, the positive inﬂuence of descriptive magnitude may reﬂect a self-set goal or
challenge effect whereby individuals who feel they are not doing well focus on the
material in order to so.

Self-Emma, Hypothesis 6 suggests that magnitude feedback perceptions affect
self-efﬁcacy over and above the effects of actual performance. There are a number of
possible tests of this hypothesis, including tests for block 1 and block 3, and for

attributional and descriptive magnitude.

106
Table 22. Regressreneguanenfoumnalattemienalmueelfrenon).

 

 

Step: Variable(s) R2 df AR2 Adf p“
1: Training Strategy .01 2 -.- --

Goal .09

Seq. .05
2: Strategy Interaction .02 3 .01 1

Goal x Seq. .16
3: Individual Differences .14* 7 .12* 4

Cognitive Ab. .27*

Learning Om. .15

Perform. Om. -.08

Negative Aff. -.05
4: Performance .16* 8 .02 1

Score (T1) -.13
5a: Diagnosticity (B1)b .l6* 9 .01 1 -.05
5b: Descriptive Mag. (B1) .19* 9 .03* 1 .20*
Sc: Attributional Mag. (B1) .16* 9 .00 1 -.05

 

a The 13’s refer to standardized regression coefficients associated with each step of the
hierarchical regression.
bSteps a, b, and c were entered independently into the regression equation.

*p<.05

107
Table 23. RegressrenequanenfeuhangemattemrenaLfoemiSelfremrt).

 

 

Step: Variable(s) R2 df AR2 Adf Ba
1: Training Strategy .051 2 -.- --

Goal .171

Seq. .161
2: Strategy Interaction .061 3 .01 1

Goal x Seq. .17
3: Individual Differences .l3* 7 .06* 4

Cognitive Ab. -.06

Learning Om. .24*

Perform. Om. .01

Negative Aff. -.09
4: Previous Judgment .52* 8 .39* 1

Att. Focus S-R (B1) .67*
5: Performance .52* 9 .01 1

Score (T7) -.12
6a: Diagnosticity (B3)b 53* 10 .01 1 .09
6b: Descriptive Mag. (B3) .54* 10 .021 1 .181
6c: Attributional Mag. (B3) .53* 10 .00 1 -.09

 

a The 13’s refer to standardized regression coefﬁcients associated with each step of the
hierarchical regression.
bSteps a, b, and c were entered independently into the regression equation.

*p<.05
1p<J0

108
Table 24. Regressieneqnatienfoninitialattentienalfeemnme).

 

 

Step: Variable(s) R2 df AR2 Adf p“
1: Training Strategy .12* 2 -.- --

Goal -.33*

Seq. .03
2: Strategy Interaction .12* 3 .00 1

Goal x Seq. -.07
3: Individual Differences .30* 7 .18"' 4

Cognitive Ab. .43*

Learning Om. -.06

Perform. Om. -.04

Negative Aff. .02
4: Performance .30* 8 .00 1

Score (T1) -.02
5a: Diagnosticityb 30* 9 .00 1 .04
5b: Descriptive Mag. .30* 9 .00 l .00
5c: Attributional Mag. .30* 9 .00 1 -.03

 

' The B’s refer to standardized regression coefﬁcients associated with each step of the
hierarchical regression.
bSteps a, b, and c were entered independently into the regression equation.

*p<.05

109
Table 25. RegressreneguatienferﬁrsuhangeinattentienaLfoeuSﬁimel.

 

 

Step: Variable(s) R2 (If .R2 Adf B.
1: Training Strategy .10* 2 -.- --

Goal -.06

Seq. .31 *
2: Strategy Interaction .10* 3 .00 1

Goal x Seq. .04
3: Individual Differences .17"‘ 7 .071 4

Cognitive Ab. -.01

Learning Om. .10

Perform. Om. -.08

Negative Aff. -.22*
4: Previous Judgment .24* 8 .07* 1

Att. Focus Time (BI) .33*
5: Performance .27* 9 .03* 1

Score (T4) -.23*
6a: Diagnosticity (132)*’ .31* 1o .04* 1 24*
6b: Descriptive Mag. (B2) .28* 10 .01 1 -.12
6c: Attributional Mag. (BZ) .29* 10 .021 l -.181

 

a The 13’s refer to standardized regression coefﬁcients associated with each step of the
hierarchical regression.
bSteps a, b, and c were entered independently into the regression equation.

*p<05
1p<J0

Table 26.

 

 

 

Step: Variable(s) R2 df AR2 Adf Ba
1: Training Strategy .09* 2 - - --

Goal .11

Seq. .27*
2: Strategy Interaction .11* 3 .02 1

Goal x Seq. .25
3: Individual Differences .17* 7 .06 4

Cognitive Ab. -. l 81

Learning Om. .171

Perform. Om. -.07

Negative Aff. -.03
4: Previous Judgment .40* 8 .23* 1

Att. Focus Time (B2) .52*
5: Performance .41 * 9 .02 1

Score (T7) -. 14
6a: Diagnosticity (B3)b .41* 10 .00 1 .05
6b: Descriptive Mag. (B3) .41* 10 .00 l -.05
6c: Attributional Mag. (B3) .43* 10 .01 1 -.15

 

a The B’s refer to standardized regression coefﬁcients associated with each step of the

hierarchical regression.

bSteps a, b, and c were entered independently into the regression equation.

*p<05
1p<J0

1 1 1
Regression results, presented in Table 27, indicate that descriptive magnitude of

negative feedback had no effect on self-efﬁcacy in block 1, B = .03, n.s.. Descriptive
magnitude did have a marginal negative inﬂuence on the change in efficacy from the ﬁrst
to the ﬁnal block, B = -.13, p < .10, as Table 28 indicates. These tables also indicate that
attributional magnitude predicts both initial self-efﬁcacy, B = -.3 8, p < .05, and the
change in self-efﬁcacy, B = -.27, p < .05 as hypothesized. This ﬁnding suggests that the
interpretation of feedback as reﬂecting internal, stable causes signiﬁcantly inﬂuences the
self-efﬁcacy judgments of trainees. Further, these results suggest that the negative effect
of magnitude on self-efficacy is attenuated for descriptive, non-attributional
interpretations of feedback.

An unexpected ﬁnding was that diagnosticity had a signiﬁcant effect on initial
self-efﬁcacy judgments. Trainees who judged their feedback to be more diagnostic
reported higher initial levels of self-efﬁcacy, B = .36, p < .05. As this effect was not
hypothesized, it should be interpreted with caution.

WW Hypothesis 7 suggests that self-efﬁcacy in turn inﬂuences
motivation to learn. In order to test this hypothesis, a hierarchical regression analysis is
run controlling for training strategies, individual differences, past performance, and
feedback perceptions. Self-efﬁcacy is entered in the last step.

Results support hypothesis 7, as Table 29 indicates, because self-efﬁcacy
signiﬁcantly predicts initial motivation to learn, B = .49, p < .05. Change analyses,
however, suggest that self-efﬁcacy does not inﬂuence the change in motivation to learn

over time. In fact, Table 30 indicates that motivation to learn is fairly stable because it is

112
Table 27. Regressionsgnatrenfminitialselﬁeﬁieaey.

 

 

Step: Variable(s) R2 df ARZ Adf B“
1: Training Strategy .02 2 -.- --

Goal .01

Seq. .14
2: Strategy Interaction .03 3 .00 1

Goal x Seq. .11
3: Individual Differences .25* 7 .22* 4

Cognitive Ab. .29*

Learning Om. .35"

Perform. Om. -.02

Negative Aff. .05
4: Performance .25* 8 .00 1

Score (T1) .02
Sa: Diagnosticityb .36* 9 .11* 1 .36*
5b: Descriptive Mag. .25* 9 .00 l .03
5c: Attributional Mag. .36* 9 .11* 1 -.38*

 

a The B’s refer to standardized regression coefficients associated with each step of the
hierarchical regression.
l’Steps a, b, and c were entered independently into the regression equation.

*p<.05

113

Table 28. Regressienequatienfemhangeinselfeﬁieaey.

 

 

Step: Variable(s) R2 df AR2 Adf 13“
1: Training Strategy .01 2 - - --

Goal .01

Seq. .1 1
2: Strategy Interaction .01 3 .00 1

Goal x Seq. .01
3: Individual Differences .21"‘ 7 .20"‘ 4

Cognitive Ab. .23*

Learning Om. .32*

Perform. Om. -.15

Negative Aff. .01
4: Previous Judgment .60* 8 .39* 1

Self-Efﬁcacy (B1) .72*
5: Performance .63* 9 .03* 1

Score (T7) .21*
6a: Diagnosticity (133)b .63* 10 .00 1 .05
6b: Descriptive Mag. (B3) .64* 10 .011 l -.131
6c: Attributional Mag. (B3) .67* 10 .04* 1 -.27*

 

8' The B’s refer to standardized regression coefﬁcients associated with each step of the

hierarchical regression.

bSteps a, b, and c were entered independently into the regression equation.

*p<.05

113

Table 28. Regressienequatienferehangeineelfzefﬁeaex.

 

 

Step: Variable(s) R2 df AR2 Adf Ba
1: Training Strategy .01 2 - - --

Goal .01

Seq. .11
2: Strategy Interaction .01 3 .00 1

Goal x Seq. .01
3: Individual Differences .21* 7 .20“ 4

Cognitive Ab. .23*

Learning Om. .32*

Perform. Om. -. 15

Negative Aff. .01
4: Previous Judgment .60* 8 .39* 1

Self-Efﬁcacy (B1) .72*
5: Performance .63* 9 .03* 1

Score (T7) .21*
6a: Diagnosticity (133)b .63* 10 .00 1 .05
6b: Descriptive Mag. (B3) .64* 10 .011 1 -.131
6c: Attributional Mag. (B3) .67* 10 .04* 1 -.27"'

 

a The B’s refer to standardized regression coefﬁcients associated with each step of the

hierarchical regression.

bSteps a, b, and c were entered independently into the regression equation.

*p<.05

113
Table 28. Regressrenequatmnﬂrehangennselﬂeﬁieaex.

 

 

Step: Variable(s) R2 df ARZ Adf [3“
1: Training Strategy .01 2 -.- --

Goal .01

Seq. .11
2: Strategy Interaction .01 3 .00 1

Goal x Seq. .01
3: Individual Differences .21* 7 .20* 4

Cognitive Ab. .23*

Learning Om. .32*

Perform. Om. -.15

Negative Aff. .01
4: Previous Judgment .60* 8 .39* 1

Self-Efficacy (B1) .72*
5: Performance .63* 9 .03* 1

Score (T7) .21*
6a: Diagnosticity (133)" .63* 10 .00 1 .05
6b: Descriptive Mag. (B3) .64* 10 .011 1 -.131
6c: Attributional Mag. (B3) .67* 10 .04* 1 -.27*

 

a The B’s refer to standardized regression coefficients associated with each step of the
hierarchical regression.
bSteps a, b, and c were entered independently into the regression equation.

*p<.05

1 14
Table 29. Regressieneqrratienferinitialmetixatienjeleam.

 

 

Step: Variable(s) R2 df ..RZ Adf 13"
1: Training Strategy .01 2 -.- --

Goal .05

Seq. .08
2: Strategy Interaction .03 3 .02 1

Goal x Seq. .25
3: Individual Differences .28* 7 .25* 4

Cognitive Ab. .21*

Learning Om. .41 *

Perform. Om. -. 10

Negative Aff. .12
4: Performance .29* 8 .01 1

Score (T1) -.10
5a: Diagnosticityb 32* 9 .03* 1 20*
5b: Descriptive Mag. .30* 9 .01 1 .12
5c: Attributional Mag. .37* 9 .O9* 1 -.33*
6. Self-Efﬁcacy (B1)c .55* 12 .14* 1 .49*

 

8‘ The B’s refer to standardized regression coefﬁcients associated with each step of the
hierarchical regression.

bSteps a, b, and c were entered independently into the regression equation.

°The ﬁnal step was entered controlling for all perception variables.

*p<.05

1 14
Table 29. Regressioneqnatienforim’tiaLmetixatienreJeam.

 

 

Step: Variable(s) R2 df AR2 Adf Ba
1: Training Strategy .01 2 -.- --

Goal .05

Seq. .08
2: Strategy Interaction .03 3 .02 1

Goal x Seq. .25
3: Individual Differences .28* 7 .25* 4

Cognitive Ab. .21*

Learning Om. .41*

Perform. Om. -. l 0

Negative Aff. .12
4: Performance .29* 8 .01 1

Score (T1) -.10
5a: Diagnosticityb .32* 9 .03* 1 .20*
5b: Descriptive Mag. .30* 9 .01 1 .12
5c: Attributional Mag. .37* 9 .09* 1 -.33*
6. Self-Efﬁcacy (B1)° .55* 12 .14* 1 .49*

 

a The B’s refer to standardized regression coefficients associated with each step of the
hierarchical regression.

bSteps a, b, and c were entered independently into the regression equation.

6The ﬁnal step was entered controlling for all perception variables.

*p<.05

115
Table 30. Regresmnequatmnfenehangernmotrxahenielearn.

 

 

Step: Variable(s) R2 df AR2 Adf Ba
1: Training Strategy .01 2 -.- -—

Goal .06

Seq. .10
2: Strategy Interaction .03 3 .00 1

Goal x Seq. .24
3: Individual Differences .23* 7 .l9* 4

Cognitive Ab. .171

Learning Om. .36*

Perform. Om. -. 10

Negative Aff. .13
4: Previous Judgment .78* 8 .55* 1

Mot. to Learn (B1) .87*
5: Performance .78* 9 .00 1

Score (T7) .01
6a: Diagnosticity (B3)b .79* 10 .01 * 1 .13*
6b: Descriptive Mag. (B3) .78* 10 .00 1 -.06
6c: Attributional Mag. (B3) .78* 10 .00 1 -.08
7. Self-Efficacy (B3)° .79* 13 .00 1 .10

 

a The B’s refer to standardized regression coefficients associated with each step of the
hierarchical regression.

bSteps a, b, and c were entered independently into the regression equation.

cThe ﬁnal step was entered controlling for all perception variables.

*p<.05
19<.10

116
predicted by the ﬁrst motivation measure, B = .87, p < .05 but its change is not

inﬂuenced by score, feedback perceptions, or self-efﬁcacy.

Post-hoe analyses reveal that the relationship between motivation to learn and
self-efﬁcacy may not be as simple as the hypothesis suggests. When motivation to learn
is entered as a predictor of self-efﬁcacy, the results are Similar to those presented above.
Using the same regression strategy, motivation to learn predicts initial self-efﬁcacy
levels, B = .43, p < .05, AR2 = .43. Furthermore, motivation to learn predicts the change
in self-efﬁcacy from block 1 to block 3, B = .21, p < .05, AR2 = .02. This ﬁnding reveals
that the current method of data collection makes it difﬁcult to ascertain the causal order
of these constructs. Both constructs are measured using self-report measures at the same
point in time. These two variables are highly related in both block 1, r = .66, p < .01, and
block 3, r = .62, p < .01. As a result, these results reveal that either variable can serve as
a signiﬁcant predictor of the other.

Iminingﬂmeemes

The ﬁnal hypotheses suggest that different training outcomes are affected by
different learning processes. These hypotheses and related analyses are also conducted
using a hierarchical regression strategy. The effects for individual differences and
feedback perceptions are controlled, but they are not discussed in this section of the
results.

Knowledge, Hypothesis 8a suggests individuals with high motivation to learn
actually learn more from their attentional focus than trainees with low motivation to

learn. This hypothesis predicts an interaction between attentional focus and motivation

117
for the knowledge and skill training outcomes. Separate regression blocks were used to

test attention with self-report and time-based attentional measures.

Regression analyses for knowledge test score are contained in Tables 31.
Controlling for training strategy, score in the last block, and feedback perceptions, neither
self-efﬁcacy nor motivation to learn predicted signiﬁcant variance in this knowledge
outcomes. Attentional focus, measured with time, was a signiﬁcant predictor of
knowledge test performance. The motivation by self-report attention interaction was
marginally signiﬁcant, providing some support for hypothesis 8a. The graph of this
interaction in Figure 7 reveals that the lowest knowledge test score occurred for
individuals with low attention and low motivation to learn.

Skill, The same series of tests were conducted for ﬁnal simulation score to
determine if hypothesis 8a is supported for skill-based training outcomes. Table 32
shows the self-efficacy has a marginal effect on ﬁnal score. The hypothesized interaction
is also marginally signiﬁcant. This provides some support for hypothesis 8a regarding
skill outcomes. The size of this effect is very small, however, accounting for less than
2% of variance. A graph of this interaction in Figure 8 reveals that the lowest
performance was obtained by individuals with low levels of attention and motivation to
learn.

Attitudes, Hypothesis 8b suggests that attitudinal outcomes are directly
inﬂuenced by motivation to learn. Tables 33 presents regression results for task

satisfaction. These results indicate that both self—efﬁcacy and motivation to learn

118
Table 31.9mﬂ1eWedgmumemeeﬁeame.

 

 

Step: Variable(s) R2 df AR2 adf Ba
1: Training Strategy .01 2 -.- --

Goal -.07

Seq. -.04
2: Strategy Interaction .01 3 .00 1

Goal x Seq. .07
3: Individual Differences .45* 7 .44* 4

Cognitive Ab. .57*

Learning Om. -.04

Perform. Om. -.28*

Negative Aff. -.02
4: Performance .54* 8 .09* 1

Score (T7) .35*
5: Feedback Perceptions .60* 11 .05* 3

Diagnosticity (B3) .04

Attributional Mag. (B3) -.29*

Descriptive Mag. (B3) .22“
6a: Process Variables .60* 14 .01 3

Self-Efﬁcacy (B3) .10

Mot. to Learn (B3) -.02

Att. Focus Self Rep. (B3) .11
6b: Process Variablesb .63* 14 .03* 3

Self-Efﬁcacy (B3) .14

Mot. to Learn (B3) .00

Att. Focus Time (B3) .20*
7a: Process Interaction .62"‘ 15 .021 1

Mot. x Att Self Re . -.781
7b: Process Interactions .63* 15 .00 1

Mot x Att. Time -.12

 

8‘ The B’s refer to standardized regression coefﬁcients associated with each step of the
hierarchical regression.

bThe two a steps are run together and the two b steps are run together, but a and b steps
are run independently with the different measures of attention.

*p<05
1p<J0

119

 

 

comu=03<

Boa . . .0 . .
53:32

E: Isl.

 

 

.208 62 632305— wﬁuomwoa 53080:: $2 8 :22?on Ea 53:83 tomathom .h oSwE

 

 

53A 3 559592
emf 33
MI
:3- “a
o
i- m.
e. 9
.. D.
. an
mo. 1
.. 3
.. m
:o S
3
m
T Mmd a
1-
It?
3..
:3

 

 

 

120
Table 32. OrteralljeatforskilleutemneefﬁnaLSimulatienseere.

 

 

Step: Variable(s) R2 df AR2 Adf Ba
1: Training Strategy .00 2 -.- --

Goal -.04

Seq. -.06
2: Strategy Interaction .01 3 .00 1

Goal x Seq. -.08
3: Individual Differences .20* 7 .20* 4

Cognitive Ab. .42*

Learning Om. -.10

Perform. Om. -.04

Negative Aff. -.08
4: Performance .56* 8 .36* 1

Score (T7) .68*
5: Feedback Perceptions .56* 11 .00 3

Diagnosticity (B3) -.04

Attributional Mag. (B3) -.02

Descriptive Mag. (B3) -.01
6a: Process Variables .59* 14 .02 3

Self-Efﬁcacy (B3) .181

Mot. to Learn (B3) -.08

Att. Focus Self Rep. (B3) -.07
6b: Process Variablesb .58* 14 .02 3

Self-Efﬁcacy (B3) .191

Mot. to Learn (B3) -.12

Att. Focus Time (B3) -.02
7a: Process Interactions .60* 15 .011 1

Mot. x Att Self Re . -.701
7b: Process Interactions .59* 15 .01 l

Mot x Att. Time -.44

 

a The B’s refer to standardized regression coefficients associated with each step of the
hierarchical regression.

e two 3 steps are run together and the two b steps are run together, but a and b steps
are run independently with the different measures of attention.

*p<05
1p<J0

120
Table 32. QxeralLtestQLskilLerrteemeofﬁnalsimulatienseete.

 

 

Step: Variable(s) R2 df .18 Adf 11“
1: Training Strategy .00 2 -.- --

Goal -.04

Seq. -.06
2: Strategy Interaction .01 3 .00 1

Goal x Seq. -.08
3: Individual Differences .20* 7 .20* 4

Cognitive Ab. .42*

Learning Om. -.10

Perform. Om. -.04

Negative Aff. -.08
4: Performance .56* 8 .36* 1

Score (T7) .68*
5: Feedback Perceptions .56* 11 .00 3

Diagnosticity (B3) -.04

Attributional Mag. (B3) -.02

Descriptive Mag. (B3) -.01
6a: Process Variables .59* 14 .02 3

Self-Efficacy (B3) .181

Mot. to Learn (B3) -.08

Att. Focus Self Rep. (B3) -.07
6b: Process Variablesb .58* 14 .02 3

Self-Efﬁcacy (B3) .191

Mot. to Learn (B3) -.12

Att. Focus Time (B3) -.02
7a: Process Interactions .60* 15 .011 l

Mot. x Att Self Re . -.701
7b: Process Interactions .59* 15 .01 1

Mot x Att. Time -.44

 

a The B’s refer to standardized regression coefﬁcients associated with each step of the
hierarchical regression.

I’The two a steps are run together and the two b steps are run together, but a and b steps
are run independently with the different measures of attention.

*p<.05
1p<J0

121

68% 85:30.30: 3:3 33332: :o3088£ Emo— 3 :o3m>3o8 3:: 53:03: tomorrow

 

 

 

...—«ca 3 53.33::
E: 33
Hr vdt
.... : «a-
I I I I 1... V. Fl
53:83. . . .
Boa. . .0 . . . . .
83:83. . . . . . .
emf III ... -- e o.

 

 

 

 

 

 

.w 2:9:

9.1033 aoueuuopod [8!!!

122
Table 33. OxemlLIesthLaffeetirLetrainingmneemmﬂasksatisfaeﬁen.

 

 

Step: Variable(s) R2 (if AR2 Adf B3'
1: Training Strategy .01 2 -.- --

Goal -.07

Seq. .06
2: Strategy Interaction .02 3 .01 1

Goal x Seq. -. 15
3: Individual Differences .17* 7 .16* 4

Cognitive Ab. .23*

Learning Om. .27*

Perform. Om. -. 1 3

Negative Aff. .09
4: Performance .34* 8 .16* 1

Score (T7) .45*
5: Feedback Perceptions .52* 11 .18* 3

Diagnosticity (B3) .21*

Attributional Mag. (B3) -.48*

Descriptive Mag. (B3) .191
6a: Process Variables .75* 14 .23* 3

Self-Efﬁcacy (B3) .25*

Mot. to Learn (B3) .51*

Att. Focus Self Rep. (B3) .08
6b: Process Variablesb .75* 14 .22* 3

Self-Efficacy (B3) .26*

Mot. to Learn (B3) .54*

Att. Focus Time (B3) .07
7a: Process Interactions .76* 15 .00 1

Mot. x Att Self Re . .22
7b: Process Interactions .76* 15 .01 1

Mot x Att. Time .29

 

a The B’s refer to standardized regression coefficients associated with each step of the
hierarchical regression.

bThe two a steps are run together and the two b steps are run together, but a and b steps
are run independently with the different measures of attention.

*p<.05
1p<J0

123
signiﬁcantly inﬂuenced satisfaction. This supports the hypothesis 8b, but also suggests

that motivation to learn is not the only predictor of satisfaction.
I l 1 . g l

The models presented in Figures 1 and 2 suggest a number of mediation effects
that are not explicitly stated as hypotheses. For example, the model suggests that the
effects of individual differences are mediated by the feedback perceptions of diagnosticity
and magnitude. To test this relationship hierarchical regression can be used to determine
if the signiﬁcant individual difference effects on self-efficacy, motivation to learn, and
attention focus disappear when feedback perceptions are controlled. A similar strategy
can be used to test whether the effects of feedback perceptions on training outcomes are
mediated by self-efficacy, motivation to learn, and attention focus. Finally, this strategy
can be used to determine if individual differences have a direct or mediated effect on ﬁnal
training outcomes. The only direct effects that were hypothesized were for ability to ﬁnal
training outcomes. All other direct effects that are found should be interpreted as
preliminary, the result of exploratory analyses. To limit the number of calculations and
case presentation, all of the following mediation analyses are tested with the self-report
attentional focus variable in the learning process block.

II 'I Di- ‘l .‘ --> --I.. . '- -I II --> unit: ' I - Anumber
of signiﬁcant effects were found for individual differences on the learning process. In the
analyses for attentional focus, cognitive ability was found to signiﬁcantly predict both
initial self-report and time-based measures of attention. In all cases, cognitive ability was
positively associated with attentional focus. When this regression, originally reported in

Table 22, is run with cognitive ability as the ﬁnal step, the predictive beta-weight and R—

124
squared values do not change dramatically. Entered after the manipulations, score, and

feedback perceptions, the parameters for cognitive ability are still signiﬁcant, B = .26, p <
.05, AR2 = .06. A similar result is obtained when the regression equation for initial time-
based attention focus is run with the variables reversed. The new cognitive ability
parameter, B = .42, p < .05, AR2 = .15 is still signiﬁcant and very similar to that shown in
Table 24. These analyses suggest that the effects of cognitive ability on attentional focus
are not mediated by feedback perceptions.

Other individual difference effects for attentional focus included learning
orientation predicting the change in self-report attention and negative affectivity
predicting the ﬁrst change in attention measured with time. The reversed equations for
these effects indicate that learning orientation does not predict the change in attentional
focus when feedback perceptions are controlled, B = .09, n.s., AR2 = .01. The effect for
negative affectivity does remain signiﬁcant, B = -.24, p < .05, AR2 = .06. Thus, while
learning orientation does not seem to have a direct effect on attentional focus, negative
affectivity directly affects the change in time-based attentional focus from block 1 to
block 2.

The regression results for self-efﬁcacy suggest that both cognitive ability and
learning orientation had signiﬁcant effects on initial self-efficacy and the change in self-
efficacy, Tables 27 and 28. In all cases, these effects served to increase the self-efﬁcacy
of trainees. Mediation analyses indicate that the effect for learning orientation remains
signiﬁcant, B = .22, p < .05, AR2 = .04 as does the effect for cognitive ability, B = .19, p <

.05, AR2 = .03 for initial self-efficacy judgments. For changes in these judgments,

125
learning orientation no longer predicts, B = -.06, n.s., AR2 = .00, and neither does ability,

B = -.06, n.s., AR2 = .00. This suggests that while cognitive ability and learning
orientation have direct effects on self-efficacy early in training, later changes in self-
efﬁcacy occur through the feedback mechanisms.

Initial levels and the change in motivation to learn were found to be inﬂuenced by
cognitive ability and learning orientation. For initial levels of motivation to learn the
effect for learning orientation continued to be signiﬁcant in reversed order, B = .31, p <
.05, AR2 = .08. For cognitive ability, the effect was signiﬁcantly reduced, B = .12, n.s.,
AR2 = .01. Neither of these variables remained signiﬁcant when the order was reversed in
the change regressions, B = -.00, n.s., AR2 = .00 for learning orientation and B = -.01, n.s.,
AR2 = .00 for cognitive ability. These results indicate that only learning orientation had a
direct inﬂuence on motivation to learn. This effect, however, only occurred for the initial
level of motivation to learn and not for the change in motivation to learn. The effect of
cognitive ability on motivation to learn was mediated by the feedback mechanisms.

W The only
signiﬁcant feedback effect for knowledge test score was for attributional magnitude of
negative feedback. Table 31 notes that descriptive magnitude of negative feedback is
signiﬁcant, but this ﬁnding is actually a suppresser effect that occurs in the presence of
the diagnosticity value. When entered separately, descriptive magnitude does not predict
knowledge test score, B = .08, n.s., AR2 = .00.

Attributional magnitude judgments continue to account for a signiﬁcant amount

of variance in knowledge test score, even when entered into the regression aﬁer

125
learning orientation no longer predicts, B = -.06, n.s., AR2 = .00, and neither does ability,

B = -.06, n.s., AR2 = .00. This suggests that while cognitive ability and learning
orientation have direct effects on self-efficacy early in training, later changes in self-
efficacy occur through the feedback mechanisms.

Initial levels and the change in motivation to learn were found to be inﬂuenced by
cognitive ability and learning orientation. For initial levels of motivation to learn the
effect for learning orientation continued to be signiﬁcant in reversed order, B = .31, p <
.05, AR2 = .08. For cognitive ability, the effect was signiﬁcantly reduced, B = .12, n.s.,
AR2 = .01. Neither of these variables remained signiﬁcant when the order was reversed in
the change regressions, B = -.00, n.s., AR2 = .00 for learning orientation and B = -.01, n.s.,
AR2 = .00 for cognitive ability. These results indicate that only learning orientation had a
direct inﬂuence on motivation to learn. This effect, however, only occurred for the initial
level of motivation to learn and not for the change in motivation to learn. The effect of
cognitive ability on motivation to learn was mediated by the feedback mechanisms.

W The only
signiﬁcant feedback effect for knowledge test score was for attributional magnitude of
negative feedback. Table 31 notes that descriptive magnitude of negative feedback is
signiﬁcant, but this ﬁnding is actually a suppresser effect that occurs in the presence of
the diagnosticity value. When entered separately, descriptive magnitude does not predict
knowledge test score, B = .08, n.s., AR2 = .00.

Attributional magnitude judgments continue to account for a signiﬁcant amount

of variance in knowledge test score, even when entered into the regression after

126
manipulations, score, and learning process variables, B = -.29, p < .05, AR2 = .04. This

suggests there is a strong, direct effect for attributional judgments on knowledge test
score.

The regression for task satisfaction indicated that all three feedback perceptions
were signiﬁcant predictors. The effect for descriptive magnitude was entered separately
to determine whether the marginal effect reported in this equation was the result of the
suppression effect mentioned earlier. Separate analysis revealed that descriptive
magnitude does have a signiﬁcant relationship with task satisfaction, B = -.28, p < .05,
AR2 = .03 controlling for manipulations, individual differences, and training score.
However, none of these effects remain signiﬁcant when entered after self-efﬁcacy,
motivation to learn, and attention focus, B = -. 14, n.s., AR2 = .01 for attributional
magnitude; B = .07, n.s., ARZ = .00 for descriptive magnitude; B = -.03, n.s., AR2 = .00 for
diagnosticity. This suggests that all effects for feedback perceptions on satisfaction are
mediated by the block of self-efﬁcacy, motivation to learn, and attentional focus.

There were no signiﬁcant feedback predictors for ﬁnal performance score, so no
mediation tests are conducted.

II I,-... .i"l : -->. I‘I'Iw ....I: --> auto. Oll‘ The
signiﬁcant individual difference effects for knowledge test include both cognitive ability
and performance orientation. When entered as the ﬁnal step in the regression contained
in Table 32, both cognitive ability, B = .39, p < .05, AR2 = .11, and performance
orientation, B = -.22, p < .05, AR2 = .04, remain signiﬁcant predictors of knowledge test

score. Thus, both individual differences directly inﬂuence this training outcome.

127
For ﬁnal skill score, reported in Table 32, the only signiﬁcant individual

difference predictor is cognitive ability. This effect does not remain signiﬁcant when
entered after manipulations, score, and learning process variables, B = .11, n.s., AR2 = .01.
This suggests that ﬁnal performance scores are not directly affected by cognitive ability.
Instead, score during training and feedback perceptions of that score mediate the effect of
cognitive ability on ﬁnal performance.

For task satisfaction, Table 33 shows that cognitive ability and learning
orientation are both signiﬁcant predictors. Mediation tests indicate that neither of these
ﬁndings remain signiﬁcant when entered after the process variables, B = -.01, n.s., AR2 =
.00 for ability; B = -.02, n.s., AR2 = .00 for learning orientation. Thus, the effects of
ability and learning orientation on task satisfaction is mediated by self-efﬁcacy,
motivation to learn, and attentional focus.
SummarLofResrrlnandQYetalIIeMMQdel

Using LISREL, path analyses were conducted to provide overall assessments of
model ﬁt. First, the hypothesized model depicted in Figures 1 and 2 is tested. The
manipulation effects are not tested because dichotomous variables violate the normality
assumption in structural equation modeling. This same issue creates problems modeling
interaction. More speciﬁcally, interaction terms violate normality assumptions because
they are not normally distributed. While this does not affect the maximum-likelihood
solutions, the chi-square and coefficient tests can be invalid (Bollen, 1989, p. 406).
Because of these problems, and because the interaction and manipulation effects were

largely marginal, the manipulations and the interaction term of motivation and attention

128
are not included. The actual model tested is presented in Figure 9. Following this

assessment, a revised model is tested based on the results of the hypotheses above. For
all of these analyses, no measurement model was tested; all variables are entered as
manifest variables.

For the following tests, speciﬁc variables needed to be selected to represent each
construct in the model. As the theoretical model was designed to predict training
outcomes, variables were selected based on their conceptual appropriateness for this task.
Initial performance was operationalized by the total score obtained in the ﬁrst block of
trials. This provides a measure of performance for the ﬁrst nine minutes of task
exposure. Feedback perceptions were taken from the second block so they follow
performance but are antecedent to the other self-report variables. The other self-reports,
self-efﬁcacy, motivation to learn, and self-efﬁcacy, were all operationalized by the values
obtained near the end of training (Block 3 values). The self-report attentional variable
was employed in these analyses, rather than the time-based measure, to simplify the
analyses. Skill was operationalized by the score obtained in the tenth and ﬁnal trial.
Both task knowledge and task satisfaction, labeled attitudes on the ﬁgure, were
operationalized as they were in the regression analyses. Initial performance and ﬁnal
skill were standardized in order to create variance terms comparable with other variables

in the analyses.

129

18233.3 33:3: :83 @3932 :8 :38me 5:: d PSwE

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

has...

 

 

 

.50 .tm

 

 

.50 :5

 

 

 

 

 

 

a... .32

 

 

 

 

 

¢ .305anme
250m All Monacoom
=35 ~m:o3:03< @0285:
5.34 8 “tom 3335
“E32329 839382 \ I
/ .mp2 .32.
$3333. monommmﬁom .wﬂz .35
880330 $30.5 wﬁﬁaoq

wage:

30:80.35
152335

 

 

 

.mzsmom EmuﬁEmmm mo Egm .3 85me

£5

 

   

 

 

 

msoom
=5 4, §o§§<

Moscoom
33083

 

 

 

 

 

 

 

 

 

 

\ bmoumoqwma
7,

 

V 53A 3

 

 

130

0323qu 5:333;

 

 

 

 

 

I

 

 

 

 

8332 A--- 38$m££

 

 

 

 

 

 

 

880830 $8on 98584 $880th
9:qu 8:339:

1 3 1
First, the hypothesized process model was tested using path analysis. The ﬁtted

covariance matrix was a poor ﬁt of the data, as indicated by a number of the ﬁt indices,
x2(63, N=110) = 233.84, p < .00, RMSEA = .16, AGFI = .63, NNFI = -.10. None of
these values suggest the model even comes close to effectively capturing the covariance
pattern in the data.

Second, based on the results above a number of paths were modiﬁed. For
example, the paths between diagnosticity and attentional focus were dropped because of a
lack of support for this hypothesis. Diagnosticity did, however, predict self-efficacy so
this link was added. The paths between self-report attentional focus and skill and
attentional focus and knowledge were dropped because those effects were not signiﬁcant.
Direct effects for attributional magnitude judgments were added to motivation to learn
and knowledge. No direct effect was added from attributional magnitude to attitudes
because mediation analyses indicated this effect was mediated by the learning process
block. Figure 10 presents the modiﬁed model which also serves as a summary of the
results for the analyses reported above.

The resulting model involves a net loss in 10 degrees of freedom as more
parameters are being estimated rather than constrained to zero. The revised model is a
poor ﬁt of the data, {(53, N=110) = 180.25, p < .00, RMSEA = .15, AGFI = .67, NNFI =
-.O7. However, a chi-square difference test of these nested models suggests that the
restrictions in the revised model provide a signiﬁcant improvement in model ﬁt, Ax2(10,
N=110) = 53.59, p < .00. None-the-less, neither of the models provides an adequate ﬁt of

the data. Further modiﬁcations to the model provide only small, incremental

132
improvements in model ﬁt. The high intercorrelation of many of the variables in the

analysis create difﬁculty in achieving adequate model ﬁt. Satisfactory ﬁt was not
obtained without abandoning theory and removing variables from the model. As a result,

no additional models are presented.

DISCUSSION

The purpose of this research is to explore how different individuals react to and
learn from difficult, early training experiences. Current research on errors in training
focuses mainly on Error Management Training (EMT), which suggests a particular
training strategy for incorporating errors in training. Current applications of EMT have
proven useful, but they are difficult if not impossible to implement during the early stages
of skill acquisition. Ordinarily, EMT suggests the addition of an entire training unit,
provided sometime during the middle of a training sequence, that involves the active
pursuit of errors.

While this area of research has successfully re-opened a long-standing debate in
the learning literature, it has done so without providing theoretical grounding in the
processes that follow the receipt of negative feedback. The goal of the current project is
to explore the psychological and behavioral processes that result from error feedback. As
EMT requires the addition of training units and is not easily applied to early stages of
skill acquisition, training strategies that can be easily applied during the earliest stages of
learning are still sorely lacking. Consequently, the process model was used to suggest
two easily implemented training strategies. These strategies were evaluated for their
usefulness in improving satisfaction and learning with error-ﬁlled early training
experiences.

The proposed process model suggests that two feedback perceptions are involved

in learning from negative feedback — magnitude and diagnosticity. These perceptions are
1 3 3

DISCUSSION

The purpose of this research is to explore how different individuals react to and
learn from difficult, early training experiences. Current research on errors in training
focuses mainly on Error Management Training (EMT), which suggests a particular
training strategy for incorporating errors in training. Current applications of EMT have
proven useful, but they are difficult if not impossible to implement during the early stages
of skill acquisition. Ordinarily, EMT suggests the addition of an entire training unit,
provided sometime during the middle of a training sequence, that involves the active
pursuit of errors.

While this area of research has successfully re-opened a long-standing debate in
the learning literature, it has done so without providing theoretical grounding in the
processes that follow the receipt of negative feedback. The goal of the current project is
to explore the psychological and behavioral processes that result from error feedback. As
EMT requires the addition of training units and is not easily applied to early stages of
skill acquisition, training strategies that can be easily applied during the earliest stages of
learning are still sorely lacking. Consequently, the process model was used to suggest
two easily implemented training strategies. These strategies were evaluated for their
usefulness in improving satisfaction and learning with error-ﬁlled early training
experiences.

The proposed process model suggests that two feedback perceptions are involved

in learning from negative feedback — magnitude and diagnosticity. These perceptions are
l 3 3

134
suggested as determinants of learning processes such as self-efficacy, motivation to learn,

and attentional focus. Training strategies of mastery framing and sequencing were then
suggested as practical means to affect these perceptions and ultimately lower the
dissatisfaction and increase the learning that occurs from these training situations.

To accomplish these objectives, a realistic learning scenario was developed with a
computer simulation task. Performance criteria were somewhat ambiguous, but the task
was designed so that all mean performance scores were negative. Score feedback
presented at the end of each three minutes trial was augmented with text stating the actual
number of errors committed during the preceding scenario. No one achieved perfect
performance, so trainees committed, and were accurately informed of, errors throughout
the training. Despite the difﬁculty of the task, motivation to learn was relatively high and
many trainees learned a great deal about the simulation. Thus, the experimental
procedure provides a challenging training environment with a novel but engaging task.
The discussion below ﬁrst discusses the ﬁndings relevant to the theoretical model. Then,
the evaluation of the training strategies is discussed. The section concludes with a
discussion of study limitations and future directions for research.

mm The results of this study suggest ﬁve speciﬁc limitations to the
proposed theoretical model. First, the distinction between “motivational” and
“information” process effects does not hold for this data. Results indicate that
diagnosticity had only minor effects on trainee’s attentional focus. Instead, diagnosticity
inﬂuenced trainees’ self-efficacy. This suggests that the notion of diagnosticity as the
predominant inﬂuence of an “informational” learning track does not hold. Furthermore,

the motivational variable of descriptive magnitude inﬂuenced attentional focus. A

1 3 5
similar cross-over effect occurred for individual differences. Learning orientation was

expected to inﬂuence magnitude and the motivational learning track only, yet it provided
signiﬁcant prediction of both attributional magnitude and diagnosticity. These results
suggest that “motivational” and “informational” tracks are not distinguishable.

Second, the model suggests that feedback perceptions mediate the effects of
individual differences on leaming processes of self-efﬁcacy, motivation to learn, and
attentional focus. Further, the model suggests that these learning processes mediate the
effects of feedback perceptions on learning outcomes. Results indicate direct inﬂuences
of learning orientation on self-efﬁcacy and motivation to learn, performance orientation
on knowledge training outcomes, and attributional magnitude on knowledge training
outcomes. Thus, the mediating effects posed in the model do'not hold. The learning
process is more complex and inter-related than the hypothesized model suggests.

Third, the original model suggests that self-efﬁcacy and motivation to learn
operate in a simple mediated relationship. To the contrary, results suggest that these
variables are highly related, each occupying similar positions in a nomological network
of the learning process. Both self-efﬁcacy and motivation to learn are inﬂuenced by
learning orientation, diagnosticity, and attributional magnitude perceptions. Both self-
efﬁcacy and motivation to learn inﬂuence attitudinal outcomes of training. And ﬁnally,
neither self-efficacy nor motivation to learn have signiﬁcant main effects on knowledge
and skill outcomes.

None-the-less, there is evidence that these constructs differ. While self-efﬁcacy
was inﬂuenced by performance throughout training, motivation to learn was independent

of performance throughout training. Self-efficacy changed over the course of training,

136
but motivation to learn remained fairly constant, as suggested by the change analyses.

Without additional data or ﬂiture research, it is difficult to identify the true relationship
between these constructs. In any case, it is clear that the hypothesized relationship is too
simple to capture the complex inter-relationship between these constructs.

Fourth, the individual difference variable of negative affectivity did not inﬂuence
magnitude perceptions. In fact, this trait had only one signiﬁcant, unhypothesized effect.
Contrary to expectations, trainees dispositions to experiencing negative affective states
did not alter the process or the outcomes of this difﬁcult training situation. The lack of
this effect is puzzling, given the recent literature asserting the importance of affect for
organizationally-relevant outcomes (e.g., George, 1992).

Fifth, the descriptive and attributional magnitude perceptions have different
relationships than originally hypothesized. While most analyses indicate these constructs
have similar effects, attributional magnitude exhibited a far greater inﬂuence on the
learning process. For attentional focus, these two constructs even have opposing effects!
Magnitude was expected to have negative inﬂuences on the learning process, but
descriptive magnitude actually increases initial levels of on-task attention. Positive
motivational effects were dismissed as a possibility early in this paper, as they were
expected to be minimal during early stages of skill acquisition. Apparently this decision
was inaccurate and should be revisited.

One perspective that may prove useful in understanding the opposing effects of
descriptive and attributional magnitude is that of perceived feedback magnitude
representing the internal assessment of goal--performance discrepancy. Research on goal

setting suggests that trainees often hold internal standards or goals regarding desired

137
performance levels (e.g., Locke & Latham, 1991). From this perspective, the larger the

discrepancy between actual performance and standard, the greater the descriptive
magnitude of negative feedback. This description of discrepancy suggests nothing about
what the individual thinks caused the current gap between actual and desired performance
level. The ascription of cause is assessed by attributional magnitude, which reﬂects
whether individuals feel their goal--performance discrepancy is a result of something
internal and stable. Those individuals who view large goal-~performance discrepancies
and see these discrepancies as caused by stable personal shortcomings should have lower
self-efficacy and motivation for training than individuals who do not view the
discrepancy this way. This explanation provides an accurate portrayal of the obtained
results. The results for individual differences also support this perspective.

The effects for individual differences were more in line with the hypotheses.
Learning and performance orientation were found to predict attributional judgments of
feedback. Thus, individuals high in performance orientation reported higher attributional
magnitude of negative feedback regardless of actual performance level. Attributional
judgments were in turn related to lower task satisfaction and lower knowledge test scores.
The direct effects of learning orientation included lower attributional magnitude
judgments, higher self-efﬁcacy, and higher motivation to learn. Through increases in
self-efﬁcacy and motivation to learn, learning orientation also indirectly improved task
satisfaction. These results support ongoing research on goal orientation effects on
training outcomes (e.g., Boyle & Klimoski, 1995; Ford et al., 1995; Kozlowski et al.,
1995; 1996) and on the attributional inﬂuences of goal orientation (e.g., Dweck, 1986;

1989). Finally, cognitive ability was also discovered to be an important determinant of

l 3 8
attributional magnitude and training outcomes. As a whole, then, individual differences

were highly predictive of important learning processes as well as training outcomes.

mm It is important to restate that the mastery goal did not elicit a
mastery state orientation in trainees. Thus, it is difﬁcult to interpret the obtained results
as effects of mastery-ﬂamed training. The failure of this manipulation is discussed later,
under study limitations, but the effects of the goal ﬂaming manipulation are interpreted
cautiously below.

The supporting evidence for the goal ﬂaming strategy involves mastery ﬂaming
lowering descriptive magnitude judgments. In other words, controlling for actual
performance scores, individuals in this condition viewed their performance as more
positive than trainees in the performance conditions. Results indicated that this effect
decreased over time. That is, differences in magnitude judgments were found for the ﬁrst
and second block, but not for the third. This pattern suggests a diminishing effect for the
manipulation.

It is likely that characteristics of the task and the training overwhelmed the
manipulation to push all trainees toward a similar interpretation of the feedback they
received. The form of the feedback (i.e., number of errors and score) and the nature of
task (i.e., short performance episodes followed immediately by this feedback) both ﬂame
performance as a critical aspect of training success. Thus, trainees in mastery conditions
may have received conﬂicting messages about the purpose and objectives of training.
The effect for mastery training may have been stronger and may not have diminished if
the feedback and structure the training were complementary to the training ﬂame.

Complementary feedback would depict information regarding the practice and

139
learning of critical skills, rather than the score that results ﬂom using these skills. The

training structure could be modiﬁed so that trainees can focus on studying or practicing
simultaneously rather than one or the other. While this solution creates potential
confounds for exploration and amount of time spent in training, these factors can be
controlled either by design or statistically.

The sequencing manipulation also has effects, including a trend for performance
scores and the post-hoc ﬁnding that sequencing lowers descriptive magnitude judgments.
The effect for performance was clear during the ﬁrst two blocks; sequencing improves the
basic skills of labeling and engaging. In the third block, performance suffered ﬂom
attention to more complex aspects of the task. Sequenced trainees were so absorbed
ﬁguring out the prioritization component of the task that their basic task performance
suffered. Beneﬁcial effects for sequencing on prioritization likely did not occur because
of the complexity and difﬁcult of the prioritization action. Few individuals performed
this action well, with cognitive ability serving as the best predictor of effective
prioritization performance. If training time was extended and trainees had more time to
practice the prioritization action, it likely that the sequencing condition would have
performed better than the unsequenced trainees. This ﬁnding would suggest a return to
the shape of the learning curves ﬂom blocks 1 and 2, and would result in greater beneﬁts
of sequencing for trainees.

An unexpected ﬁnding relating to the training strategies was that, counter to the
model, descriptive magnitude of negative feedback was found to relate positively to self-
report attentional focus. Thus, by lowering descriptive magnitude these training

strategies indirectly inhibited at least one of the mechanisms they were designed to

140
enhance. Meanwhile, training strategies had little inﬂuence on attributional magnitude

judgments. This suggests that the current training strategies did little to change the
interpretation of the negative feedback that was received. While mastery ﬂaming and
sequencing did lessen the perception of magnitude, these manipulations were not able to
stop individuals ﬂom attributing errors to internal, stable factors. Whether or not the
manipulation improvements noted above would result in manipulations that can inﬂuence
interpretations remains an interesting question, open for future investigation.

The most clear limitation of the current study is the failure of the mastery ﬂaming
to elicit a mastery state in trainees. This creates conceptual and empirical ambiguity in
interpreting the effects of the goal manipulation. The performance strategy, which
effectively raised performance state, is essentially compared to an unknown. A more
powerful design would compare a performance condition directly to a “do your best”
condition in which no ﬂaming was provided. This would create an interpretable
comparison of the effects of performance ﬂaming. To create a comparison relevant to
mastery ﬂaming, a more powerful ﬂaming manipulation must be employed.

A mastery ﬂaming condition that could be employed in future research would
involve a short exercise which actively depicts the value of learning ﬂom negative
feedback. A behavioral-modeling approach to teaching trainees how to deal with errors
may both elicit mastery states and provide skills for dealing with errors. For example,
trainees can be shown a video-tape of a successful person making a number of critical
errors, but effectively dealing with these errors by stopping, thinking, and reviewing the

error as a learning opportunity.

141
An alternative explanation for the failure of the mastery manipulation rests with

the malleability of mastery states. Without modifying the task or the feedback, it may be
difficult to focus attention away ﬂom performance. When speciﬁc feedback is presented
regarding performance, it may be difficult for trainees to ignore the information contained
therein. Further, focusing attention to performance may be an easy task because, if the
information is readily available, it simply needs to be made salient. The truthfulness of
this possibility should be investigated with future research.

Little research has been conducted on the distinction between goal orientation as a
state and as a trait. In fact, research in this area oﬂen does not distinguish between the
two, as noted by Kozlowski et al. (1995). The question of malleability or susceptibility to
situational inﬂuence is one that should be addressed. This can be accomplished by
varying the intensity of mastery-ﬂaming manipulations. In addition, changes in the form
of feedback (e.g., Field, 1996) and characteristics of the task (e. g., Carroll & Carrithers,
1984) could be explored for their inﬂuence on goal orientation states. In conducting this
research, it is clear that researchers must continue to measure both state (situational) and
trait (dispositional) aspects of goal orientation. Without measuring both forms of goal
orientation, the question of malleability cannot be addressed.

An additional limitation of this study is the artiﬁcial nature of the task and the
utilization of college students as subjects. This is a common criticism of laboratory
research, and one that has been addressed in great detail elsewhere (e.g., Locke, 1986).
None-the-less, it is important to address this issue ﬂom the standpoint of the theory tested
here. A blanket criticism of the current design is that trainees are less motivated to learn

and perform than real world trainees, and, as a result, more likely to give up or

142
withdrawal ﬂom training. The lack of real-world consequences for learning this task may

have contributed to the empirical similarity between self-efﬁcacy and motivation to learn.
Conceptually these constructs are distinct, but they may be more closely tied in short,
laboratory settings where motivation is tied to short term goal of protecting trainees’ self-
image. After all, why should a trainee exert effort learning an unimportant and difficult
task? A major source of motivation in these situations may be the self-perception of
efﬁcacy, which lead trainees to protect this self-perception by working to maintain or
improve performance. Future research is deﬁnitely needed to explore the relationship
between self-efficacy and motivation to learn in a number of realistic settings.

Having recognized this criticism, there are two reasons why I believe this
criticism is not entirely valid. First, trainees in this study were essentially compensated
employees. As an experimenter I provided them with compensation in the form of
experimental credit for trying to learn (or perform) the task. These were the experimental
instructions. Further, there were potential rewards for good performance. This type of
reward mirrors the beneﬁts or bonuses that might be obtained in a real world work
setting. Second, real world employees are not often in a “do or die” situation when it
comes to training. Current methods of training evaluation rarely create situations where
personal responsibility is demanded for transfer and maintenance of skills. Furthermore,
only the “best-in-class” organizations actually track the utilization of learned skills back
on the job (Olian, Durham, Kristof, Brown, & Pierce, 1995). Thus, while it is certainly
possible to envision instances where real world training involve higher levels and greater
persistence of motivation, those cases probably do not represent the majority of training

situations.

143
The measurement approach employed in this study provides another limit to the

generalizations that can be drawn. Many of the conclusions drawn ﬂom this study center
around the inﬂuence of attributional magnitude of negative feedback. This is a new
construct for which no other research has been conducted. Parallels were drawn to the
self-evaluation that occurs as a result of goal--performance discrepancies. However, no
direct measure of attributions or of self-set goals were collected. Furthermore, this
construct collapses attribution and self-evaluation into one measure. Other studies in this
area have directly assessed both self-set goals and attributions (e.g., Thomas & Mathieu,
1995)

Even with these differences in conceptualization and measurement, the current
ﬁndings both agree with and extend the current literature in this area. Past ﬁndings have
suggested that attributions can moderate the inﬂuence of a goal-~performance
discrepancy. The ﬁndings in this study regarding attributional magnitude suggest it is
predicted by learning and performance orientation and it affects a number of important
training processes and outcomes. These effects occur after controlling for actual
performance. In many ways, this measurement approach may more accurately reﬂect the
psychological process that occurs during performance episodes.

Other research in this area tests these effects using goal--performance difference
scores. In addition to the statistical difﬁculties with difference scores, there are
conceptual reasons to be concerned about the difference score value. Self-set goals are
collected prior to performance and assumed to remain constant until feedback is received.
Further, the weight of different self—set goals in determining self-evaluations are assumed

to be the same. That is, everyone is assumed to be equally committed to and equally

143
The measurement approach employed in this study provides another limit to the

generalizations that can be drawn. Many of the conclusions drawn ﬂom this study center
around the inﬂuence of attributional magnitude of negative feedback. This is a new
construct for which no other research has been conducted. Parallels were drawn to the
self-evaluation that occurs as a result of goal--performance discrepancies. However, no
direct measure of attributions or of self-set goals were collected. Furthermore, this
construct collapses attribution and self-evaluation into one measure. Other studies in this
area have directly assessed both self-set goals and attributions (e. g., Thomas & Mathieu,
1995)

Even with these differences in conceptualization and measurement, the current
ﬁndings both agree with and extend the current literature in this area. Past ﬁndings have
suggested that attributions can moderate the inﬂuence of a goal--performance
discrepancy. The ﬁndings in this study regarding attributional magnitude suggest it is
predicted by learning and performance orientation and it affects a number of important
training processes and outcomes. These effects occur after controlling for actual
performance. In many ways, this measurement approach may more accurately reﬂect the
psychological process that occurs during performance episodes.

Other research in this area tests these effects using goal--performance difference
scores. In addition to the statistical difficulties with difference scores, there are
conceptual reasons to be concerned about the difference score value. Self-set goals are
collected prior to performance and assumed to remain constant until feedback is received.
Further, the weight of different self-set goals in determining self-evaluations are assumed

to be the same. That is, everyone is assumed to be equally committed to and equally

144
value the goal. These assumptions have rarely, if ever, been tested. The current method

circumvents these assumptions. If the intent of a goa1--performance discrepancy is to
assess the self-evaluation conducted after feedback is received, the current method of
asking trainees about their feedback and performance seems the most parsimonious and
direct. However, as noted before, because this method is new, research should be
conducted to identify the relationship between perceived magnitude judgments and goal--
performance discrepancies. Furthermore, the relationship between traditional
attributional measures and the attributional measures employed in this study should be
explored.

The individual difference and attributional magnitude results noted here have
theoretical and practical implications for EMT and related error-based training. These
results suggest that certain individuals are prone to react negatively to errors, and, as a
result, may have their learning actually impaired by the addition of “error” training units.
Training strategies may not be effective in eliminating the negative consequences of
errors for individuals who have lower cognitive ability, lower learning orientation, and
higher performance orientation. These “at-risk” individuals can be identiﬁed through
relatively simple testing procedures, and training strategies speciﬁcally for these
individuals should be developed and evaluated.

The particular strategies tested in this study were not strong enough to prevent the
decline in motivation and learning that occurs these “at-risk” individuals. Martocchio
(1994) has investigated additional training strategies that may be useful in lowering
trainees’ tendency to ascribe poor performance to ability. In his study, Martocchio

suggested to trainees that everyone makes a great deal of mistakes and that these are a

144
value the goal. These assumptions have rarely, if ever, been tested. The current method

circumvents these assumptions. If the intent of a goal--performance discrepancy is to
assess the self-evaluation conducted after feedback is received, the current method of
asking trainees about their feedback and performance seems the most parsimonious and
direct. However, as noted before, because this method is new, research should be
conducted to identify the relationship between perceived magnitude judgments and goal--
performance discrepancies. Furthermore, the relationship between traditional
attributional measures and the attributional measures employed in this study should be
explored.

The individual difference and attributional magnitude results noted here have
theoretical and practical implications for EMT and related error-based training. These
results suggest that certain individuals are prone to react negatively to errors, and, as a
result, may have their learning actually impaired by the addition of “error” training units.
Training strategies may not be effective in eliminating the negative consequences of
errors for individuals who have lower cognitive ability, lower learning orientation, and
higher performance orientation. These “at-risk” individuals can be identiﬁed through
relatively simple testing procedures, and training strategies speciﬁcally for these
individuals should be developed and evaluated.

The particular strategies tested in this study were not strong enough to prevent the
decline in motivation and learning that occurs these “at-risk” individuals. Martocchio
(1994) has investigated additional training strategies that may be useful in lowering
trainees’ tendency to ascribe poor performance to ability. In his study, Martocchio

suggested to trainees that everyone makes a great deal of mistakes and that these are a

145
result of task difficulty, not of ability. Similarly, Thomas and Mathieu (1994) suggest

that trainers should push trainees to think of failures as resulting ﬂom external, unstable
factors. The data presented here would suggest that, if these strategies can succeed in
lowering attributional magnitude judgments, then self-efficacy, motivation to learn, task
satisfaction, and ﬁnal task knowledge would all be greater. However, the extent to which
goal orientation and cognitive ability effects on attributional judgments can be
ameliorated by training strategy is an unknown. Future research should directly assess
this issue.

In conclusion, this study suggested a theoretical model of the process of learning
ﬂom errors during early skill acquisition. A number of the paths suggested in the model
were not supported and a number of unexpected paths were found to be signiﬁcant.
While future research is needed to validate these exploratory ﬁndings, it is clear ﬂom
these ﬁndings that the learning process is not easily divisible into a motivational and
informational track. Even so, the model is useful in identifying feedback perception and
interpretation as a critical aspect of the learning process. Furthermore, individual
differences of goal orientation and cognitive ability have both direct effects on training
outcomes and indirect effects via these feedback perceptions. Thus, in considering early
stages of complex skill acquisition, the present study suggests that certain individuals are
more prone to reduce effort and withdraw. For these individuals, alternative methods of

training may be necessary.

145
result of task difficulty, not of ability. Similarly, Thomas and Mathieu (1994) suggest

that trainers should push trainees to think of failures as resulting ﬂom external, unstable
factors. The data presented here would suggest that, if these strategies can succeed in
lowering attributional magnitude judgments, then self-efficacy, motivation to learn, task
satisfaction, and ﬁnal task knowledge would all be greater. However, the extent to which
goal orientation and cognitive ability effects on attributional judgments can be
ameliorated by training strategy is an unknown. Future research should directly assess
this issue.

In conclusion, this study suggested a theoretical model of the process of learning
ﬂom errors during early skill acquisition. A number of the paths suggested in the model
were not supported and a number of unexpected paths were found to be signiﬁcant.
While future research is needed to validate these exploratory ﬁndings, it is clear ﬂom
these ﬁndings that the learning process is not easily divisible into a motivational and
informational track. Even so, the model is useful in identifying feedback perception and
interpretation as a critical aspect of the learning process. Furthermore, individual
differences of goal orientation and cognitive ability have both direct effects on training
outcomes and indirect effects via these feedback perceptions. Thus, in considering early
stages of complex skill acquisition, the present study suggests that certain individuals are
more prone to reduce effort and withdraw. For these individuals, alternative methods of

training may be necessary.

APPENDICES

APPENDIX A

APPENDIX A

Training Strategy Manipulations

S '11'1'

To help you meet the objectives you have been given, we have been investigating
different ways to make RADAR-OP easier. Through a review of research, and some
work done here, we have developed recommendations for how you should spend your
time. Following our recommendations will help achieve your objectives faster and more
easily.

Our suggestion is that you limit your attention to certain parts of the simulation for each
block. Instead of working on all parts of the program, the best way to approach this task
is to concentrate your efforts. So, before each block of trials we will give you a set of
skills to focus on during that block. When you are practicing and studying RADAR-OP
during that block, you should focus on the part of the task that we suggest.

For this ﬁrst block, we suggest that you focus on [LABELING Although there is more
to the simulation than just this one skill, LABELING is integral to all subsequent skills
and actions with the task. So, concentrate your efforts on LABELING]*. That means
that you will have 3 trials in the coming block and on ALL THREE of these trials you
should work on [LABELING]*. I will let you know when we recommend you shift your
attention to other parts of the simulation.

Do you have any questions?

* For blocks 2 and 3 these sentences were replaced to focus trainee attention on engaging
and prioritizing, respectively.

146

APPENDIX A

Training Strategy Manipulations

S 'll'l'

To help you meet the objectives you have been given, we have been investigating
different ways to make RADAR-OP easier. Through a review of research, and some
work done here, we have developed recommendations for how you should spend your
time. Following our recommendations will help achieve your objectives faster and more
easily.

Our suggestion is that you limit your attention to certain parts of the simulation for each
block. Instead of working on all parts of the program, the best way to approach this task
is to concentrate your efforts. So, before each block of trials we will give you a set of
skills to focus on during that block. When you are practicing and studying RADAR-OP
during that block, you should focus on the part of the task that we suggest.

For this ﬁrst block, we suggest that you focus on [LABELING Although there is more
to the simulation than just this one skill, LABELING is integral to all subsequent skills
and actions with the task. So, concentrate your efforts on LABELING]*. That means
that you will have 3 trials in the coming block and on ALL THREE of these trials you
should work on [LABELING]*. I will let you know when we recommend you shift your
attention to other parts of the simulation.

Do you have any questions?

* For blocks 2 and 3 these sentences were replaced to focus trainee attention on engaging
and prioritizing, respectively.

146

147
ll ll . 1'

Your purpose during this study is to learn and understand RADAR-OP. You will go
through 3 blocks of trials to give you an opportunity to learn, practice, and explore the
game. Pursuing knowledge about the game is the key to gaining skill with this
simulation.

Learning RADAR-OP is not an easy task. We recognize that you have not played
RADAR-OP before, that’s why we are giving you a series of trials to play and explore.
But, in order for you to truly understand RADAR-OP, you must work hard to learn its
features. How you perform during any one trial is unimportant, what matters is how
much you learn ﬂom each trial.

Everything you need to know to about this game was covered in the brief introductory
training or is covered in the manual. . . the rest is up to you. How much you learn ﬂom
each trial is completely under your control. RADAR-OP is challenging and complex. To
truly understand how the game works, you must commit yourself to studying and
exploring at all times.

As I demonstrated earlier, RADAR-OP keeps a tab on how you are progressing. After
each trial, RADAR-OP will give you information about the correctness of each of your
actions. You can use this feedback to help you learn about the game. The feedback is
presented in terms of different aspects or facets of the simulation, its up to you to USE
that feedback to learn as much as you can. It’s not the number of actions or errors that
you make that’s important, its how you use that information as feedback to improve your
understanding of the task.

So, remember, your objective in this study is LEARN AS MUCH AS YOU CAN
ABOUT RADAR-OP! How you perform on any one trial is unimportant... it’s what you
LEARN that matters!

Do you have any questions?

148
BE “.1.

Your purpose during this study is to perform at you maximum on RADAR-OP. You will
go through 3 blocks of trials to give you an opportunity to maximize your performance.
Your level of performance, as measured by the overall score, is critical to demonstrating
your skill at this game.

Performing extremely well on this task is difﬁcult. We recognize that you have not
played RADAR-OP before, that’s why we are giving a series of trials to get high scores
on the game. In order for you to be one of the best RADAR-OP players, you have to get
high scores by committing many LABEL and ENGAGE actions and very few errors.

Everything you need to know to perform your maximum on this game was covered in the
brief introductory training or is covered in the manual. RADAR-OP is challenging and
complex. To truly perform at the highest level of play, you must commit yourself to
striving for maximum performance at all times.

As I demonstrated earlier, RADAR-OP keeps a running count of your performance. The
overall score shows how well you have maximized the number of LABEL and ENGAGE
actions and minimized the number of errors. The best performers commit many LABEL
and ENGAGE actions and few errors.

So, remember, your objective in this study is PERFORM AT YOUR MAXIMUM
DURING ALL RADAR-OP TRIALS!

Do you have any questions?

APPENDIX B

APPENDIX B

Training Outcome Questionnaire

Wm

Please use the following scale to respond to the following statements. There are no right
or wrong answers here, please just answer honestly.

 

Strongly Strongly
Disagree Disagree Neutral Agree Agree
<-| ! l l !->
(1) (2) (3) (4) (5)

83. I was generally satisﬁed while performing this task. [TSATl]

84. The simulation was boring to me. [TSAT2-R]

85. I didn't like performing the simulation. [TSAT3-R]

86. I am glad that I participated in this experiment. [TSAT4]

87. I often thought about quitting this task. [WTHDl]

88. If I hear about other projects like this, I would be interested in participating.
[WTHDZ-R]

89. If I knew in advance what this project would entail, I would not have chosen to
participate.[WTHD3]

90. I would recommend this project to my friends. [WHTD4-R]

91. I think my ﬂiends would be interested in applying for this project. [WTHDS-R]

92. I became frustrated with my ability to improve my performance. [NEGTHTI]

93. I thought about how poorly I was doing. [NEGTHT2]

94. I was very dissatisﬁed with my performance on this task. [NEGTHT3]

95. I got mad at myself during the task. [NEGTHT4]

96. Would you be willing to participate in another session for follow-up purposes?

[WTHD6]

l - Yes, I'd like to participate in this experiment again whether or not I receive credit
or money for participating.

2 - Yes, I'd like to participate in this experiment again for credit or money.

3 - Maybe I'll participate again. I'll leave my telephone number and you can contact
me for a later appointment.

4 - Probably not. If you get in contact with me, I probably will say no.

5 - No. I'm deﬁnitely not willing to participate again.

149

150
W

This is a knowledge test about RADAR-OP. For each question there is one answer which
is the best answer, try to get as many of these questions right as possible. Awards for

knowledge will be distributed based on the number of these items you get correct.

97. Altitude of 2506 NM suggests the contact is likely which of the following: [KNl]
a. Civilian
b. Military
c. Air
(1. Surface
e. Peaceful '
98. Maneuvering Pattern of Code_Foxtrot indicates which of the following: [KN2]
a. Civilian
b. Military
c. Air
d. Surface
e. Unable to decide, need more information
99. Before a contact can be labeled, you have to do the following: [KN3]
a. Identify it as peaceful or hostile
b. Hook it using the mouse
c. Warn the contact that it is being labeled
(1. Turn the manual to page 8
e. Clear the contact ﬂom the screen
100. Communication Time of 12 Seconds indicates the contact is: [KN4]
a. Civilian
b. Military
c. Air
d. Surface
e. Unable to decide, need more information
101. Altitude of 2 NM means the contact is deﬁnitely: [KNS]
a. Civilian
b. Military
0. Air
(1. Surface
e. Unable to decide, need more information
102. If a contact has Altitude 0 NM, Maneuvering_Pattem Code_Delta, and Speed 495
NM/HR, which of the following is appropriate: [KN6]
a. Check another one value before labeling
b. Label the contact as Surface and Military
c. Label the contact as Surface and Civilian
d. Engage the contact with an “attack”
e. Check to see whether the contact is peaceful

 

151

103. A contact with Altitude of 0 NM, Maneuvering Pattern of Code_Delta, Speed of
410 NM/HR, Response Given, and Communication Time of 42 Seconds should be
labeled: [KN7]

a. Air_Military

b. Surface__Military

c. Air_Civilian

d. Surface_Civilian

e. Unable to decide, need more information
104. Before a contact can be Engaged, which of the following MUST be completed?
[KNS]

a. Turn the manual to page 10

b. Make sure no other contacts have the same communication time

c. Identify how fast the contact is going

d. Figure whether the Manuevering_Pattem is negative

e. Label the contact as Air/Surface and Civilian/Military

For the next three questions, please pick the letter that represents the correct engage

decision to pick for the described contact:

105. Contact with Threat_Level of 3 and Response Given: [KN 9]

a. Clear

b. Monitor

c. Warn

d. Attack

e. Unable to decide, need more information

106. Contact that is Air and Military with a Threat_Level of 3, No Response, current
Range of 52 NM, Speed of 27 NM/HR, and Missile Lock Clean: [KNIO]

a. Clear
b. Monitor
c. Warn
(1. Attack
e. Unable to decide, need more information

107. Contact with Threat_Level of l and Response Given, Missile is Locked_On, but
Speed is only 3 NM/I-IR: [KNl 1]

a. Clear

b. Monitor

c. Warn

d. Attack

e. Unable to decide, need more information

108. Given the following situation, what is the appropriate action? A contact has been
labeled Air and Military and Threat Level of 3. You should: [KN12]
a. Engage the contact with an “attack”

b. Go back and label the contact as surface and military
0. Engage the contact with a “warning”

d. Check more cue values before engaging

e. Engage the contact with an “attack”

109.

152
In terms of the ENGAGE decision, what does a Clean Missile_Lock tell you?

[KNl3]

110.

111.

112.

113.

a. The contact must be Cleared

b. The contact should not be Attacked

c. The contact is not dangerous to your station

d. You need more information

e. Response for that contact will likely be Given

Where are the two defensive perimeters? [KN14]

a. 12 and 20 NM

b. 12 and 256 NM

c. 10 and 256 NM

(1. 10 and 512 NM

e. 256 and 512 NM

At least once or twice during each scenario, you should: [KN15]

a. Use the zoom function

b. Write down a note about program functioning

c. Relabel a contact

d. Find the contact with the highest threat level and engage it

e. Engage the contact moving the slowest

What is a “marker” contact? [KN 16]

a. A contact that indicates the highest possible speed for vessels

b. A contact whose location is useful for identifying invisible boundaries
c. The contact that appears closest to the center of the screen

(1. The contact that must be engaged ﬁrst or points will be lost

e. The contact that is most likely to penetrate a defensive perimeter
Five contacts appear at Bearing 45 Degrees. Which of the following is the one
contact that has the greatest possibility of intruding (crossing) a defensive
perimeter? [KN 1 7]

a. Speed 80 NM/HR, Course 90 Degrees

b. Speed 80 NM/HR, Course 210 Degrees

c. Speed 120 NM/HR, Course 212 Degrees

(1. Speed 120 NM/HR, Course 52 Degrees

e. Speed 30 NM/HR, Course 90 Degrees

For the next four questions, read the scenario and choose the correct action.

114.

Two contacts appear outside the outer defensive perimeter. Contact #1 is an
Air_Military with Speed of 27 NM, Threat Level of 3, and a Course value that is
directing it toward your station. There is No Response and Missile Lock is
Locked_On. Before engaging this contact, which of the following is most
appropriate? [KN18]

a. Check another cue value before engaging

b. Check the Speed and Course on contact #2, the other contact

c. Engage contact #1 as quickly as possible

(1. Zoom out to ensure no other contacts are outside this perimeter

e. Zoom in to ensure no contacts are inside this perimeter

115.

116.

117.

153

Three contacts appear just outside the inner defensive perimeter, all around Bearing
0 degrees (or 12 o’clock). Two of the contacts are Surface_Civilian with Speed of
68 NM, Threat Levels of 1, Clean Missile_Lock, and Response Given. The other
contact is a Surface_Military with Speed of 10 NM/HR, Threat Level of 3, No
Response, and its Missile Lock is On. All contacts have Course of 169 degrees.
Which of the following is the most appropriate action? [KN19]

a. Engage one of the two Surface_Civilian contacts

b. Engage the Surface_Military contact

c. Zoom out to check out the outer defensive perimeter

d. Zoom in to verify that no other contact has penetrated the inner perimeter

e. Wait and see which direction these contacts move

Three contacts appear outside the outer defensive perimeter. Two of them are
Air_Military with speed of 45 NM, threat levels 3, no response, and Missile_Lock
On. The other is an Air_Civilian with Speed of 300 NM, Threat Level of 1,
Response Given, and Clean Missile_Lock. All of the contacts have Course
headings that will take them toward your station. Which of the following is best
action to take next? [KNZO]

a. Engage the Air_Civilian contact

b. Engage the Air_Military contacts

c. Engage the contact farthest ﬂom the defensive perimeter

d. Engage the contacts that are closest together

e. Collect more information before Engaging any contact

Contact #1 appears at Bearing 270 Degrees and has a Course of 93 Degrees, Speed
of 90 NM, and Range of 257 NM. There are two other contacts (#2 and #3) right
next to this contact, on either side. Both contact #2 and #3 have Course of 180
Degrees and Speed of 200 NM. That is all the information you have collected thus
far. Which of the following would be the best step to take next in order to avoid a
penalty intrusion? [KN2]]

a. Check the Range on the two contacts on either side of Contact #1

b. Immediately get information to Engage one of the ﬂanking contacts (#2 or #3)
c. Immediately get information to Label one of the two ﬂanking contacts (#2 or #3)
d. Immediately get information to Engage Contact #1

e. Immediately get information to Label Contact #1

THANK YOU!

APPENDIX C

APPENDIX C

Feedback and Mediating Constructs Questionnaire

Please respond to the following statements about how you think and feel right now. Use
the scale below and ﬁll in the appropriate numbers on the scantron sheet.

 

 

Strongly Strongly I
Disagree Disagree Neutral Agree Agree
<1 1 l l !>
(1) (2) (3) (4) (5)
l4
44. I plan on learning something. [MSTl] L

45. I wonder who will see my results. [PRFl]

46. I hope I don't make any mistakes. [PRF2]

47. I'd like to get a chance to discuss or investigate my mistakes. [MST2]
48. I wonder how my scores will compare with others. [PRF 3]

49. I hope this isn't something I already know. [MST3]

50. I'm eager to get started trying to ﬁgure this out. [MST4]

51. I want to look competent. [PRF 4]

52. I want to do better than others. [PRFS]

53. Its okay if I make mistakes as long as I learn something. [MSTS]

These next questions ask you to describe your training experience and your thoughts about
it thus far. Use the same scale above to rate these statements.
54. The experimenter told me to focus my attention on certain aspects of the task ﬁrst,
before moving on to others. [SEQl]
55. I plan to focus my attention on different parts of the experiment, depending on the
instructions provided by the experimenter. [SEQ2]

STOP!

Put your pencils down and turn your attention back to the computer
screen.

154

155

Please respond to the following statements based on your most recent exposure to
RADAROP and the feedback that you were provided about your performance. Use the
following scale to answer these questions:

 

Strongly Strongly
Disagree Disagree Neutral Agree Agree
< I l l I l >
(1) (2) (3) (4) (5)

Regarding all of the feedback you received for that last trial:

56. The feedback I received ﬂom RADAROP indicated that I performed poorly.

57. Based on those feedback numbers, I did not do well on that last trial.

58. The feedback tells me whether I’m meeting my objectives.

59. The feedback is useful for determining my progress.

60. I am obviously really bad at this game.

61. Based on that feedback, I will never be very good at this task.

62. With the feedback I have useful information about how I am doing.

63. The feedback will help me improve next time.

64. The feedback I got provides some clues about what I should do next.

65. When I reviewed the feedback, it seemed that I did well.

66. The score and error count are NO help in determining my progress.

67. My score and the number of errors I made indicate that I did not perform well.

68. With that score and error count, I don’t see how I’ll em do well on RADAR-OP.

69. Its clear ﬂom the feedback that I don’t have what it takes to be good at RADAR-OP.

70. With this feedback, I have a better idea what to do for the coming study and practice
sessions.

71. Even with this feedback, I’m not sure what to do next in order to get better.

Please turn to the next page and continue...

155

Please respond to the following statements based on your most recent exposure to
RADAROP and the feedback that you were provided about your performance. Use the
following scale to answer these questions:

 

Strongly Strongly

Disagree Disagree Neutral Agree Agree
<-| : : : :->
(1) (2) (3) (4) (5)

Regarding all of the feedback you received for that last trial:

56. The feedback I received ﬂom RADAROP indicated that I performed poorly.

57. Based on those feedback numbers, I did not do well on that last trial.

58. The feedback tells me whether I’m meeting my objectives.

59. The feedback is useﬁrl for determining my progress.

60. I am obviously really bad at this game.

61. Based on that feedback, I will nexer be very good at this task.

62. With the feedback I have useful information about howl am doing.

63. The feedback will help me improve next time.

64. The feedback I got provides some clues about what I should do next.

65. When I reviewed the feedback, it seemed that I did well.

66. The score and error count are NO help in determining my progress.

67. My score and the number of errors I made indicate that I did not perform well.

68. With that score and error count, I don’t see how I’ll eler do well on RADAR-OP.

69. Its clear ﬂom the feedback that I don’t have what it takes to be good at RADAR-OP.

70. With this feedback, I have a better idea what to do for the coming study and practice
sessions.

71. Even with this feedback, I’m not sure what to do next in order to get better.

Please turn to the next page and continue...

156

 

Strongly Strongly
Disagree Disagree Neutral Agree Agree
<1 : : : :->
(1) (2) (3) (4) (5)

Regarding the feedback on Labeling:

72. The labeling feedback helped me decide what I will focus on in the coming trials.
73. I could tell how well I labeled contacts ﬂom the feedback I got at the end of this trial.
74. I can use the labeling feedback to help me label better next time.

75. The label feedback provides valuable information on how well 1 labeled.

Regarding the feedback on Engaging

76. I could tell how well I engaged contacts ﬂom the feedback I got.

77. The feedback I got about engaging is useful for showing me what to do next.

78. The engage feedback clearly indicates how well I engaged.

79. I have an idea for what to focus on in order to improve my engaging for next time.

Regarding the feedback on Prioritizing

80. The feedback helped tell me how well I prioritized contacts.

81. With this feedback, I know what to do regarding prioritizing to over the next few
trials.

82. I know how I did prioritizing ﬂom that feedback.

83. I can use the prioritization feedback to prioritize better next time.

STOP!

Put your pencils down and return to RADAROP.

156

 

Strongly Strongly
Disagree Disagree Neutral Agree Agree
<-| : : : :->
(1) (2) (3) (4) (5)

Regarding the feedback on Labeling:

72. The labeling feedback helped me decide what I will focus on in the coming trials.

73. I could tell how well I labeled contacts ﬂom the feedback I got at the end of this trial.
74. I can use the labeling feedback to help me label better next time.

75. The label feedback provides valuable information on how well I labeled.

Regarding the feedback on Engaging

76. I could tell how well I engaged contacts ﬂom the feedback I got.

77. The feedback I got about engaging is useful for showing me what to do next.

78. The engage feedback clearly indicates how well I engaged.

79. I have an idea for what to focus on in order to improve my engaging for next time.

Regarding the feedback on Prioritizing

80. The feedback helped tell me how well I prioritized contacts.

81. With this feedback, I know what to do regarding prioritizing to over the next few
trials.

82. I know how I did prioritizing ﬂom that feedback.

83. I can use the prioritization feedback to prioritize better next time.

STOP!

Put your pencils down and return to RADAROP.

 

157

Use this scale to indicate what you were thinking about while studying the manual.

Never Seldom Occasionally Frequently Constantly
<: I l I l_>

(1) (i) (3') (4') (5')

84. I daydreamed while I studied.

85. I concentrated on the material.

86. I lost interest in learning the material for short periods of time.

87. I thought about other things that I have to do today.

88. I let my mind wander while I was studying the manual.

89. I looked for information to help improve (i.e., lower) my error counts.

 

 

Strongly Strongly
Disagree Disagree Neutral Agree Agree
<-| l l I l >
(1) (2) (3) (4) (5)

90. I can meet the challenges of my role in this simulation.

91. I can learn how to handle my role successfully.

92. I do NOT want to learn any more about RADAR-OP.

93. I want to perform well on RADAR-OP.

94. I am willing to commit extra time to this study in order to learn more about

RADAROP.

95. I am conﬁdent in my understanding of how information cues are related to the
decisions I have to make.

96. I am willing to commit extra time in order to improve my RADAR-OP score.

97 . I am conﬁdent that I can grasp the rules of RADAR-OP as long as I have time to
study.

98. I am certain that I can manage the requirements of my position for this task.

99. I don’t care if I ever get high scores on this game.

100. I believe I can acquire the skills needed in this game.

101. I am certain that I am capable of leaming how to do well on RADAR-OP.

102. I believe I will fare well in this task if the workload is increased.

103. I don’t care if I learn another thing about this game.

104. I believe that learning this simulation is something I can handle.

105. I am motivated to learn the RADAR-OP game.

106. I want to get high RADAR-OP scores.

107. I want to learn as much as I can ﬂom the manual and game.

108. I am certain I can cope with task components competing for my time.

109. I am motivated to perform well on this game.

STOP -- Put your pencils down and take a quick break.

157

Use this scale to indicate what you were drinking about while studying the manual.

 

Never Seldom Occasionally Frequently Constantly
<-| l I l l >
(1) (2) (3) (4) (5)

84. I daydreamed while I studied.

85. I concentrated on the material.

86. I lost interest in learning the material for short periods of time.

87. I thought about other things that I have to do today.

88. I let my mind wander while I was studying the manual.

89. I looked for information to help improve (i.e., lower) my error counts.

 

Strongly Strongly
Disagree Disagree Neutral Agree Agree
< l I I I I->
(1) (2) (3) (4) (5)

90. I can meet the challenges of my role in this simulation.

91. I can learn how to handle my role successﬁrlly.

92. I do NOT want to learn any more about RADAR-OP.

93. I want to perform well on RADAR-OP.

94. I am willing to commit extra time to this study in order to learn more about

RADAROP.

95. I am conﬁdent in my understanding of how information cues are related to the
decisions I have to make.

96. I am willing to commit extra time in order to improve my RADAR-OP score.

97. I am conﬁdent that I can grasp the rules of RADAR-OP as long as I have time to
study.

98. I am certain that I can manage the requirements of my position for this task.

99. 1 don’t care if I ever get high scores on this game.

100. I believe I can acquire the skills needed in this game.

101. I am certain that I am capable of learning how to do well on RADAR-OP.

102. I believe I will fare well in this task if the workload is increased.

103. I don’t care if I learn another thing about this game.

104. I believe that learning this simulation is something I can handle.

105. I am motivated to learn the RADAR-OP game.

106. I want to get high RADAR-OP scores.

107. I want to learn as much as I can ﬂom the manual and game.

108. I am certain I can cope with task components competing for my time.

109. I am motivated to perform well on this game.

STOP -- Put your pencils down and take a quick break.

158

Please respond to the following statements based on your most recent exposure to
RADAROP and the feedback that you were provided about your performance. Use the
following scale to answer these questions:

 

Strongly Strongly
Disagree Disagree Neutral Agree Agree
< I l l l l >
(1) (2) (3) (4) (5)

Regarding all of the feedback you received for that last trial:

The feedback I received ﬂom RADAROP indicated that I performed poorly.
Based on those feedback numbers, I did not do well on that last trial.

The feedback tells me whether I’m meeting my objectives.

The feedback is useful for determining my progress.

I am obviously really bad at this game.

Based on that feedback, I will neler be very good at this task.

With the feedback I have useful information about how I am doing.

The feedback will help me improve next time.

The feedback I got provides some clues about what I should do next.

. When I reviewed the feedback, it seemed that I did well.

. The score and error count are NO help in determining my progress.

. My score and the number of errors I made indicate that I did not perform well.

. With that score and error count, I don’t see how I’ll eye: do well on RADAR-OP.
. Its clear ﬂom the feedback that I don’t have what it takes to be good at RADAR-OP.
. With this feedback, I have a better idea what to do for the coming study and

practice sessions.

. Even with this feedback, I’m not sure what to do next in order to get better.

PWHP‘E‘PP’P?‘

I—ib—ih-It—It—db—i
Lh-kUJN—‘O

g—A
0\

Please turn to the next page and continue...

 

158

Please respond to the following statements based on your most recent exposure to
RADAROP and the feedback that you were provided about your performance. Use the
following scale to answer these questions:

 

Strongly Strongly

Disagree Disagree Neutral Agree Agree
<-| : : : :—>
(1) (2) (3) (4) (5)

Regarding all of the feedback you received for that last trial:

The feedback I received ﬂom RADAROP indicated that I performed poorly.

Based on those feedback numbers, I did not do well on that last trial.

The feedback tells me whether I’m meeting my objectives.

The feedback is useful for determining my progress.

I am obviously really bad at this game.

Based on that feedback, 1 will nexer be very good at this task.

With the feedback I have useful information about how I am doing.

The feedback will help me improve next time.

The feedback I got provides some clues about what I should do next.

10. When I reviewed the feedback, it seemed that I did well.

1 1. The score and error count are NO help in determining my progress.

12. My score and the number of errors I made indicate that I did not perform well.

13. With that score and error count, I don’t see how 1’11 216! do well on RADAR-OP.

14. Its clear ﬂom the feedback that I don’t have what it takes to be good at RADAR-OP.

15. With this feedback, I have a better idea what to do for the coming study and
practice sessions.

16. Even with this feedback, I’m not sure what to do next in order to get better.

PWSQP‘PP’N?‘

Please turn to the next page and continue...

159

 

Strongly Strongly
Disagree Disagree Neutral Agree Agree
<-| I l l !>
(1) (2) (3) (4) (5)

Regarding the feedback on Labeling:

17. The labeling feedback helped me decide what I will focus on in the coming trials.
18. I could tell how well I labeled contacts ﬂom the feedback I got at the end of this trial.
19. I can use the labeling feedback to help me label better next time.

20. The label feedback provides valuable information on how well I labeled.

Regarding the feedback on Engaging

21. I could tell how well I engaged contacts ﬂom the feedback I got.

22. The feedback I got about engaging is useful for showing me what to do next.

23. The engage feedback clearly indicates how well I engaged.

24. I have an idea for what to focus on in order to improve my engaging for next time.

Regarding the feedback on Prioritizing

25. The feedback helped tell me how well I prioritized contacts.

26. With this feedback, I know what to do regarding prioritizing to over the next few
trials.

27. I know how I did prioritizing ﬂom that feedback.

28. I can use the prioritization feedback to prioritize better next time.

STOP!

Put your pencils down and return to RADAROP.

159

 

Strongly Strongly
Disagree Disagree Neutral Agree Agree
<1 1 l l !>
(1) (2) (3) (4) (5)

Regarding the feedback on Labeling:

17 . The labeling feedback helped me decide what I will focus on in the coming trials.
18. I could tell how well I labeled contacts ﬂom the feedback I got at the end of this trial.
19. I can use the labeling feedback to help me label better next time.

20. The label feedback provides valuable information on how well I labeled.

Regarding the feedback on Engaging

21. I could tell how well I engaged contacts ﬂom the feedback I got.

22. The feedback I got about engaging is useful for showing me what to do next.

23. The engage feedback clearly indicates how well I engaged.

24. I have an idea for what to focus on in order to improve my engaging for next time.

Regarding the feedback on Prioritizing

25. The feedback helped tell me how well I prioritized contacts.

26. With this feedback, I know what to do regarding prioritizing to over the next few
trials.

27. I know how I did prioritizing ﬂom that feedback.

28. I can use the prioritization feedback to prioritize better next time.

STOP!

Put your pencils down and return to RADAROP.

 

159

 

Strongly Strongly
Disagree Disagree Neutral Agree Agree
<1 : : : :>
(1) (2) (3) (4) (5)

Regarding the feedback on Labeling:

17. The labeling feedback helped me decide what I will focus on in the coming trials.
18. I could tell how well I labeled contacts ﬂom the feedback I got at the end of this trial.
19. I can use the labeling feedback to help me label better next time.

20. The label feedback provides valuable information on how well I labeled.

Regarding the feedback on Engaging

21. I could tell how well I engaged contacts ﬂom the feedback I got.

22. The feedback I got about engaging is useful for showing me what to do next.

23. The engage feedback clearly indicates how well I engaged.

24. I have an idea for what to focus on in order to improve my engaging for next time.

Regarding the feedback on Prioritizing

25. The feedback helped tell me how well I prioritized contacts.

26. With this feedback, I know what to do regarding prioritizing to over the next few
trials.

27. I know how I did prioritizing ﬂom that feedback.

28. I can use the prioritization feedback to prioritize better next time.

STOP!

Put your pencils down and return to RADAROP.

160

Please respond to the following statements based on your most recent exposure to
RADAROP and the feedback that you were provided about your performance. Use the

following scale to answer these questions:

 

Strongly Strongly
Disagree Disagree Neutral Agree Agree
<-| : : : :->
(1) (2) (3) (4) (5)

Regarding all of the feedback you received for that last trial:

29. The feedback I received ﬂom RADAROP indicated that I performed poorly.

30. Based on those feedback numbers, I did not do well on that last trial.

31. The feedback tells me whether I’m meeting my objectives.

32. The feedback is useful for determining my progress.

33. I am obviously really bad at this game.

34. Based on that feedback, I will neuter be very good at this task.

35. With the feedback I have useful information about how I am doing.

36. The feedback will help me improve next time.

37. The feedback I got provides some clues about what I should do next.

38. When I reviewed the feedback, it seemed that I did well.

39. The score and error count are NO help in determining my progress.

40. My score and the number of errors I made indicate that I did not perform well.

41. With that score and error count, I don’t see how I’ll eye; do well on RADAR-OP.

42. Its clear ﬂom the feedback that I don’t have what it takes to be good at RADAR-OP.

43. With this feedback, I have a better idea what to do for the coming study and practice
sessions.

44. Even with this feedback, I’m not sure what to do next in order to get better.

Please turn to the next page and continue...

161

 

Strongly Strongly
Disagree Disagree Neutral Agree Agree
< : : : :->
(1) (2) (3) (4) (5)

Regarding the feedback on Labeling.

45. The labeling feedback helped me decide what I will focus on in the coming trials.
46.1 could tell how well I labeled contacts ﬂom the feedback I got at the end of this trial.
47. I can use the labeling feedback to help me label better next time.

48. The label feedback provides valuable information on how well I labeled.

Regarding the feedback on Engaging

49. I could tell how well I engaged contacts ﬂom the feedback I got.

50. The feedback I got about engaging is useful for showing me what to do next.

51. The engage feedback clearly indicates how well I engaged.

52. I have an idea for what to focus on in order to improve my engaging for next time.

Regarding the feedback on Prioritizing

53. The feedback helped tell me how well I prioritized contacts.

54. With this feedback, I know what to do regarding prioritizing to over the next few
trials.

55. I know how I did prioritizing ﬂom that feedback.

56. I can use the prioritization feedback to prioritize better next time.

STOP!

Put your pencils down and return to RADAROP.

162

Use this scale to indicate what you were thinking while studying the manual.

 

Never Seldom Occasionally Frequently Constantly
< I I I I ]->
(1) (2) (3) (4) (5)

57. I daydreamed while I studied.

58. I concentrated on the material.

59. I lost interest in learning the material for short periods of time.

60. I thought about other things that I have to do today.

61. I let my mind wander while I was studying the manual.

62. I looked for information to help improve (i.e., lower) my error counts.

 

Strongly Strongly
Disagree Disagree Neutral Agree Agree
<-| I I I I->
(1) (2) (3) (4) (5)

63. I can meet the challenges of my role in this simulation.

64. I can learn how to handle my role successfully.

65. I do NOT want to learn any more about RADAR-OP.

66. I want to perform well on RADAR-OP.

67. I am willing to commit extra time to this study in order to learn more about

RADAROP.

68. I am conﬁdent in my understanding of how information cues are related to the
decisions I have to make.

69. I am willing to commit extra time in order to improve my RADAR-OP score.

70. I am conﬁdent that I can grasp the rules of RADAR-OP as long as I have time to
study.

71. I am certain that I can manage the requirements of my position for this task.

72. I don’t care if I ever get high scores on this game.

73. I believe I can acquire the skills needed in this game.

74. I am certain that I am capable of leaming how to do well on RADAR-OP.

75. I believe I will fare well in this task if the workload is increased.

76. I don’t care if I learn another thing about this game.

77 . I believe that learning this simulation is something I can handle.

78. I am motivated to learn the RADAR-OP game.

79. I want to get high RADAR-OP scores.

80. I want to learn as much as I can ﬂom the manual and game.

81. I am certain I can cope with task components competing for my time.

82. I am motivated to perform well on this game.

STOP - Put your pencils down and take a quick break.

APPENDIX D

Please DO NOT write on this survey. Answer all questions on the scantron sheets F"
provided. Please write in and bubble the corresponding numbers for your Personal
Identiﬁcation Number (PID). If you are willing to provide your name so we can contact

APPENDIX D

Individual Differences Questionnaire

Radar Operator Experiment (RADAR-OP) Questionnaire

INSIRILCIIQNS

 

and FIRST INITIAL (you do not have to bubble in those letters). Either way all your

you about possible rewards and/or future studies, go ahead and write in your LAST NAME I

responses today are confidential.

Answer the following questions by ﬁlling in the bubble for the number that represents your
response. There are no right or wrong answers, I just want to know a little about you.
When you reach the end of page that says “STOP -- Please wait for further instructions,”
put your pencil down and stop answering the questions.

Read the INSTRUCTIONS and the SCALE on each page carefully.

Memphis:

1.

What is your sex? [sex]
(1) Male (2) Female

What is your age? [age]
(1) less than 18 yrs (2) 18-19 yrs (3) 20-21 yrs (4) 22-23 yrs (5) greater than 23 yrs

What is your overall grade point average? [gpa]
(1) 0 to .9 (2) 1.0 to 1.9 (3) 2.0 to 2.9 (4) 3.0 to 3.9 (5) 2 4.0

Have you ever played TAG (The Tactical Action Game) or TAS (The Tactical Action
Simulation) before ? [evertag]
(1) Yes (2) No (3) Uncertain

Are you left or right handed? [hand]
(1) Left (2) Right

163

164

6. Do you play with video/computer games? [playvid]
(1) Never (2) Rarely (3) Sometimes (4) Frequently (5) Always
7. How often do you work with a "mouse" on computers? [mouse]
(1) Never (2) Rarely (3) Sometimes (4) Frequently (5) Always
I 1. . l l I! . m

This set of questions asks you to describe what you think about each of the following
statements. Please use the scale shown below to make your ratings.

9°

10.

11.
12.
13.
14.
15.
16.
17.
18.
19.

20.
21.

22.
23.

Strongly Strongly
Disagree Disagree Neutral Agree Agree
< I I I I I >

 

I I | l I '

(1) (2) (3) (4) (5)

I do my best when I'm working on a fairly difﬁcult task. [L1]

I like to be fairly conﬁdent that I can successfully perform a task before I attempt it.
[Pl]

When I have difﬁculty solving a problem, I enjoy trying different approaches to see
which one will work. [LZ]

I try hard to improve on my past performance. [L3]

The things I enjoy the most are the things I do the best. [P2]

The opportunity to do challenging work is important to me. [L4]

I feel smart when I do something without making any mistakes. [P3]

I prefer to do things that I can do well rather than things that I do poorly. [P4]

The opportunity to extend the range of my abilities is important to me. [L5]

The opportunity to learn new things is important to me. [L6]

I feel smart when I can do something better than most other people. [P5]

The opinions others have about how well I can do certain things are important to me.
[P6]

I prefer to work on tasks that force me to learn new things. [L7]

I am happiest at work when I perform tasks on which I know that I won't make errors.
[P7]

1 like to work on tasks that I have done well on in the past. [P8]

When I fail to complete a difﬁcult task, I plan to try harder next time I work on it.

[L8]

165
I l. .1 ID'tﬁ

This scale consists of a number of words that describe different feelings and emotions.
Read each item and then mark the appropriate answer in the space next to the word.
Indicate to what extent you generally feel this way, that is, how you feel on the average.
Use the following scale to record your answer.

very slightly a little moderately quite a extremely

or not at all bit
I

I I I
< I I I I“">

(1) (2) (3) (4) (5)

24. Distressed [NAl]
25. Upset [NA2]

26. Enthusiastic [PAl]
27. Interested [PA2]
28. Hostile [NA3]
29. Determined [PA3]
30. Irritable [NA4]
31. Scared [NA5]

32. Aﬂaid [NA6]

33. Excited [PA4]
34. Inspired [PAS]
35. Alert [PA6]

36. Ashamed [NA7]
37. Active [PA7]

38. Guilty [NA8]
39. Nervous [NA9]
40. Strong [PA8]

41. Proud [PA9]

42. Attentive [PAIO]
43. Jittery [NAIO]

 

STOP!

Put your pencils down and wait for further instructions.

APPENDIX E

APPENDIX E

Hypotheses, Analyses, and Summary of Results

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

# Variables in Analysis Technique(s) Expected Direction of Results Support
1 IV: Mastery Focus (MF: 0,1) RM-ANOVA Lower MNF means in mastery Marginal
DV: Mag. Neg. deck (MNF) (MF) conditions
23 IV: Sequencing (SQ: 0,1) RM-ANOVA Higher DF means in sequenced No
DV: Diagnosticity of deck (DF) (SQ) conditions
2b Diagnosticity of Feedback (DF) RM-ANOVA DF increases over time Marginal
2c Diagnosticity of Feedback (DF) Paired Sample Higher DF for sequenced Yes
T-Test variable at the time it is
sequenced, compared to other
DNF values that have yet to be
sequenced within subject
2d IV: Sequencing (SQ: 0,1) Independent Higher DF for sequenced Partial
DV: Diagnosticity Feedback (DF) Sample T-Test variable at time in the sequenced
condition compared to
unsequenced subjects
3a IVS: Sequencing (SQ: 0,1) RM-ANOVA Higher MNF mean in No
Mastery Focus (MF: 0,1) performance+unsequenced
Seq. x Mastery (SQ‘MF) (SQ*MF) cell, other cells
DV: Mag. Neg. deck (MNF) equivalent
3b IVS: Sequencing (SQ: 0,1) RM-ANOVA Lower DNF mean in No
Mastery Focus (MF: 0,!) performance+unsequenced cell
Seq. x Mastery (SQ‘MF) (SQ'MF), other cells equivalent
DV: Diagnosticity Feedback (DF)
43 IV: Cognitive Ability (COG) Regression Positive correlation/beta-weight No
DV: Diagnosticity Feedback (DF) between COG and DF
4b IV: Cognitive Ability (COG) Regression Positive correlation/beta-weight Yes
DVS: Know/Skill Outcomes between COG and KO, SO
4c IV: Negative Affectivity (NEG) Regression Positive correlation/beta-weight No
DV: Mag. Neg. deck (MNF) between NEG and DNF
4d IV: Trait Goal Orientation (TGO) Regression Negative correlation/beta- Partial
DV: Mag. Neg. F dbck (MNF) weight between TGO and MNF
5 IV: Diagnosticity Feedback (DF) Regression Positive correlation/beta-weight No
DV: Attentional Focus (ATF) between DF and ATF
6 IV: Magnitude Neg. deck (MNF) Regression Negative correlation/beta- Yes
DV: Self-Efficacy (SEF) weight between MNF and SEF
7 IV: Self-Efﬁcacy (SEF) Regression Positive correlation/beta-weight Not
DV: Motivation to Learn (MOT) between SEF and MOT Tested
8a IVS: Motivation to Learn (MOT) Hierarchical Positive beta-weight on the No
Attentional Focus (ATF) Regressions MOT‘ATF interaction term
DVS: Knowledge Outcomes (KO) (each DV tested predicting K0 and SO
Skill Outcomes (SO) separately)
8b IV: Motivation to Learn (MOT) Regression Positive correlation/beta-weight Yes
DV: Affective Outcomes (AO) between MTL and ALO

 

166

 

LIST OF REFERENCES

LIST OF REFERENCES

LIST OF REFERENCES

Ackerman, P. L. (1988). Determinants of individual differences during skill

acquisition: Cognitive abilities and information processing. WWW
Bsxcholugxzﬁeneralﬂll, 288- 318

Anderson, J. R. (1983). Ihearehiteennenfcngnition. Cambridge, MA: Harvard

University Press.

Anderson, S. & Rodin, J. (1989). Is bad news always bad? Cue and feedback
effects on intrinsic motivation. WWW, 449-467.

Andrews, J. D. W. (1984). Discovery and expository learning compared. Their

effects on independent and dependent students. lenmalnfEdneaﬁonalResemehlB, 80-
89.

Azuma, H. & Cronbach, L. J. (1966). Cue-response correlations in the attainment
of a scalar concept. AmeﬁcanloumaLofEsycholoalQ. 3849.

Bailey, R. W. (1983). Hnmanenmjnenmpntensystems. New Jersey: Prentice-
Hall.

Balzer, W. K, Doherty, M. E., & O’Connor, R., Jr. (1989). Effects of cognrtrve
feedback on performance. WW, 410-433.

Baker (1991). Still need citation.

Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral
change. W 191-215

Bandura, A. (1986). WWW
meaty. Englewood Cliffs, NJ: Prentice-Hall.

Bandura, A. (1991). Social cognitive theory of self-regulation. Qrgamzatinnal
BehaxioLandﬂumanDeru'sioanssesjﬂ, 248-287

Bandura, A. & Wood, R. (1989). Effects of perceived controllability and
performance standards on self-regulation on complex decision making. lenmalof

PersonaliﬁLaniSmialPsxeholongﬁ 805-814

Biehler, F. (1978). Baehnlegxnppliedtgjeaehing (3rd ed.). Boston: Houghton
Mifflin.
167

168

Bollen. K. A. (1939)- Smremraleguaﬁonmthlatemariahles. New York:
Wiley.

Bouffard, T., Boisvert, J, Vezeau, C., & Larouche, C. (1995). The impact of goal
orientation on self-regulation and performance among college students. BunshlonmaLQf

W317- 329

Boyle, K. A. & Klimoski, R. J. (1995). Toward an understanding of goal
orientation in the training context. In R. J. Klimoski (Chair) symposium presented at the

Tenth Annual Conference of the Society for Industrial and Organizational Psychology,
Orlando, FL.

Bridges, W. (1994). JQhShiﬁ New York: Addison-Wesley.

Button, S. B., Mathieu, J. E., & Zajac, D. M. (in press). The development and
psychometric evaluation of measures of learning goal and performance goal orientation.

0"1E1'IIID"E .

Campbell, J. P. (1988). Training design for performance improvement. In
J. P. Campbell & R. J. Campbell (Eds) BMW (pp. 177-216). San

Francisco, CA: Jossey-Bass.

Campbell, J. P. (1989). An agenda for theory and research. In I. L. Goldstein (Ed)
Imimngandrlexelupmentrnmgamzanms (pp. 469-486). San Francisco, CA. Jossey-

Bass.

Carnevale, A. P., Gainer, L. J., & Meltzer, A. S. (1988). W
skiﬂsjmplgyemant. Alexandria, VA: American Society of Training and Development.

Carroll, J. M. & Carrithers, C. (1984). Blocking learner error states in a training-
wheels system. HumanEaetnmlﬁ, 377-389.

Carroll, J. M. & Mack, R. L. (1984). Actively learning to use a word processor:
By doing, by thinking, and by knowing. In J. C. Thomas & M. L. Schneider (Eds. ),

Humanfaerorsjnmmnutensxstems (pp. 13- 51). Norwood, NJ: Ablex.

Carsten, J. M. (1987). Cognitive processes in self-appraisal. Dissertation
submitted to University of South Florida.

Chen, P. Y., & Spector, P. E. (1991). Negative affectivity as the underlying

cause of correlations between stressors and strains. lmnnalnﬁApplrestxeholongé,
398-407.

169

Cohen, J- (1977)- StaustrcameeLanalxsrsfoLthehehamraLserences. New
York: NY: Academic Press.

Diener, C. I. & Dweck, C. S. (1978). An analysis of learned helplessness:
Continuous changes 1n performance, strategy, and achievement cognitions following

failure. IoumaLofBersonahﬁLandﬁoﬂalPsxcholw, 451 -.562

Diener, C. I. & Dweck, C. S. (1980). Ananalysis of learned helplessness II: The
processing of success. IoumalanersQnaliﬁLandSQciaLBsyeholongﬂ, 940- 952

Dweck, C. S. (1986). Motivational processes affecting learning. Ameriean
BsxehologisLAI. 1040-1048.

Dwyer, D. J., Hall, J. K., Volpe, C., & Cannon-Bowers, J. A. (1992). A
I IIIII I ‘-.. ‘ II I I IIIHII‘A I' IIII.I° II I.‘ (Special
Report 92- 002). Orlando, FL: Naval Training System Center.

Earley, P. C. ( 1 98 8) Computer-generated performance feedback 1n the magazine

subscription industry. QrganizaﬁonalﬁehaxioLandHumanDeeisioanssesAL 50-
64.

Ehri, L. C., Gibbs, A. L. ,& Underwood, T. L. (1988). Inﬂuence of errors on

learning the spelling of English words. W136-
253.

Elliot, A. J. & Harackiewicz, J. M. (1994). Goal setting, achievement orientation,

and intrinsic motivation. A mediational analysis. WWW
Emholchm 968- 980

Elliott, E. S., & Dweck, C. S. (1988). Goals. An approach to motivation and
achievement. WWW 5- 12

Ericsson, K. A, Krampe, R. Tesch-Romer, C. (1993). The role of deliberate
practice in the acquisition of expert performance. W, 363 -406

Farr, J. L., Hoﬂnann, D. A., & Ringenbach, K. L. (1993). Goal orientation and
action control theory: Implications for industrial and organizational psychology. In
C. L. Cooper & I. T. Robertson (Eds), IntematinnalRexiemﬂndnstrialnnd
Qrganizatmnalﬁsyehnlngy (pp. 193-232). New York. NY: John Wiley & Sons.

Fisher, S. L. (1995). The role of cognitive effort and learning outcomes demands
in skill acquisition and learning. Thesis submitted to Michigan State University.

Ford, J. K., & Kraiger, K. (1995). The application of cognitive constructs and
principles to the instructional systems model of training: Implications for needs

170

assessment, design, and transfer. In C. L. Cooper & I. T. Robertson (Eds) Intgman'onal

Rememﬂndmmdandﬂrgamzatmnalimhomwp. 1-48). New York, NY: John
Wiley & Sons.

Ford, J. K., MacCallum, R. C., & Tait, M. (1986). The application of exploratory
factor analysis in applied psychology: A critical review and analysis. Bersgmnel

W 291-314.

Ford, J. K., Smith, E. M. ,Weissbein, D. A., &Gully,S. M. (1995). 1119111315
I 'II.III.I. II II .I. I'I II IIIII. I II_"' II. III III. IIﬁII'

enyimnmem (Contract No. DAAL03 --91-C 0034, TCN Number 93- 339). Orlando, FL.
Naval Training Systems Center.

Frese, M. & Altmann, A. (1989). The treatment of errors in learning and training.

In L. Bainbridge & S. A. R. Quintanilla (Eds. ), WWW
technology (pp. 65- 86). New York: John Wiley and Sons.

F rese, M., Albrecht, K., Altmann, A., Lang, J ., Papstein, P. V., Peyerl, R.,
Prumper, J., Schulte-Gocking, H., Wankmuller, I. ., & Wendel, R. (1988). The effects of
an active development of the mental model 1n the training process: Experimental results

from a word processing system. Behammandlnfonnaﬁonleghnglml, 295- 304.

Frese, M., Brodbeck, F ., Heinbokel, T., Mooser, C., Schleiffenbaum, E., &
Theihnann, P. (1991). Errors in training computer skills: On the positive function of

errors. Human-.Comrmterlnteraetionﬁ 77-93-

Fuller, R. (1990). Learning to make errors: Evidence from a driving task.
ErgonomicLll. 1241-1250.

Gagne, R. M. (1985). Wmﬂgaming (4th edition). New York: Holt,
Rinehart, and Winston.

Gagne, R. M., Briggs, L. J., & Wager, W. W. (1992). W
design (4th edition). Philadelphia, PA: Harcourt Brace Jovanovich College Publishers.

Gick, M. L. & McGarry, S. J. (1992). Learning from mistakes: Inducing
analogous solution failures to a source problem produces later successes in analogical
transfer. 0.101.. .I'III‘I.-.. ' II In ‘eIII' U‘HI‘ III IuIII .,

623-639.

George, J. M. (1992). The role of personality in organizational life: Issues and
evidence. WM 185-213.

171

Gilliland, S. W. (1993). The perceived fairness of selection systems: An
organizational justic perspective. Dissertation submitted to the Michigan State
University.

Gist, M. E. (1987). Self-efﬁcacy: Implications for organizational behavior and
human resource management. AcademLQLManagementReximiz, 472-485.

Gist, M. E. (1989). The inﬂuence of training method on self-efﬁcacy and idea
generation among managers. WW, 787-805.

Gist, M. E., & Mitchell, T. R. (1992). Self-Efﬁcacy: A theoretical analysis of its
determinants and malleability. AcademufManagemenLReximJl, 183-211.

Glaser, R. (1990). The reemergence of learning theory within instructional
research Amenmksxchelegtstdi. 29- 39

Goldstein, I. L. (1992). Training 1n work organizations. In M. D. Dunnette & L.

M. Hough (Eds) Handbooknfindustnaljndmgmizationalpsxcholm(vol. 2, pp. 507-
620). Palo Alto, CA: Consulting Psychologists Press.

GOIdStein’ 1- L- (1993} WWW
andsxaluation. (3rd ed.). Monterey, CA: Brooks/Cole.

Goldstein, I. L. & Gilliam, P. (1990). Training systems in the year 2000.
AmenemﬁsxchelegisLAi 134-143

Greenockle, K. M. & Lee, A. (1991). Comparison of guided and discovery
learning strategies. W, 1127-1130.

Hays, W. L. (1988). Statistics (4th Ed.). Philadelphia, PA: Harcourt Brace
J ovanovich College Publishers.

Hermann, G. (1969). Learning by discovery: A critical review of studies.
loumamexpenmentaLEdueatmnhﬂ 58- 72

Howard, A. (1995). A framework of work change. In A. Howard (Ed.) Ihe
WI]: (pp.3-44). San Francico, CA: Jossey-Bass.

Howell, W. C. & Cooke, N. J. (1989). Training the information processor: A
review of cognitive models. In I. L. Goldstein (Ed.) Irainingmrganizations (pp. 121-

182). San Francisco: Jossey-Bass.

Ilgen, D. R., Fisher, C. D., & Taylor, M. S. (1979). Consequences of individual
feedback on behavior 1n organizations. W, 349- 371.

172

Ivancic, K. & Hesketh, B. (1995). Making the best of errors during training.
IminingResearclLloumaLl,103-125

Jacoby, J, Mazursky, D., Troutrnan, T., & Kuss, A. (1984). When feedback 18

ignored: Disutility of outcome feedback. .LomnaloﬁApohostychgﬂogLﬁQ, 531-.545
Kahn, S. B. (1969). Affective correlates of academic achievement. loomaLof

WM 216- 221

Kamouri, A. L, Kamouri, J. & Smith, K. H. (1986). Training by exploration:
Facilitating the transfer of procedural knowledge through analogical reasoning.

IntemationaLloumaLofManzMachineﬁmdiesJA, 171-192

Kanfer, R. (1996). Self-regulatory and other non-ability determinants of skill

acquisition. In P. M. Gollwitzer & J. A. Bargh (Eds.), Ihopsychologyofaotion;
Linkingsognitionandmotiyationjobehayior (pp. 404-423). New York, NY: Guilford

Press.

Kanfer, R. (1991). Motivation theory 1n industrial and organizational psychology.

In M. D. Dunnette & L M Hough (Eds ) HandbookoﬁndusuiaLandmanizational
psychology (pp. 75-170). Palo Alto, CA: Consulting Psychologists Press.

Kanfer, R., & Ackerman, P. L. (1989). Motivation and cognitive abilities: An
integrative/aptitude-treatment interaction approach to skill acquisition. loumaloﬂAppliod

W 657- 689

Kirk, R. E. (1982). WWW
Paciﬁc Grove, CA: Brooks/Cole Publishing.

Kluger, A. N. & DeNisi, A. (in press). The effects of feedback interventions on
performance: A historical review, a meta-analysis, and a preliminary feedback

intervention theory EthlogicalBulletin

Kluger, A. N. ,,Lewinsohn S., & Aiello, J. R. (1994). The inﬂuence of feedback
on mood: Linear effects on pleasantness and curvilinear effects on arousal.

QmanizationaLBehaﬂonarﬂﬂmnanDecisioancessesﬁﬂ, 276- 299

Kozlowski, S. W. J ., Gully, S. M., Smith, E. M., Brown, K. G., Mullins, M., &
Williams, A. (1996). xxx. In K. Smith-Jentch (Chair) and Paul Thayer (Disscussant)
symposium entitled “How, Why, and When Practice Makes Perfect,” presented at the
Eleventh Annual Conference of the Society for Industrial and Organizational Psychology,
San Diego, CA.

Kozlowski, S. W. J., Gully, S. M., Smith, E. M., Nason, E. R., & Brown, K. G.
(1995). Sequenced mastery training and advanced organizers: Effects on learning, self-
eﬁicacy, performance, and generalization. In R. Klimoski (Chair) and xx (Discussant)

173

symposium entitled “Feeling and Doing in Training...,” presented at the Tenth Annual
Conference of the Society for Industrial and Organizational Psychology, Orlando, FL.

Kraiger, K., Ford, J. K., & Salas, E. (1993). Application of cognitive, skill-based,
and affective theories of learning outcomes to new methods of training evaluation.

loumalngppliedehologyﬂ. 311-328.
Lewis, B. N. (1981). An essay on error. InsmictionaLScioncoJQ, 237-257.

Latham, G. P. & Locke, E. A. (1991). Self-regulation through goal setting.
Qrgan1zat1onalBehaymLandHumanJ2ecis1onProcesses330. 212- 247

Locke, E. A. (Rd, 1986). WWW
Lexington, MA: Lexington.

Long, V. O. & Warner, D. A. (1979). Programmed learning vs. discovery

learning. A comparison of learner attitudes. loumaloﬂnsnuctionaLBsychologLﬁ, 26-
28.

Marjoribanks, K. (1976). School attitudes, cognitive ability, and academic
achievement. lonmalofEducationaLEsychologLﬂ 653 -660

Martocchio, J. J. (1994). Effects of conceptions of ability on anxiety, self-
efﬁcacy, and learning 1n training. JoumaloonpliostychologyJQ, 819- 825.

Martocchio, J. J. (1992). Microcomputer usage as opportunity: The inﬂuence of
context in employee training. W, 529—552.

Martocchio, J. J ., & Dulebohn, J. (1994). Performance feedback effects in
training: The role of perceived controllability. Personnoiﬁsychologyﬂ, 357-373.

Martocchio, J. J., & Webster, J. (1992). Effects of feedback and cognitive
playfulness on performance in microcomputer software training. PozsonncLEsychology.
45, 553-578.

Mathieu, J. E., Tannenbaum, S. I., & Salas, E. (1992). Inﬂuences of individual
and situational characteristics on measures of training effectiveness. Aoadomuf

Managementionrnalji, 828-847.

Mathieu, J. E., Martineau, J. W., & Tannenbaum, S. I. (1993). Individual and
situational inﬂuences on the development of self-efﬁcacy: Implications for training

effectiveness. W, 125-147.

174

Mathieu, J. E., & Martineau, J. W. (in press). Individual and situational inﬂuences

on training motivation. In J. K. Ford (Ed. ), Impmymgjmmngoﬁeonycnossm
organizations, Hillsdale, NJ. Lawrence Erlbaum

Matsui, T., Okada, A. & Inoshita, O. (1983). Mechanisms of feedback affecting
task performance. 0 ... .. . . . : 114-122.

 

McCall, M., Lombardo, M., & Morrison, A. (1988). Iholossonsoicxpozience.

Lexington Books.

Mikulincer, M- (1994). LemedhelplessnesLAmpmgperspecuye Plenum

Press.

Mikulincer, M. (1989). Cognitive interference and learned helplessness. The
effects of off-task cognitions on performance following unsolvable problems. .LoumaLof

RemnalinLandSociaLEsychologLﬂ 129-135

Mikulincer, M. & Nizan, B. (1988). Causal attribution, cognitive interference,

and the generalization of learned helplessness. JoumalofBersonaliﬂLandﬁocial
Bantams: 470478

Mitchell, T. R. ,H,opper H. ,Daniels, D. George-Falvy, J. & James, L. R. (1994).
Predicting self-efﬁcacy and performance during skill acquisition. loumaLoprpliod

W 506- 517

Mone, M. A. & Baker, D. D. (1992). Cognitive, affective, and behavioral
determinants of self-set goals: An integrative, dynamic model. WW
213-234.

Noe, R. A. (1986). Trainee's attributes and attitudes: Neglected inﬂuences on
training effectiveness. Wall, 736-749.

Noe, R. A. & Ford, J. K. (1992). Emerging lssues and new directions for training

research. In Rowland & Ferris (Eds ) ResearchjnﬁersonneLandHnmanResmnces
ManagemenMVol. 10, pp. 345- 384). Nowhere, NJ: JAI Press.

Noe, R. A. & Schmitt, N. (1986). The inﬂuence of trainee attitudes on training
effectiveness: Test of a model. RezsomoLBsyoholongQ, 497-523.

Nordstrom, C. R., Wendland, D., & Williams, K. B. (1995). To err is human: An
examination of error management training. Paper presented at the Tenth Annual
Conference of the Society for Industrial and Organizational Psychology, Orlando, FL.

Ohlsson, S. (1996). Learning from performance errors. EsymologicaLReyiow.
1_Q3_, 241-262.

174

Mathieu, J. E., & Martineau, J. W. (in press). Individual and situational inﬂuences

on training motivation. In J. K. Ford (Ed. ), WW
organizations. Hillsdale, NJ: Lawrence Erlbaum

Matsui, T., Okada, A., & Inoshita, O. (1983). Mechanisms of feedback affecting
task performance. to... .- . . . - 114-122.

 

McCall, M., Lombardo, M., & Morrison, A. (1988). MW.

Lexington Books.

Mikulincer, M. (1994). MW. Plenum

Press.

Mikulincer, M. (1989). Cognitive interference and learned helplessness. The
effects of off-task cognitions on performance following unsolvable problems. loomaLof

Remonalmdﬁoeralksychomﬂ, 129-135

Mikulincer, M. & Nizan, B. (1988). Causal attribution, cognitive interference,

and the generalization of learned helplessness. loomalofBﬂsonalmLandSocral
W 470-478

Mitchell, T. R, Hopper, H., Daniels, D., George-Falvy, J., & James, L. R. (1994)
Predicting self-efﬁcacy and performance during skill acquisition. loumalgLAopliod

W 506- 517

Mone, M. A. & Baker, D. D. (1992). Cognitive, affective, and behavioral
determinants of self-set goals: An integrative, dynamic model. Homanﬁcrfonnancoj,
213-234.

Noe, R. A. (1986). Trainee's attributes and attitudes: Neglected inﬂuences on
training effectiveness. W, 736-749.

Noe, R. A. & Ford, J. K. (1992). Emerging rssues and new directions for training

research. In Rowland & Ferris (Eds), ResearchjnﬁersonneLandHumanResomces
ManagemontO/ol. 10, pp. 345- 384). Nowhere, NJ. JAI Press.

Noe, R. A. & Schmitt, N. (1986). The inﬂuence of trainee attitudes on training
effectiveness: Test of a model. BensonnoLEsychologyJQ, 497-523.

Nordstrom, C. R., Wendland, D., & Williams, K. B. (1995). To err is human: An
examination of error management training. Paper presented at the Tenth Annual
Conference of the Society for Industrial and Organizational Psychology, Orlando, FL.

Ohlsson, S. (1996). Learning from performance errors. Psychologicalﬁeyim
103, 241 -262.

175

Ohlsson, S. (1986). Some principles of intelligent tutoring. InstmctionaLSoienoe.
14, 293-326.

Olian, J., Durham, C., Kristof, A. L., Brown, K. G., & Pierce, R. (1995).
Managerial training and development: Lessons from the best. Manuscript under review.

Peters, L. H., & O'Connor, E. J. (1980). Situational constraints and work
outcomes: The inﬂuences of a frequently overlooked construct. Academyof

WM. 391-397-

Podsakoff, P. M. & F arh J. (1989). Effects of feedback sign and credibility on

goal setting and task performance. Qrgan'uatiQnaLBehayioLandHumanDecision
Processesﬂ, 45- 67.

Prather, D. C. (1971). Trial-and-error versus errorless learning: Training,

transfer, and stress. AmodomloumaigfﬂsycholongA, 377-386.

Pritchard, R. D., Jones, S. D., Roth, P. L., Stuebing, K. K. & Ekeberg, S. E.
(1988). Effects of group feedback, goal setting, and incentives on organizational

productivity lonmalanppliedPsyclmlogLMQnographLﬂ, 337- 358

Prumper, J., Zapf, D., Brodbeck, F. C., & Frese, M. (1992). Some surprising
differences between novice and expert errors in computerized office work. Bchayiour

andlnfonnatimrlechnologyzu, 319- 328.

Quiﬁones, M. A. (1995). Pretraining context effects: Training assignment as

feedback. lQnmalﬂﬂAnleEthngyﬂ 226-233

Rasmussen, J. (1990). The role of errors in organizing behaviour. Ergonomics.
33, 1185-1199.

Reason, J. (1990). Hummer-Ion New York, NY: Cambridge University Press.
Ree, M. J., Carrettta, T. R., & Teachout, M. S. (1995). Role of ability and prior

job knowledge 1n complex training performance. WWWM
721-.730

Ree, M. J. & Earles, J. A. (1991). Predicting training success: Not much more
thang. RersonnelPsychologyAA, 321-332.

Ree, M. J. & Earles, J. A. (1992). Intelligence rs the best predictor of Job
performance. WM. 86- 89

175

Ohlsson, S. (1986). Some principles of intelligent tutoring. InstmctionaLSoicnoc,
M, 293-326.

Olian, J ., Durham, C., Kristof, A. L., Brown, K. G., & Pierce, R. (1995).
Managerial training and development: Lessons from the best. Manuscript under review.

Peters, L. H., & O'Connor, E. J. (1980). Situational constraints and work
outcomes: The inﬂuences of a frequently overlooked construct. Academf

W35. 391-397.

Podsakoff, P. M. & Farh, J. (1989). Effects of feedback sign and credibility on

goal setting and task performance Organizationalﬂehayionandﬂumanmgsion
EmcessesAA, 45- 67.

Prather, D. C. (1971). Trial-and-error versus errorless learning: Training,

transfer, and stress. WW, 377-386.

Pritchard, R. D. ,,Jones S. D. ,,Roth P. L. ,Stuebing, K. K., & Ekeberg, S. E.
(1988). Effects of group feedback, goal setting, and incentives on organizational

productivity. IonmalanppliestychologyMonographlJ3 337- 358

Prumper, J., Zapf, D., Brodbeck, F. C., & Frese, M. (1992). Some surprising
differences between novice and expert errors in computerized ofﬁce work. Behayiom

andlnformatmnlmhnolmﬂ 319-328

Quiﬁones, M. A. (1995). Pretraining context effects: Training assignment as

feedback. loumalanpplredEmhologyﬂ, 226-238.

Rasmussen, J. (1990). The role of errors in organizing behaviour. Ergonomics.
33,1185-1199.

Reason, J. (1990). Homanjnor. New York, NY: Cambridge University Press.
Ree, M. J. Carrettta, T. R., & Teachout, M. S. (1995). Role of ability and prior

job knowledge 1n complex training performance. .LoumaloonohedPsychologyﬂ,
721 -.730

Ree, M. J. & Earles, J. A. (1991). Predicting training success: Not much more
than g. MW. 321-332-

Ree, M. J. & Earles, J. A. (1992). Intelligence rs the best predictor of Job
performance QmentDirectionsinPsychologrmSﬂencel, 86- 89

176

Reigeluth, C. M. & Curtis, R. V. (1987). Learning situations and instructional

models. InR M Gagne(Ed.,) Instmctionallechnologyﬂdationsmp 175- 206)
Hillsdale, NJ: Lawrence Erlbaum.

Reigeluth, C. M. & Stein, F. S. (1983). The elaboration theory of mstructron In
C. M. Reiguluth (Ed. ), I II. .. - -
cmentstatus. Hillsdale, NJ: Lawrence Erlbaum.

 

Rouse, W. B. & Morris, N. M. (1986). On looking into the black box: Prospects
and limits 1n the search for mental models. MW. 349- 363.

Sarason, I. G., Sarason, B. R. Keefe, D. B, Hayes, B. E., & Shearin,E. N. (1986).
Cognitive interference: Situational determinants and traitlike characteristics. loomaLof

BersonalmLaniSocralPsychologyaﬂ 215-226

Schmidt, R. A. & Bjork, R. A. (1992). New conceptualizations of practice:
Common principles in three paradigms suggest new concepts for training. Psychological

Science}, 207-217-

Schunk, D. H. (1991). Self-efﬁcacy and academic motivation. Special rssue:
Current rssues and new directions 1n motivational theory and research. Educational

Psychologw 207-231

Sellen, A. J. (1994). Detection of everyday errors. Special rssue: Errors, error

detection, and error recovery. WWW 475-
498.

Sellers, P. (1995, May). So you fail, now bounce back! Eortnnc, 49-66.

Singer, R. N. (1977). To err or not to err: A question for the instruction of
psychomotor skills ReyimofEducmionalResearchAl, 479-498

Singer, R. N. & Pease, D. (1976). A comparison of discovery learning and guided
instructional strategies on motor skill learning, retention, and transfer. IhoRcscarch

W. 788-796.
Skinner, B. F. (1953). Scroncoandhumanjmhayror. New York: Free Press.

Skinner, B. F. (1968). Ihcjcchnologycﬂoaching. New York: Meredith

Corporation.

Tannenbaum, S. I. & Yuk], G. (1992). Training and development 1n work
organizations. WW 399-441

177

Tellegen, A. (1985). Structure of mood and personality and their relevance to
assessing anxiety, with an emphasis on self-report. In A. H. Tuma and J. D. Maser (Eds. )

WWW (pp. 681-706). Hillsdale, NJ. Erlbaum.

Theodorakis, Y. (1995). Effects of self-efficacy, satisfaction, and personal goals
on swimming performance. IhoSmrtPsychologisLE, 245-253.

Thomas, K. & Mathieu, J. E. (1994). Role of causal attributions 1n dynamic self-
regulation and goal processes. MW, 812- 818.

Tucker, L. R., Koopman, R. F., & Linn, R. L. (1969). Evaluation of factor
analytic research procedures by means of simulated correlation matrices. Psychometrika.
31L 421-459.

Waldersee, R. & Luthans, F. (1994). The impact of positive and corrective

feedback on customer service performance. loumalofﬂrganizatronalﬁchayionﬁ, 83-
95.

Watson, D. & Clark, L. A. (1984). Negative affectivity: The dispositional to
experience aversive emotional states. PsychologicalBinlcttmio, 465-490.

Watson, D. Clark, L. A., & Tellegen (1988). Development and validation of
brief measures of positive and negative affect: The PANAS scales. loomaLof

BemnalMdSocialPsychologyzﬂ 1063- 1070

Weiner.B- (1986). Anattrihutionaljhcomofmotiyationandemoticn New
York: Springer-Verlag.

Weiner, B. (1985). An attributional theory of achievement motivation and
emotion. BMW, 548- 573

White, B. Y. & Frederiksen, J. R. (1986). WWW

aformdationmmtelligentleamingcnyircnments (Tech. Rep. No. 6277). Cambridge,
MA: Bolt, Beranek, & Newman.

Wofford, J. C. & Goodwin, V. L. (1990). Effects of feedback on cognitive
processing and choice of decision style. lmrmaloﬂApplichsychologyJi, 603- 612.

1III . I~III ‘ 3., III. ‘ ' .III ’ 30992).
Libertyville, IL: Wonderlic Personnel Test, Inc..

Wood, R. E. & Bandura, A. (1989). Impact of conceptions of ability of self-

regulatory mechanisms and complex decision making. lonmalcﬂBcrsonalinLaniSocial
W 407-415

"1111111111111111“