1:111. lit... :12 .0.
3: it! i2...!!:

 

1 .
_ +5
a

.1.

t

.. .8.
taxi .

1..

a
a!
«.13.:

.
l

 

MI...
5 3.34.? {it 31...:-
Y 1. . LIV 1.; EM..:‘A¢I..3..

.l
I... 5.
57:. .15
53‘s.}...
0).}

 

1i.
. r53)...
.. I: . .39....
3...:1r533v3
3

 

 

 

 

 

 

. . ~ ..

I .12.».Utux...»
1‘... a.
3.2.3:.

It}: Tu

 

 

 

.. :r.
uninitili
,.I.I}»!! it...
.., ..:.
X.

 

 
 

 

.7. . ..‘,.....:7
. 1.: .f . , , . «wrunu:
by”. ... Smirk}. ‘ . ;
. .I ‘ ‘ , .
1 .2. , . ,

 

 

III. III ' .|'. [Hilllll'll'
‘ I

THESts

J
200i

 

LIBRARY
Paiachlgan stats:
University

“H ‘0

 

 

 

--_l

This is to certify that the

thesis entitled

Promoting Active Learning: The Effects Of
Metacognitive Instruction And Trainee
Characteristics On Learning Processes And Outcomes

presented by

Aaron M. Schmidt

has been accepted towards fulﬁllment
of the requirements for

Master of Arts degree in Psychology

 

 

0 44; ZZ/
(.7 Major pro/fessor \

Date 5 May 1999

 

0-7 639 ‘ MS U is an Afﬁrmative Action/Equal Opportunity Institution

 

PLACE IN RETURN Box to remove this checkout from your record.
TO AVOID FINES return on or before date due.
MAY BE RECALLED with earlier due date if requested.

 

 

DATE DUE DATE DUE DATE DUE '
WW 0-: 2002
r h f} 941‘

 

t ‘C5

.3

f"
I 3"
,-
u

 

 

 

 

 

 

 

 

 

 

 

 

6/01 cJClRC/DatoDuopGS-DJS

 

 

PROMOTING ACTIVE LEARNING: THE EFFECTS OF
METACOGNITIVE INSTRUCTION AND TRAINEE
CHARACTERISTICS ON LEARNING PROCESSES AND OUTCOMES
By

Aaron M. Schmidt

A THESIS
Submitted to
Michigan State University
in partial fulﬁllment of the requirements
for the degree of
MASTERS OF ARTS
Department of Psychology
2001

Professor J. Kevin Ford

ABSTRACT
PROMOTING ACTIVE LEARNING: THE EFFECTS OF
METACOGNITIVE INSTRUCTION AND TRAINEE
CHARACTERISTICS ON LEARNING PROCESSES AND OUTCOMES
By

Aaron M. Schmidt

The resent upsurge of computer-based training has thrust issues of learner control to the
forefront. Research on metacognition suggests that those possessing weak metacognitive
skills are less able to direct their own learning and, therefore, see less beneﬁt from learner
control environments. Learners often fail to adequately and accurately monitor their
learning, resulting in sub-optimal decisions concerning how best to direct one’s learning.
In light of this problem, this study examined two interventions aimed at enhancing .
metacognitive activity during training. These interventions were placed within a model
of self-regulated learning that links training design features (e.g. metacognitive training
and prompting), trainee characteristics (e. g. goal orientations, self-efﬁcacy), learning
processes (e. g. metacognitive activity, study time), and learning outcomes (e. g.
declarative knowledge, post-training self-efﬁcacy, skill-based performance, application
performance). Results provided partial support for the model. As predicted,
metacognitive training interacted independently with both mastery goal orientation and
pre-training self-efﬁcacy in its effects on metacognitive activity throughout training.
However, the relationship between metacognitive activity and learning outcomes was

equivocal. Implications and directions for future research are discussed.

ACKNOWLEDGEMENTS

First and foremost, I would like to thank my thesis, Kevin Ford. His previous
work on related topics served as a constant source of guidance and inspiration for this
project. I am grateful for his conﬁdence in my abilities, his willingness to allow me the
freedom and time to explore and develop my ideas, and his helpful advice when
necessary to get me back on track. I would also like to thank my committee members,
Steve Kozlowski and Rick DeShon, for their helpful feedback and comments.

Thank you to my ﬁiends and family for your support throughout the process. If
not for the daily coffee runs, I would have never had the energy to complete this. I
appreciate all the words of encouragement along the way, as well as the frequent urgings
to stop obsessing on every detail and just get it done. Graduate school can be a
challenging, demanding, and sometimes ﬁ'ustrating experience, but your support has

made the process much more valuable and enjoyable.

iii

TABLE OF CONTENTS

LIST OF TABLES ............................................................................................................ vii
LIST OF FIGURES .......................................................................................................... viii
INTRODUCTION ............................................................................................................. l
Metacognition ............................................................................................................. 6
Metacognitive Monitoring ................................................................................ 10
Metamemory J udgrnents .......................................................................... 10
Feeling-Of-Knowing ........................................................................ 10

Conﬁdence In Retrieved Information .............................................. 12
Ease-Of-Learning J udgrnents ........................................................... 14

J udgments-Of-Learning ................................................................... 14
Understanding And Improving Monitoring Accuracy ............................. 16
Metacognitive Control ..................................................................................... 21
Monitoring Affects Control ..................................................................... 21
Limitations Of Metamemory Research ............................................................ 27
Generalizability ........................................................................................ 27
Motivation ................................................................................................ 30
Metacognitive Interventions ............................................................................ 31
Limitations Of Metacognitive Intervention Research .............................. 37
Generalizability ................................................................................ 37

Motivation ........................................................................................ 39

Prompting Vs. Training ................................................................... 40

Model And Hypotheses ............................................................................................ 42
Overview .......................................................................................................... 44
Transfer ............................................................................................................ 46
Learning Outcomes .......................................................................................... 47
Hypothesis la ................................................................................... 48

Hypothesis 1b ................................................................................... 49

Hypothesis 1c ................................................................................... 49

Learning Processes ........................................................................................... 50
Hypothesis 2a ................................................................................... 51

Hypothesis 2b ................................................................................... 51

Hypothesis 2c ................................................................................... 51

Hypothesis 3 ..................................................................................... 51

Trainee Characteristics ..................................................................................... 52
Goal Orientation ....................................................................................... 52
Hypothesis 4a ................................................................................... 53

Self-Efﬁcacy ............................................................................................ 53
Hypothesis 4b ................................................................................... 54
Conscientiousness .................................................................................... 54
Hypothesis 4c ................................................................................... 55

iv

Training Design ................................................................................................ 56

Metacognitive Monitoring Training ......................................................... 56
Hypothesis 5 ..................................................................................... 58
Metacognitive Prompting ......................................................................... 58
Hypothesis 6 ..................................................................................... 58
Metacognitive Training X Motivation Interaction ........................................... 59
Hypothesis 7 ..................................................................................... 59

METHOD ......................................................................................................................... 60
Participants ............................................................................................................... 60
Procedure .................................................................................................................. 61
Manipulations ................................................................................................... 63
Metacognitive Training ............................................................................ 63
Metacognitive Prompting ......................................................................... 65
Measures ................................................................................................................... 65
Control Variables. ............................................................................................ 66
Demographics and Cognitive Ability ....................................................... 66

Prior Experience ....................................................................................... 66
Pre-Training Metacognitive Skills ............................................................ 66
Motivation Variables. ....................................................................................... 68
Mastery Goal Orientation ......................................................................... 68
Pre-Training Self-Efﬁcacy ....................................................................... 68
Conscientiousness .................................................................................... 69

Learning Processes ........................................................................................... 69
Metacognitive Activities During Training ............................................... 69

Study Time ............................................................................................... 71

Learning Outcomes .......................................................................................... 71
Post-Training Self-Efﬁcacy ...................................................................... 71
Skill-Based Performance .......................................................................... 71
Declarative Knowledge ............................................................................ 72
Application Performance .......................................................................... 72

Data Analysis Plan ................................................................................................... 73
RESULTS ......................................................................................................................... 74
Manipulation Checks ................................................................................................ 74
Correlations .............................................................................................................. 76
Hypothesis Tests ....................................................................................................... 81
Hypotheses 1a, 1b, &1c ................................................................................... 81
Hypotheses 2a, 2b, & 2c ................................................................................... 83
Hypothesis 3 ..................................................................................................... 85
Hypotheses 4a, 4b, & 4c ................................................................................... 86
Hypothesis 5 ..................................................................................................... 89
Hypotheses 6 .................................................................................................... 9O
Hypothesis 7 .......................................................... ‘ ........................................... 92
Overall Model Testing .............................................................................................. 95

DISCUSSION ................................................................................................................ 102

Motivation To Learning Processes ......................................................................... 103
Metacognitive Interventions ................................................................................... 105
Learning Outcomes ................................................................................................ 1 11
Limitations .............................................................................................................. 115
Implications And Directions For Future Research ................................................. 122
APPENDICES ................................................................................................................ 125
Appendix A Informed Consent and Debrieﬁng ..................................................... 126
Appendix B: Pre-Training Measures ...................................................................... 129
Appendix C: Web Page Training ........................................................................... 134
Appendix D: Post-Training Measures .................................................................... 152
Appendix E: Metacognitive Training ..................................................................... 162
Appendix F: Performance Aid ............................................................................... 166
Appendix G: Metacognitive Prompting ................................................................. 167
REFERENCES ............................................................................................................... 168

LIST OF TABLES

Table l. Manipulation Checks ......................................................................................... 75
Table 2. Descriptive Statistics ......................................................................................... 77
Table 3. Analysis Summary ............................................................................................ 80
Table 4. Hierarchical Regression Analysis Results for Hypothesis 1 ............................. 82
Table 5. Hierarchical Regression Analysis Results for Hypothesis 2a ........................... 83
Table 6. Hierarchical Regression Analysis Results for Hypothesis 2b ........................... 84
Table 7. Hierarchical Regression Analysis Results for Hypothesis 2c ........................... 85
Table 8. Hierarchical Regression Analysis Results for Hypothesis 3 ............................. 86
Table 9. Hierarchical Regression Analysis Results for Hypothesis 4 ............................. 87
Table 10. Hierarchical Regression Analysis Results for Hypothesis 4 ........................... 88
Table 11. Hierarchical Regression Analysis Results for Hypothesis 5 ........................... 89
Table 12. Hierarchical Regression Analysis Results for Hypothesis 5 ........................... 90
Table 13. Hierarchical Regression Analysis Results for Hypothesis 6 ........................... 91 '
Table 14. Hierarchical Regression Analysis Results for Hypothesis 6 ........................... 92
Table 15. Hierarchical Regression Analysis Results for Hypothesis 7 ........................... 93
Table 16. Hierarchical Regression Analysis Results for Hypothesis 7 ........................... 95

vii

LIST OF FIGURES

Figure 1. Conceptual model ............................................................................................ 46
Figure 2. Predicted Interaction of Metacognitive Training and Motivation

Variables on Learning Processes ...................................................................... 59
Figure 3. Scree Plot of Metacognitive Skill Measure ...................................................... 67
Figure 4. Scree Plot of Metacognitive Activity Measure ................................................ 70
Figure 5. Self-Efﬁcacy X Metacognitive Training on Metacognitive Activity .............. 94
Figure 6. SEM analysis of hypothesized model, without interactions ............................ 96
Figure 7. Post-hoe model, without interactions .............................................................. 98
Figure 8. Multiple groups analysis, Metacognitive Training Absent/Present .............. 101

viii

INTRODUCTION

In order to meet the demand for quick, effective training, organizations have
begun to rely heavily upon new technologies and advancements in training design and
delivery. These technologies present new opportunities and challenges for both the
trainee and the organization. The resent upsurge of computer-based training, particularly
web-based training (WBT), provides a vivid example of the move toward technology-
based training systems. WBT offers great ﬂexibility to training designers in terms of
content, as well as the organization and presentation of that content (Brown, Milner,
Ford, & Golden, in press). WBT also provides the learner the opportunity to take greater
control over his or her own learning. Learner control refers to degree to which learners
are able to choose the method, timing, practice, and/or feedback during training (Milheim
& Martin, 1991). Although the idea of granting the trainee greater control over their
learning is not new (e.g. Tennyson, Christensen, & Park, 1984; Steinberg, 1977), web-
based training is uniquely suited to this purpose. Thus, the increased utilization of web-
based training programs has served to thrust issues of learner control into the forefront.

Research has pointed to advantages of increasing learner control. Among the
more ﬁequently cited beneﬁts are trainee attitudes and motivation. Students allowed to
choose the content, amount of study, sequencing, as well practice exercises have reported
more positive attitudes toward the training (Kinzie & Sullivan, 1989; Milheirn, 1989;
Morrison, Ross, & Baldwin, 1992). Learner control allows trainees to progress through

the training at a comfortable pace, as well as focus on content that is relevant to their

speciﬁc training objectives, leading to greater training motivation and expectations of
success (Milheim & Martin, 1991).

Despite these advantages, increased learner control brings to the table obstacles
that may counteract the advantages. Studies on the effects of learner control on post-
training achievement have failed to yield consistent results. While some have found that
individuals learn more under learner control than program control (e. g. Gray, 1987;
Hannaﬁn & Sullivan, 1995; Kinzie, Sullivan, & Berdel, 1988), others have found the
opposite (e. g. Tennyson, 1980; Tennyson & Buttrey, 1980; Tennyson & Rothen, 1979).

These inconsistent results may be explained in part by the ﬁnding that not all
learners are capable of successfully directing their learning, indicating that many learners
may fail to make the best use of the control they are given (Steinberg, 1989; Tennyson,
Christenson, & Park, 1984). A consistent ﬁnding is that individuals are often poor judges
of what or how much they need to learn (Tennyson & Rothen, 1979; Tennyson, 1980;
Williams, 1993) and, therefore, often either prematurely discontinue instruction before
mastery has been attained or continue to focus on material well past the point of mastery
(Tennyson, 1980). Given the shiﬁ toward web-based instruction, two key questions
demand consideration: 1). Why do learners so oﬁen fail to identify where to best allocate
their time and attentional resources? and 2). What can be done to help learners better
utilize their resources in a learner control training environment?

Examination of self-regulation in a learning context may shed some light onto
each of these issues. Self-regulation theories seek to explain the process by which goals
are translated into action. Two of the more prevalent theories of self-regulation are

Bandura’s Social Cognitive Theory (Bandura, 1986; Bandura, 1991) and control theory

(e.g. Carver & Scheier; Klein, 1989), both of which are built around the concept of the
feedback loop. In both social cognitive theory and control theory, the process of self-
regulation begins with a goal. One is assigned or chooses a standard by which to compare
their performance, behavior, etc. One then monitors their progress toward the standard,
examining feedback provided by the environment, as well as that provided by themselves
or others. This feedback is utilized to compare one’s level of performance to the standard.
When a discrepancy is discovered between performance and the standard, one makes
judgments about the discrepancy and chooses some action to reduce the discrepancy.

One aspect of self-regulation emerging from educational and cognitive
psychology, metacognition, may be particularly informative with respect to self-
regulation in learning or training contexts. Most broadly, metacognition refers to
”thinking about your thinking" (F lavell, 1979). Although metacognition is a broad,
nebulous construct that has been conceptualized and operationalized in a myriad of often
incompatible ways, a large portion of the metacognitive literature is focused on what
individuals know about the content of their own memories, how they come to acquire this
information, the accuracy (or lack thereof) of these assessments, and how this
information is utilized to guide subsequent learning. Individuals possessing greater
metacognitive skills are frequently found to learn more effectively than those who lack
these skills, as they actively monitor their progress, determine where problems exist, and
adjust their learning strategies accordingly (Brown, Bransford, Ferrara, & Campione,
1983)

The impact of metacognitive skills on learning effectiveness is unlikely to be

uniform across program and learner control environments. Under program control, the

extensive guidance built into the training program serves to direct learners and provides a
clear, relatively rigid structure under which they are to operate, which may serve to
reduce the impact of metacognitive skills. It is not critical that one know what, how, or
for how long they should attempt to acquire when these decisions have already been
made for the learner. Further, possessing strong metacognitive skills may be irrelevant,
or even detrimental, if the training program does not provided the ﬂexibility for trainees
to actively pursue preferred learning strategies.

In contrast, when provided the freedom to control their own learning process, it is
critical that trainees possess the skills to determine appropriate learning strategies. Those
who are unaware of what they do and do not know cannot be expected to make effective
decisions concerning how to best progress through the training. Those with strong
metacognitive skills are given a venue to fully utilize their skills by structuring their
learning experience in whatever manner they deem appropriate.

Although Kanfer and Ackerrnan (1989) have posited that, by diverting cognitive
resources off-task, self-regulatory activity has detrimental effects early in acquisition,
these effects should be less problematic under learner control. In many program
contexts, learners have pre-speciﬁed and limited time available to learn the content of the
training. If Kanfer and Ackerman’s propositions are correct, trainees have a limited pool
of cognitive resources available to allocate to learning activities in any given time period.
Thus, time spent engaging in self-regulatory activity reduces the amount of resources
spent on-task (on-task resources S total cognitive resources -— resources utilized for self-
regulatory activities). However, Kanfer et a1. (1994) demonstrated that, when given

ﬁ'equent breaks that could be utilized for self-regulatory activities, difﬁcult-speciﬁc goals

had a positive effect on learning outcomes, rather than the detrimental effects often found
on complex tasks. Learner control training contexts allow trainees the freedom to take
"breaks" whenever necessary to engage in "off-task" activities such as metacognition
without interfering with their focus on the content of the training. That is, trainees can, if
needed, devote all resources to training content and self-regulatory activities at separate
dedicated time periods. Thus, self-regulatory activities are no longer forced to compete
for limited resources during a set period of training - trainees can dedicate distinct blocks
of time to focus “on-task” and to self-regulatory activities, in effect increasing the total
time and cognitive resources devoted to the training as a whole.

How, then, can metacognition be leveraged to enhance self-directed learning?
While pre-existing metacognitive skills have been found to be positively related to
training success (e.g. Huet & Marine, 1997; Otero, Campnario, & Hopkins, 1992;
Owings, 1980), a strict reliance on preexisting trainee characteristics is inconsistent with
the goals of many training initiatives. Selecting only individuals possessing these
characteristics may not always be feasible or desirable for the organization. An
alternative is the utilization of metacognitive interventions aimed at increasing
metacognitive activity during training. This study is an attempt to examine the utility of
metacognition and metacognitive interventions for enhancing learning in a non-academic
training course. Although a number of studies have examined the utility of metacognitive
interventions for improving learning outcomes, most have focused on children within
educational settings. Further, the results of these studies have been equivocal,
particularly among studies of adult populations on work-like tasks. Additionally, these

studies have largely ignored the role of trainee characteristics, particularly those

concerned with trainee motivation. I begin with an in-depth analysis of the existing
literature on metacognition and metacognitive interventions. Limitations and
implications of this literature are addressed with respect to their applicability to
organizational training contexts. Finally a model seeking to address several key

limitations of the extant literature is presented and tested.

Metacognition

Although the concept of metacognition has been examined in various forms for
many years, F lavell is generally credited with bringing metacognition into prominence in
modern psychology. F lavell (1979) conceptualized metacognition as deliberate, planful,
goabdirected, and future-oriented mental behaviors that are directed toward
accomplishing a task. According to Flavell, metacognition can be divided into four
classes of phenomena: metacognitive knowledge, metacognitive experience, goals, and
strategies. Metacognitive knowledge refers to one's knowledge and beliefs concerning
people as cognitive beings. This knowledge consists of one's beliefs about their nature or
the nature of others as cognitive processors, knowledge about the task and its demands,
and knowledge about what strategies are likely to be effective in achieving one's goals.
Metacognitive experiences are insights or feelings one has conceming their knowledge,
such as the feeling that one does not understand. Such insights can lead one to establish
new goals or modify old goals, add to one's metacognitive knowledge, and inﬂuence the
activation of both cognitive or metacognitive strategies. Goals refer simply to the
Objectives one is attempting to achieve via their cognitive processing. Finally, strategies

are the speciﬁc actions undertaken to achieve one's goals.

Kluwe (1987) helped to ﬁrrther reﬁne the construct of metacognition. Kluwe
identiﬁed two attributes of metacognitive activity consistent with Flavell's (1979) notion
of metacognitive knowledge and metacognitive strategies, respectively. First, the
thinking subject has some knowledge of their own thinking and the thinking of others.
This attribute is based on declarative knowledge, such as the knowledge that one often
does poorly in mathematics. The second attribute consists of active monitoring and
regulation of one's own thinking, where one is acting as the causal agent of their own
thinking, and is characterized as procedural knowledge. Kluwe referred to this latter
attribute as executive processes. Executive processes were further subdivided into
executive monitoring and executive regulation. Executive monitoring processes are those
directed at attaining information about one's thinking process and include such processes
as identifying the task, checking and evaluating one's progress, and predicting the
outcomes of that progress. Executive regulation involves processes directed at
controlling or otherwise inﬂuencing the course of one's own thinking. This includes
decisions such as where to allocate one's resources, the speciﬁc steps to be used to
complete the task, and the speed and intensity at which to work on the task.

A key characteristic distinguishing metacognitive thoughts ﬁ'om other cognitive
phenomena is that metacognitive thoughts have other thoughts as their target. This raises
an interesting paradox of the thinker simultaneously being the one who observes and the
target of observation (Comte's paradox: J arnes, 1890). In attempting to resolve this
paradox, Nelson and Narens (1990) called on Tarski's (1956) solution to a similar
paradox, the Liar's Paradox, by proposing that each process consists of two inter-related

levels-an object level and a meta-level. Speciﬁcally, they proposed that consciousness is

hierarchically organized in such a manner that allows higher levels (meta-levels) to
monitor and control lower levels (object-levels).

A critical feature of this object-level/meta—level system is the direction of the ﬂow
of information between the two levels. Out of this come the two primary functions of the
metacognitive system, consistent with the executive processes identiﬁed by Kluwe
(1987) -- control and monitoring. Control is interpreted by stating that "the fundamental
notion underlying controluanalogous to speaking into a telephone handset--is that the
meta-level modifies the object level, but not vice versa" (Nelson & Narens, 1990, p. 127).
This information serves to direct the functioning of the object-level, instructing it to
initiate an action, continue an action, or terminate an action.

In order to determine what instructions to give to the object-level, it is necessary
for the meta-level to obtain information concerning the current status of the object level.
This is achieved via monitoring of the object-level. "The fundamental notion of
monitoring-analogous to listening to the telephone handset--is that the meta-level is
informed by the object level" (Nelson & Narens, 1990, p. 127). This information serves
to update the meta-level's model of the situation, which can then be evaluated with
respect to the goal the system is attempting to achieve.

The interaction between monitoring and control processes in metacognition
represents a negative-feedback loop such as that proposed by theories of self-regulation,
such as social cognitive theory (Bandura, 1986; Bandura, 1991) and control theory (e.g.
Carver & Scheier, 1982; Klein, 1989). In a learning context, the process of self-
regulation begins with the setting of a learning goal. This goal may represents the level

to which one seeks to learn the material at hand. These goals may be hierarchical in

nature, such that one has goals for particular items or components of the to-be-learned
material, as well as goals for the material as a whole. At this point, monitoring may
occur in the form of pre-learning judgments of the material. One example of such pre-
learning monitoring is ease-of-learning (EOL) judgments, wherein learners make implicit
or explicit judgments concerning how difﬁcult the material will be to learn (Leonesio &
Nelson, 1990). These and other pre-learning judgments then inﬂuence the initial learning
strategies utilized by the learner in their attempt to attain the desired level of mastery over
the material. During ongoing acquisition, the learner monitors their memory to gauge
their current level of knowledge and understanding. By engaging in this online
monitoring of comprehension or understanding, often referred to as judgments-of-
learning (J OLs), the learner is providing himself or herself with self-generated feedback
which may be utilized independently or combined with other sources of feedback. This
feedback is compared with the learning goal to determine progress toward the goal. The
learner then engages in control processes, wherein they take some form of action, such as
changing learning strategies, terminating study of the material, or simply making no
changes. It can be seen from the discussion above that a major premise of metacognitive
theory is that metacognitive processes can serve as a causal agent in the acquisition of
knowledge.

The past two decades have seen increased interest in issues such as consciousness
and metacognition, as well as improved methods of assessing such phenomena. This has
lead to a wealth of research on cognitive topics such as metacognition, leading to strong
evidence of its' causal role in leaming. This evidence is presented in the sections that

follow, in which I provide a review of the literature on metacognitive monitoring and

control. This review begins with a description of metacognitive monitoring processes,
including the various metarnemory judgments that have been examined and the
theoretical mechanisms underlying them. This is followed by an exploration of
metacognitive control processes and the link between metacognitive monitoring and
control.
Metacognitive Monitoring

Metamemog Judgments

Feeling-of-Knowing. While F lavell (1971; 1979) was responsible for stirring
much of the current interest in metacognition, a few studies with relevance to
metacognition predate his seminal work. Much of this early work centered on topics
relevant to memory monitoring, also known as metarnemory. Some of the earliest
research on metarnemory concerned a metacognitive experience referred to as feeling-of-
knowing (F OK). Even after failing to recall previously studied material, individuals are
still capable of making judgments concerning the degree to which the information is
known. For example, even though one may not currently be able to recall the winners of
Super Bowl XXV (New York Giants), they may have a good idea (feeling-of-knowing)
of whether they in fact "know" this information and may subsequently be able to retrieve
the correct answer ﬁom memory. Feelings-of-knowing are closely related to, although
less intense, than the tip-of-the-tongue experience. The tip-of-the-tongue experience is
the failure to retrieve information from memory, accompanied by a sensation of
imminent recall (Brown, 1991). Although the tip-of-the-tongue phenomenon had been
discussed at least as early as the 1940's (e.g. Woodworth, 1940), these early

investigations were lacking in several respects, most notably in their lack of

10

methodological precision, as well as their failure to examine the accuracy of these
experiences.

In 1965, Hart conducted one of the ﬁrst systematic studies seeking to determine
the accuracy of F OK judgments. Undergraduate students were presented with lists of
general-information questions, such as "Which planet is the largest in our solar system?"
After each question was presented, subjects were given 10 seconds to attempt to recall
the answer and write it down. If they were unable to recall the answer, they were asked
to make a feeling-of-knowing judgment, indicating whether they believed they knew the
answer but were unable to recall it at the moment and would be able to identify the
correct answer in a multiple-choice format. After attempting to recall the answer to each
question, subjects were presented with a multiple-choice form with the same questions.
The results of the recognition test were compared to the subj ects' feeling-of-knowing
ratings for each item to determine whether the FOK judgments could predict recognition
' performance. These results indicated that FOK judgments are accurate, albeit imperfect
indicators of the content of one's memory. Following up on this initial study, Hart (1967)
later found that the FOK judgments increased in magnitude as the number of study trials
increased.

In a more recent study, Nelson, Leonesio, Landwehr, & Narens (1986) asked a
series of general-information questions to which individuals attempted to recall the
answer and made F OK judgments for items not recalled. Each item had previously been
normed with respect to both recall probability and F OK judgments, such that Nelson et
a1. knew the normative probability of recall and subsequent F OKs for each item. It was

predicted that the normative probability of recall would be the least accurate predictor of

11

 

recognition performance, while the individuals' own F OK judgments would be the most
accurate, as the individual should be most aware of what they know. Surprisingly, this
hypothesis was not conﬁrmed. Rather, the normative probability of recall was the most
accurate predictor of recognition, followed by the individuals' own F OK judgments. That
the individuals' own FOK judgments were better predictors of recognition than normative
FOKS suggests that individuals have some access to their unique, idiosyncratic
knowledge. However, the ﬁnding that normative recall probability was a better predictor
than F OK judgments indicates that this access is somewhat limited.

Many other feeling-of-knowing studies have been conducted using various
subject populations (e. g. children: Welhnan, I977; mentally retarded: Brown & Lawton,
1977; older adults: Lachman, Lachman, & Thronesbery, 1979), types of items (e.g.
nonsense syllables: Blake, 1973; picture labels: Wellman, 1977; names of entertainers:
Read & Bruce, 1982), and criterion tests (e.g. recognition tests: Hart, 1965; cued-recall:
Gruneberg, Monks, & Sykes, 1977). The vast majority of these studies have found that
FOK judgments are accurate at moderate, but above-chance levels. This leads to the
conclusion that individuals have partial, but far ﬁ'om perfect access to their knowledge
concerning items for which they cannot immediately recall the answer.

Conﬁdence in Retrieved Information. A second category of metacognitive
monitoring is the conﬁdence one has in the answers they provide. In contrast to FOK
judgments, judgments of conﬁdence are purely retrospective in that they occur after both
acquisition and retrieval of the target information. That is, individuals are making

judgments concerning the content of their memory after they have provided an answer.

12

Conﬁdence ratings have often been obtained in a similar manner as FOK
judgments, only for recalled, rather than non-recalled items, as is the case with FOK
judgments. More speciﬁcally, FOK judgments can be made for omission errors (items
not recalled) or commission errors (items incorrectly recalled), whereas conﬁdence
judgments are made for correctly recalled items or for commission errors (N arens,
Jameson, & Lee, 1994). In the typical study, subjects are ﬁrst presented with a set of
general information questions. For each item recalled, subjects are asked to rate how
conﬁdent they are that the answer they provided was correct. These ratings can then be
compared to recall performance to determine the relationship between conﬁdence and
accuracy.

The typical ﬁnding for conﬁdence ratings is that they are only moderately well
calibrated, and generally reﬂect overconﬁdence (e.g. Bomstien & Zickafoose, 1999;
Koriat & Goldsmith, 1996; Lichtenstein & Fischoﬁ‘, 1977). The general trend of
moderate calibration and overconﬁdence has been found not only for general knowledge
questions and item lists, but for eyewitness accuracy, as well (e.g. Wells & Murray,
1984). A meta-analysis of the conﬁdence and accuracy among witnesses of staged events
found that a mere .25 correlation between conﬁdence and accuracy (Bothwell,
Deﬁ‘enbacher, & Brigham, 1987). As in general knowledge domains, the general source
of inaccuracy is overconﬁdence. Bornstein and Zickafoose (1999) directly compared
conﬁdence and accuracy across the domains of general knowledge and eyewitness
memory. They too found evidence of overconﬁdence, which was consistent across
domains. Interestingly, they found that providing individuals feedback concerning their

degree of overconﬁdence reduced their subsequent levels of overconﬁdence.

13

Ease-of-Learning Judgments. A third metarnemory judgnent is ease-of-learning
(EOL) judgnents. EOL judgnents differ from other metarnemory judgnents in that they
occur prior to acquisition, while most other judgnents are made at some point during or
after acquisition. These judgnents are essentially predictions of how easy or difﬁcult the
material will be to learn.

Leonesio & Nelson (1990) found that EOL judgnents sigiiﬁcantly predicted the rate of
acquisition (number of trials needed to attain a given number of correct recalls), with
items with higher EOLs requiring fewer trials to learn, although this relationship was
only of moderate size (r = -.22). Additionally, a small, but sigriﬁcant correlation was
found between EOL and recall, despite the fact that recall occurred four weeks after
EOLs and initial learning of the items. Hall and Bahrick (1998) found that EOL ratings
had a mean correlation of .24 with ﬁnal recall performance 5 years after initial
acquisition. Despite the long-term relationship between EOL ratings and recall, EOL

- judgnents are most predictive of early performance (Nelson & Leonesio, 1988).

A consistent ﬁnding of the EOL research to date is that EOL judgnents are
moderately accurate predictors of actual rate of learning, as well as subsequent recall
performance. This adds further support to the conclusion that individuals have at least
partial access to their existing knowledge, which allows them to estimate the ease with
which material may be learned. However, the modest size of this relationship points to
the fact that the accuracy of individual's monitoring is far ﬁom optimal.

Judgments-of-Learning. While much of the early metarnemory research focused
on the metacogritive experience of feeling-of-knowing, more recently the focus has

shifted to judgments-of-learning (J OL) as the primary metarnemory judgnent of interest.

14

J udgnents-of-learning represent one's perceptions concerning the degee to which
currently or recently studied material has been learned and can be retrieved from memory
at a later time (Leonesio & Nelson, 1990). Operationally, they are generally assessed
prior to a test of retention (generally recall), and are often followed by additional study
time. Thus, in contrast to most other metarnemory judgnents, which generally occur
either before (ease-of-learning judgnents) or after acquisition (feeling-of-knowing and
conﬁdence judgnents), J OLs represent online monitoring of one's memory, as they occur
during the process of acquisition.

Like the metarnemory judgnents discussed above, studies of J OLS typically
utilize list learning, such as paired-associates, or general knowledge questions. For
example, Arbuckle and Cuddy (1969) asked subjects to learn lists of paired-associates,
with each item presented for 3 seconds. Immediately following the presentation of each
item, subjects made "Yes" or "No" predictions referring to whether they would be able to
recall that item on a subsequent retention test. Subjects were able to predict their recall
performance at moderate, but above chance levels. In a study using a similar desigr, the
accuracy of J OLs in predicting recall performance was established for multi-trial
acquisition of paired associates as well (V esonder & Voss, 1985). Using unrelated noun-
noun pairs, Leonesio and Nelson (1990) also found that JOL ratings were moderately
correlated (r z .30) with recall performance. Additionally, JOLS sigriﬁcantly predicted
the number of trials required to acquire each item to a predetermined criteria.

While most studies of J OLs have examined their relationship with recall
performance, they have been demonstrated to predict recogrition as well (e. g. Begg,

Dutt, Lalonde, Melnick, & Sanvito, 1989; Leonesio & Nelson, 1990; Thiede &

15

Dunlosky, 1994). For example, in the aforementioned study by Leonesio and Nelson
(1990), J OLs made after initial item acquisition were correlated over .20 with recogrition
of items that initially could not be recalled on a previous test, despite the fact that the
recogrition test took place four weeks after acquisition and the recall test. Theide and
Dunlosky (1994) and Begg et a1. (1989) also found that JOLs predicted both recall and
recogrition, but found that J OL accuracy was sigriﬁcantly geater in predicting recall,
although correct guessing may contribute to the reduced accuracy of J OLs for recogiition
tests.
Understanding and Improving Monitoring Accuracy

While overconﬁdence has been found to be the rule, rather than the exception,
some steps have been effective in improving the accuracy of metacogritive monitoring.
In attempting to determine the psychological processes underlying inﬂated post-response
conﬁdence ratings, Koriat, Lichtenstein, and Fischoff (1980) suggested that individuals
selectively focus on evidence that supports their chosen answer, rather than evidence that
may contradict it. To test this hypothesis, subjects were asked to either provide reasons
supporting the chosen answer, reasons contradicting the chosen answer, both reasons for
and against chosen answer, or no reasons (control condition). Providing contradictory
reasons led to geater calibration, whereas providing supporting reasons had no effect,
suggesting that this is what individuals do naturally when making conﬁdence ratings.
Other studies have found that informing individuals of the tendency toward
overconﬁdence, as well as providing feedback concerning their own overconﬁdence

improve calibration and reduce overconﬁdence (e. g. Bornstein & Zickafoose, 1999).

16

While some research has examined the mechanisms underlying and means for
increasing the accuracy of one's post-response conﬁdence, much of the attention has been
on understanding and improving the accuracy of judgnents of learning (J OLs) that one
makes during the learning processes. Vesonder & Voss (1985) found that predictions of
trial 2 recall were signiﬁcantly more accurate when subjects had previously made recall
predictions at trial 1. Speciﬁcally, the general tendency toward overestimating one's
learning was reduced for those making JOLS on both trials, leading to increased accuracy.
J OL accuracy has also been found to increase when they follow earlier attempts at
retrieval (e.g. King, Zechmeister, & Shaughnessy, 1980). For example, subjects who
were only presented with study trials prior to a ﬁnal recall test made less accurate J OLs
than did those who alternated between study trials and test trials prior to the ﬁnal recall
test (King, Zechmeister, & Shaughnessy, 1980). However, when presented with a new
paired-associates list and study trials only, JOL accuracy between the goups was
equivalent, indicating that the increased J OL accuracy during the study-test trial blocks
was due to knowledge of previous performance, rather than the development of superior
monitoring capabilities. These results indicate that engaging in previous attempts at
retrieval is often successful in increasing monitoring accuracy. It is important to note,
however, that study time was experimenter controlled and was equivalent across items,
the implications of which will be highlighted in the following section on metacognitive
control.

In the ﬁrst of two experiments examining the effects of different kinds of
encoding on J OLs and recall, Mazzoni and Nelson (1995) sought to determine whether

J OL accuracy for lists of unrelated Italian word pairs was geater as a result of intentional

l7

 

learning in comparison to incidental learning. Subjects in the intentional learning goup
were instructed to read and leam the items, while those in the incidental learning goups
were either asked to rate the pleasantness of each word or report the number of syllables
for each word. Although recall was equivalent for all goups, J OL accuracy was geater
under intentional learning.

Given the geater accuracy of J OLs following intentional learning, Mazzoni and
Nelson (1995) conducted a second study to examine JOL accuracy for general-
knowledge items under two types of intentional learning: passive reading and active
generation of responses during acquisition. The passive reading goup was presented
with each item for 3 3, read the answer for 7 s, and then had 7 s to make a J OL. After the
3 s item presentation, the active generation goup had 7 s to attempt generate, rather than
read, the answer prior to making a J OL for each item. Active generation led to J OLS of
sigriﬁcantly geater magritude, as well as geater recall performance, than did passive
reading. These results also point to the advantages, with respect to monitoring accuracy,
of active attempts at retrieval.

One of the most robust factors inﬂuencing the accuracy of jud gnents-of-learning
is the delay between item study and the JOL rating, known as the delayed-JOL effect.
Nelson and Dunlosky (1991) had subjects make JOL ratings for each paired associate
either immediately after presentation or alter the presentation of eight additional items.
When the J OLs were made immediately after presentation, accuracy was moderate at
around .35. However, when delayed, J OL accuracy increased for all subjects, and was
nearly perfect for most (.80 overall). Thiede & Dunlosky (1994) also compared the

accuracy of immediate and delayed JOLs for predicting both recall and recogrition of

18

paired associates and found that, for both tests of retention, JOLs were more accurate
when delayed, rather than immediate. Connor, Dunlosky, and Hertzog (1994) found
evidence of a strong delayed-JOL effect in both young and older adults.

Several hypotheses have been forwarded to explain the accuracy of delayed, in
comparison to immediate, JOLs. Based on the ﬁnding that a delayed-JOL effect occurs
when J OLs are based on the stimulus alone (e. g. "dog-?") but not when based on both the
stimulus and response (e. g. "dog-table"), the transfer-appropriate—monitoring (TAM)
hypothesis has been proposed (Begg et al., 1989). This hypothesis suggests that the locus
of the delayed-JOL effect is the contextual similarity between the J OL and the recall test.
Delayed JOLs cued by the stimulus alone are more similar to a cued recall test than are
delayed J OLs cued by stimulus and response or immediate J OLs cued by either stimulus
alone or stimulus and response. To test this hypothesis, Dunlosky and Nelson (1997) had
subjects make either delayed or immediate JOLs cued by the stimulus alone or the
stimulus and response and examined the resulting correspondence between these J OLs
and a recogrition test. In the case of recogrition, the stimulus-response cue is most
similar to the retention test and thus, the TAM hypothesis would predict, should yield the
geatest JOL accuracy. However, the results showed that even when predicting
recogiition, rather than recall, J OLs cued by the stimulus alone were more accurate than
those cued by both the stimulus and response, refuting TAM as a primary cause of the
delayed-JOL effect.

The monitoring-retrieval hypothesis states that, when making judgnents about
their learning, individuals must monitor their memories for information about the item in

question (e.g. Dunlosky & Nelson, 1994; King, Zechmeister, & Shaughnessy, 1980;

19

Spellrnan & Bjork, 1992). To the extent that the information one retrieves from memory
concerning an item is predictive of eventual performance on a retention test, the J OL will
be accurate.

One particular case of the monitoring-retrieval hypothesis, the monitoring-dual-
memories (MDM) hypothesis, has been widely cited as a major source of the delayed-

J OL effect (Dunlosky & Nelson, 1992; Kelemen & Weaver, 1997; Nelson & Dunlosky,
1991). According to the MDM hypothesis, a person monitors information from both
short-term and long-term memory in parallel (W escourt & Atkinson, 1973). When J OLs
are made immediately following presentation or study of an item, information about the
item is still present in STM, which may interfere with one's ability to make a valid
assessment of whether the item has been successfully transferred to LTM. Delaying the
J 01.5 for a period of time allows the item to be cleared from STM. Thus, delayed
assessments of the presence or absence of the item in memory are based solely on
retrieval (or lack thereof) from LTM. In the case of stimulus-only versus stimulus-
response cues for J OLs, providing both the stimulus and response allows both to enter
STM, leading one to overestimate their degee of learning of the item, whereas the
stimulus alone forces one to search LTM for the response.

Several pieces of evidence support the MDM hypothesis as a major source of the
efﬁcacy of delayed J OLs. The longer the delay between item study and J OLs, the geater
the accuracy of the J Dis (Nelson & Dunlosky, 1991). Additionally, providing STM
interference between item study and JOLs dramatically improves J OL accuracy
(Kelemen & Weaver, 1997). Regardless of the explanations, it is clear the delayed-JOL

effect is a robust ﬁnding with geat implications for self-regulated learning.

20

Metacogm'tive Control

Metacogritive control refers to activities taken by individuals to inﬂuence the
course of their own thinking, including initiating, monitoring, and prioritizing mental
activities (Mazzoni & Nelson, 1998). This conceptualization of control corresponds to
Flavell's (1979) notion of metacogritive strategies, Brown's (1978) metacogritive skills,
and Kluwe's (1987) executive regulation. The wide variety of cogritive activities
subsumed under the concept of metacogritive control can be roughly divided by the stage
of learning to which they are most relevant (Nelson & Narens, 1994). During acquisition,
the key control processes are the selection of a mnemonic strategy (i.e. repetition,
visualization, etc.), the allocation of study time, both in total and to individual
components of the learning material, and termination of study time. Additional control
strategies relevant to acquisition are determining the speciﬁc steps to be taken to acquire
the information (in addition to the mnemonic strategy to be used), as well as the speed
and intensity at which one should work at the learning task (Kluwe, 1987). During
retrieval, the key control processes include the selection of a search strategy and
termination of the search for learned material.

Monitoring Affects Control. It has been demonstrated that individuals are capable
of monitoring the content of their memories with varying degees of accuracy and bias.
While the privileged access to one's own memory is an interesting phenomenon in itself,
the true value of monitoring, ﬁom a training perspective, is in its effects on control
processes during learning. Knowledge of one's consciousness plays a critical role in the
ongoing acquisition of knowledge and skills. The monitoring-affects—control hypothesis

proposes that information gained through monitoring one's memory can be utilized by

21

evaluating it relative to the goal state to determine the actions in which to engage to
achieve one's learning goals (Nelson & Leonesio, 1988). Effective self-regulation of
learning is predicated on an accurate assessment of what is known or not known
(Schoenfeld, 1997).

Many empirical examples support the role that monitoring plays in control
processes. Much of this research has focused on the inﬂuence of monitoring judgnents
on the allocation of self-paced study time. In one of the ﬁrst such studies, carried out by
Bugelski and Rickwood (1963), it was found that subjects in a self-paced study condition
outperformed those in an experimenter-paced study condition on a retention test.
Bugelski and Rickwood concluded that subjects in the self-paced condition were able to
allocate additional study time to the more difﬁcult items, which subsequently increased
retention. However, only aggegate, rather than item-by—item study time was reported,
precluding strong support for this conclusion. Other studies have found that subjects do
not allocate study time uniformly across items, but rather allocate more time to some
items than others (e.g. Belmont & Butterﬁeld, 1971; Butterﬁeld & Belmont, 1977; Zacks,
1969). This also has been interpreted as evidence of the allocation of more study time to
items of geater objective difficulty (as determined by normative probability of recall).

These studies provide indirect evidence that individuals allocate additional study
time to items of geater difﬁculty. Implicit in this conclusion is the assumption that
individuals make reasonably accurate, idiosyncratic assessments of item difficulty and/or
accurately monitor their learning during acquisition. In turn, subjects are presumed to
use these judgnents to determine how much study time to allocate to each item.

However, these assumptions had yet to be directly examined. Nelson and Leonesio

22

(1988) provided one of the ﬁrst such direct tests of the monitoring-affects-control
hypothesis. It was predicted that subjects would use both ease-of-learning (EOL) and
feeling-of-knowing (FOK) judgnents to allocate self-paced study time differentially
among the items to be learned. Consistent with their hypotheses, a negative correlation
was found between EOL ratings and item-by-item study time, such that items rated as
being more difﬁcult to learn were studied longer. FOK judgnents were also sigriﬁcantly
correlated with study time, with items receiving lower feeling-of-knowing ratings being
studied longer.

A surprising ﬁnding was that additional study time yielded little gain in terms of
performance on the retention tests, a ﬁnding that has been labeled the "labor-in-vain"
effect. Subjects who were instructed to learn the items as accurately as possible studied
up to three times longer across all items than did subjects instructed to learn the items as
quickly as possible. However, despite the extra study time, the accuracy goup only
demonstrated better retention in one of three studies and, even where sigriﬁcant, the
difference was minimal. In a similar study examining the role of EOL and J OL ratings

on self-paced study time, Mazzoni and Comoldi (1993) found results that they interpreted
. as an "item labor-in-vain" effect. Speciﬁcally, although both EOL and J OL ratings were
negatively correlated with study time, as predicted, items studied longer were not recalled
with higher ﬁ'equency. From these results, the authors concluded that "the effect makes
the choice of a correct study time strategy irrelevan " (p.48).

Nelson (1993) provided several alternative explanations for the " labor-in-vain"
effect. These explanations centered on an item selection bias, such that items were not

initially equivalent with respect to difﬁculty. Because subjects chose which items to

23

study longer, more difﬁcult items received lower EOL and JOL ratings and were
subsequently studied longer. This extra study time may have, in fact, increased the
retention of the more difﬁcult items. However, even the extra study time may not have
been sufﬁcient to bring the more difﬁcult items up to the same level of retrieval as the
less difﬁcult items. Such a situation, where extra study time leads to geater retention
than would have been observed otherwise, would represent "labor-and-gain," rather than
"labor-in-vain." Labor-and-gain can occur regardless of item retention relative to other
items. This hypothesis is known as incomplete compensation, and stands in contrast to
the complete compensation (extra study time equalizes retention of all items) and the
overcompensation hypothesis (extra study time leads to better retention).

Other studies have also demonstrated a sigriﬁcant relationship between
judgnents-of-learning and self-paced study time (Bauer, Kyaw, & Kilbey, 1984;
Dunlosky & Connor, 1997; Mazzoni, Comoldi, & Marchitelli, 1990). For example,
Mazzoni, Comoldi, and Marchitelli (1990) found that J OLs sigriﬁcantly predicted study
time, with geater study time allocated to items judged to be less well learned. These
items were subsequently recalled equally well as were items judged to be easier.
Dunlosky and Connor (1 997) investigated the utilization of memory monitoring for the
allocation of study time among undergaduates and older adults (mean age of 67 yrs).
They found that individuals from both age goups utilized their judgnents of learning to
allocate study time during later study periods.

Surprisingly, relatively few studies have investigated how the allocation of study
time is inﬂuenced by delayed J 01.8. However, Nelson, Dunlosky, Graf, and Narens

(1994) provide a notable exception, demonstrating the promise of utilizing delayed J OLs

24

to improve the allocation of study time and subsequent recall. Using a computer-
controlled variant of the familiar paired-associates paradign, participants ﬁrst learned 36
item-pairs, followed by delayed judgnents of learning for each item (based on the
stimulus alone: ex. ardhi- ?). After the delayed-JOLS for each item had been made,
subjects in a "self-chosen-items" goup chose 18 of the 36 items that would later be
restudied. This was followed immediately by a self-paced recall test for all 36 items.
After the recall test, all subjects were presented with 18 items to restudy. Subjects in the
self-chosen-items goup restudied those 18 items that had been chosen earlier. In the
worst-learned-items goup, the computer chose for each subject the 18 items to which

they had given the lowest JOLs. In the best-leamed-items goup, subjects were presented

 

with the 18 items to which they had given the mm JOLs. Finally, in the normatively-
most-difﬁcult-iterns goup, subjects were presented with the 18 least recalled items,
based on norms ﬁom previous studies. This was again followed by a self-paced recall

. test. Four additional restudy-test cycles followed.

As predicted, the lowest recall across all six test trials resulted from restudy of the
"best-leamed" items, followed by restudy of the normatively-most-difﬁcult items. The
"worst-learned" and self-chosen items goups did not differ in their recall for any of the
six trials. The "best-leamed" and nonnatively-most-difﬁcult items conditions failed to
utilize individuals' judgnents of learning to allocate study time to the items that most
needed additional study. Rather, the "best-leamed" items condition allocated restudy to
items for which their exists the least potential for improvement. That recall was lower for
the nonnatively-most-diﬁicult items condition than the remaining two conditions

indicates once again that individuals' are sensitive to idiosyncratic information in

25

memory, and that judgnents deriving ﬁom these are more useful for guiding subsequent
study than is normative information. Taken as a whole, these ﬁndings indicate that
utilization of delayed judgnents of learning during study time allows learners to allocate
the geatest study time to the items which need it the most, which can lead to increased
memory performance.

While much of the research reviewed to this point has been conducted in
laboratory settings, metacogritive monitoring has been implicated as a key causal
variable of learning in more naturalistic settings. In a study of class performance among
ﬁfth-gade students, Owings et a1. (1980) discovered that the more successful students
spontaneously and accurately monitored their comprehension as they read and studied
and, as a result, were more aware of when they were having difﬁculty. Further, the
successful students allocated geater study time to more difficult stories, whereas less
successful students allocated study time evenly across stories, indicating that the
successful students utilized their monitoring to regulate their learning. Similarly, Otero,
Campanario, and Hopkins (1992) found a strong positive relationship between
comprehension monitoring and GPA among 10th and 12th gade students. Huet and
Marine (1997) investigated the relationship between metarnemory knowledge and
memory performance on a simulated table-waiting job. Metamemory knowledge
sigiiﬁcantly predicted performance on each memory task, regardless of other memory

constraints, such as table size and perceptive cues.

26

Limitations of Metamemog Research

Generalizability. Among the most salient limitation of the metamemory research
concerns issues of generalizability. Despite the substantial empirical support for the
positive impact of metacognition on learning, one must question whether similar results
can be expected in real-world training settings. Much of the concern over the
generalizability of the results is driven by the simplicity of the learning tasks utilized in
the vast majority of the metarnemory studies. Much of the metarnemory research has
relied on list-learning as the vehicle to examine individual’s monitoring of their own
memories. Some have used lists of general knowledge items (i.e. “What is the capitol of
Finland?”), while the majority have used lists of paired associates, such as English-
Swahili equivalents for a given concept (i.e. “Soil-Adhir”).

In a 1995 address, Kraiger highlighted differences in task complexity as a key
difference between cognitive and I/O psychology paradigms, noting that the reliance on
simple tasks is often consistent with the goals of basic cogritive research. For cognitive
researchers studying learning, the focus is on fundamental learning mechanisms that are
assumed to be active across individuals and contexts. Therefore, tasks are chosen that
allow a clean test of the hypotheses without interfering with the mechanisms or practice
conditions under investigation. In contrast, l/O psychology is concerrned with learning
mechanisms and practice conditions only in so far as they are relevant in real-world
training and/or performance contexts. Therefore, geater concern is given to the ﬁdelity
of the task, the extent to which it rrnirrors critical elements of the target.

Given the goals of cognitive psychology, the use of simple learning tasks by

metarnemory researchers is neither surprising nor unjustiﬁed. However, such tasks

27

ignore differences in prior knowledge, environmental factors, are often of short duration,
etc., making attempts to generalize the ﬁndings to more complex tasks tenuous (Kraiger,
1995). While the list-learning/paired-associates paradign may resemble some of the
requirements for some jobs in which memorization of lists of information are a critical
aspect of the job, such as that of table waiting, by and large, list learrning represents a very
narrow sample of the type of knowledge, skills, and abilities that must be acquired to
perform effectively in most jobs.

Coinciding with its focus on relatively simple learning tasks, much of the existing
research on metacognitive monitoring has evaluated the impact of monitoring using a
narrow range of criteria, most often consisting of performance on a recall and/or
recognition test occurring immediately following the presentation and study of the list.
While this is understandable, given the tasks generally utilized in such studies, it serves to
compound the difﬁculty of generalizing the ﬁndings to more complex settings and
criteria. It has been convincingly demonstrated that individuals are capable of accurately
monitoring their memories under paired-associate learning tasks and that this monitoring
can inﬂuence control processes, which allow for better performance on recall and/or
recognition tests immediately following study. However, such indicators of immediate
learning or training performance may not reﬂect the degee to which training will be
successful in the long-term. It has been found that interventions which introduce geater
difﬁculty to the learner may slow acquisition and result in lower performance during
training, yet often yield substantial long-term gains relative to interventions maximizing
speed of acquisition and performance during training (Bjork, 1994; Schmidt & Bjork,

1992). From this ﬁnding, it is clear that immediate training performance is insufﬁcient to

28

capture the long-term utility of a training intervention. While a few studies have linked
metarnemory judgnents with long-term retention (e.g. Hall & Bahrick, 1998), the distal
effects of metacognitive monitoring have gone largely unexarrnined.

In addition to focusing primarily on immediate indicators of learrning, the criteria
of choice have almost exclusively been measures of verbal or declarative knowledge.
Many theories of knowledge acquisition agee that declarative knowledge must be
attained before higher-level development can occur (e. g. Ackerrnan, 1987; Anderson,
1982). Because declarative krnowledge generally serves as a foundation for further high-
order cognitive development (Anderson, 1982) and are most sensitive during the early
stages of knowledge acquisition (Kraiger, Ford, & Salas, 1993), the placement of recall
or recogrition tests immediately following study is serendipitously well timed. However,
it is in their reliance on immediate declarative knowledge as the s_ol§ measure of learrning
that the metarnemory literature has missed the mark.

Because declarative krnowledge may not discrirrninate between learners at higher
levels of cognitive development (Kraiger, Ford, & Salas, 1993), simply re-assessing
declarative knowledge at some later point during or after training is not sufﬁcient to
capture the ﬁll] range of cognitive development. A valuable addition would be measures
assessing skill-based and affective learrning outcomes. During initial skill acquisition,
performance tends to be slow, error-prone, and resource-dependent (Weiss, 1990).
However, as skill development progesses, performance becomes faster, less error-prone,
and approaches automaticity, ﬁeeing up cognitive resources which can then be directed
toward additional tasks or to the development of more elaborate task strategies. In

addition, during the latter stages of skill development, learners become more adept at

29

 

determining the appropriate situations for skill use (Gagre & Medsker, 1996), allowing
them to adapt their skills when necessary. Skill-based learning outcomes represent an
important criterion for organizational training progams, yet the role of metacogritive
monitoring in skill development remains largely unexplored.

Motivation. Another limitation of the current metarnemory research is that it has
failed to examine the extent to which individuals vary in their use of metacognitive
monitoring, or the ability of motivational learner characteristics to explain or predict this
variability. The design of most metarnemory studies is such that all individuals are
required to make monitoring judgnents before they can proceed to the next stage of the
study. Under such controlled experimental conditions, individual differences in general,
and those that inﬂuence motivation in particular, are less able to affect the frequency with
which individuals engage in monitoring. However, even under these limited conditions,
one might expect individual differences to inﬂuence aspects of metacogrition such as the
accuracy of monitoring and the extent to which individuals use the information gained by
monitoring to guide their learning via control processes.

The failure of metamemory researchers to examine the effects of motivation or
other individual differences is consistent with the goals and assumptions of much of the
cognitive literature (Kraiger, 1995). Because tlne processes under examination are
generally assumed to be common to all individuals, individual differences are regarded as
error, hence, experiments are strategically designed to minimize this source of "error
variance." However, in less tightly controlled environments, as more accurately reﬂects
most real-world learning environments, there is likely to be geat variability in the extent

to which individuals monitor their learning, as well as in the way they use the information

30

that monitoring provides. This variability in monitoring may result ﬁom any number of
differences between individuals--thus individual differences represent important sources
of variance that applied researchers must take into consideration.

Motivationally relevant individual differences may heavily inﬂuence the extent to
which individuals engage in metacognitive monitoring, as well as their reactions to and
utilization of this monitoring to inﬂuence the course of their learning. For example, when
monitoring reveals that one's current level of learning is below the desired level of
learning, some individuals may view this feedback as diagrostic, whereas others may
view this information as an indication of a lack of competence. This reaction, in tin-n,
may deterrrnine whether one utilizes this information productively, by allocating
additional study time, changing strategies, etc., or counterproductively, such as ignoring,
discontinuing monitoring of one's learrning, or physically or psychologically disengaging

ﬁom the task as a means of avoiding further negative feedback.

Metacogm'tive Interventions

Given the conceptual and empirical support for the beneﬁts of metacognition in
enhancing learning, research efforts have been aimed at enhancing metacognition during
learning. Much of this research has examined the utility of metacognitive interventions
for improving learrning outcomes among young children within academic settings. In a
study by Meloth (1990), 3rd-gade students were instructed in a manner intended to
improve their knowledge of cognition. The training resulted in modest changes in
students' knowledge of cognition over the course of a year, although this knowledge was

strongly associated with strategy use and reading comprehension. Short and Ryan (1984)

31

 

taught low-skill fourth-gade students a strategic plan (i.e. asking who, what, when,
where, and why) that was intended to aid the less-skilled readers in the monitoring of
their reading comprehension. As predicted, strategy training increased the poor readers
comprehension up to the level of the skilled readers (who did not receive strategy
training), as assessed by ﬁ'ee and probed recall. Also working with fourth-gade students,
Weed, Ryan, and Day (1990) provided some students with instructions on process
monitoring, which focused attention on interdependencies among the to be learned
material, on their use of strategies, and on how much of the material they were
remembering. Memory monitoring was signiﬁcantly related to post-test recall of
nonsense syllables and position probes. Payne and Manning (1992) taught 4th-gade
students comprehension mornitoring strategies for use during guided reading lessons.
Compared to a control goup, students receiving the comprehension mornitoring trairning
demonstrated geater reading comprehension, knowledge about reading strategies, and
more positive attitudes toward reading.

The efﬁcacy of metacognitive skills training has been examined with older
populations as well. Volet (1991) sought to develop students' metacognitive skills over
the course of a l3-week introductory computer science course by teaching a
metacognitive strategy tailored speciﬁcally to computer progamnning that included
planning, monitoring, and evaluation components. Of seven assessments of computing
performance, the trained goup only signiﬁcantly outperformed the control goup in their
ability to apply their knowledge to solve new problems. A signiﬁcantly geater number

of trained than control students passed a follow-up computing course.

32

Murphy, Schmitt, Caruso, and Sanders (1987) examined differences in
metarnemory in older and younger adults and its effects on recall of black and white
pictures. Young adults instructed to remember as many pictures as possible rehearsed
longer, recalled more, and spontaneously self-tested more than did older adults. Older
adults who were instructed to monitor their memory by self-testing prior to indicating
readiness to recall showed improved short-term recall and generalization as compared to
older adults instructed only to remember. Because no young adults were instructed to
monitor but rather were only instructed to remember as much as possible, it is not
possible to determine ﬁ'om this study whether self-testing instruction is beneﬁcial for
younger adults as well.

Among of the more successful metacognitive interventions have been those
teaching learners question generation strategies as a means of monitoring their
comprehension. Bean et al. (1986) examined the effects of question generation and
gaphic organization strategies on text recall of 10th-gade history students. The
comparison goup received instruction on outlirning, while two goups received
instruction on the use of gaphic organizers, with one of the two gaphic organizer goups
receiving previous training on question generation, which consisted of selecting or
creating statements to organize the text, writing questions based on these statements, and
evaluating the responses generated. No differences were found between the three goups
on the ﬁrst ﬁve of six lS-item quizzes. However, on quiz six and on written recall
protocols, the goup receiving both question-generation and gaphic organization training
signiﬁcantly outperformed the two other goups. Because tlnis goup also outperformed

the gaphic organizer only goup, this suggests that the locus of its effectiveness was in

33

 

the question-generation strategies rather than gaphic orgarnization, although a more
rigorous experimental design is in order to test this assumption.

In a series of studies, King and colleagues investigated the efﬁcacy of self-
questioning strategies for enhancing lecture and reading comprehension (King, 1989;
King, 1992; King & Rosenshine, 1993). The interventions consisted of an introduction to
the basic concepts and purposes of metacognition, followed by presentation of generic
question stems that require learners to ﬁll in the blanks (i.e. "How does... affect . .. ?").
After the instructor had taught and modeled the question generation strategy, students
were instructed to utilize these strategies during practice sessions, with the question stems
displayed on an overhead screen to reduce cognitive load. Students using these generic
question stems have generally been found to demonstrate geater lecture comprehension,
metacognitive process, strategy use, and knowledge structure (King, 1989; King, 1992;
King & Rosenshine, 1993) than controls, as well as students taught other learning
strategies such as note taking and summarizing. In one study with college students, tlnose
taught a surnmarization strategy recalled more than those taught question generation
immediately following the lecture (King, 1992). However, self-questioners outperformed
the summarizers on a retention test one week later.

A recent meta-analysis of self-questioning interventions provides further support
for the utility of generic question stems as a means of improving comprehension
(Rosenshine et al., 1996). Twenty-six question generation interventions were further
divided into ﬁve types: 1). those based on signal words, such as who, what, when, where,
why, and how, 2). those focusing on the identiﬁcation of main ideas, 3). those focusing

on the identiﬁcation of types of questions they were being asked, 4). those teaching

34

students to generate questions about key story gammar categories, such as setting and
main characters for reading comprehension, and 5). those utilizing generic question stems
that require learners to ﬁll in the blanks (i.e. "How are... and alike?"). While the
ﬁndings for main ideas, question types, and story gammar categories were somewhat
equivocal, signal words and generic question stems consistently led to large increases in
comprehension among average, below average, and above average students. These
interventions were particularly effective for college students, as opposed to younger
students. It is also worth noting that, at least among those interventions examined in this
meta-analysis, the most effective interventions were easy to learn and use, provided
guidance and focus, but did not demand strong cognitive skills or an extended period of
time to utilize effectively.

The list of studies attempting to train metacognitive skills outside the context of
the classroom is all too brief. Lorenc et a1. (1992) instructed 30 stroke patients to ask
themselves if they understood the information in each section immediately after reading
it. Those given the self-questioning instructions performed signiﬁcantly better on a
stroke knowledge test than did the control patients. Unfortunately, other attempts to
apply metacognitive interventions outside the classroom have been less successful.
Weissbein (1996) provided instruction on planning, monitoring, and evaluation of
learning aimed at enhancing trainees' learrning and performance on a radar-tracking
simulation. Contrary to the hypotheses, metacognitive training did not lead to geater
metacognitive activity, declarative or procedural knowledge, or task performance,
although post-hoe content analysis of metacognitive activity ratings revealed a

relationship between metacognition and many key study variables. Using the same radar-

35

tracking task, Smith (1996) provided instruction on what it means to plan, monitor, and
evaluate task strategies, coupled with three levels of discovery learning. While the
results with respect to discovery learrning were intriguing, metacognitive training only
had a signiﬁcant effect on verbal knowledge and adaptive transfer, and only when
coupled with a pure discovery learning environment. Finally, in another study using the
radar-tracking task, Brown et a1. (1997) provided subjects with strategic instruction,
which included choosing, evaluating, and modifying task strategies. However, Strategic
Instruction did not lead to improvement in learrning outcomes, as the only signiﬁcant
predictor of knowledge, strategy use, or task performance was cognitive ability.

Overall, interventions aimed at increasing metacognition among learners have
been relatively effective at increasing both metacognitive activity and learning outcomes
such as recall and test performance. This has been particularly well demonstrated among
young children in academic settings. Fewer studies have attempted to apply
.metacognitive interventions to adult learners. Those that have generally have concluded
that metacognitive interventions can be effective for adult learners as well as children
(Volet, 1991; Murphy et a1. 1987; King, 1992). Particularly pronnising are interventions
that facilitate self-questioning during learning (Rosenshine et al., 1996). Among such
self-questioning interventions, the use of generic question stems seems to be particularly
effective, resulting in an average increase of over one standard deviation in multiple-
choice and short-answer test performance across the four such studies included in the

Rosenshine et a1. meta-analysis.

36

Limitations of Metacognitive Intervention Research

Generalizability. With the exception of generic question stems, many of the
interventions reviewed above are often taught over time spans ranging from several
weeks to an entire academic year. Such extended time requirements for metacognitive
interventions reduce the feasibility of utilizing these interventions in employee training
progams. From an employee training perspective, the time required for metacognitive
interventions to be effectively implemented can have implications for both short and long
term gains in learning outcomes. In order for such interventions to have a positive impact
on learning during training and lead to geater transfer of training to the job, they must be
feasibly implemented within a reasonable time frame.

There are several potential explanations for the extended time required for most
metacognitive interventions. First, as already mentioned, most of these interventions take
place as part of other acaderrnic courses, which are generally relatively extended in time.
Because they are coupled with a long-term process of learning, brevity and time
efﬁciency of the interventions are less of a concern. Second, most metacognitive
interventions have sought to teach multiple components of metacognition, such as
planning, monitoring, and evaluation, within a single intervention. Such a holistic
approach to metacognitive training makes sense given the interrelatedness of the various
components of metacognition. However, as the complexity of the metacognitive
intervention increases, so too do the cognitive requirements to effectively acquire the
training. This translates into a ﬂatter learning curve for most learners, wherein a larger
period of time is required to effectively master and utilize the content of the intervention.

In contrast, some of the more effective interventions, such as generic question stems and

37

signal words, focus on a limited range of metacognitive phenomena. By simplifying and
better focusing the intervention, such interventions can be more quickly acquired and
assirrnilated into the learrning process.

Within the metacognitive intervention literature, few attempts have been made to
generalize the results beyond the classroom. Further, the results of those that have are
less positive than one might expect. However, it is too early to conclude that
metacognitive interventions are ineffective outside the academic context. Studies that
have attempted to enhance metacognition in non-academic settings have suffered from
several limitations that may, at least in part, account for the tepid ﬁndings. First,
Weissbein (1996), Snnitln (1996), and Brown et al. (1997) all attempted to train multiple
aspects of metacognition in a single intervention that was placed within a study lasting
four hours. Such short time periods may be insufﬁcient for trainees to acquire such
extensive metacognitive interventions. This may be compounded by the fact that subjects
were also attempting to learn and master a complex radar-tracking task, a process in
which the metacognitive skills trairning was intended to assist. This may have represented °
a dual task load, as subjects were expected to acquire both the metacognitive skills and
the skills necessary for effective performance of the radar-tracking task. If one assumes a
limited pool of cognitive resources, such a dual load may have exceeded subjects'
available cognitive resources. Under such an assumption, acquisition of metacognitive
skills, task-speciﬁc skills, or both are likely to suffer. Given that the radar-tracking task
was the most salient and emphasized aspect of the study, and in some cases was explicitly
rewarded monetarily (Brown et al., 1997), it seems likely that subjects would focus their

available resources on the task itself. Because tlnose with geater cognitive ability are

38

assumed to have a geater pool of cognitive resources (Kanfer & Ackerrnan, 1989), a
reasonable expectation resulting from this discussion would be an interaction between
cognitive ability and metacognitive training, such that those with geater cognitive ability
would more successfully acquire the metacognitive training, leading to increased learning
outcomes. However, none of the three studies reported any test of this interaction.

Motivation. Another explanation for inconsistent ﬁndings across metacognitive
interventions centers on subjects' motivation to acquire and/or apply their metacognitive
skills to the task. Because metacognition is an effort intense process, those lacking
sufﬁcient motivation will likely fail to acquire the metacognitive skills taught in the
interventions, or apply the skills that they have. Trainee characteristics may play a key
role in affecting the extent to which trainees acquire or utilize the knowledge and skills
targeted by the interventions.

For example, self-efﬁcacy has been posited to play a role in determining the
' extent to which individuals engage in metacognitive activity (e. g. Paris & Winogad,
1990). Those who believe they are incapable of acquiring the training content may have
little motivation to learn, much less monitor and regulate their learrning. Other constructs,
such as goal orientations and conscientiousness, may have an equally important role in
the extent to which individuals engage in metacognitive activity. Given insufﬁcient
motivation to engage in metacognitive activities, interventions aimed at increasing this
activity will likely have little or no effect. Little research in metacognition has exarrnined
the moderating effect of motivational characteristics. While all tlnree of the radar-

tracking studies have included motivational variables such as mastery and performance

39

goal orientations, they have not reported test for interactions between these variables and
the metacognitive interventions.

Prompting vs. Training. An implicit distinction among metacognitive
interventions is the issue of training metacognitive skills as opposed to prompting their
use. This distinction reﬂects assumptions concenning the locus of metacognitive skill use
during learrning. Studies that have sought to train metacognitive skills are based on the
assumption that the learners lack key metacognitive skills, and that this lack of skill
underlies the failure to actively engage in metacognition. To remedy this situation, they
seek to train subjects in the use of metacognitive skills, after which they are expected to
utilize the skills in the acquisition of the target material. Rather than extensively train
learners in the use of metacognitive skills, other studies have simply prompted trainees as
to when to engage in particular skills. This reﬂects an assumption that, while trainees
may have the metacognitive skills in question, they will not utilize these skills effectively
unless given explicit structure and guidance for their use.

This distinction has both practical and conceptual implications. From a
conceptual standpoint, it is important to determine whether sub-optimal engagement in
metacognition is the result of a lack of skills, or a lack of the knowledge or motivation to
utilize these skills. If it is the former underlying these deﬁcits, than our attention is best
directed toward understanding and enhancing the fundamental strategies and skills
underlying metacognition. If it is the latter, attention should be shifted to understanding
the factors that inﬂuence the use of the metacognitive skills that learners possess.
Metacognitive skills training will be ineffective if individuals do not use the skills that

they acquire. In all likelihood, some combination of krnowledge of metacognitive skill

40

and the use of existing metacognitive skills underlies the eventual use of metacognition
during learning. Understanding the nature of this relationship is critical to advancing our
understanding of metacognition at large.

The practical implications of this distinction are more obvious but no less critical.
If utilization of metacognition hinges on the knowledge of the key skills involved, than
interventions aimed at reaping the beneﬁts of metacognition would do best to work
toward increasing the learners' knowledge of these skills. Such training may be
extensive, or may be more focused in nature, but the aim would be to increase learners
understanding of various metacognitive strategies and skills that will have a positive
inﬂuence on learning outcomes. In contrast, if it is an issue of skill use rather than a lack
of the skill itself, than metacognitive interventions would be most effective if they focus
on encouraging learners to use their skills at appropriate points during learrning.
Prompting metacognition presents the most direct means of achieving this goal.

As stated above, it is most likely the case that metacognitive skill use hinges on a
combination of possessing the necessary skills and a willingness to use tlnese skills
effectively. This suggests that some combination of metacognitive skill training and
prompting would provide the most beneﬁcial results, as such an intervention would
increase the skills and provide structure and guidance for their use. However, depending
upon the type of prompts used, the skills themselves may play less of a role. For example
if the prompts are very directive the learner may not need to know the skills underlying
the prompt. Simply by following the prompt, the learner may acquire the necessary
understanding of their present state of knowledge and may be directed in the use of

optimal learning strategies. However, such prompting without knowledge of the

41

metacognitive skills and strategies underlying them may not lead one to utilize

metacognition when not explicitly prompted.

Model and Hypotheses

The literature review above identiﬁed a number of limitations of existing theory
and research on metacognition and metacognitive interventions. Although it is not
feasible to address all these limitations, this study attempts to address several of the key
limitations of this research. The ﬁrst limitation concerns the generalizability of the
ﬁndings to organizational training and various learrning outcomes. While most studies of
metarnemory have relatively simple tasks, such as word pairs, this study attempts to
apply principles derived from this literature to a complex training task similar to those
utilized for organizational training. The generalizability of metarnemory ﬁndings is also
uncertain due to reliance on relatively simple criteria, such as immediate tests of recall
and/or recognition of word lists. The present study evaluates training effectiveness by
utilizing knowledge-based, skill-based, and affective criteria, consistent with Kraiger,
Ford, and Salas' (1993) taxonomy of trairning outcomes.

Questions of generalizability plague studies of metacognitive interventions as
well. The vast majority of metacognitive interventions have been applied to and
evaluated on children in educational settings. The present study explores the
effectiveness of metacognitive interventions for improving adult trainees' mastery of non-
academic content. Previous studies attempting to generalize to this population and
context have achieved limited success. This may be, in part, due their reliance on

extensive interventions that attempt to teach several components of metacognition

42

concurrently. Given the limited time and attentional resources present in brief training
progams (i.e. 3-4 hour) such as those utilized in these studies, trainees may not be able to
acquire and apply extensive interventions of this sort in addition to the primary training
content. This issue is addressed by providing simpliﬁed, focused interventions aimed at
improving metacognitive monitoring rather than the broad range of metacognitive
activity utilized in previous studies.

Anotlner category of limitations to be addressed in this study is the limited
consideration of the role of motivationally relevant trainee characteristics in
metacognition. Studies of metarnemory have largely ignored the impact of individual
difference or motivational variables, instead choosing to rrninirnize this source of
variance. Studies of metacognitive interventions have likewise given limited
consideration to the role of motivation. This study attempts to ﬁll this gap by examining
the role of mastery goal orientation, self-efﬁcacy, and conscientiousness in the success of
the metacognitive interventions.

A ﬁnal limitation to be addressed concerrns the implicit distinction among
metacognitive interventions between training and prompting. This distinction reﬂects
underlying assumptions regarding whether sub-optimal metacognition during training is
the result of a lack of metacognitive skills, a failure to utilize these skills, or some
combination of the two. This distinction may underlie some of the inconsistent results
associated with metacognitive interventions. This study will address this question by
examining two separate interventions aimed at training or prompting the use of

metacognitive skills.

43

These limitations serve as the impetus for the present study. Next, a heuristic
model is presented incorporating the relevant study variables and speciﬁc hypotheses are

described.

Overview

Baldwin and Ford (198 8) have generated a framework for understanding factors
affecting learning and transfer of training. Three general categories of training inputs
identiﬁed as critical for learning, retention, generalization, and maintenance of training
were trainee characteristics, training design, and the work environment. Trainee
characteristics are ability, skill, motivation, and personality factors that inﬂuence the
learning process and subsequent transfer of training to the job. Training design factors
refer to the learning principles embedded in the training, the sequencing of training, and
the content of training itself. Work environment factors are those related to the
conditions under which one is expected to apply the KSAs acquired in training and
includes factors such as support and opportunity to perform trained skills. Work
environment characteristics are beyond the scope of this study, which is concerned with
factors that come into play during the training itself to affect learning outcomes.

Figure 1 presents a model for understanding factors inﬂuencing metacognition as
well as the outcomes of metacognition, based on an input-process-output framework.
Consistent with Baldwin and Ford (1988), the input factors include trainee characteristics
and training design factors. These inputs are assumed to inﬂuence the learning process,
with the focal process in this study being the extent to which the trainee engages in

metacognitive activity. These learrning processes should lead to learrning outcomes,

.0008 3330.80 ._ onE

 

 

mmoc-
30.20.0300

 

 

 

 

 

 

 

 

  

 

 

 

 

 

     

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

. ,
50050.28 >0moEm.20m
055818.... 05503-0...
0 08.... . . ,
0—: N: >030 .90... 0 a a3.— coaﬁgco _
005.528.. cosmetoton. A , .00 mm.
cos—8.39.. 5: 9.....2... cu: 8% , b: 0 s.
, 3.2.2
93.300205; 9.3.0;
3: 0000.305. N: ll , 0>...c.me020.2
9:90.000 0: -
05.0.5.5 .
93.300305.
mug—63:6 mommaochm

Swansea. unmanned wear—«0A 35000022

45

including knowledge, skills, and affect. These outcomes, in turn, should positively
impact performance on a task requiring application of the training content. In the section
that follows, each of these components will be discussed in detail, and speciﬁc

hypotheses will be posited regarding relationships among the model components.

11112831

Transfer has been deﬁned as the extent to which trainees apply the knowledge,
skills, and abilities acquired in training back to the job (Baldwin & Ford, 1988). Early
learning tlneorists held that transfer was predicated on the existence of identical elements
between two tasks (Thorndike & Woodworth, 1901), a position behaviorists have also
forwarded in the guise of stimulus-response pairings (Butterﬁeld & Nelson, 1989). This
approach led training designers to focus on how best to structure the learning
environment, in terms of reinforcement schedules, stimulus variability, etc. With the
cognitive revolution in psychology, such simplistic views of learning and transfer have
given way to geater focus on the role of the learner during the learning process.

For training to be considered successful, it is not enough that individuals acquire
the krnowledge, skills, affective outcomes targeted by the training-trainees must be able
to apply what they have learned to complex situations for which the training was
ultimately intended to address (Baldwin & Ford, 1988). Successful application of
training is contingent upon acquiring the basic declarative knowledge that underlies task
completion. Trainees must also possess the skill-based krnowledge required to effectively
and efficiently execute critical task functions. Finally, trainees must be able to cope with

and persevere through the increased complexity encountered when attempting to apply

46

what was leamed to a more complex situation. The current study operationalized and
examined transfer of training as the extent to which individuals were able to apply the
krnowledge and skills of the training progam in a complex task that mirrored the

situations to which the training was intended to apply.

Learning Outcomes

Kraiger, Ford, and Salas (1993) recently criticized I/O psychologists' tendency to
view learrning and transfer as a unidimensional construct, often focusing simply on
behavioral outcomes as an index of learning and transfer. Taking a construct-oriented
approach to learning, they drew from a broad base of literature from cognitive,
instructional, and industrial/organizational psychology to develop a multi-dimensional
and conceptually based classiﬁcation scheme of learrning, consisting of cognitive, skill-
based, and affective outcomes. Examples of cognitive outcomes include
verbal/declarative knowledge, knowledge organization, and cognitive strategies. Skill-
based outcomes include skill compilation and automaticity. Finally, examples of
affective outcomes include attitudes and motivational outcomes such as self-efﬁcacy.

Examples of research taking just such a multi-dimensional approach to learning
and transfer have recently begun to emerge. For example, Kozlowski et a1. (1995)
examined a model linking individual differences and training design features to a variety
of learning outcomes as well as adaptive transfer. It was found that declarative
knowledge, training performance, and self-efﬁcacy all added independent prediction of
performance on an adaptive transfer scenario. Ford et a1. (1998) also found that

declarative knowledge, training performance, and self-efﬁcacy were positively related to

47

transfer performance in a learner control training environment. In this study, it is
expected that the training outcomes of declarative knowledge, training performance, and
self-efﬁcacy will lead to geater transfer performance.

Declarative knowledge refers to knowledge concerning the "what" of training and
serves as a foundation for further high-order cognitive development (Anderson, 1982).
Many theories of knowledge acquisition posit that declarative knowledge is most
sensitive during the early stages of knowledge acquisition and must be attained before
higher-level development can occur (e. g. Ackerman, 1987; Anderson, 1982; Kraiger,
Ford, & Salas, 1993). Individuals with geater knowledge and understanding of the
training content should be better able to apply those skills. Said differently, without the
requisite declarative knowledge, individuals will have little to apply to the transfer task.
Models of transfer of trairning often posit knowledge-based outcomes as mediating the
relationship between training and transfer (e.g. Baldwin & Ford, 1988). Thus, knowledge
of the training content is expected to be positively related to performance on the
application task.

Hymthesis 1a. Declarative knowledge will be positively related to transfer of
training.

During irnitial skill acquisition, performance tends to be slow, error-prone, and
resource-dependent (Weiss, 1990). However, as skill development progesses,
performance becomes faster, less error-prone, and approaches automaticity, freeing up
cognitive resources which can then be directed toward additional tasks or to the
development of more elaborate task strategies. In addition, during the latter stages of

skill development, learners become more adept at determining the appropriate situations

48

for skill use (Gagne & Medsker, 1996) and adapting their skills when necessary. Thus,
the skill-based outcome of training performance is a valuable index of learning, which is
expected to be related to success on the transfer task.

Hymthesis lb. T raining performance will be positively related to performance
on the application task:

Self-efﬁcacy has been deﬁned as an individual’s belief in their ability to perform
a speciﬁc task (Bandura, 1977). Self-efﬁcacy has been posited to play a key role in the
self-regulatory process (e. g. Bandura, 1991). Those with high self-efﬁcacy generally set
higher goals and demonstrate geater persistence than do those with low self-efﬁcacy
(Bandura & Cervone, 1986). The persistence resulting from high self-efﬁcacy may be
critical in transfer, as the geater complexity that often accomparnies transfer situations
introduces new challenges and difﬁculties that must be overcome to succeed. In the
absence of such efficacy beliefs, the learner is likely to cognitively or behaviorally
withdraw ﬁ‘om the task, with resulting decrements in performance. Kraiger et al. (1993)
highlighted research indicating the criticality of self-efﬁcacy beliefs for the application of
knowledge and skills. Thus, self-efﬁcacy is expected to demonstrate a positive
relationship with application performance.

Hymthesis 1c. Post-training self-eﬁ’icacy will be positively related to

performance on the application task

49

Le_arning Processes

A large body of research provides conceptual, along with empirical support for
the key causal role of metacognitive processes in learrning (e. g. Durnlosky & Connor,
1997; Mazzoni, Comoldi, & Marchitelli, 1990; Nelson & Leonesio, 1988).
Metacognitive monitoring allows the learner to gain awareness of their current state of
knowledge and the effectiveness of their learning strategies. Self-generation of feedback
may be particularly critical in learner control environments, where little external structure
or feedback is given to guide the learner. Rather, the learner is charged with progessing
through the trairning content in whatever fashion they see ﬁt. By engaging in
metacognitive monitoring, learners can then make informed decisions about how to best
progess in their learning.

The learrning processes of metacognitive activity and study time are expected to
inﬂuence the learning outcomes declarative knowledge, training performance, and post-
“task self-efﬁcacy. Meloth (1990) found that students trained in the use of comprehension
strategies demonstrated geater reading comprehension than did control students. King
(1989) found that students who utilized metacognitive strategies to guide their learning
performed better on measures of declarative knowledge of lecture content. Further,
research in cognitive psychology has demonstrated that those who utilize the feedback
gained through monitoring by allocating additional study time to less-well krnown
material demonstrated geater recall of the training content (Nelson, Dunlosky, Graf, &
Narens, 1994). Therefore, learning processes are expected to inﬂuence the acquisition of

declarative knowledge.

50

Hypothesis 2a. Learning processes will be positively related to declarative
knowledge.

By engaging in learrning processes, individuals are expected to develop a better
understanding of how to apply the skills being trained. Volet (1991) found that
individuals trained in the use of metacognitive skills in a computer progarnming course
demonstrated signiﬁcantly geater computing performance. Thus, individuals who
engage in more learning processes should demonstrate better training performance.

Hypothesis 2b. Learning processes will be positively related to skill-based
training performance.

Ford et al. (1998) recently examined the relationship between metacognition and
self-efﬁcacy, with self-efficacy conceptualized as a learning outcome inﬂuenced by
metacognition. Consistent with their hypotheses, the results indicated a .38 correlation
between metacognition and self-efﬁcacy. Like Ford et al., Winne (1995) noted that
positive feelings of efﬁcacy will likely result from monitoring one's comprehension.

Hypothesis 2c. Learning processes will be positively related to post-training self-

eﬁicacy.

It is expected that metacognitive activity will positively inﬂuence the total time
spent on the training material. Those who engage in geater metacognitive activity are
expected to allocate more study time to material judged to be less well learned. Because
the process of monitoring one's knowledge, along with the allocation of geater study
time to less well krnown material are time consuming processes, it is expected that
metacognitive monitoring will result in more total time spent on the training material.

Hypothesis 3. Metacognitive activity will be positively related to study time.

51

Trainee Characteristics
Goal Orientation

A motivational construct with relevance for metacognition is mastery goal
orientation. A high mastery goal orientation is associated with developing competence
relative to oneself (Dweck, 1986), belief that ability is dependent upon effort, a
preference for challenging tasks, setting of difﬁcult goals for themselves (Bandura &

J ourden, 1991), persistence when faced with challenges, and a tendency to view feedback
as diagnostic rather than evaluative. Mastery orientation contains elements that can be
classiﬁed as metacognitive knowledge, which Flavell deﬁned as knowledge or beliefs
about yourself or others as cognitive processors. For example, a mastery orientation is
associated an incremental view of intelligence, a belief that intelligence is not ﬁxed, but
rather can be improved through effort. An outcome of this belief is that mastery
orientation is generally associated with a willingness to devote geater effort in the face
of challenges.

Those with a high mastery orientation should be willing to devote the effort
necessary to monitor their krnowledge, as effort is believed to lead to eventual gains in
knowledge and skills, which such individuals view as the end goal, rather than task
performance per se. Furtlner, a high mastery orientation should lead one to view the
outcomes of metacognitive monitoring as diagnostic of their current state of knowledge
and its relationship to the desired state of knowledge. When monitoring reveals large
discrepancies between current and desired states of knowledge, self-efﬁcacy should be

relatively unaffected, as should ﬁrture engagement in monitoring.

52

Some support for these propositions can be found in the existing metacognitive
literature. Schraw et al. (1995) found that those high on mastery orientation reported
geater strategy use and metacognitive awareness, which consisted of krnowledge of
cognition and regulation of cognition. Ford et a1. (1998) also examined the relationship
between goal orientations and metacognition in a trairning environment characterized by
high learner control. As hypothesized, mastery orientation was positively correlated with
a measure of metacognition, which reﬂected self-monitoring, planning, and evaluation of
progess. Wolters (1998) also found a positive relationship between mastery goal
orientation and both metacognition and use of speciﬁc learning strategies.

Hypothesis 4a. Mastery goal orientation will be positively related to learning
processes.

Self-Efﬁcacy
While post-training self-efﬁcacy is hypothesized to be inﬂuenced by
.metacognition, research suggests that pre-training self-efﬁcacy may also inﬂuence self-
regulated learning activities. Schunk (1995) has suggested that self-efﬁcacy has effects
on self-regulated learning activities, independent of problem solving skill. Those with
high self-efﬁcacy may utilize self-regulatory skills, even if their skills are deﬁcient or
otherwise faulty. At the same time, tlnose with low efﬁcacy beliefs may minimize self-
regulatory learning activities even if their skills are, irn fact, effective. Palmer and Goetz
(1988) have likewise proposed that students who perceive themselves as incompetent
may be less prone to engage in cognitive strategies. Both control theory (Carver &
Scheier, 1982; Kleirn, 1989) and social cognitive tlneory (Bandura, 1991) of self-

regulation indicate that a lack of efﬁcacy should lead to behavioral or mental

53

disengagement from the task. Monitoring is likely discontinued as a result of such
disengagement.

Additional support for the causal role of self-efﬁcacy in metacognitive monitoring
comes from research on feedback seeking. Feedback seeking research is generally
concerned with the seeking of external feedback, yet many of the ﬁndings can be
expected to generalize to seeking of self-generated feedback, such as that provided by
metacognitive monitoring. Miller and J ablin (1991) found that trainees with low self-
efﬁcacy engage in fewer feedback seeking behaviors in an attempt to avoid negative self-
evaluations.

Hypothesis 4b. Pre-training self-eﬁicacy will be positively related to learning
processes.

Conscientiousness

Conscientiousness has recently emerged as a valuable personality characteristic
that has been linked to a wide variety of organizational outcomes. A member of the Big
Five taxonomy of personality (Barrick & Mount, 1991; McCrea & Costa, 1987),
conscientiousness has been deﬁned as a trait reﬂecting responsibility, dependability,
persistence, self-discipline, hard working, and an achievement orientation. Despite early
pessimism regarding the link between personality and organizational outcomes, recent
meta-analytic evidence suggests that conscientiousness may be a signiﬁcant predictor of
nearly all performance criteria for nearly all occupational goups (Barrick & Mount,
1991). Drawing on over 30 years of research, resulting in over 130 criterion-related
validity studies including conscientiousness, Barrick and Mount found that

conscientiousness was positively correlated with the criteria composite.

54

Conscientiousness has been found to be inﬂuential in learning contexts. For .
example, Barrick and Mount (1991) found that conscientiousness was positively
correlated with training performance. Colquitte and Simmering (1998) found that
conscientiousness was related to motivation to learn during a 6-week undergaduate
management course. These ﬁndings are not surprising, given the nature of
conscientiousness and the demands of training. In order to succeed in training, it is
important for trainees to be willing to put forth the effort needed to acquire the material at
hand. Under simple trairning tasks, a small amount of effort may be sufﬁcient to ensure
success. However, when the training content is relatively complex, a sizeable
expenditure of effort is necessary on the part of the learner in order to succeed. The
persistence that is assumed to accompany high conscientiousness also should lead one to
maintain a high level of effort throughout training, even when confronted with
difﬁculties. Additionally, because conscientiousness is thought to subsume the more
speciﬁc dimension of need for achievement, those high on conscientiousness are likely to
set challenging goals and remain committed to those goals (Banick et al., 1993). Further °
highlighting the motivational impact of conscientiousness, Mount & Barrick (1995)
found it to be more strongly related to criteria that reﬂect "will do" factors rather than
"can do" or ability factors. Based on the body of research on conscientiousness, it is
expected that conscientiousness will positively inﬂuence learning processes.

Hmthesis 4c. Conscientiousness will be positively related to learning processes.

55

Training Desigp

Metacognitive Monitoring Training

Interventions aimed at teaching learners effective metacognitive monitoring skills
have often been found to a have positive inﬂuence on botln the extent to which learners
engage in metacognitive activity and on a variety of learning outcomes (e.g. Rosenslnine
et al., 1996). Particularly promising are interventions that are based on self-questioning
strategies. Self-questioning allows learners to gain insight as to their current level of
knowledge and understanding. This knowledge can then be compared to the goal state to
detect any discrepancies, and one can adjust their approach to reduce any discrepancy
detected.

Generic question stems, such as those developed by King and colleagues, have
been demonstrated to be effective for enhancing metacognitive strategy use, as well as
lecture and reading comprehension (King, 1989; King, 1992; King & Rosenshine, 1993).
Generic question stems encourage the learner to ask questions that probe their
understanding of a variety of aspects of the training content, such as main ideas, strengths
and weaknesses, relations between concepts, etc. In addition to the support for their
effectiveness, generic question stems have the advantage of being relatively easy to learn
and use, and do not appear to require strong cognitive skills.

The process of self-questioning ties in many aspects of monitoring addressed by
cognitive researchers reviewed above. Therefore, consultation of this research may help
to develop more effective metacognitive monitoring interventions by embedding
principles derived ﬁom this research. A major ﬁnding of research on nearly every

component of metacognitive monitoring is that learners tend to be overconﬁdent in their

56

estimates of their knowledge (e.g. Bomstien & Zickafoose, 1999; Koriat & Goldsmith,
1996). Due to their proposed inﬂuence on control, this overconﬁdence is likely to lead
trainees to make inaccurate or ineffective decisions concerning control of their learning.
From a self-regulatory perspective, if one believes they have attained the desired level of
learrning for a given training component, their attention may be shifted elsewhere. If this
decision is based on inaccurate information, learning and/or performance are likely to
suffer. By overestimating their knowledge or comprehension, learners are providing
themselves with false feedback, which may be detrimental to training effectiveness.

The metarnemory research has identiﬁed several sources of and potential
remedies to the overconﬁdence that pervades metacognitive monitoring. Bornstein &
Zickafoose (1999) found that simply informing subjects of the tendency to overestimate
their learning was sufﬁcient to improve the accuracy of their conﬁdence ratings.
Research on judgnents of learning (JOLs) has shown that J OL accuracy increases after
explicit attempts at retrieval (King, Zechmeister, & Shaughnessy, 1980). This ﬁnding
supports the beneﬁts of explicit self-testing for gauging one's level of comprehension,
rather than relying on more implicit judgnents concerning how well one knows the
information.

One of the more robust ﬁndings in the metarnemory literature is the delayed-JOL
effect. The accuracy of judgnents of learning increase dramatically when the judgnents
are delayed for a short period of time following item study. As the delay increases, J 01.3
approach perfect accuracy (Nelson, 1996). Similar results are obtained when item study
and J 01.8 are separated by a brief short-term memory distraction (Kelemen & Weaver,

1997). This clearly implicates STM contamination as a large source of inaccuracy in

57

metacognitive monitoring. Interventions that make trainees aware of and provide
strategies for circumventing this problem may geatly enhance the accuracy of
monitoring, resulting in ultimate gains in learning.

Hypothesis 5. Metacognitive training will be positively related to learning
processes.
Metacogjtive Prompting

Another approach for increasing metacognitive monitoring during learning is to
explicitly prompt its use. Rather than attempting to impart knowledge of metacognitive
monitoring skills, these interventions simply instruct trainees at various points during
training to assess their current level of learning. It has been indicated that individuals
often fail to utilize existing self-regulatory skills (e.g. Brief & Hollenbeck, 1985; Garner,
1990; Ridley et al., 1992). Prompting the learner to engage in self-regulatory activity has
been shown to increase the utilization of existing skills (e.g. Owings et al., 1980).
Because metacognitive prompting provides explicit structure and guidance for the use of
metacognitive monitoring, this represents a "strong" situation, as compared to no
prompting of monitoring (Bern & Allen, 1974; Weiss & Adler, 1984). That is, explicit
prompting imposes considerable demands or pressure on trainees to conform by enacting
the behaviors requested by the prompts, overcoming the tendency to not use
metacognitive skills. Thus, by cuing the use of metacognitive monitoring, metacognitive
prompting should yield many of the same beneﬁts as metacognitive training with respect
to utilization of metacognitive monitoring and ultimate effects on learning outcomes.

Hypothesis 6. Prompting metacognitive monitoring will be positively related to

learning processes.

58

Metacognitive Training X Motivation Interaction

Due to the high degee of learner control over the ultimate use of metacognitive
skills conveyed in monitoring training, there is likely to be geat variability in the extent
to which these skills are ultimately utilized during training. In many respects, a high
degee of learner control represents a relatively "weak" situation, when compared to more
formalized, instructor-led training. Relative to strong situations, weak situations provide
fewer demands or pressures to conform to given standards (Mischel, 1977).
Metacognitive training, in the absence of further prompting of its use, provides irnitial
instruction, but acquisition and implementation of these skills during subsequent training
remains the responsibility of the learners. Therefore, individual differences in motivation
are likely to play a key role in the use of trained skills and the ultimate success of the
monitoring training. It is expected that the motivational variables of self-efﬁcacy, goal
orientations, and conscientiousness will moderate the efﬁcacy of metacognitive training.

Hypothesis 7. The relationship between metacognitive training and learning

processes will be moderated by trainee characteristics (see Figure 2).

 

 

 

 

 

High Motivation‘
0.
E

3 _ _ _ __ .9 Low Motivation"

9 —————
‘ Mastery Goal onerIation. Sel-
Etllcecy. a Consderliousness
Absert Present

Metacognitive Training

Figure 2. Predicted Interaction of Metacognitive Training and Motivation
Variables on Learning Processes

59

METHOD

This study utilized a 2 X 2 fully crossed factorial design. The ﬁrst factor
consisted of metacognitive training vs. no metacognitive training. The second factor
consisted of metacognitive prompting vs. no metacognitive prompting. Thus, one goup
received only the metacognitive training, one goup received only the metacognitive
prompting, one goup received both training and prompting, and one goup served as the

control by receiving neither intervention.

Participants

Participants were undergaduate students at Michigan State University enrolled in
psychology courses who received course credit in return for participation in the
experiment. A power analysis indicated the need for approximately 144 participants.
Although few effect sizes are available for many of the relationships to be tested in this
study, the few that exist indicated the need for sufﬁcient power to detect medium effect
Sizes.

For example, Ford et al. (1998) found a .22 correlation between mastery
orientation and metacognitive activity, while Wolters (1998) found a moderate
relationship between learning orientation and metacognition (r = .45). Pintrich and
DeGroot (1990) reported a strong correlation between self-efﬁcacy and measures of
strategy use (r = .63) and self-regulated learning (r = .73). King (1992) found a moderate
effect (r = .34) for self-questioning training on strategic learning processes, while Payne

and Manning (1992) found a large effect (r = .67) with a similar comprehension

60

monitoring intervention. Ford et al. reported correlations of .29, .36, and .38 for the
relationship between metacognitive activity and declarative knowledge, training
performance, and self-efficacy, respectively. Ford et al. found moderate relationships
between declarative knowledge, training performance, and self-efﬁcacy on transfer
performance (r = .44, .29, and .39, respectively), while Bell (1999) found correlations of
.28, .34, and .40 for knowledge, performance, and self-efﬁcacy on generalization
performance.

Cohen (1992) suggests a sample size of 68 to detect a moderate correlation with
power of .80 and alpha of .05. A sample size of 107 is needed for a multiple regession
with 8 predictors and a medium effect size. Cohen states that a sample size of 36 per
goup is necessary to detect a medium effect with a four-goup factorial design. Thus,
the estimated total sample size necessary to obtain adequate power (.80) to detect the
hypothesized effects is 144 participants.

One hundred seventy-two individuals participated in this experiment. However,
two participants were excluded ﬁ'om analyses because they failed to complete the
experiment. Therefore, the sample used in this study included 170 participants, with cell

sizes ranging from 41 to 44 individuals.

Procedure
Participants completed the experiment in a university computer laboratory.
Participants were provided an intemet address and completed the experiment on a
computer equipped with the Netscape Communicator 4.6 software suite, which included

the Navigator web browser and Composer web-page design software. Upon entering the

61

intemet address, participants read instructions for completing the experiment. These
instructions included a brief description of the content and nature of the training progam.
Trainees were told that they would learn how to design their own intemet web page.
They were told that they would have the opportunity to practice what they had learned at
the conclusion of training. They were also told that at the conclusion of the training they
would take a quiz on the training material, as well as submit the intemet address for a
web page that they would create.

Next, participants read the informed consent information and indicated their
ageement to participate by entering their personal identiﬁcation number (PID) and
continuing with the trairning progam. Anyone who did not wish to participate was free to
discontinue the training progam at this point, or at any point during the training. The
consent form explained the nature and procedures of the experiment, the risks and
beneﬁts of the experiment, and their right to withdraw participation at any time without
penalty (Appendix A). Contact information was provided for the researcher, the
Psychology Department, and UCRIHS.

Participants tlnen completed several short measures of their individual
characteristics. Speciﬁcally, they answered questionnaires to establish their prior
experience with the intemet and creating web pages, self-efﬁcacy, mastery and
performance goal orientations, conscientiousness, metacognitive skills, and dernogaphic
information such as race, gender, and ACT/ SAT test scores, which as used as a proxy for
cognitive ability (Appendix B).

The web page training progam itself was presented to participants in seven major

topics, each of which was subdivided into smaller, more speciﬁc topics, each on a

62

separate web page (Appendix C). Participants were informed that their goal was to learn
the content of the training progam to the best of their abilities such that they can
successfully complete a series of exercises to be administered after training. They were
told that they were free to decide what information to study, in what sequence, and for
how long. Participants were not required by the task to access each page in the training
progam and were allowed to return to previously studied pages as often as they chose.
Upon termination of the training progam, the participants completed a series of
questionnaires (Appendix D). Speciﬁcally, they completed self-report measures of
metacognitive activity during training, and post-training self-efﬁcacy. They also
completed a multiple-choice quiz over the content of the web-page training.
Additionally, they completed a skill-based exercise, wherein they completed a number of
items requiring them to execute speciﬁc tasks using the web design software.
Participants were then provided instructions for a ﬁnal web page that they were to create.
They were asked to submit the intemet address for this web page. They then read a

debrieﬁng statement detailing the nature of the experiment.

Manipulations
Metacogrp'tive Training
The metacognitive training incorporated principles derived from cognitive and
educational research on metacognition intended to impart skills that would assist trainees
in more accurately monitoring their knowledge (Appendix E). Metacognitive training
(MCT') was presented at the beginnirng of the study--upon completion of the pre-training

questionnaires but prior to beginning the web page trairning proper. This training began

63

with a brief introduction to the concept of metacognition. Trainees were informed of the
importance of monitoring their own knowledge. They were also informed of the
tendency for learners to overestimate their level of knowledge and understanding, and the
negative impact this overestimation can have on leanning.

Participants were then provided with two speciﬁc strategies for improving the
accuracy of their monitoring, based on self-questioning (i.e. asking oneself questions
while progessing tlnrougln the trairning). The ﬁrst strategy was the use of ﬁll-in-the-blank
question stems, such as "how are _ and _ related?" Research by King and
colleagues (King, 1989; King, 1992; King & Rosenshine, 1993) found that the use of
these question stems increased metacognitive activity and learrning outcomes for a diverse
range of learners. The second strategy consisted of brieﬂy delaying self-questioning,
rather than asking oneself questions about a topic immediately after study. This strategy
was based on a large body of research demonstrating that delayed judgnents-of-leaming
(JOLs) are considerably more accurate than are immediate JOLs (e. g. Nelson &
Dunlosky, 1991), as the delay require that the answers to the self-questioning be retrieved
ﬁ'om long-term, rather than short-term memory. Finally, trainees were instructed to use
the information gained from these strategies to identify areas where they needed to spend
additional time studying. Participants in the MCT condition were also given a
performance aid (Appendix F) that summarized the metacognitive training, which they
could refer to at any point during the web page trairning. This performance aid was
provided in an effort to reduce the cognitive ability requirements needed to learn the
skills conveyed in the metacognitive training, thus putting the focus on the use rather than

the acquisition of metacognitive skills.

64

Metacognitive Prompting

In the second intervention, metacognitive prompting (Appendix G), trainees were
prompted at various points during the course of the web-page training to assess their
current level of understanding of the material they had covered up to that point. Because
these prompts were intended to encourage the use of existing metacognitive skills, rather
than instruct trainees on the use of new skills, the prompts were kept general, rather than
speciﬁc. The prompts consisted of a window that periodically appeared on the screen
asking trainees to take a few moments to reﬂect on the material they just studied and to
attempt to determine how well they had learned it. Trainees were required to respond to
two open-ended questions, one asking what information they had learned in previous
lessons and another asking what information they need to learn better. Further, they were
encouraged to spend additional time studying any material that they felt they did not ﬁrlly
understand. This window appeared in roughly 10-minute intervals, such that the screen
appeared upon clicking the ﬁrst link to another page after the 10-minute interval had
passed. During pre-testing, participants indicated that presenting the prompt every 10
minutes was often enough to encourage frequent reﬂection on the training material, but
not so often as to be distracting or disruptive. After trainees have viewed the prompt and
had responded to the two questions listed above, they were returned to the page they

initially selected.

Measures

Participants completed several survey measures at the beginning and at the end of

the training progam. The speciﬁc measures can be found in Appendices B and C.

65

Control Variables

Demogaphics and Cognitive Abilig

Demogaphic information was collected, including age, gender, race, and college
GPA. Participants were asked to provide their ACT or SAT test scores as an indicator of
cognitive ability.
Prior Expe_rience

Prior experience with computers, the intemet, and web progams were assessed
with 8 items. Participants responded on a ﬁve-point Likert scale ranging from "all the
time" (1) to "never" (5). These eight items were subjected to a common factor analysis
with varimax rotation. Examination of the resulting eigenvalues suggests a one-factor
solution. Coefﬁcient alpha for this scale was .79.
Pre-training Metacognitive Skills

Pre-training metacognitive skills were measured with a 20-item scale adapted
ﬁom Ford et a1. (1998) and Pintrich and DeGroot (1990). The internal consistency
reliability of the original Ford et a1. 12-item scale was .81. The original scale was
developed to assess the degee to which individuals typically engage in self-monitoring
of learning, choice of practice scenarios in order to address deﬁcits in learning or
performance, and self-evaluation of one's progess in learning contexts. The original
items were also targeted speciﬁcally at metacognition in the context of the radar-tracking
simulation used in that study. Thus, the scale was reworded to assess general tendencies
to engage in these activities while learning. Additionally, four items from the self-

regulation sub-scale of Pintrich and DeGroot's (1990) Motivated Strategies for Leanning

66

Questionnaire (MSLQ) were added. Individuals respond on a 5-point scale ranging from
"strongly disagee" (1) to "strongly agee" (5).

This scale was developed to assess pre-existing levels of the two primary
metacognitive constructs of interest in this study -- monitoring and control. The Speciﬁc
items in this scale are presented in Appendix B, along with headings reﬂecting the
intended dimensionality (these headings were not presented to participants). Based on
the conceptual development of the scale, a two-factor solution was expected. To
empirically examine the factor structure of the scale, a common factor analysis with
varimax rotation was performed on the 20 items. Figure 3 displays the scree plot for this
analysis. Despite the expectation of multidirnensionality, the scree plot indicates that a
single factor solution best ﬁts the data. Coefﬁcient alpha for this single factor scale was

.86.

 

 

Eigenvalue

 

 

1&545 6 i é a 1'01'11'21ﬁ31141'51'61'7181'92'0
Component Number

Figure 3. Scree Plot of Metacognitive Skill Measure

67

Motivation Variaﬂes

Mastery Goal Orientm

Goal orientations were measured with a 16-item scale by Button, et al. (1996),
which assess trait goal orientations. These scales treat mastery and performance
orientation as two distinct constructs. A sample item is "The opportunity to learn new
things is important to me." Individuals will respond on a 5-point scale ranging from
"strongly disagee" (1) to "strongly agee" (5). Previous research has consistently found
a two-factor solution, with reliability for both scales in the range of .75 - .80 (e. g. Button,
Mathieu, & Zajac, 1996; Ford et al., 1998). A common factor analysis with varimax
rotation performed on all 18 goal orientation items provided support for a two-factor
solution, with all items loading on the appropriate scale. Coefficient alpha was .80 and
.79 for the mastery orientation scale and performance orientation scale, respectively.
Pre-training Self-efﬁcacy

FTC-training self-efﬁcacy was assessed with a nine-item scale adapted from the
self-efﬁcacy sub-scale of Pintrich and DeGroot's (1990) Motivated Strategies for
Learning Questionnaire (MSLQ). The internal consistency reliability for the original
scale was .89. The original scale items were worded to assess self-efﬁcacy in a
classroom setting (ex. " I know that I will be able to learn the material for this class").
For use in this study, the items were re-worded to assess self-efﬁcacy for the trairning
course (ex. "1 know that I will be able to learn the material for this training course").
Individuals will respond on a 5-point scale ranging from "strongly disagee" (1) to
"strongly agee" (5). A common factor analysis revealed a one-factor solution.

Coefﬁcient alpha for tlnis scale was .92.

68

Conscientiousness

Conscientiousness was measured using the 12-item scale of the NBC F F I
personality inventory (Costa & McCrae, 1992). These items were designed to assess
characteristics of conscientiousness, including responsibility, dependability, work etlnic,
achievement orientedness, and perseverance. Previous research has found high internal
consistency and reliability for this scale (e.g. .90: Martocchio & Judge, 1997).
Individuals responded on a 5-point scale ranging from "strongly disagee" (l) to "strongly
agee" (5). Coefﬁcient alpha for all 12 items in this scale was .74. However,
examination of individual item-total correlations revealed that item 7 ("I seldom notice
the moods or feelings that different environments produce") correlated very weakly (r = -
.06) with the remaining items in the scale. Thus, the decision was made to remove this
item from the scale, resulting in a coefﬁcient alpha of .78.

- Learning Processes

Metacognitive Activities During Training

The extent to which trainees engaged in metacognitive activities during trairning
was assessed with a 20-item scale adapted from Ford et a1. (1998) and Pintrich and
DeGroot (1990). The internal consistency reliability of the original Ford et al. (1998) 12-
item scale was .81. The original scale items were targeted speciﬁcally at metacognition in
the context of the radar-tracking simulation used in that study. Thus, the scale was
reworded to assess the degee to which trainees engaged in these activities during the
web-page training. Five items from the self-regulation sub-scale of Pintrich and
DeGroot's (1990) Motivated Strategies for Learning Questionnaire (MSLQ) were added

and similarly rephrased to reference the web-page trairning progam. An additional four

69

items were added speciﬁcally for this study to assess strategic allocation of study time.
Individuals responded on a 5-point scale ranging from "strongly disagee" (l) to "strongly
agee" (5).

This scale was developed to assess metacognitive mornitoring and control as they
occurred during training. The speciﬁc items in this scale are presented in Appendix D,
along with headings reﬂecting the intended dimensionality (these headings were not
presented to participants). Based on the conceptual development of the scale, a two-
factor solution was expected. To empirically examine the factor structure of the scale, a
common factor analysis with varimax rotation was performed on the 20 items. Figure 4
displays the scree plot for this analysis. Despite the expectation of multidirnensionality,
the scree plot indicates that a single factor solution best ﬁts the data. Coefficient alpha for

this single factor scale was .87.

 

Eigenvalue

3

 

 

 

1

1 i 5 4 5' (5 i S 51b1'1 1'21'31'41'51'6131'81'920
Component Number

Figure 4. Scree Plot of Metacognitive Activity Measure

70

Study Time

Study time was represented by the amount of time trainees spend on the training
progam. A database recorded the amount of time elapsing between the start and
completion of the web trairning progam for each participant. This did not include time
spent on any of the pre- or post-training measures. Participants spent and average of

43.76 minutes on the trairning progam, with a standard deviation of 17.26 minutes.

Learrning Outcomes

Post-training Self-efﬁcacy

Post-training self-efﬁcacy was assessed with a ﬁve-item scale adapted from
Hollenbeck and Brief (1987). The internal consistency reliability for the original scale
was .82. The original scale items were developed to assess self-efﬁcacy on an anagam
creation task (ex. "It is just not possible for me to solve anagarns tasks at the level I
would like"). For use in this study, the items were re-worded to assess self-efﬁcacy for
the training course (ex. "It is just not possible for me to create web pages at the level I
would like"). Individuals responded on a 5-point scale ranging from "strongly disagee"
(l) to "strongly agee" (5). A common factor analysis revealed a one-factor solution.
Coefﬁcient alpha for this scale was .71.

WW

Upon completing the post-training self-efﬁcacy and declarative knowledge
measures, participants were presented with a web page containing a 17-item test. For
each item, trainees were given speciﬁc objectives to complete. Participants were asked to
edit the web page by attempting to execute the objectives described in each of the 17

items listed on the page. Some items contained multiple objectives; however, each

71

objective was independent, such that failure to complete one objective did not impair
one's ability to complete the remaining objectives, nor did successfully completing one
objective assist one in completing another. Participants were asked to spend no more
than 10 rrninutes on this measure. Revised web pages were saved as a unique ﬁle for each
participant.

Objectives were scored dichotomously, such that a score of one was given for
each objective that was successfully completed. Scores were summed across all items,
yielding a single measure of skill performance. The resulting range of scores is between
0 and 24. Due to technical problems, the skill-based learrning measure for 16 participants
could not be scored and these participants were given missing values for this measure.
The remaining participants successfully completed an average of 18.29 objectives, with a
standard deviation of 5.34. Coefﬁcient alpha for this measure was .91.

Declarative Knowledge

Declarative knowledge was assessed with a 37-item multiple-choice quiz with
eight items testing knowledge of each lesson. Responses were scored dichotomously,
such that one point was gained for each correct answer. The learning scores were
summed across all lessons to yield a single measure of declarative knowledge. Thus,
scores on the quiz can range ﬁom 0 to 37. Participants averaged 26.44ecorrect responses,
with a standard deviation of 4.15. Coefﬁcient alpha for this measure was .69.
Application Performance

Application performance was assessed at the conclusion of the study. At the
conclusion of the training progam, participants were required to develop their own web

page to demonstrate their knowledge of web page creation. Participants were informed

72

that the goal of this web page was to demonstrate what they had learned in the training
progam and, thus, they were encouraged to incorporate as much as they could of what
they had learned in the trairning progam. Participants were instructed to take no longer
than 45 minutes creating this ﬁnal web page. Web pages were scored based on the
number of elements ﬁ'om the training progam that were included in the page (see
Appendix D). Scores on this measure could range from O to 38. Due to technical
problems, the web pages for 16 participants could not be scored and these participants
were given missing values for this measure. The remaining participants successfully
incorporated an average of 16.09 elements, with a standard deviation of 4.61. Coefficient

alpha for this measure was .67.

Data Analysis Plan

The model presented in Figure 1 was tested with a series of hierarchical
regession analyses (Cohen & Cohen, 1983). The ﬁrst set of analyses examined the
impact of learning outcomes on transfer performance. The second set focused on the
inﬂuence of the learrning processes on learning outcomes. The third set examined the
inﬂuence metacognitive activity on the remaining learning processes. The fourth set
focused on the inﬂuence of the motivational variables on learning processes. The ﬁfth
set focused on the inﬂuence of the metacognitive interventions on learning processes.
The ﬁnal set of analyses examined the interaction between metacognitive trairning and
motivational characteristics on learning processes. Metacognitive training and
metacognitive prompting were represented by two dummy coded variables (1 for the

presence of the marnipulation and O for its absence).

73

RESULTS

Manipulation Checks

Upon completing training, all participants, regardless of experimental condition,
were asked a series of questions designed to assess whether they had attended to and
attempted to implement the interventions, if they were present. The ﬁrst question simply
asked participants to indicate whether they had or had not been exposed to the
intervention in question. Following this question, participants responded to 5 questions
concerning their motivation to implement the intervention.

Eighty-ﬁve of the 87 participants receiving metacognitive training correctly
indicated that they had received this training. One participant indicated that they had not
received the training, while another indicated that they were not sure whether they had
received the training or not. Among the 85 participants receiving metacognitive prompts,
78 correctly reported that they had received the prompts, 6 indicated that they had not
received the prompts, and 1 was not sure. Given the salience of the interventions, failure
to notice and correctly identify when one had received the intervention is likely due to
extreme inattentiveness on the part of these participants. Thus, the decision was made to
exclude these participants from all analyses.

With respect to implementation of the metacognitive training, participants
receiving the training indicated that they made some efforts to utilize the skills and
strategies they were taught. Table 1 presents the means for the individual items in the
metacognitive training manipulation check, as well as for the measure as a whole.

Examination of the table indicates that, although trainees indicated that they attempted to

74

Table 1. Marnipulation Checks

Metacogpitive Training

 

 

 

 

Question Mean SD
I tried to apply the learning strategies as I went through the web page 3.70 0.82
training.

I made an effort to use the strategies I was taught to help me learn to 3.63 0.74
make web pages.

I didn't attempt to use the skills I was taught when going through the 3.51 1.19
training progam. '

During training, I tried to use the ﬁll-in-the-blank questions that were 2.87 0.96
provided to me at the beginning of the study.

I felt that using the strategies I was taught would help me learn to 3.51 0.67
create web pages.

All Items 3.26 0.49
Metacogm'tive Prompting

Question Mean SD
When I received these instructions, I tried to tlnink carefully about 3.92 0.74
what I had learned.

When I received these instructions, I responded quickly and got back 3.64 0.87
to the training.

I tried to follow these instructions carefully. 3.87 0.50
I made an effort to use tlnese instructions to help me learn to make 3.77 0.69
web pages.

I felt that receiving these instructions would help me learn to create 3.70 0.88
web pages.

All Items 3,73 0.44

75

utilize the metacognitive training as a whole, there was relatively low use of the provided
question-stems.

Table 1 also presents the means and standard deviations for the individual items
in the metacognitive prompting manipulation check, as well as for the measure as a
whole. Examination of the table indicates that trainees attempted to follow and make use
of the prompts as instructed.

Correlations

Table 2 presents the descriptive statistics and correlations for the variables in this
study. Signiﬁcant correlations at p < .05 are presented in bold, while reliability estimates
are presented in parentheses on the diagonal. The correlation matrix provides basic
information regarding the relationships among the individual difference, learning process,
and outcome variables.

With regard to control variables, cognitive ability had a moderate relationship
with mastery goal orientation (r = .17) and declarative knowledge (r = .3 8), but no other
relationships with cognitive ability were signiﬁcant. As would be expected, prior
experience was positively related pre- and post-training self-efﬁcacy (r = .37 for each),
skill-based performance (r = .22), and application performance (r = .20), and was
negatively related to study time (r = -.20). Somewhat surprisingly, prior experience was
also positively related to metacognitive activity during training (r = .18).

The motivational constructs of pre-training self-efﬁcacy, mastery goal orientation,
and conscientiousness were largely unrelated. Although mastery goal orientation was
positively related to both pre-training self-efﬁcacy (r = .26) and conscientiousness (r =

.24), no other relationships among tlnese variables were signiﬁcant. The learning process

76

3. v a 8 “505%...” 08 9.0.8.230 comm—.825 ”202

 

l

 

8N. N.. 8.- 8 8.- 8.- 8.- 8... 8.2 8. 88885.. 88.8.... N.
N_. «In. E 8. 8.- 8. m. 88 8.8 8. 88.382 08.8.80 2
NI. 2. N. 8.- 8.- 8.- 8. :1. 8.8 m: 88885.. Ban-5m 8
H... o _. 8. : - 8. N.. 8. 8. 8N 8. 8889.28 means-8.. m.
8..” 8. N. Na 8. 8. .m. 8.8 8.8 8. .8356. 2E... 88 3
we 8. S. N. 8. 3. 8. 8. NZ 8. 838... 08.8882 2
8.- 8.- alN. 8 8. 2. 8.- 8. 88 .8 85:88.88 N.
NIN. 8. 8 - 3r... 8. 8.- 8. 8. N: .8 8880-28 means-2.. :
2. 2.- 8. 2 E .- 8.- 8. 9.. 8H .8 88.8.5 85.5888 o.
S. q q 8. 8. 8. 8. 3.. 8.8 8. 8.88:0 tea: a
g 8. o. 8. S. 8.- 8. 8. :.m 8. 8.8 288882 N
.2... 8.- 8 - 8.- 8.- 8. 8. 8. 8N 8. 88.88 BE N
35 ﬂ. 8 - :.- 8.- 8. 8. 8. N2 88.... 22:80 o
35 8. S. 8. 8.- NW. 8N 8. <80 m
3.5 8. 8.- z. 8. 8.. .8 880 4
3.5 EA 8. 8.. 8.2 G. ow... m
35 8. 8-. 8. 8. 8.589 2.28882 N
9.5 cm. 8. .8 8888.. 08.8882 .

N 8 n v N N _ 0m 822 z 28...;

 

8:83.0- 88880 .N 28...

77

 

 

 

 

.88.. 8. 88. I”. 88.- 88. 8..- .NM. 8. 8. 8.
.88.. mm .l." m. 8.. ﬂu N.. a 8.. 8.
:8. am 8.- 8.- 8.- VIN. 8.- 8.- .8.-
..8 8INd H.- 88. vlmu 8.- mﬂ 8..
88. 88. .8.- mﬁ 8.- 8. . 88.
.88 N.. mm 8.. mm w-w
$8 88. 8. g N..
.88 .8.- 8.. .NIN..
.88 8. 8..
.88.. .48..
G8,.
8. N. 8. 8. 8. 8. N. .. 8. 8 8

 

8.88. N 8.888

78

variables of metacognitive activity and study time did not exhibit a signiﬁcant zero-order
relationship (r = .08), although this relationship is examined more formally in the test of
hypothesis 3 reported below.

On the whole, the motivation and process variables were moderately correlated.
Two of the three motivational variables exhibited moderate correlations with
metacognitive activity, indicating that those who approached the training with a learning
focus (r = .33) and had conﬁdence in their ability to learn the material (r = .19) more
thoughtfully engaged in the training program. Only pre-training self-efﬁcacy was
directly related to study time (r = -.23), such that those who had greater efﬁcacy spent
less time studying.

The learning outcomes of post-training self-efﬁcacy, skill-based performance,
declarative knowledge, and application performance exhibited moderate to large
relationships with each other. Such relationships are not unexpected, as the outcome
constructs represent distinct but interrelated aspects of task mastery. However, the
magnitude of the relationship between declarative knowledge and skill-based
performance (r = .56) is a cause for potential concern, raising some questions as to the
distinctiveness of the underlying constructs tapped by these two measures.

Formal hypothesis testing is reported below, moving backwards through the
model. To make the hypothesis tests easier to follow and interpret, a summary of

hypotheses and the analyses used to test each is presented in Table 3.

79

Table 3. Analysis Summary

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Hyp FWS) lDV(s) lControl Analyses Supported?
Variables
la DK AP Exp, CA, PreSE No
HR . .
(umvanate effect)
1b SP AP Exp, CA, PreSE Yes
1c PostSE AP Exp, CA, PreSE No
HR (univariate effect)
2a 'MCA, ST DK Exp, CA Yes
(ST)
2b CA, ST SP Exp, CA HR No
2c lMCA, ST PostSE Exp, CA, PreSE HR Yes
3 [MCA ST Exp, CA HR Yes
4a 'MGo 'MCA, sr lExp, CA 2 HR Partial
(effect on MCA)
4b PreSE IMCA, ST lExp, CA 2 HR Partial
(effect on ST)
4c Cons 'MCA, ST Exp, CA 2 HR No
5 'MCT 'MCA, ST Exp, CA, MCS, 2 HR Partial
GO, PreSE, (effect on MCA)
gons
6 MCP 'MCA, ST E18, CA, MCS, 2 HR Partial
GO, PreSE, (effect on ST)
ons
7 MGO X MCT lMCA, ST xp, CA, MCS, 2 HR Partial
PreSE X MCT GO, PreSE, (Motivation X
Cons X MCT ons, MCT MCT on MCA:
carried by PreSE
X MCT)

 

DK = declarative knowledge; AP = Application Performance; SP = Skill-Based
Performance; PostSE = Post-Training Self-Efﬁcacy, MCA = Metacognitive Activity,
ST = Study Time; MGO = Mastery Goal Orientation; PreSE = Pre-Training Self-

Efﬁcacy, Cons = Conscientiousness; MCT = Metacognitive Training;

MCP = Metacognitive Prompting; Exp = experience; CA = Cognitive Ability,
HR = Hierarchical Regression

80

 

Hmthesis Tests

Hypotheses lale, & 1c: Lea_r;n_i_r_1g Outcomes on Application Performance.
Hypotheses 1a, lb, and lo, concerning the relationship between the learning outcomes of
declarative knowledge, training performance, and post-training self-efficacy on
application performance were tested with a single hierarchical regression. Although not
of conceptual concern in this study, cognitive ability and prior experience were entered in
the ﬁrst step as control variables. This was done due to their likely effects on all training
outcomes — it is important to demonstrate that the training outcomes have unique impacts
on application performance and, thus, that any relationships observed between the
learning outcomes and application performance do not simply reﬂect each drawing
similarly upon prior experience and cognitive ability. Additionally, because it is
important to demonstrate that any effects observed for post-training self-efﬁcacy are
unique and distinct from those contributed by pre-training self-efﬁcacy, pre-training self-
efﬁcacy was also included in the ﬁrst step of the regression. The variables of conceptual
interest in this analysis, declarative knowledge, training performance, and post-training
self-efﬁcacy, were entered in the second step of the regression. Table 4 presents the
results of the analysis.

Results indicated that cognitive ability and prior experience accounted for
signiﬁcant variance in application performance (R2 = .07, F = 4.97, p < .01). Beta-
weights indicated that prior experience was positively related to application performance
(B = .26, p < .01 at step 1). The second step of the regression, containing declarative
knowledge, skill-based performance, and post-training self-efﬁcacy, accounted for

signiﬁcant variance in application performance (AR2 = .11, AP = 5.10, p < .01),

81

supporting the most general hypothesis that learning outcomes will affect application
performance. Beta-weights indicated that skill-based training performance was positively
related to application performance (B = .26, p < .05 at step 2), supporting hypothesis 1b.
Hypothesis 1a proposed that declarative knowledge would be positively related to
application performance. Although declarative knowledge exhibited a signiﬁcant zero-
order correlation with application performance (r = .35, p < .01), this relationship became
non-signiﬁcant once the inﬂuence of the other variables in the mOdel were controlled.
Post-training self-efﬁcacy also exhibited a signiﬁcant zero-order correlation with
application performance (r = .19, p < .01), but the relationship became non-signiﬁcant
when the inﬂuence of the other variables in the model were controlled. Thus, these results
support hypothesis 1b, but did not support hypotheses 1a or 1c. Overall, the variables in

the regression accounted for 17 percent of the variance in application performance

(135.125) = 5.27, p < .01).

Table 4. Hierarchical Regression Analysis Results for Hypothesis 1

 

 

Step Predictors B at Step Final B R2 AR2
1 ‘ Ability .10 .02 .10" .10"
Prior Experience .19 .13
Pre-training Self-Efﬁcacy .16 .10
2 Declarative Knowledge .12 .12 .19M .09"
Skill-based Performance .25* .25“
Post-Training Self-Efﬁcacy .02 .02

 

n=125;*p<.05;**p<.01

82

Hymtheses 2a, 2b, & 2c: Learning Processes on Learning Outcomes. Hypotheses

2a, 2b, and 2c concerning the relationship between learning processes (metacognitive
activity and total study time) and the three learning outcomes of declarative knowledge,
training performance, and post-training self-efﬁcacy were tested with a set of three
hierarchical regressions. The three regressions differed only with respect to the DV
(HZa: declarative knowledge; H2b: training performance; H2c: post-training self-
efﬁcacy). In each of the three regressions, the covariates cognitive ability and prior
experience were entered in the ﬁrst step to control for their impact on the dependent

variables. The two learning processes were entered in the second step.

Table 5. Hierarchical Regression Analysis Results for Hypothesis 2a

 

 

Step Predictors J3 at Stem Final B R2 AR2

1 Ability .40" .39“ .18" .18“
Prior Experience .17* .23"

2 Metacognitive Activity -.01 .01 .27** .09“
Study Time .3-1** .31 **

 

n=150;*p<.05;**p<.01

Hypothesis 2a predicted that learning processes would be positively related to
declarative knowledge. Table 5 presents the results of this analysis. Results indicated
that cognitive ability and prior experience accounted for signiﬁcant variance in
declarative knowledge (R2 = .18, F = 15.75, p < .01). Beta-weights revealed that
cognitive ability (B = .40, p < .01 at step 1) and prior experience (B = .17, p < .05 at step

1) were positively related to declarative knowledge. The second step of the regression,

83

including metacognitive activity and study time, accounted for signiﬁcant variance in
declarative knowledge (AR2 = .09, AF = 8.84, p < .01), indicating support for hypothesis
2a. Examination of the beta-weights found that study time was the only signiﬁcant
parameter, demonstrating a positive relationship with declarative knowledge (B = .31, p <
.01 at step 2). Thus, hypothesis 2a was partially supported.

Hypothesis 2b predicted that learning processes would be positively related to
skill-based performance. Table 6 presents the results of this analysis. Results indicated
that cognitive ability and prior experience accounted for signiﬁcant variance in skill-
based performance (R2 = .09, F = 6.96, p < .01). Beta-weights revealed that cognitive
ability (B = .18, p < .05 at step 1) and experience (B = .28, p < .01 at step 1) were
positively related to skill-based performance. The second step of the regression,
including metacognitive activity, and study time, did not account for signiﬁcant
incremental variance in skill-based performance (AR2 = .01, AF = 0.69, p > .05). Neither
of the variables in the second step demonstrated signiﬁcant beta-weights or zero-order

correlations with skill-based performance. Thus, hypothesis 2b was not supported.

Table 6. Hierarchical Regression Analysis Results for Hypothesis 2b

 

 

Step Predictors B at Step Final B R2 AR2

1 Ability .18* .18* .09" .09"
Prior Experience .28" .31 **

2 Metacognitive Activity -.10 -.10 .10 .01
Study Time .05 .05

 

n=135;*p<.05;**p<.01

84

Hypothesis 2c predicted that learning processes would be positively related to
post-training self-efﬁcacy. Table 7 presents the results of this analysis. Results indicated
that cognitive ability and prior experience accounted for signiﬁcant variance in post-
training self-efﬁcacy (R2 = .14, F = 12.37, p < .01). Beta-weights revealed that
experience (B = .35, p < .01 at step 1) was positively related to post-training self-efﬁcacy.
The second step of the regression, including metacognitive activity, and study time,
accounted for signiﬁcant incremental variance in post-training self-efﬁcacy (AR2 = .08,
AP = 7.14, p < .01), indicating support for hypothesis 2c. Examination of the type-three
beta-weights found that study time (B = -.17, p < .05 at step 2) was negatively related to
post-training self-efﬁcacy, whereas metacognitive activity was positively related to post-

training self-efﬁcacy (B = .22, p < .01 at step 2). Thus, hypothesis 2c was supported.

Table 7. Hierarchical Regression Analysis Results for Hypothesis 2c

 

 

Step Predictors B at Step Finalj R2 AR2

1 Ability .08 .08 .30** .30**
Prior Experience .17* .13
Pre-Training Self-Efﬁcacy .41 ** .38**

2 Metacognitive Activity .19** .19** .35" .05**
Study Time .13" -.l3"

 

n=150;*p<.05;**p<.01;+p=.06

Hymthesis 3: Metacogm'tive Activig on Total Study Time. Hypothesis 3

concerning the relationship between metacognitive activity and total study time was
tested with a hierarchical regression analysis, regressing metacognitive activity onto

learning activity. Given that those with greater cognitive ability and those with more

85

prior experience are likely to require less time on the training program than those low on
these characteristics, they represent potentially large sources of variance in study time.
Thus, differences in ability and prior experience were controlled for by entering these
variables in the ﬁrst step. Table 8 presents the results of this analysis. Results indicated
that cognitive ability and prior experience did not account for signiﬁcant variance in
study time (R2 = .03, F = 2.40, p > .05). However, beta-weights revealed that prior
experience (B = -.17, p < .05 at step 1) was negatively positively related to study time.
The second step of the regression, including metacognitive activity, accounted for
signiﬁcant incremental variance in study time (AR2 = .03, AF = 4.19, p < .05).
Examination of the beta-weight indicates that metacognitive activity was positively

related to study time (B = .16, p < .05 at step 2). Thus, hypothesis 3 was supported.

Table 8. Hierarchical Regression Analysis Results for Hypothesis 3

 

 

Step Predictors B at Step Final B R2 AR2

1 Ability .02 .01 .03 .03
Prior Experience -. 17* -.20*

2 Metacognitive Activity .16* . 16* .06* .03 *

 

n=150;*p<.05;**p<.01

Hypotheses 4a, 4b, & 4c: Motivational Characteristics on Learning Processes.

Hypotheses 4a, 4b, and 4c concerning the effects of the three motivational characteristics
of mastery goal orientation, conscientiousness, and pre-training self-efﬁcacy were
examined with a set of three hierarchical regression analyses. The three regressions

differed only with respect to the DV (metacognitive activity, and total study time). Given

86

that those with greater cognitive ability and those with more prior experience are likely to
require less time on the training program than those low on these characteristics, they
represent potentially large sources of variance in study time. Additionally, questions
have been raised in the literature regarding the relationship between ability in a domain
and one’s ability to self-access (e. g., Kruger & Dunning, 1999), suggesting that cognitive
ability and prior experience may be important variables to control for when examining
the relationship of motivational variables to metacognitive activity. Thus, in all three
analyses, cognitive ability and prior experience were entered in the ﬁrst step. The three

motivational variables were entered in the second step.

Table 9. Hierarchical Regression Analysis Results for Hypothesis 4

 

 

Step Predictors B at Step Final B R2 AR2

1 Ability .05 .00 .03 .03
Prior Experience .18* .13

2 MasteryOrientation . .30“ .30** .13 .10“
Pre—Training Self-Efﬁcacy .03 .03
Conscientiousness .07 .07

 

n=150;*p<.05;**p<.01

Table 9 presents the results for metacognitive activity. The results revealed that
cognitive ability and prior experience did not account for signiﬁcant variance in
metacognitive activity (R2 = .03, F = 2.33, p > .05). The second step of the regression,
including mastery goal orientation, conscientiousness, and pre-training self-efﬁcacy
resulted in a signiﬁcant increase in variance explained (AR2 = .10, AF = 5.23, p < .01).

Examination of the beta-weights indicated that only mastery goal orientation accounted

87

for signiﬁcant unique variance in metacognitive activity (B = .29, p < .01 at step 2),
although pre-training self-efﬁcacy (r = .18, p < .05) exhibited a signiﬁcant zero-order
relationship with metacognitive activity.

Table 10 presents the results for study time. The results revealed that cognitive
ability and prior experience did not account for signiﬁcant variance in study time (R2 =
.03, F = 2.40, p > .05). However, beta-weights indicated that prior experience was
negatively related to study time (B = -.l7, p < .05 at step 1). The second step of the
regression, including mastery goal orientation, conscientiousness, and pre-training self-
efﬁcacy did not result in a signiﬁcant increase in variance explained (AR2 = .04, AF =
1.90, p > .05). Examination of the beta-weights indicated that mastery goal orientation (B
= .16, p = .06 at step 2) and pre-training self-efﬁcacy (B = -.l6, p =.08 at step 2) had
marginally signiﬁcant effects on study time. Pre-training self-efﬁcacy also exhibited a
signiﬁcant zero-order correlation with study time (r = -.23, p < .05), although this effect
became non-signiﬁcant when the other variables in the model were included. Thus,

hypotheses 4a was partially supported, whereas hypotheses 4b and 4c were not supported.

Table 10. Hierarchical Regression Analysis Results for Hypothesis 4

 

 

Step Predictors B at Step Final B R2 AR2

1 Ability .02 .Ol .03 .03
Prior Experience -.17* -.13

2 Mastery Orientation .16+ .16+ .07+ .04
Pre-Training Self-Efﬁcacy -.16+ -.16+
Conscientiousness -.01 -.01

 

n=150;*p<.05;**p<.01;+p<.10

88

Hypothesis 5: Metacognitive Training on Learning Processes. Hypothesis 5
concerning the effects of metacognitive training on learning processes was tested with a
set of two hierarchical regressions with metacognitive training dummy coded 0 (absent)
or 1 (present). The two regressions differed only with respect to the DV (metacognitive
activity and total study time). Given that the intervention was designed to enhance
learning processes beyond trainees’ initial levels, cognitive ability, prior experience, pre-
training metacognitive skills, and the motivational characteristics conscientiousness,
learning goal orientation, and pre-training self-efﬁcacy were controlled for in both
analyses by entering them in the ﬁrst step of the regression. Metacognitive training was

entered in the second step.

Table 11. Hierarchical Regression Analysis Results for Hypothesis 5

 

 

Step Predictors B at Step Final B R2 AR2
1 Ability .02 .03 .23** .23"
Prior Experience .11 .1 1
Metacognitive Skill .42" .44"
Mastery Orientation .09 .08
Pre-Training Self-Efﬁcacy -.01 -.01
Conscientiousness -.04 -.06
2 Metacognitive Training .17“ .17* .26** .03*

 

n=150;*p<.05;**p<.01

Table 11 presents the results for metacognitive activity. The results revealed that
the ﬁrst step accounted for signiﬁcant variance in metacognitive activity (R2 = .23, F =

7.33, p > .01). Beta-weights demonstrated that metacognitive skills (B = .42, p < .01 at

89

step 1) was the only signiﬁcant predictor in this step. The second step of the regression,
including metacognitive training, resulted in a signiﬁcant increase in variance explained
(ARZ = .03, AF = 5.21, p < .05).

Table 12 presents the results for study time. The results revealed that the ﬁrst step
did not account for signiﬁcant variance in study time (R2 = .07, F = 1.85, p > .05). The
second step of the regression, including metacognitive training, did not result in a
signiﬁcant increase in variance explained (AR2 = .00, AF = 0.12, p > .05). Thus,

hypothesis 5 was partially supported.

Table 12. Hierarchical Regression Analysis Results for Hypothesis 5

 

 

Step Predictors J} at Stg) FinaU3 R2 AR2
1 Ability .01 .01 .07" .07+
Prior Experience -.13 -. 13
Metacognitive Skill .07 .08
Mastery Orientation .13 .13
Pre-Training Self-Efﬁcacy -. 1 7+ -. 1 7+
Conscientiousness - -.03 -.03
2 Metacognitive Prompting .03 .03 .07 .00

 

n=150;*p<.05;**p<.01;+p<.10

Hymtheses 6: Metacognitive Prompting on Learning Processes. Hypothesis 6

concerning the eﬁ‘eCts of metacognitive prompting on learning processes were tested with
a set of hierarchical regressions with metacognitive prompting dummy coded 0 (absent)
or 1 (present). The two regressions differed only with respect to the DV (metacognitive

activity and total study time). Given that the metacognitive prompting was designed to

90

enhance learning processes beyond trainees’ initial levels, cognitive ability, prior
experience, pre-training metacognitive skills, and the motivational characteristics
conscientiousness, learning goal orientation, and pre-training self-efﬁcacy were
controlled for in each analysis by entering them in the ﬁrst step of the regression.
Metacognitive prompting was entered in the second step.

Table 13 presents the results for metacognitive activity. The results revealed that
the ﬁrst step accounted for signiﬁcant variance in metacognitive activity (R2 = .23, F =
7.33, p > .01). Beta-weights demonstrated that metacognitive skills (B = .42, p < .01 at
step 1) was the only signiﬁcant predictor in this step. The second step of the regression,
including metacognitive prompting, did not result in a signiﬁcant increase in variance
explained (AR2 = .00, AF = 0.12, p > .05). Further, prompting did not demonstrate a

signiﬁcant zero-order correlation with metacognitive activity.

Table 13. Hierarchical Regression Analysis Results for Hypothesis 6

 

 

Step Predictors B at Step Final B R2 AR2
1 Ability .02 .02 .23M .23"
Prior Experience .11 .11
Metacognitive Skill .42" .42"
Mastery Orientation .09 .09
Pre-Training Self-Efﬁcacy -.Ol -.01
Conscientiousness -.O4 -.04
2 Metacognitive Prompting .03 .03 .23" .00

 

n=150;*p<.05;**p<.01

91

Table 14 presents the results for study time. The results revealed that the ﬁrst step
did not account for signiﬁcant variance in study time (R2 = .07, F = 1.85, p > .05). The
second step of the regression, including metacognitive prompting, resulted in a
signiﬁcant increase in variance explained (AR2 = .06, AF = 9.71 , p < .01). Beta-weights
revealed that metacognitive prompting was positively related to study time (B = .24, p <

.05 at step 2). Thus, hypothesis 6 received partial support.

Table 14. Hierarchical Regression Analysis Results for Hypothesis 6

 

 

Step Predictors B at Step Final B R2 AR2
1 Ability .01 .00 .07+ .07+
Prior Experience -.13 -.13
Metacognitive Skill .07 .06
Mastery Orientation .13 .13
Pre—Training Self-Efﬁcacy -.17+ -.18*
Conscientiousness -.03 -.01
2 Metacognitive Prompting .24" .24M .13" .06“

 

n=150;*p<.05;**p<.01;+p<.10

Hyp_othesis 7: Metacognitive Training >< Motivational Characteristics on Learning

Processes. Hypothesis 7 concerning the interaction between metacognitive training and
motivational characteristics on learning processes was tested with a set of hierarchical
regressions. The three regressions differed only with respect to the DV (metacognitive
activity and total study time). Given that the metacognitive training was designed to
enhance learning processes beyond trainees’ initial levels, cognitive ability, prior

experience, pre-existing metacognitive skills, motivational characteristics (mastery goal

92

orientation, conscientiousness, and pre-training self-efﬁcacy), and metacognitive training
were entered in the ﬁrst step as control variables. The three interaction terms (mastery
goal orientation X metacognitive training, conscientiousness >< metacognitive training,

and pre-training self-efﬁcacy X metacognitive training) were entered in the second step.

Table 15. Hierarchical Regression Analysis Results for Hypothesis 7

 

 

Step Predictors B at Step Final B R2 AR2
1 Ability .03 .02 .26” .26"
Prior Experience .1 1 .12
Metacognitive Skill .44” .31 **
Mastery Orientation .08 .09
Pre-Training Self-Efﬁcacy -.01 .15*
Conscientiousness -.06 -.07
Metacognitive Training .17* .14+
2 Mastery Orientation X Training .07 .07 .31 ** .05*
Pre-Training Self-Efﬁcacy X Training -.17* -.17*
Conscientiousness X Training .06 .06

 

n=150;*p<.05;**p<.01

Table 15 presents the results for metacognitive activity. The results revealed that
the variables entered in step one accounted for signiﬁcant variance in metacognitive
activity (R2 = .26, F = 7.22, p < .01). Examination of the beta-weights demonstrates that
pre-existing metacognitive skills (B = .44, p < .01 at step 1) and metacognitive training (B
= .17, p < .05 at step 1) were the only variables entered in step 1 to account for unique
variance in metacognitive activity during training. The second step of the regression,
which included the three interaction terms, resulted in a signiﬁcant increase in variance

explained (AR2 = .05, AF = 3.00, p < .05), providing support for the general hypothesis

93

that motivational characteristics moderate the affect of metacognitive training on
metacognitive activity. Examination of the individual beta-weights revealed that the
interaction between pre-training self-efﬁcacy and training (B = -.17, p < .05 at step 2)

accounted for signiﬁcant unique variance in metacognitive activity. This interaction is

displayed in ﬁgure 5.

Figure 5. Self-Efﬁcacy X Metacognitive Training on Metacognitive Activity

 

 

 

 

 

 

 

4.6 -
ﬁg"
'2 4-1 ‘ Metacognitive
m Training
.2
:E
§’ 3-6 r —O-Absent
g - I— Present
0
5 3.1 -

2.6 u .

Low Self-Efﬁcacy Hi Self-Efﬁcacy

Table 16 presents the results for study time. The results revealed that cognitive
ability, prior experience, mastery goal orientation, pre-training self-efﬁcacy,
conscientiousness, and metacognitive training did not account for signiﬁcant variance in
study time (R2 = .07 AP = 1.60, p > .05). The second step of the regression, which
included the three interaction terms, did not result in a signiﬁcant increase in variance
explained (AR2 = .00 AF = .12, p > .05). Thus, partial support was found for hypothesis

7.

94

Table 16. Hierarchical Regression Analysis Results for Hypothesis 7

 

 

Step Predictors B at Step Final B R2 AR2

1 Ability .01 .01 .07+ .07+
Prior Experience -.13 —. 12
Metacognitive Skill .08 .07
Mastery Orientation .13 .00
Metacognitive Prompting -.17+ -.05
Conscientiousness -.03 -.Ol
Metacognitive Training .03 .03

2 Mastery Orientation X Training .05 .05 .07 .00
Pre-Training Self-Efﬁcacy X Training .00 .00
Conscientiousness X Training .00 .00

 

n=150;*p<.05;**p<.01;+p<.10

Overall Model Testing

Structural Equations Modeling (SEM) was utilized to test the model in a more

- comprehensive fashion. The hypothesized model displayed in Figure 1 was tested using
AMOS 4. The model was ﬁrst tested without examining the interactions. Listwise
deletion was utilized to address missing data (Schumacker & Lomax, 1996), which
reduced the sample size to 126. This structural model, along with the standardized
parameter estimates, is displayed in Figure 6.

This model resulted in a x2 value of 108.3, df = 31, p > .05. This value is
signiﬁcant, indicating a bad ﬁt of the hypothesized model to the observed data. A
goodness of ﬁt index (GFI) of .87 was obtained. The rule of thumb is that this value
should exceed .90 at a minimum, but .95 and higher is indication of a good ﬁt. Thus, this
again indicates a poor ﬁt for the model. Additionally, a RMSEA value of .14 was found.

This value represents the discrepancy between our data and our model, per degree of

95

O

 

 

85.5051
swaméxw

 

 

 

 

 

meaneston. ,
8:832

no.

go.

 

 

 

 

 

9

$3.265.
02.2208

 

 

6

3.

c..-

 

 

 

585.2%
m:.:_m.._...mon.

 

 

9

./ a

 

 

mEF >25

 

 

b
3.

 

 

 

€32
93.38865.

 

e

E. n <Mm§ Kw. M ED SM n ma Judo. n NR

 

 

«do:
«20.29850

 

 

 

 

38555....
0558.73.

 

 

 

we.

 

5.55.5
beams.

 

 

 

 

emm.

cc.

2.
aVN.

co.

 

3.52...
058822,.

 

 

 

 

9.8805
3.88%:

 

 

 

88858. .853 .82: Bursa: co 3..an 2mm 6 2%....

96

freedom. According to Brown and Cudeck’s criteria, which states that .05 or less
represents close ﬁt, .05 - .10 represents moderate ﬁt, and .10 or greater is a bad ﬁt, this
also indicates a poor ﬁtting model. Overall, this model presents a poor ﬁt to the data.

Consistent with the hypothesis tests described above, examination of the
parameter estimates indicated that two variables in particular did not function in the
model as expected. Conscientiousness was not related to either study time or
metacognitive activity, as speciﬁed in the model. Additionally, skill-based performance
was not predicted by either metacognitive activity or study time. Additionally, the high
correlation between skill-based performance and declarative knowledge is cause for
conceptual concern. The unexpected strength of this relationship may be a result of the
type of task used in this study. The task was heavily knowledge based and little in the
way of skill required to execute that knowledge rather than moving and clicking the
mouse. If one knows what to do, little additional knowledge or is skill needed to
determine how to do it or to actually perform the action. Thus, the primary factor
determining performance on the skill-based performance measure was declarative
knowledge, resulting in a great deal of redundancy in the two measures.

Thus, the model was re-tested with conscientiousness and skill-based performance
removed (Figure 7). This resulted in drastically improved model ﬁt. This model resulted
in a 12 value of 23.99, df = 19, p > .05. This value is signiﬁcant, indicating a lack of
model ﬁt. However, a GFI of .96 was obtained, exceeding the .95 rule-of-thumb for a
good ﬁt. Additionally, a RMSEA value of .046 was found, again indicating a good ﬁt to

the data. Because variables were removed in this model, it is not nested with respect to

97

 

 

8588.5“.
Saga?

 

 

a

*wN.

 

a

 

 

88.305.
9:95.80

9.... u <35 .8. u E0 .2 n "E .32 n N..

a

 

eon.

 

 

 

 

 

 

>08Em.._ow
9.58.763.

 

 

 

 

 

 

 

 

 

a

emN.

 

 

emu:
9:... 32m 8.
+
8.
ova.
8. 8.
32.2 2.
Exams—mos. .E
E 8.

 

 

685.28
9.....8 .721

 

 

LN.

 

 

cosmEotO
beams.

 

 

 

 

9.59...
92.3839).

 

 

 

 

91:388.“.
93.38962

 

 

28888:. 305.3 40.88 02385. .5 85E

98

the model described above and, therefore a chi-squared difference test cannot
appropriately be calculated (Maruyama, 1997).

The interactions were modeled using multiple-groups analysis, with the two levels
of metacognitive training representing different samples. The logic underlying this
approach is that, where interactions are present, parameter estimates should differ across
the two samples (Schumacker & Lomax, 1996). Two nested models were than examined
to determine the equivalence of the hypothesized model across the two levels of
metacognitive training. The ﬁrst model allowed all paths to be freely estimated across
both groups. The second model restricted the following four paths to be equivalent
across groups: 1) mastery orientation to metacognitive activity, 2) mastery orientation to
study time, 3) pre-training self-efﬁcacy to metacognitive activity, and 4) pre-training self-
efﬁcacy to study time. Thus, if metacognitive training and motivational variables
interacted in their effects on learning processes, restricting these paths to be equal should
result in a poorer ﬁt than the model allowing these paths to be freely estimated.

The results from the model comparison (x2 = 6.48 with 4 DF, p = .17) suggest that
imposing the restriction of four equal path coefficients across the two levels of
metacognitive training did not result in a signiﬁcantly worse ﬁtting model. Thus,
metacognitive training does not appear to moderate the four paths speciﬁed.

Examination of the results of the hypothesis tests above, as well as examination of the
path coefﬁcients from the present analysis suggest that the relationship between the two
motivational variables and study time is not moderated by metacognitive training,
whereas the relationship between the motivational variables and metacognitive activity is.

Thus, another pair of models was compared which constrained only the path between

99

mastery orientation and metacognitive activity, as well as the path between pre—training
self-efﬁcacy and metacognitive activity. Comparison of these two models (12 = 5.87
with 2 DF, p = .05) indicate that restricting these two paths two be equal results in a
signiﬁcantly worse ﬁtting model, suggesting that the effect of motivational variables on
metacognitive activity is moderated by metacognitive training. This analysis is

graphically displayed in Figure 8.

100

omém.

 

 

8:25:65
838:3...

 

 

me. n a an m .23 u N.. 4.

mg. n SE5. .3. n :5 .8 n .8 .32 u N.. .8358: ms... 85.525 .. N .082
NS. u $25. .8. u :6 .8 u .5 .82. u N.. ”838.3 been 2...... .2 I . .252

3

 

 

6930.265.
9396.000

 

 

 

 

$.42.

a

 

 

i 5:... 2:: >35

. .. .V
84»—

 

 

 

 

 

 

683m.._mm
3.58.535

 

 

32.2.
3.3". 93.30865.

 

 

 

 

e

 

e

Raﬁ:

 

$.43.

 

>085m.._mm
9.59.525

 

 

 

2.2m.

 

5385.5
beams.

 

 

 

 

04.35685
93.58305.

 

 

 

3885383... magma. 33.58802 £3,335 332w 0.53.92 .3 95w...—

101

DISCUSSION

The purpose of this study was to examine critical aspects of self-regulation in
training contexts. In support of this goal, the concept of metacognition was explored as it
relates to the monitoring of one’s knowledge and the subsequent inﬂuence of this
monitoring on actions taken in support of learning. A conceptual model was developed
that links trainee characteristics, training design features, learning processes, and training
outcomes (Figure 1). Trainees’ motivational characteristics were posited to inﬂuence two
learning processes - metacognitive activity undertaken during training, as well as study
time. These learning processes were linked to cognitive, behavioral, and affective
training outcomes, represented by declarative knowledge, skill-based performance, and
post-training self-efﬁcacy. These training outcomes, in turn, were linked to performance
on a task requiring application and integration of the knowledge and skills acquired
during training.

Given the conceptual and empirical support for the role of metacognition during
learning, two interventions, metacognitive training and metacognitive prompting, were
developed to support and enhance metacognitive activity during training. Metacognitive
training consisted of brief instruction prior to the start of the training program proper,
wherein trainees were informed of the concept of metacognition, the important role
metacognitive monitoring plays during learning, common mistakes trainees typically
make when monitoring their learning, and strategies to be used during training to improve
monitoring. Metacognitive prompting consisted of periodically instructing trainees to
access their learning and comprehension of the studied material. These two interventions

were expected to inﬂuence both metacognitive activity during training, as well as the

102

total time individuals spent on the training program. In addition, metacognitive training
was expected to interact with the motivational characteristics in its inﬂuences on learning
processes.

Motivation to Learning Processes

The motivational characteristics of mastery goal orientation, pre-training self-
efﬁcacy, and conscientiousness were expected to relate to both metacognitive activity and
study time. Results indicated that, when entered simultaneously into the regression
equation, these three variables explained incremental variance in metacognitive activity
beyond that accounted for by ability or prior experience. However, only mastery
orientation accounted for unique variance in metacognitive activity.

It was assumed that mastery orientation would exert a positive inﬂuence on
metacognitive activity primarily through motivational means. That is, those with a high
mastery orientation would be more motivated to acquire and master the training material,
which would lead to a willingness to exert the additional time and effort needed to
engage in metacognitive activity during training. However, post-hoe analyses revealed
that mastery orientation failed to account for variance in metacognitive activity after the
effects of pre-training metacognitive skills had been partialled, opening the possibility of
alternative mechanisms underlying this link. Those with a high mastery orientation
appear to possess superior metacognitive skills across situations (r = .54). These skills
may develop over time as a means of supporting their learning goals. Therefore, in the
present study, these individuals may have more skillfully and naturally engaged in
metacognitive activity, regardless of their interest or motivation to learn in this particular

situation. Another possibility is that the mastery orientation - pre-training metacognitive

103

skill link reﬂects a stable tendency for those with a high mastery orientation to utilize
their metacognitive skills, although they may not differ with respect to possession of the
skills per se. This general tendency to utilize existing skills may have carried over to the
training program in this study, resulting in the observed link between mastery orientation
and metacognitive activity during training.

Despite its signiﬁcant positive relationship with pre-training metacognitive skills
(r = .37), conscientiousness did not exhibit a signiﬁcant relationship with metacognitive
activity. This suggests that metacognitive activity was not dependent upon dutifulness or
achievement striving, the most prominent features of those high in conscientiousness.
This ﬁnding is consistent with those of mastery orientation described above. Thus, it
appears that motivational characteristics, at least as operationalized in the current study,
do not play as strong a role in directly inﬂuencing metacognitive activity as had been
expected. This could be due to speciﬁc characteristics of the training program used in
this study, as will be discussed in limitations below.

Motivational variables also did not exhibit as strong an inﬂuence on study time as
was expected. None of the three motivational characteristics accounted for signiﬁcant
variance in study time, whether entered individually or simultaneously into the regression
equation. However, although not signiﬁcant by typical standards, both mastery
orientation and pre-training self-efﬁcacy had marginally signiﬁcant relationships with
study time. Mastery orientation was positively associated with study time. Individuals
with high mastery orientations tend to be more concerned with and interested in learning
and, therefore, were expected to invest geater time and effort into studying the training

program. The relationship with pre-training self-efﬁcacy was more difﬁcult to speculate

104

on. While those who are low on self-efﬁcacy could be expected to withdraw ﬁ'om the
task and, therefore, exhibit a positive relationship with study time, those with higher self-
efﬁcacy may also believe, accurately or not, that they have mastered the training material
more quickly than those with lower efﬁcacy and, therefore self-efﬁcacy would exhibit a
negative relationship with study time. Results suggested that the latter may have been the
case in this study, as pre-training self-efﬁcacy was marginally and negatively related to
study time.
Metacognitive Interventions

Of primary concern in this study were the two interventions designed to enhance
metacognitive activity during training. Consistent with expectations, those receiving
metacognitive training reported greater metacognitive activity during training. Although
metacognitive training did not have a signiﬁcant zero-order correlation with
metacognitive activity, this effect became signiﬁcant after the effects of pre-training
metacognitive skills were partialled. The suppressor effect as typically observed arises
when high intercorrelations are observed among predictors, but a low or non-signiﬁcant
relationship is found between at least one predictor and the criterion (Nunnally &
Bernstein, 1994; Tzelgov & Henik, 1991). In such instances, the suppressing predictor
removes or accounts for (i.e. suppresses) “criterion-irrelevant” variance in the suppressed
predictor, increasing the predictive power of the remaining variance.

In the present situation, there is virtually no relationship between the two
predictors (r = -.02, ns). Yet, this correlation is large enough to boost the marginally
signiﬁcant effect of metacognitive training on metacognitive activity (r = .1399, p =

.0776) into statistical signiﬁcance. Examination of the regression equation demonstrates

105

that this is necessarily so, given the mechanics of the equation. Considering that
metacognitive training was a manipulated variable, with metacognitive skills being
assessed prior to this manipulation, any relationship between these variables, signiﬁcant
or not, can only be attributed to sampling error. If one sets the correlation between
metacognitive training and metacognitive skills to its “true” value of 0, the relationship
between metacognitive training and metacognitive activity mirrors the marginally-
signiﬁcant value obtained via the zero-order correlation.

Although metacognitive training and pre-training self-efﬁcacy each had only
weak and/or non-signiﬁcant direct effects on metacognitive activity, an interaction was
found among the two, as predicted. This interaction is graphically displayed in Figure 5.
Among those who did not receive the metacognitive training, pre-training self-efﬁcacy
was positively related to metacognitive activity. Those with low self-efﬁcacy have little
faith in their ability to acquire the skills being taught in the training program. As a result,
these individuals are less likely to invest the effort required to fully engage in the
training, including actively monitoring and controlling their learning processes.
Additionally, the positive correlation between pre-training self-efﬁcacy and pre-training
metacognitive skills indicates that these individuals are less likely to engage in such
activities in many learning situations.

In contrast, pre-training self-efﬁcacy did not inﬂuence metacognitive activity for
those who received metacognitive training. Examination of Figure X suggests that the
training had a beneﬁcial effect for trainees with lower self-efﬁcacy. Without training,
those with low self-efﬁcacy demonstrated lowered levels of metacognitive activity when

compared to trainees with higher self-efﬁcacy. With the metacognitive training, those

106

trainees with low self-efﬁcacy exhibited levels of metacognitive activity on par with
those with higher self-efﬁcacy, whereas the metacognitive activity of trainees with high
self-efﬁcacy was relatively unaffected by the intervention.

Counter to expectations, metacognitive training had no direct effect on study time,
nor did it interact with motivational characteristics to effect study time. This ﬁnding is
somewhat surprising, particularly given the signiﬁcant interactive effect of metacognitive
training and self-efﬁcacy on metacognitive activity. Hypothesis 3 predicted that
metacognitive activity would be positively related to study time. The basis for this
expectation was twofold. First, it was assumed that the processes of more actively
monitoring one’s learning would require time and attentional resources, adding to the
total time trainees spent on the training program. Additionally, the metacognitive
training encouraged learners to identify training information that they had not yet learned
sufﬁciently and to spend additional time studying this information. To the extent that
trainees did so, this would be expected to increase the total time spent on the training
program.

Results provided support for this hypothesis, in that metacognitive activity was
positively related to study time. However, metacognitive activity accounted for only 3%
of the variance in study time, suggesting that this link was much weaker than expected.
Given the weakness of the link between metacognitive activity and study time, it is not
surprising that metacognitive training, which was intended to primarily inﬂuence
metacognitive activity, did not have an effect on study time. Several potential
explanations can be forwarded for this ﬁnding. First, the metacognitive activity measure

may reﬂect differences in both the quality and quantity of metacognitive activity.

107

Individuals may engage in metacognitive activities that differ greatly in quality, but do
not differ in quantity or time required for their use. That is, some individuals may less
frequently monitor their learning and subsequently adjust their learning strategies, but
these few instances may be of sufﬁcient quality to yield equal or superior learning as
compared to other individuals who may more frequently utilize less effective monitoring
and control strategies. Following this logic, metacognitive training may have affected the
quality of metacognitive activities, without increasing the quantity of such activities or
the time required for their use. Although metacognitive monitoring and study time
allocation has been studied under simple word-list learning tasks, the current study is one
of the few known studies to examine this relationship under complex learning tasks,
leaving much to be learned about the interplay between these two constructs under such
tasks.

Consistent with expectations, metacognitive prompting resulted in greater study
time. The prompts, by design, required trainees to periodically reﬂect on the material
they had studied, identify what they had learned so far, as well as what they needed to
learn better. Because study time was operationalized as the time elapsing between the
initiation and termination of the training program proper, which included time spent on
the prompts, it could be argued that the positive relationship between metacognitive
prompting and study time is artiﬁcial — individuals receiving the prompts spent additional
time studying because the task required them to. In contrast, one could argue that the
time spent on the prompts is time spent actively thinking about the training material and,

therefore, legitimately belongs in the consideration of time spent on training.

108

Additional analyses may help sort out the issue. When time spent on the prompt
screen is removed from the total time spent on training, leaving only time spent on the
training content itself, metacognitive prompting no longer has a signiﬁcant effect on
study time (r = -.05, p = .49). Thus, individuals receiving metacognitive prompts did not
spend additional time on the training material itself, as was expected. One could argue
that, although the metacognitive prompts did not lead to an increase in the time spent
with the training material itself on the screen, the quality of the time spent examining this
material improved as a result of the prompts. For example, although two trainees may
have spent 45 minutes studying the training material, the one who received the prompts
would spend this time more strategically, focusing their time on the aspects they needed
to learn most. If this were the case, the relationship between study time and learning
outcomes would likely be moderated by prompting, as 45 minutes of high-quality study
time would be expected to be more strongly and positively related to learning outcomes
. than would 45 minutes of low-quality study time. However, no such interaction was
observed for any of the learning outcomes. Although null results such as this are difﬁcult
to interpret, it at least suggests that neither the quality of study time, nor the quantity,
differed as a function of the metacognitive prompting.

Perhaps a more critical question is whether the time spent on the prompts
themselves was predictive of learning outcomes. Stated differently, did the amount of
time spent on the prompts help trainees to better learn the material? Post-hoe analyses
revealed that, for those trainees receiving prompts, time spent on the prompts was
marginally correlated with declarative knowledge (r = .21, p = .06). When criterion-

irrelevant variance in the time spent on the prompts due to prior experience and cognitive

109

ability were partialled, time on prompts exhibited a signiﬁcant positive relationship with
declarative knowledge. Thus, although metacognitive prompting did not lead trainees to
spend more time on the training material per se, and was not signiﬁcantly related to
metacognitive activity (r = .16, p = .16), it does appear to have been time spent thinking
about the material, as indicated by the positive relationship with declarative knowledge.

Counter to expectations, metacognitive prompting failed to affect metacognitive
activity. One potential explanation for this failure could be that not all trainees receiving
the prompts made an attempt to fully utilize the prompts. As indicated above, individuals
receiving prompts spent signiﬁcantly more time on the training program, a difference that
is directly attributable to the time they spent with the prompts on the screen. However,
trainees may have differed in how much they attempted to truly utilize the prompts to
evaluate their learning, versus simply complying with experiment requirements. Post-hoc
analyses revealed that self-reported motivation to utilize the prompts was positively
related to metacognitive activity (r = .46, p < .01). This effect remains even after
partialling the effects of prior experience, ability, and pre-training metacognitive skills.
Thus, it appears that the prompts may have inﬂuenced metacognitive activity for those
who genuinely attempted to utilize them. However, additional post-hoe analyses failed to
ﬁnd a signiﬁcant interaction between prompting and any of the model variables in
predicting metacognitive activity. Thus, if differences in trainee motivation to utilize the
prompts are truly responsible for the lack of a signiﬁcant prompting effect, no variables
contained in the current model can account for these differences in motivation.

Another possibility is that characteristics of the prompts were not conducive to

encouraging metacognitive activity. First, the prompts were intentionally designed to

110

encourage the use of any existing metacognitive skills, rather than directing participants
in the use of speciﬁc techniques. More directive prompts may have provided participants
with more guidance on exactly how to monitor and evaluate their learning. Such
directive prompts could be particularly beneﬁcial for participants lacking skills initially.
One possible speciﬁc prompt could be requiring the use of the question-stems provided to
participants receiving metacognitive training in the current study. Speciﬁc content-
relevant questions, similar to the declarative knowledge questions asked at the conclusion
of training in this study, could also be asked periodically during training. Trainees could
be encouraged to use these questions to determine where to direct their attention and
study time.
Learning Outcomes

This study found that learning processes were related to some of the learning
outcomes. As expected, metacognitive activity was positively related to post-training
self-efﬁcacy. This effect remained even aﬁer accounting for the effects of pre-training
self-efﬁcacy. Thus, consistent with the propositions of Winne (1995) and Bandura
(1997), positive feelings of efﬁcacy appear to result from monitoring one’s
comprehension. It was assumed that this effect would result, in part, from the
hypothesized effects of metacognitive activity on knowledge. Those who more actively
monitored and controlled their learning were expected to improve their learning, which
would result in higher self-efﬁcacy. However, in the present study, no relationship was
found between metacognitive activity and cognitive or behavior indices of learning.
Thus, higher efﬁcacy appears to result from simply engaging in active monitoring and

control of learning leads one to have greater self-efﬁcacy, independent of increases in

111

 

actual learning. Bandura (1997) has stated the process of successfully guiding one’s own
learning and performance provides positive self-generated feedback concerning one’s
competence and capabilities. In support of this claim, Schunk and Cox (1986) found that
students who verbalized their cognitive strategies while they performed demonstrated
greater increases in self-efﬁcacy than students who did not exercise such self-guidance.
However, given that both metacognitive activity and post-training self-efﬁcacy were self—
report measures assessed at the same point in time, same source method bias may be L.

contributing to some of the observed relationship.

 

It may also be that individuals engaging in greater metacognitive activity did, in

‘ ,-__ ‘_M
‘a ".
- <-

fact, learn more, but the dependent measures failed to identify this learning. For
example, engaging in monitoring and controlling of one’s learning may lead to more
complete and integrated knowledge structures, which was not assessed in the current
study. Given the existing conceptual and empirical support for the relationship between

O metacognition and learning (e.g., Brown, Bransford, Ferrara, & Campione, 1983; Ford et
al., 1998; Meloth, 1990; Nelson & Narens, 1994; Volet, 1991), one must consider that the
outcome measures used herein may have failed to capture the learning gains resulting
from increased metacognitive activity.

Although post-training self-efﬁcacy was conceptualized as an outcome of

 

learning processes, the dynamic nature of learning makes the true directionality of this
relationship difﬁcult to establish. The methodology employed in the current study does
not allow for clear determination of the order of causality. The negative relationship
observed between study time and post-training self-efﬁcacy also raises questions

concerning the true causal order of post-training self-efficacy as measured and learning

112

 

processes. While it seems reasonable to conclude that possessing low self-efﬁcacy could
lead one to spend more time studying the training material, the reverse does not. That is,
it is difficult to conceptually justify that spending more time studying would lead one to
have less conﬁdence in their ability to create web pages. Thus, the negative relationship
leads to the conclusion that the aspects of self-efﬁcacy tapped by the post-training
measure may more appropriately belong causally prior to study time, rather than after, as
speciﬁed in the model. It is most likely the case that self-efﬁcacy and study time are
dynamically and cyclically related. Self-efﬁcacy beliefs may inﬂuence study time, such
that those with lower self-efﬁcacy spend greater time studying, as they believe this is
necessary to learn the material. However, as these individuals spend more time on the
material their efficacy beliefs would be expected to increase. Although the inclusion of
both pre- and post-training self-efﬁcacy was intended to capture some of the dynamic
nature of self-efﬁcacy within the learning process, the construct validity of the post-
training self-efﬁcacy measure may be questioned, as it may be measuring trainees’
perceptions of efﬁcacy as they progressed through training, rather than solely measuring
self-efﬁcacy after training has been completed. Pre-training self-efﬁcacy was only
marginally and negatively related to study time, while post-training self-efﬁcacy was
strongly negatively related to study time (r = -.54).

More intuitive results emerged concerning the effect of study time on declarative
knowledge. As was expected, individuals who studied longer demonstrated greater
declarative knowledge than those who studied less. This effect remains whether one
considers total study time including time spent on the metacognitive prompts, when

applicable, or only just the time spent on the training content itself. This result comes as

113

no surprise. Study time was, in fact, expected to play a key role in the model. One of the
key goals of the metacognitive interventions was to prevent individuals ﬁ'om disengaging
from the training prior to mastery — that is, to spend more time studying the material.
Although, as discussed above, the effect of prompting on study time may be difﬁcult to
explain simply as increases in metacognitive activity, the positive relationship observed
between metacognitive activity and study time lends some support to the notion that
those who engage in greater metacognitive skills are less prone to discontinue study time
prematurely. The additional study time, in turn, results in improved learning, as reﬂected
by declarative knowledge.

Results also indicated support for the general hypothesis that the learning
outcomes of declarative knowledge, skill-based performance, and post-training self-
efﬁcacy would be positively related to application performance. The three outcomes
entered simultaneously explained signiﬁcant variance in application performance beyond
that attributed to prior experience, ability, or pre-training self-efﬁcacy. However, only
skill-based performance accounted for unique variance in application performance,
despite the signiﬁcant zero-order correlations observed between declarative knowledge
and application performance and post-training self-efﬁcacy and application performance.

The failure of declarative knowledge and post-training self-efﬁcacy to account for
unique variance in application performance is not surprising, given the pattern of
correlations between the three learning outcomes. Skill-based performance and
declarative knowledge were correlated .56 calling into question the distinctiveness of the
constructs assessed by these measures. However, this explanation does not account for

the inconsistent pattern of relationships of these two variables with other variables in the

114

 

model. For example, declarative knowledge was positively correlated with study time (r
= .25, p < .05), whereas skill-based performance was not. It is more likely the case that,
although post-training self-efﬁcacy, skill-based performance, and declarative knowledge
are all accessing somewhat unique constructs, the conceptual relationships among them
leads to each accounting for similar aspects of application performance.

It may also be that the relationship between declarative knowledge and
application performance, as well as that between post-training self-efﬁcacy and
application, are in fact mediated by skill-based performance. Post-hoe analyses indicate

some support for this notion. Without the presence of skill-based performance,

 

declarative knowledge has a signiﬁcant direct effect on application performance,
although post-training self-efﬁcacy does not. Both variables have signiﬁcant direct
effects on skill-based performance. As indicated above, skill-based performance has a
signiﬁcant effect on application performance. Finally, the direct effect of declarative
knowledge becomes non-signiﬁcant once skill-based performance has been partialled. A
variation of this explanation is that skill-based performance may be tapping one aspect of
the more general performance domain assessed by the measure of application
performance. Post-training self-efﬁcacy and declarative knowledge may be successfully
predicting this aspect of performance, but not other aspects of application performance
that go beyond skill-based performance. These issues should be sorted out in future
research.
Limitations

Several limitations may account for some of the unexpected results in the current

study. One limitation concerns the measurement of several variables in the model.

115

Metacognitive activity was conceptualized as a multi-dimensional construct consisting of
both monitoring and control components. However, factor analyses did not support this
dimensional structure. Although eigenvalues suggested as many as 6 factors, the scree
plot indicated that the ﬁrst factor was prominent, with the possibility of one or two
additional factors. However, the pattern of factor loadings did not support the proposed
conceptualization. Post-hoe attempts to deﬁne the resulting factors on the basis of item
content led to ambiguous and largely uninterpretable constructs with a great deal of
conceptual overlap, very high intercorrelations, and relatively low reliabilities. Thus,

although metacognitive monitoring and control are conceptually distinct, the two may be

 

so closely tied that the two cannot be readily separated in practice, particularly with self-
report measures.

An additional measurement issue concerns the construct validity of self-efficacy.
In an effort to capture some element of the dynamic nature of self-efficacy in the learning
process, self-efﬁcacy was assessed both before and at the conclusion of training.
Although the two measures were highly correlated (r = .54), a factor analysis of both pre-
and post-training self-efﬁcacy items revealed a two-factor solution, with all items loading
on the appropriate scale. Thus, the two measures do appear to be measuring distinct
aspects of self-efﬁcacy. However, the negative relationship between study time and post-
training self-efﬁcacy raises concerns as to what post-training self-efﬁcacy is truly
tapping. In responding to the post-training self-efﬁcacy measure, participants may have
reﬂected on their efﬁcacy throughout the training program, rather than solely assessing
their efﬁcacy at that moment. If this were the case, this could explain the negative

relationship that remained even when taking pre-training efﬁcacy levels into account.

 

116

Additionally, pre-training self-efﬁcacy was assessed before having any exposure to the
training program. Thus, these efﬁcacy beliefs may not have been well calibrated and
could have shifted dramatically upon initial exposure to the training program. Had self-
efﬁcacy been assessed early in training, but after some initial exposure to the training
program, it may have been found that greater study time did in fact lead to higher self-
efﬁcacy when compared to these initial, rather than pre-training levels.

The remaining outcome measures represent an additional measurement concern in
this study. One problem concerns the difficulty level of the skill-based performance and
declarative knowledge measures. The average item difficulty level of the declarative
knowledge and skill-based performance measures was .76 each. The high item means
indicate that most trainees were getting most items correct, serving to reduce the variance
in the measures attributable to differences in true levels of the constructs being assessed.
The high overall means for these two measures further highlight this problem. The
average score for skill-based performance was 18.29 out of a maximum of 24, whereas
the mean overall score for declarative knowledge was 26.67 out of a maximum of 37.
Given that most trainees scored well on- both of these measures, this suggests that the
measures failed to capture the full range of learning, which may have limited their
observed relationships with other measures in the model.

In addition to the measurement issues raised above, several design issues warrant
mention as well. The ﬁrst issue concerns the choice of training topics. Web-page
creation was chosen as the training topic because it was expected to be a topic of interest

for the trainees and, therefore, at least minimal levels of motivation and attentiveness

117

 

 

 

would result. Additionally, it was expected to be a topic on which most trainees would
not have much existing knowledge.

Prior experience was found to play a much larger role throughout the model than
had been expected. Including prior experience as a covariate in all analyses only partially
solves the problem. One must wonder whether experience accounted for even more
variance throughout the model than revealed by the measures used herein. To the extent
that the measure of prior experience was not sufﬁcient to ﬁrlly assess this surprisingly
inﬂuential construct, this leads to variance in measures throughout the model that cannot
be accounted for by measures included within the model. This increase in unexplained
variance makes it more difﬁcult to ﬁnd signiﬁcant effects where they otherwise may have
existed.

Even when trainees had no prior experience with web-page creation per se,
experience with the intemet and computers in general likely still exerted inﬂuence
throughout. Those with greater familiarity with the intemet would seem likely to possess
existing schemas for the concepts covered in the training, particularly with respect to the
purpose of various elements within web pages, such as hyperlinks, tables, etc. These
schemas would be expected to assist greatly in the acquisition of the training material, as
they would need only to learn how to implement the features within web pages of their
own creation, but would already be familiar with their meaning and purpose. Those
without such schemas would in effect have more information to learn, thereby decreasing
their probability of learning any given piece of information.

While web page creation was chosen as the training content in part to ensure some

minimal degree of participant motivation, it is possible that trainees were, as a whole, too

118

motivated in the present study. That is, there may not have been sufﬁcient variance in
trainee motivation to allow the motivational characteristics to truly take effect. For
example, mastery orientation is has been found to be related to motivation to learn (e.g.,
Colquitte & Simmering, 1998), in part because individuals high in this orientation value
learning in general, which inﬂuences their probability of focusing on learning in speciﬁc
contexts. Most individuals in the present study may have adopted learning goals,
regardless of their general tendency to do so. The only signiﬁcant effect for mastery
orientation in this study was the positive relationship with metacognitive activity. That
this effect went away aﬁer the effects of pre-training metacognitive skills were removed
indicates that this effect was likely due to skills built in support of one’s general tendency
to adopt learning goals, but the motivational impact was typically associated with a
mastery orientation may not have been unique to these individuals in this context. It may
have been more revealing to examine state mastery orientation, the extent to which
individuals were focused on learning and developing their skills in this speciﬁc context.

Participants were not provided feedback concerning how well they were learning
the material in the training program. This was done in an effort to strengthen the
inﬂuence of metacognition and, relatedly, the metacognitive interventions. With no task-
generated feedback, it was expected to particularly critical for trainees to accurately
evaluate their learning and use this information to guide their learning activities.
However, the inﬂuence of task-generated feedback on metacognition has not been well
examined, allowing only speculation concerning how this may have affected the results.
It is possible that individuals who engage in metacognitive activities beneﬁt ﬁ'om the

additional information provided by the task. While participants who engage in less

119

metacognitive activities may rely solely on the task-generated feedback to guide their
learning activities, individuals who actively monitor and control their learning may use
this feedback to supplement their own evaluations of learning.

The lack of feedback may also have motivational impacts although, again, the
nature of these impacts are unclear. The lack of feedback may have been frustrating and
demotivating for some, potentially leading to withdrawal. Some individuals may have
been acquiring the skills taught in the training program, but their own self-generated
feedback may have indicated otherwise, lowering motivation. On the other hand, given
the tendency of individuals to overestimate what they know, the provision of feedback
may have resulted in more negative evaluations than that provided by self-generated
feedback alone, which could impact self-efﬁcacy, persistence, effort, etc. Finally, the
lack of evaluation may have encouraged more trainees to adopt a learning orientation,
thereby reducing the impact of trainees’ dispositional mastery orientations. Providing
evaluative feedback may have caused some participants to adopt avoidance goals, such as
avoiding being judged incompetent by oneself or others. Avoidance goals such as these
may have negative impacts on learning and performance.

Another design feature that may have impacted trainees’ learning processes was
the unavailability of practice opportunities during training. Trainees were required to
ﬁrst spend time learning the material in the training program before attempting to
perform the skills being acquired. The reason for this was two-fold. First, pre-testing,
which allowed practice during training, indicated that few trainees seemed to be
experience much difﬁcultly in acquiring the information in the training program and most

trainees were performing quite well on the outcome measures. Thus, it was feared that

120

insufﬁcient variance might result. Additionally, many trainees seemed to be able to
largely ignore the training content but learn to create web pages simply by practicing and
with and exploring the web-editing program. While this mode of learning is intriguing
and important in its own right, it was not the focus of the current study. By not allowing
trainees to practice as they learned, it put the onus on learning the information that was
contained within the training program and, in so doing, was expected to increase the
difficulty of the training program as a whole and, hopefully, result in more variance 3*
throughout the model.

Secondly, it was believed that practice would provide feedback to participants

 

concerning how well they were learning the training material regardless of how actively
they were monitoring their learning on their own. Thus, practice may have reduced the
importance or impact of metacognitive activity and the metacognitive interventions. By
not permitting practice until after completion of the training program, it was expected to
be critical that individuals actively and accurately monitored how well they were learning
material that they were studying.

As is the case with feedback, the role that practice opportunities play with
metacognition has not been well explored, leaving the nature of this relationship unclear.
Although not the focus of their study, Ford et a1. (1998) found a positive correlation
between metacognition and the extent to which trainees practiced a critical task strategy
which, in turn, was positively related to declarative knowledge and ﬁnal training
performance. Thus, practice may be a critical method by which individuals attempt to

address discrepancies detected through metacognitive monitoring.

 

121

Implications and Directions for Future Resear_c___h

While much of the extant research on metacognition has largely ignored the role
of motivational characteristics, the results of this study indicate that motivation is an
important determinant of metacognitive activity during training programs. Mastery
orientation and self-efﬁcacy were both found to impact metacognitive activity. However,
the nature of these relationships still remains to be fully understood, particularly on a
longitudinal basis. Although mastery orientation was positively related to metacognitive
activity, this effect disappeared aﬂer its relationship with pre-training metacognitive
skills is taken into account. This suggests that at least part of mastery orientation’s effect
on metacognitive activity in a given situation is due to metacognitive skills that are
developed over time. Future research should examine this possibility by examining the
relationship between motivational characteristics, such as mastery orientation, and
metacognitive activity longitudinally over time.

The interaction observed between pre-training self-efﬁcacy and metacognitive
training on metacognitive activity is of both conceptual and practical signiﬁcance. From
a practical perspective, this ﬁnding suggests that metacognitive training of the type
utilized in this study may serve the greatest beneﬁt to those who typically struggle in
learner control environments, helping these individuals to take advantage of the control
granted to them. In so doing, such interventions can help a broader range of learners
capitalize on the beneﬁts of learner control. From a conceptual perspective, it suggests
that the metacognition is somewhat malleable and can be inﬂuenced, at least for some
individuals. It also suggests that the some of the detrimental effects of low self-efﬁcacy

can be averted in certain contexts. Further research should be conducted to determine the

122

 

precise mechanisms by which the metacognitive training overcame low trainee efﬁcacy
and the speciﬁc components of the metacognitive training that were responsible for this
effect.

In contrast to some of the previous studies using adult learners (e.g., Bean et al.,
1986; King, 1989; King, 1992; Murphy et al., 1987), the metacognitive interventions
utilized in this study did not exert a direct effect on learning outcomes. As predicted,
metacognitive activity was positively related to post-training self-efﬁcacy, but was not
related to declarative knowledge, skill-based performance, or application performance, as

had been expected. One reason for the differences may be the scope of the interventions,

 

both with regard to time and the number of components of metacognition addressed.
Most of the extant studies examining interventions have taken place over relatively
extended periods of time, most often spanning weeks or months, whereas the
interventions in the current study took place over the course of a few hours. Second,
. most metacognitive interventions address multiple components of metacognition, such as
planning, monitoring, and evaluation, whereas the interventions in this study focused
primarily on monitoring. Although the simpliﬁed and focused nature of these
interventions may have been best suited to the short time period during which they were
expected to exert their effect, they may not have the same strength of inﬂuence as
interventions that utilize multiple components addressed over an extended time period.
Future studies should seek to determine the optimal duration and scope of metacognitive
interventions to achieve the desired combination of efﬁciency and utility.

Additional directions for future research have been suggested throughout, but a

few seem particularly intriguing. First, the interplay of feedback and practice

 

123

opportunities with metacognition should be examined. While one could assert that
metacognition is particularly important when these features are not present, the case
could also be made that metacognition cannot have as beneﬁcial an impact without
feedback and/or practice. Given the central role that these two features often play in
training programs and questions that have arisen concerning the conditions under which
feedback interventions may prove effective (Kleuger & DeNesi, 1996), examination of
metacognition within these contexts may yield interesting and important information for
theory and practice.

The current study examined metacognition within a learner control training
program. Again, this was expected to increase the importance of metacognition, as
participants must make decisions regarding where to devote their time and effort. Future
research should contrast learner control with program control to determine if trainee
characteristics, design features, and learning processes function differently in these two
training settings. Under program control, the guidance built into training may
compensate for lack of trainee motivation and/or skill. In contrast, under learner control,
trainee characteristics and skills may play a much larger role. However, these
propositions remain in the reahn of speculation. As organizations continue to shift
toward learner control training, the need to understand the critical n'ainee characteristics,
design features, and learning that maximize learning and, most importantly, transfer of

training back to the job will become critical for organizational success.

124

APPENDICES

125

 

APPENDIX A: INFORMED CONSENT AND DEBRIEFING FORMS

126

 

Informed Consent

 

Explanation of research

The learning behaviors of trainees in a web-based training
program will be examined.

 

Procedures and estimate of time

You will complete a training program that will teach you
how to create a web page. You will have opportunities to
practice these skills. At the end of the study, you will be
asked to create and submit your own web page. The study
is expected to take 3 hours.

 

Participation

Participation in this study is voluntary.

You may choose not to participate in some or all parts of
the study.

You may discontinue the experiment at any time without

penalty.

 

Conﬁdentiality

Your privacy will be protected to the maximum extent
allowable by law. Data gathered from you during this
study will be strictly conﬁdential. Your responses will
remain anonymous in any research reports. At your
request, the result will be made available to you.

 

Risks and costs

There are no risks or costs associated with your
participation

 

 

Principal investigator

Head of the Department of
Psychology

University Committee on
Research Involving Human
Subjects

 

Aaron M. Schmidt, schmil64@msu.edu
432—7069

Dr. Gordon Wood
355-9563

David E. Wright
355-2180

 

Agreement to Particpate

The procedures and possible risks of the experiment have been explained.
Do you understand and fully consent to participate in the study described above?

[:I Yes
[:I No

If you makred "Yes," please enter you PID number: A********

If you marked "No," please exit the experiment at this time.

There are a number of books at you local library from which you can learn
the same information without participating in a research study.

127

 

Debrieﬁng

The study in which you just participated was designed to examine how people learn
complex tasks such as web page designing. The investigator is also examining the effects
of metacognition (monitoring and regulating your own learning) on learning processes
and training performance.

If you have any questions about this study or would like to receive a copy of the results
when they are complete, please notify the investigator now, by phone at 432-7069, or by

e-mail at schrni164@msu.edu.

Thank you for participating in this study.

128

APPENDIX B: PRE-TRAINING MEASURES

129

 

Prior Experience Scale

All the Time Frequently Sometimes Seldom Never

9999:5939!“

I spend time making my own page for the web.

I surf the web for enjoyment.

I use the intemet to ﬁnd information for work or classes.

1 have taken courses on the intemet.

Making my own web page is something I've thought about doing.
I purchase products or services over the intemet.

When I surf the web, I follow links to explore.

I use the intemet to learn new things.

Please estimate the number of hours you have spent designing and creating web pages:

Self-Efﬁcacy

Strongly Agree Neutral Disagree Strongly

Agree

95”.“???pr

Disagree

Compared with others in this training program, I expect to do well.

I‘m certain I can understand the ideas taught in this course.

I expect to do very well in this training course.

Compared with others in this course, I think I'm a good trainee.

I'm sure I can do an excellent job on the tasks assigned in this training course.

I think I will perform well in this course.

My learning skills are excellent compared with other trainees in this course.
Compared with other trainees in this course I think I know a lot about the subject.
I know that I will be able to learn the material for this training course.

130

 

Mastery/Performance Orientation

Strongly Agree Neutral Disagree Strongly
Agree Disagree

1. The opportunity to do challenging work is important to me.

2. I do my best when I'm working on a fairly difficult task.

3. I try hard to improve on my past performance.

4. When I have difﬁculty solving a problem, I enjoy trying different approaches to

see which one will work.

The opportunity to learn new things is important to me.

The opportunity to extend the range of my abilities is important to me.

I prefer to work on tasks that force me to learn new things.

When I fail to complete a difﬁcult task, I plan to try harder the next time I work

on it.

9. The things I enjoy the most are the things I do the best.

10.1 feel smart when I can do something better than most other people.

11. I like to be fairly conﬁdent that I can successfully perform a task before I attempt
it.

12. I am happiest at work when I perform tasks on which I know I won't make any
errors.

13. I feel smart when I do something without making any mistakes.

14. I prefer to do things that I can do well rather than things that I do poorly.

15. The opinions others have about how well I can do certain things are important to
me.

16. I like to work on tasks that I have done well on in the past.

?°>‘.°‘S"

Conscientiousness
Strongly Agree Neutral Disagree Strongly
Agree - Disagree
1. I keep my belongings clean and neat.
2. I'm pretty good about pacing myself so as to get things done on time.
3. I am not a very methodical person.
4. I try to perform all the tasks assigned to me conscientiously.
5. I have a clear set of goals and work toward them in an orderly fashion.
6. I waste a lot of time before settling down to work.
7. I seldom notice the moods or feelings that different environments produce.
8. When I make a commitment, I can always be counted on to follow through.
9. Sometimes I'm not as dependable or reliable as I should be.

10.1 am a productive person who always gets the job done.
11. I never seem to be able to get organized.
12. I strive for excellence in everything I do.

131

H

 

Metacognitive Skills

Strongly Agree Neutral Disagree Strongly
Agree Disagree
Monitoring

1. I think about how well my tactics for learning are working.

I monitor how well I am learning the requirements of the material I am studying.
I think carefully about how well I have learned material I have previously studied.
I evaluate how well I am learning the skills of I am trying to develop.

I think about what skills need the most practice.

I try to monitor closely the areas where I need the most improvement.

I ask myself questions to make sure I know the material I have been studying.
I try to determine which concepts I don't understand well.

I think about what things I need to do to learn.

199°99‘95“?!"

Control
10. I experiment with different procedures for learning.
11. When choosing what information to study, I consider how the information will
help me to learn the skills I am trying to develop.
12. I carefully select what to focus on to improve on weaknesses I identify.
13. I use my past performance to revise how I approach learning situations.
14. I stop once in a while and go over what I have read.
15. I spend more time studying topics that I ﬁnd to be more difﬁcult.
16. I continue to study topics I am trying to learn until I fully understand them.
17. I spend additional time looking over information that I have studied earlier.

Delayed Judgments (control)
18. I wait a brief period of time aﬁer studying something before trying to determine
how well I have learned it. '
19. I distract myself momentarily after studying particular pieces of information
before thinking about how well I know it.
20. I ask myself questions about material that I have studied earlier.

132

Demographic Information

1. What is your gender?
1 = Female
2 = Male
2. What is your age?
1 = less than 18
2 = 18 - l9
3 = 20 - 21
4 = 22 - 23
5 = greater than 23
3. What is your year in college?
1 = First-Year
2 = Sophomore
3 = Junior
4 = Senior
5 = Other
4. What is your race?
1 = African-American
2 = Asian
3 = Hispanic/Latino (N on-white)
4 = White
5 = Other
5. What is your overall grade point average (GPA)?
1 = 0.0 - 1.0
2 = 1.1 - 2.0
3 = 2.1 - 3.0
4 = 3.1 - 4.0
5 = greater than 4.0
6. Please enter your ACT or SAT total test score. If you took both exams, please
enter your score on the ACT exam. _

 

 

 

133

APPENDD( C: WEB PAGE TRAINING

134

 

 

First Steps

Getting started

Opening Netscape Composer

For this training program you will need to use Netscape Composer to create your
web page from scratch. Right now, open Netscape Composer by clicking on the
Communicator menu above, and then clicking on Composer. Now that Composer is
open, you are ready to begin creating your web page. After you have opened Composer,
save your document as an HTML document. To do this you will pull down the File menu
and choose Save as..- Name your ﬁle "pagel " and save it to the "we " folder. In the box
that appears asking for a page title, click OK.

You have just created a web page! Nothing too exciting yet, just a blank white page. In
the sections that follow, you will receive instruction on how to add text, links, graphics,
and more to give your page a little life."

 

Choosing the content

One of the ﬁrst steps in creating a web page is decide what you will put on your web
page. A web page can be about anything you choose. A few ideas include special
interests, an or area of expertise you have, your family, TV shows, your favorite music,
favorite sport or team, your resume, a page advertising or selling some product or
products, or anything else you can imagine. Any of these things will add value to your
web page.

Second, think about your audience. Who do you want or expect to be visiting your web
_ page? Will it be other people who share your interest? Or will it be mostly family and
ﬁiends who want to see the latest photos you have taken? If your visitors will be family
members who rarely use the Internet, you may want to set up your web page differently
than if you are trying to appeal to a high-tech audience.

Once you have decided on the general content of what you want to showcase on your
web page, begin thinking about how you want it to appear on the page. Also be thinking
about and noting the addresses for other web sites that are related to or will support the
content on your web page. This information will be useful when you begin to add links
on your web page.

135

Design rules

If you've spent much time exploring the Web, you've almost certainly encountered some
badly designed Web sites. Since almost anyone can create Web pages, it's not surprising
that many sites are confusing, overwhelming, ugly, or incredibly slo-o-o-ow. Tantalizing
content can be hidden forever from the world when a site is poorly designed.

While it's not easy to create a well-designed site, following a few rules can help. The
following set of rules should help you to create pages that are much better organized and
easier for you visitors to appreciate:

0 Keep your pages short (200 - 500 words) and focused. If you have several
topics on your web page that are of each of moderate length (200-500 words), you
should create a separate web page for each topic, along with an index page that
has links to each of the topics. For example, each topic in this training program is
on a separate page.

0 Maintain consistency among your pages. When your web site consists of
more than one page, it is important to keep the style (colors, layout, titles, etc.) of
all the pages similar. Because one mouse-click can send someone to a new site at
any time, a consistent style provides a signiﬁcant visual cue to let people know
they're still at the same site. This has been done with this training program--all the
pages maintain a similar look format.

0 Use subheads. Insert subheads to break up large blocks of text and make
each section stand out. Subheads will draw the viewers' attention to the key
sections of your page and allow them to ﬁnd the information they need more
quickly.

Background, Text, & Other Page Basics
Web page title

Internet browsers typically have a "title bar" at the very top of the browser window. This
title bar will display the title of the web page currently in the browser window. For
example, the title displayed in the title bar for this page is "Web Page Training."

It is important to enter a carefully worded title, because the title describes your page in
many ways. Web directories such as Yahoo! use the title as a primary reference for what
the page is about. So, if you give your page a poorly worded title, it may not come up
when folks search on the very topic your page is all about.

136

 

An effective title should accurately describe the content of the page in as few words as
possible-no more than 6 to 8 words. The most important aspect of the web page content
should be included in the title.

How to Create a Title.

Netscape Composer will ask you to enter a title the ﬁrst time you save your ﬁle. By
default, it will list the document name as the title, but you can change this to whatever
title you choose.

You can easily change the title of a page you have already developed. Here is how:

1. In Composer, choose Format, Page Colors and Properties. [INSERT PICTURE].
2. On the general tab, ﬁnd the page's current title in the box marked Title
[INSERT PICTURE].
3. Click in the Title box and type in your new title.
4. Click OK.

Page colors

By default, the visitor's web browser chooses the colors for the text, links, and
background of a page. Generally, the default background will be white, the main text
will be black, and the hyperlinks will be in blue.

If you want to control how the page will appear to others, you can specify the colors you
want to be applied to your page, rather than letting the visitors web browser apply the
default colors. This adds uniqueness and personality to your page. Using custom colors
has an immediate and noticeable effect on the appearance of your page. Plus, changing
the colors is very easy to do.

You can assign custom colors to each of the following page elements:

Normal Text: All text in the page that is not a link.

Link Text: All links in the page, except those that are active or followed
(described next).

Active Link Text: A link is active for as long as a visitor is clicking it, from the
time they press the mouse button until the mouse button is released. This
speciﬁes what color the link will be while it is being clicked.

Followed Link Text: These are links that the visitor has previously used through
his or her browser. Usually this color is different ﬁom the color of the hyperlinks,
so that visitors to your web page will know what links they have already clicked
on.

Background: This is the area that sits behind the text or images in the page. The
background never covers or affects other elements in the page.

137

Here's how to change the background color:

1. In Composer, choose Format, Page Colors and Properties. [INSERT

PICTURE].
2. On the Colors and Background tab, select Use Custom Colors (Save Colors
in Page) [INSERT PICTURE].

3. Click the colored box just to the left of the words Background (Background
image overrides this color) to open a selection of color options [INSERT

PICTURE]
4. Click on the color that you want to use for the background.
5. Click OK.

To change the other colors, such as text colors and link colors, repeat these steps,
only clicking on the appropriate box in step 3.

Keywords

Most people who publish a web page want others who have an interest in the page's topic
to ﬁnd and view their page. Web search pages like Yahoo! (www.yahoo.com) and Excite
(www.excite.com) allow people to search the web for pages that cover particular topics
they are interested in. If you web page is about classic cars, you would like your web
page to be found when someone enters search terms related to your site, like "vintage
cars," etc.

Just as having the right title will help visitors ﬁnd your page when they search a topic,
you can also use "keywords" to further help interested visitors ﬁnd your web page.
Keywords do not appear anywhere on your web page. You specify them in the same
manner as the title of your web page. Adding keywords related to the content of your site
increases the chances that those interested in your web page's topic will be able to ﬁnd
your page through searches. You can have just about as many keywords as you can think
of, and the more speciﬁc names and phrases you include the better your web page will
fare in search results.

Here's how to add keywords in Composer:

1. If it is not already, open the page for which you want to add keywords.
2. In Composer, choose Format, Page Colors and Properties. [INSERT

PICTURE].
3. On the META Tags tab, click inside the User variables (META tag): box
[INSERT PICTURE].

4. Click in the Name box and type the word keywords

5. Click in the Value box and type your keywords. Put a comma (but no
spaces) between keywords [INSERT PICTURE].

6. Click OK.

138

Formatting text

One of the most important elements of any web page is, of course, the text. Although the
text generally conveys the content of your page, the ideas and information you want to
share with your visitors, it can also contribute to the look or aesthetic appeal of your
page. Netscape Composer allows you to modify a number of properties that determine
the look of your text.

Here is how to format text:

 

Font:

1. Select the characters you want to format.

2. Click the arrow on the right side of the Font list box [INSERT

PICTURE].

3. Click the name of the font you want to apply.

Size:

1. Select the characters you want to format. It
2. Click the arrow on the right side of the Font Size list box [INSERT '
PICTURE].

3. Click the font size you want to apply.

Color:

1. Select the characters you want to format.

2. Click the arrow on the right side of the Font Color box [INSERT

PICTURE].

3. Click the color you want to apply to the text.

Bold:

1. Select the characters you want to format.

2. Click the Bold icon [INSERT PICTURE].
3. To un-bold, repeat steps 1 and 2.

Italicized:

1. Select the characters you want to format.
2. Click the Italic icon [INSERT PICTURE].
3. To un-italicize, repeat steps 1 and 2.
Alignment:

1. Select the characters you want to format.

2. Click the Alignment icon on the far right [INSERT PICTURE].
3. Click the Align Left, Center, or Align Right icon [INSERT PICTURE].

 

139

Graphics

Background images

One of the easiest and most common places to use graphics is as the background of your
page. Instead of having a plain color for the background of your web page, you can have
a texture, or a picture that will cover the entire background area. Some textures include
marble, wood,

burlap, and water droplets. Click on the links above to see examples of background
images [add examples].

Just be sure that you choose a text color that can be seen over the background texture. A
backgound image that has a lot of contrast (a combination of light and dark colors) will
make it more difﬁcult to read the text that is placed over it, so choose your backgound
carefully.

Background images are "tiled" by the browser, meaning that the image is repeated across

the entire backgound. This means that even a very small image will ﬁll the entire
backgound. Click the following image to see it as a background.

To create a backgound image:

1. Make sure the image you want to use is in the same directory (folder) as

your web page.
2. In Composer, choose Format, Page Colors and Properties. [INSERT
PICTURE].

3. On the Colors and Background tab, under the heading Background image,
select Use Image [INSERT PICTURE].

4. Type in the ﬁlename of the image you want to use, or click on the Choose
File... button and select the image.

5. Click OK.

Horizontal lines

Another simple type of graphic to insert on your web page is called a "horizontal line".
These are the straight lines that appear on web pages that are oﬁen used to separate
different sections of the page. These can be plain gay lines like the ones in this training
progam, or can be more decorative to give the page more character, depending on the
image you want to convey.

140

To insert a horizontal line:

1. Click in you page at the spot where you want to insert the line.

2. Click the Insert Horiz. Line icon [INSERT IMAGE], or click Insert on the
text menu, then Horizontal Line.

Now that you have a line, you can format the line to customize the look of it. To

format a line:

1. Click on the line.

2. From the text menu, click Format, then Horizontal Line.

3 Set the following options as you prefer: [INSERT IMAGE]
0 Dimensions

0 Height: How thick the line is.

0 Width: How far across the page the line will ﬁll, in percents.
o Aligrment

0 Left: Line is positioned on the left side of the page

0 Center: Line is positioned in the center of the page

0 Right: Line is positioned on the right side of the page
0 3-D Shading: Gives the line a three-dimensional look.

 

Inserting and Resizing Images

Most webpages include at least some minimal gaphics. Graphics include both photos
and clip-art drawings. An example is the picture in the upper left corner of this page.
[INSERT IMAGE].

To insert a picture:

1. Make sure the image you want to use is in the same directory (folder) as
your web page.

2. Click in you page at the spot where you want to insert the image.

3. Click the Insert Image button on the toolbar [INSERT IMAGE] or on the
text menu up top, click Insert, then Image. .. A box will open that looks like
this: [INSERT IMAGE].

4. Select the image you want to insert by clicking on "choose ﬁle." Select the
ﬁle and click "OK". The name of the ﬁle will now appear in the white text
box under the heading Image Location

5. Click OK.

OK, so now you've got a picture on the page, but it may be either bigger or smaller than
you want. You can resize the picture to ﬁt the space you want. Keep in mind, however,
that image quality may not be as good when the image is resized, so experiment and see
what looks best to you.

 

141

 

To change the size or shape of an image:

1.
2.

Click on the image to select it (a border will appear around the image).
Position your cursor in one of the comers. The arrow-shaped cursor will
change to a double-headed arrow when you are in the right spot [insert
image]

Click and Hold down the left mouse button and drag the corner in towards
the center of the picture to reduce the size of the image, or out to increase
the size.

You can also resize a picture ﬁom the top, bottom or side edges, but if you do,
you will lose the original proportions of the image. You will retain the original
proportions if you click and drag ﬁ'om the corners.

Aligning text with graphics

By default, the ﬁrst line of text immediately following the picture will appear to the right
of the picture, near its bottom. Any text that goes for more than one line will appear
below the image (example: [INSERT IMAGE]). However, you can specify where you
want the text to appear relative to the image. You can also have the text appear all
around the image.

To change the text alignment and wrapping around the images:

p—a
O

5.

Click on the image to select it (a border will appear around the image).
Click the Insert lrnage button on the toolbar [INSERT IMAGE] or, on the
text menu up top, click Insert, then Image... [INSERT IMAGE].

In the middle of the box that will appear are the options for text aligrment,
under the heading Text alignment and wrapping around images.

Click the appropriate button for the text aligrment you want (click images
below for an example or each).

Top alignment [INSERT IMAGE]

Absolute middle aligrment [INSERT IMAGE]
Middle alignment [INSERT IMAGE]
Absolute bottom alignment [INSERT IMAGE]
Bottom alignment [INSERT IMAGE]

Text wrap left [INSERT IMAGE]

Text wrap right [INSERT IMAGE]

Click OK.

142

Linking to Stuff
Understanding links

One of the main features of a web page is the fact that it contains links to other sites on
the web. A hyperlink is a link you create between your web page and another web page
on the lntemet. The other web page can be another one of your own web pages or it can
be someone else's web page. This is done by including the address of the other web page
in your web page.

For example, the menu on the left contains links to the pages that make up this training
progam. When you click on a link, it automatically goes to the address listed.

You may want to add links on your web page. These links can point your visitors toward
other web pages you have, or to other web pages on the lntemet that are related to the
content of your web page. For example, if you have information on your web page about
your favorite musician, you might have a link to that musician's ofﬁcial web site. Or if
your web page details information about the MSU football team, you might want to
include a link to the ESPN web page that provides the Big Ten football conference
standings, or your team's statistics. In this way, you can help your visitors ﬁnd more
information that you either don't have, or don't have the time or desire to constantly
update on your web page.

A link can point to a number of resources in addition to other sites. It can point to email
addresses, newsgoups, or ﬁles such as Word documents, sound ﬁles, videos, etc. Links
_ can also point to particular places within the same web page.

Creating text links

Links have two main parts:

0 The actual text (or gaphic) that appears on your web page
0 The URL of the page, ﬁle, to be accessed when the link is clicked.

You can format the text part of the link just as you would any other text, except that the
link will be automatically underlined once a URL is associated with it.

To create a link to another web page:

1. Select the text that you want to make into a link.
Click the Insert Link icon on the toolbar [INSERT IMAGE] or, on the text
menu up top, click Insert, then Link...

3. In the box that will appear, in the white text area labeled "link to a page
location or local ﬁle," type in the URL

of the page you want to link to (ex. http://www.msu.edu).

143

NOTE: If you want to link to a page that exists IN THE SAME FOLDER as
the page on which you are putting the link, you need only type the ﬁlename
(ex. page] .htrnl). However, if the page you want to link to is NOT in the
same folder (ex: another site on the web), you MUST type the entire path,
including the http:// portion (ex. http://www.wor.com).

4. Click OK.

You can also create links to an email addresses. When a visitor clicks on one of these
links, it will open the users default email progam with a message addressed to the
address speciﬁed in the link (ex. sample link).

To create a link to an email address:

INTI"- '

1. Select the text that you want to make into a link.
2. Click the Insert Link icon on the toolbar [INSERT IMAGE] or, on the text 3

menu up top, click Insert, then Link... .
3. In the box that will appear, in the white text area labeled "link to a page

 

location or local ﬁle," type "mailto:" ,_
followed immediately by the email address (ex. mailto:sparty@msu.edu).
4. Click OK.

To remove a link, hold the mouse pointer over the link and click the right mouse
button. Select "Remove Link" from the menu.

Creating links in graphics

You may want visitors to be able to link to another web page by clicking on a gaphic
rather than by clicking on text. (You can learn more about inserting gaphics on your web
page in another lesson.) For example, if you have a photo of Beaumont Tower on your
web page, you can embed a link to the MSU home page in the gaphic. That way, when a
visitor clicks on the photo they will be linked to the MSU page.

Making an image into a link is simply a matter of attaching the URL to the image.
It is done just like creating a text link, except you select an image, instead of text,

before creating the link:

1. Select the image that you want to make into a link.

2. Click the Insert Link icon on the toolbar [INSERT IMAGE] or, on the text
menu up top, click Insert, then Link...

3. In the box that will appear, in the white text area labeled "link to a page
location or local ﬁle," type in the URL

of the page you want to link to (ex. http://www.msu.edu).

144

 

 

NOTE: If you want to link to a page that exists IN THE SAME FOLDER as
the page on which you are putting the link, you need only type the ﬁlenarne
(ex. pagel .htrnl). However, if the page you want to link to is NOT in the
same folder (ex: another site on the web), you MUST type the entire path,
including the http:// portion (ex. http://www.wor.com).

4. Click OK.

To remove a link, hold the mouse pointer over the link and click the right mouse
button. Select "Remove Link" from the menu.

Creating targets

When you click on a link, the page will load showing you the top of the ﬁle in the
browser. You can then scroll down to see the rest of the page. But there may be times
when you want to create not only a link to a page, but a link to a speciﬁc part of a page
that is not necessarily the top of the page. If you have a lot of information on your web
page, you may want to create links near the top of the page that will make it easier for
visitors to get to speciﬁc information located further down on your web page.

This is what targets do. They take a visitor to an exact spot on a page, rather than just to
the top of the page. Click here for an example [INSERT EXAMPLE]

Creating targets is a two-stage process. First, you must create the target in the
destination page (the page the link will take the visitor). Second, you must create a link
to that target. The link can be in the same page as the target, or it can link to a target on a
different page.

To create Targets in a page:

1. Open the page you want the link to lead to (it can be the same page as the
link itself).

2. Click a spot where you would like a link to lead.

3. Click on the Insert Target icon [INSERT IMAGE], or click on the Insert
menu, and then click on Target. The Target Properties box will pop up:

[INSERT IMAGE]
4. In the Target Properties box, type in a name for the target.
5. Click OK.

145

 

To create a link to a Target:

1.

2.
3

6.

Open the page that you want to put the link in (it can be the same page as

the target).

Select the text that you want to make into a link.

Click the Insert Link icon on the toolbar [INSERT IMAGE] or, on the text
menu, click Insert, then Link...
If you are linking to a target on a DIFFERENT page, in the box that will
appear, in the white text area labeled "link to a page location or local ﬁle,"
type in the ﬁlename of the page you want to link to (ex. pagel .htrnl).

If you are linking to a target on the SAME page, then skip this step.
In the white box toward the bottom, under the label "Select a named target
in current page (optional), click on the name of the target you want to link

to [INSERT IMAGE].
Click OK.

To remove a link, hold the mouse pointer over the link and click the right mouse
button. Select "Remove Link" ﬁom the menu.

Table Basics

Tables

Aligrment of text and images is a serious problem in web documents. Tabs don't work in
.web pages like they do in word processing progams like Microsoft Word. You can't use
tabs to line up columns, for example. You can't just keep hitting the space bar, either. So,
if you want to format any information or pictures in columns, you will need to use a

table.

For example, here is one way a table can help you format your web page:

 

 

 

 

Favorite players
Baseball Basketball Football
Manny Grant Hill Terrell Davis
Ramirez

 

 

 

In a normal document, you would use tabs to create the columns, but in an HTML
document you must use tables to create the columns.

A table, regardless of the medium in which it appears, is chunks of information arranged
in rows and columns. The gid of rows and columns forms the cells in which you can
organize text. A cell is the box made ﬁom the intersection of a row and column. You

can put text or images in a table cell.

146

 

A basic table is easy to create in Netscape Communicator:

1. Click the spot on you page where you want to insert a table.
Click the Insert Table icon on the toolbar [INSERT IMAGE] or, on the text
menu, click Insert, then Table...
3. At the top of the box that will appear, set the number of rows and columns
you will need.
Click OK
Click in any cell and enter text or pictures as you would anywhere else on
your page.

9'!"

Once you have created a table, you can easily add or delete rows and columns. To add
more rows, you simply hit the tab key when you are in the last cell (bottom-right cell) in
the table and another row will appear. On the text menu, click Insert, Table, then
Column.

To remove a row or column, or entire table:

1. Click in the row or column you want to delete.

2. On the text menu, click Edit, Delete Table.

3 Click on Table to delete the entire table, row to delete just the selected row,
or column to delete just the selected column.

Table Borders

You can choose to have a border that surrounds the table to give it a "ﬂamed" look. For
example, here is the same table with [INSERT TABLE] and without borders [INSERT
TABLE]. You can also adjust the size of the border around your table. When you have
no borders on a table, you will still see a blue double dashed line around the table when
working in Composer [INSERT IMAGE]. This is to help you work with you table and
will not be seen visible when the page is viewed in a web browser.

To modify table borders:

1. Click anywhere inside the table you want to format.

2. On the text menu, click Format, Table Properties. [INSERT IMAGE]

3. Click on the Table tab at the top of the box.

4. Click in the box labeled "Border line width:" and enter a number for the

width of the border.

NOTE: By default, the border size is set to 1 pixel. Choose a larger number
for a bigger border. Type 0 if you do not want a border around your table.
5. Click OK.

147

Table and Cell Colors

By default, the backgound for a table and all of its cells will be transparent, meaning that
it will take on whatever backgound you have speciﬁed for the page. However, you can
assigr custom colors to the table as a whole, or to individual cells within the table, that is
different from the rest of the page. This can add emphasis to the table and its contents
and help it to stand out from the rest of the page.

To change a table's backgound color:

Click anywhere inside the table.
On the text menu, click Format, Table Properties. [INSERT IMAGE]
Click on the Table tab at the top of the box.
Under the heading "Table Backgound", check the box to the left of the label
"Use Color: "
5. Click the colored box just to the left of the label "Use Color:" to open a
selection of color options [INSERT PICTURE]
. Click on the color that you want to use for the backgound.
5. Click OK.

PPS”?

To the backgound color of a particular cell:

Click inside the cell.
On the text menu, click Format, Table Properties. [INSERT IMAGE]
Click on the Cell tab at the top of the box.
Under the heading "Cell Backgound", check the box to the left of the label
"Use Color: "
5. Click the colored box just to the left of the label "Use Color:" to open a
selection of color options [INSERT PICTURE]
. Click on the color that you want to use for the backgound.
5. Click OK.

:P‘P’PT‘

Table Width and Alignment

By default, the table will be the full width of the page. However, you can make you
smaller than the width of the page.

The primary way to set the table width is as a percentage of the page width. For example,
you could set the table to be 80% of the page width. Then, whatever size a visitor's
browser window, the table will be 80% as wide. When the page is made smaller, the
table gets proportionally smaller, and vice versa.

148

To change a table's width:

Click anywhere inside the table.

On the text menu, click Format, Table Properties. [INSERT IMAGE]

Click on the Table tab at the top of the box.

Make sure the box to the left of the label "Table width:" is checked (by

default, this should already be checked).

5. Click the inside the box to the left of the label " Table width:" and type in the
percentage of the window you want the table to ﬁll.

6. Click OK.

FPPT“

If the table width is set to less than 100% of the page width, you will need to decide
where you want the table to be positioned (or aligred). Do you want the table to be
positioned on the left of the page (left aligiment), on the right side of thepage (right
aligrment), or in the center of the page (center aligrment)? By default, the table will be
aligned on the left side of the page.

To change a table's aligrment:
1. Click anywhere inside the table.
2. On the text menu, click Format, Table Properties. [INSERT IMAGE]
3. Click on the Table tab at the top of the box.
4. Under the heading "Table Alignment", choose left, right, or center.
5. Click OK.
Putting it on the web
Publishing your page

To load your web page on the Internet, you must save it to your space on the MSU
intemet server. All MSU students are provided with space on the MSU intemet server to
which they can post a web page.

If you haven't already converted your page to HTML format, you will need to pull down
the File menu and choose Save as HTML. If you have it in HTML format, you only need
to choose Save. In either case, save your document to the "web" folder and give it the
name "index".

Any gaphics you have placed on your web page should automatically save to the "web"
folder.

Publishing your page Using FTP progams
FTP stands for File Transfer Protocol. This progam allows you to transfer documents

ﬁom one ﬁle to another on a network to which you have access. All MSU students have
access to the MSU server through the pilot network.

149

The FTP progam you will be using is a DOS command-based progam. You have to
know the right commands in order to transfer ﬁles.

To upload you web page to the MSU server using FTP:

Click on the Windows "Start" button, then click Run...

In the Run box, type ftp and press enter.

At the "ftp>" prompt, type: open

After the word "To", type: pilot.msu.edu

After "User (pilot.msu.edu: (none)):" type your pilot e-mail login.

When it prompts you for your password, type that in.

At the next "ftp>" prompt, enter the following: cd web and press enter.
At the next "ftp>" promt type: put a:\index.html (or whatever the path name
is for where your web page ﬁle is saved; i.e. c:/mydocuments/index.html)
and press enter.

9. Now your web page is online.

?°>‘.°‘.V':“S”E\’:‘

The last step is to upload an gaphics you have on your page. You do this in the same
way.

At the "ftp>" prompt type: put c:\graphicnarne.gif (or the path name for your
graphic)

Just be sure to substitute the actual name of your gaphic, and to know whether it
is a .gif or a .jpg ﬁle.

Testing and troubleshooting

Once you have loaded your web page onto the lntemet, you should check it to make sure
all of the text and gaphics look the way you want, and that all of the links work.

If one of the links doesn't work, hold the mouse pointer over the link until the address
appears. Try to ﬁgure out why the link isn't working. Is the address correct? Did you
remember the "http://" preﬁx?

If your gaphics don't work, or your page didn't load at all, try to ﬁgure out why. Make
sure you typed in the correct ﬁle names when you loaded them. Try loading them again.

After making corrections, you will need to press the "Refresh" key on your toolbar to see
the changes take effect when you check your page again on the lntemet.

Keep checking and troubleshooting until your web page looks just right. This may take a
few tries.

150

Browser compatibility

An important thing to keep in mind when creating web pages is that the same page may
look very different in different browsers, and certain features that work geat in one
browser may not work at all in another.

The two most commonly used web browsers are Netscape Navigator and Microsoft
lntemet Explorer. You should test you page out with each of these browsers to make
sure that it displays as you intended.

Another point to consider is that different versions of the same browser may have
markedly different capabilities. You must be aware that some of the features of your
page may not work the same, if at all, on an older browser. For example, Netscape
Navigator 4.7 has more feature than does Netscape Navigator 3.2. Many intemet users
still surf with older browsers, so it is advisable to test your page with older browsers, as
well.

151

 

 

 

APPENDIX D: POST-TRAININ G MEASURES

152

 

 

Self-Efficacy

Strongly Agee Neutral Disagee Strongly
Agee Disagee
l. I have mastered web-page creation.
2. I do not create web pages as well as I would like.
3. I am certain that I can create web pages well.
4. It is just not possible for me to create web pages at the level I would like.
5. I think my performance in creating web pages could be improved substantially.

Metacognition
For each of the items below, rate the extent to which you were thinking about these issues
during the training.

Strongly Agee Neutral Disagee Strongly
Agee Disagee
Monitoring

1. I thought about how well my tactics for learning were working.

2. I monitored how well I was learning the requirements of the training progam.

3. I thought carefully about how well I had learned material I had previously studied.

4. I evaluated how well I was learning the skills of web page design.

5. I thought about what skills needed the most practice.

6. I tried to monitor closely the areas where I needed the most improvement.

7. I asked myself questions to make sure I knew the material I had been studying.

8. I tried to determine which concepts I didn't understand well.

9. I thought about what things I needed to do to learn.

Control
10. I experimented with different procedures for learning.
11. When choosing what information to study, I considered how the information
would help me to learn the skills of web page desigr.
12. I carefully selected what to focus on to improve on weaknesses I identiﬁed.
13. I used my past performance to revise how I approached this training progam.
14. I stopped once in a while and went over what I had read.
15. I spent more time studying topics that I found to be more difﬁcult.
16. I continued to study each topic until I fully understood it.
17. I spent additional time looking over information that I had studied earlier.
Delayed Judgments (control)
18. I waited a brief period of time after studying something before trying to determine
how well I had learned it.
19. I distracted myself momentarily after studying particular parts of the training
before thinking about how well I knew it.
20. Later on in training, I asked myself questions about material that I had studied
earlier in training.

153

 

Web Page Exercises
(Skill-Based Performance)

Instructions: Start Netscape Composer and perform each of the following tasks:

1.

PWNQP‘PP’E"

y—A
.9

11.
12.
13.

14.
15.
16.
17.

Save this page as student ID.html (ex. if student id is A12345678, save as
12345678.html).

Make the title of this page "Creating Web Pages"

Change the backgound color of this page to a shade of blue.

Change the color of the normal text to White.

Change the color of the following link to Yellow: Link 1

Insert the following Keywords into this page: Michigan State University
Format all the text on this line so that it is a) bold, b) italicized, and c) size 16.
Center this line:

Insert a horizontal line directly below this text:

Edit the horizontal line below by a) making the line thicker, b) set the width to 50%
of the window, and 0) position it on the right side of the window.

 

Decrease the size of the image below: [Insert Image]
Below this line, create a text link to www.msu.edu

Create a link in the image below, so that it will link to the email address
sparty@msu.edu [INSERT IMAGE]

Remove the following link, without deleting the text itself: Delete this link

Directly below this line, create a link to question #4 on this page.
Directly below this line, create a table with 3 rows and 4 columns.

On the table below, a) create a table border, b) change the table's backgound to
gay, c) add another column, and d) set the table width to 80% of the window.

 

Favorite players

 

Baseball Basketball Football

 

 

Manny Ramirez Grant Hill Terrell Davis

 

 

 

 

154

 

Declarative Knowledge

1.

Which is not a good reason to place a link on your web page?

a. To point to extra information on another web page

b. To point to another web page related to content on your web page
c. To connect to another one of your own web pages

(I. You can’t think of anything else to put on your web page

How does a hyperlink work?

a. It uses the address of another web page

b. It uses the title of another web page

c. It uses meta tags from another web page

d. It uses magic

How do you place a link on your web page?

Edit, Paste as Hyperlink

Insert, Link...

Right mouse button, Edit Hyperlink...
Right mouse button, Select Hyperlink

.o-e 9‘s»

Where do you type the link address in the dialog box?

a. Link to a page location or local ﬁle”
b. Path:

c. Base:

(1.

Named location in ﬁle:

How do you know a link is in place on your web page?

a. You can't tell

b. It is underlined

c. It is a different color

d. It is underlined and a different color

How do you remove a link from your web page?

Edit, Hyperlink...

Insert, Hyperlink...

Right mouse click, Hyperlink, Remove Link
Right mouse click, Hyperlink, Select Hyperlink

999‘.”

155

10.

ll.

12.

How do you edit a link on your web page?

Edit, Hyperlink...

Insert, Link...

Right mouse click, Hyperlink, Edit Hyperlink...
Right mouse click, Hyperlink, Select Hyperlink

9.09:.»

What name should you save your ﬁrst web page document under?

a. user
b. index

c. any name you want

d. Your pilot login name

In what folder should you save your web pages?

a. web

b. w

c. snapshots
d. any folder

How many web pages can you have on the MSU server system?

only one, and it must be named "index.htm "

several web pages; the one named "index.html" is your home page
only two; "index.html" and one other web page

only one, but it can have any name you want

9.0 9‘!»

How are gaphics ﬁles uploaded to the lntemet?

a. by saving each one separately

b. by saving them as a goup of ﬁles

c. automatically when the htrnl ﬁle is saved
(1. only by using FTP

How do you open a new browser window to locate your web page?

New, Browser window...
Tools, New window...
Edit, New, Window
File, New, Window

9‘.“ 9‘!”

156

l3.

14.

15.

16.

17.

18.

What do you need to know to ﬁnd your web page on the lntemet?

a. The page title

b. The document name
c. The address

(1. Your pilot password

Which of the following is the correct preﬁx for an MSU student's web page
address?

a. http://www.msu.edu/user/
b. http://www.msu.edu/

c. http://www.msu.edu/pilot/
d. ﬁp://www.msu.edu/user/

Which statement about changing the backgound colors of a web page is true?

a. You can change the colors, but there are only a couple from which to
choose

b. You can change the colors, and it is easy to do

c. You can change the colors, but it is difﬁcult to do

<1. You can't change the colors

Which menu is used in order to change the backgound colors on a web page?

a. Edit

b. Insert
c. Format
d. Tools

Which sets the colors for all of the non-linking text that you write?

a. Body text

b. Normal text

c. Hyperlinks

d. Followed hyperlinks

How do you set the colors for text on a web page?

Format, Font..., Effects
Format, Style...
Format, Color

Format, Text colors...

9‘.“ 9'!”

157

 

19.

20.

21.

22.

23.

24.

Why are tables useful on a web page?

Because all text on web pages must be in tables
Because you can't use the tab key to create columns

No particular reason; they're just fun to make

Tables are not useful; you can't put tables on a web page

999‘!”

How do you place a table on a web page?

a. Table, Insert table...
b. Insert, Table, Table...
c. Insert, Object...

d. You can't

Which of the following can be placed in a table?

a. Text

b. Graphics

c. Both text and gaphics
d. Neither text nor gaphics

When should you begin planning the content for your web page?

Before you do anything else

Before you post it to the lntemet

After you post it to the lntemet and see how it looks

You don't need to plan your content as long as you have links and gaphics

9.0.0:.»

Which of the following is not something to think about when planning your web
page?

Your audience

How you want your page to look

How someone else's page looks

What links you might want on your page

999‘!”

Where does the title for your web page appear?

At the top of the web page

At the top of the lntemet browser

At the bottom of the lntemet browser
You can't see it

999‘!”

158

25.

26.

27.

28.

29.

30.

Why should you have a title for your web page?

999‘!”

There is no reason to have a title

A title is required for web pages

A title makes your web page look nicer

Search engines can ﬁnd your web page more easily

What is it important for your web page title to contain?

a
b
c.
d

Your name

The lntemet address for your web page

The most important aspect of the page's content
It doesn't really matter what is in the title

How do you create a title for your web page?

.o-p 9‘!»

Format, Page Colors and Properties, General
Edit, Preferences, Title

Insert, Autotext

Insert, Title...

What happens if you don't create a title for your web page?

a
b.
c
d

No title appears in the title bar of the browser
Your web page won't load to the lntemet
Your lntemet address is used as a title

Your document name is used as a title

What can't you do with a horizontal line?

999‘!”

Move it
Hide it
Delete it
Align it

Which of the following is not a gaphic that can be inserted on a web page?

a.
b.
c
(1

Photo
Horizontal Line
Chart
Table

159

 

31.

32.

33.

34.

35.

Which menu is used in order to place a gaphic on a web page?

a. Edit

b. Insert
c. Format
d. Tools

What can you do with a gaphic once you have inserted it on a web page?

a. Resize it
b. Change its color
0. Rotate it
d. Nothing

Which of the following is not an aligrment option for horizontal lines and
gaphics?

a Left

b. Center
c Right
(1 None

What is the maximum number of words you should put on a single web page?

a. 800
b. 500
c 100
d 300

Which of the following is a valid URL for a hyperlink?

a. http://www.msu.edu

b. www.msu.edu

c. neither A or B is a valid URL for a hyperlink
d. Both A or B are valid URLs for a hyperlink

160

Final Web Page Checklist

D 500 or fewer words?

D Subheads?

D Modiﬁed Title (not just ﬁlename)?
D Title less than 8 words?

CI Modiﬁed global normal text color?
El Modiﬁed global link color?

D Modiﬁed global active link color?
0 Modiﬁed global followed link color?
:1 Modiﬁed backgound color?

CI Backgound image?

Cl Keywords?

D Modiﬁed font?

El Modiﬁed font size?

CI Modiﬁed text color?

CI Bolded text?

:1 Italicized text?

CI Modiﬁed text aligrment?

CI Inserted horizontal line?

E1 Modiﬁed line dimensions (height or
width)?

[:1 Modiﬁed line alignment?

1:] Modiﬁed line shading?

CI Inserted image?

CI Modiﬁed image size?

CI Aligred text with gaphics?
CI Created hyper link?

CI Does link function correctly?
D Created email link?

0 Does link function correctly?
:1 Created link in gaphic?

CI Created target?

Cl Created link to target?

D Does link ﬁrnction correctly?
CI Created Table?

[:1 Modiﬁed table border?

D Modiﬁed Table backgound color?

D Modiﬁed Cell backgound color?

CI Modiﬁed Table width?

[:1 Modiﬁed table aligrment?

Total Number of items checked: __

161

 

'ﬁ

 

 

APPENDD( E: METACOGNITIVE TRAINING

162

Metacognitive Training

"Before we begin the web-page desigr training, I'd like to take a few minutes to teach you some
skills that have been shown to improve learning. I would like you to use these learning skills and
strategies to help you learn during the training progam. These skills can be used outside of this
study as well, such as in your classes.

I’m going to go over three things to teach you these learning skills. First, I’m going to provide a
brief introduction, then an example, and ﬁnally teach you some speciﬁc learning skills and
strategies for you to use while learning to make web pages.

OK, ﬁrst the introduction. "These skills are what is known as metacognition--Metacogrition is
"thinking about your thinking." It includes things such as your beliefs about your intelligence,
what types of things you are good or bad at, etc.

"Metacogrition also includes your beliefs about how well you know something that you are
trying to learn. This aspect of metacogrition is known as monitoring. Monitoring is very
important when leaming new material. It will determine, what people choose to study, and how
long they choose to study it.

"So we’ve covered two new terms — metacogrition and monitoring. Metacogrition is “thinking
about your thinking,” and monitoring is “beliefs about how well you know something that you
are trying to learn.”

"Now I’m going to go over an example to help clarify the meaning of these terms.

"Imagine that you are preparing for an exam in your psychology class that covers 3 chapters in
your textbook. You ﬁrst have to decide exactly what you will study. Lets imagine that you think
you understand the information from chapters 1 and 2, but that you don't really know chapter 3
very well yet. What would we call the judgnent of how well you know the material in each
chapter? [This is an example of monitoring]. The beneﬁt of monitoring is that you realize that
since you don't yet understand chapter 3 and you would most likely begin by studying that
chapter.

This was a relatively simple example, but monitoring involves much more than deciding what
you will study before you begin studying-it also comes into play as you are learning or
studying. To be an effective learner, you have to ﬁgure out if you understand what you are
trying to learn and spend more time on the material you don't know yet.

"Most people don't try to determine how well they know the material they are trying to learn-
Instead, they simply continue to work their way straight through the material, and never really
learn what they are trying to learn. What I would like you to do is, as you progess through the
material presented in this training progam, try to determine how well you are actually learning
the material you are studying. This is important, so let me say it again: as you progress through
the material presented in this training program, try to determine how well you are actually
learning the material you are studying

163

"As you do this, keep in mind that people almost always overestimate how well they know
something. That is, people usually think that they have learned the material better than they
actually have. You have probably experienced this yourself--you studied for a test and were sure
that you knew all the material backwards and forwards. However, when it came time to take the
test, you found that you weren't able to answer the questions correctly--you didn't actually know
the material as well as you thought that you did. If you had realized that you didn't actually
know all the material, you would have spent more time studying. Can anybody give me an
example of a time when you thought you knew something you were trying to learn, but found out
later that you didn't? [If not, give personal example]. You can see that if you think you know
something, when in fact you don't, you probably won't spend as much time studying as you
really need to.

So, I’ve introduced you to metacognition and monitoring, and we’ve gone over an example. I’ve
also asked that when we get to the training that you try to determine how well you are actually
learning the materials you are studying. The only thing left is to do is to teach you some speciﬁc
strategies to help you do this.

"What I am going to do is teach you some simple strategies that you can use to more accurately
determine how well you know something you are trying to learn. First of all, when most people
do try to monitor their learning, they do so by making quick, intuitive " gut" decisions. After
studying something, they just "feel" like they know something or not. A better strategy is asking
yourself speciﬁc questions about the material you just studied. By doing this, you can get a
better idea of how well you know it. For example, assume that you are trying to learn all of the
state capitols. Can anyone give me an example of a question you could ask yourself to see how
well you know the capitols? [If not, give example "what is the capitol of Ohio?"]. If you can't
answer your own question, then you need to spend more time studying that particular piece of
information. As you go through this training program, ask yourself specific questions about the
information you are studying. If you cannot answer your own questions, spend more time
studying that information.

"To help you create questions to ask yourself, the performance aid that I have passed out
provides ﬁll-in-the-blank questions that can be used for anything that you are trying to learn.
Can anyone give me an example of a question you could ask yourself about what I have taught
you so far, using these ﬁll-in-the-blank questions? [If not, "What is the main idea of Lhﬁ
presentation? " The answer would be that people aren't very good at determining how well they
know what they are trying to learn, and I am giving you ways to solve this problem]. As you
progress though this training program, create questions based on these ﬁll-in-the-blank
questions and see if you can answer them. If you cannot, you should spend additional time
studying the material you do not understand.

So the ﬁrst strategy I’ve taught you is to create questions and see if you can answer them. The
second thing I want to teach you is when to use these questions.

164

 

"Even when asking themselves questions like these, most people still tend to overestimate how
well they know the material. They will ask themselves a question about material as soon as they
have looked it over, and often are able to correctly answer their question. They then assume that
they will be able to answer the question latter on as well. However, this may not be true.

"If you ask yourself a question about something immediately after studying it, you may be able
to correctly recall the information. Thus, you may determine that you know the material and
stop studying it. However, you may not be able to answer these same questions only moments
later when you need to use the information.

"There are a few ways to avoid this. The ﬁrst way is to wait for a brief period of time after
studying something before asking oneself questions about it. Generally, somewhere around 30
seconds to a minute is enough of a delay. Another strategy is to start thinking about something
else for a few seconds to distract you attention away from the target material, and then ask
yourself questions. Further, even after you feel you know and understand the information, it is
helpful to ask yourself questions about information you had studied earlier to be sure that you
can still recall it.

So, as you try to learn how to make web pages, try to be aware of how well you are learning and
what you are not learning as well as you need to. I have taught you three strategies to help you
do this. First, throughout training, ask yourself questions about what you are trying to learn. I
have given you some ﬁll-in-the-blank questions that can help you with this. Second, don't try to
answer the question immediately after looking at the material. Either wait about 30 seconds or
distract yourself by studying something else momentarily, then asking yourself questions about
material you had studied earlier.

"One of the most important points that I have tried to make many times already is that it is not
enough to simply monitor your knowledge. You must use the information gained by monitoring
to determine which information you will spend additional time on, to determine if your learning
strategies are working, and so on. For example, if you determine that you don't understand or
remember information you have studied earlier, spend some additional time reviewing this
information. These strategies have been shown to improve learning. As you progess through
the training, use these strategies to help improve your learning. Feel free to refer to the
Performance Aid that I have given you at any point during the training."

165

APPENDIX F

Performance Aid

Metacogrition is "thinking about your thinking"
0 Knowing how well you understand the information you are trying to learn
0 Important for decided what to study and for how long

As you move through the training program, try to determine how well you are learning
the information.

People over-estimate what they know--Research consistently ﬁnds that people
overestimate what they know. Be careful not to overestimate how well you know the
information in this training progam.

Ask yourself speciﬁc questions about the material you are studying to determine what you
know and what you need to learn better (test yourself).

The following ﬁll-in-the-blank question stems may help you create questions to ask yourself:

"What is the main idea of ?"

"How are _ and _ alike?"
"What are the strengths and weaknesses of ?"
"How does _ affect ?

"How does _ tie in with what I have learned before? "
"How is _ related to _?"
"What is a new example of _?"

"What conclusions can you draw about _?"
"Why is it important that _?"

To get a more accurate estimation of how well you know something, ask yourself questions
about it at least 30 seconds after you have studied or thought about it. Alternatively, distract
yourself by thinking of something else (such as other information from the training) brieﬂy
before asking yourself questions.

If you determine that you don't understand something as well as you should, spend
additional time studying it or modify your learning strategies.

166

 

APPENDIX G

Metacognitive Prompting

Pre-Training Instructions:

"As you go though this training progam, you should be attempting to monitoring your
knowledge-That is, you should be thinking about how well you are learning the information you
need to know in order to create your own web page. If you determine that you don't fully know
or understand information that you have already covered, you should go back and spend
additional time studying that part of the training. "

Prompt Screen:

"Please take a few of minutes to think about how well you are learning the information you need
to know in order to create your own web page.

"First, think about what information you know ﬁ'om the last lesson, and enter some brief notes
about it here:

"Second, think about what information you need to know from the last lesson, but you may not
remember it or know it well enough, and enter some brief notes about it here:

"If you feel that you don't fully know or understand information that you have already covered,
you should go back and spend additional time studying that part of the training. "

167

REFERENCES

168

 

REFERENCES

Ackerman, P. L. (1987). Individual differences in skill learning: An integation of
psychometric and information processing perspectives. Psychological Bulletip, 102, 3-27

Anderson, J. R. (1982). Acquisition of a cognitive skill. Psychological Review. 89. 369-
406.

Arbuckle, T. Y., & Cuddy, L. L. (1969). Discrimination of item strength at time of
presentation. Journal of Experimental Psychology, 81]] 1, 126-131.

Baldwin, T. T., & Ford, J. K. (1988). Transfer of training: A review and directions for
future research. Personnel Psychology, 41(1), 63-105.

Bandura, A. (1977). Self-efﬁcacy: Toward a unifying theory of behavioral change.
Psychological Review.84. 191-215.

Bandura, A. (1991). Social cogritive theory of self-regulation. Organizational Behavior
Ed humpn Decision Processes, 50, 248-287.

Bandura, A., & Cervone, D. (1986). Differential engagement of self-reactive inﬂuences
in cogritive motivation. Organizational Behavior and Human Decision Processes, 38(1), 92-113.

Barrick, M. R., & Mount, M. K. (1991). The Big Five personality dimensions and job
performance: A meta-analysis. Personnel Psychology, 4411 1, 1-26.

Barrick, M. R., Mount, M. K., & Strauss, J. P. (1993). Conscientiousness and
performance of sales representatives: Test of the mediating effects of goal setting. Journal of

Applied Psychology, 78(5), 715-722.

Bauer, R. H., Kyaw, D., & Kilbey, M. M. (1984). Metamemory of alcoholic Korsakoff
patients. Societv for Verbal Behavior 4, 175-179.

 

Begg, 1., Duft, S., Lalonde, P., Melnick, R., & Sanvito, J. (1989). Memory predictions are
based on ease of processing. Journal of Memogy & Langu_age, 28, 610-632.

Belmont, J. & Butterﬁeld, E. (1977). Instructional approaches to developmental cogritive
perspectives. In R. V. Kail & J. W. Hagen (Eds), Perspectiveson the Development of Memogg
and Cogm'tion (pp. 437-477). Hillsdale, NJ: Erlbaum.

Belmont, J. M., & Butterﬁeld, E. C. (1971). Learning strategies as determinants of
memory deﬁciencies. Cogpitive Psychology, 2, 411-420.

Bern, D. J ., & Allen, A. (1974). On predicting some of the people some of the time: The
search for cross-situational consistencies in behavior. Psychological Review. 81(6). 506-520.

169

Bjork, R. A. (1994). Memory and metarnemory considerations in the training of human
beings.

Blake, M. (1973). Prediction of recogrition when recall fails: Exploring the feeling-of-
knowing phenomenon. Journal of Verbal Learning and Verbal Behpvior. 12(3). 311-319.

Bornstein, B. H., & Zickafoose, D. J. (1999). "I know I know it, I know I saw it": The
stability of the conﬁdence-accuracy relationship across domains. Joumpl of Experimental

Psychology: Applied, 5, 76-88.

Bothwell, R. K., Deffenbacher, K. A., & Brigham, J. C. (1987). Correlation of eyewitness

accuracy and conﬁdence: Optimality hypothesis revisited. Journal of Applied Psychology, 72(4),
691-695.

Brief, A. P., & Hollenbeck, J. R. (1985). An exploratory study of self-regulating activities
and their effects on job performance. J ounirl of Occupational Behaviour 6 197-208.

 

Brown, A. L. (1978). Knowing when, where, and how to remember: A problem of
metacogrition. In R. Glaser (Ed.), Advances in Instruction_al Psychology. New York: Halsted
Press.

Brown, M. W., & Cudeck, R. (1993). Alternative ways of assessing model ﬁt. In K. A>
Bollen & J. S. Long (Eds), Testing Structural Equation Models, Newbury Park, CA: Sage.

Brown, A. L., & Lawton, S. C. (1977). The feeling of knowing experience in educable
retarded children. Develoymental Psychology, 13(4), 364-370.

Brown, A. L., Bransford, J. D., Ferrara, R. A. & Campione, J. C. (1983). Learning,
remembering, and understanding.(In J. H. Flavell & E. M. Markrnan (Eds), Handbook of child
psychology (4th ed), Vol. 3, pp. 77-166. New York: Wiley.

Brown, A. S. (1991). A Review of the Tip-of-the-Tongue Experience. Psychological
Bulletin 109, 204-223

 

Brown, K. G., Mullins, M., Weissbein, D., Toney, R., & Kozlowski, S. W. J. (1997).
Mastery goals and strategic reﬂection: Preliminary evidence for learning interference. In S. W. J.
Kozlowski (Chair), Metacogition and training. Symposium presented at the 12th Annual
Conference for the Society of Industrial and Organizational Psychology, St. Louis, MO.

Butterﬁeld, E. C., & Nelson, G. D. (1989). Theory and practice of teaching for transfer.
Educational Technology Research and Development, 37, 5-38.

Button, S. B., Mathieu, J. E., & Zajac, D. M. (1996). Goal orientation in organizational
research: A conceptual and empirical foundation. Organizational Behavior and Human Decision

Processes, 67(1), 26-48.

Carver, C. S., & Scheier, M. F. (1982). Control theory: A useful conceptual ﬁ'amework
for personality-social, clinical, and health psychology. Psychological Bulletin, 92(1), 111-135.

170

 

Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159.

Colquitt, J. A. & Simmering, M. J. (1998). Conscientiousness, goal orientation, and
motivation to learn during the learning process: A longitudinal study. J ourmrl of Applied
Psychology, 83, 654-665.

Connor, L. T., Dunlosky, J ., Hertzog, C. (1994). Age-related differences in absolute but
not relative metarnemory accuracy. Psychology & Aging, 12, 50-71.

Costa, P. T. & McCrae, R. R. (1992). The NEO PI-R professional manual. (Odessa, FL:
Psychological Assessment Resources).

Dunlosky, J ., & Connor, L. T. (1997). Age differences in the allocation of study time
account for age differences in memory performance. Memorvand Cogition, 25(5), 691-700.

Dunlosky, J ., & Nelson, T. O. (1992). lrnportance of the kind of cue for judgnents of
learning (J CL) and the delayed-JOL effect. Memory and Cogpition, 20(4), 374-3 80.

Dunlosky, J ., & Nelson, T. O. (1994). Does the sensitivity of judgnents of learning
(J OLs) to the effects of various study activities depend on when the J 01.5 occur? Journal of

Memog and Langpage, 33(4), 545-565.

Dunlosky, J ., & Nelson, T. O. (1997). Similarity between the cue for judgnents of
learning (J OL) and the cue for test is not the primary determinant of J OL accuracy. Journal of

Memog and Langgge, 36(1), 34-49.

. Dunn, W. S., Mount, M. K., Barrick, M. R., & Ones, D. S. (1995). Relative importance of
personality and general mental ability in managers' judgnents of applicant qualiﬁcations. Journal
of Applied Psychology, 80(4), 500-509.

Dweck, C. S. (1986). Motivational processes affecting learning. American Psychologist,
41(10), 1040-1048.

Flavell, J. H. (1971). Stage-related properties of cogritive development. Cogm'tive
Psychology, 2(4), 421-453.

Flavell, J. H. (1979). Metacogrition and cogritive monitoring: A new area of cognitive-
developmental inquiry. American Psychologi $1, 3310), 906-911.

Ford, J. K., Smith, E. M., Weissbein, D. A., Gully, S. M., & Salas, E. (1998).
Relationships of goal orientation, metacogritive activity, and practice strategies with learning
outcomes and transfer. Journal of Applied Psychology, 83(2), 218-233.

Gagre, R. M. & Medsker, K. L. (1996). The conditions of learning: Training
applications. (Fort Worth, TX: Harcourt Brace).

Garner, R. (1990). When children and adults do not use learning strategies: Toward a
theory of settings. Review of Educational Research, 60, 517-529.

171

Gray, S. H. (1987). The effect of sequence control on computer assisted learning.
Journal of Computer-Based Instruction, 14, 54-56.

Gruneberg, M. M., Monks, J ., & Sykes, R. N. (1977). Some methodological problems
with feelings of knowing studies. Acta Psychologipg, 41(5), 365-371.

Hall, L. K., & Bahrick, H. P. (1998). The validity of metacognitive predictions of
widespread learning and long-term retention. .

Hannaﬁn, R. D., & Sullivan, H. J. (1995). Learner control in hill and lean CAI progams.
Educational Technology Research and Development, 43(1), 19-30.

Hart, J. T. (1965). Memory and the feeling-of-knowing experience. Journal of
Educational Psychology, 56, 208-216.

Hart, J. T. (1967). Memory and the memory—monitoring process. Journal of Verbal
Learning and Verbal Behavior, 6, 685-691.

Hollenbeck, J. R., & Brief, A. P. (1987). The effects of individual differences and goal
origin on goal setting and performance. Organizational Behavior and Human Decision Processes,
40, 392-414.

Huet, N. & Marine, C. (1997). Metamemory assessment and memory behavior in a
simulated memory professional task. Contemmpag Educational Psychology, 22, 507-520.

James, W. (1980). The Principles of Psychology, Volume 1. New York: Dover.

Kanfer, R., & Ackerman, P. L. (1989). Motivation and cogritive abilities: An
integative/aptitude-treatment interaction approach to skill acquisition. Journal of Applied

Psychology, 711(4), 657-690.

Kanfer, R., Ackerman, P. L., Murtha, T. C., Dugdale, B., & et a1. (1994). Goal setting,
conditions of practice, and task performance: A resource allocation perspective. Journal of

Applied Psychology, 79(6), 826-835.

Kelernan, W. L., & Weaver, C. A., III. (1997). Enhanced memory at delays: Why do
judgments of learning irnprove over time? Journal of Experimental Psychology: Learning,

Memog, and Cogm’tion, 23(6), 1394-1409.

King, A. (1989). Effects of self-questioning training on college students' comprehension
of lectures. Contemp_o_ra_ry Educational Psychology, 14, 366-381.

King, A. (1992). Facilitating elaborative learning through guided student-generated
questioning. Educational Psychologist, 27, 111-126.

King, A., & Rosenshine, B. (1993). Effects of guided cooperative questioning on
children's knowledge construction. Journal of Experimental Education, 61(2), 127-148.

172

King, J. F., Zechmeister, E. B., & Shaughnessy, J. J. (1980). Judgnents of knowing: The
inﬂuence of retrieval practice. American Journal of Psychology, 93(2), 329-343.

Kinzie, M. B., & Sullivan, H. J. (1989). Continuing motivation, learner control, and
CAI. Educational Technology Researg and Development. 37. 5-14.

Klein, H. J. (1989). An integated control theory model of work motivation. Academy of
Management Review, 14(2), 150-172.

Kluwe, R. H. (1987). Executive decisions and regulation of problem solving behavior. In
F. E. Weinert & R. H. Kluwe (Eds), Metacogpition, Motivation, and Understanding (pp. 31-64).
Hillsdale, NJ: Erlbam.

Koriat, A., & Goldsmith, M. (1996). Monitoring and control processes in the strategic
regulation of memory accuracy. Psychological Review. 103. 490-517.

Koriat, A., Lichtenstein, S., & Fischhoff, B. (1980). Reasons for conﬁdence. 107-118.

Kozlowski, S. W. J ., Gully, S. M., Smith, A. E., Nason, E. R., & Brown, K. G. (1995,
May). Sequenced mastery training and advance organizers: Effects on learning, self-efﬁcacy,

performance, and generalization. In R. J. Klirnoski (Chair), Thinking and feeling while doing:
Understanding the learner in the learning process. Symposium conducted at the 10 Annual

Conference of the Society for Industrial and Organizational Psychology, Orlando, FL.

Kraiger, K. (1995). Paradigns lost: Applications and misapplications of cognitive
science to the study of training. 103rd Annual Convention of American Psychological
Association.

Kraiger, K., Ford, J. K., & Salas, E. (1993). Application of cogritive, skill-based, and
affective theories of learning outcomes to new methods of training evaluation. Journal of

Applied Psychology, 78(2), 311-328.

Kraiger, K., Salas, E., & Cannon Bowers, J. A. (1995). Measuring knowledge
organization as a method for assessing learning during training. Human Factors, 37(4), 804-816.

Kruger, J. & Dunning, D. (1999). Unskilled and unaware of it: How difﬁculties in
recogrizing one's own incompetence lead to inﬂated self-assessments. Journal of Personalig &

Social Psychology, 77(6), 1121-1134.

Lachman, J. L., Lachman, R., & Thronesbery, C. (1979). Metamemory through the adult
life span. Developmental Psychology, 15(5), 543-550.

Leonesio, R. J ., & Nelson, T. O. (1990). Do different metarnemory judgnents tap the

same underlying aspects of memory? Journal of Experimental Psychology: Learning, Memogg,
and Cogm'tion, 16(3), 464-470.

173

Leonesio, R. J ., & Nelson, T. O. (1990). Do different metarnemory judgnents tap the
same underlying aspects of memory? J ourn_al of Experimental Psychology: Learning, Memopy,

and Cogpition, 16(3), 464-470.

Lichtenstein, S., & F ischhoff, B. (1977). Do those who know more also know more about
how much they know? Organizationgl Behavior pnd Human Performance. 20. 159-183.

Lorene, L., Sturmey, P., & Brittain, H. (1992). Evaluation of a meta-cogritive strategy to
improve the information gained ﬁom a stroke information pack. Stress Medicine. 8(2). 111-112.

Manning, B. H., Glasner, S. E., & Smith, E. R. (1996). The self-regulated learning aspect
of metacogiition: A component of gifted education. Roeper Reviewp18(3). 217-223.

Martocchio, J. J ., & Judge, T. A. (1997). Relationship between conscientiousness and
learning in employee training: Mediating inﬂuences of self-deception and self-efﬁcacy. Journal
of Applied Psychology, 82(5), 764-773.

Maruyama, G. M. (1997). Lasics of Structural Equations Modeling. Thousand Oaks, CA:
Sage.

Mazzoni, G., & Comoldi, C. (1993). Strategies in study time allocation: Why is study
time sometimes not effective? Journal of Experimental Psychology: General, 122(1), 47-60.

Mazzoni, G., & Nelson, T. O. (1995). Judgnents of learning are affected by the kind of
encoding in ways that cannot be attributed to the level of recall. Journal of Experimental
Psychology: Learning, Memog, and Cogp'tion, 21(5), 1263-1274.

Mazzoni, G., & Nelson, T. O. (1998). Metacogrition and cogritive neuropsychology:
Monitoring and control processes.

Mazzoni, G., Comoldi, C., & Marchitelli, G. (1990). Do memorability ratings affect
study-time allocation? Memog and Cogm'tion, 18(2), 196-204.

McCrae, R. R. & & Costa, P. T. (1987). Validation of the ﬁve-factor model of personality
across instruments and observers. Journal of Personaligy and Social Psychology, 52, 81-90.

Meloth, M. S. (1990). Changes in poor readers' knowledge of cogrition and the
association of knowledge of cogrition with regulation of cognition and reading comprehension.

Journal of Educational Psychology, 82(4), 792-798.

Milheim, W. D. (1989). The effects of two learner control variables--pacing and
sequence-on learning from an interactive video lesson. .

Milheim, W. D., & Martin, B. L. (1991). Theoretical bases for the use of learner control:
Three different perspectives. Journal of Computer Based Instruction, 18(3 ), 99-105.

Miller, V. D., & Jablin, F. M. (1991). Information seeking during organizational entry:
Inﬂuences, tactics, and a model of the process. Academy of Management Journal, 16, 92-120.

174

Mischel, W. (1977). On the future of personality measurement. American Psychologist,
32(4), 246-254.

Morrison, G. R., Ross, S. M., & Baldwin, W. (1992). Learner control of context and
instructional support in learning elementary school mathematics. Educational Technology

Research and Development, 40(1), 5-13.

Murphy, M. D., Schmitt, F. A., Caruso, M. J ., & Sanders, R. E. (1987). Metamemory in
older adults: The role of monitoring in serial recall. Psychology and Aging, 2(4), 331-339.

Narens, L., Jameson, K. A., & Lee, V. A. (1994). Subthreshold priming and memory
monitoring. In J. Metcalfe & A. P. Shimamura (Eds), Metacogpition: Knowing about knowing
(pp. 71-92). Cambridge, MA, USA: Mit Press.

Nelson, T. O. (1993). Judgnents of learning and the allocation of study time. Journal of
Experimental Psychology: General, 122(2), 269-273.

Nelson, T. O. (1996). Consciousness and metacogrition. American Psychologist, 51(2),
102-116.

in!“ .

 

Nelson, T. 0., & Dunlosky, J. (1991). When people's judgnents of learning (J OLs) are
extremely accurate at predicting subsequent recall: The "delayed-JOL effect. ". Psychological

Science, 2(4), 267-270.

Nelson, T. O., & Leonesio, R. J. (1988). Allocation of self-paced study time and the
"labor-in-vain effect. " Journal of Experimental Psychology: Learning. Memory, and Cogpition,
14(4), 676-686.

Nelson, T. O., & Narens, L. (1994). Why investigate metacognition? .

Nelson, T. O., Dunlosky, J., Graf, A., & Narens, L. (1994). Utilization of metacogritive
judgnents in the allocation of study during multitrial learning. Psychological Science. 5(4). 207-
213.

Nelson, T. O., Leonesio, R. J ., Landwehr, R. S., & Narens, L. (1986). A comparison of
three predictors of an individual's memory performance: The individual's feeling of knowing
versus the normative feeling of knowing versus base-rate item difﬁculty. Journal of

Experimental Psychology: Learning, Memog, and Cogm'tion, 12(2), 279-287.

Otero, J ., Campanario, J. M., & Hopkins, K. D. (1992). The relationship between
academic achievement and metacogritive comprehension monitoring ability of Spanish

secondary school students. Educational and Psychological Measurement, 52(2), 419-430.

Owings, R. A.. (1980). Spontaneous monitoring and regulation of learning: A comparison
of successful and less successful ﬁfth gaders. Journal of Educational Psychology, 72, 250-256.

 

Palmer, D. J ., & Goetz, E. T. (1988). Selection and use of study strategies: The role of the
studier's beliefs about self and strategies. In C. E. Weinstein & E. T. Goetz (Eds), Learning and

175

study strategies: Issues in assessment, instruction, and evaluation. Educational psychology (pp.
41-61). San Diego, CA, USA: Academic Press, Inc.

Paris, S. G., & Winogad, P. (1990). Promoting metacogrition and motivation of
exceptional children. Rase: Remedial and Special Education. 11(6). 7-15.

Payne, B. D., & Manning, B. H. (1992). Basal reader instruction: Effects of
comprehension monitoring training on reading comprehension, strategy use and attitude.

Reading Research and Instruction, 32(1), 29-38.

Pintrich, P. R. & DeGroot, E. V. (1990). Motivational and self-regulated learning
components of classroom academic performance. J oumpl of Educational Psychology, 82, 33-40.

Read, J. D., & Bruce, D. (1982). Longitudinal tracking of difﬁcult memory retrievals.
Cogmtive Psychology, 14(2), 280-300.

Ridley, D. S., Schutz, P. A., Glanz, R. S., & Weinstein, C. E. (1992). Self-regulated
learning: The interactive inﬂuence of metacogritive awareness and goal-setting. J oumal of

Experimental Education, 60(4), 293-306.
Rosenshine, B., Meister, C., & Chapman, S. (1996). Teaching students to generate

questions: A review of the intervention studies. Review of Educational Research, 66(2), 181-221.

Schmidt, R. A., & Bjork, R. A. (1992). New conceptualizations of practice: Common

principles in three paradigms suggest new concepts for training. Psychological Science, 3(4),
207-217.

Schraw, G., Horn, C., Thorndike Christ, T., & Bruning, R. (1995). Academic goal
orientations and student classroom achievement. Contemmrm Educational Psychology, 20,
359-368.

Schumacker, R. E., & Lomax, R. G. (1996). A Beginner’s Guide to Structural Equation
Modeling. Mahwah, NJ: Earlbaum.

Schunk, D. H. (1995). Inherent details of self-regulated learning include student
perceptions. Educational Psychologisp 30(4), 213-216.

Seidenberg, M., Hermann, B. P., Schoenfeld, J., Davies, K., Wyler, A., & Dohan, F. C.
(1997). Reorganization of verbal memory function in early onset left temporal lobe epilepsy.

Brain and Cognition, 35(1), 132-148.

Short, E. J ., & Ryan, E. B. (1984). Metacogritive differences between skilled and less
skilled readers: Remediating deﬁcits through story gammar and attribution training. Journal of

Educational Psychology, 76(2), 225-235.

Spellrnan, B. A., & Bjork, R. A. (1992). When predictions create reality: Judgnents of
learning may alter what they are intended to assess. Psychological Science, 3(5), 315-316.

176

“ii-ml

 

Steinberg, E. R. (1977). Review of student control in computer-assisted instruction.
Journal of Computer Based Instruction. 3(3). 84-90.

Steinberg, E. R. (1989). Cognition and learner control: A literature review, 1977-2988.
Journal of Computer Based Instruction, 16(4), 117-121.

Tarski, A. (1956). The concept of truth in formalized languages. In A. Tarski (Ed.),
Logic, Semantics, Metamathematics (pp. 152-178). Oxford, England: Claredon Press.

Tennyson, R. D., & Buttrey, T. (1980). Advertisement and management stategies as
design variables in computer-assisted instruction. Educational Communication and Technology,
28(3 ), 169-176.

Tennyson, R. D., & Rothen, W. .(1979). Management of computer-based instruction:
Design of an adaptive control strategy. Journal of Computer-Based In_struction, 5. 63-71.

Tennyson, R. D., Christensen, D. L., & Park, S. I. (1984). The Minnesota Adaptive

Instructional System: An intelligent GB] system. Journal of Computer Based Instruction, 11(1),
2-13.

Thiede, K. W. & Dunlosky, J. (1994). Delaying students' metacognitive monitoring
improves their accuracy in predicting their recognition performance. J ournpl of Educational

Psychology, 86, 290-302.

Thorndike, E. L. & Woodworth, R. S. (1901). The inﬂuence of improvement on one
mental function on the efﬁciency of other ﬁnnctions. Psychology Review. 8. 247-261.

Vesonder, G. T., & Voss, J. F. (1985). On the ability to predict one's own responses while
learning. Journal of Memory and Langpage, 24(3), 363-376.

Volet, S. E. (1991). Modeling and coaching of relevant metacognitive strategies for
enhancing university students' learning. Learning and Instruction, 1(4), 319-336.

Weed, K., Ryan, E. B., & Day, J. (1990). Metamemory and attributions as mediators of
strategy use and recall. Journal of Educational Psychology, 82(4), 849-855.

Weiss, H. M. (1990). Learning theory and industrial and organizational psychology.(ln

M. D. Dunnette & L. M. Hough (Eds). Handbook of industrial and orgarnizational psychology
(2nd ed. ), 1, pp. 171-221. Palo Alto, CA: Consulting Psychologists Press.

Weiss, H. M., & Adler, S. (1984). Personality and organizational behavior. Research in
Organizational Behavior, 6, 1-50.

Weissbein, D. A. (1996). The effects of goal gpe and metacogpitive training on complex

skill aguisition: Implications of the limited resources model. Unpublished master's thesis,
Michigan State University, East Lansing.

177

 

 

JJ

Wellman, H. M. (1977). Tip of the tongue and feeling of knowing experiences: A
developmental study of memory monitoring. Child Development, 48(1), 13-21.

Wescourt, K. T., & Atkinson, R. C. (1973). Scanning for information in long— and short-
term memory. Journal of Experimental Psychology, 98(1), 95-101.

Williams, W. H. (1993). Transfer of self-instructional and metacognitive training of
communication skills for people who have learning difﬁculties. Dissertgtion Abstracts

International, 53( 7-A), 2331-2332.

Winne, P. H. (1995). Inherent details in self-regulated learning. Educational Psychologist,
3_0, 173-187.

Wolters, C. A. (1998). Self-Regulated Learning and College Students' Regulation of
Motivation. Journal of Educational Psychology, 90, 224-235.

Woodworth, R. S. (1940). Psychology. New York: Holt.

Zacks, R. T. (1969). Invariance of total learrning time under different conditions of
practice. Journal of Experimental Psychology, 82(3 ), 441-447.

178

 

ill[1|IIIEWIMIJIEIIMHII