LIBRARY
Michigan State
University

 

 

 

This is to certify that the

dissertation entitled

A PROCESS MODEL OF LEARNER BEHAVIOR AND
ENGAGEMENT DURING WEB-BASED TRAINING

presented by

Ann Williams Howell

has been accepted towards fulfillment
of the requirements for

Ph . D . degree in Psychology

 

awa- M

Major professor

Date_/é ”/7'3000

MS U is an Affirmative Action/Equal Opportunity Institution 0-12771

_,i,

 

 

 

 

6’01 CJCIHCIDueDuothS-p. 1 5

PLACE IN RETURN BOX to remove this checkout from your record.
TO AVOID FINES return on or before date due.
MAY BE RECALLED with earlier due date if requested.

DATE DUE DATE DUE DATE DUE

 

 

 

    
 

V p . Ann Williams Howell

 

I ._ A DISSERTATION
Submitted to A ..
Michigan State University
in partial fulfillment of the requirements i
for the degree of

DOCTOR OF PHILOSOPHY

‘ 32 '.-I Department of Psychology

.‘. 1r"
‘ »,‘.‘, x 2000

 

   
   
 
  

VI

     

' - W‘mmamxmmm
DURING WEB-BASED TRAINING:
-» and ”if. Hill“ Haw '- -~ '.‘i' (5.
By
Murmu- - ' '
Ann Williams Howell
unit-cm; r;

vui I?“ ‘r

Web-based training (WBT) is a new training medium that offers many advantages

' ”marl: .

bugmizations. However, it is important to understand how WBT influences the
Content r. "
individual learning process and training outcomes. In this study, a research model was
I‘ll " I
:signed and tested that measured behavioral choices (content choice and activity level)
: l’ -
and engagement choices (attentional focus and emotional engagement) made by learners
during a WBT course. Learner control was manipulated in the experiment (high control
and limited control) and was expected to interact with individual differences such as
mastery orientation, Web self-efficacy, attitude toward WBT, and microcomputer
playfulness to influence the leamer’s choices. The process choices were expected to
influence learning outcomes, including declarative knowledge, skill—based learning,
application self-efficacy and satisfaction with training. The model was built on a
motivational foundation that pulled elements from cognitive choice theories and theories

of intrinsic motivation.

In the study, 236 participants took a self-paced, WBT course on Change

Millagement in a laboratory session, and measures of individual differences, process

 

 

leading to the thought that learner control research might need to be approached from a
different perspective. The results did show some support for the relationships between
the process variables and the outcomes, supporting the idea of using process models to
study training. For example, emotional engagement related to satisfaction and to
application self—efficacy, and satisfaction was also related to attentional focus. One
conclusion of this study was that engagement is an important variable in WBT, and it
may provide an alternative (and more effective) method for studying learner control.
Content choice was related to declarative knowledge, and activity level had a marginal
relationship with skill-based learning. In addition to the outcome relationships, the study
also found relationships between mastery orientation and content choice, as well as
between attitude toward WBT and emotional engagement. A post hoc model found some
support for satisfaction as a mediator between the process variables and the outcomes
variables. This study concludes with a discussion of the results, limitations and

directions for future research.

    
   
  
 

,, action! m I‘m-u mi r..m_- I' l . neat In: f'i\ «are». but
‘ 1 'innniIIII W8 .I I IIIIIII-I i-IIII
fl”, . mam)- .‘

. ' . mmeappears on this dissertation, many people'rnade _ l
I 1,, a I -tothe project. Without these people, this research and
anotexist. I would like to give special thanks for the people that made
W A heart-felt thank you goes to Kevin Ford who participated as my
Wehairman and provided invaluable guidance, help and encouragement. Kevin
W‘me, even when the odds suggested I might not finish. I would also like to
were other members of my dissertation guidance committee, Steve Kozlowski, Neal
Selmitt and Steve Yelon, for their time and support. Steve Sneed has earned my eternal
gratitude and deserves a special thank you for the time he contributed to programming
my Web-based training course. He volunteered his help to me at a time when he was
swamped with work and anticipating the birth of his first child. Learnshare made this
pIOject possible by granting me permission to modify and use the Web—based Change
Management course in my research. Finally, I would like to thank Strategic Interactive,
specifically Tom Bohn, Dave Morrison and Mark Morrison, for the support and
assistance provided to me. SI provided me with the flexibility and encouragement that I
needed in order to complete a dissertation while working full-time.

Many other people played a less direct, but equally valuable, role in my

(3 ,7 'on : :58. First, 1 would like to share my appreciation of the MSU Psychology

 

  

"w
»I 4
l_.

 

n I

 

 

much I learned in graduate school. The content Ileamed is critical for my success, but
equally important is the ability to think in a scientific and investigative manner. I am
starting to recognize and enjoy the benefits of my education. Second, I would like to
thank all of the fellow students, friends, and family who supported me during this time. I
received invaluable help, support and encouragement from Wendy Golden, Ken Brown,
Lori Sheppard, Dan Weissbein, Sandy Fisher, Mike Wasserrnan and many others. My
parents also provided constant counseling and support and encouragement during my
entire graduate career. Most importantly I want to thank my husband, Tim. He gets
thanks and credit for supporting, helping and standing by me during this process — even
the bad days. I sincerely believe that spouses deserve an honorary Ph.D. for the sacrifices

and demands that a dissertation places on the entire family.

LIST I

LIST I

‘TTCI

ll II
ruin-l

HE‘

u.

mi

TABLE OF CONTENTS
LIST OF TABLES ............................................................................. viii
LIST OF FIGURES ............................................................................... x
INTRODUCTION ................................................................................ 1
LITERATURE REVIEW ........................................................................ 8
Non—linear Training ............................................................ 8
Learner Control ................................................................ 1 1
Learning Process and Learner Choices ..................................... 18
Engagement ..................................................................... 21
Web-based Training Research ................................................ 25
FOUNDATION FOR A NEW MODEL ....................................................... 28
Lack of Theory .................................................................. 28
Comparison to Brown (1999) Model ........................................ 33
Conceptualization of the Learning Process ................................. 37
RESEARCH MODEL ........................................................................... 40
Motivational Foundation ...................................................... 40
Overview of the Model ........................................................ 45
Individual Difference Inputs .................................................. 50
Learning Process Variables ................................................... 58
Learner Control as a Moderator .............................................. 60
Learning Outcomes ............................................................ 65
METHOD .......................................................................................... 70
Participants ...................................................................... 70
Research Design ................................................................ 70
Power Analysis .................................................................. 70
Training Course ................................................................. 71
Training Technology ........................................................... 72
Procedure ........................................................................ 73
Learner Control Manipulation ................................................ 74
Measures ......................................................................... 76
RESULTS .......................................................................................... 84
Data Analysis Strategy .......................................................... 84
Analysis of the Measures ........................................................ 84
Relationship between Individual Differences and Process Variabies.....95
Relationship between Process Variables and Outcomes .................... 99
Post hoc Analysis ............................................................... 107

E‘s-I . A. I

DP}

BBl

     
 

119
123
127
1130
I32
I“. ‘ " 117
finFuumeReummh ......... 141
)3. RC ;. . . ‘
................................................................... 147
A:' Survey and Test Items ........................................ I48 .
B: Activities and Scoring Sheets ................................ 154 '

 

 

  
   
 
   
 

tii 5 Regress; II I;
' ..... 3... '59

 

TABLE 7. Reg ,

TABLE» liq-I.
TABLE“ ‘IIIII.
TABU' If) I;

TABLI; i E . . .
TABLE 2;

TABLE I}, 3‘.
TABLE ?4 I; ,_
TABLE l5. “\(l":.", ..
TABLE to. .ermuII .. .

   
 

' '7. P081 III}. i

      
   

“Lu‘h‘lkk N‘:‘.‘:[Iul » _ “-2-

.y'jii'
. ...lit'.)

. V y!
......m;
.

-‘E‘
.‘lm

.
li'”.

-‘ ,
..IE

7.

11F
'-L;

LIST OF TABLES
TABLE 1. Collection of variables ........................................................... 76
TABLE 2. Rotated Factor Matrix for Engagement Choice Items ....................... 86
TABLE 3. Means, SDs, Reliabilities and Intercorrelations of Scales .................. 88
TABLE 4. Regression Results of Content Choices (count) .............................. 96
TABLE 5. Regression Results of Activity Level .......................................... 97
TABLE 6. Regression Results of Attentional Focus ...................................... 98
TABLE 7. Regression Results of Emotional Engagement ............................... 98
TABLE 8. Regression Results of Satisfaction with Training ........................... 101
TABLE 9. Alternative Regression Results of Satisfaction with Training ............. 101
TABLE 10. Regression Results of Application Self-efficacy .......................... 102
TABLE 11. Regression Results of Declarative Knowledge ............................ 103
TABLE 12. Regression Results of Skill—based Learning ................................ 104
TABLE 13. Mediation Analysis of Satisfaction .......................................... 105
TABLE 14. Summary of Results ........................................................... 106
TABLE 15. Additional Regression Results of Satisfaction with Training ............ 108
TABLE 16. Additional Regression Results of Application Self-efficacy ............. 108
TABLE 17. Post Hoc Regression Results of Activity Level ............................ 110
TABLE 18. Post Hoc Regression Results of Content Choice ........................... 110
TABLE 19. Post Hoe Mediation Results of Satisfaction ................................. 110
TABLE 20. Post Hoc Regression Results of Application Self-efficacy ................ 112
TABLE 21. Post Hoc Mediation Results of Application Self-efficacy ................. 112

. ‘ ‘ Ill
ii VIII
I: L .;

     

   
  
 
  
    
   
  
 
  

 

,' ‘W' "; . . its
LIST OF FIGURES
-Muitsofneelarative Knowledge - l.............l.t4

Inmofbeclarativexnowledge- -2.............ll4
foulemlctcmhl. .. 5
.II MI’IICJIJHIIZ,' '45:» v: ‘I'w’I' I

' “a! Cill‘lCt" "1 L211

genicrIt I ‘.I

ll, I" ~ ' ..
‘- 5. POST Il‘“. ‘I‘I~ ~~ . ... ,_‘-_
I Sr, '
I"
0 o t
I I
I".\ I

 

i101
HGT

5771'
IU\

   
 
 
   
   
  
   
    
  
 

if

.11"

menus
LISTOFFIGURBS

     
 

, W) is the pctfcvrrnmu' mm: . 'v. _: tzt'I/‘I'Jnofi‘ -'~' Ilw
’;A for learner mum-n] I;
Wis quickly 2am m I':I 'r 'I. I. I' t I v .- l“. s: {'2 .11.
2 w model- ~Learr1ingprocessinWBT.......... 46
ShOUId nut It 1';I , I
ioralehoices ‘ " 62
geofmctr II, a: I._ . .. ~

gagement choices ‘ ‘ .. 62

m‘ ”3‘: hi. ' Hi
. tHocModel ..... 116 '

kamlngc-w I I
‘ Mprmr

“mam .‘I'I. .

Conan

“My!

Wpfl‘lrl n. ~I‘

podtwl ;..

M’s: .r-‘PI'

“flirting -

" ' i‘nm‘II-ry . ~

_. QMhm. III-.II

  
    
 

it 35 the ;I:-;~‘ ...I.: .II ' I ' --, glut-“1k:

        
   

' training designers that '3» iii 2 7’ II. L' ‘I .' ' ' 2‘ nilI : "z. i!“..-lv'ml‘..tl
m training such as meorpwrnung spur»! Imnmznml design and

and Mr. Rm Mum

new”.

I..
“an.“

.
IPA:

‘LA I

 

3fil‘

‘ 4

 

   
    

  
 

. “tummy-«Inn and um um tor

I ten. mr- wal- rInIv :v'm: I. - I .I I g.» ' ul'ligntrtm In

; Wismeperfonnanee improvementbuuwordeflhe

; : isquickly gaining interest in corporate and university. settings,

I . 7 ' . i s .:I should not blindly jump on the WBT bandwagon (Craiger & Weiss,

‘ m; ”Mufti” technology should not distract organizations, educators and
Miran the fact that training in any medium is only worthwhile if it results in
“learning outcomes. Very little empirical research has been done to examine the

“fitting process or outcomes of WBT, so its value has not been scientifically proven.

or» reason for the lack of research is the relative newness of the technology.

Common use of the Internet only started in the mid-19905, but usage has
exploded. Use of the Web is becoming a necessary skill for business and life. The
widespread acceptance and growing use of the Web means that it could be a potentially
good tool for distance learning. Distance learning allows people to take university and
work-related courses and training from their homes and workstations regardless of where
the training originates.

Training on the Web presents many opportunities and challenges for practitioners
and researchers. From one perspective, WBT is simply another training medium —
essentially it is the next generation of computer-based training. This viewpoint should
“remind training designers that WBT still needs to fulfill all of the traditional

malfor- robust training such as incorporating sound instructional design and

‘ lameness andltranster. From another perspective; WIT-Mm

 

t
ens ”-

-II. .1

'0 ‘W‘

l P
L L.L

‘ .i
4.5!.»

l"
in. N

*JF“ r"

\l'k:

 

wk?
“A;

:T-t
I r
“Vli‘ci

he“

I“

   
  

 

’-M3W " “ )....d...’~ A A ’ 2:5“
.. $.ng behavior during training). The obligation to
.' ' . design standards needs to be considered along with the
W innovative technology. The technology is constantly changing
| ‘i‘ , . . ‘
who IS IWBTI‘ is training that is taken using a computer that is accessing a program on the
m or on a corporate intranet. It functions similarly to computer-based training
(CBT), but there are some important differences. CBT programs usually run from a
diskette or a computer hard drive, whereas WBT runs from a separate computer server.
The trainee’s computer uses an Internet connection to exchange information with the
server that hosts the training. Server technology and the Internet allow multiple users to
access the same course, permit administrators to make changes in a central location, and
collect individual performance data in a central database. Another difference is that
WBT is accessed using a Web browser such as Netscape or Internet Explorer. The Web
browser provides the window to the training. The course can be hosted either on the
Internet (public domain but usually requires password access) or a corporate intranet
(closed corporate network). WBT uses hyperlinks to offer a non-linear training
experience which can incorporate interactive exercises, graphics, sound, video, testing,
feedback, practice, personalization and more. WBT experiences can be simultaneous (all
class members logged on at the same time with instructor guidance) or asynchronous

W work alone in a self-paced environment).

A“ the instructional value has not been fully, empirically tested, these seem

 

 

it: ll

my '
[(5in

44'»:
..I.IL

‘I
«44 q.
we.”

»

1‘ I
‘NK.~
5

1w

t3; (

it.“
I‘ .

”II l?
h“ 4‘
hi .
ML) \'(

 

I
. 7‘ ».
" H-
“ l
“M
an .
5M}

   
  
  
   

edsleflexibility.1'heycanproceedattheirownpm,
Item and access needed performance support tools.

m WBT is available 24 hours a day, seven days a week. This

 

- - to take courses from home and also provides more training
“Win off-shift workers. Well—designed WBT should be easy to use for anyone
What with surfing the Web. Graphically-rich, user-friendly Web interfaces

“the-learner to navigate by pointing and clicking the mouse, and the course can offer
additional help and learning support (Stevens, 1996).

. ' WBT courses can benefit organizations by being easy to maintain and update,
offering tracking and reporting capabilities and making just-in-time training accessible to
all employees. The course material is easy to update, because it can be done with one
change made on the server. This means that small changes can be made to the content
(such as fixing errors or adding material) without requiring a huge effort such as pressing
new CD-ROM disks or copying and distributing new binders (Filipczak, 1996). Since all
information comes from one central server, the organization knows that employees are
accessing the most current information. The Web also offers sophisticated tracking
features that can record individual training performance (Filipczak, 1996), share scores
and completion information with a training administration system, and provide input into
you}: and organizational level competency reports. The Web also allows global
distillation of the training from the same server allowing the system to monitor

Wait they were all in the same room (Stevens, 1996).

 

3m.

0., b til].

let?“

 

l
.. I
”‘41.:

l. .
13. 3‘

I
'1‘1III
‘~ UH

Tiff:

ril‘y‘b‘
l‘“

4:
‘1'le

In.
I' .1'

 

   

. i Merits on WBT. Most of the literature on WBT focuses on design

- studies (eg. Khan, 1997), whereas the focus should be on the learning

 

allloutcomes. WBT provides a unique context for studying the learning process.

 
 

.4
.l
I.
.‘.

t .7 _ the learner to control different aspects of the learning and also allows the
number to collect robust data, including the actual decisions made (path taken).
mm pg of this study. Although there is limited research on WBT, many
related research areas (learner control, CBT, traditional training) provided a foundation
for exploring this new medium. From a research perspective, it was important to
understand the psychological underpinnings of the learning process during WBT.
Therefore, the focus of this study was on the learning process and outcomes in a self-
paced, learner controlled environment. Specifically, the study conceptualized the
learning process in terms of choices made by the learner. These choices included
behavioral choices (content choice and activity level) and engagement choices
(attentional focus and emotional engagement). The model was based on a motivational
foundation taken from the theories of cognitive choice and intrinsic motivation. This
investigation of the learning process included a framework for studying learner control, a
Moi the literature, a research model and an empirical study. Although this study
Md ideas that are relevant to other types of training (such as self-paced training in
M the focus was on self‘paced, nonlinear, Web-based training designed {as

. ‘
‘

mtlJVEI-IIIHI'IIIAI I . ~ l'-—~-é"|'1u<‘

    

Framework for Learner Control. In order to clarify the relationship of several

important concepts, Figure 1 presents a framework for learner control environments.

 

Web-based Training

 

 
   
    
   

Non—linear Structure

Learner
Control

 

Learning Process
- Behavioral Choices
— Engagement Choices

 

 

 

Figure 1. Framework for learner control

The framework indicates that there is a hierarchy or a series of successive design
decisions that can influence the learning process. The first decision is whether or not to
use WBT as a training medium; the second decision is whether or not to design for non-
linear structure; the third decision is how much control to give to the learner. This high-
level framework provides the basis for investigating the definition and origins of the
concept of learner control. Training design decisions influence the opportunity for
learners to make choices about their own learning. Understanding the design issues helps
define learner control, which in turn helps understand the learning process of the

individual.

A s

,.
‘9‘

us-
i

43w“
u...,‘.

\\-:l'

5...“.

wgi

”Jean

.3 >
.v .ih

ill-i

3"”.

Milllt

r I P
AWL
.

3v... .
lv‘
Lil-S

"ha a
LIIC ti

Ml. I_

PEP-2L:
’~\-.,.

»
.

l.
5 LI

 

'aheotthefimdesigo‘” “' “" ‘

   

Winnie: the training. This decision fleets all-suhequenti'

. learner control' is easier to incorporate into a self-puwd Web

~ WW6 format class. For the purposes of this study, the Web was the
WM. Once the format is chosen, training can be designed to proceed non-
M'ulinearly. This is a design decision, such that the training will or will not allow
w to vary from a pre-determined path. Defining and understanding non-linear
hitting design, as done in a later section, is an important foundation for understanding
learner control. Learner control is one type of non-linear training in which control over
the learning path is given to each individual learner. This can include control over the
sequence, content and pace of instruction. As the framework indicates, learner control is
expected to influence the learning process of individuals. The learning process refers to
the choices made by the learner (consciously or unconsciously) during the training.
Choices can be made about behavior (such as which pages to read) and level of
engagement (such as the amount of mental energy dedicated to the learning task). The
framework provided structure for the discussion and literature review of these concepts.
In a later section of this study, these concepts were incorporated into a research model
along with individual difference variables and training outcomes.

Research direction. In addition to proving a framework for investigating the

learning process, this study contributed to the literature by testing a research model for

WBT, which has rarely been done (see Brown, 1999 for an exception). This model had

Meal foundations in the motivational research -specifically cognitive choice and

 

 

5'”;
. I
lb.\\nl

\3' 51
uh,

51".

m.

'lu'.

 

(1999) but also incorporated several new elements. Specifically, this study provided an
experimental manipulation of learner control, introduced the construct of engagement to
the learning process in WBT and measured the relationships of some new individual
difference variables and outcomes. The study also used a new data collection technique
for process models in which the computer recorded all of the actions taken by the learner.
This allowed the collection of behavioral data that represented the choices made during
the learning process.

The following discussion of non-linear design and learner control was intended to
provide a rationale for the research model. The learning process formed the core of the
research model, which was tested in the experiment described in the Method section.
Several individual differences were identified that were expected to influence the
learning process, and were therefore included in the research model. These individual
differences included mastery orientation, Web self-efficacy, attitude toward WBT, and
microcomputer playfulness. Outcomes of the learning process that were investigated
included declarative knowledge, skill—based learning, satisfaction with training, and
application self-efficacy. The literature review discusses the concept of non-linear
training and learner control as well as introducing other relevant concepts. This review
sets the stage for the presentation and justification of the research model, hypotheses and

method section of the study.

iv
‘..\'1
«1, \

 

 

I.,

,th

"‘Ht.l

t.“-

*K...(”

 

 

 

LITERATURE REVIEW

This literature review introduces and analyzes the critical concepts that are related
to learner control and WBT. The literature review discusses non-linear training, learner
control, the learning process, and learner choices (behavior and level of engagement).
After the literature review, the research model section combines these concepts with
individual difference variables and training outcomes to create an empirically testable
model.

Non-linear Training

As Web-based training has gained popularity and increased in usage, one
commonly promoted benefit is that it provides a “non-linear” learning experience
(Khan, 1997). However, defining non—linear training is a challenge. The term
non-linear implies lack of organization and structure, both of which are generally
considered critical elements of well-designed training. So, is “non-linear
training” an oxymoron or can it be a reality? The linear/ non-linear concept is a
training design issue, because it deals with the way that information, ideas,
exercises and testing are presented to the learner. Specifically, it refers to who or
what controls the series of events that occur in a training session.

In linear training, all learners are presented with the same predetermined sequence
of information. Linear training is controlled by the course structure, and the instructor or
program enforces that pre-detemiined structure. For example, in order to teach a process,
the training usually starts by teaching the first step and ends with the final step. An

instructor or computer program would lead all learners through the course in the same

~14 if
[first

Cal

-Tr< ..

a. as t

It It

1"ny
- Ll
‘ 5
ll’

’1 i,

b ,<
...u

I") ‘1’.

"l\
k 5:

l.
L'kft'

H4

‘5
“\

(1‘
c

I»

IN

“2‘7 e

   
 
  
   
 

Wmmfimc hthepast.
. ,‘allyi'lmehoolehasheenthisrypeoflineaewpnw
'g,theoomsedoesnotrequirethatalllearmfbllowrhesune

   

" mmvatiety of ways to proceed through the material and learn the

W. In linear training, control comes from the pre-determined, unalterable
11 MW of the training. In non-linear training, the structure and sequence
; “Martina, so control is given to an external source (external to the course
min). The external source of control could be an instructor, adaptive computer
program or the learner. In this paper, non-linear training was defined as follows.
Non-linear training offers opportunities to vary from a pre-detennined course
structure and progression. Training content can be accessed or presented in
different ways as controlled by an influence, which is external to the actual course
structure.
This definition of non-linear training has five critical elements for self-paced,
Web-based training. The first element is that the linear/ non-linear contrast is an
instructional and technical design issue. Either the course is designed to control learning
ptogression (linear), or the course is designed to give control to an external source (non-
linear). The second element of non—linear training is that the external control source
referenced in the definition can vary. The most common external source of control
would be the learner, and this would be true for most self-paced learning. The learner
mum and what to study in the course (within the course constraints). However,

' ' . v 'riejustxone option for control in nonolineartra'ming. Nonlinear!“

     
 

mum.incasessuchasadaptivetniaingaemfl_ _ -.

a ‘ .l .‘b

3‘75”“!
(lulu

., ,y-u

tn'
Ltht

irvrt

Huh;

9 '75
shah

ill;

.‘t
my,

"flat

   
  
 

. WWIN‘W’SMCM'mHLmN

 

.- ashamed-Simnum g 1" ritzwm

r

mate: to monitor the progress ofa learner and tailor-hunte-

Won based on the level of understanding. Targeted training allows

White a pretest covering the learning objectives and receive a feedback report

 

Wmodules that can be skipped and those that need to be taken. These are not
”tally exclusive categories, so some of these control elements can be combined in a
tm'aing course (i.e. a learner gets a targeted training report and also controls which
exercises to complete).

The third element of the definition is that it is not specifically restricted to
individual, self—paced learning. Non-linear training could occur in a group setting with
the group or an instructor making decisions about what the group should do next. The
fourth element is that non-linear courses can vary in the amount of control given to the
external source. In an extreme form, the external source might control all aspects of
training such as sequence, practice, time spent, number of modules completed and more.
However, very few topics could effectively be taught without some structure, whether the
structure was obvious (prerequisite modules within the training) or implied (modules for
process steps presented in a hierarchical menu format.) Pure non-linear training without
any organizing guidance would be ignoring good instructional design principles and
likely result in confusion. In a more moderate conceptualization, the external source

Manly control some of the elements. This allows the course to be organized and

Wm a minimum amount of exposure to the material, butstill ofimrhe

 

9"] 1

when

x \

1.,
W

‘1.“
4hr.

   

, . mmnmmeem ‘ preluded ' swarm”
, : Wham. In some training, the Choices mighthcritical

 

‘ ‘ . . Mu new to takes module. This choice could influence the amountbf‘
I Willem“ and could result in critical information being missed. In
"‘ mon-linear course, the learner may be able to choose the order for proceeding

“5‘ WWe modules, but be required to complete all of them. Size of the choices can
. WW the level of the training that is targeted. Some choices would be at the
trunk level, while other choices would be at the exercise or practice level. Additional
choices could occur at the level of whether or not to click on a link for extra practice or to
View a job aid. One common type of non-linear training is learner controlled, and the
learner control research is described in more detail in the following section.
Learner Control
Leamer control is one type of non-linear training that occurs when the learner has
control over some aspects of the learning event. Learner control is an important element
of WBT, since the learner often has a choice about how to proceed. Many different
definitions of learner control exist in the current literature, but the definitions focus on
different elements and sometimes confuse design issues and behavior. Learner control
has been described in terms of the options available to the user. Murphy & Davidson
(1991) described three levels of learner control: complete learner control, program
control! adaptive control (the computer controls progression) and learner advisement

Warmvides suggestions about next steps). Reeves (1993) states that learner

_ mm control form a continuum of learner latitude that varies widely

 

Uh"

a NV!
113‘ ‘

and...
.

I
r“
(at

‘1 I n
:J\ D
U. 1.!

‘«
mw,‘

“4.1. l
l

 

   
  
   

   

' ‘ human-linear. It‘afieourseienon—heardtm be
" “henchmen resulting in learner control training. haeamy and
' issuesthat are built into the course by the instructional

h m ' ditto .,

Ia Wit learners may take the same course with the same design
.r, mu! learner control factor allows them to have different experiences.
w. Many theories of psychology such as the aptitude-treatment interaction theory

(Snow & Lehman, 1984) indicate that people and training situations interact to
create different outcomes. Different learners in the same learner control course
are likely to make different choices and have different outcomes. Some learners
would explore many of the unrestrained features, and other learners would not
deviate from the most obvious or easiest path. The behavior and choices made by
a learner when confronted with a learner control, non-linear training environment
would be defined as learner choice (Williams, 1996).

Over two decades ago, Steinberg (1977) provided a review of learner control in
computer-assisted instruction. She indicated that learners could be given control over
variables such as instructional strategy, sequence of instruction, completion time, amount
of practice and level of difficulty. The premise was that students would be more
motivated, have a better attitudes and pay closer attention to the task if they had some
Momma learning. There was not much research at that time, and the early

Wound results. For instance, Fisher, Blackwell, Garciara Mum, ' f

.
r..- \ -‘ - u

    

pen

‘Q
‘t
.r ..V“ 9‘
Iql‘“)

4-
1|.»

'r r

/ fl?“

IV“

Emir] .‘.I

u";

   
  
  
  

 

Wm learning. Another reason is that attribution theorym

   
 

“ mmwna.mmmm
' mm summumnmviwmw. m u.
“first results. It Was suspected that the early studies failed
' '. Mmadflthe psychological processes inlearning or include " ~'

. : variables. Newer lines of research focused on student strategies .
Myptuaessing and metacognitive skills) and providing support (adaptive
WNW learner control (Steinberg, 1989).
w Control can be provided to the learner through many different methods or
embination of methods. Some researchers (Merrill, 1984, Chung & Reigluth, 1992)
cimify control in terms of content, sequence, pace, display, internal processing, and
advisor strategies. Milheim and Martin (1991) focus on three primary learner control
elements: pacing (controlling speed of presentation), sequence (controlling order of
presentation), and content (controlling material presented during a lesson.) All three of
these elements were theorized to increase feelings of perceived personal control over the
learning environment. All three were also expected to influence rate of encoding during
learning. Sequence and content were expected to influence the development and use of
schemas.

Milheim and Martin ( 1991) provided several reasons for the predicted

emotiveness of learner control. From a motivational perspective, learners should
respond positively to having control for choosing personally relevant information. A

Wmmption of this perspective is that learners want control and want to have an

  

 

DU
18;

 
    

u m Finally, an information mammal.
» he ablestoorganine information and learning irrawayhnfis

' . « should increase their attention and retention.

 

t, , . ‘
I ”W Learner control can be studied using many different
i" ““mrtypieal research methods are described in this section. One type of

 

Wes two forms of learner control (full versus lean), and another type of study
warmer control version of a course with a program control version. In the full
mats lean type, learner control of pacing, sequence and content was studied by
presenting learners with either a full or lean version of a computer-based training
program. Hannafin and Sullivan (1996) studied 91 high school students taking a
computer-based geometry course. The lean version of the course contained basic
information, and learners had to use an ADD button to get additional information. The
full version contained all of the information, and learners could use a SKIP button to pass
by unneeded instruction. Results indicated that students in the full course viewed about
50 more of the optional screens than learners in the lean course and scored better on the
posttest. This study actually offered learner control to both groups, but each group was
required to do an opposite action (add or skip). The results indicate that learner control
might be more effective when students are given control to skip potentially unnecessary
information, rather than given control to add potentially helpful information.

In the second type of study, the results of students in learner control sessions are

compared to the results of students in program control sessions. Kinzie, Sullivan, and

“(mystified 98 eighth grade science students as they took a solar energycoursc

 

~ nr'fi

‘ t
-',.u

U.
1“

far.
..

1.3,;

~.>‘ .
. ...“

~

 

  
 

mm-m‘mmormmmmmmmn
' i-fw sfireprogramcontrolleamerswereautomaticallygim ‘J-

museum incorrect. They found that learners in the learner control

 

Whigher on the posttest than those in the program control condition. Even
my we'recaefulto note that students should only be given limited control, since
permissiveness is unacceptable in a learning environment. Permissiveness would
describes situation with no constraints and strucurre, and it would likely lead to neglect,
chaos and anxiety (Ryan, Connel, Deci, 1985). Limited learner control allows students to
adjust study behaviors without missing important instruction. This contrast between
allowing control, but limiting it in order to ensure a robust learning environment, is one
of the conundrums of the learner control research. As Steinberg (1977, 1989) noted in
her reviews, the learner control research has found mixed results. Some research has
found that learner control leads to more learning (Gray, 1987; Hannafin & Sullivan,
1995), but other research results are less clear.

Lee and Lee (1991) found mixed reviews within their study. They did a 2x2,
fully crossed study to examine learner control versus program control. They also
examined learning in knowledge acquisition phases versus learning in knowledge review
phases. Fifty-five eleventh grade students participated in computer training based on
their chemistry course. In learner control, students could control the number of practice

mics and the sequence of tasks and information, whereas program control students

could not change any of those elements. Analysis of the main effects indicated that

 

I‘ll:

um'Ir
Wm

 

'a'snt
Merit

 

J‘-
"mill.

 

 

 

 

 

Win theltnowledge acquisition and review phases.

literacy (1987) also found mixed results in her study comparing a linear progression
cotrse and one that offered many branching opportunities. She found that the learner
controlled branching version had a positive effect on comprehension but did not affect
retention. Furthermore, the control version resulted in more negative attitudes to the
computer program. Gray (1987) speculated that the learners did not really care if they
controlled their own learning and that too much control can be complex and distracting.
Other research indicates that program control results in more effective learning than
learner control. Pollack and Sullivan (1990) studied 152 seventh graders on recall and
recognition tasks in learner control and program control environments. They found that
program control resulted in higher achievement than learner control on the recognition
test. There were not significant differences on the recall test. In contrast to the mixed
results for learning outcomes, learner control is usually positively associated with
attitudes and motivation. Morrison, Ross, and Baldwin (1992) found that students who
couldcontrol the amount and context of supplemental practice problems reported more

,mm' .m, . '- "i “:8

.._g\;3_.'.g.1;-.\.!t‘KWtUhmmf . '9

x .

to.“

‘ t
W)"

t
. ‘

VJ".

an”

“r
ult‘.

‘1‘ .L,
.. in

I. .

.11.
‘\

‘nll‘
.4.

  
  

r1 m. Tl‘helenrnercontmlh'mmrestemshom

“ h“ A "“'r a ' ' .7. ‘fieldsandvariesextensitlelyifl‘lu‘w‘fl

 

. i993) claims that, “.. .much of the research in the field of computer-

I m is pseudoscience, because it fails to live up to the theoretical,

 

n Wumtbbgical, and/or analytic demands of the paradigm upon which it is
m 39)." He (Reeves, 1993) identified four types of problems that appear in the
finer control research.

The first problem is lack of precision in the definition of learner control. Learner
control seems to have an intuitive definition, but loose or non-existent scientific
definitions make it difficult to accurately study or compare across studies. As part of a
scientific construct, the scope and specifics of what the learner can control must be
considered. A strong definition would relate to a strong theory base in cognitive science
or learning theory (Reeves, 1993', Orey, Garrison, & Burton, 1989). The second problem
is that some of the research lacks a theoretical foundation. Milhelm and Martin (1991)
also called for a theoretical framework for understanding learner control. A theoretical
framework could help explain conflicting empirical results, which may or may not be
comparing equivalent concepts. The third problem is that learner control research is
frequented by technical and methodological problems. Some of the research
manipulations are artificial and unrealistic training situations that do not allow for
learners to have sufficient experiences and reactions. The outcome measures are

sometimes flawed, because they are not relevant to the training. The fourth problem

 

:._ L ......a 34- ' issues such as small sample sizes and large attrition rates.

, N .meanbeseenintheremmhsurdiesreviewndhdn‘sm

 

in“

12;»

Jun

g.

.c
1.;1.‘

PM

5.1» V

131:? I

3

  
 

, mm several problems thatccouldh reasonsfor‘tlle' ,

 

": L“ mmmgesdonisthatthedifferenee‘between
w ebgnitive designs of' instruction may lead to outcome differences.
W tend to have more direction, practice and feedback, and they usually

 

”Immfer of learning. Cognitive designs tend to have more student control
Matting. of the learning process, and they usually focus on far transfer of leaming.
Mm approaches focus on different levels of instruction and teaching and may not
show compatible results. A second suggestion is that individual differences such as
ability and prior knowledge influence how learners respond to learner control. These
individual differences are often not included in the research studies. A third suggestion is
that inexperience and discomfort with learner control can cause learners to respond
inappropriately. For instance, a student may decide not to explore the WBT environment,
because he is afraid of breaking the computer. Much of the learner control research also
lacks a focus on process (Milhelm, 1995). During the learning process, the learner must
make choices, but those choices are constrained by the amount and type of learner control
provided. Learner choice provides a measure of the learning process, but only a few
studies include measures of learner choice (e.g. Hancock, Thurman, &. Hubbard, 1995;
Lee & Lee, 1991; Milhelm, 1995).
learning Process and Learner Choices

The learning process can be considered in terms of the strategies used by the

talent and the behaviors that operationalize those strategies. Strategies are internal

Whether: use to select or modify their ways of attending, leanings.

 

T"? '
.u‘e ‘

.,.
—a

.jfiJ'
mlL.

L515 \-

._ 9.
Um

 

’\~‘_

so...
e

.,V
u.

M“,
l‘ll v
kw“;

n

   
  

. mailing, and elaboration (e.g. Fisher & Ford, 1998).

m. ._ assimilated to the learning process include self-regulation; ._ '

 

I ' W. activity level, effort and more. In this study, the learning process
I W has having two types of choices that were made by the learner:
1'1 mm“: (content chosen and amount of activity) and engagement choices

v" Meal focus and emotional engagement). Before presenting that research model,
this section describes other conceptualizations of the learning process and related
concepts.

Many recent learning theories have included the concept of self—regulation which
is the cognitive process related to the allocation of time and effort to activities directed
toward goal attainment (Kanfer, 1990; Farr, Hofmann, & Ringenbach, 1993). Self-
regulation theories include goal setting (Locke & Latham, 1990), control theory (Carver
& Scheier, 1990) and the integrated information processing] resource allocation theory
(Kanfer & Ackerman, 1989). All of these theories focus on the discrepancy reduction
process in which individuals work to reduce discrepancies between self-monitored
behavior and some guiding standard. Mastery orientation is suggested to impact the self-
regulation process (Farr, Hoffman, and Ringenbach, 1993). Pokay and Blumenfeld
(1990) identified three general types of learning strategies: metacognitive, cognitive and
resource management strategies. Metacognitive strategies refer to the planning,
monitoring and evaluation of one’s own cognition. Metacognition is sometimes defined

as a latent factor that explains all strategies involving monitoring, learning and adjusting

W (Erown, 1999). Cognitive strategies refer to integrating new menial

 

 

HI"
3131

"fll‘l‘!

., lisl

W L

...
"'1.”

a '0;
,.

I
put.

.5)”
U1 vii.

P‘tll‘

H w

  
    
   

  
 

. , We.

. . . m Gnllyand Salas ( 1998) studied learning strategies-in

« ' m alearnercontrolled environment. Theycreatedaleuner

t-and gave 93 undergraduate participants a choice of high, mefitun or

< . practice scenarios. The task was a complex computer simulation of a

I h. m new task which took two days to learn. After the first day of training, the

it! ”pants could choose what level of practice to receive. They were assessed at the end
6&3 second day with a complex trial designed to measure skill generalization. The
results indicated that metacognition was more influential than the other strategies.
Students who reported more metacognition performed better on a knowledge test and a
final training trial. They also reported higher levels of self-efficacy. Activity level is a
learning strategy that reflects the extent to which learners practice a skill or work on a
task. Ford, et. al (1998) found that higher activity levels were associated with greater
knowledge and better performance at the end of training.

One of the key choices that a learner can make in a learning situation is how much
effort to apply (Fisher & Ford, 1998; Steinberg, 1989). Effort can be conceptualized as
amount of effort and type of effort. It has been operationalized in many different ways
including time on task, mental workload and task persistence. Amount of effort is often
maimed as time on task. Time on task can be a contaminated measure, because an

_, , W might appear to be working on a task but focusing attention elsewhere (Fisher

_, muses; Dweclt, r986; Steinberg, 1989). Kanfer and Ackerrnan (1989) offered an

' -+._ ,

    

  
  

 

organisation, and retrieval. The-cleaning.” are
{afiflnfigga & Wager, 1992). Fisher andFedr-r

 

. p , " mat effort (reported workload) was influenced by mastery goal orientation.
W'with higher mastery reported greater workload. Participants
Wmce orientations spent more time focused on off-task activities.
”(1906) found that effort predicted achievement, whereas metacognitive and
eogniirle strategies only had an influence as mediated through effort. However, this
study did not include any content-specific measures, and the general level of the
mestions might have distorted the results.

Engagement

In their discussion of teaching effectiveness in distance learning, Webster and
Hackley (1997, p. 1284) state that “learning is best accomplished through the active
involvement of the students.” They indicate that instructors must generate student
involvement in order for the course to succeed. This reflects one of the differences
between self-paced WBT and instructor-led training. Instructor-led training has a
structured environment, which helps keep the learner engaged with the task.

Although self-paced WBT does not have an instructor to maintain involvement, it
is a medium that can offer rich graphics, interactivity, personalization and game-like
activities. It is possible that WBT is intrinsically interesting enough to maintain learner
engagement without need for an instructor. In this study, engagement was

Wind as one type of choice made during the learning process. Learners

my ommconsciously decided how much attention to focus on the training-es well

 

 

film.
nth.“

..l
«NW A
,s- .2 i
‘1‘.i‘\‘

  
   

 

‘ meharaeteristicsandoutconm.1‘hisseedoa -
, engagementand discusses it relationship to self-paced

' «: has been defined as a motivational outcome that reflects the intensity

qnlity-o‘f a student’s involvement and activity during learning (Skinner &

‘ m 1993;'Comell & Wellbom, 1991). Engaged students are behaviorally involved

ills-13 Md maintain a positive emotional tone. They are more likely to choose
mam, initiate action, and exert effort and concentration (Skinner &. Belmont,

1993). By contrast, disaffected students are passive, less involved in learning, give up
easily and often act bored, depressed and anxious. Motivational engagement has been
defined as having both behavioral and emotional components (Skinner & Belmont,

1993). The behavioral component is reflected by the student’s level of involvement, and
the emotional component is reflected by positive emotions (enthusiasm, optimism,
curiosity, interest) toward the learning experience. Kinzie (1990) incorporated a similar
concept called continuing motivation into a key role in her theory of learner control.
Webster and Hackley (1997) positioned engagement in terms of Csikzentmihalyi’s

(1975) theory of flow, which is characterized by perceptions of pleasure and

involvement. They included dimensions such as attentional focus, curiosity, and intrinsic
interest.

~Motivational engagement should be distinguished from cognitive engagement,
mm‘mflar theory in the educational psychology literature. Cognitive
‘ T to them of learning strategies such as self-regulation (Greene if; v;-
we m ~ ....1 . t .2:
. a; . , i .

'..a ,I‘ > ‘-
.,‘ _-.‘ , -.. 7 "5-" J
. ,- .

1:3: «P.-

  

  

1

Ill 1

5%

6M

"5'

Sim

 

5t tit

m.
.2-)
‘

’IF‘J

1m
lvlJlH

,3;-

var,
liill‘
“‘-\l‘

k

I
‘.
‘e‘tfi

" 1.7 j

    
      

- ”Wmtheempeermm
x assaulted toactiveinvolve‘ ' mérrt'inlearnirgpostnve'h I

 

. , , , . - u "g‘difficutt tasks and concenuanoa-(Skimer&Belmmrt,-1993;
m m fimlmc

l. w‘mm‘wmom (1993) describe how the psychological literature and
it! “literature have approached motivation and engagement in learning from two

different perspectives. Psychologists have focused on individual difference issues such
ss‘attributions (Weiner, 1986), self-efficacy (Bandura, 1986), and leaming strategies
(Mob & De Groot, 1990). Educational research has focused on teacher behaviors that
influence student motivation (i.e. Keller, 1983). They (Skinner & Belmont, 1993, p. 572)
believe that the research has advanced such that, “. . .a perspective on motivation is
emerging at the intersection of the psychological and educational literatures.” This
intersection indicates that motivation is internal to a learner. The social surround
(environment and instructor) must fulfill basic psychological needs in order for the
motivation to flourish. In their study of engagement in elementary-level classrooms,
Skinner and Belmont (1993) did find evidence that the teacher’s behavior influenced each
student’s level of engagement in the learning. This suggests a greater importance on
individual differences and their interaction with learner control.

Webster and Hackley ( 1997) examined student attitudes and engagement in
technology-mediated distance learning courses (mostly synchronous, instructor~led,

Wubased courses). They studied 29 university-credit, graduate-level courses offered

malariety of subjects. The researchers collected qualitative and quantitative data ’

 

11111

" .lr...

“Kill“

  
 

lawman Mum
‘ 'mmwas noton-locationhad less engagement whiter
. - that attitudes affected engagement and involvementin the

“new reinforce the importance of the instructor, which emphasizes how
“30m the issue of engagement in self-paced learning. The findings also

  
  
 

It mamas are important for engagement.

'iti .n-.-'Measures of engagement vary across studies. The behavioral element of
Miami engagement includes effort, attention, and persistence during learning.
Fisher and Ford (1998) studied mental effort during training. Mental effort was
operationalized as time on task and self—reported attention. This was based on Kanfer
and Aokerman’s (1989) skill acquisition theory in which attention is focused: on-task,
off-task, and on self-regulation. Brown (1999) measured attentional focus through a self-
report on—task scale and a behavioral measure of time on task.

The emotional element of engagement refers to emotional reactions during
training such as interest, happiness, anxiety and anger. Harp and Mayer (1997) explored
the differences between emotional interest and cognitive interest using definitions
provided by Kintsch (1980). Emotional interest is encouraged by adding interesting, but
irrelevant material (such as cartoons or illustrations), to a lesson. The interesting material
is hypothesized to energize the learner and therefore increase her attention, encoding and
learning. Increasing cognitive interest involves creating macro-structures (such as

W summaries or cause and effect chains) on to which learners can map new

 

iii};

em

.‘ifiii

<1 “

.1!
. lb.

.if‘x

   
 

    

‘ 2‘“ “features, available in.WBT. ’I‘heyemted fmtuyv.‘ .w n

if I ’3 math lessons with a goal to create a context for otherwise

mm They compared outcomes of intrinsic motivation. depth of

Win learning, performance, and perceived competence for 70 school-aged
mm 1001: different versions of a computer-based math training program. The
W varied on the levels of three elements: contextualization specifically fantasy
litmus (no fantasy, versus space ships and treasure hunts), personalization (references to
child’s name, friends and favorite toys) and provision of choices (allowed to make small,
non-instructional choices). They found that all three elements increased the specified
outcomes. This evidence indicates that learning elements that can increase the intrinsic
motivational appeal for student can have strong educational benefits. WBT can be
intrinsically interesting to a learner if the learner has positive attitudes to the Web and a
sense of playfulness. The role of engagement in WBT and its relationship to attitudes
and outcomes should be examined in the adult learning arena. Engagement has not been
a focus of previous WBT research, but elements of it have been included in some studies
(such as attentional focus in Brown (1999)).
Web-based Training Research

Since the Web medium itself is new, there has not been much systematic research
concerning the effectiveness of WBT (Craiger & Weiss, 1997). Fortunately, traditional
training research and computer-based training research offer some guidance about issues
faced in WBT. WBT research tends to be atheoretical and focuses on practical issues
m 1997); .One common line of study is case study analyses. Although these provide

" f , A , . ~ einfmlian. theydo notprovideaan .," 'g , ‘7 he:

 

'ib-‘l-i'rK
* i 5..

3x9”?
“.1“

  
      
  

 

‘ Why Brown (ismasewmm it

w tie-pitically tested model» Brown mammal

. \ ' ‘m control by considering the influence of individual

’ ..f .s» .. . . “Mug choices. His model indicated that immutable and malleable

' W would influence the leamer’s strategy and effort choices which

R My influences knowledge gain and post-training attitudes. His empirical test of

I! Woonsidered the relationships between individual differences (education and
W, learning and performance goals, learning and technology self-efficacy and
content utility), the learning process (operationalized as metacognition, activity level and
attentional focus) and outcomes (verbal knowledge, change in application knowledge,
and application self-efficacy). Brown (1999) defined the learning process in terms of two
types of choices: strategy and effort. He operationalized strategy as meta-cognitive
activity and measured it through a self—report scale. Effort was conceptualized in terms
of attentional focus, time on task and activity level.

Brown’s (1999) study monitored 80 employees in a Fortune 500 manufacturing

firm who were taking a new Web version of an existing, required engineering course.
Brown (1999) hypothesized that learning goals would predict meta-cognition and
attentional focus. These hypotheses were moderately supported, but metacognition was
not significantly related to any of the learning outcomes. Marginal support for some 1
omer hypotheses indicated that learning self-efficacy was related to activity level, and

.: Wgy self-efficacy was related to attentional focus. Activity level and attentional

L

,' _, M Wally related to verbal knowledge. The results were promising {my .: '7 -. . , {-

     

I')‘;
ith

73.511

Fifi '5
Lb H.

the results also highlighted some weaknesses in the model and directions for future
research. The Brown (1999) model did not develop a strong model that linked individual
difference predictors with process variables and process variables with learning
outcomes. Elements of the model were supported, but the overall structure was not. The
research method was unique and rich in that it collected field data in a controlled
environment. However, the disadvantage was that experimental manipulation was not
allowed and sample size was limited. The limited sample size along with low scale
reliabilities probably contributed to some of the null or marginal relationships. In the
current study, these shortcomings were addressed by using a controlled laboratory study
that allowed for experimental manipulation as well as allowing for a larger sample size.
The research model in this study tested a decision-making model of the learning process
in learner controlled WBT. The model included individual difference inputs, process

variables, and training outcomes.

P‘ Li.
H. t

4H-
lieu

Hails” ~
\u'. is

"mu--
iii“;

if: i r.

«lull

If it

tug;
* L

 

   
  
   
 

 

muimnmn FOR A NEW MODEL

, review highlighted the need for robust, empirical research in the

' v control, WBT and the learning process. In the next section, a resemh

I i ' ' ’ '36” me presented that attempt to describe the learning process in relation to
n ”differences, manipulation of learner control, behavioral and engagement
M Ward learning outcomes. In this section, some of the weaknesses in the existing

march are discussed in order to provide a rationale for the new research model. The
Mon and rationale for this model stem from three primary areas: the lack of theory
and research for learner control and WBT; the research conducted by Brown ( 1999); and
the decision to conceptualize the learning process in terms of behavioral choices and
engagement choices. The following discussion also provides a preview of some of the
theoretical and design choices for the research model presented in the next section.
Lack of Theogy

As discussed in the literature review, there a need for strong, theoretical research
on the topics of learner control and WBT, and this model and study hope to contribute to
the effort. The existing research in learner control has been described as pseudo-science
(Reeves, 1993), because it does not meet acceptable standards in terms of methodology
are! theory. Some of this criticism seems to be valid. The learner control research does
Mseem to have a strong theoretical foundation that allows for researchers to conduct
Whitmmy and comparable studies. In some cases the studies seem to be

‘ " ,. . » inthat theyaredesignedtotalreadvantageofadmg

    

QM

\ .

{31}

”T
‘
Lib‘oh -

m

193‘.
,‘m'.

_.
r h
./
f1
p:

‘ T ,4 ‘v
‘- Lu!\l3

t‘». '
mm
uztfx‘

 

(11
I

A._
“-HM‘

  

«- *Mofflreshfliasmlhow
"1 " ' ' -. to adult learners, Brien with thesecriticismsinmind,du

   
 
    
     

. vresmrch (hes offer some good guidance for future researchers. It

. 2' .: _ ' m relevant variables and relationships and uncovered some of the

I mummified with learner control research. As always, good research is an

 

rv mm with new studies building on the knowledge, guidance and mistakes of
~ New studies. The criticisms of the learner control research highlight the need for a

gooddefinition and clear operalization for learner control as well as a theory-based
research model. This study proposed a definition of learner control based on the concept
of non-linear training. It also empirically tested a model in a laboratory session. This
type of basic research could provide a common foundation for future research in field
settings.

Existing definitions of learner control, and the research itself, are often not tied to
a theoretical base (Milhelm & Martin, 1991). The current theory and model addressed
these issues by providing a rich definition and theoretical base. The definition of learner
control used in this paper was based on the framework in Figure 1, and it was defined as,
“...a form of non-linear training that gives some control of the instructional event to the
lama.” This definition was built from the theory and definition of non-linear training as

_ Immature review (Kinzie, 1990) also highlighted the need to include . . q

‘ -!:"" ~, .,

     

4.1“ v~wriables,andseveral(mteryorientatim,chseIMr . > L _ 9
"Yr-xi; -‘ , .1.

...—3.!
7..

  

 

.IIELIU

“”1
..5h

 
    

‘W md‘ outcomes.

  
 

“MUM have been formdtoinflrm .Wm
.theyzwere included in the new model. Itwasllopodm m:

W and microcomputer playfulness would explain further variance in ,.

kinds Mon to the need for individual difference variables, the review showed a -
“Mafocus on the learning process (Milhelm, 1995). The model presented in the
emanation is a process model that used a decision-making framework to predict choices
made by learners. This process model used a research design that allowed actual
behavioral choices to be monitored by using the computer to track the path taken by the
learner. Existing research literature was reviewed that discussed the learning process in
other terms such as cognitive strategies (Gagne et al, 1992), self-regulation (Kanfer,
1990), meta-cognition (Pokay & Blumenfeld, 1990) and choice (Ford, et al., 1998).
These theories and motivational theories (cognitive choice and intrinsic motivation) that
are discussed in the next section provided the foundation for this new process model.
Familiar elements such as activity level and attentional focus were included in this model,
but they were re-conceptualized into a new model that defined them as behavioral and
engagement choices.

How was learner control be conceptualized in this study? In order to achieve
operational clarity, it was important to specify how learner control was manipulated and
studied in the proposed model. In this new research model and study, learner control was
aInfinitmlatiortiu which some participants got a lot of control over their learning and

w W "'D "80‘ less control. This methOdOIOEY “’35 “105‘ similar to theleuner .' ’

 

lifi'ifl

. .1
r9,

1L5“

3.3m

D"- ‘ ’1
mm.

iidm

  

 

(betwoen'leamer and program control) is a’ WWW

 

‘ -i- ‘“' J questions. Chung and Reigluth (l992)‘fvrote

Wm knot whether or not learner control should be used, but rather how
Wmm's‘ ability to use the learner control available and to decide what
“if leaner control to make available (p. 19).” Although this is an admirable attempt
to sdvarieflhe research, it is based on some unsubstantiated assumptions. One
maturation is that learner control is beneficial and contributes to learning. The mixed
results of the learner control research do not clearly support this assumption. Another
assumption is that learner control is understood as part of the learning process, such that a
learner could be taught to maximize its benefits. Another implicit assumption is that the
individual differences that relate to learner control are well enough understood to
maximize an individuals use of learner control. In the development of this research
model, it was determined that these assumptions have not yet been clearly proved or
demonstrated through robust research.

If learner control can occur on a continuum of amount of control, it is important to
understand the optimal level of control. This study offered control to both conditions,
but some learners were more limited. In the limited learner control condition, the
Program directed the participant’s learning. All participants had access to the same
material, but some were forced to view it and others could control how and when they
viewed it. The operalizations of the constructs were carefully designed in order to avoid

confounding the manipulation and the measures. For example, the low learner control

m were required to complete some exercises that were not required oftheh‘igh

 

activity level or content search measures, since one condition was constrained in that
choice. It was possible that high learner control would be a negative influence on
learning if it allowed people to bypass carefully constructed lesson designs and miss
important information. Additional research is needed around the basics of learner
control, before the next level of research is examined.

During the early, idea-generation stage of this study, the focus was on the concept
of engagement. Specifically, the interest was on interventions (such as embedded
encouragement messages or a friendly character that provided a personalized form of
social interaction) that could keep a learner engaged with the material in a self-paced
Web-based training environment. Engagement was expected to encourage the learner to
take advantage of the learning opportunities in learner control and non-linear design.
However, while trying to build a research case to support the concepts of engagement and
learner control, it was realized that there was not a good theoretical justification for that
model. It was hard to justify engagement interventions to enhance learner control when
there was not clear evidence that learner control improved performance or learning. The
learner control research does not show any clear-cut outcomes for learner control.
Although some researchers have found that learner control improves learning (Gray,
1987, Hannafin & Sullivan, 1995, Kinzie, Sullivan, and Berdal, 1988), other have found
that it leads to less learning or mixed results (Pollack and Sullivan, 1990, Lee and Lee,
1991). In view of these issues, it was determined that additional, basic research with
well-defined concepts and robust design on learner control was warranted. It is important
to clearly understand the benefits and disadvantages of learner control, before designing

interventions to enhance it.

32

'3’
't.‘

.‘;-
Ln}.

»;

‘1‘.
the

as:

 

_ .
[,9 'M

<\ I“;

This study attempted to help determine whether or not there were behavioral and
engagement differences between people who had many options and people who did not
have many. Specifically, would the learning process differ for learners in high control
versus learners in low control environments? The learning process was the real focus of
this research, so the emphasis was on the choices and behavior of the learners during the
course. The manipulation examined how course design as well as individual differences
interacted to influence learning and performance in training. In this study, all of the
participants had some level of control over the course. For low learner control
participants, the core learning was controlled, but some peripheral activities (such as
accessing the glossary, help and resources sections) were under the learner’s control. It
was possible that learners would perform just as well or better in the limited, low control
environment as they did in the high control environment. Although it did not manipulate
learner control, one model that provided a foundation for this new model is the Leamer
Choice Theory model and research developed by Brown (1999).

Comparison to Brown (1992) Model

The Brown (1999) study provided a conceptual and empirical framework for
studying WBT, and the research results suggested some future directions for research.

The research model and study presented in this study built from the foundation provided
by Brown (1999), but made some important changes to the model and approach. The
most significant changes occurred in the conceptualization of the learning process, the
move from a field setting to a laboratory setting which allowed experimental
manipulation of learner control, the addition of the concept of emotional engagement, and

the introduction of new individual difference variables.

33

2"? i
M.»-

S'llli

SL’Zli

firm 1

l‘

.41":
In; lu

.
mg“.
‘ 5

The Brown (1999) study examined the learning process in terms of learning
strategies, specifically metacognition, activity level and attentional focus. Some theories
indicate that learners use strategies such as those as well as rehearsal, organizing and
elaboration in order to improve their learning, remembering and thinking (Gagne, Briggs,
& Wagner, 1992; Fisher & Ford, 1998). Brown ( 1999) used a measure of metacognition
in his research to represent the leamer’s strategy. Metacognition is a super-ordinate
strategy that involves self-monitoring of one’s learning strategy as well as use of the
strategy itself. The data did not show a significant relationship between the
metacognition learning strategy and learning. The Learning Choices theory also
considered behavioral and cognitive effort choices to be part of the learning process. In
the research, these were operationalized as activity level and attentional focus, and they
did have marginal relationships with learning. In assessing these findings related to
meta-cognition, activity level and attentional focus, it was noted that the meta-cognitive
strategy was a non-significant process variable. As a result, this study changed focus
from the meta-cognitive strategy to a focus on behavior and engagement. Furthermore,
the variables in the learning process were re-categorized (as either behavioral or
engagement choices), and new measures were added to each category.

Activity level was a promising variable in the Brown (1999) study, so it was
retained as part of the behavioral choice measure with some modification. The
behavioral choice measure was refined into two variables to provide a finer distinction
between activity level and content choices. Brown measured the activity level of the
participant in terms of the percentage of total activities completed. These activities were

limited to a list of pre-defined activities, so the measure did not capture any other actions

34

ml

“3”»

«ml

or behaviors such as accessing extra learning resources. In this new study, activity level
was operationalized as the number of Web pages accessed by the learner. This
measurement captured additional actions such as accessing a glossary definition or using
the help section. It provided a broader measure of how much activity the learner did,
including accessing supplemental learning materials. A measure similar to the one used
by Brown (1999) has been re-defined as a content choice measure. Brown indicated that
he was not able to provide a measure of the quality of the activities completed.
Specifically, he was lacking a measure of whether or not the participant was choosing the
best or most useful activities for learning the material. This study included a measure of
content choices that did assess the quality of choices by modifying activity level measure
used by Brown. The measure calculated the percentage of pre-defined, high-quality
information choices chosen by the learner. High quality information choices were
required to meet two criteria: that they were related to information contained in the
learning objectives and that they were tested in the pretest and posttest. The difference
between this measure and the activity level measure used by Brown was the quality

aspect of the designated activities. By contrast, Brown (1999) counted the completion of
all non-required activities in the course into his percentage measure. In this study, the
content choice activities had to meet the two criteria for high quality information choices.
This provided a measure of whether or not the learner could discern the most important
content in the course and then choose it. The operalization of behavioral choices in the
learning process consisted of two variables: activity level (number of pages accessed) and
content choices (percentage of high quality information choices made).

In addition to behavioral choices, engagement choices were expected to occur

35

Ullit'

[HUG

NH

1 will.
fir la.

IA"*

.0”
. l
ntl‘ \!

during the learning process. A learner made an engagement choice (consciously or
unconsciously) about how much attention and focus to exert during the course and how
much to enjoy the experience. In Brown’s (1999) study, attentional focus showed
promise with marginal relationships to self—efficacy and learning outcomes.
Consequently, it was one of the variables in the engagement choice along with emotional
engagement. Emotional engagement was measured by a scale created for this study
based on existing scales that measured interest and flow and emotional involvement
during training. The engagement construct is discussed in more detail in the
Conceptualization of the Learning Process section.

In addition to the changes in the conceptualization of the learning process, the
current study differed from Brown (1999) in terms of research setting, manipulation of
variables and the individual difference variables included. The Brown (1999) study
traded the control of a laboratory experiment for the reality of a field setting. Although
this provided field data, this choice limited the opportunity to collect certain data,
manipulate variables and access a large sample. Since WBT is a new research area, it is
also important to get a strong research foundation in a controlled environment.
Therefore, the current model was tested in a laboratory setting. This allowed an
experimental manipulation of learner control, which was expected to increase the impact
of the variable. In his discussion, Brown (1999) also discussed the importance of finding
individual differences that would predict learning activity. He found promising results
for learning goals (mastery orientation) and self-efficacy, so these variables were retained
with some modification. This research model also introduced several new individual

difference variables such as attitude toward WBT and microcomputer playfulness, which

36

 

‘3 flf

H...

Um!

u L}
Ht]

0

L
fish it.

La .
YM‘

were hypothesized to influence the learning process due to the novelty of the medium.
Since the Web is a new medium, it was expected to have some intrinsic interest for some
people. The attitude towards WBT and microcomputer playfulness variables were
hypothesized to predict more activity and involvement in the learning process which was
expected to result in better outcomes. The current study differed from Brown (1999) in
regards to the conceptualization of the learning process, research setting, and
manipulation and choice of variables.

Conceflualization of the LearningProcess

In this research, the learning process was defined in terms of the choices made by
the learner during the training. Learners faced two primary types of choices: behavioral
choices and engagement choices. Behavioral choices were reflected by the physical
actions completed by the learner during the session. In this study, the behavioral choices
were measured by monitoring where the learner clicks during the WBT course.
Behavioral choices included a choice of how much activity to do and a choice about
which content to view. This type of measure had been included in previous research and
has had relationships with individual difference variables as well as learning outcomes
(Brown, 1999). The second type of choice in the learning process was the concept of
engagement. The concept of engagement as described in this theory had not previously
appeared in the learner control and WBT research. This section briefly reiterates the
importance of engagement to the learning process in learner controlled courses, and also
distinguishes between behavioral and engagement choices.
Most learner controlled courses are also self—paced learning, which puts the

burden of maintaining interest, focus and attention on the learner instead of on an

37

iiill

t

i

I.“
the:

ill;
5

ill

iffll

instructor. Previous learner control research often focused on cognitive strategies and

behavior experienced during the training. What seemed to be missing was a more
abstract, even emotional, description of what the learner experienced during training.
What was the learner feeling? Was she interested enough to pay attention and learn the
material? Were there other distractions in the room? Was she bored by the material?
Was she clicking the buttons and selecting exercise answers without really engaging with
the material? It seems reasonable that a minimum amount of internal or intrinsic interest,
focus and attention would be critical for success in a learner controlled course, since the
teacher and classmates are not available to provide external motivation.

This concept of interest, focus and attention can be operationalized in terms of
engagement. Engagement provides a measure of the intensity and emotional quality of a
student’s involvement and activity during learning (Conner & Wellbom, 1991). It has
been linked to behaviors such as students exerting more effort, initiating action and
concentrating better (Skinner & Belmont, 1993). These behaviors are important parts of
the learning process. In this research, engagement choices included the choice of amount
of attention to pay to the material and the choice about how emotionally involved to
become in the training. Attentional focus has been measured in previous research
(Brown, 1999), but emotional engagement has not been incorporated. Together, these
two types of engagement choices could explain some of the discrepancies and missing
variance in the previous learner control studies.

Skinner and Belmont (1993) found that engagement is related to behavior during
training. These findings indicated that engagement and behavior are separate, but related,

concepts, but very few studies actually included engagement. Behavioral choices are the

38

>..Qr \
l.n'

This

[he :1

t“ .,
Nut.

Rift

. ,.
(iii:
. .

r‘fl‘r “1
hand

he}
4‘.>;.

,n'

[filth

COECE

actions (such as accessing a new page) done by the learner during the training and can be
measured objectively. Engagement choices are less objective and provide a measure of
the emotional and motivational state of the learner. Behavioral and engagement choices
are separate aspects of the learning process that are expected to have differential
relationships to inputs and outputs of the process. Discrepancies in the learner control
research could be occurring, because the researchers have not included the concept of
engagement in their models. Certain inputs and outcomes might have stronger or
different relationships with engagement, and previous models have not captured that
variance. By not separating the concepts, previous researchers might have been
confusing the results. In summary, this model defined the learning process as consisting
of behavioral and engagement choices and expected those two types of choices to have
differential relationships with input and output variables. The need for theory in learner
controlled WBT, modifications to the Brown (1999) model and the new
conceptualization of the learning process provided the foundation for the following

research model.

39

‘.' ..
{.43 .i‘
51‘“;

Ml :

l ‘. -
in ..C
‘l .1

l
.
‘.-‘R-,.
I‘Lr “
l “k
,
1:
.*

{5* ‘
'b\’ll

RESEARCH MODEL

In response to the need for more robust research, a model and study were
developed that used a motivation-based, input-process-output framework to investigate
the learning process in a self-paced, learner controlled WBT environment. In this
section, an overview and the model itself are presented, followed by detailed discussions
of the variables in the model. These discussions highlight the motivational foundation of
the model, the decision-making framework and techniques used in the study. The
experimental design for investigating these hypotheses is presented in the Method
section.

Motivational Foundation

As the literature on learner control and self-paced learning was reviewed, it
became apparent that many of the ideas had a motivational foundation. Motivation
theory provided insight into two of the main concepts in this study: learner control and
engagement. With learner control, a course offers choices to the learner and the learner
must decide what choices to make, how many to make, and how much effort to expend.
All of these choices are likely to depend on the motivation of the learner. Motivation can
also affect how the person feels during the experience, which can lead to different
outcomes. In this study, the motivation of emotions and affective variables were
operationalized in part through the engagement choices.

Motivation is often discussed in terms of its three most common and measurable
dependent variables: direction of behavior, intensity of action and persistence of

direction-specific behaviors over time (Kanfer, 1990). Direction refers to what a person

40

in

such

,. I
Mr

.
[:JMI
All.

in

does, such as the choices made in training. Intensity refers to how hard a person works,
such as amount of activity or effort and focus. Persistence refers to how long a person
works or the ability to finish a task. The outcomes of motivational processes are the
overt and covert behaviors of the person (Kanfer, 1990). This is an important definition,
because performance is not the primary outcome of the motivational process.
Performance measures the success of the behaviors, whereas the results of motivational
processes are the behaviors themselves. This distinction fits well with the process model
of learning presented in this study. The motivational foundation influences the process
choices or behaviors, which in turn have an impact on performance outcomes. Several
theories of motivation seemed relevant to this study including cognitive choice and
intrinsic motivation theories.

Cognitive choice. Cognitive choice approaches to motivation focus on cognitive
processes involved in decision making and choice. For example, some cognitive choice
theories explore the expectancy (E) and value (V) of choices (Kanfer, 1990). Expectancy
refers to the odds of receiving an outcome and value refers to the positive or negative
aspect. Both of these aspects are involved in making choices. The choices and actions
made by people depend on subjective expectations and subjective valuation of expected
consequences of alternative actions. E x V theories assume that people behave
hedonistically when choosing tasks or level of effort. They try to maximize positive
affect by engaging in behaviors that attain positive outcomes and to minimize negative
affect and outcomes. E x V models focus on individual, purposeful choices instead of

task behavior or performance (Kanfer, 1990). Each choice made by a person could have

a different outcome depending on situational constraints and differences between choices.

41

E x V models focus on choices made by people and have often been used to
predict decision making. Campbell and Pritchard (1976) defined motivation as a label for
the determinants of: the choice to initiate effort on a certain task, the choice to expend a
certain amount of effort, the choice to persist in expending effort over time. These three
determinants were all measured by variables in the study in order to better understand the
choices made by the learner. Specifically, choice to initiate effort translated to content
choice in this study. The course recorded whether or not the learner chose to do certain
tasks (such as click a link or complete an exercise). Choice to expend a certain amount of
effort was operationalized with attentional focus as a measure of mental effort. Choice to
persist was captured through the measure of overall activity level, which provides an
overall measure of effort over time. The model in this study was designed based on a
decision-makin g framework, and used many of the E x V concepts. See the description
of the process element of the model in the next section for more details.

Intrinsic motivation. Motivation can also be considered in terms of the
orientation of the motivation such as “why” the person is motivated. The same behavior
could be caused by intrinsic or extrinsic motivation. With intrinsic motivation, the
behavior is done, because it is inherently interesting or enjoyable. With extrinsic
motivation, the behavior is done, because it leads to another outcome (such as a reward;
Ryan & Deci, 2000). For example, a learner could be intrinsically motivated during a
WBT course, because he or she enjoyed the content and was interested in the novel
medium. By contrast, the same learner could be extrinsically motivated during the course
in order to get praise from a parent or receive a high grade. Conceivably, either of these

motivational states could lead to the same behavior and performance in the class. In a

42

 

m h

leumi

 

'rrr

self-paced learning environment such as this WBT course, intrinsic motivation could play
an important role in learner behavior.

Some researchers defined intrinsic motivation in terms of a task being interesting,
and others defined it in terms of the satisfactions gained by the person (Ryan & Deci,
2000). This study included both of those indications, such that mastery orientation
considered individual differences in the interest of learning and engagement considered
happiness and satisfaction during the learning process. Additionally, intrinsic motivation
can be related to reactions to learner control and other choices made by learner during the
learning process.

Intrinsic motivation has been described as a need-motive-value approach to
motivation (Kanfer, 1990). Malone and Lepper (1987) sorted intrinsic motivation
theories into three groups. The first group consisted of theories emphasizing curiosity,
incongruity and discrepancy motives in which there is a need for stimulation and arousal.
The second group involved the need to demonstrate competency and mastery. The third
group focused on need for personal control over the environment and self-determination
(e.g. deCharms, 1968). The third group seemed to relate directly to learner control. Both
concepts (intrinsic motivation and learner control) involved control over an environment,
but learner control focused on a training environment.

Csikszentmihayli (1975) presented an alternative view of intrinsic motivation. He
did not assume a static view of motivation, instead he described motivation as a dynamic
state that ebbed and flowed over time. He coined the term “flow experiences” to describe
periods of positive, intrinsically motivated behavior in which people were very focused,

goal-oriented and in control over the activity. In the current study, engagement choices

43

CL’Tll

5316 an

We»
l

“\ l‘l
{:u; 7‘"
[\‘M‘i ‘i
.111 y r:
M

during the learning were intended to measure a concept very much like “flow”.
Engagement was intended to capture amount of focus as well as positive feelings during
the learning. This framework provided a link between control and engagement concepts.

Nix, Ryan, Manly and Deci (1999) also presented a concept that linked intrinsic
motivation, control and positive emotion. Their theory was based on subjective vitality,
which they defined as the positive feeling of having energy available to oneself. This
energy was enhanced due to intrinsic motivation, which kept experiences from being
draining. They hypothesized that people who controlled their own behavior would feel
happy, content and pleased and this would lead to greater energy. The researchers (Nix,
et. al, 1999) discussed control in terms of autonomous behavior. Learner control could
be defined as a form of autonomous behavior, because the learner is choosing how he or
she will behave. Other researchers have also indicated that autonomous behavior can
lead to more personal investment and positive experiences (deCharms, 1968; Ryan &
Connell, 1989).

Summag. Many of the concepts included in this model and study had a
motivational foundation. The cognitive choice and intrinsic motivation theories both
offered guidance about the relationships between control, emotions and behavior. The
cognitive choice theories raised the issue of considering the effect of expectancy and
value on choices and decision making. Campbell and Pritchard (1976) presented three
determinants decision making that were Operationalized in the study described in the next
section as: content choice, attentional focus and activity level. The theories of intrinsic
motivation highlighted the importance of considering “why” the person is motivated — for

intrinsic or extrinsic reasons. Since the environment in the study was a self-study

44

 

litre

mire

environment, intrinsic motivation seemed to be a relevant concept. In this study, intrinsic
motivation in learning was captured through the individual difference variable of mastery
orientation. Intrinsic motivation can also be seen as part of the engagement variable
which measured satisfaction during the learning process (Ryan & Deci, 2000). The work
of de Charms (1968) also linked intrinsic motivation to control as represented through
learner control in this study. Many of the specific elements in this model and study as
described in the next section relied on these motivational concepts.

Overview of the Model

 

The model presented in Figure 2 considered the influence of learner control and
individual differences on the learning process and learning outcomes. The model was
designed with an input-process—output format. Individual differences were inputs that
were expected to influence behavior and choices during the process, which in turn would
influence the final outputs or training outcomes.

Eggs, The primary inputs to the model were four individual difference variables,
which included mastery orientation, Web-self-efficacy, attitude toward WBT and
microcomputer playfulness. All of these variables were identified as ones which could
help explain why learners make different choices during the learning process. An
understanding of these variables could help explain differences in learning outcomes as
well as help identify people who are likely to be successful in self-paced, learner

controlled environments.

45

Fi. ..IIL- rwwflilgg l. illju _. .-..rll. lul ~ _.ii

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Behavioral _
Choices Declarative
Knowledge
Content
Choice
Individual . . .
Differences Activity Sklll-b ase d
Level Learning
Mastery T
Orientation
High/ Low
‘ Le Engagement Application
web met 0‘0”“ Self-Efficacy
Self-efficacy Control
« 1 Attentional
Attitude to \ Focus Satisfaction
WBT . With
. Emotional Tra‘mng
Mrcro- Engagement
computer
Playfulness

 

 

 

 

 

 

 

Figure 2. Research Model — Learning Process in WBT

46

‘ L
Hl A
.‘h‘t it

it. 'lt
Stilt

;
.I‘r.

 

fill]

4;“
:2 "IV“
”1* w

Process. In order to study learner control issues, it is critical to understand the

 

learning process experienced by the participants (Milhelm, 1995). The research model in
this study considers the learning process in terms of choices and operationalizes those
choices in terms of a decision-making framework. In the decision-making literature,
process models focus on the intervening steps between the introduction of informational
inputs and the decisional outcomes (Ford, Schmitt, Schechtman, Hults, and Doherty,
1989). WBT creates an ideal situation for studying training in terms of a process (input-
process-outcome) model. The individual differences are the inputs to the model; the
behavioral and engagement choices during the learning process can be measured; the
learning outcomes are the outputs to the model.

Process tracing methodology used in the decision-making literature was a good fit
for the input-process-outcome model presented in this study. In process tracing, the steps
leading to the decision are defined by the pre-decisional behavior that is observed. By
using this framework, this model can build on the rich process detail in the decision-
making literature (e. g. Beach & Mitchell, 1978). The decision-making framework can
also provide guidelines for data collection by considering methods used in process
models. WBT with learner control can be viewed as a series of decision-making steps.
For instance the learner must make decisions such as whether or not to follow the linear
order of the modules, to view resource information and to access optional practice
exercises. By tracking the sequence and type of these decisions, learning differences
between students can be identified.

Many decision-making theories focus on absolute outcomes that can be

categorized as either correct or error choices (for example, see Payne, 1982). When a

47

ten

10 ‘lr
Ugh.

w»
vii-h

475.
t

. .tri
~

 

learner takes a Webvbased training course, he or she must make a series of small
decisions about what step to take next. These decisions are usually not “correct” or
“incorrect”, but they can be used to assess the learner’s thought process. From a cost/
benefit, contingency model perspective, these types of process steps are not highly
significant decisions, since the decisions can be reversed and the learner is in control
(McAllister, Mitchell, and Beach, 1979). Based on those characteristics, the contingency
model would indicate that these simpler decisions are made by applying simple rules and
heuristics. Although the decision making may not be complex in WBT, the process can
be analyzed in order to study learning. Since the decision making is less complex, this
research does not require the SOphisticated methodology employed in the many decision-
making studies.

The current model was a process model in that it focused on the steps chosen to
proceed through the course, as opposed to only focusing on the final decision. Process
models focus on the steps that occur after input variables are introduced and that lead to
decision outcomes (Ford et al, 1989). When studying process models, it is important to
collect processing data as often as possible during the decision-making (Svenson, 1979).
These data are traditionally collected via the two primary process tracing methods: verbal
protocols and information boards. Verbal protocols require decision makers to think
aloud during the process, and information boards require them to explicitly search for
information within available alternatives.

Web-based training provides an opportunity to collect processing data through a
third method. If designed properly, a WBT course can record the path, choices and

timing used by the learner to proceed through the course. The computer can track data

48

such

llh'fll

Diff

such as: the order in which modules were accessed, number of exercises completed,
number of times reference materials were accessed, time spent in each section and the
overall course as well as performance data. This information can be operationalized to
represent the learning that occurred during the training. Learning inputs and outcomes
can be correlated with the type and amount of activity done by the learner.

In a processing model, the key elements of the learning process for the course
must be identified. In their review of the process tracing literature, Ford et. a]. (1989)
defined the salient features of the search process to be (1) depth of search, (2) sequence
of search, (3) content of search, and (4) latency of search. Depth of search refers to the
total amount of information searched. Sequence of search refers to the temporal pattern
in which the information is acquired and assessed. Content of the search refers to an
analysis of the specific content of the information acquired. The latency of the search
refers to the amount of time spent examining a specific piece of information and! or to
make a final decision. These features of the learning process can be measured by
tracking the learner’s progress.

This current study incorporated two of these search process features to
operationalize the behavioral choices: depth and content. Depth of search was
operationalized in terms of the activity level of the learner, which provided a measure of
the overall amount of activity. Content of search was operationalized in terms of whether
or not the participant accessed optional, but relevant material. Specifically, the computer
tracked whether or not the learner accessed specific information or exercises that had
been designated as critical learning information. Critical learning information was

defined as information that was tested in the pretest and posttest and highlighted in the

49

Tie t

learning objectives in each module. Sequence of the search was not included in this
study, because it would have been confounded with the learner control manipulation (the
sequence of some learners was restricted). Latency of the search was operationalized in
terms of the time spent viewing critical learning information, but the measure was not
used in the final analyses.

Another part of the learning process was the level of engagement of the learner.
Learners make choices about how engaged to become with the training. Skinner and
Belmont (1993) defined engagement as having a behavioral and emotional component.
The current model presented two similar components of engagement — attentional focus
(how much attention the learner focused on the training and material) and emotional
engagement (the positive or negative tone of the learner’s involvement in the course).

Outputs. The model indicated that the learning process variables were expected
to influence several different outcomes, including declarative knowledge, skill-based
learning, application self-efficacy and satisfaction with training. As discussed in later
sections, these outcomes were all expected to have importance for self-paced Web-based
training. The specific elements of the model and the hypotheses are described in the
following sections. The discussion begins by describing the individual differences and
the learner control manipulation and continues to describe the outcomes of the model.
Individual Difference Inputs

Beach and Mitchell (1978) developed a contingency model of decision making

that presented strategy as being a function of the task environment, characteristics of the
task, and personal characteristics of the decision maker. In this case, the task

environment was the WBT course, the characteristic of the task was the learner control

50

 

:0 pr:

1

m 4
In \. ,1
..lk'bt

design and the personal characteristics of the decision maker was the individual
differences of the learner. The task environment (the course) was consistent across the
learners, since they all took the WBT course. The characteristic of the task varied across
the learner control manipulation with some learners having more control options than
others. Within and across the two manipulation groups, additional variance was expected
to occur due to individual differences. People respond differently to situations based on
individual characteristics (Mischel, 1977), so they are a critical part of understanding the
learning process. Kinzie (1990) also highlighted the need to include individual
differences in learner control research. Several individual differences were hypothesized
to predict variations in the learner’s behavior, engagement and learning outcomes in this
model. As discussed in more detail below, these individual difference variables included
mastery orientation, Web self-efficacy, attitude toward WBT, and microcomputer
playfulness.

Many variables have been hypothesized to influence the learning process and
training effectiveness such as self efficacy, trainee attitudes, interests, values, and
expectations (Noe & Schmitt, 1986). Warr and Bunce (1995) conducted a longitudinal
training study of 106 managers to test the influence of 11 trainee characteristics on
learning scores, reactions to training and changes in rated job behaviors. They found that
general positive attitudes to the training led to better learning scores. Noe and Schmitt
(1986) found that job involvement had an influence on learning and indirectly on
performance. Noe (1986) proposed that attitudes such as reactions to training
assignments (such as assessment feedback), career and job attitudes and expectancies

influences motivation to learn which in turn influences learning.

51

recur

lrlrii

Dill

Trainee characteristics are often not the primary focus of research studies, instead
the data are collected in conjunction with other theories. However, a recent study
attempted to analyze the research concerning trainee characteristics and their influences
on motivation to learn and training outcomes. Colquitt, LePine, and Noe (unpublished)
conducted a meta-analysis in order to test the Noe (1986) model and examine the
research in 106 studies on trainee attributes and attitudes. They found support for many
of the relationships. For instance, they found anxiety had a strong negative relationship
to motivation to learn (r c = -.57) and to self-efficacy (r c = -.57). Anxiety had weaker
relationships to declarative knowledge (r c = —.16), skill acquisition (r c = -.15) and
reactions (g c = -.23). Self-efficacy was found to have strong relationships with
motivation to learn (r c = .42), declarative knowledge (r c = .30), skill acquisition (r c =
.32), post-training self-efficacy (r c = .59), motivation to transfer (5 c = .33), actual
transfer (5 c = .47) and job performance (r c = .22). These results demonstrate the
importance of individual characteristics on the training outcomes.

Theories such as the person-situation interaction theory by Michel (1977) indicate
that different environments or situations can influence the characteristics of a person,
such that the individual will think and act differently in different situations. In this
research model, the situation was the learner control manipulation. It was hypothesized
that individuals who were high and low in the individual difference variables would react
differently to the high and low learner control manipulations. Furthermore, these
reactions would differ between the behavioral choices and the engagement choices.
Learners who had low scores on the four individual difference variables were expected to

respond differently to the high/ low learner control manipulation while making

52

Wilt?

t l :‘

qt‘rfll.

if.“
.

Mir
iti‘?i
sire
{rm
Thu -
5W
ifu' 1t“
:rrzp.

l‘.
a

iva

A, r

i 3

~_:.
(‘3
(_ .

behavioral choices but were not expected to be influenced while making engagement
choices (they would always have low engagement responses). In contrast, people who
were high in the individual difference variables were expected to be influenced by the
high/ low learner control manipulation in making their engagement choices, but not when
making their behavior choices (they would always have high behavioral choices).
MasteerOrientation. One important individual difference variable related to the
learning process is mastery or learning orientation. Mastery orientation describes how
people approach learning tasks, and it has been shown to influence persistence, focus, and
achievement. People with mastery orientations believe that effort leads to successful
outcomes, and that their ability for a task can be improved (Ames, 1992; Dweck, 1986).
They engage in learning activities in order to understand the new task or concept and
enjoy the challenge of learning. In contrast, people with performance orientations are
less interested in learning the task and more interested in performing better than their
comparison group. Performance oriented people believe that successful outcomes come
from high ability and that their ability for a task is fixed and cannot be improved (Ames,
1992). Research has indicated that mastery and performance learning orientations are
two conceptually and statistically separate dimensions (Button, Mathieu, Zajac, 1996).
Much of the research on learning orientations was conducted in classroom
settings. These results indicated that classrooms emphasizing mastery goals encouraged
students to use more effective learning strategies, prefer challenging tasks, approach the
class more positively and believe that success follows from effort. Furthermore, mastery
goals encouraged students to persist when faced with errors or difficulties. In contrast,

classrooms emphasizing performance goals encouraged students to focus on their current

53

smut
son:
are:
3'1} Eli

item

v

rill t.

m

,
9

(‘r fl»;
‘vi

alt

ability level, evaluate their ability negatively, and attribute their failures to lack of ability.
Performance oriented goals lead students to avoid challenging tasks and have less
persistence when dealing with difficulties (Ames & Archer, 1988; Dweck, 1986; Dweck
& Leggett, 1988; Elliott & Dweck, 1988).

In the classroom studies, learning orientations were usually manipulated as
situational variables (Dweck, 1986). However, learning orientations can also be
considered trait variables or individual differences. A recent paper classified learning
orientation as, “...a somewhat stable individual difference variable that may be influenced
by situational characteristics (Button, et al., 1996).” The dispositional learning
orientation predisposes people to act a certain way, but the response patterns can also
vary across situations. Some recent research indicates that goal orientation can function
as a trait or a state variable, but the constructs are distinguishable (e. g. Fisher, 1998).
These studies indicate that goal orientation might have a fixed, individual component and
a malleable component that is situation specific. For the purposes of this study, learning
orientation was treated as an individual difference trait.

Students with a strong learning goal orientation should be more intrinsically
motivated to learn in a training program if the program is perceived to be building
competence (Farr, Hofmann, & Ringenbach, 1993). Mastery orientation has implications
for the learning process such as amount of effort expended. Fisher and Ford (1998)
found a link between effort and mastery goal orientation. As mentioned earlier, learners
with high mastery orientation are more likely to explore a task and have intrinsic interest
in learning (Ames, 1992; Dweck, 1986). Therefore, participants with high mastery

orientation scores were expected to access more of the critical learning content and do

34

€13.10}

Mitt?

:5 (An
\iiiku‘

more activities overall. The higher intrinsic interest was also expected to result in more
engagement choices, such as more attentional focus and higher emotional engagement.

H1: Participants with high mastery orientations will make more of the

targeted content choices, have a higher activity level, have more

attentional focus, and have more emotional engagement than participants

with low mastery orientations.

Self-Efficacy. Self-efficacyrefers to an individual’s belief that he or she can
perform a specified behavior (Mathieu, Martineau, & Tannenbaum, 1993; see Gist &
Mitchell, 1992 for a more extensive review). Social cognitive theory addresses self-
efficacy and indicates that self-efficacy beliefs are expected to influence behavior
(Bandura, 1991). A meta-analysis by Sadri and Robertson (1993) found a positive
relationship between self-efficacy and performance (r = .36) and between self-efficacy
and behavioral choice or intention (r = .30). Pintrich and DeGroot (1990) found a
positive relationship between self-efficacy and two outcomes: cognitive engagement
(rehearsal, elaboration and organizational strategies) and performance.

When considering self-efficacy, a specific, target behavior should be identified.
Therefore, computer self-efficacy refers to an individual’s confidence level in his or her
own ability to perform computer-related tasks. Negative emotions such as anxiety or low
self-efficacy toward computers can affect how a person approaches a computer-related
task and can negatively affect success at that task (Ray & Minch, 1990). Similar
constructs have been studied under the names of computer alienation and computer
anxiety (Ray & Minch, 1990). Previous research has shown that computer self-efficacy

does predict behavioral intentions and actual behavior concerning the use of computer-

related technology (Hill, Smith & Mann, 1987). Specific to WBT, Brown (1999) found

55

C'tldc

‘3‘. V‘
Cillkn

1"

mm L

lfil l
3323;

L
l...
1745*.-
“Karl

evidence that self-efficacy is related to activity level and attentional focus. Web self-
efficacy is expected to act similarly to computer self-efficacy and the Brown (1999)
measures. Confidence about successfully using the Web training medium is likely to
influence the leamer’s performance. Based on this research, it was expected that people
who were confident in their ability to use the Web would perform better in the learning
process.

H2: Participants with high Web self-efficacy will make more of the

targeted content choices, have a higher activity level, have more

attentional focus, and have more emotional engagement than participants

with low Web self-efficacy

Attitudes. Learner attitudes toward a new technology, such as the Web, may

influence the success of the new method. Attitudes have been defined as internal states
that influence the choice of personal action (Gagne, 1984). They can serve functions
such as providing information about reactions to other individual, events or objects and
guiding behavior (McCormick & 1] gen, 1985). Attitudes toward training have been
found to have significant relationships to learning outcomes (Alli ger & Janak, 1989).
Attitudes can be inputs or outcomes of the learning process. In the model of learning
outcomes proposed by Kraiger, Ford, & Salas (1993), attitudes are classified as affective
learning outcomes.

Successful technology implementations (such as introducing WBT) often depend
on the attitudes and opinions of users (e. g. Davis, Bagozzi, & Warshaw, 1989). Webster
and Hackley (1997), as described earlier, found that attitudes are an influence in distance
learning environments and specifically on learner engagement. This model hypothesizes

that attitude toward WBT would influence behavior choices and engagement choices.

Lerr

[lliilii

q
u,

. ‘ t F

i‘ L“.

|

r "'1 ’z.,

‘w‘ \ .
a, a;

‘x

.i' “.r

X‘Illr‘l

Learners with positive attitudes to WBT were expected to do more and focus more on the
training.

H3: Participants with positive attitudes toward WBT will make more of

the targeted content choices, have a higher activity level, have more

attentional focus, and have more emotional engagement than participants

with less positive attitudes toward WBT.

Microcorrgmter playfulness. Webster and Martocchio (1992) introduced and
validated a construct called microcomputer playfulness which could also influence
learning in a Web format. They saw evidence that attitudes about computers were
shifting from a fearful to a positive nature (e. g. Gardner, Young, & Ruth, 1989). Part of
this shift could be attributed to the fact that microcomputers can be fun to use once the
initial fear is overcome. Computers can encourage a state of playfulness by providing
quick responses, ease of use, and personalized features (Webster & Martocchio, 1992;
Starbuck & Webster, 1991). They (Webster & Martocchio, 1992) defined
microcomputer playfulness as a situation-specific, individual, trait characteristic that
describes an individual’s tendency to interact spontaneously, inventively and
imaginatively with microcomputers. It was proposed that people with high levels of
microcomputer playfulness would feel more involved, have a positive mood and
experience more satisfaction. Furthermore, it was proposed that playful individuals
would require less external stimuli (such as performance feedback) to sustain their
motivation. Webster and Martocchio (1992) found internal consistency, concurrent
validity, discriminant validity, predictive validity and test-retest reliability evidence to
SUpport their seven item scale of microcomputer playfulness. They also found that the

measure related positively to computer attitudes, competence and efficacy and negatively

to computer anxiety. The measure also related positively to training outcomes of

57

I .
m". ‘
it I

roul‘

illfllli

all s:

learning, mood, involvement and satisfaction (Webster & Martocchio, 1992; Martocchio
& Webster, 1992).

In this study, it was expected that people high in microcomputer playfulness
would have high interest and motivation to perform training in a novel Web-based
format. The novel format was expected to make them perform more learning behaviors
and engage more with the training.

H4: Participants with high microcomputer playfulness will make more of

the targeted content choices, have a higher activity level, have more

attentional focus, and have more emotional engagement than participants

with low microcomputer playfulness.

Learning Process Variables

 

The individual difference variables were expected to influence the learning
process of the individuals. The learning process was conceptualized as including
behavioral and engagement choices. Behavioral choices include the amount of work
completed (activity level) as well as the quality of the work completed (content choice).
The Web supports this type of data collection, since the computer can record every click
of the mouse or bit of information entered into the course by the learner. Activity level
was measured by the total number of pages accessed by the learner during the course,
providing a measure of overall activity. This measure assessed the amount of primary
and extra information accessed by the participant during the learning session. Every
participant who completes the experiment accessed a minimum number of pages in the
course. However, there were many opportunities to access more than the minimum
number of pages in the course, such as accessing the support material (glossary,
resources, help and feedback sections). These support materials were available

throughout the entire course, but were not required for completing the course or scoring

58

till
[0321]

these

“in,”

15%“ ll

well on the tests or activities. Another opportunity was to use the BACK button to return
to earlier information in order to review information. Participants who took advantage of
these opportunities and showed more activity were expected to perform better on the
learning outcomes. The activity level measure did not include accessing the screens
related to the content choice measures. Even so, the measures were expected to be
correlated, since a participant who chose one type of optional activity was likely to make
the other as well. However, activity level provided a measure of the overall quantity of
activity and content choice focused on the selection of targeted, high quality choices.
This conceptualization of activity level differed from the Brown (1999) conceptualization
which was closer to this study’s definition of content choice. Brown (1999) limited his
assessment to specific target behaviors, so he did not get a measure of the pure quantity
of activity within the course.

Content choice was the second type of behavioral choice in the learning process.
Content choice was measured by the percentage of targeted activities completed during
the training. The target activities were chosen, because they teach critical learning
information. Activities that were considered to teach critical learning information met
two criteria: their content was included in the learning objectives for the module and the
content was included in the pre/posttest. This was intended to provide an indication of
how well the learner discerned and chose critical learning information. Specifically, this
measure considered whether or not the participant completed optional exercises, accessed
extra learning information or viewed linked vocabulary words. Participants who Opted

not to access this extra learning information still had all of the information that they

needed in order to perform well on the tests and outcome exercises. This measure of

59

content choice was similar to the Brown (1999) measure of activity level, but the Brown
measure did not require included activities to meet any criteria.

The learner also made engagement choices, which reflected the level of
involvement and focus of the learner. Engagement choices included the amount of
attention focused on the course as well as the level of emotional involvement.
Engagement provided a measure of the intensity and emotional quality of a student’s
involvement and activity during learning (Conner & Wellbom, 1991). Attentional focus
has been measured in previous research (Brown, 1999), but emotional engagement has
not been incorporated. The attentional focus measure captured the level of concentration
that the participant had during the training. It asked about the participant daydreaming
during the training. Participants who were not paying attention to the material were less
likely to learn well. The emotional engagement measure looked at the participant’s level
of interest and involvement and emotional state during the training.

Learner Control as a Moderator

Learner control was manipulated in the study, such that some participants had a
lot of control and others had only minimal control. Learner control was expected to
influence the type of choices (behavioral and engagement) made by the learner. It was
hypothesized to moderate the relationship between the individual difference variables and
the behavioral and engagement choices made during the learning process. Earlier
research (Milhelm & Martin, 1991) indicated that learners should be motivated by having
the control to choose relevant information. However, individual differences are also
likely to influence how different people respond to this situational characteristic (Mishel,

1977). The influence of the moderation was expected to vary based on high and low

60

0i Ell

Salli

molt

lit ll

[’3 31'

M- is
b

scores for the individual difference variables as well as for the type of choice (behavioral
or engagement). All of the individual difference variables were oriented such that high
scores were expected to relate to more learning behaviors and more engagement. The
moderating effect of learner control was therefore expected to act similarly for each of
the four individual difference variables.

Figures 3 and 4 provide illustrations of the hypothesized interactions. As
seen in the figures, it was expected that individuals who were high and low in the
individual difference variables would react differently to the high and low learner
control manipulations. Furthermore, these reactions would differ between the
behavioral choices and the engagement choices. Figures 3 and 4 show two

general interactions, which apply to all four individual difference variables.

61

High score on ID variable

\

Behavioral Low score on ID variable

Choices Negative -

 

1

Positive 4

 

J L
F T

Low High

Amount of Learner Control

Figure 3. Behavioral Choices Interaction

High score on ID variable

Positive 4-
Engagement /

ChOICCS Low score on ID variable

I

Negative -1’

 

L l
l T
Low High

Amount of Learner Control

Figure 4. Engagement Choices Interaction

62

Flgur

Cll‘ft’

.t q .;
.ll'ltt

iui‘ll' I l

341"};
Md] g.

i'lflwo},

‘1
hrs)”
*1

 

l‘. r A
«ll, ,
‘ \

... 1,;
"wall

5., -

Niall:
\

it:

Figure 3 represents all four individual difference variables and how they were
expected to relate to behavioral learning choices across the learner control
conditions. In this figure, the performance scale reflects behavioral choices that
are high (high activity level and more content choices) and low (low activity level
and fewer content choices). Figure 4 represents all four individual difference
variables and how they relate to engagement learning choices across the learner
control conditions. In this figure, the performance scale reflects engagement
choices that are high (high attentional focus and high emotional engagement) and
low (low attentional focus and low emotional engagement).

Behavioral choices interaction. In comparing Figure 3 and Figure 4,
learners who had low scores on the four individual difference variables were
hypothesized to respond differently to the hi gh/ low learner control manipulation
while making behavioral choices but were not expected to be influenced while
making engagement choices. In other words, low scores were expected to interact
with learner control to influence behavior but not to cause any interaction that
influenced engagement.

Low scores on mastery orientation, Web self-efficacy, attitude toward
WBT and microcomputer playfulness were expected to indicate that the
participant would be less likely to work hard or show active involvement in the
training. Since they would have low intrinsic motivation, the low individual
difference score participants were expected to prefer the low learner control
condition for behavioral choices. They would not want to expel any extra energy,

would want to be told what to do, and would perform the minimum required.

63

Ind l
liltll

s
:5‘
W

tr-
; 1‘,
Illl,~

aging
"hit,

it

Low individual difference scores were expected to have a flat floor effect on
engagement choices, such that no interaction was expected across the learner
control conditions. They were not expected to become any more than minimally
involved with the training.

H5: Learner control will moderate the relationship between the individual

difference variables and behavioral choices in the learning process.

Participants with low scores on the individual difference variables will

make more of the targeted content choices and have a higher activity level

in the low learner control condition than in the high learner control
condition. Participants with high scores on the individual difference
variables will not perform differently across the learner control conditions
for behavioral choices.

Engagement choices interaction. In contrast, people who were high in the
individual difference variables were expected to be influenced by the hi gh/ low
learner control manipulation in making their engagement choices, but not when
making their behavior choices. People with high scores on the individual
difference variables were expected to be more likely to explore the training and
find it interesting and involving. These people were expected to make high
behavioral choices regardless of the learner control condition resulting in a flat
ceiling effect for behavioral choices. They would be as active as possible in
either condition. However, this same desire for the control to explore was
expected to lead to low engagement choices in the low learner control conditions.
Although they would still be expected to perform well, the low learner control/
high individual differences people were expected to resent the low control. These

explorers were expected to want to control their learning and show less focus and

involvement when denied the opportunity.

ullm;
r 1 .
Gil fill
Filth
his a
an

illtt’.

F-r'grn
(2;: in
lung

\jH

ain‘t L

H6: Learner control will moderate the relationship between the individual
difference variables and engagement choices in the learning process. Participants
with high scores on the individual difference variables will have more attentional
focus and higher emotional engagement in the high learner control condition than
in the low learner control condition. Participants with low scores on the
individual difference variables will not perform differently across the learner

control conditions for engagement choices.

Learning Outcomes

All attempts to understand and intervene in training situations should be
ultimately focused on the end goal or the intended learning outcomes. In order to
develop empirically testable variables, training outcomes should be defined as
psychological constructs that are distinguishable from learning processes (Brown, 1999).
This allows for constructs that can be assessed by different methods, but still link to
training interventions and job performance. Training outcomes should also relate to the
objectives and structure of the course itself.

For the past few decades, the concept of training outcomes have followed the
model established by Kirkpatrick (1974). This model specifies four steps for training
evaluation: reactions (how well the trainees liked the program); learning (principles, facts
and skills absorbed by the trainee); behavior (changes in on-the-job behavior); and results
(organizational outcomes such as improved efficiency). Recent literature has indicated
that this model has some conceptual problems (Alliger & Janak, 1989). It is limited

because it assumes that the four steps are in ascending order, assumes that the levels are
causally linked and it does not provide a clear taxonomy of learning outcomes (Alliger &
Janak, 1989).

In response to such criticisms, Kraiger, Ford and Salas (1993) developed a model

Which divided learning outcomes into three categories: affective, behavioral and

65

cognitive. Affective outcomes refer to attitudes and motivation that can be influenced
during training. Behavioral outcomes refer to skill-based activity including psychomotor
skills. Cognitive outcomes refer to verbal knowledge and higher order knowledge
organization and cognitive strategies such as procedural knowledge. The research model
in this study included all three types of these variables.

Affective outcomes. Affective outcomes refer to attitudes and motivational
constructs such as self-efficacy and satisfaction. Attitudes were not incorporated into the
Kirkpatrick (1974) model, except in the limited form of reaction ratings. In the Kraiger,
Ford and Salas (1993) model, attitudes and motivation are more than just training inputs,
since they can also be outcomes. If a trainee’s values undergo a change due to the
training, then learning has occurred. In previous research, affective outcomes to training
have included creative individualism (Schein, 1968), organizational commitment (Louis,
Posner and Powell, 1983), group norms (Feldman, 1984), tolerance for diversity (Geber,
1990), and changes in values (Kraiger, Ford and Salas, 1993). Affective outcomes also
include motivational constructs such as mastery and performance orientations, self-
efficacy, and goal setting. In this model, two affective outcomes were measured:
satisfaction with the training and application self—efficacy.

The relationship between satisfaction and learner control has varied in earlier
research. Gray (1987) found that learner control led to negative attitudes about the
computer program and speculated that too much control was distracting. By contrast,
Morrison et a1. (1992) found that control led to more positive attitudes. This model
hOped to sort out this conflict by providing a finer level of distinction. It was expected

that satisfaction would be tied to the level of engagement (which had not been previously

66

ll'iéi

sallsl

3L1, ' "
tilOlk

Tlil‘tll

1"]
'rs-ta
Q's

investigated) and not to behavior. As an affective outcome, it was expected that
satisfaction would be influenced by the engagement choices instead of the behavioral
choices. Satisfaction is an emotional response to how involved the participant was with
the training. Recent research indicates that satisfaction is not highly correlated with
learning or predictive or transfer (Alliger, Tannenbaum, Bennett, Traver, Shotland, 1997;
Warr and Bunce, 1995). Even so, satisfaction is a relevant outcome for training,
especially in a new medium such as WBT, which is quickly growing in popularity.
Satisfaction is likely to influence whether or not a learner would opt to take future WBT
course and could influence the general acceptance of the medium. In this study,
engagement choices were expected to influence satisfaction.

H7: Participants with high attentional focus and high emotional

engagement will be more satisfied with the training than participants with

low attentional focus and low emotional engagement.

In addition to satisfaction, application self-efficacy was another affective
outcome, which was included in the model. In contrast to the satisfaction measure, self-
efficacy has been shown to predict transfer. Self-efficacy is an individual’s belief in his
or her capacity to successfully perform a particular act or behavior. Bandura (1997)
suggests that peOple that are high in self-efficacy for a task are more likely to engage in
and persist with that task. Self-efficacy has also been shown to predict the maintenance
of new skills (Gist, Stevens and Bavetta, 1991). A measure of self-efficacy for WBT was
included as an individual difference input for the model. As an outcome measure,
application self-efficacy examines the effect of the learning process on final feelings of

efficacy to use the material taught in the course. The confidence that a trainee has after

training can influence generalization and transfer.

67

t’l‘ '
..ts

“r \ \f
”U‘LL

Slrii

,3...
k. Ii,»

Brown (1999) did not find a relationship between higher activity levels or
attentional focus and application self-efficacy. In order to determine how the learning
process influences application self-efficacy, this model considered a finer distinction.
Similar to the hypotheses for satisfaction, the affective outcome of application self—
efficacy was hypothesized to be related to engagement and not behavioral choices. Due
to Brown’s null finding for attentional focus, the relationship was only expected to occur
with emotional engagement. Specifically, people who were interested and involved in
the training were more likely to visualize themselves as using the content in the future.

H8: Participants with high emotional engagement will have higher

application self-efficacy than participants with lower emotional

engagement.

Cognitive. Cognitive outcomes encompass the most traditional idea of learning.
An accepted definition of learning is “a relatively permanent change in knowledge or
skill produced by experience” (Weiss, 1990, p.172). Cognitive learning is often assessed
using achievement tests (multiple choice knowledge tests), and can more accurately be
described as declarative knowledge. Cognitive learning outcomes are common measures
for training research and organizations (e. g. Colquitt & Simmering, 1998; Fisher & Ford,
1998). Some evidence indicates that declarative knowledge measures cannot
discriminate among trainees at higher level of experience, but they can be useful in the
early stages of skill acquisition (Kraiger, Ford, and Salas, 1993). In this study, behavioral
and engagement choices were expected to influence learning (acquisition of declarative

knowledge).

68

.nr sl
[wk

H9: Participants who make more of the targeted content choices, have a

higher activity level, have more attentional focus and higher emotional

engagement will have more declarative knowledge than participants who

make fewer of the targeted content choices, have lower activity levels,

have less attentional focus and lower emotional engagements.

Behavioral. Behavioral outcomes include skill—based measures. An important
concern of training in the workplace is whether or not the acquired skill can and will be
used back on the job (Baldwin & Ford, 1988). Skill practice scores have been shown to
predict generalization of skill and positive transfer (Ford, et al, 1998). In this study, the
skill measure was assessed using transfer application activities. Skill development and
transfer were expected to be related to the behavioral choices made by the trainee. More
activity and better choices were likely to reinforce skill-based learning.

H10: Participants who make more of the targeted content choices and have

a higher activity level will have more skill-based learning than participants

who make fewer of the targeted content choices and have lower activity
levels.

69

METHOD

mm

The participants were 249 students taking introductory psychology courses from a
large mid-west university who received course credit in exchange for participation.
Thirteen participants were removed from the analyses due to computer crashes, illness or
failure to complete final measures. The final sample size was 236 participants.

Research Design

 

The design was an experimental study. There were two conditions in the
manipulation — high learner control versus low learner control. Some of the data on
individual differences, process variables and outcome variables were collected through
self-report, online surveys. Other process variables and outcomes were collected through
computer tracking and paper-based activities. Although participants took the course
during scheduled lab sessions, each participant completed the study individually in a self-
study format.

Power Analysis

Many of the relationships hypothesized in this study have not been previously
studied, so relationships from similar studies were used to provide a baseline. In terms of
attentional focus, Fisher and Ford (1998) found a correlation of -.35 between off-task
attention and verbal knowledge. Brown (1999) found correlations between verbal
knowledge (pretests and posttests) and time on task and activity variables that ranged
from .26 to .45. Many of the significant correlations found in this study fell in a similar,
medium effect size. According to Cohen and Cohen (1983), a sample size of 230 with

medium effect sizes provided a power of .99 to reject the null hypothesis of no

70

relrzl

Oil {If

pm
tons?-
sped;
irritil
rile?
the I?

'l

“I 153.:

l
i]. \‘t uh

-l.
l0}: |‘

CT“.
~L)1L(

In?“ J
c n‘
e

:‘Uiif'l

.l‘ra'r
{“1

relationship at an alpha level of .05 for a standard model. This finding provided guidance
on needed sample size, but the interactions included in this model made it complicated to
asses the true power of the sample size for this model.
Training Course

The course used in this study is called Change Management and teaches basic
principles of change in organizations. The course is owned by Learnshare, which is a
consortium of Fortune 500 companies that share training resources. Learnshare granted
special permission for the course to be used in this research study. The Web-based
training version of the course was designed and developed by Strategic Interactive using
materials provided by Learnshare. The Change Management course was chosen because
the material was suitable for a college-aged audience, the course is based on sound
instructional design principles, and it could be modified to allow high and low conditions
of learner control.

The original course was modified in several ways to meet the needs of the study.
It was significantly shortened from sixteen hours, by removing several “specialty”
modules that dealt with specific roles related to Change Management (such as the Leader
role). The scaled-down course used modules from the primary tutorial that taught the
basic concepts of change management. The shortened course was expected to offer
enough interaction with the course to detect differences in learning processes and
outcomes. Extra screens were also added that provided information specific to the
experiment (such as instructions). The course was modified to allow online collection of
the scales and to have the results stored in a database. Special programming allowed the

collection of behavioral data such as number of pages accessed (activity level measure),

’1

I
I9?
.w.

lltilC;

i, .
I'ItmI
l1.
9

at _
,
41% a
‘5

specific choices made (content choice measure) and time spent in the course. The
original course had an option for audio in which quotes and interactions are spoken. For
the purposes of the study, audio was not an Option — everyone received the text-only
version. This maintained standardization across the participants. Finally, two versions of
the course were created to provide the two conditions of high learner control and low
learner control. Throughout the remainder of this paper, references to the course

indicated the shortened version of the course used in the study.

The course was designed with a coffeehouse, conversational theme. The learner
enters a coffee shop and reads headlines in the newspaper about change problems that are
occurring in organizations. Other coffee shop patrons engage in a conversation about the
change problems. These interactions are interspersed by instructional elements that teach
the concepts of Change Management. The course includes two primary modules: The
Concept of Delta and The Concept of Momentum. In addition, the course includes an
introduction, glossary, resources, help section and email feedback. The two modules
have learning objectives, text information and interactive exercises. Sample screen shots
from the course are included in Appendix C.

Training Technology

The course was designed to be compatible with different computer systems and
different browsers, since Leamshare’s users come from many different organizations.

The technical design specifications for the course called for Netscape 3.0 or Internet
Explorer 3.0 or higher and 800 x 600 pixel screen resolution. Data were collected in an
Access database, and no sound cards or special plug-ins were required. Computers in the

university computer laboratories were well-suited for this course. The course was hosted

72

or a
main
data I
rite

[vol I
rats
for pr
lift“)!

iron;

”6 l
is; _
“1st HUI

3‘ ll:

on a server owned and maintained by Strategic Interactive. Strategic Interactive
maintained high levels of server and user security. The data were well-secured during the
data collection process. Participants accessed the course through the Internet from a
university computer lab.
Procedure
Participants signed up for the experiment through the Psychology subject
pool web site and through special class sign-ups. A computer lab on campus was
reserved for the laboratory sessions. Participants received 2 hours of experimental credit
for participating. Many of the participants did not require this amount of time. It was
necessary to allow plenty of time, because the participants worked at different speeds
through the self—paced course. Participants worked individually in a self-study format,
and the researcher did not offer any assistance in completing the course.
The first step of the program was the consent form, which was presented online to
the participant who indicated acceptance or non-acceptance of the consent agreement.
All participants consented to participating in the experiment. After consent, participants
completed the login screen by indicating their name and identification number. The
name was only be used to give participation credit to the participants. The final dataset
did not include the names; it only contained the student identification numbers in order to
provide a unique identifier.
After the login, the computer assigned the participants to either the high control or
the low control condition (it alternated every other person). The WBT program started
by collecting the individual difference variables (mastery orientation, Web self-efficacy,

attitude toward WBT and microcomputer playfulness). The next screen presented

73

lllOIl

information about the experiment and thanked Strategic Interactive and Leamshare for
their assistance. The next screen presented an explanation of the manipulation, which
was intended to strengthen the effect. Each person was told which manipulation (you are
in the high/ low condition) he or she was receiving. The next step was for the learner to
complete the pretest, after which the actual course material and training began. The
participant proceeded through two modules and completed exercises along the way. Two
process variables (emotional engagement and attentional focus) were collected in the
middle of the course between the two modules. After completing the course the
participant took the posttest and completed the satisfaction with training scale and the
application self-efficacy scale on the computer. After they completed the course and all
of the scales, participants saw a screen indicating that the computer section of the study
was complete. They received a paper-based copy of the skill exercise from the researcher
and completed it. After the final exercise was complete, the researcher provided a
description of the study that included contact information for the researcher and
dismissed the participants.
Learner Control Manipulation

Participants were randomly assigned to one of the two experimental conditions of
high or low learner control. There were 117 participants in the low control condition and
119 participants in the high control condition. The manipulation was reinforced on an
introductory page that described the manipulation in general terms and told the
participants which type of control they were going to have. Although this reinforcement
was artificial, it was intended to have some similarities to real-life situations. As learners

become more accustomed to taking WBT, they will learn the range of possibilities for

total.

control and interaction. In future training, they will have a sense for whether or not they

have a lot of control in comparison to other courses.

In the high learner control condition, participants had more choices and were able

to take advantage of the non-linear aspects of the course. They were able to:

Review the modules in any order. The course introduced each module and
provided links to take participants to either module one or module two.
Review the content in each module in any order. A breakdown of the module
was presented including the tutorial content, exercises and extra information
sections. In the high learner control section, these elements were all links
allowing the participants to jump to different elements of the course.

Take or skip the Optional practice exercises. Five total exercises were
presented in each module, and two of them were labeled as “Required”.
However, participants in the high control condition were not required to
complete any of the five exercises in order to continue.

In the low learner control condition, participants had fewer choices and were forced to

learn the material in a more linear progression. They had some control over optional

exercises and other learning elements (such as using the glossary), but their progression

through the content was more limited. They were forced to:

Pass through the modules in a pre-deterrnined order (no control). The course
introduced each module, but there was not a link to the modules. Participants
had to start with module one.

Review the content in each module in a linear order. The same breakdown of
the modules was presented including the tutorial content, exercises and extra
information sections. In the low learner control section, these elements were
not links, so the participant had to click the next button and proceed linearly.
Be required to complete the two “Required” exercises. Five total exercises
were presented in each module, and two of them were labeled as “Required”.
Participants in the low control condition were required to complete the two
“Required” exercises before they were able to continue with the course. In
order to avoid confounding the manipulation and the measures of content
choices and activity level, only the three exercises that are optional for both
conditions were counted in the measures.

V14
,...-
(L.
-—c

 

Adeasures

Online surveys were used to collect the self-report data, except for the skill

exercise, which was completed on paper at the end of the study. The web server tracked

all other process and outcome variables to an online database. All of the survey scales

are included in Appendix A. Table 1 indicates when each measure was collected, the

method of collection (self-report (SR) versus computer tracking (CT)) and number of

items in the original scale.

Table 1: Collection of variables

 

Before Training

During Training

After Training

 

Individual
Difference
Variables

o Mastery Orientation (SR, 8)
0 Web Self-Efficacy (SR. 5)
0 Attitude toward WBT (SR,

8)

0 Microcomputer Playfulness
(SR. 7)

 

Learning
Process

0 Content Choice (CT)

0 Activity Level (CT)

0 Attentional Focus
(SR, 6)

0 Emotional

Eggagemcnt (SR. 11)

 

Outcomes

 

L

 

 

 

 

o Declarative
knowledge (SR, 14)

0 Application SE (SR,
5)

0 Satisfaction with
Training (SR, 8)

0 Skill-based Learning
(SR, essay on paper)

 

Mmtemorientation. Mastery orientation was measured by the eight-item scale

 

developed by Button, Mathieu, & Zajac (1995). The items were rated on a five-point

Likert scale ranging from Strongly Disagree (1) to Strongly Agree (5). The eight-item

mastery orientation scale had an alpha coefficient of .74.

76

 

In!

3‘. |:r‘

or.)
‘

Performance orientation. Although not hypothesized, performance orientation
was also measured as a counter-point to mastery orientation. It was measured using the
eight-item scale deveIOped by Button, Mathieu, & Zajac (1995). The items were rated on
a five—point Likert scale ranging from Strongly Disagree (1) to Strongly Agree (5). The
eight-item performance orientation scale had an alpha coefficient of .82.

Web self-efficacy. Web self-efficacy was measured with a five-item scale based

 

on the self-efficacy scale of Pintrich and De Groot (1990). The scale was modified to
measure self-efficacy toward general use of the Web. A sample item is, “I am certain I
can master the skills needed to effectively navigate the World Wide Web.” The items
were rated on a five-point Likert scale ranging from Strongly Disagree (1) to Strongly
Agree (5). The five-item Web self-efficacy scale had an alpha coefficient of .87.

Attitude toward Web-based trainirgp. Attitude toward WBT was measured using

 

an eight-item scale modified from the Attitude toward the Internet scale used in Howell,
Mullins, Fisher, Schmitt, and Kozlowski (1998). The revised scale included items such
as, “I enjoy taking Web-based training courses.” The items were rated on a five-point
Likert scale ranging from Strongly Disagree (1) to Strongly Agree (5). Scale analysis
showed that Item 7 was double barreled and did not fit well with the rest of the scale, so it
was removed. The final seven-item Attitude toward WBT scale had an alpha coefficient
of .86.
Microcomputer playfulness. Microcomputer playfulness was measured using the

seven-item scale developed and validated by Webster and Martocchio (1992). This scale

presented a list of seven adjectives. Participants were asked to choose the response

(strongly disagree to strongly agree) that best matched a description of themselves for

77

crcl
int:

Cfllt

.‘I .
[it til
A
slut:

Brim

L \.~rd
Vat

EISIt

5.4«
(”—1)
,_.

 

each adjective. In their validation of this scale, Webster and Martocchio (1992), found an
internal consistency of .86 to .90 for the seven items as well as finding proof of
concurrent validity, discriminant validity, predictive validity and test-retest reliability.
The items were rated on a seven-point scale ranging from Strongly Disagree ( l) to
Strongly Agree (7). In this study, the seven-item microcomputer playfulness scale had an
alpha coefficient of .78.

Activity level. Activity level was measured by the total number of pages accessed
by the learner during the course. This provided a measure of overall activity including
accessing optional screens and referencing SUpport material. This differed from the
Brown (1999) operalization of activity level, which was more conceptually similar to this
study’s definition of content choice. The number of total pages accessed was collected
by the computer. In order to keep measure operationally distinct, the final activity level
score was calculated by subtracting the number of pages connected to the required
exercises and the number of pages counted as part of the Content Choice measure from
the total number of pages accessed in the training.

Content choice. Content choice was operationalized using three separate, but
related, measures. The first measure was a count of the number of pre-determined
activities completed during the training. The pre-deterrnined activities were all chosen,
because they taught critical learning information. Activities that were considered to teach
critical learning information met two criteria: their content was included in the learning

objectives for the module and the content was included in the pre/posttest. This was
intended to provide an indication of the quality of effort exerted, because it monitored

how well the learner discerned and chose critical learning information. This measure was

78

similar to the Brown (1999) measure of activity level, but the Brown measure did not
require included activities to meet any criteria. There were three exercises in each
condition that were Optional, so these were included in the count for the content choices
(six total). In addition, two extra examples in each module (four total), and eight
clickable vocabulary words in the first module and seven clickable vocabulary words in
the second module (15 total) were part of the content choice count. Content choice
(count) scores ranged from zero to twenty-four.

The second measure of Content Choice was a latency or time measure. The same
critical learning information was targeted. However, instead of a count, the measure was
the amount of time (in seconds) spent viewing the targeted items. The latency or time
measure of content choice ranged from zero to 520 seconds.

The third measure of Content Choice was a combination of the first two. The
measure was the average amount of time in seconds spent looking at the target items.
Specifically, it was an average calculated by dividing the time measure by the count
measure. This was intended to provide a depth Of processing measure that differentiated
how much attention the individual items were given by the learner. The average time
measure ranged from zero to 60 seconds.

Although the range for the measures started with zero, only five of the
participants had zero as their content choice score. All of the count and time measures

were examined for the possibility of being overdispersed, which would create the need to
transform the variables before hypothesis testing. The results indicated that this was not
necessary, so the raw scores were used in the analyses. Although all three of the content

choice measures were included in the descriptives and correlation analyses, only the

79

,3,
,‘tr.

“all

u‘
. P.“-
“N“.

content choice (count) measure was used in the hypothesis testing. See the Results
section for a more thorough discussion of these decisions.

Attentional focus. This was measured with a scale similar to the one used in
Brown (1999) that asked about on-task and Off-task behavior. Attentional focus was the
amount of attention devoted to the course materials as Opposed to unrelated topics or
material. The measure was a self-report Likert scale asking the extent to which trainees
thought about the task-related and task-unrelated subjects. The scale was adopted from
Fisher (1995) and Brown (1996). It included statements such as "I am letting my mind
wander while I was learning the material". Most of the items in this scale were derived
from Kanfer and Ackerman’s (1989) measure of Off-task attention. The six-item measure
used in this study was most similar to the measure employed by Brown (1996). Items
about attention to off-task topics were reverse scored so that high scores reflect greater
task-related attentional focus. Two items (Iteml and Item 6) were removed from the
scale due to low item-total correlations and due to loading on a different factor (see factor
analysis discussion in the Results section). The final four-item attentional focus scale had
an alpha coefficient Of .85.

Emotional engagement. Emotional engagement was measured with an eleven-
item, self-report scale develOped for this study. Skinner, Wellbom, and Connell (1990)
measured engagement using a teacher report scale that assessed the level Of active
participation and the emotional tone during the class. Both of these elements were
incorporated into the self-report scale developed for this study. The items were rated on a
five-point Likert scale ranging from Strongly Disagree (1) to Strongly Agree (5). Scale

analyses and a factor analysis with attentional focus (see factor analysis discussion in the

80

Results section) indicated that three items (Item 3, Item7, Item 8) should be removed
from the scale. This resulted in an eight—item scale with an alpha coefficient of .88.

Declarative knowledge. Learning was operationalized through a measure of
declarative knowledge. Learners completed a fourteen-item, multiple choice test that
covered concepts from the course. The same test was used for the pretest and posttest
and the two test administrations were correlated at r=.28. Declarative knowledge was
measured by using the posttest score as a dependent variable in the analyses. The pretest
score was entered in the first step of the regression analysis as a control.

Skill-based learning. Skill—based learning was measured using the score on an
application exercise. Application knowledge refers to the trainee’s ability to apply
concepts from the course to new problems. Participants completed two essay activities,
which were intended to tap the knowledge taught in the course. The activities were
written by the researcher for the purposes of this study.

An answer key was developed for each activity that included target items,
vocabulary words and a rating scale. The target items were ones which would be
elements of a good answer based on the material in the course. Participants received
rating points for each item that they included in their essay answer. They were not
required to include all Of the target items. The scoring system was tested by comparing
results for two scorers on a sub-set of the essays. A correlation between the raters
indicated that the raters were consistent in their scoring (r=.9l). Based on this indication
of the robustness of the scoring system, the essays were all scored by the primary
researcher.

For each target item, the participant received a rating based on a three-point scale.

81

Participants received zero points if they did not mention the target item in their essay.
They received one point for partial responses, such as making a reference to the item
without explanation. They received two points if they completely discussed the target
item in their essay. For each activity there was also a list of target vocabulary words.
The participants received one point for each word that was used in the essay answers.
The rating scores were added up for each participant to result in a final score for the
application exercise. There were twelve target items (0-2 points possible per each) in the
first exercise and thirteen target vocabulary words ( 1 point possible per each) for a total
of 37 possible points. There were twelve target items in the second exercise and thirteen
target vocabulary words (37 total possible points). The scores for the activities were
combined, so the minimum score was zero and the maximum possible score was 74. The
results showed that the scores actually ranged from one to forty.

Application self-efficacy. Application self-efficacy was measured with a five-

 

item scale modified from Brown (1999). The Brown (1999) scale had an internal
consistency reliability of .73. The modified scale included items such as, “I am confident
that I learned enough to improve the way I handle change events in my life.” The items
were rated on a five-point Likert scale ranging from Strongly Disagree (1) to Strongly
Agree (5). The alpha coefficient in this study for the five-item scale was .82.

Satisfaction with training. Training satisfaction was measured using an eight-item
scale modified from the task satisfaction scale used in Williams (1997). The Williams
(1997) scale has a coefficient alpha of .86. The modified scale included items such as, “I
am glad I had a chance to take this course”. The items were rated on a five-point Likert

scale ranging from Strongly Disagree (1) to Strongly Agree (5)- The alpha COCffiCient for

82

the eight-item scale used in this study was .91 .

Learner control condition. There were two randomly assigned experimental
conditions of high and low learner control as described earlier. The high control
condition was labeled as a one and the low control condition was labeled as a zero. There
were 117 participants (49.6%) in the low control condition and 119 participants (50.4%)

in the high control condition.

Time in course. Although not hypothesized, the total time in the course was also

 

measured. The time was measured in minutes and ranged from 15 to 73 minutes.

83

RESULTS

Data Analysis Strategy

The first data analysis step in this study was to examine the robustness of the
measures. This involved examining the unidimensionality of some of the scales using
alpha reliability analyses, considering square root transformations and using factor
analysis to consider the factor structure of two of the measures. The scales were
modified to reflect suggested changes and inter-scale correlations were run. The
hypotheses were tested using a series Of hierarchical regressions. Finally, some post-hoe
analyses of an alternative model were run.

Analysis of the Measures

Alpha reliabilities. Alpha reliabilities were run for the self-report scales including
mastery orientation, performance orientation, self-efficacy, attitude to WBT,
microcomputer playfulness, attentional focus, emotional engagement, application self-
efficacy and satisfaction with training. Three scales, attitude to WBT, emotional
engagement, and attentional focus, were modified as a result of the analysis. The details
of the changes are reported in the Method section Of this report. All of the resulting alpha
reliabilities were acceptable and ranged from .74 to .91.

Factor analysis. When initial correlations were run between the scales, it was
noted that there was a correlation between the two engagement choice measures
(attentional focus and emotional engagement; r = .47). According to the theory, these
two measures were intended to represent the choices made by the learner about how

involved (level of attention and emotional involvement) to become with the training.

84

They were expected to be conceptually related, yet empirically distinct, variables. The
high correlation indicated that a relationship did exist, but it was important to examine
the factor structure of the variables to determine distinctness. Fifteen items were
subjected to a common factor analysis. Nine items were from the emotional engagement
scale (items 7 and 8 had already been removed due to the reliability analysis) and six
items were from the attentional focus scale. An initial factor analysis was set to pull
factors with eigenvalues greater than one and to rotate with Varimax rotation. The results
of this initial analysis found three factors that explained 53% of the variance. Few items
loaded on the third factor. The scree plot indicated that a two factor solution would be
more apprOpriate.

Another factor analysis was run which specified two factors and Varimax
rotation. The rotated factor matrix is shown in Table 2. The two factors accounted for
48% of the variance. The rotation converged in eight iterations. All of the emotional
engagement items loaded on the first factor, except for item three, which loaded fairly
evenly on both factors. Items two through five of the attentional focus scale loaded on
the second factor. Items one and six Of the attentional focus scale loaded fairly evenly on
both factors.

These results seemed to suggest a two factor solution with factor one being
emotional engagement and factor two being attentional focus. The three items that cross-
loaded did not fit well with the other items in the respective scales from a conceptual and
methodological perspective. Item three on the emotional engagement scale dealt with
actively participating in the course as Opposed to the emotional response to the training.

The first and sixth items on the attentional focus scale were both reverse scored from the

85

other items, which might have contributed to the cross-loading. In addition, the first
attentional focus question was poorly written due to being double-barreled.
Consequently, the three items that cross-loaded were removed from their respective
scales. The revised scales were still significantly correlated (r=.39), but the correlation
was lower indicating more independence. Resulting reliabilities for the modified scales
were reasonable and are reported in the Method section. These results suggested that the
two scales were related but distinct constructs.

Table 2: Rotated Factor Matrix for Engagement Choice Items

Factors
Items 1 2
Emotional Engagement 1 pg) .008
Emotional Engagement 2 (R) J .205
Emotional Engagement 3 * §8_3_ Qfl
Emotional Engagement 4 L32 .221
Emotional Engagement 5 ,__2_2 .190
Emotional Engagement 6 fi .148
Emotional Engagement 9 i .008
Emotional Engagement 10 i9; .290
Emotional Engagement 11 L608 .218
Attentional Focus 1 * .2_4_l ._2_3_4
Attentional Focus 2 (R) .106 flZ
Attentional Focus 3 (R) .245 .Sfl
Attentional Focus 4 (R) .008 _._3_2
Attentional Focus 5 (R) .142 .8_8_
Attentional Focus 6 * Ali .___3_l

(R) = Reverse Scored items; * Items dropped from final scales.

86

Descrggtives. The descriptive statistics for the scales were examined in order to
check the distributional properties of the variables. All of the self-report, psychological
scales had descriptives within normal ranges. The time and count variables displayed
much more variability with some extreme ranges. Time and count variables have the
possibility of experiencing overdispersion, which occurs when the variances exceed the
means (Long, 1997; Brown, 1999).

In order to approximate normal distribution, the variables were submitted to a
square root transformation. The descriptives and correlations of the transformed
variables were compared to the descriptives and correlations of the untransformed
variables. The descriptives of the transformed variables accounted for some extreme
values and more closely resembled normal distribution. However, the real test was to
consider the effect of the transformed variables on relationships with the other variable in
the study. The transformed and untransformed variables had slightly different
correlations with the other variables. Some of the relationships were higher and some
were lower. However, the correlations showed the same pattern of significant
relationships for the transformed and untransformed variables. Interpreting transformed
variables can be challenging, since the metrics do not reflect the original metrics for the
variable. Based on the interpretational difficulty of transformed variables and the
consistent pattern of significant relationships, it was decided to use the original,
untransformed variables in the analysis for this study. All future references to the time

and count variables are for the untransformed variables.

87

Correlations. Table 3 (see Appendix A) presents the descriptive statistics and
correlations for the variables used in the study. Means and standard deviations for each
scale are included in the first two columns. Alpha reliability coefficients are listed in
parentheses in the diagonal when apprOpriate for the measure. The correlation matrix
shows the relationships between all of the variables hypothesized in the study. In
addition, statistics are included for performance orientation and total time Spent in course,
which were not formally hypothesized. Performance orientation was included as a
counter-point measure to mastery orientation, but it was not significantly correlated to
any of the variables in the study. Total time in the course was significantly and positively
related to all of the process and outcome variables, except application self-efficacy
(r=.08). It was logical that learners who made more of the behavioral learning choices
would ultimately spend longer in the course, because each extra action takes time. The
results also suggested that learners who engage more with the course spend longer in the
course. Time in the course was related to outcomes such as post-test scores (r=.35),
performance on the skill-based exercise (r=.27) and satisfaction (r=. 19). The time the
learner spent in the course related to many of the Outcomes. This could reinforce the
need to identify ways to promote engagement and interaction with the self-paced learning

in order to keep the learner involved for longer.

88

OEE 20.5 8823 E0: 83:88 8:
05 no 385528 E $3532 .8. v a a ESE—cw?- 85065 I. 93 S. v a a EmucEmG 3:565 ...... .8925

a 326m :25 2a 035 fi

 

 

 

 

E E 3E8 :< 35:22“: man—a Be 3:086
v 935% can SSE E805 8828 03. a:

m

 

 

 

 

  

 

 

 

 

 

 

 

 

 

 

 

 

 
 

 

 

 

 

  

 

 

 

 

 

 
 

 

 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

.13 Ian. 3.3.. 8.- 8.- 8. 8.- E - 2. 8.: 78:8 2:: 8-58 52.: e .
8. 3.8. 3.8. *8. 8. 10m. 8. 5.- 8. 8. 78m 1%]
we. 8. 3. S. .2. .2. :2. we; :8 we. %

....LN. .18. *3. S. 8. B.- Z. 5.- *9. 38 SAN

tam. tom. :8. 8.- 8.- 8. 8.- 8.- 8. 8mm 8.2 anozmom .m-
8. 8. oo. *1. 8. 3.8. 8. 8. 8. oi and 322m .m|_r
8. *3. s2. 8. 8. if. 2. m8. 8. 8. and E25 m cm _a=o__oEm.m_
*3. *2. *2. No: No. wo. mo. 8.. _ _. mm. 88 Soon .22.:22 .:l
*3. .13. 3.8. *2. 8. 8. 8.- 8.. 8. 8.8 2.8 3.15 .323. .2
- 18. 32. 3.- S.- 8. 8.- 8.- :. m; 8.2 35:. >~ egosuaoEoUd

- ...... _ w. co.- .L; .- vor 00.. 8.- 2. 88: 3.2; A88 E 0E5 02.050 32.50 .m

-- 8.- 2m? 8.- 2.- 8.. 8. 8.0 8.0

- 8. 8. 8. 8. 8. On. On. 5:680 6:on 5:33 .0
$5 3.8. $9.. 8. 1.8. 8. 36 3053 «E .8: EoooEE.mL

GM: 18. 8. 8. am. Em km? 2 2552 .e
E: .2. :8. 8. 54. oeoEo-rom .ml

8w; 3. mm. 84m :osflcoio ocean—gotcha .N

3: 8. 8.4 SEE-25 .232;

m w m o H w w m a Om :32

 

 

 

 

mofiom mo 803288885 use mozanazom £83335 neeucmam .2802 ”m 29d-

89

one: 203 3820 Eu: 000E800 Baa 8—00..” 35.: Pa 038 £5 5 3.00m =<
50:53:20.. SE? 20 3:086 05 :0 335530 E 33:52 .8. v a an 3onch 330:3: ... 0:0 5. v a 00 0:005:30. 80006:. ......

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

-- 3.9. we. Isl-N. 1.8. *2. *2. ......om. 3:. 9.258500525:80-73
20.0 1.8. ......2. 3:8. 0—. 1.3.. 1.8. S. 5:008:38:
Q: 2. ti. 5. :8. .....wm. 8.- Seam-gum 8:83:52
.. «*Om. frog. *2. *2. gal. NEG-804 883-508 .m—
.. ......wm. *w_. *Ofi. ......VN. “mo-Smom .V—
-- 8. no. ....LN. 688m .2
:8 ram. mo. 2550385226592
G3 00. 800m 3:00:82 .2
-- .33 £23. .2
AoEclwév 02050 62:00 .0
$.03 BEE 020:0 22:00 .w
05003220 0:00:00 .N-
520:00 35:00 .0503 0
30:.382m 5288000022 .m
Hm? 9 003:2. .v
80050.:an .m
coagcoto accustotom .N
:2§:otO 8:332 A
2 N1 dfi mfi E 8 NH 3 cm

 

 

 

8.83 m use-w

90

Although conceptually distinct, some of the individual difference variables
showed moderate correlations with each other. For example, microcomputer playfulness
correlated with mastery orientation (r=.27), self-efficacy (rt-.40) and attitude to WBT
(r=.25). Self—efficacy also correlated with mastery orientation (r=.25) and attitude to
WBT (r=.33). These correlations are not surprising considering that all of the individual
differences have a motivational element in them.

With a few exceptions, the correlation matrix indicated that the individual
difference variables were not related to the behavioral and engagement choices. Mastery
orientation, self-efficacy and attitude to WBT were not significantly correlated with many
of the process variables. Attitude to WBT was correlated with emotional engagement
(r=.18). This suggested that learners who had a positive attitude to the training medium
became more emotionally involved with the training. Contrary to expectations,
microcomputer playfulness had a negative relationship with the two measures of content
choice (m-JS for count measure) and (r=-. 13 for time measure). This indicated that
learners who expressed an interest in playing or exploring with the computer actually
made fewer of the valuable content choices and spent less time viewing them. One
possibility was that those learners were spending time exploring other pages, but that
does not appear the case. There was not a significant relationship between
microcomputer playfulness and overall activity level (r=.02). Some of the individual
difference variables were also correlated with outcome variables, especially application
self-efficacy.

The learner control manipulation did not seem to have much of an effect on the

participants. Learner control only had a relationship with one of the predicted variables.

91

People in the higher learner control condition had a higher activity level (r=.15). There
was a significant relationship between learner control version and the pretest score
(r=.14), but that was probably due to an experimental artifact. Learners were informed of
their level of control immediately before completing the pretest. Learners in the low
control condition scored lower on the pretest, perhaps due to the negative information of
being in the low control condition. However, the version effect seemed to mostly
disappear after that initial effect. People in the higher control condition were also more
satisfied with the course (r=.15).

The correlation matrix indicated that there were more significant relationships in
the second part of the model than in the first part. Many of the process variables
correlated with each other. For example, the content choice count variable correlated
with the other behavioral choice measure, activity level (r=.33) but also with attentional
focus (r=.13) and emotional engagement (r=.l6). The final scales for attentional focus
and emotional engagement were correlated (r=.39), but were demonstrated to be separate
constructs in the factor analysis described earlier.

The three versions of content choice were examined to determine which
measure(s) should be used in further analyses. The average time content choice measure
was more strongly correlated with the other time measure of content choice (r=.65,
p<.01) than with the count measure (r=.l9, p<.Ol). Further examination showed that the
average time measure of content choice had patterns of correlations similar to that of the
time measure, but fewer of the relationships were significant. Due to the weaker showing
of the average time measure, it was not included in the analyses of the hypotheses. Not

surprisingly, the count and time measures of content choice had a high zero-order

92

correlation (r=.81, p<.01). They showed similar patterns of correlations with the other
variables, so the two measures seemed redundant. Due to the high correlation and
similarity in relationships, the count measure of content choice seemed adequate to
represent the variable. The count measure of content choice was used in the hypothesis
testing.

All of the predicted relationships between the process variables and the outcome
variables were significant, except that attentional focus was not significantly correlated
with the posttest (r=.10). These results were promising for the second half of the model.
Choices that learner made in the training did seem to be positively related to training
outcomes. Attentional focus was correlated with satisfaction (r=.35) as predicted and
also showed unexpected relationships with application self-efficacy (r=.28), skill-based
learning (r=.13) and time in the course (r=.20). This indicated that learners who paid
attention to the material enjoyed the course more, had more confidence about applying
the information, were able to apply the information in an essay format and spent longer in
the course. Emotional engagement showed a correlation with satisfaction (r=.75). Some
of the strength of the correlation could be due to a similarity in the constructs, but they
did have conceptual differences. The emotional engagement variable was a process
variable collected in the middle of the training that captured the leamer’s immediate state
and feelings at the time. Satisfaction was an outcome variable collected at the end of
training that measured the leamer’s overall reaction to the course. Emotional
engagement also showed marginal relationships with the posttest (r=.15), skill—based
learning (r=.l3) and time in the course (r=.l6). As expected, emotional engagement also

showed a positive, significant correlation with application self-efficacy (r=.49). Learners

93

who were emotionally engaged with the course seemed to enjoy the course more and
have more confidence in their ability to use the information in the future.

The content choice (count) variable showed the expected correlation with the
posttest (m.28) and a marginal relationship with skill-based learning (r=. I4). Learners
who made more of the targeted learning choices did better on the posttest and the final
essays. Unpredicted correlations also occurred between content choice and satisfaction
(1:22) and time in the course (r=.41). Activity level showed an expected relationship
with the posttest (r=.24) and a marginal one with skill-based learning (r=. 14). It indicated
that learners who did more during the training (accessed more pages) performed better on
the posttest and the final essays. Activity level was also strongly related to time in the
course (r=.47). These relationships were further explored through regression analyses
based on the hypotheses.

Hypothesis Testinfig. The hypotheses were tested using a series of regressions.

The results showed almost no support for the hypotheses linking the individual difference
variables and the process (behavioral choices and engagement choices) variables. As a
whole, mastery orientation, self-efficacy, attitude toward WBT and microcomputer
playfulness did not seem to influence choices made during the training. The relationships
between behavior and engagement choices and the outcome vauiables were more
promising. Some of the hypotheses were supported, suggesting that what the learner did
during the training did affect declarative knowledge, skill-based learning, application

self-efficacy and satisfaction.

94

Relationshig; between Individual Differences and Process Variables

The effect of the individual difference variables on the learning choices was
assessed through a series of regression analyses. For most of the analyses, a process
variable was the dependent variable and the other variables were entered in three steps.
The first step entered the four individual difference variables. The second step entered
the learner control variable and the third step entered the four possible interaction
variables.

H1, H2, H3, and H4 predicted that participants with high mastery orientation,
Web self-efficacy, attitude toward WBT, and microcomputer playfulness would make
more of the targeted content choices, have a higher activity level, have more attentional
focus and have more emotional engagement than participants low in the individual
differences. H5 and H6 predicted moderation effects of individual difference variables
and the learner control condition on behavioral choices and engagement choices,
respectively. As explained below, none of the interactions were significant, so none of
them were graphed.

Content Choice. Table 4 tested the effect of the four individual differences and
the moderators on content choice. Table 4 showed that the four individual difference
variables explained 5% of the variance in content choice (count) while the entire analysis
accounted for a total of 7% of the variance. Most of the variance was explained by
microcomputer playfulness (B = -.19; p<.01) with some explained by mastery orientation
(B = .13; p<. 10) which was marginally significant. The negative relationship of
microcomputer playfulness and content choice was significant but was actually Opposite

the hypothesized positive relationship. This finding for microcomputer playfulness

95

suggested that learners who considered themselves exploratory and imaginative computer
users chose fewer of the valuable learning activities. None of the other individual
difference variables had significant relationships with content choice.

The results in Table 4 did not strongly support the hypotheses. N 0 support was
found for the content choice elements of H2 (self-efficacy) or H3 (attitude to WBT). The
finding regarding microcomputer playfulness was actually counter to H4, which
predicted that learners with high microcomputer playfulness scores would make more of
the targeted content choices. Marginal support was found for H1 for the relationship
between mastery orientation and content choice. Learners with high mastery orientations
made more of the targeted learning choices. As seen in Table 4, the learner control
condition did not have a significant effect on content choices. However, the moderator
analysis showed marginal support for an interaction between the control condition and
self-efficacy.

Table 4: Regression Results of Content Choices (count)

 

 

 

 

 

 

 

 

 

 

 

Stg) Variable(s) B at step R2 AR2 (if
1 Mastery Orientation .13a .05* -- 4, 231
Web Self-efficacy -.06
Attitude toward WBT -.02
Microcomplter Playfulness -. l9**
_ 2 Learner Control Condition -.05 .05 .00 5, 230
3 Condition*mastery .68 .07 .02 9, 226
Condition*self—efficacy -.85 ‘
Condition*attitude .39
; Conditionjplayfulness .13

 

** indicates Significant at p < .01; * indicates Significant at p < .05; indicates srgnrficant at p < .lO

The marginally significant interaction between learner control condition and self-efficacy
was not strong enough to be a reliable result, especially since learner control was not

Significant in the previous step. These results showed no support for the content choice

96

element of H5 and H6, which predicted the moderation relationships.

Activity Level. Further analysis of H1 through H6 also showed disappointing
results for the influence of the individual difference variables on activity level. Table 5
shows the regression, which indicated that all of the variables only accounted for 5% of
the variance. None of the four individual differences had a significant relationship with
activity level. This suggested that learners were not influenced in how much to do in the
course by mastery orientation, self-efficacy, attitude toward WBT or microcomputer
playfulness. The second step of the regression did show a significant relationship for the
learner control condition ([3 = .15; p<.05). This indicated that learners in the high control
condition had higher activity levels than learners in the low control condition. None of
the interaction variables were significant in the regression. No support was found for
Hypotheses 1-6 in relation to activity level as a dependent variable.

Table 5: Regression Results of Activity Level

 

 

 

 

 

 

 

 

 

 

 

 

Step Variable(s) B at step R2 AR2 (if

1 Mastery Orientation .06 .01 -- 4, 231
Web Self-efficacy -.11
Attitude toward WBT .06
Microcomputer Playfulness .04

2 Learner Control Condition .15* .04 .03* 5, 230

3 Condition*mastery .96 .05 .01 9, 226
Condition*self-efficacy -.51
Condition*attitude -. 10
Condition*playfulness -.32

 

** indicates Significant at p < .01; * indicates Significant at p < .05; indicates Significant at p < .10

Attentional Focus. As with activity level, analysis of Hl-H6 for attentional focus

 

did not demonstrate any significant relationships as seen in Table 6. Collectively, the
variables accounted for only 4% of the variance in attentional focus. These results

Suggested that choices about how hard to focus or concentrate on the course were not

97

influenced by the measured individual differences. The amount of learner control and
interactions were also not significantly related to attentional focus.

Table 6: Regression Results of Attentional Focus

 

 

 

 

 

 

 

 

 

 

 

 

Step Variable(s) at step RI AR2 df l

l Mastery Orientation .ll .02 -- 4, 231
Web Self-efficacy .01
Attitude toward WBT .08
Microcomputer Playfulness -.03

2 Learner Control Condition -.02 .02 .00 5, 230

3 Condition*mastery -.59 .04 .02 9, 226
Condition*self-efficacy -.89
Condition*attitude .28
Condition* playfulness . 16

 

** indicates significant at p < .01; "‘ indicates significant at p < .05; a indicates significant at p < .10

Emotional Engagement. For emotional engagement, the variables accounted for
6% of the variance as seen in Table 7. The only significant relationship was for attitude
toward WBT (B = .18; p<.01). This suggested that learners who had a positive attitude to
WBT became more emotionally involved with the course. This supported the emotional
engagement aspect of H3.

Table 7: Regression Results of Emotional Engagement

 

 

 

 

 

 

 

 

 

 

 

 

 

Step Variable(s) B at step R2 AR2 (if 1
l Mastery Orientation .08 .04* -- 4, 231
Web Self-efficacy .03
Attitude toward WBT .l8**
Microcomputer Playfulness -.03
_J Learner Control Condition .07 .05 .005 5, 230—]
3 Condition*mastery -.39 .06 .012 9, 226
Condition*self—efficacy -.64
Condition*attitude .13 1
~ Conditiorfiplayfulness .72

 

** indicates significant at p < .01; * indicates significant at p < .05; 8 indicates significant at p < .10

98

Summary. As summarized in Table 14, the data did not support most of the
hypothesized relationships between the individual difference variables and the process
variables. Mastery orientation was related to content choice in partial support of H1, but
none of the other relationships in H1 were significant. Web self-efficacy did not relate to
any of the process variables, so no support was found for H2. Partial support was found
for H3 due to a positive, significant relationship between attitude toward WBT and
emotional engagement. Microcomputer playfulness showed a significant, relationship
with content choice, but that was Opposite the hypothesized relationship. None of the
other relationships in H4 were significant, so it was not supported as written. H5 and H6
hypothesized moderation from the learner control condition between the individual
differences and the process variables. However, the learner control variable did not have
many significant zero-order correlations and only had a significant relationship with
activity level in the regression analyses. The learner control manipulation was not a
strong influence in the study. No support was found for learner control condition as a
moderator, so no support was found for H5 and H6.

Relationship between Process Variables and Outcomes

H7 through H10 dealt with the relationships between behavioral and engagement
choices and the training outcomes of declarative knowledge, skill-based learning,
application self-efficacy and satisfaction with training. These hypotheses were tested
using hierarchical regression. The four individual difference variables were entered in
the first step, before entering the hypothesized variables. This provided a control for

individual differences, before considering the process/ outcome relationship.

99

S_atisfaction with trainimg. H7 predicted that the engagement choices of
attentional focus and emotional engagement would relate to satisfaction with training.

The results of the analysis are seen in Table 8. The six variables (including individual
differences) accounted for 57% of the variance in satisfaction with training. Although
not hypothesized, some of the variance was explained by a direct relationship between
attitude toward WBT and satisfaction ([3 = .20; p<.01). However, the overwhelming
majority of the variance in satisfaction was explained by emotional engagement ([3 = .71;
p<.01). This suggested that people who were more emotionally involved during the
course were more pleased with the course overall.

The initial zero—order correlations showed a significant correlation between
attentional focus and satisfaction (r=.35, p<.01). However, the regression in Table 8 did
not show a significant relationship for attentional focus, probably due to the strength of
the emotional engagement variable. To explore the effect of attentional focus, another
regression was done as seen in Table 9. In this analysis, attentional focus was added into
the regression a step before emotional engagement. The analysis in Table 9 demonstrated
that attentional focus was significantly related to satisfaction ([3 = .33; p<.01), when the
variance was not first accounted for by the emotional engagement variable. This
indicated that learners who concentrated more during the training were more pleased with
the training. These results supported H7 and also highlighted an unhypothesized

relationship between attitude toward WBT and satisfaction with training.

100

Table 8: Regression Results of Satisfaction with Training

 

 

 

 

 

 

 

Step Variable(s) B at step R2 ARZ (if
1 Mastery Orientation .09 .05* -- 4, 231
Web Self-efficacy .03
Attitude toward WBT .20**
Microcomputer Playfulness -.03
2 Emotional Engagement .71 ** .57** .52 6,229
Attentional Focus .07

 

 

 

 

 

** indicates significant at p < .01; * indicates significant at p < .05; a indicates significant at p < .10

Table 9: Alternative Regression Results of Satisfaction with Training

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Step Variable(s) B at step RT ARZ df
1 Mastery Orientation .09 .05* -- 4, 231
Web Self-efficacy .03
Attitude toward WBT .20**
Microcomputer Plaflrlness -.03
2 Attentional Focus .33** .16** .11** 5,2307
3 Emotional Engagement .71** .57** .41 ** 1 6,229—l

 

 

** indicates significant at p < .01; * indicates significant at p < .05; 3 indicates significant at p < .10

Application self-efficamy. H8 predicted that emotional engagement would relate

 

to application self-efficacy. Table 10 showed the results of this analysis. Step 1 showed
an unhypothesized direct relationship between mastery orientation and application self-
efficacy ([3 = .18; p<.01). This indicated that learners with an exploratory learning style
were more likely to feel confident about their ability to apply the information learned in
the course. The initial zero-order correlations showed correlations between application
self-efficacy and the other individual difference variables as well: self-efficacy (r=.l9;
p<.01), attitude (r=.]5; p<.05) and microcomputer playfulness (r=.13; p<.05). However,
these relationships did not remain significant when entered as a block into the regression
analysis.

The individual difference variables and emotional engagement accounted for 28%

101

of the variance in application self—efficacy. As hypothesized in H8, emotional
engagement was related to application self-efficacy (B = .46; p<.01). Learners who were
emotionally involved in the course had more confidence in their ability to apply the
information.

Table 10: Regression Results of Application Self-efficacy

 

 

 

 

 

 

 

 

 

 

Step Variable(s) B at step R2 ARZ df
l Mastery Orientation .18** .08** -— 4, 231
Web Self—efficacy .ll
Attitude toward WBT .ll
Microcomputer Playfulness .01 .
2 Emotional Engagement .46** .28** .20** 5,230 I

 

 

 

** indicates significant at p < .01; * indicates significant at p < .05; 3 indicates significant at p < .10

Declarative Knowledfigg. H9 predicted that the four process variables would relate
to declarative knowledge. The dependent variable for this analysis was the posttest score,
so the pretest score was entered in the first step as a control variable. Results indicate
that the variables account for 19% of the variance in declarative knowledge, but only
11% of the variance after the pretest is factored out as seen in Table 11. Some of the
variance is explained by the pretest control variable ([3 = .28; p<.01). The individual
difference variables were not significantly related to declarative knowledge. Only one of
the four process variables was related to declarative knowledge, providing partial support
for H9. Content choice was related to declarative knowledge ([3 = .24; p<.01). This
suggested that learners who chose the targeted learning items performed better on the
posttest. This was a logical conclusion, since one criteria for an item to be a targeted

learning item was that it related to the material on the pre and posttests.

102

Table 11: Regression Results of Declarative knowledge

 

 

 

 

Step Variable(s) j at step R2 AR2 df
l Pretest score .28 .08** -- 1,234
2 Mastery Orientation .11 .10 .02 5,230

Web Self-efficacy -.08
Attitude toward WBT .09
Microcomfliter Playfulness -.04
3 Attentional Focus .02 .19** .09** 9,226
Activity Level .09
Content Choice (count) .24**
Emotional Erygzgement .06

 

 

 

 

 

 

 

 

** indicates significant at p < .01; * indicates significant at p < .05; 8 indicates significant at p < .10

 

Skill-based learning. H10 predicted that the behavioral choice variables (content
choice and activity level) would be related to skill-based learning. This hypothesis was
only marginally, partially supported in the analysis using the count measure of content
choice as seen in Table 12. The first two steps of the analysis with six variables only
predicted 7% of the variance in skill-based learning. Step 1 showed a direct relationship
with mastery orientation (B = .14; p<.05) and a marginal direct relationship with Web
self-efficacy (B = .12; p<. 10), neither of which was hypothesized. The hypothesized
relationship shown in Step 2 showed a marginal relationship between activity level and
skill-based learning (B = .12; p<. 10). This indicated that learners who accessed more
pages did better on the essay exercises. Content choice (hypothesized) and the
engagement choice variables (Step 3, not hypothesized) were not significantly related to
skill-based leaming in the analysis. The results indicated marginal, partial support for

H10 for activity level but no support for content choice.

103

Table 12: Regression Results of Skill-based Learning

 

 

 

 

 

 

 

 

 

 

Step Variable(s) B at step RT AR2 (if

1 Mastery Orientation .14* .04* -- 4, 231
Web Self-efficacy .12‘
Attitude toward WBT -.11
Microcomputer Playfulness -.04

2 Activity Level .12 a .07* .03* 6,229
Content Choice (count) .10

3 Attentional Focus .08 .09 .02 8,227
Emotional Engagement .08

 

 

** indicates significant at p < .01; * indicates significant at p < .05; 3 indicates significant at p < .10

Mediation. Although not specifically hypothesized, the model indicated that
individual difference variables would be mediated through the process variables to affect
the outcome variables. For mediation to occur, the variables must meet certain criteria
(James & Brett, 1984). Due to the poor performance of the model in this study, many of
these requirements were not met which indicated that mediation was not occurring.

One requirement is that the input variables must be related to the mediating
variable. The analyses only showed three relationships that met this requirement
(mastery orientation and content choice: Table 4; microcomputer playfulness and content
choice: Table 4; attitude toward WBT and emotional engagement: Table 7). The second
requirement is that the mediating variable must be related to the outcome variable. Many
more of the variables fit this criteria, but only three relationships involved the variables
that also met the first requirement (content choice and declarative knowledge: Table 11;
content choice and satisfaction: Table 9; emotional engagement and satisfaction: Table
9). The third requirement is that the input variables must be related to the outcome
variables. Only one relationship that met the previous two requirements also met this

requirement (attitude toward WBT and satisfaction). The one path of mediation that

104

with:
was ;
the 11
if Iht‘
modt
satisl
13 $1

mcdi

‘* 1nd

{malt
Lillie}
reliti
SUPP
lnclu.
P<.l(

Signil

withstood the first three requirements (attitude to emotional engagement to satisfaction)
was analyzed in a mediation test as shown in Table 13. The fourth requirement was that
the input variable (attitude toward WBT) would not be significantly related to satisfaction
if the mediating variable (emotional engagement) was partialled out of the analysis. The
model that was tested was that attitude predicted emotional engagement, which predicted
satisfaction, and attitude only affected satisfaction through emotional engagement. Table
13 showed that the fourth requirement was fulfilled, indicating that attitude was fully
mediated through emotional engagement to have a relationship with satisfaction.

Table 13: Mediation Analysis of Satisfaction

 

 

 

Step Variable(s) Bat step R2— ARZ (if
1 Emotional engagement .75** .56** -- 1, 234
2 Attitude .06 .56** .00 2, 233

 

 

 

 

 

 

 

 

** indicates significant at p < .01; * indicates significant at p < .05; 3 indicates significant at p < .10

Summary. Table 14 gives a summary of the results found for the regression
analyses of the hypotheses. When applicable, the hypotheses were broken down into the
different dependent variables. The column labeled “Support?” indicated whether the
relationship was significant (Yes) at p<.01 or p<.05 or non-significant (No). Partial
support indicated that the relationship held for one, but not all of the relationships
included in the hypothesis. Marginal support indicated a significant relationship at the
p<.10 level. As the table clearly demonstrates, most of the hypotheses were not

significant. The research model did not perform well in the analysis.

105

Tab]

"1 jifilE—l

fl __

’ l

_ 1771—71—1 '

Table 14: Summary of Results

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Activity Level

 

Hyp. Dependent Variable Independent Variables Support?

1 Content Choice Mastery Orientation Yes

1 Activity Level Mastery Orientation N o

1 Attentional Focus Mastery Orientation No

1 Emotional Engagement Mastery Orientation No

2 Content Choice Web Self-efficacy No

2 Activity Level Web Self-efficacy No

2 Attentional Focus Web Self-efficacy No

2 Emotional Engagement Web Self-efficacy No

3 Content Choice Attitude toward WBT No

3 Activity Level Attitude toward WBT No

3 Attentional Focus Attitude toward WBT No

3 Emotional Engagement Attitude toward WBT Yes

4 Content Choice Microcomputer Playfulness No (opposite

sigpificant rel.)

4 Activity Level Microcomputer Plaqulness No

4 Attentional Focus Microcomputer Playfulness N o

4 Emotional Engagement Microcomputer Playfulness No

5 Behavioral Choices Individual Differences and No
Interactions with learner
control condition

6 Engagement Choices Individual Differences and No
Interactions with learner
control condition

7 Satisfaction Attentional Focus Yes
Emotional Engagement

8 Application Self- Emotional Engagement Yes

efficag

9 Declarative Knowledge Content Choices Partial (Content
Activity Level Choice)
Attentional Focus
Emotional Engagement

10 Skill-based Learning Content Choices Partial (marginal -

activity level)

 

106

 

t1

Post hoc Analysis

The results of the hypothesis testing were disappointing, and the research model
did not perform well. Even so, the correlation table indicated that there were some
relationships between the variables, even if they were not originally hypothesized. In
order to explore these relationships further, some post hoc analyses were conducted.
These analyses were intended to seek alternative relationships and models in the hope of
gaining some understanding of what occurred. This type of post hoc analysis could
provide valuable information about future directions for research. Two additional
regression analyses were run that followed the basic framework of the model. In
addition, some new models were developed and tested, based on a logical assessment of
possible relationships. It was possible that the data could tell a story framed by an
alternative model. One model with satisfaction as a mediator to the other outcome
variables showed promise in the post hoc analyses.

AdditionaLprocess variables and outcomes. The zero-order correlations indicated
that relationships were occurring between some of the process and outcome variables. In
one case, content choice was significantly correlated with satisfaction with training
(r=.22). An additional regression analysis (Table 15) was run that entered the two
behavioral choice variables and time in the course as a final step. The final step indicated
that content choice predicted satisfaction (B = .10; p<.05), even beyond the variance
claimed by the emotional engagement variable. This suggested that pe0ple who chose
more of the targeted learning items were more pleased with the course. With the
additional variables, 58% of the variance in satisfaction was explained as compared to

57% without the final step. This was not a large gain in explained variance. Although

107

 

the beta of content choice was significant, the R2 change was not significant. This
indicated that content choice did not contribute much to the explanation of variance in

satisfaction.

Table 15: Additional Regression Results of Satisfaction with Training

 

 

 

 

 

 

 

 

 

 

 

Step Variable(s) B at step RT AR2 (if
1 Mastery Orientation .09 .05* -- 4,231
Web Self-efficacy .03
Attitude toward WBT .20**
Microcomputer Playfulness -.03
2 Attentional Focus .33** .l6** .11** 5,230
3 Emotional Ergggement .71** .57** .41** 6,229
4 Activity Level -.02 .58 .01 9,226
Content Choice (count) .10*
Total time in course .03

 

 

** indicates significant at p < .01; * indicates significant at p < .05; 3 indicates significant at p < .10

A similar analysis was run for application self-efficacy as seen in Table 16. An
initial zero-order correlation between attentional focus and application self—efficacy

(r=.28; p<.01) did not remain significant in the regression analysis in Step 3.

Table 16: Additional Regression Results of Application Self-efficacy

 

 

 

 

 

 

 

 

 

 

Step Variable(s) j3pt step R2 AR2 (if

1 Mastery Orientation .18** .08** -- 4, 231
Web Self-efficacy .11
Attitude toward WBT .11
Microcomputer Playfulness .01

2 Emotional Eggaiement .46** .28** .20** 5,230

3 Attentional Focus .08 .29 .01 9,226
Activity Level -.07
Content Choice (count) -.01
_ Total time in course .02

 

** indicates significant at p < .01; * indicates significant at p < .05; 8 indicates significant at p < .10

108

 

These results suggested that content choice was related to satisfaction, but the
non-significant R2 change made the interpretation ambiguous. This relationship should
be investigated further in future research. These two post hoc analyses did not provide
much insight into the nature of the data and relationships. Since analyses based on the
original model were not providing good results, additional models were considered.
These post hoc models were constructed from existing variables, but were based on
reasonable, theoretically consistent ideas.

Alternative Model #1. Considering the constructs, it seemed possible that the
variables could be re-organized into an alternative model. A model was conceptualized
and tested that indicated that engagement choices could relate to behavioral choices
which would in turn mediate the relationship to the final outcomes. The logic was that
learners made their emotional engagement choices in terms of their amount of focus and
emotional involvement. This level of focus and involvement would dictate the overall
amount of activity and the tendency to choose targeted learning items. Learners who
were more focused and engaged would do more and make more of the targeted choices.
Amount of activity and targeted choices would affect final learning outcomes. This
model was tested as seen in Tables 17 and 18. The model as a whole did not show
support. For the most part, the engagement variables did not relate to the behavioral
choice variables, even without the individual difference variables included. In Table 17,
neither emotional engagement nor attentional focus was related to activity level. In Table
18, attentional focus was not related to content choice either. However, emotional
engagement showed a marginally significant relationship to content choice. Even with

this marginally significant relationship, only 3% of the variance was explained.

109

Table 17: Post hoc Regression Results of Activity Level

 

 

 

Step Variable(s) Batstcp R2 AR2 df l
1 Emotional Engagement .10 .01 -— 2,233 J
Attentional Focus -.04

 

 

 

 

 

 

** indicates significant at p < .01; * indicates significant at p < .05; 8 indicates significant at p < .10

Table 18: Post hoc Regression Results of Content Choice (count)

 

 

Step Variable(s) B at step RI2 AR2 df
1 Emotional Engagement .13 ‘ .O3* -- 2,233
Attentional Focus .08

 

 

 

 

 

 

 

 

** indicates significant at p < .01; * indicates significant at p < .05; 3 indicates significant at p < .10

The variables that did not perform well in Tables 17 and 18 were dropped from
further analysis, so the focus was on emotional engagement and content choice.
Reviewing earlier analyses, satisfaction was the only final outcome variable that was
related to both emotional engagement and content choice. A mediation analysis was run
to consider the relationship between emotional engagement and content choice, with
content choice mediating the relationship to satisfaction. The results did not strongly
support a mediated model. Emotional engagement was only marginally related to content
choice (as seen in Table 18). The results in Table 19 indicated that emotional
engagement had a strong direct relationship with satisfaction even when content choice
was partialled out. These findings did not support a mediated model.

Table 19: Post hoc Mediation Results of Satisfaction

 

 

 

 

 

 

 

 

 

gen Variable(s) [3 at step R2 ARZ df “I
4 Content Choice .22** .05** -- 1,234T
_2 Emotional Engagement .73** .57** .52** 2,233

 

 

** indicates Significant at p < .01; * indicates Significant at p < .05; indicates srgnrficant at p < .10

110

 

Alternative Model #2. Considering the significant correlations between the
process and outcome variables, it seemed possible that something additional was
occurring. Many of the variables had significant relationships with satisfaction, including
the other final outcome variables. One possibility was that satisfaction had a role as a
mediator to the other outcome variables instead of being a final outcome itself. Earlier
analyses (Table 15) showed that attentional focus, emotional engagement and content
choice were all related to satisfaction. Attitude to WBT also had a direct relationship
with satisfaction. Since satisfaction had many strong relationships, it was chosen for the
focus as a mediator in this analysis. Building on these finding, several additional
analyses were run to test satisfaction as a mediator between the process variables and the
final outcomes.

The analysis in Table 20 examined the relationship of satisfaction to application
self-efficacy, after controlling for the process variables. The results indicated that
satisfaction was significantly related to application self-efficacy, and the five variables
explained 40% of the variance in application self-efficacy. Learners who were more
satisfied with the course had more confidence in their ability to apply the information
after training. Table 20 also shows a relationship between emotional engagement and
application self—efficacy. It appeared that learners who were more emotionally involved
with the course also had higher application confidence. To test this relationship further,
the analysis in Table 21 entered the process variables after satisfaction. The results
showed that emotional engagement was no longer significantly related to application self-
efficacy when satisfaction was partialled out of the equation. This indicated that the

relationship between emotional engagement and application self-efficacy was fully

111

mediated through satisfaction. None of the other process variables showed significant
relationships.

Table 20: Post hoc Regression Results of Application Selfoefficacy

 

 

 

 

 

 

 

 

 

Step Variable(s) B at step R2 AR2 df
1 Activity level -.06 .25** —- 4,231
Content Choice -.02
Emotional Engagement .46**
Attentional Focus .10
2 Satisfaction with Training .59** .40** .15** 5,230

 

 

** indicates significant at p < .01; * indicates significant at p < .05; 3 indicates significant at p < .10

Table 21: Post hoc Mediation Results of Application Self—efficacy

 

 

 

Step Variable(s) B at step RT AR2 df
1 Satisfaction with Training .62** .38** -- 4,231
2 Activity level -.06 .40** .02 5,230
Content Choice -.08
Emotional Engagement .04
Attentional Focus .06

 

 

 

 

 

 

 

 

** indicates significant at p < .01; * indicates significant at p < .05; a indicates significant at p < .10

Similar analyses were run for skill-based learning. As seen in Table 23,
satisfaction had a marginally significant relationship with skill-based learning, when the
process variables are controlled. The five variables accounted for 6% of the variance in
skill-based learning. None of the process variables showed a relationship with skill-
based learning. An earlier analysis (see Table 12) found a marginal relationship between
activity level and skill-based leamin g. This marginal relationship did not stay significant
when all of the process variables were entered into the analysis in one block. When
satisfaction was entered into a regression in Step 1 without controlling for the process
variable, it showed a stronger relationship to skill-based learning (B = .09, p<.01) and

explained 4% of the variance.

112

 

Table 22: Post hoc Regression Results of Skill-based Learning

 

 

 

 

Step Variable(s) parstep R2 1 AR2 1 df J
1 Activity level .1 1 .05* -- 4,231
Content Choice .08
Emotional Engagement .08
Attentional Focus .09 p
2 Satisfaction with Training .17= .06“ .01: [5,230 J

 

 

 

 

 

 

** indicates significant at p < .01; * indicates significant at p < .05; a indicates significant at p < .10

Similar analyses for declarative knowledge indicated that satisfaction is
significantly related to declarative knowledge. For these analyses, the pretest was entered
in Step 1 as a control. Table 23 shows that satisfaction, content choice and the pretest
were significantly related to declarative knowledge. Satisfaction and making targeted
learning choices affected performance on the posttest. The six variables account for 21%
of the variance in declarative knowledge (12% without pretest variance). The
relationship between content choice and declarative knowledge was further explored in
Table 24. Results indicated that content choice was still significantly related to
declarative knowledge, even after controlling for satisfaction. This indicated that content
choice has a direct relationship to declarative knowledge and a partially mediated
relationship to it through satisfaction.

Table 23: Post hoc Regression Results of Declarative Knowledge - l

 

 

 

 

Step Variable(s) B at step 1 R2 1 AR2 L Mi
1 Pretest .28** .08** L -- 1 1,235—\
2 Activity level .10 .18** .10** 5,230

 

Content Choice .23**
Emotional Engagement .07
__ Attentional Focus J .03

3 Satisfaction with Training \ .29** X .21** \ .03** X 6,229—\

 

 

L

** indicates Significant at p < .01; * indicates Significant at p < .05; indicatcs Significant at p < .10

113

Table 24: Post hoc Regression Results of Declarative Knowledge - 2

 

 

 

 

 

 

 

 

Step Variable(s) Bat step RT J AR2 1 df_l
1 Pretest .28** .08** I -- P1,235J
2 Satisfaction with Training .25** .14** I .O6** 2,234_j
3 Activity level .1 1 .21 ** .07** 6,229
Content Choice .2 l * *
Emotional Engagement -.14
Attentional Focus .01 J

 

 

 

 

 

** indicates significant at p < .01; * indicates significant at p < .05; 3 indicates significant at p < .10

Model Fit. The results of the different regression analyses indicated several
significant relationships. Overall model fit for the significant relationships was tested
using structural equations modeling (AMOS program). Several modifications were made
to the model based on feedback from initial analysis runs. One of the best fit models was
represented in Figure 5. The included values are standardized beta weights and
correlations. The results indicated good fit for the model. The model had a Chi Square
value of 39. 16 (P=. 12) with 30 degrees of freedom. The CFI was .98, the GFI was .97,
the AGFI was .94 and the RMSEA was .03. The data did support the model.

Summag. The resulting post hoc model indicated the importance of satisfaction
in meditating the relationship between the process variables and the final outcome
variables. Activity level showed no significant relationships and was not included in the
model. Emotional engagement had a direct relationship with satisfaction and a
relationship with application self-efficacy that was fully mediated through satisfaction.
Attentional focus related to satisfaction, but not directly to the final outcomes. Content
choice related to satisfaction and also had a direct relationship to declarative knowledge.

The three process variables predicted 58% of the variance in satisfaction. Satisfaction in

114

 

turn had significant relationships with application self-efficacy, skill-based learning and
declarative knowledge. Mastery orientation showed some direct relationships to content
choice, application self-efficacy and skill-based learning. Attitude toward WBT was
related to emotional engagement and microcomputer playfulness was related to content
choice. The key point of interest in this model was the importance of satisfaction as an

intermediary between the process variables and the final outcomes.

115

 

~0UOE 00TH HmOnm .W Duswmm

u E 323/
.meozfiotoo new 3230.5 Eon “03689:“; 08 Esme E .

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

@ _ zoo“ 8.2101 >e_n_
.300 9. E280 a; ano-o__2
S. 9. mo. 6
8 9 .....r.
@ 9:5me 9 C... \3 \ .0500“. N.
. ___xm .wzmm doze.
mo. / mm.
a an.
e. m .0985 t Fm>> 2
.OEM mw. 0U3:3< N.
t.
m mm
. e
. 9.
mm mp
C.
5on

beams.

DISCUSSION

This study was designed to consider the effects of individual differences and
learner control on choices made by learners during a Web-based training session. The
behavioral and engagement choices made by the learners were expected to influence final
training outcomes. The research model presented in Figure 2 hypothesized relationships
moderated by learner control between four individual differences (mastery orientation,
Web self-efficacy, attitude toward Web-based training, microcomputer playfulness) and
behavioral and engagement choices of the learners during training. Behavioral choices
included activity level and content choice, while engagement choices included attentional
focus and emotional engagement. These choices were expected to relate to the final
outcomes of declarative knowledge, skill-based learning, application self-efficacy and
satisfaction.

This model had a motivational framework based on the theories of cognitive
choice and intrinsic motivation. Motivational concepts tied to many of the variables in
the study. For example, the content choice variable represented the direction element of
motivation, because it measured what the participants did in the training. Content choice
also translated to the ‘choice to initiate effort’, which was a component of the cognitive
choice theory (Campbell & Pritchard, 1976). Activity level and attentional focus
represented the intensity element of motivation, since they measured how hard the
participants worked. In the cognitive choice theory, activity level could represent the
‘choice to persist’, and attentional focus could represent ‘choice to expend a certain

amount of effort’. Intrinsic motivation provided insight into the concepts of learner

117

control and engagement. Engagement was intended to capture focus and feelings in the
training, which related to intrinsic motivation concepts such as Csikszentmihayli’s (1975)
theory of Flow. Additional research (Nix, Ryan, Manly, Deci, 1999) indicated that
control over behavior led to contentment and energy, suggesting the importance of
learner control for training. The motivational theories provided a foundation for the
model. The cognitive choice theories highlighted the importance of expectancy and
values on decision making in training. The intrinsic motivation theories highlighted the
importance of considering “why” the person was motivated.

Regardless of the theoretical foundation of the model, the results did not indicate
a high level of support for the model. The individual difference variables did not display
many of the hypothesized relationships. Mastery orientation and microcomputer
playfulness were Si gnificantly related to content choice, although the microcomputer
playfulness relationship was opposite from the hypothesized relationship. Another
significant relationship occurred between attitude toward WBT and emotional
engagement. The learner control manipulation for the study did not Significantly relate to
any of the variables or act as a moderator as hypothesized. More promise was seen in the
second half of the model in the relationships between the process variables and the
outcomes. As hypothesized, the engagement variables, attentional focus and emotional
engagement, were both related to satisfaction. For the behavioral variables, content
choice was related to declarative knowledge and activity level was related to skill-based
learning. These significant findings suggested that there is value in using a process
model to study training.

The findings of the study are described in more detail in the following sections,

118

which are broken down by discussions of learner control, engagement choices, and
behavioral choices. The primary discussion is followed by an exploration of the post-hoe
analyses, the problems with the individual difference variables, limitations and future
research Opportunities.

Learner Control

The experimental manipulation varied the level of control given to two groups of
participants. One group had a hi gh-level of control over the order and content of the
training, and the other group had less control. It was expected that participants would
make different behavioral and engagement choices during training based on the
individual differences and level of control. Control was hypothesized to moderate the
relationship between the individual difference variables and the choices. Hypothesis (H)
5 predicted that more behavioral choices (content choice and activity level) would be
made by participants with low scores on the individual differences in the low control
condition as compared to the high control condition. H5 predicted no difference across
conditions for people with high scores on the individual difference variables. H6
predicted that participants with high individual difference scores would show higher
engagement scores (attentional focus and emotional engagement) in the high control
condition than in the low control condition. No difference across conditions was
predicted for participants with low individual difference scores.

The results did not Show any support for H5 or H6. Learner control was not a
Significant variable in any of the analyses, and none of the interaction variables were
Significant for any Of the predicted relationships. The learner control manipulation did
not have much of an effect on any of the variables, even most of the zero-order

correlations were not significant. Learner control showed small correlations which were

119

significant at p<.05 level with activity level (1:15), the pretest (r=.14) and satisfaction
(r=.15). These initial correlations suggested that participants in the higher control
condition accessed more pages, did better on the pretest and were more satisfied with the
course. These results, especially the significant correlation with the pretest score,
suggested that experimental artifacts from the design might have influenced results. In an
attempt to strengthen the effect of the manipulation, participants were clearly told that
there were two manipulations of control and which manipulation they had. This
information was given to the participants immediately before taking the pretest. Leaming
that they were in the low control condition might have caused some of the participants to
put less effort into the pretest and perform more poorly. This influence seemed to
disappear by the posttest, since the posttest was not significantly correlated with the
learner control condition. This finding caused some suspicion about the other two
correlations as well. They might have been legitimate correlations, or they might have
been caused by motivational differences as participants were intentionally told they have
less or more control. In any case, the correlations did not lead to significant relationships
in the regression analyses.

Unfortunately, these results contributed some additional confusion to the
complicated learner control findings. Learner control studies have differed on
methodology and findings. Kinzie, Sullivan and Berdal (1988) found that learner control
had a positive impact, since students with more control (learner control) performed better
than students with less control (program control). However, they used a different design
that focused on learner control of exercise feedback, as opposed to the design in this

study which focused on control over pace, sequence and content. The methodological

120

differences make it difficult to compare Studies, but initial comparisons indicate
differences in findings. In contrast to Kinzie et a1. (1988), Lee and Lee (1991) found
mixed results. They found that the influence of the different types of control differed for
different types of outcomes. Specifically, program control was more effective for
knowledge acquisition, and learner control was more‘effective for knowledge review.
The current study measured four process variables and four outcome variables, but
learner control did not relate to them. The learner control literature and the current
research show many differences in results, methodology and design. These differences
make it difficult to accurately compare the research. This study did not help bring clarity
to the concept Of learner control. As discussed in a later section, one conclusion might be
that it is time to stop studying learner control using the existing paradigm and find a new
one such as considering it as an element of engagement.

The current study tried to learn from previous research and address some of the
concerns raised about the learner control research (Reeves, 1993), but was not completely
successful. Reeves (1993) specified a need for a precise definition of learner control.
This study developed a definition of learner control based on the concept of non-linear
training, which is controlled by the learner. This provided a framework for considering
learner control and also for identifying contrasting types of control, such as non-linear
training controlled by a computer. This study had some success at addressing the
conceptual, definition issue. The second problem noted by Reeves (1993) was the need
for a theoretical foundation. This study attempted to develop a model of learner control
based on the principles of motivation. However, the results indicated that the model was

not supported, so this study did not further the learner control research. If motivation is

121

one of the true foundations Of the learning process in WBT, this study did not provide
any supporting evidence. However, as noted later, many of the significant relationships
in the post hoc model were the variables with a motivational foundation. These
motivational aspects offer some promise and should be explored in future research.

This study did not overcome Reeves’ third concern, which was the frequency of
technical and methodological problems. Learner control seems to be a difficult design
issue to manipulate. Similar to previous research, this study had difficulty creating an
effective and realistic manipulation of learner control. For example, this study told the
participants which condition they were in immediately prior to the pretest and
consequently created an artificial effect. This effect was apparent through the correlation
between the learner control condition and the pretest score (n.14). Learners in the low
control condition performed more poorly on the pretest, likely due to lower attention and
motivation as a result Of the condition assignment. The effect of the control condition did
not play a role in later relationships, so the effect seemed to wear off after the pretest. It
was hoped that the problem with the artificiality of revealing the condition would be
overcome by the benefit of strengthening the manipulation.

The null effects for the rest of the learner control relationships indicated that the
manipulation was probably was not very effective. Many researchers have tried different
manipulations of learner control with mixed results. Learner control is a very
individualized process, and perhaps it cannot be effectively manipulated to find solid
results. In the limitations section, this issue of manipulating learner control was
discussed in more detail. One proposition was that current methods of Studying leamer

control are not fruitful, and alternative approaches should be considered.

122

Reeves’ final concern was with analytical issues, such as small sample sizes and
large attrition rates. This study did overcome those issues. The sample size of 236 had
sufficient power for the analyses, and very few people dropped out of the study (usually
dropped due to computer troubles). However, the large completion rate was also due to
the study design. Having peOple take the course in a lab setting encouraged completion
Of the course. Completion rates would probably have been much lower if participants
were instructed to complete the course on their own, as is common in real work settings.
In summary, this study did not find any indication that learner control influenced process
choices or final training outcomes. However, it was difficult to tell whether the null

results were due to methodological problems or a genuine effect.

Engagement Choices

The engagement concept showed more promise than learner control. Engagement
referred to the level of involvement of the learner. It was expected to be an important
element of the learning experience, especially since this study involved a self-paced
learning environment. When the learner is responsible for directing his or her own
learning experience and for completing the material, involvement and motivation could
be a critical factor for good performance. Skinner and Belmont (1993) found differences
between the behavior of engaged and disaffected students, and this study also found
effects for engagement.

In this study, engagement was conceptualized as a process variable that reflected
choices made by the learner during the training. The engagement choices were
operationalized as attentional focus and emotional engagement. Attentional focus was
the learner’s assessment of the amount of attention given to the training. Emotional

engagement reflected the learner’s emotional involvement and positive feelings during

123

the course. Hypotheses 14 all partially dealt with the influence of the four individual
differences on the two engagement variables. NO support was found for the relationships
of attentional focus or emotional engagement to mastery orientation (H1) or to Web self-
efficacy (H2) or to microcomputer playfulness (H4). No significant relationships were
found during hypothesis testing or even with the zero—order correlations. These null
results were surprising, especially the lack of relationship between mastery and the
engagement variables. Earlier research found that engaged students were more likely to
chose difficult tasks, initiate action and exert effort and concentration (Skinner and
Belmont, 1993). These elements sound very similar to the description of people with
high mastery orientation who like to explore tasks and have an intrinsic interest in
learning (Ames, 1992; Dweck, 1986). Even so, this study found no relationship between
mastery orientation and engagement. One possible reason for the null results was that
mastery orientation and emotional engagement focused on different elements Of the
learning process. Specifically, mastery orientation focused on tasks and performance.
Similarly, content choice was a behavioral choice that was task focused, and mastery
orientation did have a positive relationship with it. By contrast, emotional engagement
focused on feelings. These two elements of learning (task focus and emotional focus)
might be different enough that they do not affect one another. A second possibility is that
the mastery orientation variable did not have enough variance to Show differences. The
mean of mastery orientation was 4.02, indicating that many of the learners rated
themselves high in it. The SD was only .42, indicating that there was limited variance in
the measure. This homogeneity might have limited the relationship with emotional

engagement. A third possibility is that the experimental design did not stimulate the

124

 

 

relationship. For example, the short duration of the course might have limited the effect
of mastery orientation and emotional engagement, because the course was not long
enough to call out differences.

Although emotional engagement was not related to mastery orientation, it did
have a significant relationship with attitude toward WBT (H3). This provided the only
significant relationship for H3. Learners who had a positive attitude about training on the
Web were more emotionally involved in the experience. This supported previous
research that also found a link between attitudes and engagement for distance learning
(Webster and Hackley, 1997). The leamer’s attitude before the training could be
considered an emotional state, and that emotional State influenced the leamer’s emotions
during the training. Attitude toward WBT was not significantly related to attentional
focus as also predicted in H3.

The first part of the model did not strongly support the engagement variables, but
more results were seen for the second part of the model. The engagement variables
showed relationships to two of the hypothesized training outcomes. Both attentional
focus and emotional engagement were positively related to satisfaction as predicted in
H7. Learners who chose to pay attention to the course and become emotionally involved
were more satisfied with the training. Engagement during the training related to final
feelings toward the training.

The results also supported H8, as emotional engagement was positively related to
application self-efficacy. Learners who were emotionally engaged with the training were
more confident in their ability to apply the information. H8 differed from the other

hypotheses in that only one of the four process variables was predicted to influence the

125

application self-efficacy outcome. This decision was made for several reasons. First, this
study expected that application self-efficacy would be related to the engagement
variables and not to the behavioral variables. Similar to satisfaction, application self-
efficacy is an affective outcome. Affective outcomes capture emotional responses to the
training. The engagement variables are also affective in nature and therefore were
expected to have more influence on application self-efficacy than the behavioral
variables. Second, previous research failed to find a relationship between attentional
focus and application self-efficacy (Brown, 1999). This finding led to the hypothesis
being narrowed further, such that attentional focus was not expected to have a significant
relationship with application self-efficacy. Third, emotional engagement had not been
tested in relation to attentional focus. It seemed conceptually logical that the two could
be related, since they are both affective and emotional in nature. Consequently, H8 only
predicted a relationship between emotional engagement and application self-efficacy. As
indicated earlier, H8 was supported, and additional analyses were done to confirm the
lack of relationships with the other process variables. Post-hoe analysis (Table 16)
supported the expectation that attentional focus and the behavioral variables were not
related to application self-efficacy. There was an initial zero-order correlation (r=.28)
between attentional focus and application self-efficacy, but this was not supported during
the regression analysis. These results indicated that application self-efficacy could be
largely explained by emotions during the training.

The model was examined to see if any mediation was occurring between the
individual difference variables and the outcomes through the process variables.

However, only one path met the criteria for mediation and was tested. The tested

126

mediation hypothesized that attitude toward WBT related to emotional engagement,
which related to satisfaction, and attitude only related to satisfaction through emotional
engagement. AS seen in Table 13, this path of mediation was supported. Engagement
facilitated the relationship between pre-training attitudes and post-training affective
reactions. All three of the involved variables are affective and emotional in nature,
indicating the importance of feelings during learning experiences.

NO support was found for H9, which predicted a relationship between the two
engagement variables and declarative knowledge.

Behavioral Choices

 

In addition to the relationships between engagement variables and individual
differences and training outcomes, the behavioral choices also showed some
relationships. It was expected that choices, such as whether or not to view a page or
complete an optional exercise, would relate to the individual differences and outcomes.
Learners who had high scores on the individual difference variables were expected to
make more behavioral choices during the training. The logic was that learners who were
active during training and made good choices would perform well on outcome measures.
This study found limited support for some of these expectations.

Behavioral choices were operationalized as content choice and activity level.
Content choice measured the number of times the learner accessed pre-defined
“important” learning items, and activity level considered the overall number of non~
required pages accessed by the learner. Hypotheses 1-4 all included relationships
between the individual difference variables and the behavioral choice variables. Results
indicated that activity level was not related to any of the four individual differences,

providing no support for that part Of H1-H4. Similarly, no relationship was found

127

 

between content choice and Web self-efficacy (H2) and attitude toward WBT (H3).
However, content choice was related to mastery orientation and microcomputer
playfulness. As predicted in H1, learners who were more mastery oriented in their
learning made more of the targeted learning choices. This finding supported previous
research. People with mastery orientation believe that effort leads to successful outcomes
and enjoy the challenge of learning (Ames, 1992; Dweck, 1986). This translated well to
the self-paced learning environment of the WBT course. Specifically, it seemed as if the
learners with high mastery orientation made the extra effort of identifying the important
learning concept and of choosing the targeted items. Previous research found a link
between mastery orientation and effort (Fisher & Ford, 1998). However, this current
study seemed to suggest that choosing good learning items was more important than
unqualified, overall effort. These results indicated that learners did not simply exert more
overall effort when they were high in mastery orientation, because the activity level
element was not significant. More specifically, learners high in mastery orientation
expended more effort on choosing important learning concepts. Mastery orientation
seemed to help learners focus their extra effort on making good choices.

A significant relationship was also found between content choice and
microcomputer playfulness. However, the significant relationship was Opposite of the
hypothesized relationship, so no support was found for that element of H4. It was
originally hypothesized that learners who interact positively and imaginatively with
computers would make more of the content choices, due to interest in the novel training
format. The Opposite was found in that peOple high in microcomputer playfulness made

fewer of the important learning choices. One possibility was that learners who enjoyed

128

playing on the computer were off-task, exploring other elements of the course and were
distracted from the content. However, the lack of relationship between microcomputer
playfulness and activity level indicated that the learners were probably not off exploring
other sections of the course. Of all the relationships hypothesized between the individual
difference variables and the process variables, only three were significant. Two of those
three involved content choices, but one of those relationships was in the opposite
direction.

Some relationships were also found between the behavioral choice variables and
the two hypothesized outcomes. H9 predicted that the four process variables would relate
to declarative knowledge, as measured by the posttest. As mentioned earlier, the
engagement variables did not relate to declarative knowledge. Activity level also proved
not to be related to declarative knowledge as seen in Table 11. Choosing to view more
pages did not seem to influence final performance on the posttest. However, content
choice was positively and significantly related to declarative knowledge in partial support
of H9. Targeted learning choices were designated as such, because the content related to
the material on the pre and posttests, and they were represented in the learning objectives.
Since the material in the targeted items was directly related to the material on the posttest,
this significant relationship is very logical. Learners who chose to read or complete the
targeted learning items performed better on the test. It is likely that the learners were
consciously noting the material in the pretest and attending to the learning Objectives in
order to ascertain which pieces Of content were important. By choosing the important
pieces, the learners who made more content choices had corresponding success on the

posttest. The pretest and the learning objectives were probably acting as advanced

129

organizers to inform learners about the important concepts of the training.

H10 predicted that content choice and activity level would predict skill-based
learning. Skill-based learning was measured by performance on two essays that asked
the learners to apply the change management concepts from the WBT course. Results (as
seen in Table 12) indicated that content choice was not related to performance on the
essays. However, a marginally significant relationship was found between activity level
and skill-based learning, providing partial support for H10. This indicated that learners
who viewed more pages during the training performed better on the transfer essays.
Although the information on the extra pages was not necessary for strong performance, it
was likely that the learners benefited by having concepts reinforced.

One of the interesting findings of this study was the link between mastery
orientation, content choice and declarative knowledge. Mastery orientation was related
to making more of the important learning choices. Making those important learning
choices was related to performing well on the posttest. Although these relationships did
not meet the criteria for testing mediation, this series of relationships highlighted the
importance of considering learning in a process model.

Post-hoe Models

The model hypothesized in this study did not perform well. However, several
significant relationships were found indicating that choices made during training can
affect outcomes. There were enough significant relationships and significant zero-order
correlations to suggest that something measurable was occurring, even if it was not the
hypothesized model. Following this assumption, several post hoc analyses (Tables 15-
24) were conducted to try to determine what story the data could tell.

Some of the analyses expanded on the original model and entered additional

130

variables into some of the regressions. For example, Table 15 showed that content choice
was related to satisfaction even after accounting for the hypothesized relationships
(attentional focus and emotional engagement). This indicated that learners who chose the
targeted learning items were more satisfied with the course. Perhaps participants who
made more of the good content choices were satisfied, because they used their additional
knowledge to perform well on the posttest.

Except for identifying unhypothesized relationships, it was not productive to be
constrained by the original model. Consequently, effort was focused on identifying an
alternative model. The first step to identify a new, motivational model was to consider a
foundation that could tie an alternative model together. In this case, satisfaction seemed
to be a likely candidate. Satisfaction had more significant relationships in the regressions
and the zero—order correlations than the other variables. One concern was that the
training literature did not seem to view satisfaction as a critical learning outcome.
However, examination of the correlation matrix (Table 3) indicated that satisfaction was
significantly and positively correlated with the other outcome variables: declarative
knowledge (r=.27), skill-based learning (r=.19) and application self-efficacy (r=.62).

This raised an interesting question about the possibility of satisfaction mediating the
relationship to the other outcome variables. A model was tested that included the

variables that had Shown significant relationships with satisfaction or the other outcome
variables. As seen in Table 20, satisfaction was related to application self-efficacy, even
after controlling for the process variables. Learners who enjoyed the training experience
had more confidence that they could apply the information. The analysis also showed the

positive relationship between emotional engagement and application self-efficacy. The

131

mediation test in Table 21 supported the idea that the relationship between emotional
engagement and application self—efficacy was fully mediated through satisfaction. This
suggested that people who felt happy and emotionally involved during the training were
more satisfied, and this satisfaction helped them feel more confident about applying the
information later. Additional analysis (Table 22) showed that satisfaction was marginally
related to skill-based learning. Satisfied learners performed better on the essays.
Satisfaction and content choice were both significantly related to declarative knowledge
(Table 23). Content choice remained significantly related to declarative knowledge, even
after accounting for satisfaction. This indicated that content choice was only partially
mediated through satisfaction, because it also had a direct relationship with declarative
knowledge. Satisfied learners and ones who made the targeted learning choices scored
better on the posttest. Model fit analyses of the post hoc model in Figure 5 indicated
good fit for the model. Satisfaction with training did appear to be an important part of the
relationship between the process variables and the final outcomes. This post hoc model
suggested that satisfaction with training could be an often overlooked, but important,
element of training.
In_dividual Differences

This study found disappointing results for the individual difference variables.
Very few of the hypothesized relationships were significant. Mastery orientation was
related to content choice in partial support of H1, and attitude toward WBT was related to
emotional engagement in partial support of H3. Additionally there was a relationship
between microcomputer playfulness and content choice, but it was Opposite the

hypothesized direction. It indicated that people who had playful. positive attitudes

132

-..-1

toward the computer actually made fewer of the important learning choices.

None of the other hypothesized relationships were significant, but some
unhypothesized relationships were. For example, attitude toward WBT had a direct
relationship with satisfaction as seen in the post hoc analyses. Mastery orientation was
directly related to application self—efficacy and to skill—based learning. Brown (1999)
found a similar relationship between mastery orientation and application self-efficacy.
However, he also found that application self-efficacy was related to performance
orientation and technology self-efficacy, and these relationships were not supported in the
current study.

There are many potential explanations for the poor performance of the individual
difference variables in this study. These possible explanations include: the possibility
that individual differences do not affect process variables, poor choice of variables
included in the study, and design constraints. The first possible explanation is that
individual differences do not affect process variables (or at least the process variables
included in this study) as much as they influence the final outcomes. Many existing
training studies considered the effect of individual differences on training outcomes
without considering process variables. For example, Warr and Bunce (1995) found that
positive attitudes to training led to better learning scores. The meta-analysis described
earlier by Colquitt, LePine, and Noe (unpublished) also found relationships between
trainee attitudes and outcomes such as self-efficacy, declarative knowledge, skill
acquisition and reactions. These studies focused on a direct relationship between
individual differences and outcomes, without considering the learning process.

However, some studies have found relationships between individual differences

133

and process variables. Brown (1999) considered the relationships between individual
differences and process variables, and found results that conflict with the current study.
Specifically, Brown (1999) found that learning goals (mastery orientation) and
performance orientation and technology self-efficacy were related to attentional focus.
None of these relationships were supported in the current study; attentional focus was not
related to any of the individual differences. The results from Brown (1999) suggested
that individual difference variables could affect process variables. This is logical since
individual differences need some mechanism for translating themselves into differences
on training outcomes, and the learning process is a likely mechanism. Although previous
research focused on relationships between individual differences and final training
outcomes, the results found by Brown (1999) highlighted the need for additional research
on the relationship between individual differences and process variables.

Th second possible explanation is that the current study chose the wrong
individual difference variables, and other ones would have had more impact. No
demographic information was collected which precluded analyses such as the correlation
between age and computer savvy or the differences in the learning process between men
and women. It might also have been useful to measure some individual differences, such
as personality characteristics, that differed more from the other variables. The attitude
toward WBT, computer playfulness and self-efficacy variables were all directly tied to
reactions toward computers. With a fairly homogenous sample (all university students),
there was not as much difference in computer experience. In fact, participation in the
study required basic Web knowledge, because participants had to sign up for the sessions

using a Web-based subject pool system. In hindsight, the study should not have focused

134

so many individual difference variables on computer issues.

Other individual differences might prove more valuable in a similar process
mode]. Some possibilities would be a personality measure such as conscientiousness and
a motivational measure such as motivation to learn. It seems likely that
conscientiousness could relate to process variables such as attentional focus and content
choice. Conscientious participants would be more likely to pay attention to the content
and choose the critical learning information. Conscientiousness could also be closely tied
to completion, if a study was conducted in non-lab setting. If participants were
responsible for their own time and completion, conscientiousness might give them the
volition they need to persevere and complete the course. Motivation to learn could also
have been a valuable individual difference measure. In this study, participants received
credit for taking part in the study, and they did not have another external reasons (such as
a grade or job performance) to attend to the material and learn it. Differences in the
learning outcomes might be a result of the differences in motivation to learn, with
participants who had an intrinsic interest in the material performing better. Additionally,
another individual difference that might have had an influence was a measure of general
ability or cognitive ability (g). General ability could be tied to the ease of learning new
concepts. It could also be linked to more non-linear learning, if higher ability helped
participants recognize Opportunities for personal control.

Design constraints could have been a third reason that the individual difference
variables did not perform well. As mentioned earlier, participants may or may not have
had interest in the content, which might have affected their motivation to learn. The

content in the course was about Change Management, which was not directly related to

135

class work or job needs for many of the participants. The material was intended to be
understandable but novel for participants. However, a few of the participants mentioned
that they had studied the same concepts in their business and management courses.
Research conducted in the field, such as Brown (1999), should have had more relevance
for the learners and therefore engaged them better in the learning. Since it was not
measured, it was unknown whether relevance or familiarity influenced these results. The
context of the instruction could have been another design issue. The course was
primarily focused on teaching declarative knowledge such as vocabulary and concepts.
There were some case studies and exercises that attempted to translate the knowledge
into practical application, but they were not the focus of the course. It was possible that
this method of teaching was less stimulating for the participants and did not catch their
intrinsic interest. A course designed with more practical application (and relevant
content for the participant) might have improved interest in the material and affected
results. Another design issue might have been related to the basic nature of WBT.
Specifically, the technology of WBT might have distracted learners from the content.
Learners might have been clicking on screens to see what would happen, instead of
clicking to learn the content. The process variables in the study were measuring the
actions and reactions of the participants during the training. If the actions and reactions
were dictated by the training design, it was possible that individual differences did not
have much impact on the process.

There are many possible reasons that the individual difference variables did not
have many significant relationships in this study. Due to some of the study constraints, it

would be premature to conclude that individual differences do not affect the learning

136

process. It would be more responsible for future research to include alternative
individual difference variables, chose participants with varied computer experience and
measure the motivation to learn the content.

Limitations

Some of the limitations of this study were discussed in the previous section, but
some additional concerns include methodological issues and the manipulation of learner
control.

Methodological issues. There were several methodological issues with this study.

 

One issue was that the study was conducted with a student sample in a laboratory setting.
This decision was made in order to allow the use of two experimental conditions (high
and low learner control) and to ensure larger sample sizes. However, the trade-off was
that the participants were not in a work setting with actual work-related performance
outcomes related to the training. As mentioned earlier, the laboratory setting made the
learning itself less relevant to the participants and removed any consequences of poor
performance. They were not required to know the content for a course or to perform
better on their job. This dynamic might have changed their interest in the content and
consequently their motivation to learn. To determine the true impact of learner control
and engaement and WBT in a work setting, field data should be collected. Participants
who see the training as relevant and important are likely to proceed and perform
differently.

Along the same lines, the laboratory setting, allowed control and consistency in
the learning environment, but that did not necessarily reflect the reality of WBT in
organizations. WBT courses are often taken by learners on their own, in a non-protected

environment such as at their desk in an open cubicle area. Non-laboratory settings open

137

the opportunity for many distractions such as ringing phones, co-worker interruptions and
other noise. Furthermore, the learner would probably be trying to focus on training,
while other peOple in the area are doing other work tasks, adding another distraction. It
could be important to study the learning process for WBT in the workplace with these
distractions as a measured variable. A first step might be to collect some qualitative and
quantitative data concerning the effect of environmental distractions. This could provide
a catalog of common distractions and eventually lead to possible solutions for them.
Another step would be to consider how distractions and individual differences such as
conscientiousness and perceived utility of content could influence persistence and
completion of the course.

Manipulation of learner control. Another limitation was the manipulation of

 

learner control. It is possible that learner control has no significant affect on the learning
process as indicated by this study. However, it is equally likely that the manipulation of
learner control used in this study was ineffective and therefore did not stimulate any
differences. The manipulation varied the learner’s control over taking required exercises
and jumping to different sections of the course. These are some of the classic variations
of control as described in the literature (see Merrill, 1984, Chung & Reigluth, 1992).
However, perhaps these variations are not the key ones.

In the search for clarification of the learner control results, the manipulation used
in this study was more closely examined. Specifically, it needed to be determined
whether or not the learner control manipulation in this study was successful. Did
participants in the high control condition use their control and choose paths that were

more non-linear? This could help determine whether the manipulation itself failed or

138

whether learner control truly did not have an effect on training behaviors and outcomes.
In order to explore this question, the data were examined to determine the timing and
order that pages were hit in the course. Due to the possibility of limitless variation in
page order, a specific combination was investigated based on the participant’s page
choice after the jump menu page. The jump menu page was the first and primary page
that allowed non—linear access, because the content list was linked for the high control
condition and not linked for the low control condition. Of the 236 participants, only 11
people (4.6%) varied from the linear path at that choice point. Of the 11 non-linear paths,
three of the participants did not make content choices, instead, they chose to explore the
resources sections (such as vocabulary, etc.). Of the three vocabulary paths, two of those
were in the low control condition. The final breakdown showed 10 participants in the
high control condition (8.4% of condition) who chose to take a content-related, non-linear
path through the course. This did indicate that some participants did take advantage of
the non-linear path, so learner control research may have some hope in the future.
However, the numbers were not strong enough to assert that this specific manipulation
was successful. It has been a repeated conclusion from previous research that learner
control is difficult to measure and study. Either the manipulation of it needs to be re-
thought, or the construct itself should be revised.

Additional thought should go to identifying other elements of training that might
effect the learning process if controlled or not controlled. Since earlier research found
mixed results with learner control manipulations, this study attempted to strengthen the
effect by telling the participants which level of control they had. This was a very

artificial manipulation, and did not improve the study. The only noticeable effect was a

139

negative one, in that it seemed to influence scores on the pretest. The participants took
the pretest immediately following the announcement of their control condition.
Participants with less control seemed to lose some interest and performed more poorly on
the pretest. This effect seemed to disappear after that point.

The mixed results surrounding learner control research and the difficulties of
manipulating it could be a Si gn that researchers are approaching the issue incorrectly.
The questions up to this point have been about the value of learner control and how to
strengthen the manipulation of learner control. Perhaps these are the wrong questions. It
may not be useful to study learner control in its current form, instead it could be studied
as part of a larger construct. Cordova and Lepper (1996) found compelling evidence that
engagement can be enhanced through the types of features available in WBT. They
varied their programs on three elements: contextualization, such as fantasy themes (no
fantasy versus space ships and treasure hunts), personalization (references to child’s
name, friends and favorite toys) and provision of choices (allowed to make small, non-
instructional choices). They found that all three elements increased the specified
outcomes. In the Cordova and Lepper (1996) conceptualization, learner control was only
one element of the bigger construct of engagement. Engagement was a key concept in
the current study, but learner control was treated as a separate concept. The results for
learner control might have been stronger if it had been conceptualized as part of the
engagement concept. Previous null and conflicting results for studying learner control
indicated that a new tactic might be needed, and considering learner control as part of a
larger concept might be the solution. Instead of searching for ways to strengthen the

manipulation of learner control, it might be more fruitful to re-conceptualize it as part of

140

the engagement concept.

Summary. Based on the results of this study and the limitations, one conclusion
was that the accepted methods for studying learner control should be re-evaluated. It is
possible that learner control should not be studied as a primary focus. Instead the focus
should be on larger concepts such as engagement, which might include the concept of
learner control. Another conclusion could be that learner control should be studied, but
the control given to learners should be over non-instructional elements. That would
guarantee a minimal level of exposure to important material for each learner. The learner
would have a feeling of control, while still having a robust learning experience. One
overall weakness of the current study was that the initial focus was on the Web medium,
and the research design followed from that focus. The proper focus should have been
self-paced learning, with the Web medium being just a research design choice. The key
to this study was the learning of the participants, but the focus was Often on the medium.

Directions for Future Research

Satisfaction. One direction for future research would be to test the importance of
satisfaction in other learning environments, both self-paced and instructor-led.
Satisfaction is often added to research as an after-thought outcome, but this study
indicated that it might deserve more primary attention. Satisfaction could be treated as a
mediator, as in the post hoc model. It could also be looked at longitudinally to see if
changes in satisfaction affected motivational direction, intensity or persistence.

Another direction would be to consider how course design could influence
satisfaction. Satisfaction is one element of the ARCS model developed by Keller (1988).
The ARCS model was intended to explain individual motivation in learning and help

designers integrate motivational strategies into instruction. The model contains four

141

elements related to the motivation to learn: attention, relevance, confidence, and
satisfaction. Attention involves arousing and sustaining curiosity and attention, similar to
the idea of volition. Relevance involves connecting the instruction to important needs
and motives. Confidence involves developing confidence in success and generating
positive expectancies, similar to the idea of task self-efficacy. Satisfaction involves
motivation based on the learner’s reaction to the course and outcomes.

According to Keller and Suzuki (1988) satisfaction affects continuing motivation
as long as effort is consistent with expectations and outcomes, and the learner feels good
about the outcomes. Keller and Suzuki’s model (1988) approached satisfaction from a
motivational framework, which is consistent with the theoretical basis of this study.
Using the Keller and Suzuki (1988) model as a guide, future research could investigate
the instructional design strategies that could influence and enhance training satisfaction.

Volition. As discussed earlier, motivation includes three elements: direction,
intensity and persistence (Kanfer, 1990). Direction and intensity were measured in the
current study through content choice, attentional focus and activity level. However,
persistence was one element that was not strongly represented. One direction of future
research could be to include a persistence element as operationalized through volition.

In self-paced learning, one of the greatest challenges is keeping the learner
focused and on—task long enough to complete the course and adequately learn the
material. One source indicated that as many as 30-50% of students who start a distance
education course drop out before finishing (Moore & Kearsley, 1996 cited in Cornell &
Martin, 1997). Many recent learning theories have studied the effect of pre-training

motivation and learning strategies on learning goal attainment (such as completing the

142

course and learning the material). One element that is sometimes missing from this
research is a representation of the motivational process during the learning event.

Some researchers have added this representation by indicating that, . .volitional
processes mediate the road from intention to attainment (Garcia, McCann, Turner, &
Roska, 1998, p. 392)”. Volitional processes are thoughts and behaviors that maintain the
learner’s intention to attain a goal (such as complete the course) despite internal and
external distractions (Garcia, et. al, 1998; Como & Kanfer, 1993). Persistence and effort
are dimensions of motivation, but distractions can occur to divert the effort and
persistence required for a task (Garcia, et. al, 1998; Kanfer & Ackerrnan, 1989). Many
school tasks such as term papers and preparing for finals require sustained effort over
days or weeks. In self-paced WBT, students usually take the course in short segments
over time at their own discretion. At any point during this time, learners could get
distracted and never complete the task. Volition is a measure of the motivation for
maintaining persistence and effort to the goal.

In the Action-control theory, Kuhl (1984) divided goal-directed activity into a
pre-decisional phase and a post-decisional phase. In the pre-decisional phase, the
motivation to learn is established and goal setting and efficacy analyses are initially done.
This provides the energy and direction for the action. The post-decisional phase involves
implementation of goals by using learning and volitional strategies to accomplish the
academic task (Garcia, et. al, 1998). Volition could actually play a mediating role
between pre-training motivation and learning strategies. When implementation of an
intention is threatened by internal or external distraction, volitional control would activate

in order to counter the competing distraction.

143

Garcia, et. al (1998) studied the effect of volition on learning strategies and the
motivational antecedents of volition. They surveyed 487 university students about goal
orientation, task value, self-efficacy, test anxiety, volition and learning strategies related
to their course. Their results indicated that volition was positively and significantly
related to the learning strategy measures (time management, concentration, information
processing, use of study aids, self-testing, and test-takin g strategies). Furthermore, 21 %
of the variance in volition was accounted for by four variables (intrinsic goal orientation,
task value, test anxiety, self-efficacy). They supported the Action-control theory (Kuhl,
1984) by finding that volition did mediate between intention to learn and goal striving.

Volition could be an important factor in the motivational process of learning.
Trainees with high measures of volition would be more likely to stay focused and on-
task. This feature could be especially relevant in self-pace WBT in which the learner
must maintain his or her own motivation and contend with many environmental
distractions. Future motivational models of the learning process in self-paced
environments such as WBT should include the concept of volition. Volition could be
especially important for field research in the workplace, because of the high potential for
environmental distractions.

Encouraging engagement. This study provided some evidence of the importance
of engagement, feelings and satisfaction during training. Another direction for future
research would be to test different ways to encourage and promote engagement. This
research could help determine the strategies that keep the interest of learners and
encourage attention and completion. One possibility for encouraging engagement would

be to simulate social interaction in WBT. This could occur through personalized

144

messages including use of the learner’s name and feedback tailored for the learner.
Another possibility would be to have a guide character that could appear throughout the
course and provide feedback and encouragement to the learner. One facet of WBT that is
growing fast is to use a hybrid approach to training, which also increases social
interaction. This combines WBT with an instructor-led portion. Often, the learners
complete pre-work and assessments online and learn basic knowledge through a WBT
course. After completion of the Web portion, learners are brought together for an
instructor-led portion. After the course is over, support information can also be provided
on the Web. This approach might encourage engagement with the Web sections of the
course. Learners might expend more effort and focus if they know they will be required
to know the information in a public, social setting.

Another method for encouraging engagement could be to enrich the learning
environment — especially in self-paced environments. In WBT this could be done
through rich graphics, increased interactivity, and more use of simulations and case
studies. As always, it is important to make sure that the instructional design takes
precedence over support materials such as graphics. Harp and Mayer (1997) tried to
encourage emotional involvement by adding interesting, but irrelevant material (such as
cartoons or illustrations), to a lesson. The interesting material was hypothesized to
energize the learner and therefore increase attention, encoding and leamin g. As
described earlier, Cordova and Lepper (1996) found that contextualization,
personalization and provision of choices could affect outcomes and intrinsic motivation.
Most of the research done on encouraging engagement in self-paced learning has been

done in schools with children, and it would be interesting to see a study done with adults.

145

Adults might react differently to engagement tactics, such as having a negative reaction
to a theme that was juvenile or condescending. Engagement promises to be an important
element in WBT, but it still needs to be researched and investigated for further

understanding.

146

APPENDICES

147

APPENDIX A: SURVEY AND TEST ITEMS

148

* = removed items
Manipulation Check — PILOT test only (completed after all other measures)

MPPN“

I felt like I had a lot of control over my learning in this Web-based training course.
The computer allowed me to make my own learning choices.
The computer controlled me too much during the training (R).

I was able to decide how to proceed through the course.
I had the option to change the order in which I learned the material in the course.

Mastery Orientation

l.

l"

wagwew

I do my best when I’m working on a fairly difficult task.
When I have difficulty solving a problem, I enjoy trying different approaches to see

which one will work.

I try hard to improve on my past performance.

The opportunity to do challenging work is important to me.

The opportunity to extend the range of my abilities is important to me.
The opportunity to learn new things is important to me.

I prefer to work on tasks that force me to learn new things.

When I fail to complete a difficult task, I plan to try harder the next time I work on it.

Web Self-efficacy

MPPNH

I am confident I can have a basic understanding of the World Wide Web (WW).
I am confident that I can do an excellent job of using the WW.

I am confident that I will be able to learn and gather information using the WW.
I am certain I can master the skills needed to effectively navigate the WWW.
Compared with other students, I think I know a great deal about the WWW.

Attitude toward Web-based Training

MPWN~

N?"

I enjoy taking Web-based training courses.

I would rather take a course on the Web than in a classroom.

I do not have any interest in taking a Web-based training course. (R)

I think Web-based training has great potential for expanding education.

I believe there is a lot of course material that could be taught through a Web-based

course.
I think I can learn a lot through a Web-based course.

Ithink the Web is only good for games and is not good for learning. (R). *

I like that I will be able to guide my own learning through a Web-based training

course.

149

Microcomputer Playfulness
The following questions ask you how you would characterize yourself when you use

computers. For each adjective listed below, please select the number that best matches a
description of yourself when you interact with computers. (seven point scale - strongly

disagree to strongly agree)

sewew-

Spontaneous
Unimaginative (R)
Flexible

Creative

Playful

Unoriginal (R)
Uninventive (R)

Attentional Focus

omeww—

I am thinking about how well or how poorly I am doing. *

I am daydreaming while I am learning.

I am losing interest in learning the material for short periods of time.
I am thinking about other things I have to do today.

I have let my mind wander while I am learning the materials.

I am concentrating on the training materials (R) *

Emotional Engagement

owflgwew-

The graphics and interactivity of this course are keeping me interested.

As I am taking the course, I am bored. (R)

I am actively participating in learning the content during this course. *

I am interested in the course.

I am curious about the functionality and design of the course.

I am enjoying taking this course.

I feel anxious while taking this course. (R) *

I am just clicking through this course without focusing too much on it (R). *

As I am taking this course, I am happy.

10.1 feel personally involved with the course.
11. This type of training arouses my imagination.

150

Declarative knowledge test (Pretest and Posttest)

Module 1: Concept of Delta
1. Transitioning from the present state to the future state is a concept called:

0.)

A

111

\1

a) Transition
b) Delta

c) Synergy
d) Momentum

Which of the following concepts refers to the size of the change between what is and
what is desired to be?

a) What Is/ What Not Change

b) Delta deviance

c) Change differential

d) Magnitude of the gap

Which of the following is an element of change?
a) Quality

b) Resources

c) Consequences

d) All of the above

Which of the following is NOT a helpful hint for presenting a change to employees?
a) Focus on feelings first and information later

b) Focus on including employees for a win-win situation

c) Discuss the importance of the change for national economies

d) Acknowledge the need for help from internal resources

The role of a sponsor is to:

a) Be accountable for the outcomes related to change
b) Carry out the daily change plan

c) Redesign all internal processes and procedures

(1) All of the above

. Transitioning from the present state to the future state requires:

a) A formal change plan
b) A vision statement describing the future state
c) Leadership commitment and support throughout the process

d) All of the above

. In defining change, which of these factors does NOT belong?

a) A majority of the employees must agree with the change

b) The quality of the change
c) The consequences of not achieving the future state

d) When the change should take place

151

8. A change agent is:

a) A member of management who helps identify the target population
b) A person accountable for putting the change in place

c) A person who self-selects to lead change

d) A team of employees selected to accelerate change

9. Momentum is defined as:

10.

11.

12.

l3.

14.

a) Process of creating organizational support for a change
b) The high-speed journey to change

c) The pace of communication needed in change efforts
d) Environmental pressures that force a change

Change has occurred when:

a) The change agent says it is complete

b) The large majority of those affected accept it

c) Other companies in the industry recognize the change
d) Performance by individual employees improves

A stakeholder matrix is an analytical tool that can be used to enroll the target
population. What are the two axes of the matrix?

a) Cost of the change & Impact on the change

b) Job title & Level of support for the change

c) Impact on the change & Level of support for the change

(:1) Tenure with the company & Union membership

"Unfreezing" an organization refers to: . .
a) Thawing each employee’s emotions through individual communication
b) Breaking up natural day-to-day inertia to get change started

C) Warming up employees to the nature of change

(1) The process of understanding obstacles to making the change

Critical mass is created when:

a) External problems make change necessary

b) All communication plans have been activated .

C) The most critical problems are given the most attention

d) A majority of employees are committed to working on the change

Which of the following is NOT one of the four actions of momentum to move from a

present to future state?
a) Create

b) Prevent

c) Stop

(1) Reengineer

152

Satisfaction with Training

WNQMPPN?‘

I enjoyed this Web-based course.

I feel fairly satisfied with how much I learned in the course.

Most people will be fairly pleased with this course.

I am satisfied with the chance I had to learn new things in this course.
I worked well by myself in this course.

I am satisfied with the amount of control I had during the course.

I am glad I had a chance to take this course.

Overall, I am satisfied with my experience with this course.

Application self-efficacy

1.

2.

3.

Even though I may have some difficulty, I know that I will be able to develop a
change plan.

I am concerned that I do not know enough to effectively implement a change plan.
(R)

I am confident that I can use the skills from the change management course.

I am comfortable applying the change management process to problems in my own
life.

I am confident that I learned enough to improve the way I handle change events in my
life.

153

APPENDIX B: ACTIVITIES AND SCORING SHEETS

154

ID#

 

Final Activities

Read the following scenario and consider what you would do in that situation. Write
your answers in the empty spaces. Both activities refer to the same scenario and require
that you put some thought and effort into your answers.

Scenario

You are the manager of a music store in a college town. The owner of the store has
decided to start selling CDs over the Internet and has put you in charge of managing the
change effort. You will still need most of your sales staff to work in the store, but 25%
of the employees will have to be re-trained to work in customer service for the Internet
site. There is some grumbling in the company, because the employees think you are
trying to "do away" with salespeople. You know that you will need salespeople for the
store and customer service people for the website. Until now the music store has not
changed much for the thirty years that it has been in business. You will have to carefully
consider how to manage this change for the store.

You now have two activities to complete. In the first activity, you will make a list of
tasks that need to be done to make the change successful. In the second activity, you will
write a speech to explain the change to the other employees.

Activity 1

Develop a change plan for this scenario. Make a list of tasks that you would do or
organize from start to finish to make this change successful. Explain each step. You
should focus on the change issues and do NOT need to focus on technical issues such as
setting up the website.

Activity 2

Your boss wants you to make a speech about the change to the employees. Write the
speech you would use to describe the change to them. You can make up facts and
statistics to support your presentation.

155

Application Exercise
Activity Test Key

Based on the content presented in the course, acceptable answers for each activity have
been identified. There are two ways to earn points - by including target items and by
mentioning specific vocabulary words. All of the rating points will be added up (for both

ways to earn points in both activities) to get a total score for each participants.

The first way of earning points involves presenting a list of the target items for
each activity. Some items have italicized examples of acceptable answers. This list is
the full set of possible answers. The essay answers for each participant will be analyzed
to determine if the answer has the items in the target list. Each participant will have a

rating for each item on the list. The rating scale for each item will be as follows:

0 = Participant does not mention the target item in their essay.

1 = Participant mentions the item in the essay. Does not completely explain it or
demonstrate knowledge or only mentions part of target item.

2 = Participant completely discusses the target item. Correctly answers and

applies knowledge of the item.

Participants are not expected to discuss all of these items. Any items not covered

by this list will not be counted at all.

The second way to earn points will be to include specific target vocabulary words.
Participants will receive one point for each vocabulary word they include from the target
list. Each vocabulary word can only earn one point within an activity even if it is used

more than once. A word can earn two pints total if it is used in both of the activities.

156

Activity 1: Scoring

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Item
Develop a change plan. Organize the effort; Make a strategy.

2 Set up a timetable for the change steps. Decide schedule or deadlines —-
when things will happen.

3 Describe the business case for change. Determine reason the change is
important — competition; new Internet opportunities; egpand business.
Hold an employee meeting to announce the change.

5 Invite questions and dialogue. Questions are encouraged and welcome.
Ask emploKes to give input and feedback.

6 Have ongoing meetings or informal discussion with employees to keep
track of their feelings toward the change effort.

7 Send the message through different organizational levels. Have
sppervisors talk to their direct reports.

8 Review progress against goals. Track performance. Keep records and
communicate how the Chang is progressing.

9 Share information about competitive or benchmark companies that have
implemented similar changes. It has worked for other stores.

10 Find a sponsor for the change. Choose someone to be in charge.

1] Present a vision of the future. Plan to be top Internet retailer in 5 years
or increase revenues by 25 %.

12 Recognize personal achievements and team efforts. Provide monetary
rewards. People who do well will get raises.

Vocabulary Words

Delta

Transition

 

Present state (present)

 

Future state (future)

 

S onsor (s)

 

Magnitude of Gap (Gap in Magritude)

 

Milestone chart (milestones)

 

Change Plan (Plan or Plan for Change)

 

Timetable/ timeline

 

Business Case

 

Unfreeze or freeze or refreeze

 

Stakeholders

 

 

Crisis situation

157

 

 

 

Activity 2: Scoring

 

 

 

 

 

 

will go up. Costs will go down.

r Item
1 States that there will be a change to start selling CDS over the Internet.
2 States that some of the salespeople will have to be re-trained to work as
customer service representatives.
3 Some sales people will stay in their current jobs. The store will not be
“doing away” with salespeople. No threat to yobs; Lob security.
4 Describe the business case or specific benefits for change. Determine

reason the change is important - competition; new Internet
opportunities; expand business. You will earn more money. Revenues

 

now. Move tgfuture.

5 States that the business has not changed much, but it is time to change

 

6 Share information about competitive or benchmark companies that have
implemented similar charges. It has workedjor other stores.

 

Asks if there are any questions.

 

Discusses the timing of the change (how soon or when the change is to
take place). When training or site design will start.

 

9 Discusses the consequences of the change. What will happen if change
does not occur --go out of business; lose customers.

 

the word “you ”.

10 Discusses the change from the employee’s point of view. Change may
be scary. You may be worried. You will have great opportunities. Uses

 

11 Includes new statistics to support the speech.

 

 

 

12 Uses motivational language to get people excited. This will be great for
us! We can all pull together and make this great!

 

 

Vocabulary Words

 

‘ Delta

 

 

Transition

 

Present state (present)

 

Future state (future)

 

Sponsor (5)

 

Magnitude of Gap (Gap in Magnitude)

 

Milestone chart (milestones)

 

Change Plan (Plan or Plan for Change)

 

jmetable/ timeline

 

Business Case

 

 

Unfreeze or freeze or refreeze

 

Stakeholders

 

 

 

 

 

1 Crisis situation

158

 

 

BIBLIOGRAPHY

159

BIBLIOGRAPHY

Alliger, G. M. & J anak, E. A. (1989). Kirkpatrick’s levels of training criteria:
Thirty years later. Personnel Psychology, 42_, 331-342,

Alliger, G. M., Tannenbaum, S. 1., Bennett, W., Traver, H., Shotland, A. (1997).
A meta-analysis of the relations between training criteria. Personnel Psychology, 50,
341-358.

 

Ames, C. (1992). Classrooms: Goals, structures, and student motivation. Journal
of Educational Psychology, _8_4, 261—271.

 

Ames, C., & Archer, J. (1988). Achievement goals in the classroom: Student

learning strategies and motivation processes. Journal of Educational ngchologyLSO,
260-267.

 

Baldwin, T. T., & Ford, J. K. (1988). Transfer of training: A review and
directions for future research. Personnel ngholigyg 41, 63-103.

Bandura, A. (1986). Social Foundations of Thought and Action. Englewood
Cliffs: Prentice-Hall.

Bandura, A. (1991). Social cognitive theory of self-regulation. gganizational
Behavior and Human Decision Process, fl, 92-113.

 

Bandura, A. (1997). Self-efficacy: The Exercise of Control. New York: W. H.
Freeman.

 

Beach, L. R., & Mitchell, T. R. (1978). A contingency model for the selection of
decision strategies. Academy of manzgment Review, 3, 439-449.

Brown, K. G. (1996). Motivational and informational consequences of errors in
earlv skill acquisition: The effects of individual differences and training strategy on
pgceptions of nega_tive feedback. Unpublished Masters Thesis. Michigan State
University, East Lansing, MI.

Brown, K. G. (1999). Individual differences in choice duringgleaming: The

influence of learner goals and attitudes in Web-based training. Unpublished dissertation.
Michigan State University, East Lansing, MI.

Button, S. B., Mathieu, J. E., & Zajac, D. M. (1996). The development and

psychometric evaluation of measures of learning goal and performance goal orientation.
Organizational Behavior and Human Decision Processesy61, 26-48.

160

Campbell, J. P., McCloy, R. A., Oppler, S. H., & Sager, C. E. (1993). A theory of
performance. In N. Schmitt & W. C. Borman (Eds), Personnel selection in organizations
(pp. 71-98). San Francisco, CA: Jossey-Bass.

Campbell, J .P. & Pritchard, R. D. (1976). Motivation theory in industrial and
organizational psychology. In M.D. Dunnette (ed.), Handbook of Industrial
Organizational Psycholcgy(pp.63-l30). Chicago: Rand McNally.

Carver, C. S. & Scheier, M. F. (1990). Origins and functions of positive and
negative affect: A control-process view. Psychological ReviewL97, 19-35.

Chung, J. & Reiguluth, C. M. (1992). Instructional prescriptions for learner
control. Educational Technology, 32, 14-20.

Cohen, J ., & Cohen, P. (l983LApplied Multiple Regression/ Correlation
Analysis for the Behavioral Sciences. Hillsdale: Lawrence Erlbaum Associates.

 

Colquitt, J. A., LePine, J. A., & Noe, R. A. (1998). Trainee attributes and
attitudes revisted: A meta-analytic structural equation modeling analysis of research on
training motivation, Unpublished Manuscript.

Colquitt, J. A., & Simmering, MS. (1998). Conscientiousness, goal orientation,
and motivation to learn during the learning process: A longitudinal study, Journal of
Applied Psychology 8} (4), 101-120.

Connell, J. P., & Wellbom, J. G. (1991). Competence, autonomy and relatedness:
A motivational analysis of self-esteem processes. In M. R. Gunnar & L.A. Sroufe (Eds),
Self processes in development: Minnesota Symposium on Child Psychology (Vol. 23, pp.
43-77). Hillsdale, NJ: Erlbaum.

Cordova, D. I., & Lepper, M. R. (1996). Intrinsic motivation and the process of
learning: Beneficial effects of contextualization, personalization, and choice. Journal of
Educational Psychology, 88 (4), 715-730.

Cornell, R. & Martin, B. L. (1997). The role of motivation in Web-based
Instruction. In B.H. Khan (Ed), Web-based Instruction (pp. 5-18). Englewood Cliffs,
NJ: Educational Technology Publications.

Corno, L., & Kanfer, R. (1993). The role of volition in learning and performance.
Review of Research in EducationLl9, 301-341.

Craiger, J. P. & Weiss, R. J. (1997). Traveling in cyberspace: Web-based
instruction. The Industrial-Organizational ngchologiyt, 3;, 11-18.

16]

Csikzentmihalyi, M. (1975). Beyond Boredom and Anxiety. San Francisco:
J ossey —Bass.

 

Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of
computer technology: A comparison of two theoretical models. Manggement Science,
35, 982-1003.

 

DeCharms, R. (1968). Personal Causation. New York: Academic Press.

 

Dweck, C. S. (1986). Motivational processes affecting learning. American
Psychologyst, 41, 1040-1048.

Dweck, C. S., & Leggett, E. L. (1988). A social-cognitive approach to motivation
and personality. Psycholpgigcal Review, 95, 256-273.

Elliott, E. S., & Dweck, C. S. (1988). Goals: An approach to motivation and
achievement. Journal of Personaliyy and Social Psychology, 54, 5-12.

Farr, J. L., Hofmann, D. A., & Ringenbach, K. L. (1993). Goal orientation and
action control theory: Implications for industrial and organizational psychology. In
C.L.Cooper & I.T.Robertson (Eds), International Review of Industrial and
Organizational Psychology (pp. 193-232). New York: Wiley.

 

Feldman, D. C. (1984). The development and enforcement of group norms.
Academy of Management Review, 9, 47-53.

Filipczak, B. (1996, Feb). Training on the intranets: The hope and the hype.
Training, 24-32.

Fisher, S. L. (1995). The role of cognitive effort and learning outcome demands
in skill acquisition and learning. Unpublished Master’s Thesis, Michigan State
University.

Fisher, S. L. (1998) The role of goal orientation in a self-regulation framework. .
Unpublished Doctoral Dissertation, Michigan State University.

Fisher, M. D., Blackwell, L. R., Garcia, A. B., & Greene, J. C. (1975). Effects of
student control and choice on engagement in a CAI arithmetic task in a low income

school. Journal of Educational Psychology, _6_7_, 776-783.

Fisher, S. L. & Ford, J. K. (1998). Differential effects of learner effort and goal
orientation on two learning outcomes. Personnel Psycholggy, 5 L 397-420.

Ford, J. K., Schmitt, N., Schechtman, S. L., Hults, B. M., & Doherty, M. L.
(1989). Process tracing methods: Contributions, Problems, and neglected research
questions. Organizational Behavior and Human Decision Processes, 43, 75-117.

 

162

Ford, J. K. Smith, E. M., Weissbein, D. A., Gully, S. M., & Salas, E. (1998).
Relationships of goal orientation, metacognitive activity, and practice strategies with
learning outcomes and transfer. Journal of Apglied ngchologyjl 218-233,

Gagne, R. M. (1984). Learning outcomes and their effects: Useful categories of
human performance. American Pstholoygist, 39, 377-385,

Gagne, R.M., Briggs, L.J., & Wager, W.W. (1992). Principles of instructional
desigr_i (4‘h Ed.). New York: Harcourt Brace Jovanovich College Publishers.

Garcia, T., McCann, E. J ., Turner, J. E. & Roska, L. (1998). Modeling the
mediating role of volition in the learning process. Contempprary Educational
Psychology, 23, 392-418.

Gardner, E. P., Young, R, & Ruth, S. R. (1989). Evolution of attitudes toward
computers: A retrospective review. Behaviour and Information Technology, 8 (2), 89-98.

Gerber, B. (1990). Managing Diversity. Training, 2, 23-30.

Gist, M. E., & Mitchell, T. R. (1992). Self-efficacy: A theoretical analysis of its
determinants and malleability. Academy of Management Review, 1], 183-211.

Gist, M. E., Stevens, C. K., & Bavetta, A. G. (1991). Effects of self-efficacy and
post-training intervention on the acquisition and maintenance of complex interpersonal
skills. Personnel Psycholggy, 44, 837-861.

Gray, S. H. (1987). The effect of sequence control on computer assisted learning.
Journal of Computer-based Instruction, 1_4, 54-56.

Greene, B. A., & Miller, R. B. (1996). Influences on achievement goals,

perceived ability, and cognitive engagement. Contemgoraryjducational Pysychology, 21,
181-192.

Hancock, T. E., Thurman, R. A., & Hubbard, D. C. (1995). An expanded control
model for the use of instructional feedback. Contemrafl Educational chholpgy, 20,
410-425.

Hannafin, R. D., & Sullivan, H. J. (1996). Preferences and learner control over
amount of instruction. Journal of Educational Psychology, 88 (1), 162-173.

 

Harp, S. F. & Mayer, R. E. (1997). The role of interest in learning from scientific
text and illustrations: On distinction between emotional interest and cognitive interest.

Journal of Educational Psychology, 89 (1), 92-102.

163

Hill, T., Smith, N. D., & Mann, M. F. (1987). Role of Efficacy expectations in
predicting the decision to use advanced technologies: The case of computers. J oumal of

Applied Psychology, 12 (2), 307-313.

Howard-Rose, D. & Winne, P. H. (1993). Measuring component and sets of
cognitive processes in self-regulated learning. Journal of Educational Psychology, 85 (4),

591-604.

Howell, A. W., Mullins, M. E., Fisher, S. L., Schmitt, N., Kozlowski, S. W. J.
(1998, April). Using the Internet for Competitive Advantage. Poster presented at the
Thiteenth Annual Conference of the Society for Industrial and Organizational

Psychology, Dallas, TX.

James, L. R., & Brett, J. M. (1984). Mediators, moderators, and tests for
mediation. Journal of Applied Psychology, _6_9 (2), 307-321.

Kanfer, R. (1990). Motivation theory and industrial and organizational

psychology. In M. D. Dunnette & L. M. Hough (Eds), Handbook of Industrial and
Oganizational ngphology, 2nd edition, vol. 1. Palo Alto: Consulting Psychologists Press.

Kanfer, R. & Ackerman, P. L. (1989). Motivation and cognitive abilities: An
integrative aptitude-treatment interaction approach to skill acquisition [Monograph].

Journal of Applied Psychology, 74, 657-690.

Keller, J. M., (1983). Motivational design of instruction. In C. M. Reigeluth
(Ed), Instructional-Design Theories and Models: An Overview of their Current Status

(pp. 383-434). Hillsdale, NJ: Lawrence Erlbaum Associates.

Keller, J. M., & Suzuki, K. (1988). Using the ARCS motivation model in
courseware design. In D. H.. Jonassen (Ed), Instructional Designs for Microcomputer
Courseware (pp. 401-434). Hillsdale, NJ: Lawrence Erlbaum Associates.

 

Khan, B. H. (1997). Web-based Instruction (WBI): What is it and Why is it? In
B.H. Khan (Ed), Web-based Instruction (pp. 5-18). Englewood Cliffs, NJ: Educational

Technology Publications.

Kintsch, W. (1980). Learning from text, levels of comprehension, or: Why
anyone would read a story anyway. Poetics, 9, 87-98.

Kinzie, M. B. (1990). Requirements and benefits of effective interactive
instruction: Learner control, self-regulation, and continuing motivation. Educational

Technology Research and Develggment, 38, 1-21.

 

Kinzie, M. B., Sullivan, H. J., & Berdel, R. L. (1988). Learner control and
achievement in science computer-assisted instruction. Journal of Educational

Psychology, 80, 299-303.

 

164

Kirkpatrick, D. L. (1974). Evaluation of training. In R. L. Craig (Ed), Training
and Develoypyment Handbook (2"d ed., pp. 18-1118-27). New York: McGraw Hill.

Kraiger, K., Ford, J. K., & Salas, E. (1993). Application of cognitive, skill-based,
and affective theories of learning outcomes to new methods of training evaluation.
Journal of Applied Psychologyfl8, 311-328.

Kuhl, J. (1984). Motivational aspects of achievement motivation and learned
helplessness: Toward a comprehensive theory of action control. In B. A. Maher & W. B.
Maher (Eds), Prgggess in Eyperimental Personality Research (Vol. 13, pp. 99-171), New
York: Academic Press.

Lee, S. & Lee, Y. H. K. (1991). Effects of learner-control versus program-control
strategies on computer-aided learning of chemistry problems: For acquisition or review?
Journal of Educational Psychology. 83, 491-498.

Locke, E. A., & Latham, G. P. (1990). A theory of goal setting and task
performance. Englewood Cliffs, NJ : Prentice Hall.

Long, J. S. (1997). Regression models for categprical and limited dependent
variables. Thousand Oaks, CAzSage.

Louis, M. R., Posner, B. Z. & Powell, G. N. (1983). The availability and
helpfulness of socialization practices. Personnel Psychology, 36, 857-866.

 

Malone, T. W. & Lepper, M. R. (1987). Making learning fun: A taxonomy of
intrinsic motivations for learning. In R. E. Snow & M.J. Farr (Eds), Aptitude. Leaminyg
and Instruction: Cognitive and Affective Process Analysis (Vol. 3, pp.223-253).
Hillsdale, NJ: Erlbaum.

Martocchio, J. J., & Webster, J. (1992). Effects of feedback and cognitive
playfulness on performance in microcomputer software training. Personnel Psychology,
4._5, 553-578.

Mathieu, J. E., Martineau, J. W., & Tannenbaum, S. I. (1993). Individual and
situational influences on the development of self-efficacy: lrnplications for training
effectiveness. Personnel Psychology,46, 125-147.

McAllister, D. W., Mitchell, T. R., & Beach, L. R. (1979). The contingency
model for the selection of decision strategies: An empirical test of the effects of
significance, accountability and reversibility. Organizational Behavior and Human
Performance, 24, 228-244.

165

 

McCormick, E. J. & Ilgen, D. R. (1985). Job satisfaction, attitudes and opinions.
In E. J. McCormick & D. R. 11 gen (Eds), Industrial Organizational Pygnonygyw‘“
edition, pp 307-324). Englewood Cliffs, NJ: Prentice Hall.

 

Merrill, M. D. (1984). What is learner control. In R. K. Bass & C. D. Dills
(Eds), Instructional Develogment: The State of the Art 11. Dubuque: Kendall! Hunt.

Milheim, W. D. & Martin, B. L. (1991). Theoretical bases for the use of learner
control: Three different perspectives. Journal of Computer-Based Instructionyl8, 99-
105.

Mischel, W. (1977). The interaction of person and situation. In D. Mgnusson &
N. S. Endler (Eds), Personality at the crossroads: Current Issues in Interactional
thology, (pp. 333-352). Hillsdale, NJ: Erlbaum.

Moore, M. G., & Kearsley, G. (1996). Distance education: A systems view.
Belmont, Ca: Wadsworth.

 

Morrison, G. R., Ross, 8. M. & Baldwin, W. (1992). Learner control of context
and instructional support in learning elementary school mathematics. Educational
Technolgy Research and Development, 40, 5-13.

Murphy, M. A. & Davidson, G. V. (1991). Computer-based adaptive instruction:
Effects of learner control on concept learning. Journal of Computer-Based Instruction,
18, 51-56.

Nix, G. A., Ryan, R. M., Manly, J. B., & Deci, E. L. (1999). Revitalization
through self-regulation: The effects of autonomous and controlled motivation on
happiness and vitality. Journal of Experimental Social Psychology, 3_5_, 266-284.

Noe, R. A. (1986). Trainee’s attributes and attitudes: Neglected influences on
training effectiveness. Academy of Management Revieygl 1, 736-749.

Noe, R. A. & Schmitt, N. (1986). The influence of trainee attitudes on training
effectiveness: Test of a model. Personnel Psyghology, 3% 497-523.

Orey, M. A., Garrison, J. W., & Burton, J. K. (1989). A philosophical critique of
null hypothesis testing. Journal of Research and Devemment in Education, 2; (3), 12-
21.

Payne, J. W. (1982). Contingent decision behavior, Psychological Bulletin, 9__2_,
382-402.

166

Pintich, P. (1986). Motivation and learning strategies interactions with
achievement. Paper presented at the annual meeting of the American Educational

Research Association, San Francisco.

Pintrich, P. R. & de Groot, E. V. (1990). Motivational and self-regulated learning
components of classroom academic performance. Journal of Educational Pgmhology,

_8_2, 33-40.

Pokay, P. & Blumenfeld, P. C. (1990). Predicting achievement early and late in
the semester: The role of motivation and the use of learning strategies. Journal of
Educational Psychology, Q (1), 41-50.

Pollock, J. C. & Sullivan, H. J. (1990). Practice mode and learner control in
computer-based instruction. Contempgrary Educational Ps‘mologyyli 251-260.

Ray, N. M. & Minch, R. P. (1990). Computer anxiety and alienation: Toward a
definitive and parsimonious measure. Human Factors, 32 (4), 477-491.

Reeves, T. C. (1993). Pseudoscience in computer-based instruction: The case of
learner control research. Journal of Computer-Based Instruction,20, 39-46.

Ryan, R. M., & Connell, J. P. (1989). Perceived locus of causality and
internalization: Examining reasons for acting in two domains. Journal of Personality and

Social Psyc_hology, 5_7, 749-761.
Ryan, R. M., Connell, J. P., & Deci, E. L. (1985). A motivational analysis of self-

determination and self-regulation in education. In C. Ames & R. E. Ames (Eds),
Research on motivation in Education: Vol. 2. The Classroom Milieu (pp. 13-51). San

Diego: Academic Press.

 

Ryan, R. M. & Deci, E. L. (2000). Intrinsic and extrinsic motivations: Classic
definitions and new directions. Contemporary Educational Psycholggy, 25, 54-67.

Sadri,G., & Robertson, 1. T. (1993). Self-efficacy and work-related bhaviour: A
review and meta-analysis. Applied Psyghology: An lntemational Review, 42 (4), 139-

152.

Schein, E. H. (1968). Organizational socialization and the profession of
management. Industrial Management Review, 9, 1-16.

Skinner, E. A. & Belmont, M. J. (1993). Motivation in the classroom: Reciprocal
effects of teacher behavior and student engagement across the school year. Journal of

Educational Psychology, 85 (4), 571-581.

167

Skinner, E. A., Wellbom, J. G., & Connell, J. P. (1990). What it takes to do well
in school and whether I’ve got it: The role of perceived control in children’s engagement
and school achievement. Journal of Educational Psychology, 82, 22-32.

Snow, R. E., & Lohman, D. F. (1984). Toward a theory of cognitive aptitude for
learning from instruction. Journal of Educational ngchology, 76, 347-376.

Starbuck, W. H., & Webster, J. (1991). When is play productive? Accounting,
Managemegt, and Information Technology, 1 (1), 71-90.

Steinberg, E. R. (1989). Cognition and Learner Control: A Literature Review,
1977-1988. Journal of Compgter-Based Instmctioml6, 117-21.

Steinberg, E. R. (1977). Review of student control in computer-assisted
instruction. Journal of Computer-Based Instruction, 8, 84-90.

 

Stevens, L. (1996). The intranet: Your newest training tool? Personnel Journal,
July, 2-32.

 

Svenson, O. (1979). Process descriptions of decision making. Organizational
Behavior and Human Performance, _28, 86-112.

Warr, P. & Bunce, D. (1995). Trainee characteristics and the outcomes of Open
learning. Personnel Psychology, 48, 347-375.

Webster, J. & Hackley, P. (1997). Teaching effectiveness in technology-mediated
distance learning. Academy of Management Journal, 40 (5-6), 1282-1309.

Webster, J ., & Martocchio, J. J. (1992). Microcomputer playfulness:
Development of a measure with workplace implications. MIS Quarterly, June, 201 —

226.

 

Weiner, B. (1986). Attribution, emotion, and action. In R. M. Sorrentino & E. T.
Higgins (Eds), Handbook of motivation and cognition: Foundations of social behavior.

New York: Guilford Press.

Weiss, H. M. (1990). Learning theory and industrial and organizational
psychology. (In M. D. Dunnette & L. M. Hough (Eds). Handbook of industrial and
organizational psychology (2nd ed., Vol. 1, pp. 171—221). Palo Alto, CA: Consulting

Psychologists Press.)

 

Williams, A. E., (1997). Grassroots changg: Variables affectingrole innovation
Unpublished Masters Thesis. Michigan State University, East Lansing, MI.

 

168

Williams, M. D. (1996). Learner-control and instructional technologies. In D.
Jonassen (Ed), Handbook of education research technology (pp. 957-983). New York:
Simon & Schuster McMillan.

I69

l
N!- —‘ _tlr; ' ‘

    

GAN STATE

' Hill mum

   

31293 021774