1”...th
8"". 1.:
‘ -\.>~p»
u I - «

 

.o...’...-.

.A e
I . '.'
uu‘l'

‘ a1:

”'2‘ «{Efgﬁfgm :4.

little

532% ‘
"5‘5

5?“ and
m" 93233?”
Ma’s: L-
-§.§:;ask::r.‘

_, ‘-
Min

m“
‘&

anktgda.‘ p c s.

as» ‘ - J — ~.’

w. t a .‘ .

-& ‘.1w~,,¢r., "M". m . \ 53:223.? “i

- . _.~‘ - 1*"- “F.

,v "2-7pm: it
31%;?!” '3" '
12"” ‘J: x

4.3 ‘
d-
vzif‘
,u

3'
l

., \.

h ,gbr_
3“" - ’1‘-
a:

$351,; .. .. Q"
fw‘w‘i‘t...
“I: n

..
. v,” N,
'Iz“

'l

.r. f. h .
, .....-vr" ‘

...
v .r
,_...

..v-.~

 

llllllllllllzl;

     

 

 

WillHill!llllllllllllllllll

301055 3356

This is to certify that the
dissertation entitled
THE EFFECTS OF COMMUNICATING APPLICATION TASK
REQUIREMENTS ON STUDENTS' LEARNING PROCESSES
AND ACHIEVEMENT
presented by

Jun Young Shin

has been accepted towards fulﬁllment
of the requirements for

PAP degreein EdUCa+/.md
Ricki/off

 

V Major professor.\7%’l

Date Feé. 24,. 122,2

MSU is an Afﬁrmative Action/Equal Opportunity Institution 0-12771

 

Err; am. oh}. !
5525322“ tiff? :y. f

O
E?3 a 3
.'1""
33.'l.'--'r,p}133:1 J

 

 

 

PLACE IN RETURN BOX to remove this checkout from your record.
TO AVOID FINES return on or before duo duo.

DATE DUE DATE DUE DATE DUE

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

MSU to An Afﬂnnotlvo Action/Equal Opportunity Institution
amount

THE EFFECTS OF COMMUNICATING APPLICATION TASK
REQUIREMENTS ON STUDENTS' LEARNING PROCESSES
AND ACHIEVEMENT

By

Jun Young Shin

A DISSERTATION

Submitted to
Michigan State University
in partial fulfillment of the requirements
for the degree of

DOCTOR OF PHILOSOPHY

Department of Counseling, Educational Psychology, and
Special Education

1992

?

K
.1

(

57

{75’-

ABSTRACT

THE EFFECTS OF COMMUNICATING APPLICATION TASK
REQUIREMENTS ON STUDENTS' LEARNING PROCESSES
AND ACHIEVEMENT
by

Jun Young Shin

Most research on the instructional effects of objectives has shown
that objectives clearly communicate instructional requirements for recall
tasks more effectively than application tasks. This study asked: How can
instructors successfully communicate test requirements through objectives
for application tasks learned from text? The researcher derived two
qualities of objectives that would be likely to affect student's learning
processes and achievement: (a) the deﬁnition of the subject matter domain
in the objective, and (b) the concreteness of the example of the test task in
the objective.

A posttest only control- group experimental design was used. Fifty-
six volunteer graduate students were randomly assigned to one of four
treatments: objectives that were (a) deﬁned and concrete, (b) deﬁned and
mt concrete, (c) concrete and EM deﬁned, and (d) neithJ deﬁned M
concrete. Subjects' were asked to read text, select relevant content from
text, choose practice exercises, explain their reasons for their selections and
take an application task posttest.

Analysis of the data showed that:

1. Subjects presented with deﬁned objectives were able to select
relevant content. They used objectives as a guide to limit their choices.

2. Subjects presented with concrete objectives behaved no

differently than subjects without them in selecting the right exercise for
preparing for the posttest. They made selections based on the examples in
the text as well as the example in the objectives.

3. Students' posttest performance was positively and substantially
affected by the deﬁned Objective treatment. However, there was an
indirect link in the path from the deﬁned objective treatment to the posttest
through choosing relevant content and performing well on an exercise.

4. Subjects presented with the deﬁned objective treatment were not

different in their reading time than subjects without the treatment.

ACKNOWLEDGEMENTS

I wish to express appreciation for the contribution of the many

individuals who made the completion of this study possible.

I am especially grateful to Dr. Stephen Yelon, my advisor and
committee chairman, who gave so generously of his time and expertise to
patiently guide this research. He provided enthusiastic support which
resulted in its completion. I can't express my appreciation for his
contribution in written word. I am indebted to him for my academic life
at MSU. He supported me not only academically, but also ﬁnancially and
personally. He guided my study at MSU from the ﬁrst step to the last. I
wish there was a way to repay him for his kindness other than

remembering his generosity in the future. "Thank you, Sir."

I would also like to acknowledge my gratitude to Dr. Ralph Putnam,
Dr. William Anderson, Dr. Stephen Raudebush, and Dr. Michael Seltzer.
Dr. Ralph Putnam provided academic encouragement since his arrival to
MSU, and he supported this study with his insightful criticism and valuable
advice. Dr. William Anderson supported this study with continuing
interest and intellectual insight. Dr. Stephen Raudebush taught me
research methods at MSU and encouraged this study with his expertise.
Dr. Michael Seltzer provided opportunities to discuss my statistical data
and guided me to search beyond the Obvious. Thanks to all of you.

I could not have begun work on my doctorate without the support

and encouragement of my father and mother, whose love and support have

iv

been a constant source of inspiration and encouragement to me in pursuit

of education and life goals.

Many thanks to my wife and best friend, Kum Sook Choi, whose
love, support, patience, belief in my abilities, and dedication have provided

energy and encouragement so that I could concentrate on my studies.

A Special thanks to my son, Sang Young Shin, who has grown up
with Daddy always taking some class or working on a paper. He helped
me realize more than ever the importance of family life. He has brought
joy into my life. Now, I'll be able to spend more time with him.

TABLE OF CONTENTS

LIST OF TABLES .......................................................................
LIST OF FIGURES ......................................................................

Chapter I

INTRODUCTION ........................................................................
Problem .............................................................................
Research Questions ..............................................................
Need for the Study ..............................................................
Outline of the Study .............................................................

Chapter 11
REVIEW OF THE LITERATURE ................................................
An Overview of Theoretical Relationships Among Objectives,
Learning Activities, and Student Achievement ........................
Research on Objectives ........................................................
What is an Objective? .................................................
Effects of Objectives on Guiding Learning ....................
Effects of objectives on recall and application
tasks .................................................................
Effects of objectives on relevant and incidental
outcomes ..........................................................
Effects of general and specific Objectives on
learning ............................................................

Summary

vi

Page
xi

xiv

8
12
13
13

15

18

21
22

Characteristics of Effective Objectives .......................... 22

Deﬁned objectives ............................................. 22
Concrete objectives ............................................ 24
Summary .......................................................... 25
Adjunct Questions as a Concrete Indicator in an
Objective ................................................................... 26
Summary .......................................................... 28
Research on Students' Cognitive Processing during
Learning ............................................................................ 28
Attention ................................................................... 29
Summary .......................................................... 31
Strategies ................................................................... 31

Strategies related to choosing relevant content of

text .................................................................. 33

Strategies related to choosing exercise ................. 34

Summary .......................................................... 36

Research Questions .............................................................. 36
Chapter III

METHOD .................................................................................... 38

Research Design .................................................................. 38

Sample ............................................................................... 42

Pilot Study .......................................................................... 43

Instrumentation ................................................................... 44

Posttest ...................................................................... 44

Questions Regarding Prior Knowledge ......................... 45

Objectives .................................................................. 46

vii

Text .......................................................................... 46

Exercises ................................................................... 47
Method to Assess the Choice of Relevant Content ........... 49
Method to Assess the Choice of Exercise ....................... 49
Reliability ........................................................................... 50
Data Analysis ...................................................................... 50
Statistical Analysis ...................................................... 51
Preliminary analysis .......................................... 51
Main analysis of the research questions ................ 51
Subsidiary analysis of relationships ..................... 53
Chpater IV

FINDINGS .................................................................................. 54
Data Analysis ...................................................................... 54
Statistical Analysis ...................................................... 54
Preliminary analysis .......................................... 55
Prior knowledge of objective ..................... 55
Prior knowledge of correlation .................. 58
Descriptive information about posttest ........ 60

The relationship of the posttest to prior
knowledge of objectives and correlation ..... 62
Main analysis of the research questions ................ 63
Research question one ............................... 63
Research question two ............................... 66
Research question three ............................. 67
Research question four .............................. 74
Subsidiary analysis of relationships ..................... 76

viii

Qualitative Analysis .................................................... 78

Analysis of reasons given for selection of text 78

content .............................................................
Analysis of reasons given for selecting an 83

exercise ............................................................
Summary and Integration of the Results ........................ 87

Chapter V

DISCUSSION AND CONCLUSIONS ............................................. 91
Discussion of the Findings .................................................... 91
Prior Knowledge of Objectives .................................... 91
Prior Knowledge of Correlation .................................. 93
Selection of Relevant Content ...................................... 93
Selection of Exercise .................................................. 97
Posttest Performance .................................................. 97
Reading Time ............................................................ 100
Path Analysis ............................................................. 101
Limitations ......................................................................... 102
Conclusions ........................................................................ 104
Implications of the Study for Instructional Design .................. 106
Application Objectives ................................................ 106
Students' Use of Objectives .......................................... 107
Design Implications .................................................... 107
Recommendation for Future Study ........................................ 109
LIST OF REFERENCES .............................................................. 111

ix

APPENDIX
APPENDIX
APPENDIX
APPENDIX
APPENDIX
APPENDIX
APPENDIX
APPENDIX
APPENDIX
APPENDIX
APPENDIX
APPENDIX
APPENDIX

Table 1.

Table 2.

Table 3.

Table 4.

Table 5.

Table 6.

Table 7.

Table 8.

Table 9.

Table 10.
Table 11.
Table 12.
Table 13.
Table 14.

LIST OF TABLES

Page
Means and Standard Deviations of Prior Knowledge of
Objectives .................................................................... 55
One-Way ANOVA Results for Prior Knowledge of
Objectives ..................................................................... 56
Means and Standard Deviations of Prior Knowledge of
Objectives as a Function of Treatment Condition ............... 57
Two-Way ANOVA Results for Prior Knowledge of
objectives ...................................................................... 57
One-Way ANOVA Results for Prior Knowledge of
Objectives ..................................................................... 58
Means and Standard Deviations of Prior Knowledge of
Correlation ................................................................... 59
One-Way ANOVA Results for Prior Knowledge of
Correlation .................................................................... 59
Means and Standard Deviations of Prior Knowledge of
Correlation .................................................................... 60
Two-Way ANOVA Results for Prior Knowledge of
Correlation .................................................................... 60
Descriptive Data for the Posttest ...................................... 61
Frequency of Posttest Scores by Treatment ....................... 61
Correlational Matrix for all the Study's Variables ............. 63
Mean Scores of Relevant Content by Type of Objective ...... 64
Two-Way ANOVA Results for the Selection of Relevant
Content ......................................................................... 65

xi

Table 15.

Table 16.

Table 17.

Table 18.

Table 19.

Table 20.

Table 21.

Table 22.

Table 23.

Table 24.

Table 25.

Table 26.

Table 27.

Table 28.

Table 29.

Table 30.

Mean Scores for Selecting Irrelevant Content by Types of

Objective ....................................................................... 66
Two-Way ANOVA Results for the Selection of Irrelevant
Content ......................................................................... 66

Frequency and Chi-square Test Results of Selection of

Exercise by Type of Objective ......................................... 67
Means of Posttest Scores by Treatment with and without

Outliers ......................................................................... 68
Two-Way ANOVA Results for the Posttest Score .............. 69
Mean Ranks of Posttest Scores by Type of Objective .......... 71
Nonparametric ANOVA Results for Posttest Ranks ........... 72
95% Conﬁdence Interval of b .......................................... 74
Means of Reading Times by Type of Objective .................. 75
Two—Way ANOVA Results for Reading Time ................... 75

Interrater Agreement on Subjects' Use of Objectives in
Selecting Relevant Content by Treatment .......................... 79
Frequency of Words and Phrases Used by Subjects to

Select Content ................................................................ 80
Number of Subjects Using Objectives to Limit Content by
Treatment ..................................................................... 81
Abbreviated Reasons Given by Subjects not Using

Objectives to Select Content ............................................ 82
Interrater Agreement on Subjects' Use of Objectives in
Selecting an Exercise by Treatment .................................. 84
Frequency of Words and Phrases Used by Subjects to

Select Exercises ............................................................. 84

xii

Table 31. Abbreviated Reasons Given by Subjects not Using

Objectives to Select Exercises .......................................... 86

xiii

LIST OF FIGURES

Page

Figure 2-1. Theoretical Relationships Among Objectives, Learning
Activities, and Student Achievement ........................... 10

Figure 4-1. A Path Analysis of the Effects of Deﬁned Objective on
Students' Posttest Achievement ................................... 78

xiv

Chapter I
INTRODUCTION

Problem

In elementary school, high school, college, and even graduate school,
students are often given assignments to learn from a text. They are told to
read the text and to do some exercises. In elementary school, a student
may be asked to read from a social studies text to ﬁnd out how to interpret
maps. In high school, a student may be asked to read from an art history
text to be able to understand art styles. In college, a student may be
assigned to read from an economic text to learn how to understand the law
of supply and demand. Each of these students must make choices about
which content in their text to pay most attention, and given a choice about
exercises at the back of the chapter, they must make decisions about which
exercise to do. Speciﬁcally, they must determine what content in their
reading is important and what practice they must do to learn from text.

To aid these students in attending to relevant content and in choosing useful
learning activities, and thereby to learn more effectively and efﬁciently,
instructors and instructional designers use orienting devices to

communicate requirements to students.

One type of orienting device is an instructional objective. An
objective is a statement of "what students will be able to do or how they
will be expected to behave after completing a prescribed unit of course of
instruction (Kibler & Bassett, 1977, p. 55)." An objective includes certain

elements such as Speciﬁc conditions of testing, observable test behavior,

1

deﬁnition Of criteria, and the lower limits of acceptable performance

(Y elon, 1991). For the purpose of this study objectives will contain
conditions and behavior. For example, the elementary school teacher may
tell students that on the test they will be given a map and will be asked to
ﬁnd distances between two speciﬁc locations. The high school teacher
may tell students that on the test they will be given an art work and will be
asked to state the characteristic of the particular work that ﬁt a certain style
of art. The college professor may tell students that on the test they will be
given a supply and demand data for one year and will be asked to predict

the cost of a product for another year.

In exploring the best ways to orient students via objectives,
researchers have found that certain variables best influence their effect.
The objectives must be speciﬁc and complete, that is, the content must be
deﬁned and the task must be presented concretely. (Hamilton, 1985;
Melton,l978; Frase & Kreitzberg, 1975; Kaplan & Rothkopf, 1974;
Rothkopf & Kaplan, 1972; Yelon, 1991).

A deﬁned course objective is a statement that limits the domain of
subject matter clearly. If an objective is deﬁned, students can attend to
relevant content from a text because the label assigned to the domain or
the attributes of the deﬁned domain are clear. For example, the
elementary school teacher might tell students more than: "From this text,
you will interpret maps." The teacher might say: "You will learn to
calculate the distance between.any two cities on a map. " The high school
teacher might also tell students more than: "From this text, you will

understand styles of art." The teacher might say: "You will learn to

identify the art style of impressionists." The college professor might also
tell students more than: "From this text, you will understand the law of
supply and demand." The professor might say: "You will learn to predict
a cost of a particular product."

A concrete course objective is a statement that denotes the referent
test clearly. If an objective is concrete, that means it describes the test
conditions and behavior clearly, so students can choose appropriate means
to prepare for the test because the test situation and performance are clear.
For example, within an objective, the elementary school teacher might
provide an example of a test question based on the reading such as: "There
is a Michigan map, you will calculate the distance between Lansing and
Ann Arbor." The high school teacher might provide in an objective an
example of a test question based on the reading such as: "Here is a Van
Gogh painting you haven't seen before (Starry night). State the
characteristics of the painting which make it an impressionistic painting."
The college professor might provide in an objective an example of a test
question based on the reading such as: "Here is the price of petroleum in
1991. Here is data that show estimated supply and demand of world
petroleum for 1993. Predict the cost of the petroleum in 1993."

Researchers have found that it is relatively easy to successfully
communicate requirements via objectives for recall tasks such as
deﬁnitions, facts, and formulas (Duchastel, 1980; Hamilton, 1985; Ho,
Savenye, & Haas, 1986; Kaplan & Rothkopf, 1974; Rothkopf & Kaplan,
1972). But, researchers have found that it is difﬁcult to effectively

communicate requirements by way of objectives for application tasks

(Barker & Hapkiewicz, 1979; Duell, 1974). An application task is a task
in which learners are not merely engaging in rote learning, but attempting
to use the idea in new situations. Thus, teachers wishing to teach
interpretation of maps, understanding an art work, and understanding of
the law of supply and demand need further research to guide their design
of the application objectives. Therefore, the main question of this
research is: How can instructors successfully communicate requirements
through objectives for application tasks to be learned from text?
Subsidiary questions are: How can instructors help students to know what
to attend to in their text assignments, and to know which learning exercise

to select?

This was a study for exploring the relationship among the three
major factors: (a) communication of test requirements via objectives for
application tasks to be learned from text, (b) students' cognitive learning
processes as they use objectives to learn from text, and (c) student
achievement of application tasks learned from text. An experimental
research design supplemented with questionnaires was used. Fifty-six
graduate students participated and were asked to learn a statistical
application task from text. The experimental protocol attempted to
simulate a typical reading assignment. The students were ﬁrst given
objectives with varying characteristics, the degree to which they were
deﬁned and concrete. Second, subjects were given a text selection to read.
Third, subjects were given a choice of practice exercises similar to those at

the end of a reading.

Research Questions

The following questions served as a framework for the study:

Question one: Will students presented with the deﬁned objective
treatment be able to select more relevant content than students not

presented with that treatment?

Question two: Will students presented with the concrete objective
treatment be better able to select appropriate exercises for practice

than students not presented with that treatment?

micstion three: Will students presented with the deﬁned and
concrete objective treatment be able to get a higher the score on the
application ppsttest than students not presented with that treatment?

mestion four. Will students presented with the deﬁned and concrete
objective treatment be able to use less time to read the text than
students not presented with that treatment?

Need for the Study

This study was undertaken to investigate two relatively unexplored
areas: (a) the effects of objectives on application tasks, and (b) the
dynamics of subjects' thinking as they use objectives. Previous studies
have shown that objectives are useful tools for communicating test

requirements for recall tasks. But research has not demonstrated that

objectives can serve to help subject's learning application tasks.

Thus, this study will investigate how objectives should be stated to
promote learning of an application task from text. This study will show
the inﬂuence of certain qualities of objectives that thus far have not been
systematically combined. The study's principal value therefore lies in
what it contributes to present knowledge of the effects of objectives and
students' learning processes regarding application tasks. It will have
implications for the design of any instruction where subjects must learn to
do a task other than recall from a text. Further, this study could provide a
more detailed idea about what makes an objective speciﬁc and complete.

There have been no studies exploring what students say they are
doing with the objectives that they have been given. Thus, this study will
examine the dynamics of the use of objectives. The study will look at the
mediating variables of attention to text content, and of choice and use of
text exercises as they inﬂuence a student's learning of an application task
from text. Learning how students use objectives to choose content and
exercises may verify what researchers have hypothesized as to the role that
objectives play in directing attention and guiding study. It may also alert
designers as to other factors which affect the use of orienting devices like

objectives.

Outline of the Study

Chapter 11 presents a review of literature as background for the

study and leads to the questions that provide a framework for this study.

Chapter 111 describes the research design, the subjects involved in the
study, and the methods employed. The results of the pilot study and the
procedures used to collect and analyze the data are also reported. Chapter
IV presents the results of the investigation and reports the ﬁndings related
to the research questions. Chapter V contains a discussion of the ﬁndings,
conclusions, limitations, implications of the study, and recommendations

for future research.

Chapter II
REVIEW OF THE LITERATURE

Two major questions are being asked in this study: (a) What is the
effect of communicating application task test requirements on students'
cognitive processes during learning of application tasks?, and (b) what is
the effect of communicating application task test requirements on students'
achievement of those application tasks? Consequently, this study has two
purposes: (a) to investigate the effect of communicating course
requirements via objectives, and (b) to explore students' cognitive
processing of test requirements through given objectives to choose relevant
content and practice exercises. Therefore the review of literature is
divided into two sections. The ﬁrst body of literature to be reviewed
considers ways to communicate to students application task test
requirements via objectives. The second body of literature to be reviewed

addresses students' cognitive processing during learning.

The researcher used the Educational Resources Information Center
(ERIC) service to review the recent literature on the study of objectives,
students' cognitive processing, attention, and learning. In addition, the

researcher carefully checked and traced reviews of related literature.

An Overview of Theoretical Relationships among Objectives,

Learning Activities, and Student Achievement

The review of literature is structured to support the following

theoretical relationships. In the rest of the chapter, evidence is presented

8

to substantiate these relationships and to lead toward the present research

questions.

To learn from prose text in formal learning settings, students must
choose relevant content. Further, to prepare themselves for tests in
instructional settings, Students also must often choose relevant exercises for
practice. To make relevant choices of content and practice, students must

know the test requirements.

One strategy, to provide students with the knowledge of what is
required on the test, is the instructional objective. If students clearly
understand the concept of what is on the test as explained in the objective,
students will be able to selectively attend to the content and the exercises

that will help to produce achievement.

Instructors can best communicate using instructional objectives when
the objectives are deﬁned and concrete (Yelon, 1991). A deﬁned Objective
is a statement clearly describing the limits of the domain of the subject
matter of the test. For example, an instructor might deﬁne a domain in an
objective by stating that students will be expected to identify examples of
four types of validity. In contrast, an instructor might leave the objective
undeﬁned by stating that students are to identify examples of a
measurement idea. If an objective is deﬁned, students can attend to
relevant content of a text, because the label assigned to the domain or the

attributes of the deﬁned domain are clear.

10

A concrete objective is a statement that denotes the referent test
clearly. If an objective is concrete, that means it describes the test
conditions and behavior clearly. For example, a concrete objective might
be: Given a written case vignette such as "a researcher correlated a
perceived social support measure with a general self-esteem measure to
Show that perceived social support was a part of the self-esteem variable,"
students will choose which type of validity the example illustrates from
these choices: content, concurrent, construct, and predictive validity.
Students can then choose appropriate means to prepare for the test, because
the test situation and performance are clear. If students can attend and
study the relevant content and choose the best practice for a test, they are

likely to achieve what the test requires.

 

 

f cram-acumen
Relevant . content

‘jneﬁnedcom-s.
Requirement

" " ‘ Selective _
' Attention
etc
'5;r_-,~;R.equirement'

 

 

Cflfearerchdiceof.

2‘5 *.P“3¢ﬁ¢°' Exam

 

 

 

 

 

 

ezmzm<w~mo>

Figure 2-1. Theoretical Relationships among Objectives,

Learning Activities and Student Achievement

Figure 3-1 shows a model based on the described relationships
among objectives, learning activities, and student achievement . Note how

deﬁned requirements aid students in attending selectively to relevant

11

content, how concrete requirements aid students in attending selectively to
practice exams, and how relevant content and appropriate practice lead to

achievement.

This study investigates the effects of communicating application task
requirements on students' learning processes and achievement. An
application task is a task in which learners are not merely engaging in rote
learning, but attempting to use the idea in new situations. For example,
application tasks may include identiﬁcation of new examples, explanation,

or prediction Of new cases.

Most research on the instructional effects of objectives has shown
that objectives clearly communicate instructional requirements for recall
tasks but are not as effective in orienting students for tasks other than recall
(Barker and Hapkiewicz, 1979; Duell, 1974; Duchastel, 1980; Ho, Savenye,
& Haas, 1986; Kaplan & Rothkopf, 1974; Rothkopf & Kaplan, 1972).
Perhaps the qualities present in a recall objective and the qualities present
in an application objective must be different to be effective. It may be that
the domain of subject matter to be learned to achieve an application task is
not as clear to students as the domain in a recall task, and therefore, the
domain in an application task objective must be clearly deﬁned to be
effective. It may also be that the task of recall is apparent to students.
When there is a recall test, students generally know what the test will be
like. But the test for an application task is not obvious, it must be
specified. Thus, an application task objective must include a concrete
example of the test question. The logic continues that students receiving a

deﬁned application objective will be able to efﬁciently direct their attention

12

to the deﬁned content in an assigned reading and students receiving a
concrete application objective will be able to select an exercise matching
the example given in the objective. When students attend to relevant
content and choose exercises that match the test conditions and behavior,

they are likely to learn the application task efﬁciently and effectively.

Thus, there is one main research question: Does the presentation of
application task objectives which deﬁne a subject domain and concretely
exemplify a required test affect students' learning process and resulting
achievement? There were two subsidiary research questions: (a) Does the
presentation of application task objectives which deﬁne a subject domain
help students to know what to attend to in their assigned reading?, and (b)
does the presentation of application task objectives which concretely
exemplify a required test help students to know which exercise to select
when given a choice? There is also a practical question posed by this
study: Are objectives an effective tool in helping instructors successfully

communicate instructional requirements of application tasks?

Research on Objectives

This section of the review of literature on objectives is divided into
research on the effects of objectives on learning, and writings about
characteristics of effective objectives. The subsections of the review of
research on effects of objectives on learning deal with: (a) effects of
objectives on recall and application tasks, (b) effects of objectives on

relevant and incidental outcomes, and (0) effects of speciﬁc and general

13

objectives. The subsections on the characteristics of objectives deals with

effects of deﬁned objectives and concrete objectives.

What is a_n Objective?

Kibler & Bassett (1977) referred to objectives as statements of "what
students will be able to do or how they will be expected to behave after
completing a prescribed unit of course of instruction (p. 55)." Similarly,
Hamilton (1985) deﬁned objectives as "preprose statements that are
intended to focus the student on the to be tested material (p. 66)." An
objective includes certain elements such as speciﬁc conditions of testing,
observable test behavior, deﬁnition of criteria, and the lower limits of
acceptable performance (Y elon, 1991). Davis, Alexander, and Yelon
(1974) deﬁned the behavior in an objective as "the component of a learning
objective that describes the behavior of a student after instruction (p. 33)."
They also deﬁned the conditions as "the component of a learning objective
describing the situation in which the student will be required to
demonstrate the terminal behavior; the component that describes the test
conditions (p. 37)." Although some authors refer to criteria and lower
limits of objectives, for the purpose of this research, discussion of

objectives is concentrated only on conditions and behaviors.

Effects of Objectives on Guiding Learning

The use of objectives is one way to communicate test requirements to
students. In theory, providing students with objectives enables them to

orient their learning activity towards the speciﬁc desired behaviors that

14

must be mastered to complete the instructional requirements satisfactorily
(Kibler & Bassett, 1977).

Most early experiments on the use of objectives asked only if the
presence of objectives produced learning and resulted in unreliable and
inconclusive ﬁndings (Kibler and Bassett, 1977). Sometimes objectives
had a positive effect on leaming and sometimes they did not. Thus, mere
presence of any sort of objective, constructed in any manner, was not
sufﬁcient to produce effects on learning. In their early review of
literature on the use of objectives, Kibler and Bassett (1977) also said that
research had not consistently demonstrated any differential effects on
student learning attributable to the way in which objectives were stated.

At that time most research dealt with gross differences between general and
speciﬁc objectives. The researchers at that time did not vary the
characteristics of speciﬁcity or clarity of objectives except in a very broad
manner, that is, whether the objectives were behavioral or not behavioral,
whether or not they included traditional parts of objectives. In addition,
Kibler and Basset pointed out that in 1977 there were too few studies
available on these questions to infer conclusions. However, in a more
recent review, Hamilton (1985) resolves some questions about the effects of
objectives, but still points out the varying effects of objectives on different
tasks. He notes that there are only a couple of studies on the effects of

objectives on application tasks.

Hamilton as well as other researchers state that research on
objectives has suggested some clear relationships as well as some puzzles

yet to be solved. Three important relationships that may serve as guides to

15

further research are: (a) the effect of objectives on recall versus applied
tasks, (b) the effect of objectives on relevant versus incidental outcomes,

and (c) the effect of general and speciﬁc objectives on learning.

Effects of objectives on recall and applicatjon tasks

Objectives are effective for the learning of verbal information
(Duchastel, 1980; Ho, Savenye, & Haas, 1986; Kaplan & Rothkopf, 1974;
Rothkopf & Kaplan, 1972). However, objectives have not been found as
effective for learning of application tasks as they have for the learning of
verbal information. For example, Barker and Hapkiewicz (1979) found
that a group with no objectives achieved the same score as a group with
objectives on a posttest calling for an evaluation task. Duel] (1974) also
found no difference on an application task between an experimental group

with objectives and a control group without objectives.

Researchers agree that the effectiveness of objectives to inﬂuence
learning differs as a function of type of knowledge of the task to be learned
(Hamilton, 1985; Kibler & Bassett, 1977; Lewis, 1981; Melton, 1978).
After reviewing quite a large number of research studies on objectives,
they all concluded that objectives have consistently produced positive
effects on achievement for recall tasks while producing no effects or small
effects on achievement for application tasks. Hamilton showed that recall
objectives with fewer than two of the traditional components suggested by
Mager (1962), that is, conditions, behaviors, and criteria, produced one of
the highest difference scores of the treatment groups in a set of

experiments. Hamilton stated that when an objective points out the

16

information to be learned in a text, the learning effect will be strong.
When more information is provided than the information to be learned for

a recall task, the learning effect is likely to hindered. Hamilton stated:

The lack of need for speciﬁcity or completeness of the goals/objectives to produce
positive effects may be a function of the low level of learning outcome. That is,
too much information may interfere with the processing of to be learned information
when only a very superﬁcial level of processing is required. The need to present
complete and speciﬁc objectives (per Mager's definition) may occur only at higher
level learning outcomes. The relationship between speciﬁcity of goals/objectives

and measured learning outcomes should be the focus of future research (p. 78).

Thus, one interpretation of the ﬁndings that objectives are effective
for the learning of verbal information but not for application tasks is that
the most effective objectives for the learning of verbal information are
spe_ci_11g and complete for those purposes, whereas the objectives for
application tasks are not as speciﬁc and complete for those purposes.
Students not provided with clear orienting directions for an application task
would probably expect to be given a recall test and would not expect a
higher level test over the text material (Duell, 1974).

To be speciﬁc and complete, the objectives for application tasks must
be clearly deﬁned and concrete in nature. When given a recall objective
describing a test of recall of verbal information, such as the direction to
write the four causes of World War II, it is relatively easy for students to
select relevant content from a text and to know what to expect on the test.

Therefore, students can prepare for the test by choosing the most relevant

17

content and by using the learning activity that is most compatible with the
described end state. For example, if the desired outcome is rote recall of
four causes of WW II, the relevant content is clearly deﬁned and perhaps
the most appropriate activity is practicing recall of the four causes.
Suppose, however, that the desired outcome is an application task, that is,
comprehension of the signiﬁcance of information contained in given
material, such as, to analyze a story, or to apply the information to a novel
problem, such as, to know the appropriate statistic for a given problem.
Then, students might have difﬁculty in selecting relevant content from a
text and knowing what to expect on the test, since the content is not deﬁned
nor is the task stated concretely. For application tasks objectives, to be
effective in fostering learning, it seems plausible that the domain of content
must be deﬁned and a concrete description and example of the task must be
included.

Because application tasks require the additional Skills of identifying
and selecting the relevant pieces of information that are useful in solving
new problems (Lyon & Gettinger, 1985), if the content is not deﬁned nor
concrete, students may have difﬁculty distinguishing what content is
important and what practice is appropriate. In other words, when the
objectives are not deﬁned and not concrete as they are in most commonly
written application objectives, the relevant content and practice is not
obvious. In support of this view, Glover, Plake, and Zimmer (1982)
found that objectives for higher-order learning outcomes were more
difﬁcult to describe and classify than were objectives for lower-order
outcomes. What exactly is the content domain? Which stories? Which

problems? Which statistics? What is the test form? How will stories and

18

problems be given? How will the students respond? What will be

relevant information and what will be the appropriate learning activity?

Thus, the research question in this study concerns the degree to
which students learn something when confronted with various kinds of
objectives about an application test. If an application objective lacks some
important characteristics to make it speciﬁc and complete, then it cannot be
of help to students in orienting themselves to desirable learning activity for
the application test. Although students must modify their learning activity
in light of their ideas of the nature of the test, it is a difﬁcult task for
average students. It is made more difﬁcult when students' ideas about an
application test are based on an undeﬁned and an abstract objective. That
may be why some objectives are effective for learning verbal information
but not for learning application tasks. Therefore, deﬁned and concrete
application objectives seem likely to help students to modify their learning
activity for the desired outcomes. The ability to modify one's activities in
light of changes in the nature of the test is an essential factor in efﬁcient

learning for that test.

Effect of objectives on relevant and incidental outcomes

Researchers have been concerned about what difference objectives
can make to students on relevant learning. There are a large number of
studies that support the effectiveness of objectives on relevant learning
(Duchastel & Brown, 1974; Kaplan, 1974; Kaplan & Rothkopf, 1974;
Kaplan & Simmons, 1974; Rothkopf & Billington, 1975, 1979; Rothkopf
& Kaplan, 1972). However, some researchers (Barker & Hapkiewicz,

19

1979; Duell, 1974) found that objectives made little difference to students
on relevant learning. Melton (1978) discussed the anomaly by concluding:
"Clearly, it is not sufﬁcient to simply provide students with behavioral
objectives. They must also be aware of them (p.293)." He further
concluded that behavioral objectives made little difference to students who
were highly conscientious, or well motivated. Such highly motivated
students can achieve regardless of whether or not the objectives are
speciﬁed. Although there are some exceptions in the effectiveness of
objectives on relevant learning, researchers generally conclude that
objectives are effective for the learning of relevant content (Hamilton,
1985; Lewis, 1981; Melton, 1978).

Researchers are also concerned that objectives may depress incidental
learning. Some researchers claim that objectives indicate to students what
is required of them, and as a result incidental learning is depressed
(Duchastel & Brown, 1974; Frase & Kreitzberg, 1975; Rothkopf &
Billington, 1975, 1979). They argue that objectives discourage students
from expanding their horizons by encouraging them to conﬁne their
learning to speciﬁed tasks, and as a result incidental learning is depressed.
On the contrary, some researchers found that objectives had little or no
effect on depressing incidental learning (Duell, 1974; Kaplan & Rothkopf,
1974; Rothkopf & Kaplan, 1972). In fact, Kaplan and Rothkopf concluded
that the provision of objectives enhanced incidental learning as well as

relevant learning.

The reason that objectives are effective in helping students to choose

and learn relevant content is apparently that clearly deﬁned and limited

20

objectives cue students to attend to speciﬁc categories of content. So it is
no surprise that well speciﬁed, deﬁned objectives affect the learning of
relevant content. However, researchers do not explain the reason why the
presentation of objectives may sometimes depress learning of incidental
content and at other times make little difference in incidental learning.
Researchers have had little to say about the reasons for students' learning
of incidental content because they have not investigated the students'
learning activities or mental processes during preparation for the test.
Research thus far simply included a posttest to measure the incidental

learning.

One reason for the conﬂicting results on incidental learning might be
that incidental learning is unaffected by objectives when objectives are
poorly deﬁned. If students have no clear idea of what is to be learned,
they are likely to learn incidental as well as relevant content. When
objectives are precisely deﬁned in terms of the domain of content to be
learned, then incidental learning is likely to be depressed. Another reason
might be that incidental learning may be unaffected by objectives when the
objectives are not concrete. If an objective does not specify the conditions
and behavior and no examples are given of the task, students may seek
more information than speciﬁed already to be sure they are prepared for

any event.

In sum, if students have difﬁculty distinguishing what content is
important, they will also have difﬁculty studying. Quite simply, one
cannot selectively attend to important material in the absence of precise

limits to what is important. Therefore, to the extent that application

21

objectives are deﬁned and concrete, students can direct their attention to

relevant content and practice exercises.

Effects of general and smpiﬁc objectives on learning

Although some studies have found no difference between the effects
of general objectives and speciﬁc objectives (Duell, 1974), speciﬁc goals
are generally thought to enhance performance. Frase and Kreitzberg
(1975) conducted a study which assessed the effects of general and speciﬁc
goals on students' achievement. The group given the speciﬁc goals
performed better than a control group given no objectives and better than a
group given general goals, while controlling for any potential encoding
speciﬁcity effect. Several other researchers (Rothkopf & Kaplan, 1972;
Kaplan & Rothkopf, 1974) also noted that the performance of students
provided with precisely stated objectives was signiﬁcantly better than that
of students provided with either vaguely stated instructional objectives or
short paragraphs of information. Stein (1978) also conﬁrmed this view by
noting that when objectives are vague, students do not perform well.

Thus, it appears that speciﬁc objectives enhance performance.

But, what makes an objective speciﬁc? Researchers have found that
increasing the number of goals reduced students' performance (Kaplan &
Rothkopf, 1972; Rothkopf & Billington, 1975; Rothkopf & Kaplan, 1974).
Increasing the number of goals may lead to broad coverage of content to be
learned, thereby, reducing students' performance. Thus, perhaps to
increase a performance, a speciﬁc objective should be deﬁned to help a

student focus. Further, if objectives do not include examples of tests,

22

students expect retention types of tests, even though tests are supposed to be
higher order questions (Duell, 1974). Therefore, a speciﬁc objective may
be thought of as concrete as well. Thus, an objective can be considered

speciﬁc when the content domain is deﬁned and when the task is concretely

described and illustrated.

Sammy

An objective that is abstract and vague is likely not to be helpful to
students' learning. Perhaps that is the reason that application objectives,
which are usually abstract and vague, usually have been found not to affect
on learning. However, if application objectives are deﬁned and are
concrete they should produce greater achievement than the usual abstract

or vague application objectives.

Characteristics of Effective Objegtives

There are good reasons to believe that if students are given
objectives carefully structured for a given task, the objectives will have an
effect on learning. As mentioned, objectives are likely to be inﬂuential if

they are deﬁned and concrete.
Deﬁned objgtjves
First, objectives must be deﬁned, that is, they must limit the content.

When objectives are deﬁned, a student can choose relevant content from

the text they study.

A deﬁned objective is a form of an objective in which the domain of
subject matter is speciﬁed and the range of subject matter on the test is
limited. In a deﬁned objective, the labels assigned to the content
requirements or the attributes of the content domain are so clear that a
reader could choose examples of relevant test content with little doubt and
with little error. Brown, Campione, and Day (1981) supported the
usefulness of deﬁned objectives by arguing that exact speciﬁcation of the
rules that could be used to achieve a goal was an extremely effective

instructional route.

In prose learning, because an entire document is usually not relevant,
it must be searched selectively, and an optimal solution maximizes accuracy
and minimizes time. Guthrie (1988) suggested the optimal solution as a
component that can activate the selection of categories to develop students'
ability locating information in documents. He proposed a cognitive
processing model to account for performance on locating information in
documents. An important and ﬁrst component of his model is a form of
objective that can lead readers to relevant information. According to

Guthrie, effective instruction should include that component.

Therefore, if an objective is deﬁned, students can pick out relevant
content from their texts. A deﬁned objective includes conditions in which
classes or types of the test content are speciﬁed, and classes and types of the
behavior and its object are speciﬁed. Therefore, a deﬁned application

objective should limit all content variations in conditions and behavior.

24

Concrete objwjves

Though potentially useful, limited Objectives alone cannot facilitate
performance better than do more general objectives. Second, therefore,
objectives must probably also be concrete, that is, they must give the
student a clear idea of the test form. Duell (1974) suggested that when
orienting directions are not clear, students would probably expect recall
questions even though tests were supposed to be hi gher-order questions.
With a concrete idea of the required task, students can think of the most

appropriate means to study for the test.

Concrete objectives denote the referent test clearly. Because a
concrete objective matches the referent test, students can pick out a relevant
test form and therefore choose the most appropriate exercises or practices.
Guthrie (1988) suggested that readers must be able to verbalize an
objective in the form of information that is to be found in the document.
Therefore, objectives that include the form of information in the document

may be essential components in successful reading.

Concrete objectives enable learners to recognize that a given test or
practice exercise belongs to a class that Shares a common characteristic or
property. Once learners have the concrete objectives, they can correctly
identify the examples of appropriate practices and tests from nonexamples.
When teachers orient students with objectives, they are attempting to teach
them a concept of the test. How should the concept of the test be
communicated most clearly? Researchers suggest that a learner's ability to

correctly apply concepts is signiﬁcantly inﬂuenced by the speciﬁc

25

combination of examples and nonexarnples used to teach the concept
(Tennyson & Park, 1980; Tennyson & Cocchiarella, 1986). Thus, to give
students a clear concept of the test, teachers should use concrete examples
or concrete descriptions of what the test will be like. If the concept of the
test is made clear, then students should be able to pick out examples of a

practice test or matching exercise to prepare for the test.

A concrete objective includes concrete conditions in which the
example of the test and the description of the variations in test conditions
are stated, and concrete behaviors in which observable behaviors and
objects of the verbs are denoted. Therefore, in an application objective,
concrete conditions have to Show the variations in their descriptions and
examples, and an operational statement of behavior. In other words, the
test situation and the test performance should be clear. Duchastel (1977)
even suggested that instructors should give students practice tests items to
insure the best use of objectives. It makes sense then to add practice items

to objectives to make them clear.

Summary

For an objective to be speciﬁc and complete it must apparently
deﬁne the content to be learned and supply a concrete example of the test.
When given an objective that is concrete as well as deﬁned, students can
selectively attend to relevant information and choose the most appropriate

exercises to prepare for a test.

26

Adjunct Ouestiorps as a Concrete Indicator in an Objective

Using questions is an effective instructional strategy. Questioning
can be traced back to Socrates who used a chain of questions to lead
students to conclusions. Following the tradition of Socrates, several
researchers have tried to develop theories of learning using questions
(Anderson & Faust, 1974; Collins, 1978; Sigel & Saunders, 1979).
Recently, Collins and Stevens (1983) developed a cognitive theory of

inquiry teaching that emphasizes the importance of questioning in learning.

Using questions can also an effective orientation strategy to alert
students to what is going to be on a test. Adjunct questions, a concept
developed by Rothkopf (1965), are questions presented before, during or
after text to help students learn. For example, adjunct questions may alert
students as to what is going to be on a test based on the text. It is a subset
of research on "mathemagenic" activities. Rothkopf suggested that a
teacher might provide additional activities to the traditional prose text that
would induce the reader to more actively participate in the reading and

learning process.

In the research on adjunct questions, students are given texts to read
with questions inserted either before or after paragraphs that contain
relevant content. In these studies, prequestions usually facilitate verbatim
and factual learning (just like the effects of objectives on recall tasks),
while post questions facilitate conceptual learning or learning of

information not speciﬁc to the question (Boker, 1974).

27

For example, Watts and Anderson (1971) compared three types of
postquestions: the repeated example, application questions, and control.
The repeated example was for the retention of a concept or principle, and
the application questions were for the transfer of a concept or principle.
The group given application adjunct questions achieved better on

application posttest questions than did the other two groups .

Felker and Dapra (1975) conﬁrmed Watts and Anderson's results by
comparing ﬁve types of adjunct questions: (a) post-adjunct comprehension
question group, (b) pre-adj unct comprehension question group, (0) pre-
verbatim adjunct question group, (d) post-verbatim adjunct question group,
and (e) control group. They found that adjunct comprehension post
questions produced signiﬁcantly better performance on the problem-

solving test than all other types of questions.

Why are prequestions effective for facts, while postquestions are
effective for application tasks? Fact prequestions are easy to interpret and
direct students' attention to relevant content while comprehension
prequestions may not be as easy to interpret. On the other hand, even
though they may be harder to interpret, comprehension postquestions give
students an opportunity to reread the relevant content and infer the

objective as well as an opportunity for monitoring their comprehension.

Frase (1968) compared the effect of "broader" adjunct questions and
"speciﬁc" adjunct questions on recall learning. He found that broader
questions led to poorer posttest performance. He explained that broad

28

questions may have altered the subjects' conception of the task so that they
did not attend to the speciﬁc material that was included in the posttest.

Researchers attribute the effect of adjunct questions on learning to an
increase in attention caused by questions (Reynolds & Anderson, 1982;
Reynolds, Standiford, & Anderson, 1979) and an enhancement of the
opportunity for using comprehension monitoring (Brown, Bransford,
Ferrara, & Campione, 1984). Thus, the literature on the use of adjunct
questions and the use of objectives is tied to the literature on students'

cognitive processing during learning.

Summm

Adjunct prequestions usually facilitate factual learning, while adjunct
post questions facilitate conceptual learning. In general, speciﬁc adjunct
questions are more effective than are broad ones. The present study asks:
Would an example of a speciﬁc adjunct prequestion inserted in an objective
as a concrete indicator of the test serve to help students choose relevant

content and choose appropriate practice?

Research on Students' Cognitive Processing during Learning

The effects on telling students the course requirements depend
largely on what the learners think about during learning. Leamers must
actively process the orientation messages they are given. Objectives don't
automatically produce learning. Objectives give students an idea of the

test format, and the relevant content. But students must apply the

29

Objectives to infer the best content and the best practice to prepare for the
test. Two important factors for the application of objectives are attention
students give to signiﬁcant or relevant information, and strategies they

tailor to the learning situation.

Attention

Attention is a student thought process that may help to explain some
of the student's learning (Reynold & Anderson, 1982; Reynold & Shirey,
1988; Wittrock, 1986). Students vary in their attentive capacities (Hagen
& Hale, 1973; Miller & Weiss, 1981). Some students can be fully attentive
for long periods of time, others for short periods only. Some are more

distractable than others.

More importantly, some can selectively attend to important material
while others have difﬁculty to attend to what is important. Many
educators stress the importance of paying attention during learning, since
the ability and willingness to pay attention is a major factor in school
learning. It is this process where active mental effort is expended and
comprehension takes place. Those students who can pay attention most
efﬁciently to important material may learn the most and achieve the most.
Many researchers of learning disabilities see attentional deﬁcit as the most
critical defect of the learning disabled child (Hagen & Hale, 1973; Rutter,
1989).

Achievement is generally regarded to be closely related to attention.

Higher achieving students are more inclined to attend to learning (Peterson,

30

Swing, Braverrnan, & Buss, 1982; Peterson, Swing, Stark, & Wass, 1984)
and they make more effective use of the cognitive strategy of attention

allocation than do lower achieving students (Reynolds and Shirey, 1988).

Good learning strategists are attentive to the demand placed on them
(Pressley, Goodchild, Fleet, & Evans, 1989). In order for the strategy of
attention allocation to be effective, the student must have the capability to
utilize the parameters of the task and the text to determine the importance

of text elements.

To understand, learners must seek information about the signiﬁcance
or relevance of facts. Therefore, students must engage in active strategies
to ensure increased attention to important material that will not be retained
automatically. As human beings mature, they become better able to
identify what are the essential organizing features and crucial elements of
texts. If students have difﬁculty distinguishing what is important, they
will also have have difﬁculty studying. Quite Simply, one cannot
selectively attend to important material in the absence of a ﬁne sensitivity
to what is important. Therefore, if students use objectives, they can use

them to direct their attention to relevant content and study means.

Objectives may indicate to the learner the information in the text that
will be the focus of the test (Hamilton, 1985). Speciﬁc objectives are
especially helpful in locating information in textbook reading. Guthrie
(1988) suggests a theoretical model to account for a student's use of an
objective in ﬁnding relevant information in text. The model includes ﬁve

components: (a) goal formation, (b) category selection, (c) extraction of

31

information, (d) integration, and (e) recycling. The model states that a
speciﬁc goal or an objective as an important component to orient readers to

important information.

Researchers have also found that objectives function by inﬂuencing
selective attention (Anderson, 1982; Shirey & Reynolds, 1988). In
research on the effects of giving students objectives, an attentional model
has provided a useful explanation of the ﬁndings (Wittrock,1986).
Anderson (1982) proposed an attentional theory of prose learning as

follows:

(1) Text elements are processes to some minimal level and graded for importance.
(2) Extra attention is devoted to elements in proportion to their importance.
(3) Because of the extra attention, or a process supported by the extra attention,

important text elements are learned better than other elements (p. 292).

Su_m_m_ant

Objectives are useful only when students use them as a tool to guide
their attention to the important content in text and to the best practice
exercises. Thus, attention is an important mediating variable leading to

achievement.

Strategies

One of the primary modes through which students acquire

information and knowledge in an academic setting is by reading expository

32

prose (Calfee & Drum, 1986; Hamilton, 1985; Just & Carpenter, 1987).
Researchers studying reading comprehension have shown that a reader's
ability to use orienting devices as objectives while processing a prose
passage is positively related to the ease of comprehension and retention of
the prose passage (Anderson, 1982; Brown, Campione, & Day, 1981;
Guthrie, 1988; Guthrie & Kirsch, 1987; Hamilton, 1985). Orienting
devices help students to focus on important content and to use study time
efﬁciently. Thus, students are able to focus adequate amounts of quality

attention on the important information in order to learn it.

Learning strategies inﬂuence student behaviors and thoughts about
the best way to learn (W ittrock, 1986). Therefore, successful strategies
are learning processes that, when matched to the requirements of tasks,

facilitate performance.

Some strategies can be used only in very speciﬁc situations in
particular domains (Pressley, Goodchild, Fleet, & Evans, 1989). There
are sets of particular strategies tailored to each of these situations (Pressley
et al., 1989) and students must tailor their activities precisely to the
competing demands of requirements in order to become effective learners

(Brown, Campione, & Day, 1981).

There is a long history of interest in the types of strategies students
bring to the task of learning from texts. Some learning strategies are
rehearsing information, elaborating information, and organizing
information. For example, students may use notetaking, underlining,

adjunct aids, question asking, and outlining to rehearse, elaborate, and

33

organize. Comprehension monitoring is one of the most important
strategies for learning information and ﬁnding relevant content in text. It
is also of use in test preparation. Another learning strategy is engaging in
the preparation and practice for a test. All of the strategies mentioned are
related to students' selection of text content and selecting exercises or

practice in preparation for a test.

Strategies related to choosing relevant content of text

Students must tailor their learning strategies to the demand placed on
tasks. If the desired outcome is rote recall, perhaps the most appropriate
strategy is mnemonic elaboration. If, however, the desired outcome is
comprehension of the signiﬁcance of information contained in the material
or the application of the information to a novel problem, then the
appropriate activity would change. Guthrie and Kirsch, ( 1987) found that
locating information in text and reading comprehension are two separate
factors in text reading. Therefore, an appropriate learning activity must

be one that is compatible with the desired end state.

The knowledge of textual importance, knowledge of suitable
strategies, and estimation of one's current state of mastery have been found
in a series of school-like tasks such as notetaking, outlining, summary
writing, and retrieval-cue selection. Within the series of studies conducted
by Brown and her colleagues, qualitative differences were repeatedly found
in the types of notes, summaries, and outlines produced by spontaneous

users of a comprehension monitoring strategy (Brown, 1980; Brown &

34

Day, 1983; Brown & Smiley, 1978). Comprehension monitoring may be
the most important strategy that affects the use of objectives.

Comprehension monitoring is closely related to students' use of
objectives (Reynolds & Shirey, 1988). Comprehension monitoring
requires the student to establish learning goals for an instructional unit or
activity, to assess the degree to which these goals are being met, and, if
necessary, to modify the strategies being used to meet the goals.
Comparisons of good and poor comprehenders have consistently shown
that poor comprehenders are deﬁcient in the use of active learning
strategies needed to monitor understanding (Weinstein & Mayer, 1986).
Palincsar and Brown (1984) demonstrated that students' learning was
improved by orienting them to a particular direction and monitoring it.
Therefore, if students are given an objective that is clearly deﬁned and
concrete, students can use it to choose relevant content of a text and can

monitor the degree to which the requirements are being met.

Stpategies relateg to choosing exercises

Practice simulating the actual nature of a criterion task may be the
most important student learning activity. Several researchers (Anderson,
1980; Hannaﬁn, 1987; Mayer, 1984) emphasized the importance of practice
during instruction. Duchastel (1977) suggested that to insure that
objectives will be used most efﬁciently, students should be given valid
practice with objectives and the class of material to be learned. Hannaﬁn
(1987) found that the combination of practice and orienting activity

produced a signiﬁcant interaction as well as a powerful effect of practice.

35

He contended that orienting activity alone was not a signiﬁcant instructional
component. Nitch (1977) also showed that the kind of practice students
engage in has an important effect on type of test, that is, recall or
application. In Nitch's study, students who had received practice that
required them to act in varying contexts performed better on an application
test than students who had received practice that required them to act in the
same context. Varied-context practice was better preparation for the
application task requiring use of a concept, whereas same context practice
produced faster rote learning of the particular exemplar in the original

task.

When objectives are not concrete, in the absence of other cues,
students cannot be sure of the speciﬁc requirements of the test and they
cannot choose or make the most out of practice opportunities. How can
students be expected to perform well when confronted with a test that they
are not prepared to handle adequately because they chose the wrong
practice? Thus, when students are told concretely what a test will be like,
students can adjust their learning strategy in light of their knowledge
concerning the actual nature of the test, that is, they can choose the best
practice for the test. The ability to adjust one's practice activities in light
of information about the nature of the test is an essential factor in efﬁcient

learning.

Students make decisions in study to practice in certain ways. If
students are given a clear notion of the test requirements, then they should
be able to pick a practice exam that matches the test. In fact, to be

successful, students must ﬁnd matching practice because practice interacts

36

with the orienting activity (Hannaﬁn, 1987; Hamilton,l985). To do well
in an exam for a particular objective, a student must engage in a related
practice. In other words, objectives alone are not a signiﬁcant
instructional component. It is what students do with objectives to guide
their choices of practice that is signiﬁcant. Therefore, if students are
given an objective that is clearly deﬁned and concrete, students can use it to

choose appropriate practice.

Summa_ry

Deﬁned and concrete objectives by themselves will not be effective.
They are tools to help guide students' attention. Students must employ
learning strategies to use the information in the objectives to select the most
important content in text and to choose the best practice exercises for the
test. Attention acts as a mediating variable leading from the deﬁned and
concrete objective to appropriate learning strategies and from there to

achievement.

Research Questions

The theoretical question of this study is what combination of
characteristics of an objective most effectively and efﬁciently inﬂuence the
application of ideas learned from text? This study is based on a hypothesis
which states: the more concretely stated the task in the objectives and the
more carefully deﬁned the subject matter stated in the objective (other
things being equal), the more effective the inﬂuence on the application of

ideas learned from text. The reasoning for this result is that when given a

37

speciﬁc deﬁned domain, students can selectively attend to the relevant
information in the text; when given a concrete idea of a task, students can
think of the most appropriate means to study the text. Furthermore, when
given the most relevant content and the most appropriate practice, subjects

are likely to achieve the most.

As a consequence, speciﬁc research questions are:

Question 1: Will students presented with the deﬁned objective
treatment be able to select the more relevant content than students not
presented with that treatment?

Question 2: Will students presented with the concrete objective
treatment be better able to select appropriate exercises for practice than

students not presented with that treatment ?

Question 3: Will students presented with the deﬁned and concrete
objective treatment be able to get a higher the score on the application
posttest than students not presented with that treatment?

Question 4: Will Students presented with the deﬁned and concrete
objective treatment be able to use less time to read the text than students not
presented with that treatment?

Chapter III
METHOD

The purpose of this chapter is to describe the research design and the
methods of investigation employed in the study. The pilot study, sample,
instrumentation, and data analysis are also reported.

Research Design

An experimental research design was used to answer four research
questions. The research design is a posttest only control- group experiment
(Campbell and Stanley, 1963). The posttest only control-group design was
used because a pretest could act as an orienting device, and may confound

the treatment effects.

Fifty-six volunteer students in education participated in this study.
Subjects were randomly assigned to one of four treatments: deﬁned and
concrete objective, deﬁned and pp; concrete objective, concrete and n_ot

deﬁned objective, and p91 deﬁned and pp; concrete objective.

Subjects participated individually or in groups of between two to
ﬁve, each receiving his or her own treatment. The researcher
administered the experiment. Before the experiment, all subjects
answered questions about their previous experience with objectives and
with correlation, the subject matter to be studied. Next, subjects read
along as they listened to a tape describing instructions for the experiment.

Then they read their own objective of an application task dealing with the

38

39

statistical subject: correlation.

The experimenter asked the subjects to learn to perform the
application task: "choosing the appropriate correlational technique for a
given set of data. " To choose an appropriate technique for a given set of
data, a student must recall the type of data associated with the correlational
technique. But the task involved more than recall. Subjects had to
remember the attributes of each correlational technique required and apply
those attributes to new examples to be able to identify the right technique.
Thus, the task was an example of concept identiﬁcation, and as such, was a
short mental skill with as least ﬁve steps: (a) study the data, (b) recall
characteristics of types of data, (c) identify the types of data, ((1) recall the
technique associated with the type of data, and (e) choose the appropriate

name of the technique.

One group was given the most complete objective including a clearly
limited domain of content required for the posttest, a precise description of
the task behavior and the task conditions, as well as a concrete example of a
test item. Following is an example of the most deﬁned and concrete

course requirements in the form of an objective:

Given data regarding £1! combinations of continuous and artiﬁcial dichotomy variables,
such as:

Student SAT Scores Algebra Test (Successzl; Failure=0)

Doyle 350 0

Sabers 450 1

Glass 550 1

etc.

you are to circle the name of the appropriate correlation technique from m these choices:
a Pearson Product-moment correlation

b. Tetrachoric correlation

c. Biserial correlation

The second group was given an objective including a clearly limited
domain of content required for the posttest. However, the objective's
behavior was somewhat vague and no example of the type of item was
included. Following is an example of the deﬁned but not concrete

objective:

For m combinations of continuous and artificial dichotomy variables, you are to know
the appropriate correlation technique from oply the following choices:

a. Pearson Product-moment correlation

b. Tetrachoric correlation

c. Biserial correlation

The third group was given an objective with a concrete example of
the type of test item to appear on the posttest. The test behavior was
precisely stated, but the general description of the test conditions was
relatively vague. Also the subject matter domain was not limited to
certain types of correlations. Following is an example of the concrete but

not deﬁned course requirements:

Given some data, such as:

Student SAT score Socioeconomic Status (Hi gh=l; Low=0)

41

Moll 350 1
Chipman 450 0
Hopkins 550 1
etc.

you are to circle the name of the appropriate statistical technique from a list of techniques
given such as:

a. Pearson Product-moment correlation

b. Tetrachoric correlation

c. Biserial correlation

The fourth group was given an objective including a general and
abstract description of the task and the content. The domain was not
limited and an example of the test item was not included. Following is an

example of the course requirements that are neither deﬁned nor concrete:

Know the appropriate statistical techniques.

All four groups read the same text about correlation containing
relevant and incidental content as it pertained to the posttest. All subjects
were asked to highlight the relevant content as they saw it. In addition, to
determine how they were using the objectives, all subjects were asked to
answer a question about why they chose the content they did. Students
were allowed to use written objectives given to them and to write on the

text and make notes on the printed text.

The experimenter measured the time for reading the material for all

subjects. The time was measured by subtracting the time of starting

42

reading the material from the time of asking for exercises.

' After reading the text, subjects were asked to return the written
objectives and text material they used, and to choose one among four types
of sample exercises which could help them to prepare for the posttest.
After choosing an exercise type from the samples, students were asked to
perform a set of exercises like the one they chose. Again, to determine
how they were using the objectives, all subjects were asked to answer a

question about why they selected the exercise they did.

Then all students took a test applying the content read by choosing
the appropriate correlational technique for a given set of data. The total

time for the procedure was estimated to be 30 minutes.

Sample

Fifty-six volunteer graduate students from Michigan State University
were recruited from classes in the College of Education and from personal
contact. Among the 56 subjects 12 were male and 44 were female. Out
of 56 subjects, 46 subjects were from the College of Education. The other
subjects' majors were advertising (2), family and child ecology (1),
management (1), zoology (1), psychology (1), linguistics (1), computer
science (1), and geography (1). One subject didn't indicate a major.
Nineteen subjects were in doctoral programs and 37 subjects were in
master's programs. Regarding native language, 40 subjects used English
as their native language while 16 subjects used English as a second

language. They were randomly assigned to one of the four groups: (a)

 

 

43

deﬁned content and concrete task, (b) deﬁned content, (c) concrete task,

and (d) neither deﬁned content nor concrete task.

The University Committee on Research Involving Human Subjects
(UCHRIS) at Michigan State University reviewed a form for protecting
human subjects and approved the study (Appendix A, B, and C). The
researcher maintained conﬁdentiality throughout the study. Only the
researcher and the researcher's adviser could access the data. During the
whole process of the study, no complaints or procedural problems were

encountered.

Pilot Study

A pilot study was carried out before the main experiment. The
primary purpose of the pilot study was to test the instruments for this study
with a sample of eight graduate students enrolled in Michigan State

University.

During the pilot study subjects were strongly encouraged to give
feedback and suggestions as they progressed through the experiment. The
researcher used the information collected to reﬁne the instrument for this
study. Two major problems were encountered in the pilot study. First,
the researcher found that the readability of the text material caused
confusion for some subjects. As a result, the researcher worked with a
faculty member on the researcher's dissertation committee to reﬁne the
text material. The researcher also found that some subjects ignored

written instructions about the experiment. As a consequence, the

44

researcher included tape-recorded instructions in the main experiment.
The researcher identiﬁed and corrected any problems which were

encountered with the procedure of experiment.

The pilot study was also to determine whether the research questions
were worth asking. Even based on the limited results for eight subjects,

the results seemed promising.

Instrumentation

The instruments used in this study were prequestions, objectives, a

text, exercises, and a posttest.

ﬁgs—“Bit

A posttest of ten questions applying the ideas of correlation was
created along with a description of the task in deﬁned/broad -
concrete/abstract terms (see Appendix K). The posttest was assessed by
the researcher and the researcher's adviser for structure and content
validity against the text. The posttest was scored by giving one point for

each correct answer. The posttest included ten questions such as:

1. Here is some data:

Student GRE scores Socioeconomic Status
(Hi ghzl; Low=0)

Anderson 350 l

McLeod 450 0

 

45

Short 550 1

etc.

circle the name of the appropriate correlation technique from these choices:
a. Pearson product-moment correlation

b. Biserial correlation

c. Tetrachoric correlation

Questions Regarding Prior Knowledge

Before the experiment began, subjects were asked about their
previous experience with the subject matter domain of correlation and with
the use of objectives (see Appendix D). Four questions were asked about
self-rated prior knowledge of correlation, and four questions were asked

about self-rated prior knowledge of objectives.

The four questions about prior knowledge of correlation were about
the subjects' experience in taking a course including correlation,
understanding the meaning of correlation, being able to calculate
correlation, and applying correlation ideas. Subjects responded to the last
three questions on a ﬁve-point Likert scale ranging from very well to
adequately to very poorly. Subjects' responses were scored with 5 points
for very well, 4 points for well, 3 points for adequately, two points for
poorly, and one point for very poorly. Subjects' responses to the last
three questions were added to form a score for prior knowledge of

correlation.

The four questions about prior knowledge of objectives were about

46

the subjects' experience in taking a course about objectives, understanding
the meaning of objectives, being able to write objectives, and applying
ideas about objectives. Subjects' responses were scaled and scored as were

the correlation knowledge questions.

The prequestions were assessed by the researcher and the

researcher's adviser.

grim

The validity of the objectives and their degree of narrowness and
concreteness were ranked by the researcher and the researcher's adviser.

The complete forms of objectives can be found in Appendix E.

Text

Text was paraphrased from two major statistics textbooks (Borg &
Gall, 1983; Glass & Hopkins, 1984) and sentences were assessed as to
relevant and irrelevant with regard to the most explicit objective by the
researcher's adviser and a faculty member on the researcher's dissertation
committee (see Appendix F). The readability of the text was also checked

by the researcher's adviser and the faculty.

The text consisted of eleven paragraphs: six relevant paragraphs and
ﬁve irrelevant paragraphs. Each paragraph was labeled from A through
K. A blank preceded each paragraph for students to check. Subjects
were asked to check the blank if they thought the paragraph contained

47

relevant content.

The content of the text was assessed by the researcher and the
researcher's adviser for relevant and irrelevant against the deﬁned and

concrete objective.

Exercises

Exercises were based on the four objectives. Therefore, four
categories of exercises were made: a deﬁned and concrete exercise, a
deﬁned and po_t concrete exercise, a concrete and pg deﬁned exercise, and
an exercise that was pe_ith£ deﬁned and p9_r concrete (see Appendix J).
Each exercise was accompanied by a separate page with answers to give
subjects feedback. The exercises were assessed by the researcher and the
researcher's adviser for structure and content validity for adherence to the

four objectives.

The deﬁned and concrete exercise was matched with course
requirements; hence, it included consistent conditions, behaviors, and
content to course requirements. The exercise included three questions like
this:

Here is some data that include continuous and artiﬁcial dichotomy variables:

Student GRE Scores SES (I-li gh=1; Low=0)
Hart 350 1
Levin 510 0

Cuerton 470 1

etc.

circle the name of the appropriate correlation technique from these choices:
a. Pearson product-moment correlation

b. Tetrachoric correlation

c. Biserial correlation

The deﬁned and g); concrete exercise was only matched to the
content of course requirements. The exercise included three questions like
this:

A Pearson product-moment correlation is to be used when relating two continuous

variables.

True False

The concrete and n_ot deﬁned exercise was only matched to the
conditions and behaviors of course requirements. The exercise included

three questions like this:

Here is some data:

Student GRE scores Marital status (Marriedzl; Not married=0)
Clark 556 l

Blase 550 0

Walker 460 1

etc.

circle the name of the appropriate statistical technique from these choices:
a. Kendall's tau

b. Rank-difference correlation, rho

49

c. Phi correlation

The neither deﬁned and M concrete exercise was matched with
none of the course requirements. The exercise included three questions
like this:

Kendall's Tau is more likely to be misinterpreted than rho.

True False

Method to Assess tIhe Choice of Relevant Content

After highlighting the relevant content, all students were asked to
answer a questionnaire (Appendix G) that included the following question:

Why did you choose and highlight the content you did as the most relevant?

The highlighting was scored for number of relevant paragraphs
chosen and the number of irrelevant paragraphs chosen by giving one point
for each relevant paragraph and one point for each irrelevant paragraph.
The points were assigned based on the judgement of the researcher and
another rater. The open-ended answers to the question were summarized
by categorizing the reasons for the choices by the researcher and the

researcher's adviser.

Method to Assess the Choice of Exercise

After choosing the exercises, all students were asked to answer a

questionnaire (Appendix I) that includes following question: Why did you

50

select the exercise you did? The reason for the choice of exercise were
summarized by categorizing them for the choices by the researcher and the

researcher's adviser.

The choice of exercise was scored as "best exercise" or " not the best
exercise". The choice of the most deﬁned and concrete exercise was
regarded as the "best exercise", and the choice of deﬁned but do_t_ concrete

exercise, concrete but n_ot deﬁned exercise, and neither deﬁned and nor

 

concrete exercise were regarded as "not the best exercise".

Reliability

The posttest was evaluated for internal reliability using the Cronbach
Alpha formula. The estimate of Cronbach Alpha is the degree to which
the item responses correlate with the total score.(MehrenS & Lehmann,
1984). The internal reliability was .53. Possible reasons for this
moderate reliability are discussed in chapter V.

Data Analysis

The data analysis has two main parts: a statistical analysis and a
qualitative analysis. The statistical analysis has three parts. First, data
from the experiment were analyzed to describe the characteristics of the
sample. Second, data from the experiment were analyzed to answer the
four main research questions. Third, data from the experiment were
analyzed to answer important questions not addressed in the research
questions. For qualitative analysis, the data collected from each subject

51

about his/her reasons for selecting text content and exercises was analyzed
to help explain the quantitative results and to help shed light on the mental
processing subjects use when thinking about objectives. Section one of the
qualitative analysis concerns subjects' selection of content and section two
concerns subjects' selection of exercises. The researcher coded and
analyzed the data. 3

Statistical Analysis

Preliminary analysis

The preliminary analysis was concerned with three variables: prior
knowledge of objectives, prior knowledge of correlation, and posttest
score. For prior knowledge of objectives and correlation, preliminary
analysis was concerned with any preexisting differences among treatments.
For posttest score, the preliminary analysis was concerned with the
inﬂuence of outliers. The preliminary analysis included descriptive data,

ANOVA test results, and correlation data for these three variables.

Main analysis of the research question;

The main analysis includes inferential analyses regarding the four

research questions.

Question one stated: "Will students presented with the deﬁned
objective treatment be able to select the more relevant content than students

not presented with that treatment?" To answer the question, data were

52

analyzed using a 2 x 2 Analysis of Variance (ANOVA), where the number
of relevant sentences chosen by subjects was the dependent variable and the
deﬁned objective treatment (yes versus no) and concrete objective

treatment (yes versus no) were the two independent variables.

Question two stated: "Will students presented with the concrete
objective treatment be better able to select appropriate exercises for
practice than students not presented with that treatment?" To answer the
question, data were analyzed using the chi-square test, for group

differences in the choice of exercise.

Question three stated: "Will students presented with deﬁned and
concrete Objective treatment be able to get a higher the score on the
application posttest than students not presented with that treatment?" To
answer the question, data were analyzed using a two-way of ANOVA to
compare the posttest performance among treatments, where the subjects'
posttest score was the dependent variable and the deﬁned objective
treatment and concrete objective treatment were the two independent
variables. Also question three was analyzed using nonparametric Puri and
Sen ANOVA test (Harwell, 1988). Effect sizes and regression coefﬁcients

were also calculated to assess the size of treatment effects.

Question four stated: "Will students presented with the deﬁned and
concrete objective treatment be able to use less time to read the text than
students not presented with that treatment?" To answer the question, data
were analyzed using a two-way of ANOVA, where the subjects' time for

reading text was the dependent variable and the deﬁned objective treatment

53

and concrete objective treatment were the two independent variables, to

analyze the time used to read the text.

Subsidiary analysis of relgtionships

To relate the results from the AN OVA, the nonparametric,
regression, and correlational analyses the researcher performed a path
analysis. The path analysis aims to verify the theoretical model discussed
in Chapter III.

Chapter IV
FINDINGS

This study investigates the effects of communicating application task
requirements on students' learning processes and achievement. This
chapter presents the data from the experiment and reports the ﬁndings
which are related to the four research questions posed in Chapter I and
elaborated on in Chapter III. The data collected on the sample of the 56
subjects involved in the experiment were statistically analyzed to test the

four questions.

Data Analysis

The data analysis has two main parts: a statistical analysis and a
qualitative analysis.

Statjstical Analysis

The statistical analysis has three parts. First, data from the
experiment were analyzed for the overall characteristics of the sample.
This procedure was to support the main analysis. Second, data from the
experiment were analyzed to answer four main research questions. Third,
data from the experiment were analyzed to answer important questions not

addressed in the research questions.

55
Prelimipag analysis

The preliminary analysis was concerned with three variables: prior
knowledge of objectives, prior knowledge of correlation, and posttest
score. For prior knowledge of objectives and correlation, preliminary
analysis was concerned with any preexisting differences among treatment
groups. For posttest score, the preliminary analysis was concerned with
the inﬂuence of outliers. The preliminary analysis included descriptive

data, ANOVA test results, and correlation data for these three variables.

Prior knowledge of objectives. Self-rated prior knowledge of
objectives ranged between 3 and 15 points out of 15 possible maximum
points. Out of 56 subjects, 41 subjects had taken a course that taught
objectives and 15 subjects had not taken any such course. Subjects who
had taken the course about objectives had a mean score of 12.07 with a
standard deviation of 2.44 and subjects who had never taken any course
about objectives had a mean score of 7.20 with a standard deviation of
3.67. Table 1 summarized the means and standard deviations of prior

knowledge for subjects who had an objective course and who had not.

Table 1. Means and Standard Deviations of Prior Knowledge of

 

Objectives
_M<nn SL3 N
Subject who had taken a course 12.07 2.44 41
about objectives
Subjects who had never taken a 7.20 3.67 15

course about objectives
Total 10.77 3.54 56

 

56

The researcher performed a one-way ANOVA test to compare the
group differences with regard to the prior knowledge of objectives. The
test result showed that there was difference in self-rated knowledge about
objectives between subjects who had taken a course on objectives and
subjects who had never taken a course on objectives, F(1,55) = 32.97,
pr:.000. The results implied that subjects who had taken a course on
objectives regarded their knowledge higher than subjects who had never
taken a course on objectives. Table 2 showed the ANOVA results for the

group differences with regard to the prior knowledge of objectives.

Table 2. One-Way ANOVA Results for Prior Knowledge of

 

Objectives
Variable d_f g ms E p
Taking a course about 1 260.80 260.80 32.97 .000
objectives
Residual 54 427. 18 7.91
Total 55 687.98

 

The researcher performed further preliminary analysis concerning
any preexisting differences among treatments regarding prior knowledge
of objectives. Subjects who were in the defined and concrete objectives
treatment had a mean of 11.93 with a standard deviation of 2.37. Subjects
who were in the deﬁned only treatment had a mean of 10.50 with a
standard deviation of 4.45. Subjects who were in the concrete only
treatment had a mean of 9.21 with a standard deviation of 3.37. Subjects
who were in the neither deﬁned nor concrete treatment had a mean of

11.43 with a standard deviation of 2.98. Table 3 displayed the means and

57

Standard deviations of subjects in treatment groups for prior knowledge of

objectives.

Table 3. Means and Standard Deviations of Prior Knowledge of
Objectives as a Function of Treatment Condition

 

Defined Nondefined YC
Mean SD Mean SD Mean SD
Concrete 11.93 2.37 9.21 3.37 10.57 3.37
N onconcrete 10.50 4.45 11.43 2.98 10.96 3.75
YD 11.21 3.57 10.32 3.51 *10.77 *3.54

 

(YD: means and SD for the deﬁned objective treatment; YC= means and
SD for the concrete objective treatment; * = Grand mean and SD)

The researcher performed a two-way ANOVA test to assess any
preexisting differences on students' prior knowledge of objectives before
treatments. As shown in Table 4, there were no differences on students'
prior knowledge of objectives with regard to both the deﬁned treatments,
F(1,55)=.92, p=.34, and the concrete treatments, F(1,55)=.18, p=.67.
Also, there was no interaction effects at the .05 level of signiﬁcance,
F=3.85, p=.055.

Table 4. Two-Way ANOVA Results for Prior Knowledge of

 

objectives
Variable _d_f_ s_s E E p
Deﬁned 1 11.16 11.16 .92 .34
Concrete 1 2.16 2.16 .18 .67
D * C 1 46.45 46.45 3.85 .055
Residual 52 628.21 12.08
Total 55 687.98

 

58

However, the researcher was concerned about the interaction effect
between the deﬁned objective treatment and the concrete objective
treatment. Because the effect was very close to .05 level of signiﬁcance, it
could not be ignored. It suggested that one of the cell means may be
different from the others. AS a result, the researcher analyzed the
subjects' prior knowledge of objectives with a one-way ANOVA. As
shown in Table 5, there was no difference in self-rated prior knowledge of
objectives among the four groups. In other words, the four treatment
groups were homogeneous regarding the self-rated prior knowledge of

objectives.

Table 5. One-Way ANOVA Results for Prior Knowledge of

 

Objectives
Variable d: s_s @ E p
Treatment 3 59.77 19.92 1.65 . 19
Residual 52 628.21 12.08
Total 55 687 .98

 

Prior knowledge of correlation. Subjects' scores on self ratings on
their prior knowledge of correlation ranged between 3 and 15 points out of
15 possible maximum points. Out of 56 subjects, 35 subjects had taken a
course that taught about correlation and 21 subjects had not. As shown in
Table 6, subjects who had taken a course including correlation had a mean
score of 9.77 with a standard deviation of 2.50 and subjects who had never
taken any course including correlation had a mean score of 5.95 with a

standard deviation of 2.87. Table 6 summarized the results.

59

Table 6. Means and Standard Deviations of Prior Knowledge of

 

Correlation
___M%n .S_D. .1!
Subjects who did take a course 9.77 2.50 35
including correlation
Subjects who did not take a course 5.95 2.87 21
including correlation
Total 8.34 3.22 56

 

The researcher performed a one-way AN OVA test to assess group
differences with regard to prior knowledge of correlation. The test result
showed that there was a difference in self-rated correlation knowledge
between subjects who had taken a course including correlation and subjects
who had never taken a course including correlation, F(1,55): 27.41,

p=.000. Table 7 showed the ANOVA results.

Table 7. One-Way ANOVA Results for Prior Knowledge of

 

Correlation
Variable d_f s_s @ E p
Taking a course about 1 191.43 191.43 27.41 .000
correlation
Residual 54 377. 12 6.98
Total 55 568.55

 

The researcher performed further prelirrrinary analysis concerning
any preexisting differences among treatments regarding prior knowledge
of correlation. Subjects who were in the deﬁned and concrete objectives
treatment had mean of 8.36 with a standard deviation of 3.50; those in the
deﬁned only treatment had mean of 8.14 with a standard deviation of 2.88;
those in the concrete only treatment had mean of 8.36 with a standard

60

deviation of 2.87; and those in neither the deﬁned nor the concrete
treatment had mean of 8.50 with a standard deviation of 3.86. Table 8

summarized the results.

Table 8. Means and Standard Deviations of Prior Knowledge of

 

Correlation
Defined N ondefined YC
Mia-a SI.) M1111 SI) Mean S_D
Concrete 8.36 3.50 8.36 2.87 8.36 3.14
N onconcrete 8.14 2.88 8.50 3.86 8.32 3.35
YD 8.25 3.15 8.43 3.34 *8.34 *3.22

 

(YD: means and SD for the deﬁned objective treatment; YC= means and
SD for the concrete objective treatment; * = Grand mean and SD)

The researcher also used a two-way AN OVA test to analyze any
preexisting differences of students' prior knowledge of correlation. The
results showed that there were no differences of students' prior knowledge

of correlation between treatments. Table 9 Shows the results.

Table 9. Two-Way ANOVA Results for Prior Knowledge of

 

Correlation
Variable d_f_ § ms E p
Deﬁned 1 .45 .45 .04 .84
Concrete 1 .02 .02 .00 .97
D * C 1 .45 .45 .04 .84
Residual 52 567.64 10.91
Total 55 568.55

 

Descriptive information about the ppsttest. As shown in Table 10,
posttest scores ranged between 2 and 10 points. The average score was

7.91 out of 10 possible points with standard deviation of 2.57. As shown

61

in Table 11, out of 56 subjects, 27 had 10 points (48.2%), Six had 9 points
(10.7%), two had 8 points (3.6%), four had 7 points (7.1%), four had 6
points (7.1%), four had 5 points (7.1%), six had 4 points (10.7%), one had
3 points (1.8%), and two had 2 points (3.6%). The distribution of

subjects' posttest scores were negatively skewed.

Table 10. Descriptive Data for the Posttest

 

Mean 7.91 SE Mean .34 Std. Dev 2.57
Variance 6.59 Kurtosis - .70 SE. Kurt .63
Skewness - .85 SE. Skew .32 Range 8.00
Minimum 2.00 Maximum 10.00 Sum 443.00

 

As shown in Table 11, there were two outliers among four
treatments, one in the deﬁned and concrete objective treatment and one in
the deﬁned only objective treatment. An outlier is a research subject
whose scores differ markedly from the general pattern established by other
subjects in the sample. Note the lowest scores in each of those two

categories.

Table 11. Frequency of Posttest Scores by Treatment

 

 

 

 

Score Deﬁnedﬁ& Concrete Deﬁned only Concrete only Neither N
10 10 8 6 3 27
9 2 2 2 6
8 1 1 2
7 1 1 2 4
6 1 2 1 4
5 1 1 1 1 4
4 1* 2 3 6
3 1 1
2 1* 1 2

* Outliers

62

The relationship of the posttest to prior knowledge of objectives an_d
correlation. The researcher considered the possibility of prior knowledge
of objectives and correlation as covariates. The correlation between the
posttest scores and prior objective knowledge scores was -.37 with a
signiﬁcance level of .01. That correlation meant that subjects who had
highly rated their prior knowledge of objectives had low posttest scores.
Because of this negative correlation, the researcher further analyzed the
data by computing correlation between the self-rated prior knowledge of
objectives score and the posttest score for each treatment group. The
correlation between the posttest scores and prior objective knowledge
scores was -.46 for the deﬁned and concrete treatment group, -.33 for the
deﬁned only treatment group, -.72 for the concrete only treatment group,

and -. 18 for the neither deﬁned nor concrete treatment group.

The correlation between posttest and prior correlation knowledge
was .14 and was not signiﬁcant even at .05 level. The correlation between
posttest and prior correlation knowledge was also analyzed by computing a
correlation between the self-rated prior knowledge of correlation score and
the posttest score for each group. The correlation between the posttest
scores and prior correlation knowledge scores was .11 for the deﬁned and
concrete treatment group, -.35 for the deﬁned only treatment group, .12
for the concrete only treatment group, and .51 for the neither deﬁned nor

concrete treatment group.

Because of the negative relationship between posttest score and prior

knowledge of objectives, and low relationship between posttest score and

63

prior knowledge of correlation, two variables were not appropriate

covariates for the analysis of the posttest results.

Table 12. Correlational Matrix for all the Study's Variables

 

Pcor Deﬁn Concr RT RC RI CS EX Post
Pobj -.01 .13 -.05 -.33* -.05 -.12 .03 -.14 -.37**

Poor -.03 .01 .32* .20 .26 .11 .06 .14
Deﬁn .00 .07 .45** -.22 -.22 .35** .29*
Concr .15 .14 —.07 .00 .35** .13
RT .41 ** .35** .18 .20 .29*
RC .21 -.06 .50** .51**
RI .48** -.09 -.22
CS ' -.38** -.40**
EX .51**

 

(Pobj=Self-rated Prior Knowledge of Objective, Pcor=Self-rated Prior
Knowledge of Correlation, DeﬁnzDeﬁned Objective, Concr=Concrete
Objective, RT=Reading Time, RC=Selection of Relevant Content,
RI==Selection of Irrelevant Content, CS=Choice of Sample Exercise,

EX=Exercise Performance, PostzPosttest Score)
** < .01; * < .05

Main analysis of the research guestions

The main analysis includes inferential analyses regarding the four

research questions. Let us consider each question in turn.

Research question one. To answer question one, "Will students
presented with the deﬁned objective treatment be able to select more
relevant content than students not presented with that treatment? ", the
researcher used a two-way ANOVA to determine the differences in

selecting relevant content between treatments.

64

As shown in Table 13, the group with the deﬁned objective treatment
had a mean score of 4.89 out of 6 possible points while the group with the

nondeﬁned objective treatment had a mean score of 3.43.

Table 13. Mean Scores of Relevant Content by Type of

 

Objective
Defined Nondefined YC
Mean SD Mean SQ Mean Si)
Concrete 4.71 1.49 4.07 1.54 4.39 1.52
Nonconcrete 5.07 1.14 2.79 1.53 3.92 1.76
YD 4.89 1.31 3.43 1.64 *4.16 *1.65

 

(YD: means and SD for the deﬁned objective treatment; YC= means and
SD for the concrete objective treatment; * = Grand mean and SD)

The AN OVA test was to determine the effect of the deﬁnition on
students' selection of text content. As noted in Table 14, the highly
signiﬁcant AN OVA result showed that there was a treatment effect of the
deﬁned objective treatment on students' selection of relevant text content.
This implies that subjects were more likely to choose relevant content while
reading the text if they were given a defined rather than a nondeﬁned
objective treatment. However, the effect of deﬁnition appeared only in the
nonconcrete objective condition. As shown in table 13, the effect of
deﬁnition was prominent in the nonconcrete objective condition (5.07 vs.
2.79), but not in the concrete objective condition (4.71 vs. 4.07). The post
hoc test, Studentized Newman-Keuls (SNK) method, showed that the
difference between 5.07 and 2.79 was signiﬁcant at .05 level, but was not
signiﬁcant between 4.71 and 4.07.

65

The ANOVA test also assessed the treatment effect of concreteness
on the choice of relevant content. Similarly, though the ANOVA results
indicated that concreteness did not inﬂuence subjects' choices of relevant
content, there was an effect of concreteness. But the effect appeared only
in the nondeﬁned objective condition. As shown in table 13, the effect of
concreteness was prominent in the nondeﬁned objective condition (4.07 vs.
2.79), but not in the deﬁned objective condition (4.71 vs. 5.07). The SNK
test conﬁrmed the Signiﬁcant difference between 4.07 and 2.79 at .05 level.

Table 14. Two-Way ANOVA Results for the Selection of
Relevant Content

 

Variable d_f s_s pp E p
Deﬁned 1 30.02 30.02 14.58 .00
Concrete 1 3.02 3.02 1.47 .23
D * C 1 9.45 9.45 4.59 .04
Residual 52 107.07 2.06

Total 55 149.55

 

The researcher also checked if the deﬁned objective treatment
inﬂuenced the choice of irrelevant content. AS shown in table 15, the
group with the deﬁned objective treatment had a mean choice of 1.43 with
standard deviation of 1.87 out of 5 possible choices while the group with
the nondeﬁned objective treatment had a mean choice of 2.25 with standard
deviation of 1.82 out of 5 possible choices. For the concreteness effects,
the group with the concrete objective treatment had a mean choice of 1.96
with Standard deviation of 1.82 out of 5 possible choices while the group
with the nonconcrete objective treatment had a mean choice of 1.71 with

standard deviation of 1.96 out of 5 possible choices.

66

Table 15. Mean Scores for Selecting Irrelevant Content by
Types of Objective

 

Defined Nondefined YC
Mean _S_I2 Mean SQ Mean SD
Concrete 1.36 1.60 2.57 1.87 1.96 1.82
Nonconcrete 1.50 2.18 1.93 1.77 1.71 1.96
YD 1.43 1.87 2.25 1.82 *1.84 * 1.88

 

(YD: means and SD for the deﬁned objective treatment; YC= means and
SD for the concrete objective treatment; * = Grand mean and SD)

As shown in Table 16, a two-way ANOVA test was used to test the
group difference on subjects' choice of irrelevant content. The results
indicated that the presence or absence of the deﬁned or the concrete
objective treatment had no effect on students' choice of irrelevant content.
There was no interaction effect on students' choice of irrelevant content,

either.

Table 16. Two-Way ANOVA Results for the Selection of
Irrelevant Content

 

Variable d_f s_s E E p
Deﬁned 1 9.45 9.45 2.71 . 1 1
Concrete 1 .88 0.88 .25 .62
D * C 1 2.16 2.16 .62 .43
Residual 52 181.01 3.48

Total 55 193.55

 

Research q_1_restion two. To answer question two, "Will students
presented with the concrete objective treatment be better able to select
appropriate exercises for practice than students not presented with that
treatment?", the researcher used a chi-square test to determine the

differences between experimental conditions in choosing an exercise type.

67

AS noted in Table 17, the test results showed that there was no difference in
the frequency of choosing the best exercise between those students with or
without the concrete objective treatment. The test results also showed that
there was no difference in the frequency of choosing the best exercise

between those students with or without the deﬁned objective treatment.

Table 17. Frequency and Chi-square Test Results of Selection
of Exercise by Type of Objective

 

Right Wrong
Concrete 16 12
Nonconcrete 16 12
x2 (1) = .00, p: 1.00
Right Wrong
Defined 19 9
Nondefined 13 15

x2 (1) = 2.63, p: .11

Resgch question three. To answer question three, "Will students
presented with the deﬁned and concrete objective treatment be able to get a
higher the score on the application posttest than students not presented with
that treatment?", the researcher used a two-way of AN OVA test to
determine the effect of the deﬁned and the concrete course requirement on

students' posttest achievement.

As shown in Table 18, the subjects receiving the deﬁned objective
treatment (Yd) had a mean posttest score of 8.64 points with a standard
deviation of 2.26 while the subjects receiving the nondeﬁned objective
treatment had a mean posttest score of 7.18 points with a standard deviation

0f 2.68 (N=56). The subjects receiving the concrete objective treatment

68

(Yo) had a mean posttest score of 8.25 points with a standard deviation of

2.46, while the subjects receiving the nonconcrete objective treatment had a

mean posttest score of 7.57 points with standard deviation of 2.67 (N=56).

 

Table 18. Means of Posttest Scores by Treatment with and
without Outliers
Defined Nondefined YC
N ___an 5.1.) 15. _Mwn Si) 1! _Mean SD

Concrete
with all subject 56 8.57 2.56 56 7.93 2.40 56 8.25 2.46
w/o one outlier 55 9.08 1.80 55 7.93 2.40 55 8.48 2.17
w/O two outliers 54 9.08 1.80 54 7.93 2.40 54 8.48 2.17
Nonconcrete
with all subject 56 8.71 2.02 56 6.43 2.82 56 7.57 2.67
w/o one outlier 55 8.71 2.02 55 6.43 2.82 55 7.57 2.67
w/o two outliers 54 9.00 1.78 54 6.43 2.82 54 7.67 2.67

YD
with all subject 56 8.64 2.26 56 7.18 2.68 56 *7.91 *2.57
w/o one outlier 55 8.89 1.89 55 7.18 2.68 55 *8.02 *2.46
w/o two outliers 54 9.04 1.75 54 7.18 2.68 54 *8.07 *2.45

 

(YD: means and SD for the deﬁned objective treatment; YC= means and
SD for the concrete objective treatment; * = Grand mean and SD)

As noted earlier, there were two outlying scores on the posttest.
Without one outlier, the mean posttest score and standard deviation of each
treatment showed as follows: 8.89 with 1.89 for the deﬁned objective
treatment group and 7.18 with 2.68 for the nondeﬁned objective treatment
group; 8.48 with 2.17 for the concrete objective treatment group and 7.57
with 2.67 for the nonconcrete objective treatment group. Without two
outliers, the mean posttest score and standard deviation of each treatment
showed as follows: 9.04 with 1.75 for the deﬁned objective treatment
group and 7.18 with 2.68 for the nondeﬁned objective treatment group;

69

8.48 with 2.17 for the concrete objective treatment group and 7.67 with

2.67 for the nonconcrete objective treatment group.

The researcher performed a two-way ANOVA test including all
subjects (N =56), then the researcher performed the ANOVA without one
outlier (N255), and then without both outliers (N=54) to assess the effect

of treatments without those outliers.

Table 19. Two-Way ANOVA Results for the Posttest Score

 

Variable d1: s_s _m§ E p
Deﬁned
56 1 30.02 30.02 4.93 .03
55 1 40.45 40.45 7.65 .008
54 1 46.64 46.64 9.13 .004
Concrete
56 1 6.45 6.45 1.06 .31
55 1 11.91 11.91 2.25 .14
54 1 8.38 8.38 1.64 .21
D * C
56 1 9.45 9.45 1.55 .22
55 1 4.44 4.44 .84 .36
54 1 6.84 6.84 1.34 .25
Residual
56 52 316.64 6.09
55 51 270.14 5.30
54 50 255.28 5.1 1
Total
56 55 362.56
55 54 326.99
54 53 317.13

 

As described previously, a deﬁned course objective is a statement

deﬁned course objectives inﬂuenced the subj ects' achievement on the

that limits the domain of subject matter clearly. As listed in Table 19, the

70

posttest. Subjects given the deﬁned objective treatment were substantially
more likely to attain higher scores than subjects given the nondeﬁned
objective treatment, F (1,54) = 4.93, p = .03. Without the outliers, the
deﬁned objective treatment showed an even more signiﬁcant result.
Without one outlier, the deﬁned objective treatment produced a positive
effect that was signiﬁcant at less than .01 level, F( 1,53) = 7.65, p =.008.
Without two outliers, the effect was increased to a signiﬁcance of less than
.005 level, F(1,52) =9.13, p=.004.

Because of the ceiling effect on the posttest, the normality
assumption for parametric tests was violated. Note that the distributions
of the posttest scores in three of the four treatment groups were negatively
skewed (see Table 11). Thus, the researcher double checked the
parametric results by using a nonparametric test, the Puri and Sen test,

based on the rank score of the subjects' posttest performances.

As shown in Table 20, the subjects receiving the deﬁned objective
(Yd) had a mean rank posttest score of 33.39 while the subjects receiving

the nondeﬁned objective had a mean rank posttest score of 23.37. The
subjects receiving the concrete objective (Y c) had a mean rank posttest

score of 30.96 while the subjects receiving the nonconcrete objective had a

mean rank posttest score of 26.00.

The Puri and Sen test (Harwell, 1988) has the general form, PS =
(N—l) 0 ~ X 2P9 (1—or), where PS is the Puri and Sen test statistic and Bis a

measure of explained variation, that is, a ratio of the explained variation in

the ranks to the total variation. The results of the Puri and Sen test were

71

compared to a chi-square value at a chosen level of signiﬁcance based on
pq degrees of freedom, where p was the number of dependent variables
and q was the number of independent variables. The Puri and Sen test of
rank factorial ANOVA conﬁrmed the parametric results by showing that
the deﬁned course objectives inﬂuenced the subjects' posttest scores,
PS=5.70 (df=1), p< .05. Table 21 showed the analysis of nonparametric

test results.

Table 20. Mean Ranks of Posttest Scores by Type of Objective

 

Defined Nonconcrete YC
N Moan N Mean N Mean
Concrete
with all subject 56 34.14 56 27.79 56 30.96
w/o one outlier 55 35.65 55 26.79 55 31.06
w/o two outliers 54 34.65 54 25.84 54 30.09
Nonconcrete
with all subject 56 32.64 56 19.43 56 26.00
w/o one outlier 55 31.64 55 18.46 55 25.05
w/o two outliers 54 32.31 54 17.71 54 24.74
YD
with all subject 56 33.39 56 23.37 56 *28.48
w/o one outlier 55 33.57 55 22.63 55 *28.00
w/o two outliers 54 33.48 54 21.79 54 *27.42

 

(YD: means for the deﬁned objective treatment; YC= means for the
concrete objective treatment; * = Grand mean)

The researcher also used the nonparametric ANOVA to analyze the
data without the outliers. The results showed a similar strong effect of the
deﬁned course objectives on subjects' posttest scores. Without one outlier,
the deﬁnition effect was signiﬁcant at less than .01 level, PS: 7.46 (df=1),
p< .01. Without two outliers, the deﬁnition effect was signiﬁcant at less
than .005 level, PS=8.41 (df=1), p< .005.

72

Table 21. Nonparametric ANOVA Results for Posttest Ranks

 

if. ﬁg 55 .53
Deﬁned 1 570* 7.46** 8.41***
Concrete 1 1.45 2.35 1.76
D * C 1 .70 .28 .52

 

(***=<.m5;** =<.01;*=<-05)

In addition to tests of signiﬁcance, the researcher calculated the
magnitude of the effect of the deﬁned objective treatment in standard units.
The effect size estimate, or d—index (Borg & Gall, 1983) is deﬁned as the
difference between the means of two experimental conditions divided by
the standard deviation of the variable. The standard deviation used in this
analysis was pooled standard deviation derived from all subjects. The
statement that d=1.00 indicates that an observed mean difference between
treatment conditions was equivalent to one standard deviation. Therefore,
effect sizes of .2 and .5 would respectively indicate a small and a moderate
effect Size (Borg & Gall, 1983). The effect size of the deﬁned objective
treatment was .5 when all the subjects were included in the analysis.
Without one outlier, the effect size was .6. Without two outliers, the

effect size was .7.

The researcher also employed a regression analysis to assess the
effect that the deﬁned objective treatment had on students' posttest scores.
The effect of the deﬁned objective treatment, the concrete objective
treatment, and the interaction of the treatments were used to explain the

posttest score.

73

These are regression models derived from those variables:

Y = 6.84 + 1.46D + .68C - . 411NTER (N=56)
Y = 6.71 + 1.72D + .93C - . 281NTER (N=55)
Y = 6.78 + 1.86D + .79C - . 36INT ER (N=54)

(D=1 if a subject was in the deﬁned treatment group, D=0 if a subject was
in the nondeﬁned treatment group; C=1 if a subject was in the concrete
treatment group; =0 if a subject was in the nonconcrete treatment group;
INT ER=1 if a subject was in the deﬁned and concrete treatment group or
in the neither deﬁned nor concrete treatment group, INTER=-1 if a subject
was in the deﬁned only treatment group or in the concrete only treatment

amp)

where D was the effect of the deﬁned Objective treatment, C was the effect
of the concrete objective treatment, and INTER was the interaction effect
of the treatments. The munbers in parentheses indicate the number of
subjects used in each analysis.

The deﬁned objective treatment had a regression coefﬁcient of 1.46
with 56 subjects, 1.72 with 55 subjects, and 1.86 with 54 subjects. The
regression coefﬁcients indicate the amount of change in the dependent
variable produced by one unit of change in the independent variable. For
instance, as a student moves from a nondeﬁned objective treatment setting
to a deﬁned objective treatment setting, we would expect an increase in
posttest scores of 1.46 points. Likewise, as a student moves from a
nonconcrete objective treatment setting to a concrete objective treatment
setting, we would expect an increase in posttest scores of .68 points. This
result means that, other things being equal, the presence or absence of a
deﬁned objective treatment could produce a 1.46 point difference for the
posttest score in the case of 56 subjects, a 1.72 point difference for posttest

74

score in the case of 55 subjects, and a 1.86 point difference for posttest
score in the case of 54 subjects. The regression coefﬁcients also indicate
that the effect of the concrete objective treatment was about half as the
effect of the deﬁned objective treatment.

The researcher also checked the 95% conﬁdence interval of the
regression coefﬁcients. The lower and upper boundaries of regression
coefﬁcient for the deﬁned objective treatment are .14 and 2.79, -.64 and
2.00 for the concrete objective treatment, and -1.07 and .25 for the
interaction effect with 56 subjects. The lower and upper boundary of
regression coefﬁcient mean the smallest and highest value for the true
effect of the deﬁned objective treatment respectively. Table 22 shows the
95% conﬁdence interval of the regression coefﬁcients with 56, 55, and 54
subjects. Note that without outliers the lower boundaries of regression

coefﬁcient for the deﬁned objective treatment is getting higher.

Table 22. 95% Conﬁdence Interval of b

 

& i .55
bd 0.14 - 2.79 0.47 - 2.96 0.62 - 3.10
bc — 0.64 - 2.00 - 0.32 - 2.10 0.45 - 2.02
binter - 0.07 — 0.25 - 0.91 - 0.34 - 0.97 - 0.26

 

Research question four. To answer question four, "Will students
presented with the deﬁned and concrete objective treatment be able to use
less time to read the text than students not presented with that treatment?",
the researcher used a two-way ANOVA test to determine the differences in

reading time between experimental conditions.

75

Table 23. Means of Reading Times by Type of Objective

 

Defined Nondefined YC
Mean S_D Mean ED Mean E12
Concrete 9.36 2.47 1 1.57 6.17 10.46 4.75
Nonconcrete 10.71 4.68 8.93 5.73 9.82 5.21
YD 10.04 3.74 10.25 6.00 *10. 14 *4.95

 

(YD: means and SD for the deﬁned objective treatment; YC= means and
SD for the concrete objective treatment; * = Grand mean and SD)

As Shown in Table 23, the subjects receiving the deﬁned objective
treatment (Yd) had a mean reading time of 10.04 minutes with a standard
deviation of 3.74 while the subjects receiving the nondeﬁned objective
treatment had a mean reading time of 10.25 rrrinutes with a standard
deviation of 6.00. The subjects receiving the concrete objective treatment
(YC) had a mean reading time of 10.46 rrrinutes with a standard deviation of
4.75, while the subjects receiving the nonconcrete objective treatment had a

mean reading time of 9.82 with standard deviation of 5.21.

Table 24. Two-Way ANOVA Results for Reading Time

 

Variable d_f s_s m E p
Deﬁned 1 .64 .64 .03 ' .87
Concrete 1 5.79 5.79 .23 .63
D * C 1 56.00 56.00 2.26 .14
Residual 52 1286.43 24.74

a Total 55 1348.86

 

As displayed in Table 24, ANOVA results showed that there were no

treatment effects on subjects' reading time for the text material. The

76

researcher found neither a deﬁned objective treatment effect, F = .03
(p=.87), nor a concrete objective treatment effect, F = .23 (p=.63). There
was no interaction effect either, F = 2.26 (pz. 14).

 

Subsidigy analysis of relationships

To relate the results from the parametric, the nonparametric,
regression, and correlational analyses, the researcher performed a path
analysis. Consider the correlations among all the variables of concern in

the study as noted in Figure 1.

The path diagram in Figure 1 reveals a number of interesting
signiﬁcant direct effects on posttest performance. The deﬁned objective
treatment, choosing relevant content, and the exercise performance, all
showed a positive Si gniﬁcant relationship with the posttest score in the

correlational analysis (see Table 12).

From the preceding analyses it is apparent that subjects who were
given the deﬁned objective treatment were more likely to get higher
posttest scores than subjects who were not given the deﬁned objective
treatment. However, the path diagram showed that the deﬁned objective
treatment affected the posttest performance indirectly. As shown in the
path diagram, the path coefﬁcient between the deﬁned objective treatment
and the choice of relevant content was .45. This coefﬁcient was the same
as correlation coefﬁcient between the deﬁned objective treatment and the
choice of relevant content. Then, the path coefﬁcient between the choice

of relevant content and the exercise performance was .50. This coefﬁcient

77

was also the same as the correlation coefﬁcient between the choice of
relevant content and the exercise performance. However, the path
coefﬁcient between the exercise performance and the posttest was .34, and
the path coefﬁcient between the choice of relevant content and the posttest
was .33. These path coefﬁcients were less than the correlation coefﬁcients

of those two relationships.

The reductions from correlation coefﬁcients to the path coefﬁcients
were due to interrelationships among all variables. In the path analysis,
each successive step counted only the variance which was not counted in the
previous step. Therefore, the decreased coefﬁcient from .51 to .33
between the choice of relevant content and the posttest performance meant
that the deﬁned objective treatment already explained some portion of the
posttest total variance. Likewise, the decreased coefﬁcient from .51 to .34
between the exercise performance and the posttest meant that the deﬁned
objective treatment and the choice of relevant content already explained

some portion of the posttest total variance.

The path diagram also showed that the path coefﬁcient between the
deﬁned objective treatment and the posttest performance was .02. Note
that the correlation coefﬁcient between the deﬁned objective treatment and
the posttest performance was .29. Therefore, the path diagram clearly
showed that the deﬁned objective treatment affects the posttest performance
through the selection of relevant content and exercise performance. All
this means that the deﬁned objective treatment affects the posttest, but does
so indirectly which supports the theoretical model in chapter 111.

 

78

.02 37'
__... Reading Correct

Treatment

(Deﬁned) 4\5‘ ‘50 .34
.35

Exercise

Posttest

Figure 4-1. A Path Analysis of the Effects of Defined
Objective on Students' Posttest Achievement

Qualitative Analysis

The researcher collected data from each subject about their reasons
for selecting text content and exercises. Subjects' reasons for their
decisions may help explain the quantitative results and may Shed light on
the mental processing subjects use when thinking about objectives. The
ﬁrst section of the qualitative analysis concerns subjects' selection of

content and the second section concerns subj ects' selection of exercises.
Analysis of reasonagiven for selection of text content

After reading the text and choosing relevant content, subjects were
asked to write their answer to the question, "Why did you choose and
highlight the content you did as the most relevant?" In analyzing subjects'
written reasons for selection of text content, the researcher was looking for
ways that subjects used of the kind of objective that they were given in each
treatment. However, the researcher was equally interested in reasons

other than objectives for their selections.

79

The researcher and researcher's adviser checked subjects' written
remarks to see whether they commented or related their choice of relevant
text content to objectives. The subjects' written comments were classiﬁed
according to whether they used words or phrases that denoted objectives.
In an interrater agreement, both raters stated that 10 out of 14 subjects in
the deﬁned and concrete objective treatment group gave reasons that
related to objectives for their choice of relevant content. In other
treatment groups with objectives, both raters reported results that were
comparable to the most complete treatment: 8 out of 14 in the deﬁned only
objective treatment group, and 10 out of 14 in the concrete only objective
treatment group. Note that according to both raters only 5 out of 14
subjects in the group with the neither deﬁned nor concrete objective
treatment group related their choice of relevant content to objectives.
Overall, the interrater agreement of the classiﬁcations of reasons given for
all treatments was 86.8%. Table 25 shows the interrater agreement among

treatments.

Table 25. Interrater Agreement on Subjects' Use of Objectives
in Selecting Relevant Content by Treatment

 

Raterl Rater’z A gree A greement

Deﬁned and concrete 10 10 10 100%
Deﬁned only 10 8 8 80%
Concrete only 10 10 10 100%
Neither 8 5 5 62.5%
Total 38 33 33 86.8%

 

Subjects in all treatments used similar words or phrases in

mentioning objectives as the reason for selecting relevant content. Some

80

subjects used words with direct reference such as "objective", "instructional
objective", "instructions", and "introduction". Some subjects used phrases
that denoted the contents of the objectives such as "3 [statistical] tests we
would be tested on." Some subjects took phrases right out of the
objectives such as "would be used when", taken from "On the test you are
to know when to use correlational techniques." Some subjects referred to
the purpose of the objective with phrases such as " [needed the content for
the] question given later. " Table 26 summarizes the the words and phrases
given by subjects for choosing the contents and includes the frequency for
each response category. The importance of words and phrases used by
subjects can be best understood by referring to Appendix L which states the

objectives given in each treatment.

Table 26. Frequency of Words and Phrases Used by Subjects to
Select Content

 

Deﬁned & Concrete Deﬁned only Concrete only Neither

Objective* 7 8 8 4

Instructions 2 0 2 0

Those

(three) to 1 2 0 1

be tested

When to use 0 0 0 3
N 10/ 14 10/14 10/ 14 8/14

 

(*Includes "instructional objective")

Most subjects in this study were certain as to why they chose content
as relevant. When subjects referred to objectives as their reason for
selection, they used similar terminology. However, subjects Showed
marked differences in the way they used objectives in making their choices

of content. In other words, some subjects commented on the use of

81

objectives in selecting relevant content, but they failed to use them
appropriately, that is, in limiting content. Nine out of ten subjects in the
deﬁned and concrete objective treatment group who used objectives in
selecting relevant content commented on how the objective helped them see
the limits of the relevant content. In comparison, six out of eight subjects
in the deﬁned only objective treatment group and four out of ten subjects
in the concrete only objective treatment group who used objectives in
selecting relevant content made similar comments. Note that no subjects in
the neither deﬁned nor concrete objective treatment group who used
objectives in selecting relevant content commented on how the objectives
helped limit the content. Table 27 summarizes the number of subjects who
commented on using objectives to see the limits of the text content. While
subjects in the neither deﬁned nor concrete objective treatment group may
have known how to use objectives to limit content, they did not because the
objective was so general it gave little information as to the limits of the

content.

Table 27. Number of Subjects Using Objectives to Limit
Content by Treatment

 

_N_ Commenting on objective Limit content

Deﬁned and Concrete 14 10 9
Deﬁned only 14 8 6
Concrete only 14 10 4
Neither 14 5 0

 

Those subjects who did not use objectives in their selection of content
provided other reasons. Subjects often made comments that related to

their prior knowledge of correlation. Some said they selected content as

82

relevant because the content they chose were frequently used correlation
techniques. Some stated that their selections were based on what they
already knew were important correlation techniques. Thus, some subjects
seemed to rely on their prior experience to select relevant content rather
than use the objectives they were given for that purpose. Table 28
summarizes reasons given by subjects who did not report using objectives

in their selection of content .

Table 28. Abbreviated Reasons Given by Subjects not Using
Objectives to Select Content

 

Deﬁned and Concrete
Key word in the test.
Main deﬁnition and examples.
Subject of correlation between two variables.
Comparing two variables.
Deﬁned only
Frequency of usage.
Pearson is important, and the other could be looked in most
statistical books anytime.
Chose content because ed. researchers should learn all those kinds.
Highlighted areas that I felt were uniquely signiﬁcant to
understanding each coefﬁcient.
Concrete only
Most widely used correlation techniques.
Concept and application.
Most commonly used formula.
Terminology.
m
It related to correlation technique used in various data.
They are related to the content about correlation.
Because they are important conelations.
Core concept to understand the correlation.
The content gave basic information about correlation.

 

83

Analysis of reasons given for selecting an exercise

 

After stating their reasons for choosing relevant content and
choosing an exercise to practice for the test, subjects were asked to write
their answer to the question, "Why did you select the exercise you did?"

In analyzing the subjects' written reasons for their selection of an exercise,
in a fashion similar to the preceding analysis on content, the researcher was
looking for ways that subjects used of the kind of objective that they were
given in each treatment. Again, the researcher was equally interested in

reasons other than objectives for the subjects' selections.

AS before, two raters checked the subjects' written remarks to see if
they related their selection of an exercise to objectives. The subjects'
written comments were classiﬁed according to whether they used words or
phrases that denoted objectives. In an interrater agreement, both raters
stated that four out of fourteen subjects in the deﬁned and concrete
objective treatment group gave reasons that related to objectives for their
choice of exercise. In other treatment groups with objectives both raters
reported results that were comparable to the most complete treatment:
four out of fourteen in the deﬁned only objective treatment group, three
out of fourteen in the concrete only objective treatment group, and three
out of fourteen in the neither deﬁned nor concrete objective treatment
group. Overall, the interrater agreement of the classiﬁcations of reasons
given for all treatments was 74%. Table 29 shows the ratings and

interrater agreement for each treatment.

84

Table 29. Interrater Agreement on Subjects' Use of Objectives
in Selecting an Exercise by Treatment

 

Treatment Raterl Rate12 A gree A greement
Deﬁned and concrete 5 4 4 80%
Deﬁned only 6 5 4 57%
Concrete only 4 4 3 60%
Neither 3 3 3 100%
Total 17 17 14 70%

 

The numbers of subjects referring to objectives as the reason for
selecting an exercise was relatively low and quite similar in all groups. In
stating their reasons for selecting an exercise, subjects who referred to
objectives used words such as "[instructional] objectives", "criteria [for
selection]", "[general instructions]", " [example of] posttest", and "[three

choices in the] ﬁrst paper."

Table 30. Frequency of Words and Phrases Used by Subjects to
Select Exercises

 

Q+_C Q Q Neither
Objective 2 3 2 3
(General) instruction 0 0 2 0
Example of posttest 1 0 0 0
Criteria 0 l 0 0
Three choices in the ﬁrst paper 1 0 0 0
Correct answer was present 0 1 0 0
It tested what I am supposed to be tested 1 0 0 0
The 3 tests that should be found is included 0 1 0 0
on the ﬁnal test
Example of both Pearson correlation and 0 1 0 0
artiﬁcial dichotomous
N 5 7 4 3

 

Subjects also used phrases that denoted objectives such as "Correct
answer was present", "It tested what I am supposed to be tested", "The 3

85

tests that should be found are included on the ﬁnal test", and "Example of
both Pearson correlation and artiﬁcial dichotomous." Table 30
summarizes the words and phrases given by subjects for choosing the

exercise and includes the frequency for each response category.

Many subjects used criteria other than objectives in their selection of
an exercise. Subjects often made comments that related to reading
material. Some said they selected the exercise as relevant because the
exercise they chose included examples shown in the text they read. Some
stated that their selections were based on familiarity of examples. Thus,
some subjects seemed to rely on their memory of examples shown in the
text to select a relevant exercise, rather than use the objectives they were
given for that purpose. Table 31 summarizes reasons given by subjects

who did not report using objectives in their selection of exercise.

Note that fewer subjects gave reasons that related to objectives in
selecting an exercise than in selecting relevant text content. Perhaps
students were making use of the text as well as objectives to make their
decisions on the choice of an exercise for practice. Those subjects using
objectives to make their selection of an exercise did so two ways. Some
subjects were trying to match the exercises to the examples they saw in the
objective and text. Other subjects were trying to match the content they
recalled from the objective with the content they noticed in the exercise
sample. Many subjects referred to the examples given in the objective and

the text when stating reasons for selecting an exercise.

86

Table 31. Abbreviated Reasons Given by Subjects not Using
Objectives to Select Exercises

 

Deﬁned and Concrete
Looked a little more interesting.
The question will help me recall deﬁnition of the concept.
Like seeing the example.
Most familiar to the way I had set up my study pattern.
I knew the answer.
It doesn't require to remember too many details from the text.
Most familiar with rank (Spearman) order vs. nonrank (Pearson).
The answers ﬁt the material I had studied, in the form I studied it.
It showed the ranking of each country.
Deﬁned only
The data looked familiar to what I had been reading.
The more information I had available the better I could/would do.
While reading the text, I analyzed the data to be able to distinguish
between the techniques.
Because I can apply my understanding.
The one I select is described in the content.
Expected some tests that may deal with (mental) recall without
engaging calculations.
Concrete only
The term was familiar to me.
I remember the most information about the exercise I selected.
It is appropriate to apply the understanding of variables.
Selected exercise seemed to provide a little bit more information.
Chose the one I can study more.
Easiest to identify scores/rank.
The format allows me to determine based on the information which
technique would be applicable given the type of data.
It was the reﬂecting best the text read.
The true-false format would give me enough information to recall
the little bit of knowledge I had gathered.
The most comfortably with continuous score (at least one). The
three selections were (a,b,c) most familiar.
Neither
Basically, it was just a matter of choosing one at random. I'd do
about the same on any of the tests.
I thought I understand the material.
Familiar with this kind of data analysis.
Remember a little something on ranks.
The one I feel I understand relatively.

87

Table 31 (cont'd).

It makes you practice when to use a speciﬁc kind of correlation
technique.

I like the type.

I can't select the correct answer better than when data/information is
given rather than selecting T or F without an example.

It related to correlational techniques.

It had familiar looking data arranged in a format that is used
regularly in my work related reading.

I remember same thing about rank.

 

8mm and Integration of the Results

This study investigated the effects of communicating application task
requirements using objectives on students' learning processes and

achievement when learning from text.

Before the experiment, the researcher was concerned that the
subjects' prior knowledge of objectives and the subjects' prior knowledge
of correlation would affect the posttest performance and might confound
the treatment effects on the posttest score. Corresponding to the
researcher's concern, the results showed that subjects who had taken a
course about objectives rated their knowledge higher than subjects who had
never taken a course about objectives. The results also indicated that
subjects who had taken a course including correlation rated their
knowledge of correlation higher than subjects who had never taken a
course including correlation. However, the correlation between self -rated
knowledge of Objectives and correlation and the posttest score was negative

(p-.37) and low (. 14) respectively. These results could be interpreted as

88

meaning that the subjects who felt that they knew more about objectives
scored lower on the posttest than subjects who felt that they knew less about
objectives, and that subjects who felt they knew more about correlation
scored on the posttest about the same as subjects who felt they knew less
about correlation. The results also indicated that treatment groups were
homogeneous with regard to self-rated prior knowledge of objectives and
correlation. Therefore, self-rated prior knowledge of objectives and
correlation were eliminated from consideration as covariates. Preliminary

analysis also showed two outliers with respect to posttest performance.

The test results of research question one, about the effect of the
deﬁnition on selecting relevant content, showed that there was a deﬁnition
effect. But the deﬁnition effect appeared only in the nonconcrete objective
condition. Similarly, there was a concreteness effect, but the concreteness

effect appeared only in the nondeﬁned objective condition.

The test results of research question two, about the effect of
concreteness on selecting the appropriate exercise, showed that there was
no concreteness effect, no deﬁnition effect, and no interaction effect. The
results indicated that there was no difference in the frequency of choosing
the best exercise between the subjects with or without the concrete

objective treatment.

The test results of research question three, about the effect of the
deﬁnition and the concreteness on posttest performance, showed that there
was a deﬁnition effect, but not a concreteness effect nor an interaction

effect. The results indicated that the deﬁned objective treatment

89

inﬂuenced the subjects' achievement on the posttest. Subjects given
objectives which deﬁned the subject domain were substantially more likely
to attain higher scores than subjects given objectives which did not deﬁne
the subject domain as precisely. Nonparametric analysis results also
conﬁrmed the parametric results of the deﬁned objective treatment effect.
The deﬁnition effect was stronger without outliers. The effects size of
deﬁned objective treatment was .5 when all the subjects were included in
the analysis. Without one outlier, the effects size was .6. Without two
outliers, the effects size was .7. The regression coefﬁcients of the deﬁned
objective treatment added more detailed features of the effect. The
deﬁned objective treatment had a regression coefﬁcient of 1.46 with 56
subjects, 1.72 with 55 subjects, and 1.86 with 54 subjects. The regression
coefﬁcients indicated the amount of change in the dependent variable
produced by one unit of change in the independent variable. For instance,
a change of one unit in the deﬁned variable would produce 1.46, 1.72, and
1.86 points higher than in posttest score with 56, 55, and 54 subjects
respectively. Therefore, other things being equal, subjects who received
objectives which deﬁned the subject domain got 1.46, 1.72, and 1.86 points
more than subjects given objectives which did not deﬁne the subject domain

as precisely with 56, 55, 54 subjects respectively.

The test results of the research question four, about the effect of the
deﬁned and the concrete objective treatment on reading time, showed that
there was no deﬁnition effect, no concreteness effect, and no interaction
effect. The results indicated that those subjects' who received objectives
which deﬁned the subject domain were not as efﬁcient as those subjects

who were given objectives which did not deﬁne the subject domain as

90

precisely. The results also indicated that those subjects' who received
objectives with a concrete example were not as efﬁcient as those subjects

who were given objectives which did not have a concrete example, either.

The path analysis showed that there is an indirect link in the path
from the deﬁned objective treatment to the posttest through choosing
relevant content and performing well on the exercise. This demonstrated
the effect of the deﬁned Objective treatment on the process of learning and

on students' achievement.

Subjects in eaCh condition showed similarity regarding frequency in
using objectives for selecting relevant content. However, their use of
objectives in limiting content showed marked differences. Subjects' with
the more deﬁned objective treatment were more likely to be able to use the
objectives as a guide to limit text content. No subjects in the neither
deﬁned nor concrete objective treatment group used objectives as a guide
to limit text content. Subjects used concrete examples to limit content

when they had no deﬁned content.

Subjects in all treatment groups made relatively less use of objectives
in selecting an exercise to prepare for the posttest as compared to their use
of objectives as a guide for selecting relevant content. Overall, subjects in
all treatment groups used objectives to select an exercise approximately
equally. For the selection of an exercise, subjects depended on the

examples in the text as well as the example in the objectives.

Chapter V
DISCUSSION AND CONCLUSIONS

The purpose of this chapter is to present a discussion of the ﬁndings
reported in chapter IV and to relate them to the research questions. This
chapter also contains the limitations and implications of the study, and

recommendations for future research.

Discussion of the Findings

In reading each section of the discussion, keep in mind that the main
research question of this study was: Does the presentation of application
task objectives, which deﬁne a subject domain and concretely exemplify a
required test, affect students' learning processes and resulting achievement
when learning from text? The two subsidiary research questions were:
(a) Does the presentation of application task objectives which deﬁne a
subject domain help students to know what to attend to in their assigned
reading?, and (b) does the presentation of application task objectives which
concretely exemplify a required test help students to know which exercise

to select when given a choice?

Prior Knowledge of Objectives

Before the experiment, to assess inﬂuence on the results, the
researcher asked subjects to rate their knowledge of objectives. Note that
self-rating was used instead of a pretest to avoid teaching subjects about

objectives. The results showed that the correlation between self-rated

91

92

knowledge of Objectives and the posttest score was negative (r=-.37). This
result meant that the subjects who felt that they knew more about objectives
scored lower on the posttest than subjects who stated they did not know as

much.

One reason for this relationship may be due to the ceiling effect on
the posttest. The plot between prior knowledge of Objectives and the
posttest showed that the relationship would have been zero if there was no
ceiling effect. But suppose the effect is not just a measurement artifact.
Perhaps this effect came about because subjects who knew how to use
objectives were frustrated and confused when less than perfect objectives
were given to them. Consequently they did poorly on the posttest. In
support of this explanation the researcher noticed that some subjects made
remarks about their frustration when stating reasons their use of objectives.
Some subjects commented the lack of guidance in objectives such as "I was

totally confused".

Note that this result contradicts studies that showed that subjects
trained in the use of objectives make more effective use of objectives
(Duchastel, 1977). Although this result is inconsistent with past research,
it raises the question about what happens when students encounter
instructional support that does not conform to what they expect. Teachers
may encounter this condition when moving students from a dependent
instructional mode to a more independent mode such as from a master's

level class to a doctoral one.

93

Prior Knowledge of Correlation

Just as with the knowledge of objectives, to assess inﬂuence on the
results, the researcher asked subjects to rate their knowledge of
correlation. Here also a pretest may have alerted students to the particular
content to be learned and may have inadvertently taught them. The results
showed that the correlation between self-rated correlation knowledge and
the posttest score was low (r=.14). This result could be interpreted as
meaning that subjects who felt they knew more about correlation scored on
the posttest the same as subjects who felt they knew less. This could mean
that the instructional intervention accounted for the effect on the posttest.
This result also shows that although prior knowledge is a factor in learning
and achievement, it must combine with other factors, such as instructional

support or instructional materials to produce achievement.

Selection of Relevant Content

Does the presentation of application task objectives which de_ﬁ_n_e a
subject domain help students to know what to attend to in their assigned
reading? The results showed that those subjects who received objectives
which deﬁned the subject domain were more likely to choose relevant
content than those subjects who were given objectives which did 991 deﬁne
the subject domain as precisely. However, note that subjects who received
objectives with a concrete example were not as likely to choose relevant
content as those subjects who were given objectives which did rat have a

concrete example. This supports the theoretical notion that the deﬁned

94

objective treatment would have a stronger effect on choosing relevant

content.

However, the effect of deﬁnition appeared only in the nonconcrete
objective condition. Recall that the mean score on relevant content for the
nonconcrete objective treatment with deﬁnition was 5.07 compared to 2.79
for the nonconcrete Objective treatment without deﬁnition. In contrast,
the mean scores in the concrete objective conditions with and without
deﬁnition were not much different: 4.71 and 4.07. The post hoc test,
Studentized Newman-Keuls (S NK) method, showed that the difference
between 5.07 and 2.79 was signiﬁcant at the .05 level, but the difference
between 4.71 and 4.07 was not. Similarly, though the ANOVA results
indicated that concreteness did not inﬂuence subjects' choices of relevant

content, there was a simple main effect of concreteness in the nondeﬁned
objective condition (4.07 vs. 2.79) conﬁrmed by the SNK test at .05 level.

These results may be interpreted to mean that objectives which limit
the subject matter domain are useful cognitive tools to help students direct
their attention and make decisions when students have no clear idea about
the speciﬁc appearance of the test. Perhaps the limitations of content help
students focus their processing and help ease the load on short term
memory. But students must believe that the test will truly be limited to the
domain speciﬁed by the objective to act appropriately. These results also
mean that objectives which concretely describe a test are useful cognitive
tools to help students direct their attention and make decisions when
students are not given a limited subject matter domain. Thus, to help

students attend to the most relevant content in a reading assignment,

95

teachers may specify either the limits of the content or the exact nature of

the test.

The results about choosing relevant content may be tempered
somewhat by noting that the correlation between choosing relevant content
and choosing irrelevant content was relatively low (r=.21). Because the
correlation was not negative, it may be said that subjects who chose
relevant content also chose some irrelevant content. Subjects who selected
the relevant content correctly were discriminating, but not always
discriminating very precisely. Some subjects may have gone beyond the
deﬁned limits described by objectives for other reasons. This result, along
with the reasons supplied by students for their choices, demonstrated that
subjects use their own strategies to choose content as well as those that were
suggested by instructors. Thus, objectives did not necessarily hinder
subjects from looking at and possibly learning incidental content as some
researchers contended (Duchastel & Brown, 1974; Frase & Kreitzberg,
1975; Rothkopf & Billington, 1975, 1979). This supports Kaplan and
Rothkopfs notion (1972,1974) that objectives increase both relevant and
incidental learning. This result should allay the fears of teachers who
believe that giving students objectives will narrow their attention. On the
contrary, students are not like computers, they have their own agendas and

strategies which supplement the strategies provided by teachers.

Why did subjects choose other content beside that which was relevant
to the objective? There could be many reasons. First, the subjects in this
study may not be very good at choosing relevant content; they may not be

very efﬁcient learners. But this reason is not convincing because the

96

subjects were graduate students and were probably efﬁcient learners.
Second, based on their classroom experience with teachers stating
objectives which have no relation to the test, subjects may not believe that
the objectives describe the test and therefore ignore them (Melton, 1978).
Hamilton (1985) supported this reason when he suggested that students'
experiences with objectives are a signiﬁcant factor in affecting students'
learning. Third, subjects may realize the limitations of describing a test
through an objective and thus go beyond what the objective states to cover
what they interpret as the important aspects of the subject. Fourth,
subjects may be inﬂuenced by the context of the research. Several subjects
commented on the importance of the concept of correlation in a general
research setting. Perhaps subjects may have felt compelled to cover more
about correlation than asked for in the objectives. Fifth, subjects may
simply have a drive and a curiosity which inﬂuences them to go beyond the
immediate objective. If students are so highly motivated, they may
achieve the objectives regardless of whether or not test requirements are
speciﬁed (Melton, 1978). On the other hand, perhaps because this was

" just an experiment," students were not motivated to focus as accurately as

they would if the test counted for a grade.

The ﬁndings and the possible reasons seem to imply that, as Melton
suggested, it is not sufﬁcient to simply provide students with behavioral
objectives. Students must also understand and trust in the relationship
between the objective and the test, and they must use the objective
purposely to prepare for the test. Thus, teachers must ﬁrst show students
that the objectives describe the test, explain how to use the objective as a

tool, and then follow through by providing a valid test.

Selection of Exercise

Does the presentation of application task objectives which concretely
describe and exemplify a required test help students to know which
exercise to select when given a choice? The result showed that there was
no difference in the frequency of choosing the best exercise between the

subjects with or without the concrete Objective treatment.

One puzzling ﬁnding was that subjects who selected more irrelevant
content were more likely to choose the right exercise. To solve this
puzzle, note that subjects were given only a single example in the concrete
objective treatment which may have satisﬁed and possibly misled them.
Meanwhile, subjects who were given objectives without an example might
have been motivated to check various examples in the text to clarify the
task. In support of this solution the researcher observed that about half of
the remarks given by subjects who chose the right exercise implied that
they relied on examples in the text material rather than on the Objectives.
The reason for this may be due to a recent memory effect, where

information processed most recently affected the students' use of examples.

Posttest Performance

Does the presentation of application task objectives which deﬁne a
subject domain and exemplify a required test help students to get a higher
score on the application posttest? The results showed that the deﬁned
course objective treatment substantially inﬂuenced the subjects'

98

achievement on the posttest. Subjects given objectives which deﬁned the
subject domain were more likely to attain higher scores than subjects given

objectives which did not deﬁne the subject domain as precisely.

Although the effect of the deﬁned objectives was clear in the
subjects' posttest performance, it was even clearer without outliers. The
effect Size of deﬁned objectives was .5 when all the subjects were included
in the analysis. Without one outlier, the effect size was .6. Without two
outliers, the effect size was .7. The effect sizes, d=.5, .6, and .7 indicate
that observed mean differences between the deﬁned objective treatment and
the nondeﬁned objective treatment were equivalent to .5, .6, and .7
standard deviations. In other words, the deﬁned objective treatment group
got scores one half of a standard deviation higher than the nondeﬁned
objective treatment group. Furthermore, the regression coefﬁcients of the
deﬁned objective treatment Showed that for those students in that treatment
compared to those without that treatment, we would expect an increase in
posttest scores of 1.46, 1.72, and 1.86 points for 56, 55, and 54 subjects
respectively. On the ten item test, this means that students affected by the
deﬁned objective treatment got one and a half points or more higher than
others, a substantial impact. This would be like getting 15 to 18 points

more on a one hundred point test.

One ﬁnding from the preliminary analysis on the posttest score was
the distribution of the scores. The posttest score was negatively skewed.
That means that many subjects had scores that were close to the maximum
possible score. Since the distribution of the posttest score did not justify

the assumption of normality, nonparametric tests were performed to verify

99

the parametric test results. The results showed similar outcomes to the
parametric test results. The negatively skewed distribution of the posttest
score also implied a ceiling effect on the score. A ceiling effect means that
the range of difﬁculty of the test items was limited. Therefore, the test
score might not measure the entire range of achievement possible on the
posttest performance affected by the treatments. The ceiling effect may
also inﬂuence the reliability of the posttest. Since the reliability measured
the degree to which the item responses correlate with the total test score,
the reliability of the posttest might be relatively low due to the ceiling
effect. Thus, the ceiling effect placed an artiﬁcial restriction on the
distribution of gain scores across treatments. In other words, the estimate
of effect of the deﬁned objective treatment was conservative due to a
ceiling effect. The defined objective treatment probably would have
produced even greater achievement than the nondeﬁned objective treatment
had the text covered a wider content area. Thus, if the posttest included
more items and wider range of difﬁculty by expanding the content of the
text material, subjects' posttest scores between treatments may have shown

greater differences than the present ﬁndings.

While the deﬁned objective treatment inﬂuenced posttest
performance of an application task learned from text, the concrete
objective treatment did not. The concrete objective treatment was
supposed to help students to decide on the exercise which best matched the
sample presented in the objectives, so they would practice the desired
performance under appropriate test conditions, and therefore succeed on
the test itself. But objectives with a concrete example of the test did not

help students to select the best exercise. It may be that students used the

100

example primarily to note the content domain represented and the example
duplicated the help provided by the limitations of content already provided.
The simple effect of the concrete objective treatment on the selection of
relevant content supported the notion that students were probably using the
example to deﬁne the domain. Perhaps students need to learn how to use
the example to match exercise conditions and behaviors. Perhaps one
example is not enough. Perhaps the example was not concrete enough to
be effective. Teachers and researchers may therefore consider
experimenting by providing a number of concrete examples to show the
test format and by explicitly Showing students the relation of these

examples to the test.

Reading time

Does the presentation of application task objectives which deﬁne a
subject domain and concretely exemplify a required test help students to
use less time to read the text? It was thought that if students could
selectively attend to only that content needed and those conditions and
behaviors desired they should save time in their reading. The results
showed that neither the deﬁned nor the concrete objective treatment made
any difference in reading time. In fact, subjects without the deﬁned
objective treatment who spent more time reading were as good at choosing
relevant content as those given the deﬁned objective treatment. It seems
like unguided students may ﬁnd relevant content accidently because they
select and read everything. However, students given a deﬁned objective
treatment, may spend as much time reading to be absolutely sure they have

found what they need to meet the objective. Thus, the deﬁned objective

101

treatment may not help students be any more efﬁcient in terms of reading
time. But, given the same reading time as those without the deﬁned
objective treatment, students with the deﬁned objective treatment do score

higher on the posttest. So, in that sense, Objectives foster efﬁciency.

Path Analysis

The results from the correlation analysis, the parametric analysis, the
nonparametric analysis, and the regression analysis were used in a path
analysis to verify the theoretical model proposed in chapter III. The path
diagram revealed a number of interesting signiﬁcant effects on posttest
performance. From the ANOVA, nonparametric, and regression analyses,
it is apparent that subjects who were given the deﬁned objective treatment
were more likely to get higher posttest scores than subjects who were not
given the deﬁned objective treatment. The analysis showed a strong effect
for the deﬁned objective treatment on the posttest score. However, the
path analysis showed that the deﬁned objective treatment effect was not
direct. As proposed in theoretical model, the effect was indirect through
choosing relevant content and exercise performance. Thus, the path
diagram supported the model proposed in this study. This result suggests
that teachers should provide deﬁned objectives, text with content relevant
to the task, and appropriate exercises to produce posttest achievement.
Some students, on their own, will choose the content and use the exercise to
achieve. Other may have to be taught how to use the objectives to choose

content and exercise.

102

Limitations

There were two types of limitations to this study: threats to internal
validity and threats to external validity:

The internal validity of an experiment is the extent to which
extraneous variables have been controlled by the researcher (Campbell &
Stanley, 1963). This study has four possible threats to internal validity.

1. Subjects in this study had treatments under slightly different
environmental conditions. They either participated individually or in
groups up to ﬁve, each receiving his or her own treatment. Thus, some
subjects may have been inﬂuenced to stop reading sooner or to continue

reading because of the actions of others.

2. The researcher may have inadvertently inﬂuenced subjects'
thinking by asking for reasons for their choice after they ﬁnished their
reading and after they selected an exercise. For example, when subjects
were asked to justify their choices of content, they may have been alerted
to the notion that they needed a rationale for their choices. The rationale
may have reminded them of the objectives. If they had read irrelevant as
well as relevant content, being reminded of the objectives may have helped

them focus an relevant content and retrieve only that content.

3. The reliability of the posttest may have affected the strength of
the treatment effects. Note, that in spite of the moderate reliability, clear

effects were present. If the posttest did not have a ceiling effect, the

103

reliability would have been greater and perhaps the treatment effects even
more pronounced. For a criterion referenced task, such as this, it may

have been more appropriate to use another kind of estimate of reliability.

4. The researcher administered the experiment and coded the data,
and the qualitative data were analyzed by the researcher and researcher's
adviser. These procedures might have caused experimenter bias. For
those who would like to check the results of the qualitative measures, the
subjects' verbal responses are recorded in Appendix L and M.

External validity is the extent to which the ﬁndings of an experiment
can be applied to particular settings (Campbell & Stanley, 1963). This
study has three possible threats to external validity.

1. One must be cautious about generalizing the results of this study
due to the nature of the subjects of this study: a volunteer sample of
graduate students at one university. Graduate students may learn from
text in ways unlike undergraduates, high school, or elementary school

students

2. Generalization may also be limited by the environmental
conditions of this study. This experiment was an isolated learning
situation outside of a regular class. Perhaps learners would learn

application tasks from text differently in the context of a real class.

3. The moderate reliability of the posttest may affect the
generalization of ﬁndings in this study. It may be that other subjects

104

taking this posttest may get a somewhat diminished or increased score

simply due to test characteristics.

Conclusions

Keeping in mind the limitations, the following four conclusions are

supported by the ﬁndings.

1. Students presented with speciﬁcally deﬁned objectives were able
to select relevant content for an application task to be learned from text.
Speciﬁcally deﬁned objectives help students most when there is no concrete
example of the test. In contrast, in general, students presented with the
concrete objective treatment were not any more able to select relevant
content from text for an application task than students without them.
However, when students were not presented with limits of the content to be
learned, concrete examples of a test helped students to select relevant
content. In addition, objectives did not restrict or inhibit students from
learning content incidental to that prescribed by the objectives. Though
students thought about objectives when selecting relevant content, the
objectives were not the only criteria they used to study. Subjects also used
their own criteria. Therefore, it is recommended that to serve as a guide
for students' selection of content from text for an application task, teachers
Should include in their objectives statements which deﬁne the limits of the
content to be tested. If the content cannot be deﬁned, then teachers should
include a concrete example of the test. Also teachers should not be
surprised when students use their own criteria to attend to content

irrelevant to the objectives.

105

2. Students presented with the concrete objective treatment were no
different than students presented without the concrete objective treatment
in selecting the right exercise for preparing for a posttest of an application
task learned from text. Therefore, it cannot be recommended on the basis
of this study that teachers guide students in selecting the best practice for an
application task learned from text by providing only a single example of

the test task in the objective.

3. Students' posttest performance on an application task learned
from text was positively and substantially affected by the deﬁned objective
treatment. Although the reliability of the posttest was moderate, the
results clearly showed a substantial effect of deﬁnition. Therefore, it is
recommended that to affect the number of posttest items answered
correctly for an application task learned from text, teachers should include
in their objectives a deﬁnition of the limits of content to be tested. The
concrete objective treatment did not produce a Si gniﬁcant effect on
students' achievement. Perhaps this is so because more than one example
is needed to help students' selection of the best practice for a test.
However, the concreteness showed its potential effectiveness by acting as a

deﬁner when the content was not deﬁned.

4. Students presented with the deﬁned objective treatment were not
different in their reading time than students without them. Neither were
the students presented with the concrete objective treatment. Students
seemed to think about objectives especially when selecting relevant content.

But, the objectives were not the only criteria they used to study. Subjects

106

used their own criteria in addition to objectives. Therefore, It is
recommended that teachers ﬁnd tools and techniques other than objectives

to inﬂuence reading time.

Implications of the Study for Instructional Design

Two unique aspects of this study were investigating application
objectives and students' thinking as they used objectives while learning.

Each aspect has implications for instructional design.

Application Objectives

This study showed that objectives with certain characteristics could
be as effective in inﬂuencing posttest results on an application task as well
as inﬂuencing posttest results on recall tasks as reported in previous studies
(Duchastel, 1980; Ho, Savenye, & Haas, 1986; Kaplan & Rothkopf, 1974;
Rothkopf & Kaplan, 1972). But teachers need to state objectives
differently for application tasks than recall tasks. The ﬁndings showed
that teachers should consider including a clear deﬁnition of the content
domain in application objectives, and they should experiment with concrete

examples of test items in the objectives.

Some educators may be concerned about "teaching for the test",
implying that the objective will just assist recall and that incidental learning
will be neglected. As just mentioned, this study showed that telling
students what the objective is will not merely produce recall, it will aid in

learning of application tasks. However, in addition, this study

107

demonstrated that giving students objectives will not restrict their learning.

They will continue to learn incidental content.

S_tu_dents' Use of Objectives

Researchers on objectives have said that selective attention plays an
important role in the way students use objectives (Reynold & Anderson,
1982; Reynold & Shirey, 1988; Wittrock, 1986). However, they failed to
show what selective attention means and how students use it. This study
showed that students use more than one criterion to direct their attention.
They use objectives, but they also use text examples and their own prior

experience to select content.

The ﬁndings of this study imply that teachers should consider
providing deﬁned and concrete objectives, relevant text, and proper
exercises for learning application tasks from text. Then students will be
able to use the objectives as in this study to properly process information.
However, it is probably not sufﬁcient to simply provide students with
objectives. They must be instructed to use them efﬁciently.

Design Implications

Providing objectives, relevant text content, and exercises can help
teachers plan for students' leanring in elementary school, high school, and
college. Instructional designers may also create computer-aided-
instruction (CAI) and individual modules using the same design features as

in a typical text assignment.

108

To produce achievement of application tasks when learning from
text, teachers can match the experimental protocol which is similar to
typical school assignments. For example, suppose an elementary school
teacher wanted to have students learn from text how to interpret a map.
According to the results of this study, the teacher should assign text which
states how to interpret maps and give exercises which match deﬁned and
concrete objectives like this one: Given a map such as the one on page 45
of the text, you will state the distance between any two given points such as
"What is the distance between Oscoda, MI and Detroit, MI." Also suppose
a high school teacher wanted to have students learn from text how to
identify impressionistic art. According to the results of this study, the
high school teacher Should assign text which states how to identify
impressionistic art and give exercises which match deﬁned and concrete
objectives like this one: Given a painting not seen in the text such Van
Gogh's Starry night, you will state the characteristics of that painting which
are characteristic of impressionistic style. Also suppose a college
professor wanted to have students learn from text how to predict the cost
of a product using supply and demand. The college professor should
assign text which states how to predict the cost of a product using supply
and demand and give exercises which match deﬁned and concrete
objectives like this one: Given data such as that on page 56 of the text that
show estimated demand and supply of world petroleum in 1993, you will

predict the cost of the petroleum in 1993.

109

Recommendations for Future Study

There are eight recommendations for future studies:

1. Expand the content of the text material to be learned for an
application task to produce a greater effect of the deﬁned objective
treatment. The present study was planned to take about 30 nrinutes of time
in consideration of the subjects' possible fatigue. As a result, the
researcher limited the length of text material and posttest items. These
limitations might have resulted in a limited treatment effect as well as
limited differentiation among treatment groups.

2. Replicate the study with a larger number of posttest items.
Increasing the number of posttest items may be desirable to improve the
reliability of the posttest as well as to measure the treatment effect more
precisely.

3. Replicate the study with different levels of students. It should
reveal whether these results are a characteristic of only graduate students
or whether they apply to elementary, junior high, senior high, and
undergraduate students as well.

4. Include more examples as well as nonexamples of the test task in
the concrete objective treatment. Additional varied examples are likely to
produce a treatment effect for concreteness.

5. Conduct the research in an actual classroom setting to produce the
natural behaviors of learners.

6. Conduct the research with other application tasks with more
complex content. Consider other analysis, synthesis, and problem solving

tasks.

110

7. Conduct research that repeats the treatment with variation in
content.

8. Extend the type of instruction to other methods than text.
Perhaps a video lecture or an audio tape can be used as the method of

instruction.

LIST OF REFERENCES

Anderson, J. (1980). Cognitive psychology and its implications. San
Francisco: Freeman.

Anderson, R. (1982). Allocation of attention during reading. In A.
Flammer & W. Kintsch (Eds), Discourse processing (pp. 292-305).
New York: North-Holland.

Anderson, R., & Faust, G. (1974). Educational psychology: The science
of instruction and learning. New York: Dodd, Mead.

Barker, D., & Hapkiewicz, W. (1979). The effects of behavioral Objectives
on relevant and incidental learning at two levels of Bloom's taxonomy.
Journal of Educational Research, 72, 334—339.

Boker, J. (1974). Immediate and delayed retention effects of interspersing
questions in written instructional passages. Journal of Educational
Psychology, 66, 96-98.

Borg, W., Gall, M. (1983). Educational researcher: An introduction (4th
ed.). New York: Longman.

Brown, A. (1980). Metacognitive development and reading. In J. Spiro,
B. Bruce, & W. Brewer (Eds), Theoretical issues in reading
comprehension (pp. 453-481). Hillsdale, N.J.: Erlbaum.

Brown, A., Bransford, J ., Ferrara, R., & Campione, J. (1984). Learning,
remembering, and understanding. In J. Flavell & E. Markman (Eds),
Handbook of child psychology (Vol. III, pp.77-166). New York:
Wiley.

Brown, A., Campione, J., & Day, J. (1981). Learning to learn: On
training students to learn from texts. Educational researcher, 10, 14-21.

Brown, A., & Day, J. (1983). Macrorules for summarizing texts: The
development of expertise. Journal of Verbal Learning and Verbal
Behavior, 22, 1-14.

Brown, A., & Smiley, S. (1978). The development of strategies for
Studying texts. Child Development, 49, 1076-1088.

111

 

112

Calfee, R., & Drum, P. (1986). Research on teaching reading. In M.
Wittrock (Ed.), Handbook of research on teaching (3rd ed., pp. 804—
849). New York: Macmillan.

Campbell, T., & Stanley, J. (1963). Experimental and quasi-experimental
designs for research on teaching. In N. Gage (Ed), Handbook of
research on teaching (pp. 171-246). Chicago: Rand McNally.

Collins, A. (1977). Processes in acquiring knowledge. In R. Anderson, R.
Spiro, & W. Montague (Eds), Schooling and the acquisition of
knowledge (pp. 339-363). Hillsdale, N.J.: Lawrence Erlbaum.

Collins, A., & Stevens, A. (1983). A cognitive theory of inquiry learning.
In C. Reigeluth (Ed.), Instructional-design theories and models: An
overview of their current status (pp. 247-278). Hillsdale, N.J.:
Erlbaum.

Dalis, G. (1970). Effect of precise objectives upon student achievement in
health education. The Journal of Experimental Education, 39, 20-23.

Davis, R., Alexander, L., & Yelon, S. (1974). Learning system design:
An approach to the improvement of instruction. New York: McGraw-

Hill.

Duchastel, P. (1980). Learning objectives and the organization of prose.
Journal of Educational Psychology, 72, 300-309.

Duchastel, P. (1977). Functions of instructional objectives: Organization
and direction. Great Britain: the Open University. (ERIC
Documentation Reproduction Service No. ED 135 774)

Duchastel, P., & Brown, B. (1974). Incidental and relevant learning with
instructional objectives. Journal of Educational Psychology. 66, 481-485

Duell, O. (1974). Effect of type of objective, level of test question, and
judged importance of test material upon posttest performance. Journal

of Educational Psychology, 66, 225-232.

Felker, D., & Dapra, R. (1975). Effects of question type and question
placement on problem solving ability from prose material. Journal of
Educational Psychology, 67, 380-384.

113

Frase, L. (1968). Effect of question location, pacing, and mode upon
retention of prose material. Journal of Educational Psychology, 59,
244—249.

Frase, L., & Kreitzberg, V. (1975). Effects of topical and indirect
learning directions on prose recall. Journal of Educational Psychology,
67, 320-324.

Glass, G., & Hopkins, K. (1984). Statistical methods in education and
psychology (2nd ed.). Englewood Cliffs, N.J.: Prentice Hall.

Glover, J ., Plake, B., & Zimmer, J. (1982). Distinctiveness of encoding
and memory for learning tasks. Journal of Educational Psychology, 74,
189-198.

Guthrie, J. (1988). Locating information in documents: Examination of a
cognitive model. Reading Research Quarterly, 23, 178-199.

Guthrie, J., & Kirsch, I. (1987). Distinctions between reading
comprehension and locating information in text. Journal of Educational
Psychology, 79, 220-227.

Hagen, J ., & Hale, G. (1973). The development of attention in children.
In A. Pick (Ed), Minnesota symposia on child psychology (Vol. 7, pp.
1 17-140).

Hamaker, C. (1986). The effects of adjunct questions on prose learning.
Review of Educational Research, 56, 212-242.

Hamilton, R. (1985). A framework for the evaluation of the effectiveness
of adjunct questions and objectives. Review of Educational Research,

55, 47-85.

Hannaﬁn, M. (1987). The effects of orienting activities, cueing, and
practice on learning of computer-based instruction. Journal of
Educational Research, 81, 48-53.

Harwell, M. (1988). Choosing between parametric and nonparametric
tests. Unpublished manuscript.

Hedrick, C. (1976). Learning production systems from examples.
Artiﬁcial Intelligence, 7, 21-49.

114

Ho, C., Savenye, W., & Haas, N. (1986). The effects of orienting
objectives and review on learning form interactive video. Journal of

Computer-Based Instruction, 13, 126-129.

Just, M., & Carpenter, P. (1987). Ihe psychology of reading and language
comprehension. Newton, MA: Allyn and Bacon.

Kaplan, R. (1974). Effects of learning prose with part versus whole

presentations of instructional objectives. Journal of Educational
Psychology, 66, 787-792.

Kaplan, R., & Rothkopf, E. (1974). Instructional objectives as directions
to learners: Effects of passage length and amount of objective-relevant
content. Journal of Educational Psychology, 66, 448-456.

Kaplan, R., & Simmons, F. (1974). Effects of instructional objectives used
as orienting stimuli or as summary/review upon prose learning. Journal
of Educational Psychology, 66, 614-622.

Kauer, K. (1984). Intentional and incidental learning with instructional
texts: A meta-analysis for 1970-1980. American Educational Research
Journal, 21, 323-339.

Kibler, R., & Bassett, R. (1977). Writing performance objectives. In L.

Briggs (Ed), Instructional design: Principles and applications (pp. 49-
95). Englewood Cliffs, N.J.: Educational Technology Publications.

Lewis, J. (1981). Answers to twenty questions on behavioral objectives.
Educational Technology, 21, 27-31.

Lyon, M., & Gettinger, M. (1985). Differences in student performance on
knowledge, comprehension, and application tasks. Journal of
Educational Psychology, 77, 12-19.

Mager, R. (1962). Preparing instructional objectives. Palo Alto, CA:
Fearon.

Mayer, R. (1984). Aids to text comprehension. Educational Psychologist,
19, 30-42.

Mehrens, W., & Lehmann, I. (1984). Measurement and evaluation in
education and psychology (3rd ed.). New York: Holt, Rinehart, and
Winston.

115

Melton, R. (1978). Resolution of conﬂicting claims concerning the effects
of behavioral objectives on student learning. Review of Educational
Research, 48, 291-302.

Miller, B, & Weiss, M. (1981). Children's attention allocation,
understanding of attention, and performance on the incidental learning
task. Child Development, 52, 1183-1190.

Nitch, K. (1977). Structuring decontextualized forms of knowledge.
Unpublished doctoral dissertation, Vanderbilt University.

Palincsar, A., & Brown, A. (1984). Reciprocal teaching of
comprehension-fostering and comprehension-monitoring activities.

Cognition and Instruction, 1 , 117-175.

Peterson, P., Swing, S., Braverrnann, M., & Buss,R. (1982). Students'
aptitudes and their report of cognitive processes during direct
instruction. Journal of Educational Psychology, 74, 535-547.

Peterson, P., Swing, S., Stark, K., & Waas, G. (1984). Students'
cognitions and time on task during mathematics instruction. American
Educational Research Journal, 21, 487-515.

Pressley, M., Goodchild, F., Fleet, J ., Zajchowski, R., & Evans, E. (1989).
The challenges of classroom strategy instruction. The Elementary
School Journal, 89, 301- 342.

Reynolds, R., & Anderson, R. (1982). Inﬂuence of questions on the
allocation of attention during reading. Journal of Educational
Psychology, 74, 623-632.

Reynolds, R., & Shirey, L. (1988). The role of attention in studying and
learning. In C. Weinstein, E. Goetz, & P. Alexander (Eds), Learning
and study strategies: Issues in assessment, instruction, and evaluation
(pp. 77-100). New York: Academic press.

Reynolds, R., Standiford, S., & Anderson, R. (1979). Distribution of

reading time when questions are asked about a restricted category of text
information. Journal of Educational Psychology, 71, 183-190.

Rosch, E. (1978). Principles of categorization. In E. Rosch, & B. Lloyd
(Eds.), Cognition and categorization (pp. 9-31). Hillsdale, N.J.:

116

Erlbaum.

Rothkopf, E. (1965). Some theoretical and experimental approaches to
problems in written instruction. In J. Krumboltz (Ed), Learning and
the educational process (pp. 193-221). Chicago: Rand McNally.

Rothkopf, E., & Billington, M. (1979). Goal-guided learning from text:
Inferring a descriptive processing model from inspection times and eye
movements. Journal of Educational Psychology, 71, 310-327.

Rothkopf, E., & Billington, M. (1975). Relevance and similarity of text
elements to descriptions of learning goals. Journal of Educational
Psychology, 67, 745-750.

Rothkopf, E., & Kaplan, R. (1972). Exploration of the effects of density
and speciﬁcity of instructional objectives on learning from text. Journal
of Educational Psychology, 63, 295-302.

Rutter, M. (1989). Attention deﬁcit disorder/Hyperkinetic syndrome:
Conceptual and research issues regarding diagnosis and classiﬁcation. In
T. Sagroden, & T. Archer (Eds.), Attention deﬁcit disorder: Clinical
and basic research (pp. 1-24). Hillsdale, N .J .: Erlbaum.

Shirey, L., & Reynolds, R. (1988). Effect of interest on attention and
learning. Journal of Educational Psychology, 80, 159-166.

Siegel, I., & Saunders, R. (1979). An inquiry into inquiry: Question
asking as an instructional model. In L. Katz (Ed), Current topics in
early childhood education (pp. 169-193). Norwood, N.J.: Ablex.

Stein, B. (1978). Depth of processing reexamined: The effects of the

precision of encoding and text appropriateness. Journal of Verbal
Learning and Verbal Behavior, 17, 707-714.

Tennyson, R., & Park, 0. (1980). The teaching of concepts: A review of
instructional design literature. Review of Educational Research, 50, 55-
70.

Tennyson, R., Cocchiarella, M. (1986). An empirically based instructional
design theory for teaching concepts. Review of Educational Research,
56, 40-71.

VanLehn, K. (1985). Learning one subprocedure per lesson. Unpublished

117

manuscript.

Watts, F., & Anderson, R. (1971). Effects of three types of inserted
questions on learning from prose. Journal of Educational Psychology,
62, 387-394.

Weinstein, C., & Mayer, R. (1986). The teaching of learning strategies.
In M. Wittrock (Ed), Handbook of research on teaching (3rd ed, pp.
315-327). New York: Macmillan.

Wittrock, M. (1986). Students' thought processes. In M. Wittrock (Ed),
Handbook of research on teaching (3rd ed, pp. 297-314). New York:
Macmillan.

Yelon, S. (1991). Writing and using instructional objectives. In L. Briggs,
K. Gustafson, & M. Tilman (Eds). Instructional design: Principles and
applications (2nd ed, pp. 75-121). Englewood Cliffs, N .J .: Educational
Technology Publications.

APPENDICES

118

APPENDIX A

Invitation to Volunteer to Participate

Hello:

Want a chance to help with some research on instructional psychology, and
learn more about learning skills and statistical knowledge? How? Well,
more details will be provided if you decide to participate, but the basic idea
involves taking something like a short course. Approximately 30 minutes
will be needed.

To indicate your interest in participating, please print your name and
telephone number on this sheet and return it to me. Even if you
volunteer, you will be free to decline to participate if you happen to change

your mind. Thank you for your consideration,

Jun Young Shin

Graduate Student
Yes, I would like to be considered:
Name
Phone Number
If you have any questions or concerns, contact Jun Young Shin (Tel. 355-
3012).

 

 

119

APPENDIX B

Informed Consent Form

Prior to signing this form, please read all of the following paragraphs.
The paragraphs are numbered to make it easier to ask questions about
speciﬁc points.

1. This research in which you would participate is being conducted to
fulfill one of the requirements of Jun Young Shin's doctoral program at
Michigan State University. The purpose of the research is to study the
effects of communicating course requirements on students' learning
processes and achievement. There will be four types of course
requirements. You will be randomly assigned into one of them and
you will be asked to read a course requirement and text material, and to
complete an exercise and posttest. Participation in this study will
require approximately 30 minutes of your time. The time can be at
your convenience - any time, any day that is open.

2. Signing this form expresses your agreement both that the research has
been explained to you and you understand the explanation. Please note
that this research involves all the risks usually associated with taking a
test including both the risk of stress while answering the questions and
the risk that one's performance may not meet one's expectations.

3. Please note that no beneﬁcial effects are guaranteed for participating this
research.

4. Please note that, even if you agree to participate, you may end your
participation at any time without fear of reprisal.

5. All results of this study will be treated with strict conﬁdence regarding
the identity of any participant. Individual participant will be able to
obtain their own results if they wish.

6. Signing this form expresses your agreement that the results of your
posttest will be shared with the researcher in order to give an
understanding of your learning.

7. Signing this form will indicate that you have freely consented to
participate in the research referred to in the preceding paragraphs. If
you so agree, please sign and date this form in the space provided below

Participant's Signature Date

 

 

120

APPENDIX C

Confidentiality

The following procedures will be used to protect conﬁdentiality
which will be assured to each potential subject. Only the researcher and
trained raters will have access to the posttest results and questions used in
the study. Each posttest and question sheet will be numbered and will not
contain the name of the subject.All responses sheets will be transformed to
number codes for the purpose of computer analysis. Information from the
study will be treated in aggregate and it will not be possible to identify any
individual taking part in the study or the responses that they have made.
The names of the individual subjects or collection sites will not be used in

any reports of the results of this research.

 

121

APPENDIX D

Prequestions

Please answer these questions by checking the appropriate space.
1. Have you ever taken any course that taught you about correlation?

Yes No

If yes, which course?

 

2. How well do you understand the meaning of correlation?

 

 

 

 

 

very well well adequately poorly very poorly

3. How well can you calculate correlation?

 

 

 

 

 

very well well adequately poorly very poorly

4. How well can you use correlation ideas?

 

 

 

 

 

very well well adequately poorly very poorly

5. Have you ever taken any course that taught you about instructional
objectives?

Yes No

If yes, which course?

 

6. How well do you understand the meaning of instructional objectives?

 

 

 

 

 

very well well adequately poorly very poorly

122

7. How well can you derive and write instructional objectives?

 

 

 

 

 

very well well adequately poorly very poorly

8. How well can you apply ideas about instructional objectives?

 

 

 

 

 

very well well adequately poorly very poorly

9. I am in the following sort of program.

 

BA MA ED.S. Ph.D.

 

 

Subject

 

123

APPENDIX E

Instructional Objective

Read These Immrtant Instructions:

1. Today you will read and learn about correlation.

2. You will be given a p_<_>sttest after reading some text and doing a practice
exercise. The posttest will assess your attainment of the instructional
objective.

3. The instructional objective below describe the msttest you will get after
reading and practice.

4. The instructional objective describes what you are to learn from the text
you will be given to read.

5. Consider the instructional objective careful. so
a. you can direct your attention to the most relevant content,
b_.a_nd you ca_n choose the best approach to practice.

6. You do not have to read all the content - just read what you need to do
well on the posttest. Read selectively.

Instructional Objective (Test Description)
Read very carefully.

1. On the test you will be given data regarding only combinations of
continuous and artiﬁcial dichotomy variables, such as:

 

 

Student SAT Scores ﬂgebra Test (Success=1; Failure=0)
Borg 350 0
Smith 450 1
etc.

2. On the test you will be given gmy these choices:
a. Pearson product-moment correlation
b. Biserial correlation
c. Tetrachoric correlation
3. On the test you are to circle the name of the appropriate correlation
technique.

On the next page you will ﬁnd the text relating to the test you will take.

 

124

Read These Important Instructions:

1. Today you will read and learn about correlation.

2. You will be given a ppsttest after reading some text and doing a practice
exercise. The posttest will assess your attainment of the instructional
objective.

3. The instructional objective below describe the ppsttest you will get after
reading and practice.

4. The instructional objective describes what you are to learn from the text
you will be given to read.

5. Consider the instructional objective careful, so
a. you can direct your attention to the most relevant content

b. and you can choose the best approach to practice.

 

 

6. You do not have to read all the content - just read what you need to do
well on the posttest. Read selectively.

Instructional Objective (Test Description)
Read very carefully

I. On the test you will be given data regarding only combinations of
continuous and artiﬁcial dichotomy variables.

2. On the test you will be given only these choices:
a. Pearson product-moment correlation
b. Biserial correlation

c. Tetrachoric correlation

3. On the test you are to circle the name of the appropriate correlation
technique.

On the next page you will ﬁnd the text relating to the test you will take.

125

Read These Impprtant Instructions:

1. Today you will read and learn about correlation.

2. You will be given a msttest after reading some text and doing a practice
exercise. The posttest will assess your attainment of the instructional
objective.

3. The instructional objective below describe the msttest you will get
after reading and practice.

4. The instructional objective describes what you are to learn from the text
you will be given to read.

5. Consider the instructional objective careful, so

a. you can direct your attention to the most relevant content,

b. and you can choose the best approach to practice.
6. You do not have to read all the content - just read what you need to do

well on the posttest. Read selectively.
Instructional Objective (Test Description)
Read very carefully

1. On the test you will be given some data, such as:

 

 

Student SAT Score College GPA
Zimmerman 350 3.7
Spalding 450 3.2
etc.

2. On the test you will be given a list of techniques given such as

a. Pearson product-moment correlation
b. Biserial correlation
c. Tetrachoric correlation

3. On the test you are to circle the name of the appropriate statistical
technique.

On the next page you will ﬁnd the text relating to the test you will take.

126

Read These Immrtant Instructions:

1. Today you will read and learn about correlation.

2. You will be given a ppsttest after reading some text and doing a practice
exercise. The posttest will assess your attainment of the instructional
objective.

3. The instructional objective below describe the msttest you will get
after reading and practice.

4. The instructional objective describes what you are to learn from the text
you will be given to read.

5. Consider the instructional objective careful so
a. you can direct your attention to the most relevant content
b. and you can choose the best approach to practice.

6. You do not have to read all the content - just read what you need to do
well on the posttest. Read selectively.

 

 

Instructional Objective (Test Description)
Read very carefully

I. On the test you are to know when to use correlational techniques.

On the next page you will ﬁnd the text relating to the test you will take.

127

APPENDIX F
Text
Instructions:

1. Read the following text to prepare yourself to take a posttest described
by the objective.

2. You are not responsible for reading all paragraphs.

3. Direct your attention to the content you feel is most important to help
you prepare for the posttest.

4. Please check using the marker given to you only paragraphs that are
related to the instructional objective. For example, if you feel that
following paragraph is important content, check the left top blank as
follows:

X A. Correlational techniques are frequently used in statistical analyses.

5. Take time as much as possible until you master the important content.
6. The posttest will be closed book test.

Correlation Techniques

_ A. In this section we discuss seven correlational techniques that can
be used to analyze the degree of relationship between two variables. The
purpose of the correlation coefﬁcient is to express in mathematical terms
the degree of relationship between any two variables. A coefﬁcient of
correlation is a statistical summary of the degree and direction of
relationship between two variables. A perfectly consistent relationship is
expressed as 1.0. The form of the variables to be correlated and the
nature of the relationship determine which technique is used. Variables in
relationship studies are usually expressed in one of four forms: continuous,
rank, artiﬁcial dichotomy, and true dichotomy.

_ B. Continuous scores are values of a variable that has an indeﬁnite
number of points along its continuum. For example, these data,

 

Student Stanford-Binet Intelligence Test
Fuson 142

Gelman 137
Harel 126

BIC.

128

would be continuous scores.

_ C. A rank score expresses the position of a person or object on a
variable, relative to the positions held by other persons or objects. For
example, these data,

Countg S limit Ranks
Italy 87mph 1
France 81mph 2
U. S. 65mph 3
etc.

would be rank scores.

_ D. The term dichotomy refers to a variable that has only two values.
An artiﬁcial dichotomy results when individuals are placed into two
categories on the basis of difference on a continuous variable. For
example, these data,

Student Socioeconomic Status (Highzl; Low=02
Inhelder 1
Zucker 0
Rochel 0
etc.

would be artiﬁcial dichotomy scores. When individuals are divided into
two groups on the basis of a variable that can have only two values, the
dichotomy is referred to as a true dichotomy. For example, these data,

 

Student Marital Status (Marriedgl: Not MarrieEQ)
Sawyer 1
Taylor 0
Stanley 1
etc.

would be true dichotomy scores.
Pearson Product-Moment Correlation, r
_ E. The Pearson product-moment coefﬁcient, r, is used when both

variables that we wish to correlate are expressed as continuous scores.
For example, if we administer an intelligence test such as the Stanford-

129

Binet test and an achievement test such as the CI‘ BS Achievement Test to
the same group of individuals, we will have two sets of continuous scores,
each individual having a score on each of the two tests. The data may look
like this:

Student Stanford-Binet CI‘ BS Achievement Test
Glass 120 560
Grant 135 470
Allison 127 640
etc.

The relationship between these two sets of scores would be expressed by a
product-moment coefﬁcient of .35. Because most educational measures
yield continuous scores, this is the most frequently used correlational
technique. The product-moment correlation has a smaller standard error
than the other correlational techniques and is generally preferred when
appropriate.

Rank-Difference Correlation, rho

_ F. The rank-difference correlation, rho, is a special form of the
product-moment correlation. The rank-difference correlation is used to
correlate two variables when one or both of these variables are available
only in rank form. For example, studies correlating speed limit with
trafﬁc fatalities over the world would generally employ the rank-
difference correlation because each country's standing is expressed as a
rank. To use this correlational technique, however, both variables must be
expressed as a rank, so in this case the speed limit of each country and
fatalities, which are available in the form of continuous scores, would have
to be converted to ranks before the correlation could be calculated.
Converting continuous scores to rank scores involves the simple procedure
of listing the continuous scores in order of magnitude and then assigning
ranks. The data may look like this:

Country Smed Limit (X) Fatalities (_Y_) _)_(_ _Y_
Italy 87 mph 6.4 1 3
France 81 mph 8.0 2 2
USA. 65 mph 3.3 3 4
Spain 62 mph 12.4 4 1

etc.

130

Kendall's tau

_ G. Tau is another form of rank correlation that has some theoretical
advantage over the better known Spearman's rho. Like rho, tau is used to
correlate two sets of ranks. The data may look similar to that used in the
explanation of rho. Data not in rank form can be converted to ranks if
one desires to use tau. Its principal advantage is that it has a more normal
sampling distribution than rho for a sample under ten. It is more difﬁcult
to calculate than rho and yields lower correlation coefﬁcients when

computed from the same data.
The Biserial Correlation s.

_ H. The biserial correlation is used when one of the variables is in the
form of continuous scores and the other variable is in the form of an %
artiﬁcial dichotomy. For example, if we wish to determine the
relationship between success and failure in algebra course and scores on an
algebra aptitude test, we would use the biserial correlation. The data may
look like this:

 

Student Algebra Aptitude Test Score _Spccess of Algebra Course
(Successzl; Failure=0)

Lewis 350 0

Morgan 470 1

Morse 510 1
etc.

In this case the aptitude test yields continuous scores, while the record of
each subject as having passed or failed algebra takes the form of an
artiﬁcial dichotomy. As a rule, the correlation coefﬁcients obtained using
the biserial technique are somewhat higher than those obtained on the same
data using the product-moment technique.

The Point Biserial Correlation

_ I. The point biserial correlation is used when one of the variables we
wish to correlate is in the form of a continuous score and the other variable
is in the form of a true dichotomy. This type of correlation is used in
studies relating gender to different continuous variables, such as
intelligence, verbal ﬂuency, reading ability, and achievement. In such
studies gender provides the true dichotomy, and the other measure
provides the continuous variable. The data may look like this:

131

 

 

Student SAT Score Gender (Malezl: Female=0)
Adams 620 0
Berk 420 1
Jackson 550 1
etc.

The Tetrachoric Correlation

__ J. Occasionally we encounter a situation in educational research
where both variables that we wish to correlate are in the form of artiﬁcial
dichotomies. Under this conditions the tetrachoric correlation statistic is
used. Use of this coefﬁcient requires the assumption that the variables
underlying the dichotomies in the tetrachoric correlation analysis are
continuous and normally distributed. Also, the tetrachoric coefﬁcient is
considerably less stable than the product-moment coefﬁcient. The data
may look like this:

Student Socio-Economic Status Success of Algebra Course

(Hi ghzl; Low=0) (Successzl; Failure=0)
Bennett 1 0
Duke 1 0
Cooper 0 1

etc.
The Phi Coefﬁcient

_ K. The phi coefﬁcient is used to correlate two variables that are both
true dichotomies. Because we deal with relatively few true dichotomies in
education, phi coefﬁcients are seldom calculated in educational research.
The main use of this technique is to determine the correlation between two
items on a test during item analysis. Each subject's response to each item
can be classiﬁed as either correct or incorrect, thus giving two true
dichotomies. The data may look like this:

 

Student Marital Status my Out of College
(Marriedzl; Not Married=0) (Dropped Out=1; Remained=0)
Davis 0 1
Eder 0 0
Metz l 1
etc.

When you are ﬁnished reading. raiLse your h_and and ask for an exercise

132

APPENDIX G

Questions

For the Selection of Text Content

Why did you choose and highlight the content you did as the most relevant?

133

APPENDIX H

Sample of Test Items

Instruction: Choose the type of test item that will best prepare you for the
test. There are four exercise types. Read them and tell the proctor which
exercise you wish to use.

1. Here is some data:

Country Rank of Area Rank of GNP
USA. 4 1
Russia 1 4

India 6 26
(30 countries more)
etc.

circle the name of the appropriate statistical technique from these choices:

a. Rank-difference correlation

b. Kendall‘s Tau

c. Point biserial correlation

d. Phi coefﬁcient

2. Here is some data:
Student College GPA Socioeconomic Status (I-Iigh=1;Low=0)
Jackson 3.2 1

Wayne 2.4 0
Wagner 2.9 1
etc.

circle the name of the appropriate correlation technique from these
choices:

a. Pearson product-moment correlation

b. Biserial correlation

c. Tetrachoric correlation

3. A biserial correlation is to be used when relating two artiﬁcial
dichotomy variables.
True False
4. Kendall's tau has a more normal sampling distribution than rho for a
sample under ten.

True False

 

 

134

APPENDIX I
For the Selection of Exercise

Why did you select the exercise you did ?

135

APPENDIX J
Exercise Form A

1. Here is some data:

Student GRE scores Socioeconomic Status (Hi ghzl; Low=0)

Anderson 350 1

McLeod 450 0

Short 550 1
etc.

circle the name of the appropriate correlation technique from these
choices:

a. Pearson product-moment correlation

b. Biserial correlation

c. Tetrachoric correlation

2. Here is some data:

 

Student MEAP Test Scores College GPA
Ross 470 3.0
Warren 360 2.7
Rutter 420 3.2
etc.

circle the name of the appropriate correlation technique from these
choices:

a. Pearson product-moment correlation

b. Biserial correlation

c. Tetrachoric correlation

3. Here is some data:

Student Stress Levels( High=1; low=0) Anxiety Levels(High=l; Low=0)
1

 

Tanner 1

Sinclair 0 1

Potter 1 0
etc.

circle the name of the appropriate correlation technique from these
choices:

a. Pearson product-moment correlation

b. Biserial correlation

c. Tetrachoric correlation

136

Exercise Form B

1. A tetrachoric correlation is to be used when relating two artiﬁcial
dichotomy variables.

True False

2. A Pearson product-moment correlation is to be used when relating two
continuous variables.
True False

3. A biserial correlation is to be used when relating a continuous variable
with an artiﬁcial dichotomy variable.
True False

 

 

137

Exercise Form C

1. Here is some data:

County Rank of Area Rank of Population
China 2 1
Russia 1 2
Brazil 3 3
(30 countries more)
etc.

circle the name of the appropriate statistical technique from these choices:
a. Rank-difference correlation
b. Kendall's Tau

c. Point biserial correlation

(1. Phi coefﬁcient

2. Here is some data:

 

Student SAT scores Gender (Malezl ', Female=0)
Gordon 556 1
Jones 550 0
Leeper 480 1
etc.

circle the name of the appropriate statistical technique from these choices:
a. Rank-difference correlation
b. Kendall's Tau

c. Point biserial correlation

(1. Phi coefﬁcient

3. Here is some data:

Student Gender Drop Out of College
(Male=1; Female=0) (Dropoutzl; Remained=0)
Green 0 0
Evert 1 0
Carter 1 1
etc.

circle the name of the appropriate statistical technique from these choices:
a. Rank-difference correlation
b. Kendall's Tau

c. Point biserial correlation

d. Phi coefﬁcient

138

Exercise Form D

l. Biserial correlations are somewhat lower than those obtained on the
same data using the Pearson product-moment correlation.
True False

2. Tetrachoric correlation is considerably less stable than the Pearson
product-moment correlation.
True False

 

3. Kendall's tau is more likely to be misinterpreted than rho.
True False

 

139

APPENDIX K
Posttest
1. Here is some data:
People Annual Income Annual Saving
Duke $ 30,000 $ 3,000
Cazden $ 54,000 $ 4.000
Calfee $ 43,000 $ 5,000

etc.

circle the name of the appropriate correlation technique from these
choices:

a. Pearson product-moment correlation

b. Biserial correlation

c. Tetrachoric correlation

2. Here is some data:

Countg Population (unit: million) Number of Soldier (unit: million)

USSR 220 5. 12

Britain 50 0.32

Germany 60 0.45
etc.

circle the name of the appropriate correlation technique from these
choices:

a. Pearson product-moment correlation

b. Biserial correlation

c. Tetrachoric correlation

3. Here is some data:

 

Countg Degree of Industrial Development Welfare (Goodzl: Poor=0)
(Developedzl; Underdeveloped=0)
1

A 1
B 1 0
C 0 0

etc.

140

circle the name of the appropriate correlation technique from these
choices:

a. Pearson product-moment correlation

b. Biserial correlation

c. Tetrachoric correlation

4. Here is some data:

 

Student Satisfaction of Home Environment GPA
(Satisﬁedzl; Unsatisﬁed=0)

Cooper 1 3.7
Lein 0 2.6
Heath 1 3.1

etc.

circle the name of the appropriate correlation technique from these
choices:

a. Pearson product-moment correlation

b. Biserial correlation

c. Tetrachoric correlation

5. Here is some data:

Baseball Player Batting Average Years in Major League
Miller .327 3
White .287 7
Enos .252 2
etc.

circle the name of the appropriate correlation technique from these
choices:

a. Pearson product-moment correlation

b. Biserial correlation

c. Tetrachoric correlation

6. Here is some data:

People Approval of Abortion Home Environment
(A greezl; Disagree=0) (Goodzl; Poor=0)

141

Keddie 1 1

Wells 0 1

Woods 0 0
etc.

circle the name of the appropriate correlation technique from these
choices:

a. Pearson product-moment correlation

b. Biserial correlation

c. Tetrachoric correlation

7. Here is some data:

People Legalization of Marijuana Socioeconomic Status
(Agree=l; Disagree=0) (High=l; Low=0)
Wild 1 1
Zacks 1 0
Eder 0 0
etc.

circle the name of the appropriate correlation technique from these
choices:

3. Pearson product-moment correlation

b. Biserial correlation

c. Tetrachoric correlation

8. Here is some data:

 

Country Average Height at 18 WelfargGooEl; Poor=0)
A 172 cm 1
B 168 cm 0
C 173 cm 1
etc.

circle the name of the appropriate correlation technique from these
choices:

a. Pearson product-moment correlation

b. Biserial correlation

c. Tetrachoric correlation

142

9. Here is some data:

Family Number of child Socioeconomic Status (Highzl ; Low=0)

Webb 2 1

Young 1 1

Mehan 3 0
etc.

circle the name of the appropriate correlation technique from these
choices:

a. Pearson product-moment correlation

b. Biserial correlation

c. Tetrachoric correlation

10. Here is some data:

 

Quarterback He' gl_1t Passing Yardage
Yarema 182 cm 2278 Yards
Chucklong 185 cm 2754 Yards
Ware 175 cm 1567 Yards
etc.

circle the name of the appropriate correlation technique from these
choices:

a. Pearson product-moment correlation

b. Biserial correlation

c. Tetrachoric correlation

143

APPENDIX L

Subjects' Written Comments for the Selection of Text Content

1. Deﬁned a_nd Concrete Treatment

81:

82:

S3:

S5:

S6:

S7:

Both written + accompanying instructions stated that this was the
info. we would be tested on. I did, however, read background info
to give me a point of reference.

They gave me the key words in the text.
New information, objectives related to different types for posttest.

I chose that content because the objective informed me what areas
would be covered on the exercise/practice sheet. This helped me
focuses my attention on what I needed to cover and understand. Any
extra knowledge might have confused me or interfered with my
ability to absorbed new information.

Key words from the objective. Key vocabulary needed to
understand explanation of the 3 correlations. So confusing - I
wanted only to focus on the most relevant inf. so my mind could stay
clear.

- I selected the main deﬁnitions.
- When examples make it easier to understand the point which is
being developed.

1. I look at the test questions.

2. Because I knew next to nothing about correlations, I read the
intro. paragraphs. The background+deﬁnitions were very
helpful.

3. Then, I weeded out the extraneous + non-pertinent material. This
deleted several of the correlation processes.

4. I eliminated the paragraphs under Pearson's product-moment
model as the instruc. obj. indicated. I wouldn't need really need
to know much about them; that is, learning would take a different
direction.

5. The remaining paragraphs were the introductory paragraph+the 3
on each of the correlation processes considered to be important in
the instruc. obj.

88:

S9:

S10:

$11:

812:

$13:

814:

144

Because the three types of correlations which were to be tested were
given at the beginning of the exercise. I was looking for examples
of two three types of correlations which were presented in the ﬁrst
instructions.

It dealt with the 3 statistical tests that were listed in the instructional
objective. Highlighting helped me to remember.

Because it was comparing two variables that were the same or had
the same correlation. It showed the relationship or correlation
clearly.

Because it directly related to the subject of correlation between 2
variables.

According to the statement of the objective, I needed to know only
about three speciﬁc types of correlation, so I read about only those
three. In describing two of the three, an familiar term arose
(artiﬁcial dichotomies). I felt I needed to know about it so I found
the appropriate text (which I had brieﬂy skimmed earlier) and read
it. Since it was essential, I checked that section. I was already
familiar with the essential concept of continuous variables, so I did
not read that section, although I noted it from a brief skimming of
the text.

Each paragraph pertained to something in the instructional objective.
The ﬁrst couple gave deﬁnitions of the terms I would be needing.
The last three were descriptions of the three correlations stated that I
would be tested on. The instructions made it clear that I did not
need to read all of the paragraphs. I only looked for material that
was considered important via the instructional objective and the
sample test instructions.

I highlighted those to be tested, i.e., the 3 methods of doing
correlation tests. The introduction is already understood, so I did
not highlight it.

2. Deﬁned Only treatment

SIS:

I highlighted the paragraphs that dealt with the terms in the
instructional objective. I tried to only focus on the part of each
paragraph that explained the ﬁve main terms that were stated. The

 

S16:

817:

818:

$19:

820:

$21:

822:

145

instructional objective told me I would need to tell when to use a
speciﬁc correlation technique - so that is what I focused on. I tried
to memorized the usage for each correlation.

The areas I highlighted were areas that I felt were uniquely
signiﬁcant to understanding each coefﬁcient.

Some was background information I needed to know and the rest
related to the 3 tests we would be tested on.

Chose key words either in title of each section or in a quick scan of
the section looking for the three statistical tests we were told we
would be tested on. Chose paragraph A because it deﬁned
correlation and paragraphs B,C, & D to answer myself "we” were
using the same terminology to describe the 3 tests. I highlighted
within sections words that clariﬁed purpose of each test (when to use
it) and strengths/weaknesses of each test to be further discriminating
in selecting the appropriate test to use in the "practice". Scanned
examples to "cement" the theory.

I felt that every paragraph was relevant to the instructional
objectives. One speciﬁc technique was discussed in each paragraph
well enough for me to understand the basic use for the technique.
The language was clear and understandable, and nearly every
sentence was purposeful-the text was not wordy, but was quite to-
the-point.

The ﬁve paragraphs checked related to the objective
Continuous variables
Artiﬁcial dichotomy
Pearson - two continuous
Biserial continuous - art. dich.
Tet. - two art. dich.

I scanned the words listed in objectives and read parts of these
paragraphs concentrating on differences among Pearson, Tetra, and
Biserial correlation.

Because the instructional objective clearly stated what I would be
required to know and what speciﬁcally I would be given (i.e., the
choices, only the sets of numbers) to assist me in reaching my
answer. The paragraphs I highlighted (checked) contained helpful

146

information, deﬁnition, and examples so that I could retain the
information for the "test".

823: 1. It pertained speciﬁcally to the deﬁnition of terms that I will read
an understanding too in order to then identify the different types
of correlation of variables examples.
2. It explained speciﬁcally what the correlation involved and linked
to the deﬁnition of terms in the objectives.

S24: I chose the content based on the frequency of usage.
825: Because the ones I choose are relevant to the objectives.

826: I chose those contents because educational researchers should learn
about all those kinds of correlations. Knowing different statistical
models, speciﬁcally correlation, will help the researcher to apply the
appropriate model in a given context. Learning about all those
models will make the instruction complete. one is not expected to
master all the models, basic knowledge about all the models should
be included in the instructional design.

827: The objective stated (1) that $111 combination of continuous
correlation and artiﬁcial dichotomy would appear on the test, and (2)
that choices would be from Pearson-moment correlation, biserial,
and tetrachoric whatevers. Since I didn't know from memory or
application what any other stuff in #1 or #2 were about and which of
them were a description of #1, I read those applying to both #1 &
#2. I didn't read the other paragraphs because they weren't
mentioned in #1 or #2 of the objective. I didn't even recall anything
part #1 & #2 in the objective because I was so upset that I had to take
a test on something I didn't know anything about. It took my awhile
to calm down enough to take any information from the paragraph I
was reading. I don't know I will retain the information because I
didn't get a chance to practice anything before I took the "test."

828: Understanding different correlations is important especially
Pearson's Product Moment Correlation. The other correlations are

not as applicable, and could be looked up in most statistics books
anytime.

3. Concrete Only Treatment

829: 1. Deﬁnitions are boring and lengthy.

S30:

S31:

S32:

S33:

S34:

S35:

S36:

S37:

838:

147

2. I anticipated these will be examples.

3. The terminology made sense to me, so I decided to memorize the
terms and from them to derive the meaning, rather than using the
given deﬁnitions.

The objective was to select an appropriate statistical correlation. I
felt that a review of all correlations was necessary, as these are not
all familiar to me.

It discussed! -Two set of continuous scores
- Most commonly used formula.

I think that the understanding of variables would be the ﬁrst step for
learning correlation. I think that the most widely used correlation
techniques are most important in this text.

Because the instructions said that the past test was about correlation
(a different types of correlation). I also highlighted some on the
deﬁnitions provided because the aided in my understanding of the
different correlation types that I am not familiar with.

The ﬁrst paragraph was an overview of how the correlations work -
to show relationships. The next three choices dealt with the tests that
were named on the ﬁrst sheet you gave us - describing the objectives
and the post tests.

First, the deﬁnition of some basic concepts and terms regarding
correlation has been discussed. Then, 6 kinds of correlation methods
are explained based on the above discussion, which draw a clear
picture of both concept and application about the chosen topic.

Because the instructions told me to look for those 3 areas only.

Because the instructional objective explained that I would be tested
an my ability to recognize when a certain technique would be
applicable and circle the correct technique.

As the objective of the handout is to know the correlational methods
such as Pearson Product-Moment Correlation, Biserial Correlation,
Tetra-Correlation, I have to pay attention to those words that are
relevant to the objective.

S39:

S40:

S41:

S42:

148

I ﬁrst considered the instructional objective and then I highlighted
the content that I thought would be most relevant. Time was also a
factor in what I highlight.

They were the 3 types listed in the instructions.

I checked all paragraphs to refresh my memory. Even after 3 terms
of statistics, correlation came quite early (full) and it was obvious
that I could not recall from memory alone the different techniques.
Even the simpler notions of "rank", "continuous variables",
"dichotomies" etc. needed to be reviewed. As the instructional
objective stated that I would be given some data and be asked to
circle the best technique, I felt it important to be thorough. Having
to read all the directions told me that I do not have an automatic
grasp of correlation. I circle many key words so when I ﬁnish I
could go back and review quickly my understanding. Admittedly,
the names of the techniques were not clear, so I resorted to looking
at the key words and matching them with name: For example
Name Variables

Continuous
Rho Rank True Dich/False
Biserial True Dichotomy True/T rue

Point Biserial Artiﬁcial Dichotomy Artiﬁcial/Artiﬁcial

The items I chose were explanations of when the various types of
correlations would be used. It applies to my instructional objective
of choosing the proper correlational method for given data.

4. Neither Deﬁned nor Concrete Treatment

S43:

S44:

It was the only one that stated when a correlation technique would be
used " would be used when ..." or words to that affect. The other
paragraphs seemed to be explaining facts, i.e., correlational
techniques rather than when you would use correlational techniques
in the ﬁrst place.

The instructional objective mentioned that I should know when to use
correlation techniques and the answer to this question is in paragraph
A, that is when we want to know the relationship between two
variables.

S45:

S47:

S48:

S49:

850:

851:

852:

853:

SS4:

855:

149

Because it stated speciﬁc instances of n_sg which I felt might be
useful. The other paragraphs contained material of a more general
nature (and/or deﬁnitions which I am familiar with at this time).

Because I thought the content I check was the basic, but core concept
to understand the correlation. The reason why I didn't mark on

some content was that it was not necessary to learn for my practical
beneﬁt.

The sections highlighted contained information about which type of
correlation was used given the different types of data. The objective
asked me to learn when to use correlational techniques which had to
be done in context of understanding the different variables and how
they are represented.

Because it related to correlation technique used in various data.
There were seven types of techniques used, all of which were
developed or designed to draw correlations between variables, age,
gender, test scores, etc. to help draw conclusions from data.

Told me what I needed to know when I what a correlational ﬁgure
did so now I know when to use it.

Because it was the only content that directly matched the objective, in
its most basic form.

Because it told me when to use correlational techniques (when I want
to ﬁnd the degree of correlation between two variables).

The content that gave basic information about correlation,
dichotomy, etc.were important.

I choose those content because they are related to the content about
correlation.

I choose the content I marked because it seems important for
correlations. Anything that related to the 2 variables measured the
relationship of - I checked.

I chose all of the text as relevant because the instructional objective
stated the learner was to know when it was appropriate to use
correlational techniques. 7 different types of correlation were
discussed, depending on the level of measurement of the 2 variables.

150

To know when each was appropriate, one needs to associate the type
of correlation with the different possible levels of measurement.
This require some rehearsal time for me since I have not worked
with correlation for a year.

SS6: Because those the content needed for the question given later.

151

APPENDIX M

Subjects' Written Comments for the Selection of Exercise

1. Deﬁned and Concrete Treatment

81: No particular reason. It looked a little more interesting I guess.
82: The type of question will help me recall deﬁnition of the concept.
S3: I like seeing the example.

S4: I selected that exercise because it best ﬁt what I had read about.
The objective stated what I would be "tested” on and I felt the
particular exercise would show me if I grasped the content which I
read.

SS: It was most similar to the way I had set up my study pattern to learn
the information.

S6: It doesn't require to remember too many details from the text.
S7: I thought I knew the answer.

S8: Because it matched the examples which were given in the original
reading. Also the three choices in the 1st paper were the same three
choices of correlation techniques which were in this exercise.

S9: The answers fit the material I had studied, in the form I studied it.
810: Because it show the ranking of each country.

81 1: Because it was a choice between 1 & 2 which I am most familiar
with rank (Spearman) order versus nonrank (Pearson).

S12: It most accurately reﬂected the type of test the objective talked
about. I rejected the others because:
- one represented a true/false question, not a multiple choice
type that I expected from the objective
- one mentioned a type of correlation which I was told I didn't
have to know about in the objective

$13:

152

- one gave data that was not appropriate for any of the three
types of correlation I was told I would have to know about

The exercise was an example of the post test stated in the example.
It had the same variables listed.

814: Because it tested what I am supposed to be tested.

Ldeﬁned; Only Treatment

815:

816:

$17:

818:

$19:

820:

S21:

$22:

I selected the exercise that I did because the data looked familiar to
what I had been reading.

Because I can use the either or alternative, I do not like process of
elimination because these seems to be a tendency at times for these
response categories to overlap or just almost seem like the right
answer.

The 3 tests that should be found is included on the ﬁnal test.

Because I learned the criteria for selection of each test sol selected
the test form that allowed me to look at the "situation" and apply the
criteria then select the test.

Because while reading the text, I analyzed the data to be able to
distinguish between the techniques. Words can sometimes be
confusing, but numbers on paper should be straight-forward.

The ﬁve paragraphs checked related to the objective
Continuous variables
Artiﬁcial dichotomy
Pearson - two continuous
Biserial continuous - art. dich.
Tet - two art. dich.

1 and 4 have names of correlation other than those in objective.
As a exercise, I thought T-F types is easier to see feedback.

I am a visual learner and I feel I have some photographic memory.
Also, I like to see example and apply my knowledge before making
my choice. I just felt the more information I had available the better
I could/would do.

S23:

824:

S25:

S26:

S27:

S28:

153

It used the information in the instructional objectives in a similar
fashion - Linking the name of the correlation to an example of data.
- I work rather ﬁtting examples to deﬁnitions.

Because I can apply my understanding.

Because the one I select is described in the content.

I chose form 3 because I was expecting same tests that may deal with
(mental) recall without engaging calculations since time is short for

such exercise.

It contained examples of both Pearson Product-Moment Correlation
and artiﬁcially dichotomous whatevers.

The correct answer was present and there were the fewest number of
choices.

3. Concrete Onlv Treatment

829:

S30:

S31:

S32:

S33:

S34:

1.It was the reﬂecting best the text read.
2.It included the names (terminology) I used to memorize the text, in
the way I exp_e§ted it.

The objective is to name an appropriate correlation technique, and
(2) is the only one which asks clearly & directly to do this.

The most comfortably with continuous score (at least one). The
three selections were (a,b,c) most familiar.

It is appropriate to apply the understanding of variables.

Because it was similar to sample shown in the general instruction and
the samples I read in the text.

I felt I had learned enough about a totally unfamiliar subject during
my brief reading of it to have a basic idea of what they were about.
I knew I hadn't learned anything that would help me apply the new
knowledge. I wasn't even sure of what I had learned! I felt the
true-false format would give me enough information to recall the
little bit of knowledge I had gathered.

S35:

S36:

S37:

S38:

S39:

S41:

S42:

154

Compared to the ﬁrst question, I have more conﬁdence in selecting
question2, because it seems that I can have two answers in questionl.

Because this is the area of instruction I read on.

Because the format allows me to determine based on the information
which technique would be applicable given the type of data.

Because the term was familiar to me, I wanted to know more about
it.

I was totally confused. I thought I remember the most information
about the exercise I selected obviously. I was wrong because I failed
the test.

Biserial - easiest to identify scores/rank.

The direction stated "to prepare you for test" selection 3 and 4
simply assessed memory and did not provide visual information
(though would be easier to answer). Selection 1 seemed to provide
a little bit more information than number 2 (told how many more
countries -needed in case of a small 11 of less than 10) (provided one
additional answer (4) compared to (3) in number 2).
Confusion: #1 said "statistical technique” which could mean all
techniques.
#2 said "correlational technique" which would delimit
and only give correlational techniques.

Exercise 2 looked like it ﬁt the description of my behavioral
objective - choosing the correct correlational method for various
forms/types of data.

4. Neither Deﬁned nor Concrete Treatment

S43:

It related most closely to the objective and content I selected and
included words like "... is used when..."

S44: Because it make you practice when to use a speciﬁc kind of

correlation technique.

S45:

S47:

S48:

S49:

850:

S51:
852:

SS3:

SS4:

SSS:

SS6:

155

I chose 3 because of the word "when" in the question. If the
objective is to know when, then that is the question to be speciﬁcally
addressed.

Because I like the type

I selected exercise #1 because I can't select the correct answer better
than when data/information is given rather than selecting T or F
without an example.

I chose the exercise because I thought I understand the material,
apparently not, because I reversed thinking on all the question asked.

I remember same thing about rank?

Basically, it was just a matter of choosing one at random. My
knowledge of correlation is not good enough to spend a lot of time
trying to ﬁgure out on which test I'll do best - I think I'd do about
the same on any of the tests. ‘

Because it related to correlational techniques.

I selected the exercise because it had familiar looking data arranged
in a format that is used regularly in my work related reading. The
columns for rank and types data seemed familiar, therefore easy
understand.

I feel familiar with this kind of data analysis.

I seem to remember a little something on ranks. Also it seems
interesting because I haven't done this type of correlation.

Q3 and Q4 were true & false. They did not ask the learner to
choose among the different correlations, so these were ruled out.Q2
had fewer option (3) than did Q1 (4), so I thought the likelihood of
choosing correctly to be less a matter of luck. Q1 asked which
correlation measure was appropriate given data of certain level of
measurement which tested on the objective.

Because that exercise was the one I feel I understand relatively.

 

"tuttut