© 1978

PRITT FRANK HIOB

ALL RI GHTS RESERVED

© Copyright by
FWQHT’FFUMVKJHHDB

1978

DEVELOPMENT AND PILOT TESTING
OF A SELF INSTRUCTIONAL MANUAL
FOR THE DESIGN OF LEARNING MODULES:

A RESEARCH AND DEVELOPNlENT STUDY

By

P . F rank Hiob

A DISSERTATION

Submitted to
Michigan State University
in partial fulfillment of the requirements
for the degree of

DOCTOR OF PHILOSOPHY
Department of Secondary Education

1978

ABSTRACT

DEVELOPMENT AND PILOT TESTING
OF A SELF INSTRUCTIONAL MANUAL
FOR THE DESIGN OF LEARNING MODULES:
A RESEARCH AND DEVELOPMENT STUDY
By

P. Frank Hiob

In spite of widespread use of self instructional materials
and a plethora of books and manuals on how to develop such
materials, it was not known whether a self instructional manual
could be used to enable college level instructors with very little
background in instructional design to produce self instructional
modules which involved higher level cognitive skills. To answer
this question the author designed and tested a self instructional
manual . Six college level instructors were asked to design
modules using the cognitive skills and step by step guidance
provided by the Manual. These instructors received no additional
formal instruction regarding module design.

The completed modules were evaluated on a criterion-
referenced evaluation form by six evaluators. All six modules
reached the criterion level suggesting that the‘ Manual was effective
as a learning experience. The designers were positive about their
experience. However, all designers indicated that they required

additional outside information to fully complete their modules.

DEDICATION

This dissertation is dedicated

to my Mother and late Father.

ii

ACKNOWLEDGEMENTS

I wish to acknowledge my indebtedness to Dr. A. Abedor,
Chairman of my Committee for the support, advice and guidance
which he so readily and competently provided. Other members
of my Committee, Dr. R. Farace, Dr. C. Gentry and Professor
B. Wenberg all provided excellent assistance in their own particular
ways and to each I am grateful. The help and steadfast support
provided by Professor Wenberg goes much further than her
involvement on my Committee, and includes continuous support
and, through a graduate assistantship, an inspired introduction
to what was to me a whole new field of Dietetics.

A number of people have made important contributions in
one way or another: the designers who produced modules in
Australia and Hawaii, Mary Jo Morrissey for her module,

Dr. Walter Dick of Florida State University for permission to
use various materials, the Dietetic Faculty at Michigan State
University for form atively evaluating the Manual and suggesting
improvements.

To my wife Elizabeth for her love and help over the last

three years I owe much of this work.

iii

TABLE OF CONTENTS

LIST OF TABLES O O O O O O O O O O O O O O O O O

LISTOFFIGURES................

Chapter

II.

I.

GENERAL INTRODUCTION. . . . . . .

mUOm>

Subject Area . . . . . . . . . . . .
Statement of Problem. . . . . . . . .
Statement of Need . . . . . . . . . .
Research Questions . . . . . . . . .
Limitations of the Study . . . . . . .

RE\/IEW OF LITERATURE. . . . . . . .

organization 0 O O O O O O O O O C O O O

A.

B.

Critical Observations on Literature
Reviewed..............
1 . Lack of Standardized Terminology .
2. Problems of Educational Innovations
3. Systems Approach . . . . . . . .
Justification for Information Chosen
for Inclusion in the Manual . . . . . .
1 . Learning Outcomes . . . . . . . .
2. Mastery Learning . . . . . .
3. Learning Processes . . . . . . .
4. Analysis of Learning Outcomes . .
5. Sequencing of Content. . . . . . .
6. Criterion-Referenced Testing and
Measurement
7. Formative Evaluation. . . . . . .
8. Modular Instruction. . . . . . . .
9. Diffusion of Educational Innovations

iv

Page
vii

viii

mum-he

11

11

11
13
16
18

21
22
26
29
33
34

36
38
42
43

Chapter Page

III. METHODOLOGY . . . . . . . . . . . . . . . 40
A. Type of Research . . . . . . . . . . . . . 46
8. Development of the Self Instructional
Manual................. 49
1. Philosophical Basis and Format
ofthe Manual . . . . . . . . . . . . . 49
2 . Identification of the Manual '5 Content . . 55
3. Cognitive Objectives of the Manual. . . . 58
4. Product Objectives of the Manual . . . . 60
5. Formative Evaluation of the Manual . . . 62
C. Method of Pilot Testing: An Overview . . . . 63
1. Sample Selection . . . . . . . . . . . 63
2. Data Collection .. . . . . . . . . . . . 64
3. Attitudinal Questionnaire . . . . . . . . 65
4. Summative Evaluation . . . . . . . . . 66
IV. DESCRIPTION AND RESULTS OF PILOT
TESTING OF THE MANUAL . . . . . . . . . . 67
Description . . . . . . . . . . . . . . . . . 67
Findings................... 70
V. SUMMARY AND CONCLUSIONS . . . . . . . . 82
Summary of the Pilot Testing of the Manual . . . 82
Conclusions of the Study . . . . . . . . . . . 83
Heuristics . . . . . . . . . . . . . . . . . . 87
Recommendations for Further Research . . . . 90
Improvements to the Manual . . . . . . . . . . 90
Improved Methods of Investigation . . . . . . . 92
Concluding Remarks . . . . . . . . . . . . . 94

BIBLIOGRAPHY...............
APPENDICES
A. Evaluation Instrument for Module

and Rationale for Evaluation . . . .

Survey on Competency Based Education

Results of Summative Evaluation

of a Module Designed from ﬁne Manual
as Partial Requirement of a Masters'

Degree (Morrissey, 1978) . . . . .

The Manual . O O O O O O O O O C 0

Post Modul e Devel oprn ent Su rvey:
totheDesigner. . . . . . . . . .

Summary of Progress of the Dietetic

Undergraduate Resources Committee (DURC).

vi

Page

95

114

121

125

138

379

381

Table

LIST OF TABLES

Page
Evaluation of Modules . . . . . . . . . . . . 74
Evaluation of Modules--Sunmary Table . . . . 77

Results of T-test for Difference Between

Pretest and Posttest Percentage Mean Scores

in Lecture and Module Treatments for

Michigan State University and University

of Hawaii Dietetic Students

(Morrissey, 1978). . . . . . . . . . . . . . 79

vii

Figure

LIST OF FIGURES

System Approach for Designing Instruction
from Dick and Carey, The Systematic Design
of Instruction, Scott and Foresman 8: Co. .

TheM.S.U. Dietetic Model. . . . . . . .

Page

0 O O 57

O O O 58

CHAPTER 1

GENERAL INTRODUCTION

A . Subject Area

The subject area for this dissertation is instructional design,
educational systems and educational technology. The major focus
is the design of self instructional programs in higher education,

particularly where higher level cognitive skills1 are required.

B. Statement of Problem

In the area of Dietetic Education (as in a number of other
professionSQ) the national controlling body has called for an improve-
ment in the educational preparation of its pre-service students

(Study Commission, American Dietetic Association, 1972).

 

The term "higher level cognitive skills" refers to those skills
which on Gagne's learning hierarchies (1977) are designated as
"rule" or "problem solving".

Medical Education is another example, see A Handbook
for Change, Recommendations of the Joint Comm ittee on
Medical Education. Wm . F. Fell Co. Philadelphia,
Pennsylvania, 1972.

 

1

Part of the concern is the level at which subject matter is
currently being taught. Cox (1970) found that the cognitive level
at which the average college level instructor teaches is at the
comprehension level on Bloom's (1956) taxonomy, a low level
with reliance mainly on memory. Instructors claim (Cox, 1965,
1970) that they expect their college students to operate at a much
higher level such as problem solving, where a number of concepts
and principles are combined to reach a solution to a complex

problem . This problem is fully discussed in the Manual (Appendix D).

In response to the challenge to improve the quality of education
various innovative programs are being established throughout the
country. At Michigan State University, on advice from the
University's Learning and Evaluation Service, and funded by a
Kellogg grant, a Competency Based Education (CBE) Program 1

has been initiated in the Department of Food Science and Human

 

Nutrition.
1 The implied characteristics of CBE are:
1 . Instruction is individualized and personalized.
2. The learning experience of the individual is guided
by feedback.
3. The program as a whole is systematic.
4. The emphasis is on exit, not entrance, requirements.
5. Instruction is modularized.

W. Robert Houston, "Competency Based Education" in
Houston, W.R. , Exploring Competengy Based Education,
McCutchan Publishing Corp. , Berkeley, 1974.

 

3

One feature of CBE is that it requires instruction to be
individualized and presented through modules. Individualized
instruction ﬁnrough modules has become the fastest growing
movement in education (Novak, 1973; Cross, 1976; Vermilye,
1976). The use of modules is also central in all other methods
of improving education through individualization (Goldscrmid, 1974).

For instructors to fully participate in CBE or any plan of

individualized instruction, it will be necessary for them to be able

to design modules as only rarely will ﬁney find suitable modules
avail able for purchase.

The task of designing modules should be recognized as requir-
ing a great deal of knowledge on learning, teaching, educational
materials, ﬁne specification of learning outcomes, analysis of
content, an understanding of ﬁne interrel atedness of parts of a
learning system, and the cybernetic1 nature of an instructional
system . Most of this knowledge is new to university instructors
wiﬁnout an education background (Chase, 1968; Cl ark and HOpkins,
1969; Gideons, 1970).

There are texts available on ﬁne design of modules but they
usually require a workshop, prior experience, or an intensive

period of training, e.g. , System Approach for Educatiog, (Corrigan

 

Cybernetic is taken to mean control through feedback.
Von Bertal anffy's (1966) explanation of the cybernetic system is
presented on p.

4

et al , 1969). Self instructional manuals on modular instruction are
available, e.g. Criterion Referenced Instruction (Mager, 1961).
Unfortunately ﬁnere is no research study ﬁnat provides evidence

that useful modules actually result from working through a manual
without additional intervention. As Branson (1978) points out, it
may even be impossible to find a generic, all purpose manual or
model to suit ﬁne specific needs and target population of a particular
program .

Given ﬁne realities of constraints on instructors' time, and an
understandable reluctance to "go back to school", one alternative
would be a self instructional manual on the design of modules
developed for a particular population and with its needs in mind.

Up to now the identification of need has been on a macro level.
The target population at such a level is diverse, and according to
Branson (1978) it would be unlikely that a model could be devised
that adequately meets the needs of such a heterogeneous group.
Consequently, it is necessary to look at the micro level in order to
determine the needs of a more homogeneous group. Such a focus
would be more specific in identifying needs, and it would be more
likely that these specific needs could be met.

The population at the micro level for this dissertation is the
Co-ordinated Undergraduate Program in Dietetics at MSU . In

1974, MSU was awarded a grant by the Kellogg Foundation to

develop a co-ordinated undergraduate program in Dietetics. In
its proposal for the grant, MSU stated that it would develop the
program along the lines of a competency based education system .
Subsequently a committee, designated as DURC1, began to work
to bring about a CBE program .

So as not to interupt ﬁne flow of ﬁnis General Introduction,
the proceedings of the Committee's work are stated in Appendix F.
The pertinent aspect of DURC was a difficulty the committee
experienced in accepting highly conceptual new ideas concerning
Competency Based Education. It was stated by the Committee
that actual examples of CBE concepts were necessary if progress
was to be made.

Given ﬁne articulated need for knowledge on CBE concepts
and a need for actual examples of modules, ﬁne auﬁnor decided that
there was a need for a manual which would attempt to meet ﬁnese
two needs. To the author ﬁne major problems appeared to be:

a. On the part of some key faculty members there was
a rejection of rigid CBE methodology which ﬁney equated with

behavioral ism . These faculty looked for a less rigid program

 

Dietetic Undergraduate Resource Committee.

6

which would accomodate their concerns. For example, they
reﬁJsed to participate in an early step of ﬁne CBE program (that
of sequencing competencies) by stating that it was not possible

to sequence competencies when competencies had not and could
not be adequately determined through ﬁne use of a Magerian
formula. Their concern is often expressed in literature (Eisner,
1975; Elam, 1972; Neff, 1975; Ainswortln, 1977)

b'. For a number of faculty, particularly ﬁnose heavily
engaged in research, the concepts of CBE were quite novel.

They held an Open mind to a program which would improve
learning, but did not have sufficient information on CBE to
become committed to the program .

c . Lack of concrete exam pl es of what the innovation (CBE)
was all about led to a great deal of uncertainty. The commonly
used term "module", which was chosen by DURC as ﬁne unit of
instruction in CBE and ﬁnerefore a key concept, was perceived by

many as threatening .

C. Statement of Need

 

Given the above problems, the author, after consulting with
a number of people from DURC, decided ﬁnat the major needs in
which he could have some impact were:

1 . To modify ﬁne CBE concepts so ﬁnat they responded to
the needs and aspirations of the target population; thereby being

perceived as rewarding and consequently accepted.

7

2 - The dissemination process of the acceptable CBE
concepts through a product. Concrete evidence should be produced
to make CBE concepts visible.

3. The product referred to in (2) should be a module as
modules are central to CBE (Houston, 1974).

4. The acceptable CBE concepts, and how to translate
these concepts into a module, shOuld be presented through a manual
So as to be convenient and fit wiﬁnin faculty assumed method of
information assimilation.

5. The manual should address the needs expressed (on pp. 1-6)

under Statement of Problem . For example, it should deal with

 

higher order cognitive learning rather than lower level information
Skills.

Given this background the author decided to develop a manual
to address ﬁnese needs. That such a manual would be effective

became ﬁne focus of interest in the present pilot study.

D . Research Questions

There are two major research questions to be addressed in
this pilot study.

First, can a self instructional manual on designing modules
be developed where those who conn pl ete the manual reach mastery
level on its contents?

Second, will attainment of mastery of the self instructional

manual result in a module which meets some minimum acceptable

standards?

There is also a third question: will the module so developed
obtain results at least as good as those obtained through traditional
methods of instruction? Because of the time required it is unlikely
that such data can be obtained. Furthermore, a number of uncontrollable
factors could bias such data. Therefore, data related to this third
question will be considered as additional supporting evidence rather
than the primary focus of ﬁnis dissertation.

The following are stipulations to the research question:

a. The designer of the module will be a faculty member in
higher education in a field other than education and will have had
very little formal training in education.

b. The learning outcomes of ﬁne modules must include
higher order cognitive skills; on Gagne's (1975) classification they
need to reach at least defined concept level or higher.

c. The designers of modules will need to be in geographic
locations different from that of the authOr so as to control for possible

auﬁnor input.

E . Limitations of the Study

The following limitations reduce the general izabil ity of this
pilot study. The major limitation is the relatively small number
of modules avail able for evaluation. One reason for ﬁne small

number is ﬁne tremendous amount of work involved in

9
developing a module. A conservative estimate is ﬁnat the design takes
someﬁning like 100 hours. On top of this is time taken for developing
materials (e.g. , if the module is on PLATO, it may take another
100 hours to develop the PLATO program). For the self instructional
manual to be mastered and a module produced may take six months.
The estimates stated in this paragraph are based on the author's own
experience in developing and testing the module RENAL which is on
Control Data Corporation's PLATO systems . House (1974) gives an
account of his experiences in developing modules on PLATO; his
experiences parallel those of the author1 .

Another limitation is the lack of controls because ﬁne modules
will be developed independently of the author, without his involvement.
Thus factors which cannot be controlled in this study include:

a) ﬁne variability of ﬁne learner/designer entry skills, b) the input

from ﬁne designer's environment in addition to the Manual, and

c) ensuring that the designer proceed through the Manual correcting

his/her own embedded tests which dictate progress or remediation.
There are no effective controls or way of monitoring this process;

if ﬁne designer does not reach mastery on a test, will he/she in fact

go through ﬁne materials again as required, or continue wiﬁn the next

 

Rosinski (1977) gives an account of ﬁne difficulties in recruit-
ing faculty to develop modules even when an honorarium was
offered-~the anticipated number of modules went down from
from twenty five to just one.

10

step in the manual? In other words, there are no effective
controls which allow only ﬁnose who have mastered the manual
to proceed on to the design stage.

While these limitations are significant, it is felt that for
purposes of the present pilot study ﬁney must be tolerated since

further studies will lead to refinement.

CHAPTER 11

REVIEW OF LITERATURE

O rgani zation

The review of literature is divided into two sections. The
first deals with some critical observations on literature on the
design of instruction. The second addresses research findings
pertinent to the choices made in designing the Manual, with a

concluding review on the diffusion of educational innovations.

A. Critical Observations on Literature Review

 

The review of literature on self instructional manuals is a
complex task. Part of ﬁne complexity is ﬁne abundance of idiosyn-
cratic, loosely defined terminology. For example, self instructional
manuals are referred to by a variety of alternative appelations includ-
ing: Systems Engineering and Training (SET), Learning Systems Design

(LSD), the Systematic Design of Instruction (SDI), Systems Approach

11

 

to Training (SAT), Personalized Instruction (PI), Instructional
Systems Development (ISD), just to mention a few.
In this disse rtation the acronymn SAT (for Systems Approach

to Training) will be used to represent self-instructional manuals .

Differences between the manuals range from super-

ficial variations in terminology to fundamental variations in
philosophy. A basic issue on which disagreement exists is the
degree to which the instructional design process can be reduced to
a linear sequence of generally applicable, prescriptive procedures.
The positions that have been espoused range from Eckstand's (1964)
statement that the design of instruction is primarily an art, to
the hypothesis that course design can be reduced to a series of well
defined procedures which can be carried out by untrained personnel .
Of the dozens of methods contained in the different manuals and
in the related literature, none seem to have attained widespread
acceptance. Campbell (1971) points out that a major problem with

the avail able manuals is ﬁnat they do not indicate how their particular

r‘Tnethodology differs from other methodologies or fronn traditional ways

Of developing educational programs . At the present, the use of self-

‘

1 A number of manuals state in the preface that no prior training

is required, e.g. Modular Instruction by James Russell states:
"The only prerequisite is a desire to improve student learning." (preface).

13

instructional programs is growing faster than any oﬁner method

of improving instruction (Novak, 1973; Cross, 1976) and ﬁne
required ingredient in the title of ﬁne program seems to be
"systems approach" even when there is little agreement as to
what that term means . This confusion is primarily the result
of three factors: lack of standardized terminology, problems
associated wiﬁn educational innovations, and the evolutionary
nature of ﬁne systems approach to manual design. Each of these

will be discussed in some detail in this section.

1 . Lack of Standardized Teminolggy

 

Manuals seem to use similar terms: objectives, task analysis,
methodology, media selection, sequencing, objective performance
rnneasure, criteria referenced testing, individualized instruction
and evaluation.

The use of a common terminology creates ﬁne initial impression
of high content similarity among the manuals. However, closer
inspection of the operational definitions given to ﬁnese terms
shows ﬁnis impression to be false as the following example will
illustrate.

Virtually all of ﬁne manuals use the term "task analysis"

but ﬁne operational definitions of task analysis provided by the

 

14

manuals differ both in content and in degree of detail. While some
manuals leave much to the discretion of the analysts, others are
so specific as to provide a form broken down into hierarchical
categories such as role, duty and activity, or job, task, and
element; others provide litﬁe or no structure as to the number

or the types of categories to be used. Some require each task

to be classified as psychornotor, cognitive or affective; others
rate each task on each of these categories; still others ignore
this breakdown.

Manuals generally provide only one procedure for task
analysis. When alternative procedures are not provided, the
assumption is made that ﬁne method given is universally applicable.
This assumption is not warranted by the literature on task analysis.
The Gilbreths (reported in Swain, 1962) in their pioneering work
on improving industrial efficiency, developed the first formalized
task analytic methodology. Their procedures were useful in time
and motion studies on production line tasks. However, in the 1950's,
R.B. Miller found ﬁnat the Gilbreth's procedures did not allow for the
identification of human attributes used in com pl ex tasks (Swain, 1962).
Miller developed a methodology entitled "task-demands analysis"

because he believed that existing meﬁnods did not provide adequate

15

data concerning the demands which tasks make on the operator.
Since 1960, ﬁne number of available task analytic methodologies
has risen dramatically. A number of theorists, after reviewing
this state of affairs, have concluded ﬁnat no single method of task
analysis can be generated which is val id in all circumstances
(Gustafson, Honsberger, and Michelson, 1960; Folley, 1964;
DeGreen, 1970; Rankin, 1974).

The degree to which task analysis can or should be procedural-
ized is controversial. The trade-off is that although higher degrees of
proceduralization result in narrower ranges of application, they may
permit the use of less qualified, less costly analysts. DeGreen's
(1970) analysis of this problem led him to conclude that: (a) reduction
of task analysis to a routine checklist procedure results in "a deluge
of useless data"; (b) task analysis must always be viewed as a means
and not as an end; and (c) the usefulness of task analytic data is a
function of the degree of expertise of the analyst.

Although the example used here is "task analysis", a similar
presentation could be made concerning each of the terms shared by
the majority of manuals: behavioral objectives, media selection,
methodology selection, criterion referenced testing, objective
performance measurement, sequencing, evaluation. Research and

analysis are needed to determine for each of ﬁnese concepts, the

16

degree to which procedurealization can be achieved, the generality

of those procedures, and the skills necessary to apply them .

2 . Problems of Educational Innovations

 

The degree of acceptance of a given innovation is shaped
by factors other than its inherent advantages and limitations.
Campbell (1971) stated that educational innovations have historically
followed a predictable life cycle, and constructed a three-stage
model of that process. In the first stage , a new technique appears
and develops a large following of advocates who cl aim to have
successfully applied the technique. The second stage consists of
numerous modifications of the basic technique. The third and final
stage in the life cycle of educational innovations is ﬁne appearance of
criticism by a few vocal opponents, which grows into an inevitable
backlash. According to the model, this criticism does not serve to
stimulate improvement of the technique, but to stimulate ﬁne devel op-
ment of a new technique. At that time the cycle starts anew.

While Campbell's model is primarily descriptive, Milsum
(1968) presents a phenomenon called the "bandwagon effect" which

hel ps explain the model. The bandwagon effect serves to transform

researchable hypotheses (educational innovations) into political entities,

thereby t riggering the mechanism which leads to the innovation's

17'

downfall. The mechanism works as follows. As the number of
researchers, developers, theorists, administrators, laboratories,
schools, etc. , who have vested interest in the innovation grows,
the resistance to critical examination of the innovation and to the
consideration of alternatives also grows. In addition, claims are
made for and have become entrapped in the political arena.
The process by which the innovation attains the reputation of
a panacea has an unwanted side effect. The greater the number of
people who attempt to use innovations based on unfulfillable promises,
the greater the number of people will be who are disappointed by it.
As this number grows, the criticism and backlash predicted by
Campbell's model occur and eventually result in the downfall of
the innovation. According to Campbell (1971), SAT is the current
innovation, and is following the life cycle predicted by his model.
That SAT has been touted as a panacea and has fallen victim
to the bandwagon effect was first documented in 1968 by Hartley. He
concluded that the SAT literature is "long on persuasion and short on
critical self appraisal ". He believed this to be the result of over-
2eal ousness in attempts to use the new meﬁnodol ogy wiﬁnout a clear
Understanding of what it was supposed to produce. Carter's 1969
article, "The Systems Approach to Education: Mystique and Reality"
provides not only a review of the problems created by the bandwagon

Effect but also a realistic assessment of what can be expected from SAT.

 

18

Sugarman, Johnson and Hinton (1975) provide further support data

and analysis in ﬁnese two areas.

3. Systems Approach

The systems approach to training, SAT, evolved from "systems
analysis" (alternatively called "the systems approach"), a methodology
developed during World War II, to solve problems created by rapidly
advancing weapons systems technology. After the war, the meﬁnodol ogy
was found useful in the solution of problems in a variety of fields
(Saettl er, 1968).

In the late 1950's, ﬁne first attempts to apply systems analysis
to the design of training programs were undertaken by the Rand
Corporation (Kershaw and McKean, 1958), and by the Human Resources
Research Organization (Hoehn, 1960).

The goal of early SAT developers was to generate tools which
could aid training programs design personnel in their day-to—day work.
These tools consisted of models, ﬁnat is, formalized sinnpl ifications
of methods and techniques which other experts had found useful.

These models were intended to be used, modified or ignored, in any
particular situation, based on ﬁne discretion of the user. They were

not intended to relieve him of his responsibility as a decision maker.

 

19

The early 1960's witnessed ﬁne emergence of a new technology
which greatly affected the evolution of SAT. This new technology
was based on the hypoﬁnesis that if training program design experts
could formalize models of ﬁne meﬁnods and techniques ﬁnat made them
successful, then laymen could follow these models and produce the
same result at lower cost. The main thrust of developmental efforts
under this technology has been the production of manuals which attempt
to reduce ﬁne design of training programs to a linear sequence of
procedures which can be carried out by personnel inexperienced in
training program design (Dicket son, Shuiz and Wright, 1970).

During the middle and late 1960's, the procedural ized SAT concept
generated a great deal of literature. The original, generic concept
of SAT, which remained relatively dormant during ﬁnat period, has
been the subject of renewed interest during ﬁne 1970's. This is,
at least partially, a result of a re-evaluation of ﬁne state-of-the-art
of educational psychology (Campbell, 1971 ; Glaser and Resnick, 1972;
McKeatchie, 1974), which has concluded that ﬁne avail able theory and
empirical evidence on the process of learning and teaching do not support
the procedural ization of ﬁne training program deveIOpment process.
As McKeatchie (1974) pOints out, psychologists are much less sure of

the "l aws of learning" than ﬁney were a few years ago.

20

Recent research is attempting to develop methods, models, and
techniques which training experts can use, modify or ignore. This
reﬂects a belief that deveIOpm ent of training is a complex problem,
not solvable by procedural ized . methods. For example, in the
selection of training media, Braby et al (1975), and Boucher, Gottlieb
and Morganl ander (1973) have produced media selection models which
specifically state that their goal is to assist rather ﬁnan to replace

the experienced specialist.

To conclude ﬁnis section here are two statements, made

twenty years apart, which illustrate our present problem:

"There is a simple job to be done. The task can .
be stated in concrete terms . The necessary techniques
are known. Nothing stands in the way but cUltural inertia."
B.F. Skinner.(1954)

"There is a complex job to be done. The task
cannot be stated in concrete terms. The necessary
techniques are not ﬁJlly known, The equipment cannot
always be easily provided. Other things - primarily
our ignorance of the complexities of human learning -—
stand in the way, as well as cultural inertia. "

J. Hartley (1974)

21

A. Justification for Information Chosen for Inclusion in the Manual

In light of the previously discussed problems in current manuals
on self instructional program design, a major task would be the
selection and justification of information to be included in the Manual
used in the present study. It was necessary to review the literature
for educational practices and theories. This provided information
which would enable the objectives of the Marual to be reached.

The author was not aware of any literature wlnich sets out the
process of selecting content for self instructional manuals. Shore
(1973) states that there is no literature on this process and suggests
that the way to decide on content is to look at existing manuals to
determine their common elements found in module development.

The common elements are: state objectives, order objectives
(sequence), devise pre and post tests, design instructional activities,
make avail able suitable resource materials.

Klingstedt (1971) has recommended a series of steps very closely
resembling those of Kurtz (1971). Summarized, ﬁnese steps are:

Step 1. Learning outcomes are determined.

Step 2. The learning outcomes are analyzed into
smaller objectives and sequenced.

Step 3. Tests are constructed which measure entry
skills and objectives.

22

Step 4. Instructional activities are designed to help

the learner master the objectives.

Step 5. A post test measures the student's achievm ent

of ﬁne objectives .

This author has chosen content of the Manual on ﬁne basis of
what is required to follow his seven step model and conceptual know-
ledge required to understand that model . DevelOpment of the model
is presented in Chapter III p . 49. The major concepts are:

A. learning outcomes

B. mastery learning

C. learning processes

D. analysis of learning outcomes

E. sequencing of content

F. criterion referenced testing and measurement

G. formative evaluation, media selection and

ind ividualizing instruction
H. modular instruction

The literature providing the research basis for these concepts is presented
next, followed by a statement of philosophical basis and format of the Manual.
In determining content the following were also kept in mind:

criticisms outlined in the review of literature, part one; information

 

obtained in interacting with DURC (presented in Statement of Need);

and views currently being expressed in journals.

A. Learning Outcomes

The starting step of this author's model is the specification of
learning outcomes. The use of the term "learning outcome" is a

departure from ﬁne usual term — objective, or behavioral objective.

23.

First a review of the behavioral objective movement is presented
and ﬁnen the reasons for ﬁne preference of ﬁne term "learning outcomes" .

The need for objectives is documented by Tyler (1934) who
advocated ﬁneir use as both a goal for teaching and a measure of
its effectiveness. The behavioral objective movement came into
its own as part of the systems and accountability movements in
education which required measurable (therefore observable)
objectives. Mager (1965) responded to ﬁnis need wiﬁn his three part
formula for writing Objectives . As his first book was on objectives
for programmed learning and as programmed learning was mainly
due to Skinner, people have equated Mager's objectives with behavior-
alism . Mager himself has never termed his objectives as "behavioral";
his concern was to provide measurable objectives (a form of account-
ability).

Since Mager's 1965 text there have been many others who have
proposed a formula for writing objectives . Gagne (1974) proposes
five parts to an objective: situation, learned capability, object,
action and tools . Miller (1962) proposes a skills analysis and in
his view obj ectives should include: 1 . an indicator on which the
activity-rel evant indication appears; 2. the indicator or cue which
calls for a response; 3. the control object to be activiated; 4. the

activation or manipulation to be made; 5. ﬁne indication of response

adequacy or feedback .

24

The level of detail used in writing such descriptions is about
the same as would be used for writing a set of technical instructions
useful to a novice.

Recently there has been a growing reaction against ﬁne use of
behavioral type of objectives (Ainsworth, 1977). Various reasons
are given. Ashby (1965) points out that a learner's performance
changes, according to the testing environment. Dressel (1977)
describes covert learning which would not be measurable (or acceptable)
if we insist that objectives call for observable behavior.

In their later writings, Gagne (1974) and Bloom (1978) have
moved from strictly behavioral objectives to a much wider concept
of "learning outcomes " which allow for certain learning to be covert
and testing to be spread over a period of time.

MacDonald-Ross (1973) lists 16 objections to behavioral objectives.
Geiss (1977) believes ﬁnat there is a pl ace for behavioral Objectives in
training but not in education.

Gronlund (1974) supports the concept of learning outcome and
emphasizes that understanding is ﬁne objective of learning, rather than
behavioral outcomes which are the responses made after understanding
occurs.

A number of authors (Harrow, 1972; Armstrong, 1971;

McAshen , 1 977) believe in"l earning outcom es" replacing 'obj ectives '1

25

They see thatﬁne learning outcome has two components 4 a goal and
an evaluation component. The evaluation component may accept
covert learning and different responses according to the environment.
McAshen (1977) expresses what may be the concern of many:
Once an educator understands ﬁnat the learning outcomes or
con petencies are not ﬁne same thing as the behavioral objectives
(which rely on one-time observations of responses) he must question

the value of any objective that is stated in behavioral terms only.
The Manual proposes an approach which is a synthesis of

a number of researchers (Sirnons, 1973; Armstrong, 1971;

Smiﬁn, 1972; Harrow, 1972) who advocate a more ﬂexible
"learning outcome" approach to the behavioral objective formula

advocated by Mager. Learning outcomes are defined as goals,
competencies or specific learning intents (McAshen, 1977). In

the learning outcomes approach the objective has two components -

a goal and an evaluation component. In evaluating the learner the
focus will be on the achievement of the goal to be conducted ﬁnrough

a sample of a number of behaviors over a period of time and accepting

ﬁne existence of covert learning (Dressel , 1977).

Designers who prefer to present objectives in the format
proposed by Mager or Gagne will not be discouraged from doing

so but others who prefer a less rigorous learning outcome

approach may do so.

26

2. Mastery Learning

 

The concept of mastery learning is central in the move to
individualize instruction. With resistance building up against CBE
it is being suggested in literature that "mastery based education"
replace CBE as the major focus for improving education ﬁnrough
design (Block, 1978). In the Manual, mastery learning is the
central philosophy. The concepts of mastery learning are fully
treated in the Manual ﬁnrough two modules . DURC faculty reaction
has been very receptive.

The basic prenise that most students can learn what needs to
be learnt, if ﬁne process is approached sensitively and systematically,
is a very Old one. The Jesuit schools emphasized this before the 17th
century, Pestalozzi in the 18th century, and it has been part of the
English universities' tutor system since its inception (Klaus, 1971).

John Carroll's Model of School Learning (1963) and currently
called Mastery Learning, is the modern approach to the notion that
most students can attain a high level of learning capability. The auﬁnor's
interpretation of Carroll's Model is that if students are normally
distributed with respect to aptitude for some subject and all students
are given exactly the same instruction, then achieven ents measured

at the completion of the subject will be normally distributed. Under

27

such conditions ﬁne relationship between aptitude measured at the
beginning of the instruction and achievement measured at the end
of the instruction will be relatively high (typically about + .70).
Conversely, if students are normally distributed wiﬁn respect
to aptitude, but ﬁne kind and quality of instruction and learning
time allowed are made apprOpriate to the characteristics and
needs of each learner, the majority of students will achieve
mastery of the subject. And, ﬁne correlation between aptitude
measured at ﬁne beginning of instruction and achievement measured
at the end of instruction should approach zero.

Carroll's (1963, 1970) model states that the level of
mastery reached by a learner on any instructional task is a
function ‘of the time actually spent learning the material and the
amount of time he needs to master the material. The amount of time
a student actually spends learning thematerial depends on two factors:
time allowed, and his perseverance. The amount of time needed by
ﬁne stude nt is dependent on ﬁnree factors: aptitude, quality of the
instructional materials, and his ability to understand the instructional

materials .

28

Research ﬁndings on mastery learning have been impressive
(Keller, 1968; Block, 1971; Bloom, 1971). However, as Cross
(1976, p.77) observes

Those two stalwarts of the school system - grades

and semesters - become almost meaningless when mastery
learning is implemented. Ideally, all students would earn

A grades, and they would take as much time as
necessary to accomplish this level of mastery.

The Manual has two modules on mastery learning, which take
a total of over four hours to complete. The emphasis is on concepts
and supporting research findings to give the designer a strong
knowledge base for possible attitude change (from traditional norm-
referenced approach to a mastery approach). The problem of
grades and semesters raised by Cross are not discussed as ﬁnese
are mainly organizational problems vvlnere the individual faculty
member has very little control. It is a speculation on the part of
ﬁne author that given the two modules on mastery learning, ﬁne
designer will gradually incorporate as many mastery concepts

as organizational constraints will allow.

29

3 . Learning Processes

The Manual places great stress on the learning process; it
continually asks the reader to start the design process by asking
"how can ﬁne learner learn ﬁnis objective?" rather than the common
approach which starts wiﬁn "how should I teach this?" .

Since Pavlov and Thorndike began their studies of learning,
thousands of experiments have been conducted on the learning
process. The Manual does not follow any particular school of
learning or researcher, but is a synﬁnesis of ﬁne generally
accepted theories . Consequently ﬁne theories of Thorndike (1921 ),
Skinner (1968), Ausubel (1968), Bruner (1966, 1971) and Gagne (1974)
are prominent.

Bruner (1966, 1971) is concerned wiﬁn indUcing active participation
in the learning process on the part of the student, catalyzed by a
"discovery-l earning" environment, and by frequent challenges to
solve novel problems. To Bruner there are three major stages of
intellectual development; these are ﬁne enactive, the iconic and the
symbolic representations. He has based some of his work on Piaget
and in turn, much of Gagne's hierarchies are based on these three

basic representations .

Ausubel (1968) is primarily concerned with meaningful
reception learning and the acquisition and retention of knowledge.
The major emphasis of his ﬁneory is on the inferred processes
presumed to be in operation. The major concepts taken from
Ausubel and used in the Manual are: advance organizer,
anchoring ideas, cognitive structures, subsumption and
assimilation .

Skinner (1968) and Thorndike (1921) are very similar and
are treated togeﬁner. Skinner, as is typical of the S-R tradition,
limits his attention and discussion to observables. He is concerned
almost exclusively wiﬁn input-output relationships and does not
write about inferred variables . The concepts used in ﬁne Manual
which are directly derived from these two researchers are:
stimulus, reinforcement and contingencies of reinforcement,
chaining, shaping, respondents and operants.

Gagne (1974) emphasizes the learning of several kinds of
learning outcomes, each requiring a different kind of mental
process. The conditions for learning involve the interaction of
internal conditions of ﬁne learner, and the external conditions of
the learning environment. Each kind of learning outcome requires

a different set of these internal and external conditions.

31

There is one additional theory of learning which is used in
the Manual and ignored in oﬁner manuals reviewed. That is ﬁne
information processing theory. According to this theory, the
processes that one must conceive in order to explain the phenomena
of learning are those ﬁnat make transformation of inputs to outputs
in a fashion somewhat anal agous to the workings of a computer.
These various forms of transformations are what goes on "inside
the learner's head" - ﬁne learning process. Theoretical accounts
of the information processing theory are: Atkinson and Shiffrin
(1968), Norman (1970), Anderson and Bower (1972), Lindsay and
Norman (1972).

The information processing theory is closely related to
cybernetic concepts in biological-behavioral sciences. Von Bertalanffy(1966)
explains the cybernetic system in this way:

"The minimum elements of a cybernetic system are

a 'receptor' accepting stimuli (or information) from

outside as an input; from this information a message is

led to a 'center' which in some way reacts to the message

and, as a rule, amplifies the signals received; ﬁne

center, in its turn, transmits ﬁne message to an 'effector'

which eventually reacts to the stimulus with a response

as output. The output, however, is monitored back, by

a 'feedback' loop, to the receptor which so senses the

preliminary response and steers ﬁne subsequent action

of the system so that eventually the desired result is
obtained. " (p.40)

32

The different learning theories mentioned so far are used
thoughout the Manual particularly in Step 4 (planning of the lesson)
and Step 5 (construction of the lesson), when ﬁney are most applicable.
The auﬁnor sees no problem in this approach as ﬁne theories used are
not contradictory but rather complementary. Full statements of
learning theory would be beyond the SCOpe of ﬁne Manual. Brief
abstracts accompany ﬁne major bibliography for ﬁnose who wish
to pursue any particular ﬁneorists .

There is one exception - that of information processing theory.
A module on information processing ﬁneory is included in the Manual
in response to interest in this ﬁneory expressed by faculty during

formative evaluation .

4. Analysis of LearningQutcomes

Usually SAT refers to this step as "task analysis" and part one
of the review of literature points out that all ﬁne manuals reviewed
presented only their own methodology without reference to other
alternatives, thereby giving the beginning designer a naive view
of straightforwardness and simplicity of the process.

In the present Manual ﬁnree alternate methods are recommended,
according to the learning involved. The designer is to choose the
method which suits the type of learning outcome which is desired.

If the learning outcome involves procedural tasks then the approach
of Davis, Alexander and Yelon (1974) is recommended. When the
learning outcome involves mostly cognitive processes ﬁnen the
approach of Gagne (1974) is recommended and when the learning
outcome involves both procedural and cognitive processes then ﬁne
approach of Singer and Dick (1974) is recommended.

Through interviews with target faculty and during formative
evaluation, the author found that the actual choice of topic as a
suitable "chunk" proved difficult. The work of Cook and Walbesser
(1973) addresses this concern and ﬁneir methodology is included.
Brieﬂy, they suggest ﬁne use of a topical hierarchy before developing

an instructional hierarchy.

34

The Manual is mostly concerned with the teaching of higher
cognitive skills and therefore ﬁne work of Gagne and Briggs (1974)
is used more extensively. Much of their methodology calls for
hierarchical analyses and sequencing; research findings in ﬁnis

area follow.

5. Sequencitg of Content

Duncan (1972) indicates that there have been three main
approaches to research on sequencing. Some investigators have '
taken a ﬁneoretical approach - basing their work eiﬁner on
Skinner's ideas about shaping behavior or on Gilbert's ideas
about the chaining of sequences of responses (Gilbert, 1962;
Mechner, 1967). Oﬁner investigators have looked in more detail
at Mager's approach of determining ﬁne sequence preferred by ﬁne
learner rather than that preferred by the teacher (Mager, 1961;
Mager and Clark, 1963). Oﬁner investigators have based their
approach to sequencing by manipulating ﬁne internal structure of
the subject matter with which they were dealing, an approach
owi ng much to the ﬁneoretical position of Gagne (Gagne, 1970;

Gagne and Briggs, 1974). The Manual suggests ﬁnat where the
content has mostly cognitive skills then the methods advocated by

Gagne be followed. However, as research does not identify a "best"

method the Manual also mentions alternate methods (but in less detail).

35

Each of these approaches will now be briefly considered .
AS a part of his approach to programming, Gilbert advocated that
a sequence which could be classified as chain-like ( a followed by b
then c then d etc.) should be taught retrogressively. That is, the
learner should make the last response first. Gilbert advocated that
doing the last response--completing ﬁne chain—was the easiest
response to make and the most motivating to reinforce. He ﬁnerefore
advocated that learners should, in effect, be programmed to make
their responses to a chain in the correct order but always building
up to completing the end of ﬁne chain. Experiments have failed to
indicate any superiority for ﬁnis approach (Wilcox, 1974).

A number of studies have offered support for Mager's
learner-controlled raﬁner than instructor-controlled sequencing
(Horn, 1964; Briggs, 1968; Issing and Eckert, 1973; Newkirk, 1973)
although there have been exceptions (Allan and McDonald, 1966).

In many of these studies, though, the results are confounded wiﬁn
other variables which could have affected the results (Merrill, 1973).

The most interesting approach to sequencing, however, has
concerned itself wiﬁn ﬁne implications of the argument ﬁnat if a
subject matter has an internal structure, then ﬁnere should be a
logical teaching sequence consequent upon it. Gagne argues that

many subject matters have a hierarchical structure. What one has

36

to do to discover this structure, is to ask, "What does a learner
have to know in order to do ﬁnis task when given only the instructions
to do it?" An answer to ﬁnis question provides material to which
one can apply the same question, and so on, defining in a sense a
subset of skills, or more technically, a cumulative hierarchy of
sub-tasks . The phrase "cumulative hierarchy" simply implies that
the learner must be able to succeed at one level before he can go
to a higher one.

The results from recent experiments have not always
supported Gagne's approach (White, 1973), but the hierarchical
notion still continues to attract attention (Airasian and Bart, 1974;

Kozma, 1974; Phillips, 1974; White, 1974).

6. Criterion-Referenced Testingand Measurement

In a questionnaire administered to DURC, a series of
questions concerned the members' practices in grading. The results
showed that only one member utilized the philosophy of criterion-
referenced testing. A check of term grades posted outside offices
also demonstrated ﬁnat students' grades were closely along a normal
curve. Consequently, criterion-referenced testing is prominent in
the Manual, however, norm-referenced testing is also recommended

for certain purposes .

37

Criterion-referenced tests are specifically designed to meet
the measurement needs of irGtructional programs following a
systematic design. In contrast, the better known norm-referenced
tests are designed principally to produce test scores for ranking
individuals on the ability measured by the test.

A very flexible definition of a criterion-referenced test has
been proposed by Glaser and Nitko (1971): ". . . [a test] that is
deliberately constructed so as to yield measurements ﬁnat are
directly interpretable in terms of specified performance standards. "
(p.653). According to Glaser and Nitko (1971):

The performance standards are usually specified

by defining some domain of tasks ﬁnat the student should

perform . Representative samples of tasks from this

domain are organized into a test. Measurements are

taken and are used to make a statement about the

performance of each individual relative to ﬁnat

domain. (p.653)

Furﬁner distinctions between norm-referenced tests and criterion-
referenced tests have been presented by Ebel (1971), Glaser (1963),
Popharn and Husek (1969), Glaser and Nitko (1971), Hieronymous
(1972), and Livingston (1972).

Hambl eton and Norvik (1973) have noted that the primary
problem in criterion-referenced measurement is that of classifying

a student into one of several mutually exclusive mastery states or

categories. Mastery states are introduced to represent different

38

levels of performance on the domain of items measuring each
objective covered in the criterion-referenced test. Typically, a
cut-off score or mastery level score is set to permit the teacher

to assign students, on the basis of ﬁneir performance on each

subset of items measuring an objective covered in the criterion-
referenced test into one of two mutually exclusive categories -
masters and non-masters. (See Millman, 1973, and Block, 1972,
for disoussions of guidelines for setting cutting scores .) In ﬁne
Hambl eton-Novick formulation, criterion—referenced test

reliability takes the form of an index indicating the consistency

of decision-making across parallel forms of the criterion-referenced
test or across repeated measurements (Swaminaﬁnan, Hambleton and
Algina, 1974). Validity takes the same form except, of course, that

a new test or some other appropriate measure serves as the criterion.

The Manual advocates the use of criterion referenced tests when
evaluating student progress but asks the designer to consider norm
referenced tests for post tests . This method is advocated by Yel on
(1976) who points out ﬁnat employers need to know how students

stand in relation to each other.

7 . Formative Evaluation

 

A major feature of this author's model is the heavy emphasis
on formative evaluation. The Manual recommends ﬁnat formative
evaluation be conducted while the module is being developed . Such
evaluation provides information to developers that would allow ﬁnem

to modify and improve their product.

39

The inadequacy of current use of formative evaluation
procedures in the development of products has been well documented
by Komoski (1971). He notes that less ﬁnan one percent of all the
14,000 textbook titles being sold have been formally evaluated . Of
the 80 manuals on ﬁne design of courses which this author has
reviewed, only two show evidence of formative evaluation.

The heavy emphasis on formative evaluation in this author's
program is based on the belief that in order to assess ﬁne effective-
ness of instructional systems, a variety of data types need to be
utilized in ﬁne program .

Formative evaluation is conducted through the following:
embedded or in-program tests, pre and post tests, and attitudinal
surveys. Each of these will be treated in turn.

1 . Ennbedded tests have been found to be "very useful "
(Baker and Al kin, 1973).. Their major usefulness is to test whether
the student is mastering ﬁne sub-objectives (Dick, 1968), to give
constant feedback to the learner (Crowder, 1960) and as a diagnostic
function, indicating what particular discriminations need additional
attention. Glaser (1966) suggested that wiﬁnin-program errors
represented an inadequacy of the program itself. Recommendations,
based on research, on the sorts of data to collect in program develop-

ment have been forwarded by Markle, 1967; Baker, 1973; Dick, 1978.

40

A number of researchers have offered formative evaluation models:
Scriven (1972) and Stake (1972) imply that it is best that the formative
evaluation be performed by someone external to ﬁne program . Scriven calls
his approach "goal free evaluation" and establishes a rule in which the
formative evaluator collects his or her own data' and renders assessment
on the actual effects of the program .

Stufflebeam(1971) and Al kin (1969) emphasize the necessity of
structuring evaluation so that it serves decision making purposes by
producing appropriate and timely information.

There are numerous models and checklists for product development
processes . These vary from conceptual schemes (Schutz, 1970) to
comprehensive step by step procedural checklists (Borg and Hood, 1968).
All models strongly advocate the use of formative evaluation as stated by
Baker and Al kin, 1973:

" . . . at the core of each model, regardless of its esotericism or

practicality, is the realization and recognition that product

development and formative evaluation are intertwined as snake

and staff and that product revision depends upon the generation

of formative evaluation data."

The Manual synﬁnesizes most of the concepts and procedures

discussed here; these form Step 6 of the Manual which is enclosed

as Append ix D .

41

2 . Post test data. The form and frequency of gaﬁnering
post test data is not clear. Husek and Sirotrik (1968) and Shoemaker
(1972) have described a procedure to reduce the amount of testing
time requisite for program revision. Samples of test items are
administered to samples of learners and the sum across all items
is used as an index of program success.

The number and use of subjects is an important point.

Robeck (1965) used a single student as the data source and showed
that this economical method Significantly improved ﬁne product.
The above procedure has been verified by Fleming, 1973, and
Markle, 1967. The main problem was to determine which student
to select.

A variation of the single student procedure is advanced by
Abedor (1972) who used a small group com bined with a debriefing.
Baker (1973) commenting on Abedor's study points to an interesting
byproduct of ﬁne study: the analysis of ﬁne data suggests ﬁnat obtaining
feedback through procedures utilizing a student debriefing session may
serve as instructional product development training.

A number of studies have shown that when data from formative
evaluation is used to revise the product then considerable improve-
ment is achieved (VanderMeer, 1964; Gilbert, 1962). However,
no research has been conducted into ﬁne form that formative evaluation
should take and how early in product development should evaluation take

pl ace .

42

8. Modular Instruction

This general heading includes ﬁne concept of individualized
instruction through ﬁne use of modules . The effectiveness of
individualized instruction through ﬁne use of modules has been
demonstrated in a large number of studies (Celinski, 1968;
Ferster, 1968; Keller, 1968; Lloyd, 1969; Myers, 1970; Bern,
1971; Johnson, 1971; Kulik, 1974).

There is ample evidence in literature that modular instruct-
ion, be it a small part of curriculum or ﬁne whole curriculum,
is working. Brown et al (1976) describe the function of College IV,
one Of five Grand Valley State Colleges in Allendale, Michigan.

At this college, all courses are taught by a self-paced modular
system of instruction. The school has attracted large numbers of
non-traditional and adult students who cannot attend pre-arranged
classes. The College has developed a flexible administration to
allow for continuous registration and enrollment.

Bridge (1976) surveyed 43 modular, self-paced courses in
physical , social and applied sciences being taught in England and
Ireland. The results show an overwhelming approval of self-paced
modular approach both by students and faculty.

Taveggia (1976) presents an overview of 14 studies which

com pared learning outcomes of self-paced modular instruction and

43
conventionally taught courses. Student performance on course
content examinations showed ﬁnat modular instruction is superior
to conventional meﬁnods.

The above are just three studies. Journals such as Higher
Education, Educational Technology, carry a large number of
similar studies. A recent issue of ﬁne Journal of Personalized

Instruction (3,1 of Marcln, 1977) carried a list of 54 major articles

 

which give research findings supportive of modular instruction.
The basic theme of ﬁne Manual is that much of university

instruction should be presented ﬁnrough ﬁne use of well-designed

modules where the subject matter and situational constraints allow

ﬁnis (as outlined in Step 1 of ﬁne Manual).

9. Diffusion of Educational Innovations

The factors which encourage or impede the diffusion and
acceptance of innovations in education have been discussed widely
in literature (Lippitt, Watson, and Westley, 1958; Marcum, 1968;
Bennis, Benne, and Chin, 1969; Havelock, Huber, and Zimmerman,
1969; Rogers and Shoemaker, 1971). The kinds of factors that
students of planned change have identiﬁed as generally supporting
innovation in education include, after ﬁne outline of Glasser (1971):

organizational attitudes that support change; clarity of goal structures;

44
organizational structures ﬁnat favor innovation; professional ism of
staff; organizational autonomy; and strong vested interests in
preserving status quo methods of operation.
Of particular concern to this study is ﬁne use of a product

to encourage the diffusion and acceptance of an innovation. Under

 

the Statement of Need (p.6) it was stated that the Manual would also
serve as a change inducing mechanism . The author saw the Manual
as one of the means of diffusing an educational innovation (which in
this study is CBE) by having designers produce modules (visible
products). The author sees the availability and trialability of
products to be a necessary part of an acceptance process, because
the innovation (CBE) amounts to a conceptual approach to education,
made up of many concepts. The Manual transposes these concepts
into concrete visible products which can be demonstrated, tried

and tested.

The theoretical position for the author's assumption for ﬁne
need for concrete products is supported in literature. First, the
difﬁusion of an innovation is a process. This process is described
by Katz, Levin, and Hamilton (1972) as starting wiﬁn acceptance,
over time, of an idea by individuals in the systenn . Rogers and
Shoemaker (1972) also see diffusion of innovation as a process over
a series of stages. They see the process as starting wiﬁn awareness
of ﬁne innovation, then a Show of interest in ﬁne innovation, followed

by opportunities to evaluate the characteristics of the innovation,

45

then ﬁne innovation must be trial able before it is finally adopted.
Rogers and Shoemaker (1972) found that the acceptance of

an innovation depended on five characteristics or attributes of that

innovation as perceived by the target population: (1) ﬁne relative

advantage of the innovation over other methods or products, (2)

the compatabil ity of ﬁne innovation with ideas already held, (3) the

relative complexity of the innovation (a less complex innovation

has a better chance of acceptance over a more comm ex innovation),

(4) that the innovation be trialable and (5) the innovation be observable.
AS stated earlier, the conceptual nature of the innovation (CBE)

required that it should be presented in a trial able and observable

format. Rogers (1972) provides further guidelines as to ﬁne attributes

of an innovation which determine its acceptance; ﬁnis is further

discussed on p .

CHAPTER III

M ETHO DO LOGY

A. Type of Research

 

This dissertation is a pilot study. According to Borg and
Gall (1971‘) a pilot study is used to experiment with a variety of
approaches, ideas, procedures ﬁnus allowing an appraisal of
their adequacy, and also allowing unforseen problems to surface
(pp. 60-61).

The dissertation follows what Borg and Gall (1971) term as
"research and development" (R & D) methodology. This type of
research differs from most basic and applied research projects

1 that can

in a number of ways. Its objective is a finished product
be used effectively by a designated population. The most critical
difference between basic and applied research and educational R & D
is ﬁne sequence that is followed. The typical steps in the R & D
sequence are (after Borg and Gall, 1971, p.31 ):

1 . Develop a set of objectives that ﬁne product should achieve.

2. Conduct research or review previous research to discover

the deficiencies of current products and to identify approaches

 

The "product" in ﬁnis study is the Manual

46

47

ﬁnat are likely to overcome these deficiences.

3. Develop a product to the point where one may reasonably
expect that it will accomplish its objectives .

4. Test ﬁnis product and evaluate its effectiveness in meeting
objectives.

5. Revise the product on ﬁne basis of the field test results.

6. If it is successful, put ﬁne product into Operational use.
The R 8: D sequence proposed by Borg and Gall (1971),

was followed in this way:

1 . Develop a set of objectives that the product should achieve.

A set of cognitive objectives, followed by production
objectives, is presented on p. 58. The designer will acquire ﬁne
cognitive skills and ﬁnen transfer the cognitive skills to ﬁne actual
production of a module .

2. Conduct research or review previous research to discover

the deficiencies of current products and to identity approaches
ﬁnat are likely to overcome these deficiencies .

 

Much of ﬁnis is done in ﬁne Review of Literature (pp. 11-45).

and in the Statement of Problem (pp. 1-6)

3. Develop a product to the point where one may reasonably
expect that it will accomplish its objectives .

 

Research underlying ﬁne major components of the product

(the Manual) are presented in the Review of Literature. The meﬁnod

48

of develOping the Manual is described on p. 49. Development also
included rigorous formative evaluation.
4. Test ﬁnis product and evaluate its effectiveness in meeting
ogjectives.
Testing takes ﬁnree forms:
a. Formative evaluation of the Manual;
b. Evaluation of the products of the Manual (the module)
which is done ﬁnrough evaluation by a panel of expert
judges using criteria set out in Appendix A;
c. A field test of ﬁne modules as explained in Summative

Evaluation .

The final revision of the product, so that it can be put into

operational use, is not part of this study.

49

B. Development of ﬁne Self Instructional Manual

The process of developing the Manual was through four main
steps: a) determining ﬁne philOSOphical basis and format, b) identifying
content, c) identifying cognitive objectives, d) identifying product
objectives, and e) formative evaluation. The Manual is enclosed as

Appendix D .

1. Philosophical Basis and Format of the Manual

PhiIOSOphically, ﬁne Manual is to provide information in
possible techniques; the term "techniques" being used in its broad
sense as defined by Ellul (1967): technique is nothing more than
means and ﬁne ensemble of means (p. 19).

The Manual follows a philosophy which calls for the integration
of a systems approach to instruction with that of a humanistic approach
to education. Fox and DeVault (1974) found ﬁnat the best examples of
individualized instruction are those that blend these two approaches.
This approach is fully explained in ﬁne Manual .

In ﬁne vernacular of ﬁne target population, the Manual offers an
educational cafeteria of techniques instead of the more common
educationa "plat du jour". The auﬁnor does not suggest a radically
different view or urge the substitution of a new system for an existing

one. Instead, the Manual presents a varied array of choices to

50

achieve ﬁne same objective so as to accommodate ﬁne increasing
heterogeneity of those involved in education, an approach favored
by Fantine (1976). In a number of instances the Manual carries
suggested further readings for alternate approaches and views.
Its methods and approaches are in direct response to the needs
expressed by ﬁne target pOpul ation .

Competency based education, modular instruction, or individ-
ualized instruction constitute an innovation to most faculty in ﬁne
target population. Therefore the findings of Rogers and Shoemaker
(1971) are helpful on how to gain acceptance. Rogers and Shoemaker
advocate a gradual introduction of new concepts which sould be
perceived as compatible with existing methods and beliefs. Through
gradual introduction, the innovation should also demonstrate its
advantages, be trialable, and the process should be observable.
Consequently, the Manual introduces new concepts slowly, giving
concrete exam pl es from higher education, particularly dietetics,
and presents alternatives to choose from .

Generally the format of the Manual may resemble a programmed
text, but cannot be equated with Skinnerian meﬁnods. The ﬁneoretical
base for Skinner's (1954) views are severely criticised by Annett

(1969; 1973) and others .

51

The Manual itself is in a modular format; therefore it often
acts as an exemplar. Various techniques of presentation are used,
some for the purpose of illustrating alternatives, at other times
because ﬁney appear ﬁne most effective way of teaching .

The format of ﬁne Manual is print wiﬁn some graphs, flow-
charts and models. The use of print is considered the most
effective way to achieve ﬁne learning outcomes as advocated by
Brigg's (1978) method of selecting appropriate media.

Chunks of information usually vary according to the difficulty
of content but usually are large, in line with ﬁne target pOpulation's
capabilities as advocated by Holland (1965) and Leith (1966). As
Markle (1969) points out, programs of today contain various step
Sizes; the size of ﬁne step is largely determined by what is
corsidered appropriate for ﬁne learner, and the nature of ﬁne
material.

The Manual used mainly linear techniques, but some branching
is used for accelerated progress and remedial work. Holland (1965)
and Leith (1966) showed ﬁnat when comparing linear and branching
programs for older, intellegent learners, there was no difference
in test results, but branching programs were superior in terms of
time taken to learn.

Frequency of tests is varied in ﬁne Manual . Research by

52

Holland (1965), Anderson (1967) and Annett (1969) Show that low
ability learners benefit from immediate knowledge of results.
For above average learners, testing can be varied.

Attention needs to be paid to the nature of ﬁne subject matter
being learned (easy - difficult), the kind of response being made
(overt - covert), its frequency, and ﬁne kind of feedback deemed
appropriate. A study by Grundin (1969) indicates how these features
may interact. In Grundin's study, overtresponding produced better
results than did covert responding, but ﬁne older the learners, ﬁne
less the difference. However, in his study, the effect of overt
responding interacted wiﬁn the frequency of feedback. In ﬁnis
experiment, with overt responding, frequent knowledge of results
seemed superﬂuous, or even detrimental .

While immediate knowledge of results does not always lead
to increased learning, this does not mean ﬁnat the technique of
providing immediate feedback is always erroneous. Indeed,
providing knowledge of results is probably one of the most important
principles of teaching (Annett, 1969). In the context of programmed
learning, Annett (1973) says, " Reinforcement of each step? Well
you can hardly argue with that. One does not have to be an ardent
S-VR theorist to see that at some stage the student must be aware

of what the right answer is . "

53

Annett's argument is with the reinforcement interpretation
of this principle (Annett, 1969; 1973), not with the idea of frequently
telling a student how well he is doing. Information about the adequacy
of a response is more important than reinforcement. Consequently,
in the Manual answers are often lengﬁny, providing a "suggested"
or "expected" answer.

Research on overt versus covert responding produced similar
mixed results. In many cases no Significant differences were
found between the test results of learners who wrote their answers
down or learners who merely "thought" them . Again, however,
there were indications of where overt responding was superior -
for example (i) when ﬁne learners were young children; (ii) when
the material was difficult; (iii) when the programs were fairly
lengﬁny; and (iv) when particular (novel or specific) terminology
was being taught (Holland, 1965; Leith, 1966).

A more careful analysis is required of what is meant by the
phrase "active responding". AS Annett (1973) wrote, activity on the
part of the learner seems a good idea, but what sort of activity?

It is important to note that with intelligent learners covertly
responding to programmed instruction there is still far more
questioning of the learner than there is in a conventional teaching-

learning situation. There is evidence in oﬁner instructional contexts

54

to indicate how important this questioning might be (Glaser and
Resnick, 1972; McKeachie, 1974; Prosser, 1974).

There are times when the Manual suggests that for certain
types of learning there be a departure from individual learning to
pairs or even groups. Several experiments have been performed
which have conn pared ﬁne results obtained from learners working
in pairs, in trios, in small or even in large groups (Hartley, 1968).
In the majority of cases no significant differences in test results
have been reported. However, for learning outcomes which deal with
ﬁne affective domain, group interaction has resulted in better
clarification of values and more lasting learning.

The Manual also suggests that at times ﬁne instructor and
modular learning be integrated. Hartley (1972) gives research
evidence that the instructor and module working together provide
a more efficient teaching technique than eiﬁner working in isolation.

Individualized learning is usually self-paced, and that is also
the meﬁnod advocated in ﬁne Manual. However, ﬁnere are reminders
throughout that a steady pace needs to be maintained, for as Landy

et al (1969) points out, the rate of work expands to fit ﬁne timetable.

55

2, Identification of the Manual's Content

The need for the Manual . is based on the Statement of

 

Problem (p. 1 ) and ﬁne Statement of Need (p.5 ) earlier in this
disse rtation .

To meet the stated needs, the Manual must serve two
functions. First, provide the cognitive skills necessary for
the construction of modules. Second, provide ﬁne step by step
guidelines for designing and producing a module.

For the first function of the Manual (that of providing the
necessary conceptual information) ﬁne content was identified
partly through the use Of Gagne's (1975) hierarchical analysis,
which (paraphrased) states:

What simpler knowledge would a learner have to
possess in order to learn skill x, the absence of which
would make it impossible for him to learn skill x.

That is to say, in conducting such an analysis, one

seeks to identify essential prerequisites, those sub-

ordinate skills which are actually incorporated into

the skill to be learned.1

A full account of ﬁne hierarchical analysis which followed

 

1 Research on Gagne's hierarchical analysis is in the
Review of Literature.

56

is too lengthy for inclusion here, however the main required
intellectual skills which were identified included the concepts
associated with systems approach, mastery learning, criterion
referenced evaluation, specification of learning outcomes and
hierarchical analysis of cognitive skills.

The format and amount of content presented was partly
decided on ﬁne evidence provided by questionnaires and quizzes
which faculty completed during DURC Learning Sessions
(mentioned in Statement of Problem) as well as by the questions
asked by faculty when interviewed in the summer of 1976.

The second function of ﬁne Manual is to provide quidelines
step by step, for ﬁne design and production of a module. One very
useful conceptual model was developed at Florida State
University and presented on the next page (Figure 1). The Florida State
Model has been in use for nearly ten years. The author's main
criticism of this model is that its concurrent activities are confusing
in a self instructional manual. Some of ﬁne other models considered
were ﬁne IDI1 and Learning System Design (Davis, Alexander and
Yelon, 1974). The IDI, with its nine major steps appeared too
task oriented and lacked sufficient guides for product development.

From the three models just mentioned, as well as a number of

 

Instructional Developnn ent Institute of National Special
Media Institute (1972).

57

 

 

 

 

 

 

lmntvmg
IMNUCIIOMI
Goal _

Canducnng

instrucuonu

Anal 1m:

 

 

 

 

 

 

 

 

 

 

Fig. 1 -- Systems Approach Model for Designing Instruction from

 

launching

Entry acumen.

CMIOCICHSUC!

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Rmmmg
lMlmctIon
r
Wmmt Dcwloomg OONIOOmg 9.9.1001” Douqnmg and
Performance C'Itcnon- Instrucuonal —. and 5.1mm, n—a— Conducting
Oblocmu Rohnncoo Strum IMVUCNOO FOM‘OWO
Tom EVIIUIIIOO
'1 " —J I
Dos-90mg and
Conducting
Summon”

 

Evaluation

 

Dick and Carey, The Systematic Design of Instruction,

Scott Foresman and Co.

others the author developed a conceptual model intended to meet the
particular needs of DURC. The seven step model is presented on
the next page (Figure 2) followed by the objectives for each step.

It was primarily from this conceptual model that the content of the

 

Manual was derived.

 

 

58

 

 

 

 

 

Step 5

Step 6

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Construct
lesson

 

 

 

Fonrnative

evaluation

- one to one

-small
group

- field
evaluation

 

 

 

Step 1 Step 2 Step 3 Step 4
Develop
Specify WW9 criterion Plan lesson:
F... learning - learning tests motivation
outcomes outcomes - entry level techniques
- pretest media
- embedded
- post test
- attitude
Step 7
Recycle

 

 

 

 

Fig , 2 -— The M . S.U. Dietetic Model
The Model is named the M . S.U . Dietetic Model as it

was developed largely ﬁnrough support and co-operation
provided by the M. S.U. Dietetic Faculty.

3. Cognitive Objectives of the Manual

Step 1. Specify Learning Outcomes

Be able to identify learning outcomes which meet ﬁne
criteria for initiating the development of a module.
Be able to state learning outcomes for a module of

your own choosing. The learning outcome will meet

the major criteria required for initiating the development

of a module .

 

 

59

Step 2. Analyze the Learning Outcomes

Be able to identify and describe procedural , hierarchical
and combination approaches to instructional analysis.

Be able to describe the relationship among the tasks which
are identified through an instructional analysis.

Be able to apply instructional analysis techniques to identify
subskills required to attain specified learning outcomes.

Be able to describe entry behaviors and distinguish them
from general characteristics of students in a target

population .

Step 3. Develop Criterion Tests

Be able to identify the purposes for entry tests, pretests,
embedded tests and post tests.

Given a variety of learning outcomes, be able to write
appropriate criterion referenced test items which reflect
the learning required of the learner as stated in the learning

outcomes .

Step 4. Plan Lesson
Be able to identify and describe the major components of an
instructional strategy.

Be able to develop an instructional strategy for a set of

objectives for a particular group of learners.

60

Step 5. Construct Lesson

Given an instructional strategy, be able to describe the
procedures for developing instructional materials .

Be able to develop instructional materials based on a

given instructional strategy.

Step 6. Formative Evaluation of the Lesson

Be able to state the purpose of formative evaluation.

Be able to state the purpose of summative evaluation.
Be able to state the procedures of formative evaluation.
Be able to state various methods of collecting data for

fo rm ative evaluation .

Product Objectives of the Manual

 

You will write an instructional goal for a module of your
own choosing.

You will identify the skills required to reach your instruct-
ional goal through the application of the appropriate
instructional analysis technique.

You will derive the entry behaviors from your instructional
analysis .

You will write performance objectives for the skills which

have been identified in your instructional analysis.

10.

11.

61

You will write apprOpriate criterion-referenced test items
which match the behavior required in your behavioral
objectives.

You will pl ace your identified subskills, behavioral
objectives, and criterion-referenced test items in a chart

to enable the evaluation of your design. In addition, you

will draw an instructional analysis diagram which illustrates
the sequence and relationship of your subskills . The
components of the instructional analysis should be numbered
to correspond to the subskills in your evaluation chart.

You will describe the instructional strategy for your set

of objectives and target population.

You will develop a module based upon your instructional
strategy.

You will develop an appropriate formative evaluation plan
and instruments for your instructional module.

You will identify the problems and revise your module based
on the data from your formative evaluation.

You will write a report which describes the design, develop-
ment evaluation and proposed revisions of your instructional

modules, according to the guidelines provided in this Manual.

62

5. Formative Evaluation of Manual

This evaluation was carried out along the guidelines suggested
in Step 6 Of the Manual presented in Appendix D.

Formative evaluation of the Manual was conducted in three
phases. First a one to one phase; second, a small group phase;
and third, one extended small group situation.

The one to one testing was conducted with three subjects:

a graduate student from Instructional Development and Technology
Department, a graduate student from Food Science and Human
Nutrition and an instructor from Food Science and Human Nutrition.
The form of the testing was for the author and the "l earner" to go
through all the components word by word. The learner would
comment on anything which seemed unclear and the author noted

the comment and recorded the possible alternatives. As a result

of this phase, the Manual was extensively rewritten.

The extended small group phase was conducted with DURC
starting August 1977. Participants were asked to work through a
segment of the materials and make comments and suggestions prior
to coming to the weekly meeting . At the meeting participants
expressed their reactions to the materials and took the post test.
All the materials were collected, comments considered and tests

eval uated .

63

After revisions based on this phase were concluded, it
seemed that the Manual could be Put to its intended use. In
addition, a Masters' candidate (Morrissey, 1978) in the Depart-
ment of Food Science and Human Nutrition selected the Manual as

ﬁne tOpic for her Unesis .

(3. Method of Pilot Testing: An Overview

Pilot testing consisted of an evaluation of the modules and
the reports which were produced by the designers as a result of
working through the self instructional Manual . These reports
provided documentation on the processes which went into developing
the module and were thereby helpful in the evaluation of the module.
Guidelines as to what should be included in the report are stated

in the Manual .

1 . Sarmale Selection

Instructors in higher education who expressed interest in new
methodology to improve their teaching were approached at random
to see if they were interested in developing a module. The instructor

needed to be in a field other than education and not have had formal

54

training in instructional design. The interested instructor
(designer) was given a copy of the Manual and encouraged to
follow the directions. The designers were fully briefed as to ﬁne
whole process and what the process entailed in tenms of resources.
It was agreed earlier that ﬁne minimum number of instructors
to be included in ﬁnis study should be at least five while ﬁne actual
number turned out to be seven.
Designers were geographically dispersed with some in
Australia and others in Hawaii; this prevented input or 'contamin-

ation' by the author.

2. Data Collection

 

Each module was evaluated by different pairs of instructional

designers who were either faculty or advanced degree students

in Instructional Development and Technology at Michigan State
University. The evaluation consisted of assessing each module
with respect to each of 26 questions developed by the auﬁnor.

Each question was worth a stated number of points for a total

of 100 and each question had a stated criterion level. Points

were awarded by each evaluator as outlined in ﬁne rationale

which accompanied the evaluation instrument. (The evaluation
instrument and rationale for awarding points are appended as

Appendix A.) Development of the 26 question evaluation

65

instrument, rationale and objectives was started by the author as
partial requirement for IDD 633 at Florida State University with
Dr. L. Briggs, as instructor. At ﬁne time, modifications had

to be made to ﬁne evaluation instrument and ﬁne rationale to

bring about agreement with the stated objectives. Since that
time, changes have been made to the objectives and consequently
to ﬁne evaluation instrument. In April 1978, the author visited
Florida State University and took the objectives and evaluation
instruments to Dr. W. Dick, another author in the field. Dr. Dick
suggested several changes and also gave a c0py of an evaluation
instrun ent which he has developed, for possible further develOp-
ment of ﬁne auﬁnor's evaluation instrument.

Inter-evaluator reliability was determined by comparing the
judgements of ﬁne two evaluators who independently reviewed each
module and its accompanying report.

Evaluators were guided by the rationale which accompanies the

criterion referenced evaluation instrument.

3, Attitudinal Questionnaire

 

Each designer was asked to complete an attitudinal questionnaire
at ﬁne completion of ﬁne study. The purpose of ﬁnis questionnaire
was to guage the designer's attitude to the whole process of learning

through ﬁne Manual, and the design and testing of ﬁneir module.

66

Questions of particular interest are: would ﬁney design another
module; if ﬁney designed anoﬁner module how long would it take in
relation to their first attempt; would ﬁney recommend this method
to a colleague; has ﬁnis experience changed their attitude and

methods to ﬁneir everyday teaching? A copy of the questionnaire

is in Appendix E.

4- Summative Evaluation

Summative evaluation, e.g. experimental comparison of
modules wiﬁn conventional instruction, was not included in ﬁnis
dissertation due to the extensive logistics involved. Designers
were not able to carry out their summ ative evaluation and report
their ﬁndings before the deadline set for ﬁnis dissertation.

However, the summative evaluation of one module (Morrissey, 1978),
which was produced as partial requirement for a Masters' degree,
is reported in Chapter IV and Appendix C as a possible example

or for replication studies.

CHAPTER IV

DESCRIPTION AND RESULTS OF
PILOT TESTING OF THE MANUAL

The purpose of this chapter is to report the results of the
pilot testing of the Manual. The chapter is divided into two

sections, 1) description and 2) report of findings.

Description

The purpose of ﬁnis pilot study was the evaluation of the Manual.
As outlined in Chapter III, this was done through the evaluation of
six products (modules), designed along the conceptual, information
and step by step procedures provided in the Manual. Evaluation of
each module was carried out by two competent instructional
designers using a criterion referenced instrument (enclosed as
Appendix A) and following a set rationale for the awarding of
points (also in Appendix A).

The designers were instructors in higher education, three
in Australia and three in Hawaii; each designer produced one
module. Choosing designers geographically distant from the

auﬁnor controlled for author "contamination".

67

68

Designers in Australia were afﬁliated with the Caulfield
Institute of Technology in Melbourne and ﬁne designers from
Hawaii included a former professor attending the University
of Hawaii as a graduate student, an instructor at the University
of Hawaii, and a former professor now working as a consultant,
living in Honolulu.

The designers in Australia were contacted when the auﬁnor
visited Australia in December, 1977 and ﬁne designers in Hawaii
were contacted while the author worked at the University of Hawaii
in January to March, 1978.

A designer was accepted when in conversation it became cl ear
that he/ she had very little, if any, training in ﬁne theory or methods
of education. This lack of education background needed to be later
verified by ﬁne responses to a pre entry test which the prospective
designer was asked to complete. This pre entry test was essentially
the same as used in DURC except that questions 10 and 11 were
omitted, and questions 3 and 4 were changed to percentages for the
prospective designers in Australia because grade points are not
used there. Questions 3 and 4 were also marked as not applicable
to ﬁne prospective designers in Hawaii who was working as a
consultant. The pre entry test is enclosed as Appendix B; it was
still titled "Survey on CBE" as ﬁne auﬁnor considered such tenms
as "test" or "examination" to be inappropriate with the prospective

designers .

69

Results of the pre entry test or "Survey on CBE" are also
found in Appendix B. Suitable prospective designers were given a
copy of the Manual (as in Appendix D) and asked to work through it.
Designers were instructed to keep in touch to report ﬁne progress
made on the module. Completed modules and reports were
expected to be wiﬁn the author by June 30, 1978.

The first module arrived in late May and the last module
towards ﬁne end of July. Designers were then sent a Post Module
Development Survey which they were asked to complete and return
to ﬁne auﬁnor. This survey is enclosed as Appendix E.

When a module arrived, the author sought out competent
evaluators. The evaluation was carried out by three advanced
degree students and four faculty, all associated with Instructional
Development and Technology at Michigan State University. No
evaluator was asked to evaluate more ﬁnan two modules and for
each module that was evaluated by a graduate student ﬁne oﬁner

evaluator was a faculty member.

7O

Findi s

Altogether there are six findings which can be supported by
data. Findings 1 ,2, and 3 relate directly to the research questions
posed in Chapter 1:

1 . Can a self instructional manual on designing modules
be developed where ﬁnose who complete the manual reach mastery
level on its contents?

2. Will attainment of mastery of ﬁne self instructional
manual result in a module which meets some minimum acceptable
standards?

3. Will the module so developed obtain results at least
as good as those obtained ﬁnrough traditional methods of instruction?

Findings 4,5, and 6, while not directly related to the above

research question, are, neverthless of interest.

Finding 1: Mastery of Key CBE concepts was acquired through
ﬁne Manual.

Data relevant to this finding is presented in Table1 . Key
concepts, considered as essential for ﬁne design of a module had
earlier been identified (p. 21 ). Mastery of ﬁnese concepts was

operationally defined as scoring above criterion on ﬁne 26 item

71

evaluation instrument (in Appendix A) on which the modules and
accompanying reports were evaluated.

Perusal of Table 1 indicates that the designers achieved
considerable mastery of key concepts as measured by the modules
actually produced. Of ﬁne 312 evaluations made, only nine were
below the criterion level. However, each of the nine below

the criterion level, judgennents were made by one evaluator but

not the oﬁner evaluator. From personal interviews with evaluators
it seems that the "fail" judgements were either because of ﬁne
ambiguity of the rationale given in ﬁne evaluation instrument or

that the item did not seerm applicable for ﬁnat particular module

or its intended users. For example, item VI A, preinstructional
activities, was not fully addressed by two designers who maintained
that they knew their learners to be highly motivated and therefore
saw the need for only minimal amount of motivational preinstructional
activity. In both of ﬁnese cases one evaluator rated ﬁne item 0 but
the other evaluator awarded at least the criterion level of points

as a sign of accepting the designer's rationale. A close study of all
of ﬁne items which were judged as not reaching the criterion level

revealed that the designer understood ﬁne concepts involved

72

but considered their use in the module as inappropriate.
As mentioned earlier, the designers chosen for ﬁnis pilot
study had very little knowledge of key CBE concepts. From
the data in Table 1 it is cl ear that the vast majority of key concepts
haVe been mastered by all designers and thus it is suggested ﬁnat these

key concepts can be and were acquired primarily ﬁnrough ﬁne Manual.

Finding 2: The Manual led to development of acceptable modules
A total of six modules were designed. All six modules
reached ﬁne criterion level; findings are reported in Tables 1 and 2.

It is reasonable to conclude that the Manual led to the development
of acceptable modules as all designers had stat ed at the outset
that they had no knowledge of how to design modules and their
knowledge of key concepts was very low (see Appendix B). The
question which remains unanswered concerns the amount of
assistance ﬁne designers required and received apart from the
information provided in the Manual.

Judging frorm the evaluation questions sent to the auﬁnor, four
stated that ﬁney required further reading of ﬁne recommended texts,
particularly Gagne (1975); three stated they required no further
readings. No designer reported having sought any consultative help

from anyone .

73

There was one deficinecy in the Manual expressed by the
designers. They indicated the need for a finished product as a
model of what was expected; this requirement was expressed by
all designers. As the designers were the first group to produce
modules there were no finished products avail able as models.

While there were no finished products which met the criterion set
down, there were some which in some ways resermbled the required
product. The first such module was the one developed by the author,
called 53921 which is on Control Data Corporation's ﬂatg system .
The designers in Hawaii had access to this program but the ones

in Australia did not. To fill ﬁnis need for models, the designers
were referred to Briggs (1977) where ﬁnree modules are included.
However, those modules had been developed on a different format
and therefore would have only been of minimal help to the designers
(perhaps even confusing).

However, the point was made that the designer experienced
certain (unnecessary) anxiety which could have been avoided by the
inclusion of a completed product. The absence of a finished product
as a model for the designers in Australia seermed to have very little
effect on the quality of their modules as the evaluations were equival-
ent to the evaluations of the modules designed in Hawaii where the

designers had ﬁne Opportunity to see a module.

74

Docs. Tove: otm cowcmﬁto 26wa mmcoom

 

. mamﬁumuma
Hmoowuuauumaw you
m>m£ mmmzu m:0au¢u«aoaw
pom dowumasooo umwumu
mo monumﬁumuumumno
Hmumcmw mo coauoauummp .<
coaumasoom
umwumH mo coauoauommo

.HHb

 

CH m

CH

N mHmHumuma smacks» pmcumma

on cu wHHme mm pwpoaodﬁ

amonu mam whoa>mnmn muudm

OH

on pmvdaudw mHHﬁxmnom

comaumn coaumuwmaumau .Q
. . .mHHwanom wcoam oﬁnm
Idowumawu mo doauoﬁuomom .u
.mHHaxmnom mo
amuwmwp mo mmwomuwaoaou .m
.pmuomaow

SmOHOponuwa How mam:owumu .<

mammamo< HodowuoouumaH

.HH

 

 

 

 

 

 

 

. . . . . . . HnmlemGOHu
nusuumow mo upmamumum .m

. . . . . . . . . . AHmow

Hmaoauuauumaﬁ wdﬁuumamm

wow wow: mauwuauUV
manpoa Haw mHmaowuwu .<
Hmoo HddOHuusuumoH

.H

 

~.>m H.>m

 

N.>m H.>m

 

Q

~.>m HSm

 

N.>m H.>m

U

 

m

~.>m HSm

N.>m H.>m

 

<

 

 

mmADQOZ

 

Hm>wa
Goa

lumuﬁuu

 

mudﬁom,

wamz

 

mmaapoz mo coaumoam>m P manna

 

75

our; Toot: mLm cowLmﬁLo 266p mmgoom

 

(“N

NQ

F4
«ncﬂ

o'm

NM

<fN
m
m
M

Nq
N<f
NQ

m

(Or-l

Him

0

<TN

N<f

mamaumuma
spas uamduwcou ammumuum .m
mmwuﬁ>wuum canounul3oaaom .n
.wmauH>Auum
coaumoﬁoﬁuumo ucwpsum .0
.GOHumuowmmuo coaumauomcw .m
.mmﬁu
IH>ﬁuum HmGOHuosuumaamuo .<
mwmumuum Hmooauu=uumcH

O O O O O O O O I O

.H>

 

. . Aummu phonemmunlwuudw
.umou poppmnﬁw .uwmulumoo
.mummumuov Goaumaoooo
umwumu pom mamﬁumuma coo:
momma mummu oumﬁuoouoom .U
.vmuuauameu saummau
musmeuomumo mo Hm>mH

mumumma no Hm>ma cowumuwuu .m

.. mwaouudo maﬁaumma
cu memo“ mo oﬁzmdowumamu .<
momma pmucmummmmnaowuwuauo

 

 

 

 

 

 

 

 

waoouoo
woﬁcumma ago we unmamumuw .m
. mamhamom Hmcowu
nonhumaw Scum coaum>ﬁump .<
mwaouuoo ononmA

.>H

 

~.>m H.>m

m

~.>m H.>m

 

m

 

~.>m H.>m

n

 

U

~.>m H.>m

 

N.>m H.>m N.>m H.>m
m <

 

 

 

mmADQOZ

 

Hm>wH
cow

muoﬁoq
Boa

luouwuo Iaxmz

 

.e.uaoo F magma

 

76

omchopc: mLm cowtmﬁco >6qu mmLoom

 

olwqu «1
[\

NNN 00
O)

NNN 00
O)

FiFqu c:
m>

Hv-ir-il m
h-

A<HOH

. . . . . . . .mmuopmoouo .0
O O O 0 O O O O O O mummu 0m
0 O O O 0 O O O mﬁﬂumug O<

madamﬁ>mm pmummwwom.HHH>

 

 

 

 

 

 

 

M

. . . . . . . .NNHAHWHG
pom humEEDm pump .3
. . . . . . coauumaaoo
dump you new: mmuapmu
loud pom muowaouumoﬁ .m
. . . mpoum coauomam>m
m>wumauom mo owwmmp .N
. . . . . . .muaumﬁumu
lumumno docum mHoewm .H
mmuspmuouo
coaumsam>w docum
IHHmam mo coauoﬁuommp .u
. . . . . . . . . wﬁﬁummu
maeloulmdo coo: comma
moowmg>mu pom wuasmwu .m
. . . . . . . . . nonstop
loud coauumaaou pump

MGOIOUIODO MO GOHUQHHUWQV .<

coaumdam>m m>HumEuom

.HHP

 

N.>m H.>m N.>m H.>m
m m

 

 

N.>m H.>m N.>m H.>m

 

a

U

 

m

N.>m H.>m ~.>m H.>m

 

<

 

 

Immense:

 

Hm>wH
so“

lumuwuo

muoﬁom.
ESE

 

Iﬁxdz

 

.p.u:ou F manna

 

77

 

 

 

hm mm mm? mm» m
cm mm wow wow m
mm mm wow wow 0
Ow am wow wow 0
mm mm mm» mow m
mm ow wow wow <

N H N H

HoumDHw>m Hounde>m mﬁomoz
Dmptmgm accoumuﬁuu mnu

mucaoo Hmuoa

umma 01602 23 3o

 

.Hm>mH cowuwuﬁuo mnu umE manpoz ecu we poxmm mama mucumaam>m

amEEDW <Ilmmazpoz mo coaumaamawm I N manna.

78

Finding 3: There is some evidence ﬁnat students taught through
a module produced fronn ﬁne Manual would perform,
on the average, as well as those taught by a traditional
lecture method .

Due to complex logistics, none of the designers overseas
were able to complete an experimental comparison of their module
in time to be reported in this study. However, one module was
experimentally tested1 . The designer was able to arrange
three experirmental situations where equal ability students were
divided into two groups. Students in the control group were taught
using conventional lecture methods while students in the experimental

group were taught the sarme topic in the form of a module. A full

account of the meﬁnodology used and the results are reported in Appendix C.

Both groups (experinn ental and control) in all ﬁnree experiments
showed considerable gains. Table 3 is presented to illustrate this
gain, other tables showing pre and post test differences are presented
in Appendix C. When the post test means of the control groups
(traditional lecture method) were com pared with the post test means
of ﬁne experiemtal groups (the module) in each case the means of
the experimental group were higher; however ﬁne differences were
not statistically significant at the . 05 level .

In this experirment ﬁne students taught through the module
received, on the average, as good if not better results as the

students taught by the conventional meﬁnod.

 

This module was designed to satisfy ﬁne requirements of a
Master's Thesis. The forrmative evaluation of this particular
module is not reported in this dissertation because it went through
evaluation processes required by ﬁne designer's committee which
were different to those required for ﬁnis dissertation.

79

2—60. v d an u:d0w.uw:m.mmw

 

 

~NN.N mNm. mac.N pouho pumpcaum
mom.m~ mNN.m woN.m~ :oﬂumw>op pumpcaum
muoo. um NN.- moom.um moov.mm ooom.~c cam:
Amm u 2C mango:
now.N o~n.~ muv.N gouge duodenum
NNN.H~ HNo.o «mm.o :oﬁuma>op duodenum
mace. «A mo.w NooN.mN mnmm.cm NooN.No cam:
Hmu u 2C unsound
a we oaum> Aumououa umouumoa ammuoua ozone
e -umouumoav
mucouommMo

 

8ka Smmmttoév .3523

owuouoﬁo «nose: up xuwmuo>ﬁca 0:» use zuwmuo>wca cumum :mwﬂguwz How mucoEumoHH o~=p¢2 pea
ousuuuu :a museum coo: ammucouuoa amouumoa use amououa coozuom oucohommﬁo now «mouth mo mu~amo¢uu.mwo~awb

80

Finding 4: The Manual promoted a positive attitude toward CBE
and module design.

When asked in the Post Module Development Survey (Appendix
E) if ﬁney would design another module following the Michigan State
University model, all designers answered yes. A further question
in ﬁne same survey asked: would you recommend ﬁnis model to
anyone else? The answer again was yes from all designers. Wiﬁn
these positive findings and an absence of negative findings, it may
be concluded that the Manual develops positive attitudes to CBE

and module design .

Finding 5: Designers reported that ﬁne Manual had affected
their method of teaching.

When designers were asked in ﬁne Post Module Develooment
Su rvey’if designing ﬁneir module had changed their way of teaching,
the answer was yes. Three designers reported that ﬁne experience
had greatly changed ﬁne way they teach while the oﬁner three designers
reported ﬁnat their method of teaching had not changed a great deal

but that ﬁney approached the process of teaching in a different way.

Finding 6: The diffusion and acceptance of CBE concepts by
DURC was partly attained through the co-operative
developrm ent of ﬁne Manual .
As reported earlier in this dissertation, there had been

resistance to CBE concepts by DURC; it was hoped that the co-operat ive

 

* Question , Appendix E

81

develOpment of the Manual would overcome ﬁnis resistance. While
the evidence is not tangible or measurable, ﬁnere are signs that
CBE concepts have become more acceptable to members of DURC.
The supporting evidence is as follows:

a. One member of ﬁne group has completed a module
(following the Manual) which has been summ atively evaluated.

b. One module is being designed for ﬁne purposes of
a Doctoral Dissertation.

c. Three instructors are in the process of designing
modules for their own use wiﬁn learners.

d. Seven members of the DURC group are presenting
the Manual as a workshOp at ﬁne annual convention of the American
Dietetic Association.

e. The Manual has been cited in a number of presentations

at conventions by members of DURC.

In the next chapter (Chapter V, Summary and Conclusions)

these findings are further discussed.

CHAPTER V

SUMMARY AND CONCLUSIONS

This concluding chapter has four sections (a) summary of

the pilot testing of the Manual, (b) major conclusions of the study,

1

(c) heuristics related to ﬁnis study and (d) recommendations for

further research .

Summarlof the Pilot Testing of the Manual

The purpose of ﬁnis study was to see if a self instructional
Manual could be developed, for a broadly defined group, so that
mermbers from that group could independently design modules for
use in higher education where higher level cognitive skills were required.
At least five modules were to be designed. The completed modules

were to be evaluated on a criterion-referenced instrument. Designers

___

By the term "heuristic" ﬁne auﬁnor takes to mean findings
based on evidence of experience from the conduct of this study,
which may need further verification by more scientific methods.
Some of ﬁne conclusions need to be presented as heuristic because
(a) the small N upon which these generalizations are based do not
allow for more definitive statements and (b) these heuristics are
worﬁny of furﬁner consideration by other instructional developers.

82

83

were also asked to complete an attitudinal questionnaire on their
perception of the effectiveness and usefulness of ﬁne Manual as
well as their reaction to the whole process of designing a module.
A total of six modules were developed for ﬁnis study, three
in Australia, and three in Hawaii. A furﬁner module was developed
at Michigan State University; it was designed and tested as partial
fulfillment of a Master's Degree and was not evaluated on the
criterion-referenced instrument and therefore is not reported in
Tables 1 and 2 (pp.74-77). However, as the module was tested

experirm entally, those findings are reported in Chapter IV and Appendix C.

Conclusions of the Study

Conclusion 1: INFORMATION PROVIDED IN THE MANUAL
RESULTED IN THE DESIGN AND PRODUCTION
OF SATISFACTORY MODULES.
Of concern to educators is the frequent inability of the learner,
who has demonstrated possession of certain cognitive knowledge, to
transfer such knowledge into practice. The designers in this pilot

study cl early transferred ﬁne cognitive skills they acquired from

the Manual to ﬁne design and production of their module.

84

Conclusion 2: HIGHER LEVEL COGNITIVE SKILLS WERE
TAUGHT THROUGH THE MANUAL.

The cognitive skills to be mastered in the Manual ranged
from low level discrirmination to higher order rules. Designers
demonstrated acquisition and capabilities of these skills by
designing and producing modules where each module included
higher order rules. By judging ﬁne evidence provided in ﬁne
designer's formative evaluation of ﬁneir own module, it was

cl ear that their subjects also reached mastery of higher order

rules.

Conclusion 3: THE MANUAL WAS FOUND TO BE USEFUL
IN MEETING THE NEED FOR WHICH IT WAS
DESIGNED.

Common criteria for assessing ﬁne usefulness of an educat-
ional product such as the Manual is to question its validity, feasibility,
and effectiveness .

1 . Val idity—Did the Manual do what it was supposed to do?

The Manual would be considered valid if (a) designers were able
to gain knowledge of key CBE concepts and infonmation from it

and (b) if those concepts would result in an acceptable module.

85

With respect to ﬁne criterion of validity, evidence in this
pilot study indicated that designers did gain cognitive knowledge
of ﬁne key CBE concepts. Further, all designers produced modules
which exceeded set criteria. On ﬁne evidence avail able, the Manual

was val id .

2. Feasibility--The Manual would be considered feasible
if five or more college level instructors, wiﬁn only minimal know-
ledge of CBE, and geographically distant from ﬁne auﬁnor, would
complete modules which met ﬁne criterion level.

With respect to ﬁne criterion of feasibility, evidence in this
pilot study showed that six college level instructors, all meeting the
stipulations of this study, began and completed their modules, and
all met the criterion level. Tlnerefore, for ﬁne conditions set out

in ﬁnis pilot study, the Manual can be considered feasible.

3. Effectiveness--The Manual would be considered
effective if (1) designers reached mastery level on key CBE concepts
and (2) the modules Which ﬁney designed and produced would meet

a criterion level .

86

Evaluation indicated that on (1) there were 52 evaluations
made on each designer and 312 for ﬁne group of six. Of the total
of 312 evaluations, nine were below the criterion level. Of these
nine, some judgements may be reversed because of dubious
validity of the test item . Because of the large number of positive
evaluations (303 or more) ﬁne Manual can be accepted as effective
wiﬁnin the design of ﬁnis pilot study as enabling designers to reach
mastery level .

Evaluations on (2), that the modules would meet a criterion
level, were all positive. Consequently, ﬁne Manual was effective

in this pilot study.

87

Heuristics

As a consequence of the development and pilot testing of
the Manual ﬁne auﬁnor formulated certain heuristics which may be
of interest to other instructional developers. The heuristics mentioned
here are only a sample (in the author's view, the most important ones).
However, a large number of heuristics emerged throughout the form-
ative evaluation stage of the Manual which is not a part of this pilot
study and therefore not reported here.

Heuristic 1: A democratic responsive development of a

product is more likely to result in its acceptance.

Often, in instructional development, potential clients or
designers are given a model which ﬁney are to follow. This method
amounts to the imposition of a method of accomplishing goals perceived
by the developer of the model raﬁner than ﬁne target population. The
situation is compounded where the designer has not met the target
audience and is not aware of ﬁne particular group's concerns.

In this pilot study the Manual was developed for ﬁne expressed
needs of the target population. This meant ﬁnat every word of the
Manual was critically read by at least fifteen members from the target
population. As a result of the feedback, areas of concern to the target
population were re-written and submitted for further review until the

refinements were accepted and no major concerns were found .

88

Heuristic 2: Learners are likely to closely follow a
completed product.

All modules develOped for this pilot study followed closely
the format of the Manual itself. That is, they all used print format,
incorporating many of the techniques of programmed learning.
While it was suggested to ﬁne designers that the modules which
make up the Manual could serve as models, it was also stated that
the format and medium of their own modules should be determined
by an analysis of ﬁne learning outcomes; on ﬁnat criterion, a number
of modules should have utilized visuals, movement, and in one case,
a musical instrument. It appears that the novice designer requires
exam pl es of the different media mentioned in the Manual to fully
operational ize cognitive knowledge.

Heuristic 3: Technical terms and jargon of education should

be avoided, if possible.

Many of the terms used in CBE are threatening to people
outside education. Often the ﬁnreat is due to an impression that
the term represents technical precision, and implies an entirely
new forrm of operation which required a great deal of effort for
anyone unfamiliar with it. For example, the term "module" seems
to arouse such fears; the term is used in other fields such as the
building industry, electronics and space technology, and to add to
the confusion, in education it is used to mean different things, such

as a block of tirmetable or components of a school system .

89

In such a situation, the author found it helpful to introduce
new educational terms ﬁnrough terms ﬁnat were already understood.
For example, ﬁne term "module" was introduced initially as a "well
designed lesson". Then as time went on and exarmpl es of modules
became avail able and the components of a module became known,
the correct technical terrm came into more frequent and accepted use.
Heuristic 4: The design of modules is a time consuming,
frustrating and difficult task; only people wiﬁn
special motivation will see the task through.
The auﬁnor knew from previous experiences and from
literature that it is very difficult to get faculty members to design
and report on complex modules. The fact that, for this pilot study,
six faculty members started and completed ﬁneir modules is due to
a combination of factors. At the start of the project, the author developed
a mental profile of the faculty member most suitable for the pilot study.
The person must feel a personal need to irmprove ﬁne quality of teaching.
Such a person is usually already a good teacher and has drawn attention
to him/herself for this reputation, but now is seeking further improve-
ment and is looking for outside help. Usually this person is a strong
scientist or researcher in his/her own field and feels a need to also

be successful as a teacher.

90

Recommendations for Furﬁner Research

 

This was a pilot study. Pilot studies are used to refine a
product, the methodology used and measures employed. As a
result of the present study, a number of recommendations for
further developrment and research can be made. These recommend-
ations can be classified as (1) improvements or refinements to the
Manual , and (2) improvements in the meﬁnods of investigation to

make the findings more generalizable.

Improvements to the Manual

A number of improvements need to be made to the Manual.
According to the feedback received from ﬁne designers, it is
necessary to include a finished product with the Manual to serve
as a model. This seems an obvious point as the diffusion of
innovation literature stresses the necessity for innovators to see
a finished product. Research needs to be carried out as to the
type of model to be provided, for example, should it be 51 ide/tape,
print, video, or a combination. Would a model using one medium
influence designers to design similar medium modules? What should
the length of the model be? Would a lengthy model encourage similarly
lengthy products? How elaborate should ﬁne model be? How does a
costly, professionally designed and produced module affect the
beginning designer, for example, would it be threatening or would

it be motivational?

91

Designers commented on ﬁne presentability of the Manual.
The point was made ﬁnat the total number of pages approaches 300
and it is difficult to find a section of interest without spending a lot
of time searching. Suggested changes include ﬁne use of colored pages
according to a code, e.g. white paper denotes conceptual background
information, green paper for the seven steps of the model and so on.
Another improvement could be the use of plastic tabs attached to ﬁne
different sections which would act as "thumb tabs".

One unanswered question concerns the adequacy of ﬁne Manual;
is all the necessary material avail able in the Manual? The evidence
indicates that designers need to go to further outside sources for
more information. The author considers this to be acceptable; to
include everything that is possibly required in ﬁne Manual would
make it a frightfully heavy volume. In the present Manual ﬁnere are
a lot of references to which the designer could turn for furﬁner
information. However problems cwld arise where the references
are not avail able, particularly in another country.

Designers also pointed to a few typographical errors, areas
where improvements in expression are needed and a total of three
missing pages .

The obvious shortcomings in the Manual need to be corrected
and further development cycles could look into the question of the

amount of inforrmation included. Further research should also look at

92

the different styles of expression used in the Manual. At the present,
the styles are varied, ranging from harangues to adopt different
attitudes, to hum or, to matter-of-fact presentations of research
findings.

Generally there were insufficient comments on the construction,
content and presentation of ﬁne Manual itself, possibly because such
information was not actively sought by the author. Further develOp-

ment cycles need to solicit more feedback frorm the designers .

Improved Methods of Investigation
There is a need for better evidence on the validity, feasibility
and effectiveness of ﬁne Manual. The meﬁnods used in this pilot
study provided tentative evidence until the next cycle of refinement.
In ﬁne next cycle of refinement the following areas could be considered.
Val idity.-- The validity of the evaluation items (in Appendix A)
should be further studied to see if they correspond with, and consequently
measure, the content of the Manual. Further, do the evaluation items
accurately measure ﬁne designer's knowledge and product? It seems
that evaluation item VI A in particular needs to be further tested .
The validity of pre and post test items, throughout the Manual

need to be established.

93

Feasibil ity.--One area to be further researched is ﬁne
feasibility of the Manual as a self instructional entity. In this
pilot study ﬁne six designers were guided to some extent by
corresponding wiﬁn ﬁne author where they were constantly encour-
aged to keep to their time lines and complete their modules.
However, without this constant monitoring would the designers
carry on their task to completion? What factors could be built
into the Manual which would ensure completion, e.g. the effect-
iveness of tirme lines designed into ﬁne Manual for purposes of

self pacing?

Effectiveness.-- In the next cycle of research and develop-
ment better measures of effectiveness need to be used; measures
which can provide better quantitative evidence on ﬁne effectiveness
of ﬁne Manual . Two areas are of immediate interest. One area
concerns the designer's knowledge gain and clnanges in attitude
measured ﬁnrough pre and post test results. Such measures
ShQJld provide clear data as to the designer's knowledge of key
concepts and attitudes towards CBE, for pre and post treatment
comparisons.

The second area is the effectiveness of ﬁne modules
produced by the designers compared to traditional teaching methods .
The experimental design presented as Appendix C is a possible guide

to such experimentation and deserves replication.

94

Concl udigRemarks

The Manual provides ﬁne cognitive skills and ﬁne procedural
guidelines for ﬁne design and development of a module. Two additional
features of ﬁne Manual are (1) that it is self instructional and (2) ﬁnat
it particularly addresses the acquisitionofhigher order skills. The
development of the Manual was through responsive interaction with
a particular target group (college level instructors with minimal
CBE background) where this author attempted to respond to the needs
of ﬁnat group.

In this pilot study, the Manual has been successful; it is now
ready for a further cycle of development and evaluation. In this
next cycle, summative evaluation in the form of comparing perform-
ances of control and experimental groups should provide further data
as to ﬁne validity, feasibility and effectiveness of ﬁne Manual .

It is hoped that ﬁne product (the Manual) and the principles
used in its develOprment will prove useful to educators who are
sufficiently concerned with the needs of learners to engage

in ﬁne learning and growth process ﬁnemselves.

BIBLIOGRAPHY

BIBLIOGRAPHY

Abedor, A.J. Second draft technology: Development and field test of
a model for formative evaluation of self instructional multi

media learning systems. View points, 1972, 48 (4), 9-44.

 

Ainsworth, D. Examining the Basis for Competency Based Education.
Journal g£_Higher Education, 48(3). 1977. ’

 

Airasian, P.W. The role of evaluation in mastery learning. In J.H.
Block (Ed.) Mastererearning: Theory and Practice. New York.
Holt, Reinhart and Winston, 1971.

 

 

Airasian, P.W. and Bart, W.M. The use of ordering theory in the
identification and specification of instructional sequences.

Boston College, Chestnut Hill, Massachusetts. 1974.

Alkin, M.C. Evaluation and decision making. P1anning_and Changing,
1973, 3 (4), 1-4.

 

American Dietetic Association Committee on Minimum Academic Requirements-
Plan IV 1972. (Mimeographed) A.D.A. Chicago.

Anderson, R.C. "Educational Psychology". 1967. Annual Review 92

 

Psychology, 18. 129-164.

 

Anderson, R.C. and Bower G. Psychologng£_Learni g and Motivation.
Academic Press, N.Y. 1972. '

Annett, J. Feedback and Human Behavior. Penguin Harmondsworth, 1969.

95

96

Annett, J. "Psychological bases of educational technology", in Budgett,

R. and Leedham, J. (Eds.) Aspects g§_Educationa1 Technology, VII,
Pitman, London, 1973.

Armstrong, R.J. et.a1. The Development and Evaluation 3: Behavioral
Objectives. Worthington, Ohio: Charles A. Jones Publishing Co.,
1970.

 

Ashby, W.R. Ag Introduction £2_Cybernetics. Methuen and Co., London,
1965 (Second Edition).

 

Atkinson, R.C. Adoptive instructional systems: Some Attempts to
Optimize the learning process. In D. Klahr (Ed.), Cognition and
Instruction. Wiley, N.Y. 1976.

 

 

Atkinson, R.C. and Shiffrin, R.M. Human Memory: A Proposed System
and its control processes. In K.W. Spence and J.T. Spence (Eds.),
The Psychology g£_Learnigg and Motivation. Vol II. Academic
Press, N.Y. 1968.

 

 

Ausubel, D.P. Educational Psychology: IA_Cognative View. Holt Rinehart
and Winston. N.Y. 1968.

 

Baker, E.L. The Technology of instructional development. In RJM.U.
Travers (Ed.), Second Handbook 2: Research gg_Teaching. Rand
McNally Chicago, 1973.

Baker, E.L. and Alkin, M.C. Formative evaluation of instructional

development. Audio Visual Communication Review. Vol. 21, No. 4,
1973.

Battig, W.F. Paired-associate learning, in T.R. Dixon and D.L. Horton
(Eds.), Verbal Behavior and General Behavior Theory. Englewood
Cliffs, N.J.: Prentice Hall, 1968.

97

Benathy, B.H. Instructional Systems. Palo Allo, California: Fearon
Press, 1968.

Bertalanffy, Von.L. Organismic Psychology and Systems Theory. Clark

 

University Press Worchester, Massachusetts. 1966.

Best, W.J. Research ig_Education. Prentice-Hall Inc. Englewood
Cliffs, 1977. (3rd Edition).

Block, J.H. The effects of various levels of performance on selected
cognitive, affective, and time variables. Unpublished doctoral
dissertation, University of Chicago. 1970.

Blalock, H.M. Casual Inferences 13_Nonexperimental Research. University
of North Carolina Press. Chapel Hill, 1964.

 

Block, J.H. (Ed.) Mastery Learnigg: Theory and Practice. Holt,
Rinehart and Winston. N.Y. 1971.

 

 

Block, J.H. Student Evaluation: Toward the Setting of Mastery Performance

Standards. Paper presented at the meeting of AERA, Chicago.
1972.

Block, J.H. The "C" in CBE Educational Researcher. (7), 5. 1978.

 

Bloom, B.S. (Ed.) Taxonogy-gf Educational Objectives-The Classification
gf_educational goals-Handbook I: Cognitive domain. David McKay
Co. N.Y., 1956.

 

Bloom, B.S. (Ed.) The role of the educational sciences in curriculum

development. University of Chicago, 1964.

Bloom, B.S. Mastery Learning. In J.H. Block (Ed.) Mastery Learnigg:

 

Theory and Practice, 1971.

 

98

Bloom, B.S., Hastings, J.T., Madaus, G.F. Handbook gg_formative and

 

summative evaluation 2: student learning. McCray-Hill, N.Y.
1971.

 

Bloom, B.S. Human Characteristics and School Learnigg, McGraw Hill,
N.Y. 1976.

Bloom, B.S. Mastery Learning. In J.H. Block (Ed.), Mastery Learnigg:

 

Theory and Practice. Holt, Rinehart and Winston N.Y. 1971.

 

Borg, W.R. and GAll, M.D. Educational Research David McKay Co., Inc.
N.Y. 1971. (Second Edition).

 

Born, D.G. Proctor Manual and Instructor Manual for Development of a

PSI Course. University of Utah, Salt Lake City, 1971.

Born, D.G., Gledhill, S.M. and Davis, M.L. Examination performance in
lecture discussion and personalized instruction courses. Journal
2; Applied Behavior Analysis. 5:33-53, 1972.

Boucher, B.G. Gottlieb, M.J. Morgenlander, M.L. Handbook and Catalgg

 

for Instructional Media Selection. Ed. Tech. Publications, Englewood
Cliffs N.J. 1973.

 

Braby, R. Ag_Evaluation‘g§ Ten Techniques g£_Choosigg_Media. TAEG

 

 

 

Report No. 8, December 1973. Naval Training Equipment Center,
Orlando, FL.

Branson, R.K. The Interservice Procedures for Instructional Systems
Development. Educational Technology Volume XVIII No. 3 March,
1978. p 12.

 

99

Bridge, W. Self study courses in undergraduate science teaching: The

report of a survey. Higher Education 5, 211-224, 1976.

 

Briggs, L.J. Seguencing pf Instruction i3 Relation £2 Hierarchies pf

 

 

Competence. American Institutes for Research. Pittsburgh,

 

Pennsylvania, 1968.

Briggs, L.J. Instructional Desigp: Principles and Applications.
Educational Technology Publications Englewood Cliffs, N.J. 1977.

 

Brown, T., Holland, R. and Malous, M.L. A self paced learning system
in action in higher education. Educational Technology. 16(3) 43-
45, 1976.

Bruner, J.S. Toward 5 Theory gf_Instruction. Cambridge: Harvard

 

University Press, 1966.

Brunet, J.S. The Relevance g£_Education. New York: Morton, 1971.

 

Campbell, D.S., Schwen, T.M. Beyond the Remedial Loop: Toward the
integration of task and learner analysis for a process approach
to instructional development. Paper presented at AECT Annual
Convention, Philadelphia, PA. 1971. (ED 049 599).

Campbell, D.T. and Stanley, J.C. Egperimental and Quasi-Egperimental
Designs for Research. Rand McNally Chicago, 1963.

 

Campbell, J.P. Personnel Training and Management Development. In

 

Mussen, P. and Rosensweig, M. (Eds.), Annual Review of Psychology,
Volume 22, 1971, 565-566.

Carroll, J.B. A Model of school learning, Teachers Collegg Record,
1963, 64, 723-733.

 

100

Carroll, J.B. Ploblems of measurement related to the concept of learning

for mastery. Educational Horizons, 1970, 48, 71-80.

 

Chase, F.S. The National Program gf_Educational Laboratories: Report

 

 

2§_§_Study pf Twenty Educational Laboratories and Nine University
Research and Develgpment Centers. washington, D.C.: U.S. G.P.O.,
1968.

 

Chu, G.L., and Schramm, W. Learnigg from television. Washington,
DOC. N.A.EOB.’ 1967.

 

 

Clark, D.L. and Hopkins, J. A Report on Educational Research Development
and Diffusion Manpower, 1964-1974. Bloomington, Indiana: Indiana
University Research Foundation, 1969.

Cook, J.M. and Walbesser, H.H. How £g_meet accountability with behavioral
and learning hierarchies. University of Maryland, College Park,
1973.

 

Corrigan et.a1. 'A System Approach for Education. (SAFE) R.E. Corrigan
Garden Grove, California, 1969.

 

Cox, R.C. Item Selection Techniques and Evaluation of Instructional
Objectives. Journal pf Educational Measurement. 1965, 2, 181-
185.

 

Cox, R.C. and Wildemann, C.C. Toxonogy.g§_Educational Objectives:
Cognative Domain. ‘A§_Annotated Bibliqgraphy. Pittsburgh:

 

 

University of Pittsburgh, Learning Research and Development
Center, 1970.

Cross, K.P. Accent gp_Learning. Jossey—Bass San Francisco, 1976.

Crowder, N.A. Automatic programming by intrinsic programming. In As
A. Lumsdaine and R. Glaser (Eds.), Teaching Machines and Programmed

 

 

Learning: A Source Book. A.E.C.T. Washington, D.C., 1960.

101

Davis, K.K. Competency Based Learning: Technology Management and
Design. McGraweHill Book Co., N.Y. 1973.

 

Davis, R.H., Alexander L.T. and Yelon, S.L. Learnign System Desigp.
McGraerill Book Co., N.Y. 1974.

 

Davis, R.H. .A Behavioral Chang§_Model With Implications for Faculty
Development. (mimeograph) Learning and Evaluation Service, Michigan
State University, East Lansing, MI 1978.

Dick, W.A. A methodology for the formative evaluation of instructional
materials. Journal 9f Educational Measurement. 1968, 5(2) 99-
102.

 

Dick, W.A. Methodology for the formative evaluation of instructional
materials. Journal pf Educational Measurement. 1978. 5(2), 99-
102.

Dick, W.A., Hagerty, N. Topics ig_Measurement: Reliability and
Validity. McGraweHill, N.Y. 1971.

 

Dressel, P.L. The Nature and Role of Objectives in Instruction.
Educational Technolqu. (l7)5, May 1977, 7—15.

Duane, J.E. Individualized Instruction Programs and Materials.
Selected Readings and Bibliography. Educational Technology
Publications. Englewood Cliffs, N.J. 1973.

Duncan, K.D. Strategies for analysis of the task, in Hartley, J.
(Ed.) Strategies for Prqgrammed Instruction. London, Butterworths
1972.

Ebel, R.L. Criterion-referenced measurements: Limitations, School
Reviews, 1971, 69, 282-288.

102

Eisner, E.W. Educational Objectives: Help or Hinderance? School
Review, 75 (Autumn 1976).

Elam, S. Performance-based Teacher Education: What is the State of
the Art? American Association of Colleges for Teacher Education,
1972.

Ellul, J. The Techological Society. Vintage, N.Y. 1967.

 

Fantine, M.D. (Ed.) Alternative Education: .5 Source Book for Parents,
Teachers, Students and Administrators. Doubleday N.Y. 1976.

 

 

 

Flanahan, J.C. Functional education for the seventies. Phi Delta
Kappau, 1967, 49, 27-32.

Flanagan, J.C. The PLAN system for individualizing education. NCME

Measurement ip_Education, 1971, 2, 2.

 

Folley, J.D. Guidelines for Task Analysis NAUTRADEUCEN. U.S. Naval
Training Center, Port Washington, N.Y. 1964.

Fox, G.T., and DeVault, M.V. Technology and humanism $2 the classroom:

 

 

Frontiers g§_educationa1 practice. Educational Technology. 1974,
14(10), 7-13.

Gagne, R.M. The acquisiton of knowledge. Psychological Review, 1962,
69, 355-365.

 

 

Gagne, R;M. The Conditions 2: Learning. New York: Holt, Rinehart and
Winston, 1965. (2nd edition, 1970.)

Gagne, R.M. Essentials gf_Learning. Dryden Press, 1974.

103

Gagne, RlM. The Conditions g£_Learning, 3rd ed. Holt, Rinehart and
Winston. New York. 1977.

 

Gagne, RmM., and Paradise, N.E. Abilities and Learning sets in knowledge
aquisition. Psycholqglcal Monographs, 1961, 75, (518).

Gagne, R.M., Major, J.R., Gerstens, H.L. and Paradise, N.E. Factors
in acquiring knowledge of a mathematical task. Psycholggical
Monogrephs, 1962, 76, (526).

 

 

Gagne, R.M. and Staff. University of Maryland Mathematics Project.
Some factors in learning non-metric geometry. Monographs of the
Society for Research in Child DevelOpment, 1965, 30, 42-49.

Gagne, RJM. and Briggs, L.J. Principles gf_Instructiona1 Desigp.
Holt, Rinehart and Winston, Inc. N.Y. Second Edition 1975.

 

 

Geis, G.L. Education, Training and Behavioral Objectives. Educational
Technology 17(5), May 1977, 32-36.

 

 

Gentile, J.R., Roden, A.H. and Klein, R.D. An analysis of variance
model for the intra subject replication design. Journal pf
Applied Behavior Analysis. 5:193—198, 1972.

Gilbert, T.F. Mathetics-the technology of Education in Merrill, M.D.

(Ed.) Instructional Design: Readings. Englewood Cliffs, Prentice
Hall, 1971.

Gideanse, H.D. Research, development, and the improvement of education.
Science, 1 and 2, 541-545, 1968.

Glaser, R. Instructional technology and the measurement of learning
outcomes. American Psycholggist, 1963, 18, 519-521.

Glaser, R. Psychological based for instructional design. A.V. Communications
Review. 1966, 14(4), 433-449.

 

104

Glaser, R. Adopting the elementary school to individual performance,

 

in Proceedings gf_the Invitational Conference 22_Testing Problems.
E.T.S. Princeton, New Jersey, 1968. pp3-36.

Glaser, R. Evaluation of Instruction and changing Education Models, in
Wittrock, M.C., and Wiley, D.E. (Eds.), The Evaluation pf Instruction.
New York: Holt, Rinehart and Winston, 1970.

 

 

Glaser, R., and Nikro, A.J. Measurement in learning and instruction.
In R.L. Thorndike (Ed.), Educational Measurement. Washingon:
American Council on Education, 1971.

Glaser, R., Resnick, L.B. Instructional Peychology Annual Reveiw pf
Psychology. 1972, 23, 207-276.

 

 

 

Goldschmid, B. and Goldschmid, M.L. Modular Instruction ig_Higher

 

Education: A_Review. Center for Learning and Development,
McGill University. Montreal, 1972.

Goldschmid, B. and Goldschmid, M.L. Individualizing Instruction in
Higher Education. A review, Higher Education, 5:9-33, 1974.

 

Gronlund, N.E. Stating Behavioral Objectives for Classroom Instruction.
The MadMillan Co. N.Y. 1974.

Grundin, H.U. "Response mode and information about correct answers in
programmed instruction: a discussion of experimental evidence
and educational decisions." In Mann, A.P. and Brunstrom, C.K.
(Eds.) Aspects gf_Educationa1 Technology, Ill, Pitman. London.
1969.

Gustafson, H.W. Hornsberger, W.D., Michelson, 8. Determination pf

 

Task Analysis Content. Unpublished Dissertation Florida State
University, Tallahassee, 1960.

 

105

Gustafson, K.L. Improving Instructional Development: Faculty as
Learners. Educational Technolggy, 1975, 15(5), 34-38.

 

Guideonse, H.D. Educational Research and Development $3 the United
States. Washington, D.C.: U.S. G.P.O., 1970.

 

 

Hambleton, R.K. Testing and Decision-Making Procedures for Selected
Individualized Instructional Programs. In Review p£_Educationa1
Research Vol. 44, No. 4. 1974.

 

Hambleton, R.K. and Norvik, M.R. Toward an integration of theory and
method for criterian-referenced tests. Journal p£_Educational
Measurement. 1973, 10, 159-170.

 

 

Harrow, A.J. .é Taxonomy p£_the Psychomotor Domain: .5 Guide for
Developing Behavioral Objectives. David McKay Co., 1972.

 

 

Hart, M. Competency-based Education. Journal pf the American Dietetic
Association. Vol. 69, December 1976. 616-621.

 

 

Hartley, H.J. Educational Planning Prggrammipg.Budggtipg; §:Systems

 

Approach. Prentice Hall Inc. Englewood Cliffs, N.J. 1968.

Hartley, H.J. Programmed Instruction 1954-1974: A Review. Programmed
Learnipg and Educational Technology. Vol II, No. 6, Nov. 1974.
p. 288.

 

Hartley, J. Some factors affecting student performance in programmed
learning. Programmed Learning and Educational Technolqu, 5.
1968. 206-217.

 

 

Hartley, J. (Ed.) Strategies forProgrammed Instruction. Butterworths.
London. 1972.

 

Hartman, D.P. Forcing square page into round holes: Some comments on
"An analysis of variance model for the intra subject replication
design." Journal pf Applied Behavior Analysis. 7:635-638, 1974.

106

Havelock, R.G. .é Guide 52 Innovation in Education. C.R.U.S.K. Ann
Arbor, 1970.

 

Havelock, R.C., Guskin, A., Frohman, M., Havelock, M. Hill, M., and
Huber, J. Planning for Innovation Throngh Dissemination and
Utilization_pf_Knowledge. C.R.U.S.K. Ann Arbor. 1969.

 

 

Havelock, R.C., Huber, J.C., and Zimmerman, S. Major WOrks'pp_Change
ip_Education. C.R.U.S.K. Ann Arbor. 1969.

 

Joyce, B. and Well, M. Models pf_Teaching. Prentice-Hall Englewood
Cliffs, N.J. 1972.

Keller, F.S. Goodbye Teacher. Journal gf_Applied Behavior Analysis,
1968, 1, 79-89.

 

Kershaw, J.A., McKean, R.N. Decision making in_the Schools: ‘Ap_0utsider's
View. Rank P-1886, 1960.

 

 

Kim, H. The mastery learning project in the middle schools. In J.H.
Block (Ed.), Mastery Learning: Theory and Practice. New York:
Holt, Rinehart and Winston, 1971. (Abstract).

 

Klaus, D. Instructional Innovation and Individualization. -Second
Printing. American Institute for Research for the Agency for
Instructional Deve10pment. U.S. Department of State, Kensington,
Maryland. 1977. ,

Klausmeier, H.J. and O'Hearn, G.T. (Eds.) Research and Development
Toward the Improvement of Education. Journal p£_Experimental
Education, 37(1968), 1-163.

 

Klingsredt, J.L. Philosophical basis for competency education. Educational

 

Technology 21, No. 11, November, 1972.

107

Komoski, P.K. Statement to the U.S. House of Representatives Committee
and Education and Labor In Hearings: .Tp_Establish.g_National
Institute pf Education, 92nd Congress, First Session. Washington:
U.S. Govt. Printing Office, 1971.

Kozma, R.B. The effect of empirically validated versus re-ordered
inter and intra sequencing of instruction on learning. Center
for Research on Learning and Teaching, University of Michigan,

Ann Arbor, MI. 1974.

Krathwohl, D., Bloom, B. and Masia, B. Taxonogy 2: Educational Objectives,
Handbook II: Affective Domain. David McKay Co., New York, 1964.

 

Kulik, J.A. PSI: a formative evaluation. In Personalized Instruction

gn_Higher Education: Proceedings p£_the Second National Conference.

 

B.A. Green (Ed.) Center for Personalized Instruction, Washington,
D.C., 1976.

Kulik, J., Kulik, C. and Carmichael, K. The Keller plan in science
teaching. Science 183: 379-383, 1974.

Landy, D., McCue, K. and Aronson, E. Beyond Parkinson's Law. Journal
pf_Applied Psycholggy, 53, 1969. 236-239.

Lawless, C. Book Review of Learning System Design: An Approach to

the Improvement of Instruction.

Leith, G.O.M. .A Handbook pf Programmed Learning_(2nd ed.). University
of Birmingham: Educational Review Occasional Publication No.1.
1966.

Lindsay, P.H., and Norman, D.A. Human Information Processing: .An

Introduction £g_Psychology. New York: Academic Press, 1972.

 

Lippitt, R., Watson, J. and Westley, B. The Dynamics pf Planned
Change. Harcourt Brace and Co. New York. 1958.

108

Livingsten, S.A. Criterion-referenced applications of classical test

theory. Journal p£_Educational Measurement, 1972, 9, 13-26.

 

Lloyd, K.E. and Knutzen, N.J. A Self paced programmed undergraduate
course in the experimental analysis of behavior. Journal of
Applied Behavior Analysis.~ 2:125-133, 1969.

 

McAshan, H.H. Behavioral Objectives: The History and the Promise.
Educational Technology, 17(5), May, 1977.

 

McKeachie, W.J. 'Instructional Psychology. In Annual Review p£_Psychology,
Volume 25, 1974.

 

MacDonald-Ross, M. Behavioral Objectives - A Critical Review. Instructional
Science, 2(1973), 1-52.

Mager, R.F. On the sequencing of instructional content, in Davies,
I.K. and Hartley, J. (Eds.) Contributions pp 32 Educational

 

 

Technology. Butterworths, London. 1961.

 

Mager, R.F. Preparing Instructional Objectives. Fearon, Polo Alto.
1962.

 

Mager, R.F. and Clark, P.C. Explorations 1n_student controlled instruction.
Psychological Reports 13, 71-76. 1963.

 

 

Markle, S.M. Empirical testing of Progress. In P.C. Lange (Ed.),
Programmed Instruction. The sixtysixth Yearbook of the National
Society for the study of Education, Part II. Chicago NSSE, 1967.

 

. Markle, S.M. Good Frames and Bad (2nd ed.). Wiley, N.Y. 1969.

 

Mechner, F.M. Behavioral analysis and instructional sequencing in

Lange, C.P. (Ed.) Programmed Instruction N.S.S.E. Chicago, 1967.

 

109

Merrill, M.D. and Boutwell, R.C. Instructional Development: Methodology
and Research. In Kerlinger, F.N. (Ed.), Review p£_Research_yg
Education. Vol. 1, F.E. Peacock, Hasca, Illinois, 1973.

Miller, A. (Ed.) Teaching_andLearning ip_Medica1 School, Harvard
University Press, Cambridge, Massachusetts, 1962.

 

 

Millman, J. Passing Scores and test lengths for domain-referenced
measures. Review pf Educational Research. 1973, 43, 205-216.

 

Milsum, J.H. Technosphere, biosphere, and sociosphere: An approach
to their systems modeling and optimization. In Ludwig Von Betalanffy,
A. Rapport and R.L. Meier (Eds.), General Sygtems: Yearbook pf

 

the Society for General Systems Research, Volume XII, 1968, 37-
48.

 

Morrison, H.C. The Practice_g£ Teaching ;p_the Secondary School.

 

Chicago: University of Chicago Press, 1926.

Morrissey, M.J.K. The Application of the Problem-Oriented Medical
Record to Client Nutritional Care: A Self Instructional Module
for Dietetic Students. Unpublished Masters Thesis. Michigan
State University, 1978.

Myers, W.A. Operant Learning Principles Applied to Teaching Introductory
Statistics. Journal pf_Applied Behavior Annlysis, 3(1970), 191-
97.

 

Neff, F.C. Performance-based Teaching: .A New Orthodoxy? in Ralph
Smith (Ed.) Regaining Educational Leadership: Critical Essays on
PBTE/CBTE, Behavioral Objectives and Accountability. John Wiley
and Sons, N.Y. 1975.

 

Norman, D.A. (Ed.) models p£_Human Memory. New York: Academic
Press, 1970.

 

110

Novak, J. The Future p£_Modular Instruction. Center for Improvement

 

of Undergraduate Education Notes, No. 6 Ithace, N.Y.: cornell
University, CIUE, 1973.

Payne, D.A. The Specification and Measurement pf Learning Outcomes.
Blaisdell Waltham, MA. 1968.

 

Phillips, E.R. Validating learning hierarchies for sequencing mathematical
tasks. (Doctoral dissertation, Purdue University), Ann Arbor,
Michigan: University Microfilms, 1971, No. 72-2008, 171.

Pophaur, W.J. Normative data for criterian-referenced tests? Phi
Delta Kappan, 1976, 58, 593-594.

 

Prosser, G.V.P. The role of active questions in learning and retention

of prose material. Instructional Science, 2 1974. 451-476.

Rankin, W.C. Task Description and Analysis for Training System Design.
TAEG Technical Memorandum 74-2. January 1975, Orlando, FL.

Robeck, M.D. A study of the revision process in programmed instruction.
Unpublished Master's Thesis, University of California, Los Angeles,
1965 0

Rogers, E.M. Diffusion pf Innovations. The Free Press. New York.
1962.

 

Rogers, E.M. and Shoemaker, F.F. Communication p§_Innovations. The
Free Press, N.Y. 1971.

 

 

Rosinski, F. An Approach to Developing a Competency-Based Curriculum.
The California Story in Action Programs for Developing Allied
Health Educators. The W.K. Kellogg Foundation, Battle Creek,
1977.

 

 

Russell, J.D. Modular Instruction. Burgess Publishing Co., Minneapolis,
MN. 1974.

Saettler, P. .2 History p£_Instructional Technology, McGraw-Hill
Inc., N.Y. 1968.

Schramm, W. Bingedia, Little Media. Hollywood, CA: Sage Publications,
1977.

Schutz, R.E. Developing the "D" in Educational R and D. Theory into
Practice, 6 (1967), 73-76.

 

Scriven, M. The Methodology of Evaluation. In R,W. Tyler, RJM. Gagne,
and M. Sriven (Eds.), Perspectives pf_Curriculum Evaluation.
AERA Monograph Series on Curriculum Evaluation, No. 1. Chicago:
Rand McNally, 1967. '

Scriven, M. Curriculum Evaluation: Potential and Reality, in J. Weiss
(Ed.), Curriculum Theory Network. Toronto Institute for Studies
in Education, Toronto, 1972, pp 182-192.

Semb, G. Research strategies in higher education. Division.pf_the
Teaching pf Psychology Newsletter, 1973 pp 9-13.

Semb, G. The effects of mastery criteria and assignment length on
college student test performance. Journal pf Applied Behavior

Analysis 7:61-69, 1974.

Shoemaker, D.M. Evaluating the effectiveness of computing instructional
programs. Educational Researcher, 1972, 1(5), 5-8, 12.

 

Share, E.M. Strategies for the Implementation of Modular Instruction
and their Implications in University Education. The Journal pf
Higher Education, December, Vol. 44, No. 9, 1973.

 

112

Singer, R.N., and Dick, W. Teaching Physical Education: Q Systems

 

Approach. Houghton-Mifflin, Boston. 1974.

Skinner, B.F. Are Theories of Learning Necessary? Psychological
Review 57 193-216, 1954.

 

Skinner, B.F. (1954). The Science of Learning and the Art of Teaching
in Lundsdaine, A.A. and Glaser, R. (Eds.). Teaching Machines and

 

 

Programmed Learning. Washington, N.G.A. 1960.

 

Skinner, B.F. The Technology pf Teaching. Appleton-Century-Crofts,
N.Y. 1968.

 

Smith, K.D. and Smith, M.F. Cybemetic Principles pf_Learning and
Educational Design. Holt, Rinehart and Winston. N.Y. 1966.

 

 

Smith, W.L. First steps first. Competency Based Teacher Education.

 

Palo Alto, California. Science Research Associates, 1972.

Spencer, R. et al. An experimental comparison of two forms of personalized

instruction, a discussion procedure and an independent study

 

procedure, In Personalized Instruction 1n_Higher Education:

 

Proceedings pf the First National Conference. Ruskin, R.S. and
Bono, S.F. (Eds.). Washington, D.C. 1975 pp 11-20.

Shufflebeam, D. The use of experimental design in educational evaluation.

Journal pf Educational Measurement, 8(4).

Sugarman, R.C. et.a1. SAT REvisited-A Critical Post Examination of the
Systems Approach to Training. Paper presented at the Proceedings
of the Human Factors Society 19th Annual Meeting, October 1975,

Dallas, Texas.

Supper, P. The uses of computers in education, Scientific American,
1966, 215, 206—221.

 

113

Swain, A.D. System and Task Analysis, a major tool for designing the
personnel subsystem. Sandia Corporation, SCR-457, Technical
Information Division, Albuquerque, N.M. 1962.

Swaminathan, H., Hambleton, R.K., and Algina, J. Availability of
Criterian-referenced tests: A decision-theoretic formulation.

Journal pf Educational Measurement, 1974, 11.

 

Taieggia, T.C. Personalized instruction: A summary of comparative
research, 1967-1974. American Journal of Physics 44(11), 1028-
1033, 1976.

 

Thorndike, E.L. Educational Psychology. Vol. II: The Psychology of

 

Learning. Teachers College, Columbia University, 1921.

Tyler, R.W. Constructing Achievement Tests. Ohio State University,
Columbus, 1934.

Vermilye, D.W. (Ed.) Individualizing_the System: Current Issues in_

 

 

 

Higher Education, San Francisco: Jossey-Boss, 1976.

 

Washburne, C.W. Educational measurements as a key to individualizing
instruction and promotions. Journal of Educational Research,
1922, 5, 195-206.

White, R.T. Research into learning hierarchies. Review pf_Educationa1
Research, 1973, 43, 361-375.

 

White, R.T. The validation of a learning hierarchy. American Educational
Research Journal, 1974, 11, 121-136.

 

 

Williams, R. A behavioral typology of education objectives for the
cognative domains. Educational Technology 15;6, 1977 pp 3946.

Yelon, S. Instructional Design and Technology. Reneod Michigan State
University, 1976.

APPENDIX A

1 . Evaluation Instrument for Module

2 . Rationale for Evaluation

MODULE EVALUATION

 

 

Maximum Criterion Points
Points Level Awarded
I. Instructional Cool
A. rationale for module (criteria used for selecting
instructiouslgoel)................ S 4
B. stetementofinstructionslgoel.......... 2 2
II. Instructional Analysis
A. rationale for methodology selected . . . . . . . . . 2 ___:l__
B. completeness of diagram of subskills . . . . . . . . 10 __l!__ '—
0. description of relationship nong subskills . . . . 2 ___;L__ ‘—
D. clarification between subskills included as entry ——
behaviors and those included as skills to be learned
throughmsterisls................. 2 I __
III. Description of Target Population ,
' A. description of general characteristics of target
population and implications these have for instruc-
tionslmsterials.................. 4 3
IV. Learning Outcomes
A. derivation from instructional analysis . . . . . . . 4 3 _
B. statues: of the learning outcome . . . . . . . . . 6 4
V. Criterion-Referenced Tests
A. relationship of items to learning outcomes . . . . . 4 3
B. criterion level or mastery level of performance
clearlyidentified................. 3 2
C. appropriate tests based upon materials and target
popultsion (pretests, poscrest, embedded test, entry
beheviorstest).................. 9 7
VI. Instructional Strategy
A. preinstructional activities . . . . . . . . . . 2 1 .___..
B. informationpresentation.............. _£g__ 3 __
c. student participation activities . . . . . . . . . . __43__ g
D. testing (covered in No. V) . . . .
E. follow-throughscrivities............. 2 l .—
F. strategy congruent with materials . . . . . . . . . 6 5
VII. Formcive Evaluation
A. description of one-co-one data collection procedures 6 3 _
3. results and revisions based upon one-to-one testing 6 4
C. description of small-group evaluation procedures
1. sample group characteristics . . . . . . . . . . 2 1 _
2. design of formative evaluation study . . . . . . 4 ; __
3. instruments and procedures used for date col-
lection.................... 4 __§__ ..._..
4. dstssimsrysnddispley............ 4 3 __
VIII. Suggested Revisions
A.msterisls..................... __§!_. __1L__ __
B.tests....................... ___1!__ ‘_Jl__ __
C.procedures..................... 2 __2L__ _
TOTAL 100 73

Ill:

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

llS

RATIONALE FOR

MODULE EVALUATION

I. Instructional Goal

A.

The designer should clearly indicate the rationale for selecting
the topic for their instruction. The rationale should

include such items as their expertise in the content area,
ability to provide the instruction within a reasonable amount
of time, the availability of students to try out the materials
in the formative evaluation, and the need for this type of
instruction.

The goal statement should be a clear indication of the outcome
of the instruction. Although the goal does not need to be
stated in precise behavioral terms, it should reflect, in
general, what students should be able to do when they have

completed the instruction.

ll. Instructional Analysis

A.

The designer should indicate which instructional analysis
methodology has been chosen and why it was selected.

The designer should include a complete instructional analysis
chart that lists all the subskills associated with the
instructional goal. The subskills should be stated in
behavioral terms. Entry behaviors should be included on the
chart and clearly indicated as such.

The designer should describe the diagram previously described
in B and indicate the relationship of the subskills to each

other.

ll6 ~
The designer should describe the entry behaviors that have
been identified and explain the rationale for including
them. For instruction that has no relevant entry behaviors,

students should explain why none are required.

Ill. Description of Target Population

A.

The designer should describe the general characteristics of

the target population for whom their instruction is being
prepared. The target populations may vary from a particular
classroom group to a national category of students. Important
aspects of this component are that the target population is
identified and that there is some statement of the implications
of the relationship between the general characteristics of

target students and the instruction to be provided for them.

IV. Performance Objectives

A.

The designer should include in the documentation report a
chart that indicates in one column the subskills and
instructional goal as identified in the instructional analysis
and in a second column, the corresponding performance
objectives and terminal objective for the instructional unit.
Subskills and objectives should be numbered similarly in
order to indicate their relationship.

The objectives for the instruction should be stated using the
three components suggested by Mager. These include the
conditions, performance, and criteria. The alternative
approach to writing objectives suggested by Gagne and Briggs
is also appropriate. Student should indicate which format

they have used for the objective statements.

ll7

V. Criterion-Referenced Tests

A.

The designer should provide a clear indication of the
relationship between their test items and their instructional
objectives. This relationship can be indicated by assigning
the same numbers to test items as was used to number the
corresponding subskills and objectives, or, a chart can be
developed which indicates the item number of each performance
objective and the corresponding test items.

The designer should indicate the criterion used to judge the
mastery level of learners' performance relative to each of
the objective and to the total test.

After reviewing the test, the instructor should determine

if there are adequate number of test items and whether the
items are appropriate for the target population and the

content of the instruction.

VI. Instructional Strategy

A.

The designer should describe how their instruction will
motivate learners, inform them of what they are going to learn,
and remind them of related information which they already know.
The designer should describe how the content will be presented
to learners. This explanation should not simply be a running
description of the specific content, but rather an indication
of how ”chunk" size will be determined and sequenced.

A description should be included of the activities provided

to learners to practice the behavior taught in the instruction.
The provision of feedback for learners' performance should

be described.

ll8
The evaluation of testing strategies was discussed previously
in Section V.
Although the designer is not typically required to develop
any remedial or enrichment activities, a brief description
of suggested ones should be included.
After examining the instructional strategies described by
the designer, the instructor should examine the instructional
unit itself to determine whether the materials are consistent
with the strategies described by students in their documentation

sections.

VII. Formative Evaluation

A.

The designer should describe the preparation of instruction
for the one-to-one formative evaluation. This description
should include the format of materials to be evaluated,

the characteristics and entry behaviors of learners who
participated in the evaluations, the criteria for the selection
of the learners, and the procedures followed in conducting

the one-to-one evaluation.

The designer should describe the results of the one-to-one
evaluation, particularly the major comments received from
learners during the evaluation. There should be a description
of the types of revisions made in the materials and the
rationale for these revisions based on the results of the
one-to-one evaluation.

The designer should provide a complete description of the
characteristics and entry behaviors of learners who
participated in the small-group evaluation. There should also

be a complete description of the procedures employed in

l19

implementing the evaluation. The description should include
the format of the materials, the procedures which were used
for testing, the instructions given to learners, and the
feedback received at the end of the instruction. There should
be a description of all instruments and procedures used to
collect data. These descriptions can be used to document

the effectiveness of the instruction and to identify points

in the materials that still need revision. Students should
summarize the data collected, including the display of

such items as pretest and posttest performance, performance on
embedded-test items, learning time, performance by objectives,

and learner attitudes as indicated on questionnaires.

VIII. Suggested Revisions

A.

The designer should describe, based on the data and information
gathered during the small-group evaluation, the revisions
which remain to be made in their materials. The relationship
between these revision recommendations and the data obtained
during the small-group evaluation of the materials should be
apparent.

The designer should indicate revisions that remain to be

made in the test instruments as a result of the data and
information gathered during the small-group evaluation.

The designer should indicate any instructional procedures
that need to be revised as a result of information gathered
during the small-group evaluation. They should describe

any changes in procedures or materials needed if materials

were used in a regular classroom setting.

ll9/l

Criteria for Evaluating Designer-Prepared Instructional Materials

 

 

l. Materials should include objectives, pretests/posttests,

and the instruction.

2. The instruction should follow the instructional strategy

established for the unit as described in the documentation

report. Some general guidelines:

a.

Have adequate motivation and attention getting ideas

 

been provided for the content to be taught and the

characteristics of the target learners?

 

Have learners been informed of necessary prerequisite
knowledge or skills?

Have students been informed about what they will be
learning to do (common language objectives)?

Have ideas and concepts been presented in a logical
sequence?

Is enough explanation provided for pupils to actually
”learn" the ideas, concepts, and skills?

Are accurate and adequate examples included of ideas,
concepts, and skills being taught?

Are there embedded tests to provide pupils with the
opportunity to practice skills?

Is adequate and accurate feedback provided to pupils

responses to the embedded tests?

3. Are materials legible, clear, and easy to follow?

 

120
It may be expected that the instructional materials will be
prototype or first drafts that are inexpensively produced. Drawings
and illustrations should be stick figures or magic marker drawings
and materials should be inexpensively duplicated. This ”draft”
nature of the materials should not detract from the students'
evaluation of their materials. The important issue is that the
materials include the information on the right pages, and in the
right order, so that it can be tested with target learners, analyzed,

and revised before being produced in a more polished format.

APPENDIX B

SURVEY ON

COM PETENCY BAS ED EDUCATION

3.

Surveygon CBE

 

 

 

 

 

 

 

 

Mager suggests that there are three components to a well written
objective. What are these components?

A.

B.

C.

In the space below write an objective (no.matter in what field).
Taking your previous classes as a guide, what percent of your students
do you:

.A. expect will get 3.5 or better? z

B. expect will get satisfactory 2.0 to 3.02 2

 

C. expect of fail, less than 2.0? 2

 

IZI

l22

4. Do you believe that with proper instruction and more time (if

needed) nearly all of your students could earn 4.0?

 

5. What do you understand by the term "competency"?

 

6. Do you agree with the following statement:

At the undergraduate level we should assign challanging work
so that the good students can stand out and be identified for

graduate work or for leadership in the field?

7. Do you identify the knowledge and skills of the students who come

to your class at the start of an academic quarter?

Yes/No

If yes, how?

123

Please indicate your true belief and considered opinion to the

following statements:

S.Asstrongly agree Mbneutral S.D-strongly disagree

A -agree D-disagree N.API don't understand

S.A A N D S.D N.A

A. In most cases if a student
fails to learn the teacher

should be considered

 

primarily responsible

 

B. Teaching is the application

 

of scientific principles.

C. A teacher's job primarily

should be to give students

 

information.

 

 

D. Instructional technology

is the use of machines in

 

 

 

 

 

 

teaching.

l2“

9. How well do you know the following concepts? Please indicate

with:
N-nothing L=a little 0.K.=enough to explain to
others
N L 0.K.

A. Mastery Learning

 

.- . -._-..--...._ _..

B. Competency Based Education

 

C. Bloom's Toxonomy

 

D. Gagne's hierarchy

 

E. Criterion referenced measures

 

—-——_ . ..»-~...—..—- g.

F. Norm V. criterion referenced

 

u...- .-

tests

G. System approach to education . -_

 

H. Principles of learning

 

 

 

 

I. Formative Evaluation . r

10. Are there concepts which we use in DURC which you are uncomfortable

with and would like to have explained?

 

 

 

11. Do you think that DURC meetings devoted to working through CBE and

related concepts would be:

Very helpful Helpful Waste of time

APPENDIX C

Results of Summative Evaluation
of a module designed as partial

requirement of a Master's degree

CHAPTER V

RESULTS/DISCUSSION

These null hypotheses have been written: (A) dietetic students
at MSU and the University of Hawaii given a self-instructional module
or equivalent lecture presentation to learn the application of the POMR
concept to client nutritional care will show no significant gain in
score from pretest to posttest (both tests measuring equivalent content)
to indicate that learning has taken place, (B) dietetic students at MSU
given a self-instructional module to learn the application of the POMR
concept to client nutritional care will show no significant difference
'in achievement on the posttest compared to dietetic students at MSU
given an equivalent lecture presentation on POMR and same posttest and
(C) dietetic students at MSU given a self-instructional module to learn
the application of the POMR concept to client nutritional care will show
Ia significant difference in achievement on the posttest compared to
dietetic students at the University of Hawaii given the same self-
instructional module and same posttest.

Also an achievement at the Mastery level of ninety percent
accuracy on the posttest would indicate that the dietetic student can
apply Problem-Oriented Medical Record (POMR) knowledge into accurately

writing a dietetic problem, plan of action and SOAP progress note.

IZS

126

When the hypothesis predicts that differences will exist in
the data and allows this difference to be in either direction (larger
or smaller), a two-tailed test of its null hypothesis is performed since
the difference can fall in either tail of sampling distribution (Nie et
a1., 1975).

The T-test provides the capability of computing whether or not
the difference between two sample means is significant. Two types of
tests may be performed.

1. Independent samples: the cases are classified into two groups

and a test of mean differences is
performed for specific variables.

2. Paired samples: for paired observations arranged casewise, a
test of treatment effects is performed. An
example would be the same (or similar) individual
being measured before and after treatment.

The goal of the T-test statistical analysis is-to establish
whether or not a difference between two samples is significant. "Sig-
nificant" is used to mean "indicative of" or "signifying" a true dif-
ference between the two samples (Nie et a1., 1975).

A significance level for testing the null hypothesis was chosen
and set at equal to or less than .05. This significance level is
exactly the probability of rejecting the null hypothesis when it is true
(Nie et a1., 1975).

If it is known whether the two populations have the same vari-
ance, an F test of sample variances may be performed. The null hypo-
thesis with alternative hypothesis and a significance level a' is
chosen. An a' level of .05 was chosen. From the sample variances, F

was computed. If the probability for F is greater than a', the t value

based on the pooled-variance estimate is used to determine probability.

127

If the probability for F is less than or equal to a', the t value is
based on the separate variance estimate to determine probability (Nie

et a1., 1975).

Results of Hypothesis Testing
Each hypothesis is presented along with a report of findings.
Five of the protests of the Michigan State University Conventional Study
Plan Dietetic Students were misplaced so the data from these students
could not be included in the analysis of data.

Hypothesis A: Dietetic students at MSU and the University of

 

Hawaii given a self-instructional module or equivalent lecture presen-
tation to learn the application of the POMR concept to client nutri-
tional care will show no gain in score from pretest to posttest to indi-
cate that learning has taken place.

This hypothesis was tested using the dependent T-test. Results
are reported on Table 1.

Findings: The null hypothesis was not supported. There was a
significant gain from pretest to posttest for all groups (a < .001).
The pretest scores for both treatments showed greater deviation compared
to the deviation in the posttest scores, which may also indicate that
more students learned the POMR concept in an equivalent manner.

Hypothesis 8: Dietetic students at MSU receiving the self—

 

instructional module on POMR will not show a significant difference in
achievement on the posttest compared to dietetic students at MSU given
an equivalent lecture presentation on POMR and same posttest.

This hypothesis was tested using the independent T-test. As

the dietetic curriculum had two offerings, the Coordinated Study Plan

128

 

.89 v a r. scmonhamdmm

 

 

 

 

 

HNA.~ mmm. mmo.~ house enueceum
mom.mH mmm.m www.mH conumw>oe oumecnum
also. am NA.HH moom.~m moov.mm ccom.~o can: v
Q... u 5 32.2.1
i
saw.~ can.“ m~v.~ nouns pudendum
~N~.- HNo.o vmm.m :oﬂumw>oc uncecnum
aSo. «a mo.w noom.m~ mmmm.om noc~.no emu:
Ame u 2V unsaved
a we on~o> mumouosm umouumon unmoved asowo
e unmouumodv
coconommwo

 

 

.mucopsum
owuouoﬁo “Mose: mo xuﬁmso>ﬂca on» men xuwmwo>ﬁ== madam ccmﬁnoﬁz mom mucoEHmoab oasmoz can
ousness cw mopoom coo: eunucoouon «monumom one awesome coozuom oucoaommﬁa pom amoune mo muﬁsmomlu.~ o~nm

 

129

and Conventional Study Plan, the data were collected separately for
students enrolled in each plan. Results are reported on Table 2.
Findings: The null hypothesis was supported. The difference
in achievement on the posttest scores for the module and lecture groups
was not significantly different. Further analysis revealed that the
Conventional Study Plan students in the module group had the lowest
standard deviation. The Coordinated Study Plan students in the module
treatment may have been a more homogeneous group. The pretest scores
were analyzed at the same time the posttest scores were analyzed and so
were also included on Table 2. There was a significant difference in
pretest scores between the lecture and module groups of the MSU Coor-
dinated Study Plan students (.001) whereas pretest scores in the lecture
and module treatments for the MSU Conventional Study Plan students were
notrsignificantly different.

Hypothesis C: Dietetic students at Michigan State University

 

receiving the self-instructional module on POMR will show a significant
difference in achievement on the posttest compared to dietetic students
at the University of Hawaii given the same self-instructional module
and posttest.

This hypothesis was tested using the independent T-test. As the
dietetic curriculum had two offerings, the Coordinated Study Plan and
Conventional Study Plan, the data were collected separately for students
enrolled in each plan. Results are recorded on Table 3.

Findings: The null hypothesis was not supported. The differences
in achievement on the posttest percentage mean scores were not

significant.

130

 

.pom: mm: oumsnumo oocmnno> pudendum <

.moms mm: unmanumo oonmnne> ponoom <

 

 

 

o n
.8. v a n... nﬁunmnemnmm
mam.n man. was.~ nmc.~ nonno enmecmnm
mnm.a mme.~ oao.v ons.e connmn>oe unneemnm
poem. an kn.n owmv.nm ooov.ew
omen. km.nn mn.n ooom.mm ooon.~m can: .
umouumom
men.v eso.~ Nan.m onm.m nonnm enmeemnm
nso~.nn ncm.n Nmo.n one.On connnn>oe enmecmnm
mnnm. en vo.n nmm~.mn cocv.me
pence. wn ma.m ooom.mv ooo~.oo can:
amouona
Anuzd nonuzv amaze Acnuzd
.nonn we osnm> .>:ou .>:oo .>:ou .enoou
nnmn-o3n n nnaza: .amz nnmzn: am: nmon
cameo: onsuooq

 

 

 

 

 

I4

.mncopsum team xpsum m.>:ouv ﬁneonuco>cou one

a.onooov moumtnmnoou xuwmno>nca madam damage“: you nanoseconh onsmoz mom onsuooq toosuom

monoom coo: ommucoonon amonnmoa en ooeonommna new unsound an outonommﬂa now swoon? u:op=odom:~--.~ unnoh

 

 

131

 

41

.mom: was oqunumo oocmnnn> ounnmmom <0

.mom: was unmanumo ooconno> monooa <

a

.mo. v a nu ncmunnnnmnma

 

 

 

mma. mam.m mam. mnn. nonno enmecmnm
mno.m mnm.m mno.m ~ms.n connmnsoe enmenmnm
owns. nm.o as. nooo.mm owne.nm
omen. mm on.n noeo.ma ooom.mm can:
nmonnmoa
nan.m men.v nan.m osc.~ nonno enmenmnm
«so.Vn nvo.nn svo.sn nom.m connmn>oe ennecanm
gene. on mm.n ooo~.mo nmw~.mn
namno. mn nn.~ ooo~.mo ooon.mv can:
umoumhn—
Amnuzd Anuzd Amnuzd nonuzd nmon
.nonn we o=Ho> .>:ou .>:ou .>:ou .mnoou
nnan-ozn n nnazm: am: nnmzn: am:
unspoz cameo:

 

 

 

 

 

.mutomsum tend

nesum a.>:ou nnmzmzv nmconueo>eou nnmzmz mo nunmno>ncs onu new mucoosum unnouono tend nusnm
ﬁ.>:oov unconnco>cou men n.mnoouv pounenpnoou nunmno>nta oumum townnon: now utosumonb onsmoz :n
monoom one: ownncoonoa anonymod en coconoMMnn can «monond :n ooconomwno now unease utomeomomcnuu.m once

 

 

132

Further analysis (Table 3) revealed that the pretest percentage
mean scores of the Michigan State University Coordinated Study Plan
students and the University of Hawaii Conventional Study Plan students
were significantly different (.013). In comparing the Conventional
Study Plan students at Michigan State University and the University of
Hawaii Conventional Study Plan students, it is seen that neither their
pretest or posttest percentage mean scores were significantly different
as also seen on Table 3.

The dietetic students at Michigan State University and the
University of Hawaii achieving ninety percent on the posttest are
recorded on Table 4. The majority of students in each group achieved

the ninety percent criterion level.

Reliability of Posttest Evaluators
The calculation of a Reliability Coefficient was used for esti-
mating the reliability of the two independent evaluators of the
posttest (Ebel, 1972).

The formula is:

 

 

anyg- Zx§y
r:
Jlnzxz - (EX)21.[n£y2 - (XX)21

The results of the formula indicated that the reliability coefficient
was .69 indicating that the two evaluators had sixty-nine percent
reliability. The score can range from zero to one with scores near one

indicating high reliability between raters.

l33

Table 4.--Number of Dietetic Students in Lecture and Module Treatments
of Conventional and Coordinated Study Plan Students at
Michigan State University and the University of Hawaii
Achieving Ninety Percent on the Posttest.

 

 

 

Number of Number of Students
Treatment Students Achieving 90% on
Posttest
{Itr‘
Lecture
MSU
Conventional S 4
Coordinated 10 9
(Module
MSU
Conventional 7 ‘ S
Coordinated 10 9
Module
University of
Hawaii
Conventional 15 14
TOTAL 47 ' 41

 

 

 

 

134

Comments

The students' feelings (affect) regarding the module were
recorded. They felt that more time was needed to be Spent on the
Assessment Component of the SOAP Progress Note and less time on
learning about Dietetic Problems. The students felt that they derived
benefit from the embedded test questions while going through the
module. The affective aspects of the module (music, cartoons and
statements about their success, e.g., You're Doing Fine!) were enjoyed
by all of the students and recommended to be left in the module. The
dietetic students at the University of Hawaii thought that the cartoons
of lesson two of the module needed to be more relevant to the content.
Students reported there was confusion when they had to move back and
forth between the slide tape and student booklet but this confusion
decreased as the student kept going through the module. Also clarity
as to whether or not only one answer is acceptable for the embedded
test questions is needed. Those students attending the lecture treat-
ment felt that the lecture was lengthy and repetitious but that the

case study examples used were helpful. The students overall felt that

they learned how to write SOAP notes in the correct format.

l35

CHAPTER VI

CONCLUSIONS/RECOMMENDATIONS

Conclusions

 

Major conclusions from this study concern Hypotheses A, B and
C. Null Hypothesis A states that dietetic students at MSU and the
University of Hawaii given a self-instructional module or equivalent
lecture presentation to learn the application of the POMR concept to
client nutritional care will show no gain in score from pretest to
posttest (both tests measuring equivalent content) to indicate that
learning has taken place. Hypothesis A was rejected since there was a
significant gain in score from pretest to posttest to indicate that
learning had taken place. Null Hypothesis B states that dietetic
students at MSU given a self-instructional module to learn the appli-
cation of the POMR concept to client nutritional care will show no
significant difference in achievement on the posttest compared to
dietetic students given an equivalent lecture presentation to learn
the application of the POMR concept to client nutritional care and
same posttest. Null Hypothesis 8 was accepted since there was not a
significant difference in posttest percentage mean scores between the
module and lecture groups in either the Coordinated Study Plan or
conventional Study Plan dietetic students at Michigan State University.

Null Hypothesis C states that dietetic students at MSU given a

136

self-instructional module to learn the application of the POMR concept
to client nutritional care will Show a significant difference in
achievement on the posttest compared to dietetic students at the
University of Hawaii given the same self-instructional module to learn
the application of the POMR concept to client nutritional care and same
posttest.

Null Hypothesis C was rejected since the posttest scores of the
dietetic students at the University of Hawaii receiving the module
treatment were not significantly different than the posttest scores
of the Michigan State University dietetic students in the module
groups. The University of Hawaii's dietetic students' high posttest
scores may be accounted for by the fact that they were being offered
an opportunity to participate in'a project being offered by another
large mainland university. These students could have developed
interest and curiosity with a desire to perform well.

Forty one of the forty seven students (93%) participating in
this study were able to achieve at the ninety percent criterion level
on the posttest regardless of being in the lecture or module group.
Over ninety percent of the University of Hawaii dietetic students
achieved ninety percent on the posttest which may indicate that the
POMR module has the potential of being used with other dietetic
programs with their students achieving the ninety percent criterion
level on the same posttest.

The raters of the posttest for all groups showed a sixty nine
percent reliability coefficient between each other. The use of the

performance criteria checklist used in scoring the posttests probably

I37

accounted for this reliability and further clarification of the

criteria may yield higher results.

Recommendations

 

A major recommendation would be that the module be revised for
clearer understanding based on the comments of the students. The
whole module is being reviewed by the designer's Graduate Committee
Members and Clinical Faculty of the Coordinated Study Plan in Dietetics
at Michigan State University in order to convert it from a prototype
into a Professional Instructional Module.

A self-instructional module should be produced using a content
area that does not change frequently in order to have the time spent
in design and production worthwhile.

Evaluation of more pre- and posttests of the module with'more
dietetic students could be conducted to increase the validity of the
module.

It appears that the hierarchical process used to develop a final
learning outcome (Hiob, 1978) has application for other types of cogni-
tive learning in order to develop a sequential path leading the student
to mastery of a subject. 'This thesis study has shown that the hier-
archical process is successful in aiding the dietetic student in
learning the application of the POMR concept to client nutritional care
and, in the designer's opinion, be applied to the develOpment of other

materials in the cognitive domain in dietetic curriculums.

APPENDIX D

THE SELF INSTRUCTIONAL MANUAL.

A Self Instructional Manual for the

Systematic Development of Modules:
The M.S.U. * Dietetic Model.

Frank Hiob
January, 1978.

*Michigan State University

The development of this Manual and the M.S.U . Dietetic Model
were partly supported by a grant (71-2926) from the W. K. Kellogg

Foundation .

138

139

Acknowledgements

The development of this Manual was partly supported by
W. K. Kellogg Foundation-Grant 21-2627.

A number of people have been very supportative throughout this
project, especially Professor B. Wenberg, together with the faculty
and graduate students of the Department of Food Science and Human
Nutrition at Michigan State University

Many of the modules in this Manual have been adapted from
existing materials. Whenever possible, permission to adapt the
original materials has been sought from authors when they could be
identified.

Two sources of materials need further acknowledgement: first,
unpublished material acquired mainly as class notes at Florida State
University, especially the text The Systematic Desigp_of Instruction
by Walter Dick and Lou Carey (roneod) which has subsequently been
published by Scott, Foresman and Co., Glenview, Illinois (1978) under
the same title; and second, unpublished materials from various
sources at Michigan State University, particularly from Dr. C. Gentry's
classes.

Many people have contributed in providing suggestions and
testing the Manual -- to all these people I wish to express my

gratitude.

IllO

WELCOME

This is a self instructional Manual on how to design and
produce modules. It has been formatively tested and is currently
going through wider field testing . The author would be grateful
for comments and suggestions to the further improvement of
this Manual .

The Manual is made up of a series of modules . You are
asked to work through them all. Tests in various forms through-
out the Manual will give you an indication of your progress;
there may be a number of modules which you can skim through
while others may need considerable concentration.

The more effort you put into your work the greater will

be your reward (anon. ).

141

Introduction

 

The use of individualized self instructional programs is growing
faster than any other method of improving instruction (Cross 1976). The
reason for this focus is quite simple: students differ along numerous
dimensions and well designed individualized instruction considers these
individual variations and attempts to provide for them. The effectiveness
of individualized instruction has been demonstrated in a large number of

studies (Kulik 1974, Keller 1968).

What is individualized instruction? If asked, each reader could
probably provide a definition of individualized instruction and believe
it was fairly accurate and reflective of the current thinking of most
educators about individualization. However, if these definitions were
shared, it would be found that there are almost as many definitions of
individualized instruction as there are persons using the term. Current
definitions vary from those which indicate that students will proceed at
their own rate through a prescribed set of materials to reach a predeter-
mined set of objectives, to definitions which indicate that students
will be free to select their own means of achieving their own objectives.
These two types of definitions reflect in part the two traditions which
have contributed the most to the growth of individualized instruction.
These two approaches may be characterized as the humanistic approach and

the behavorial science or systems approach.

142

Educators who consider themselves in the humanistic camp have a
genuine interest in the total development of individual children. They
recognize the importance of individual differences and believe that the
essence of an outstanding education is the display of genuine care and
concern for students as they attempt to define those areas of learning
which.are important and relevant to them. There is a strong focus on the
personal growth and development of the individual student. This emphasis
on individual personal development and human relationships is an attempt
to counteract the increasing alienation which students encounter in
their society and perhaps in their own homes. The rapid growth of the
book sales in the personal development area and the establishment of
various personal development groups reflect our society's interest in
these same problems.

Those.educators who favor the behavioral science or systematic
approach to education have also had a significant impact on the individ-
ualization of instruction during the past decade. This approach emphasizes
the consideration of the inputs, processes and outputs of the education
process. Research has been conducted on how to identify content for
inclusion in the curriculum, how to better assess the knowledge and
abilities of students, how to better design classroom procedures and in-
structional materials, and how to make instruction more effective.

The systematic approach to instruction had its initial impact in
the development of programmed instruction. This medium of instruction
emphasizes the importance of a precise definition of what it is that
the student will learn and the importance of careful structuring of in-
structional materials. Programmed instruction utilizes active student
participation in instruction to facilitate achievement of given objectives.

iMany of the teaching principles of programmed instruction are applied

143

today in the systematic design of instructional materials, although
programmed instruction per se is not in great use.

Both of these instructional approaches - the humanistic and the
systems - emphasize the significance of individual differences and the

necessity of providing appropriate instruction to the student.

 

Self Test

After each of these statements write (H) for humanistic or (S) for
systematic approach:

1. Programmed learning is an example of which major approach? ( )

2. Behavioral science prefers an approach which is ( ).

3. A precise definition of leaning is a feature of ( ) approach.

4. Personal growth and development of the total individual is the

major concern of ( ) approach.

m

It should be noted that while in academic circles representatives
of humanism and behaviorism debate the merits of their approaches, there
is little evidence of this conflict when one views individualized instruction
in use in classrooms. Recent studies by Fox and DeVault (1974) have indicated
that the best examples of individualized instruction are those which blend
the best of both the humanistic and the systems approaches to instruction.
While the MSU dietetic model will advocate the behavioral science
approach to designing, developing, and evaluating instructional materials,
the authors are in full accord that humanistic and systems approaches must

be integrated to provide the best atmosphere for effective student learning.

 

Feedback: 1. Systematic
2. Systematic
3. Systematic

4. Humanistic

 

IAS

There are three major components in the implementation of individ-
ualized instruction. The first involves the role of teachers; the
second is the preparation of instructional materials; and the third is
the delivery of instruction to students in the classroom. With the
emphasis on selfinstructional materials, the instructor no longer plays
the role of disseminator of all information. However, it is not totally
agreed upon as to what the new role of the instructor should be. In
some settings the instructor's role has been downgraded to a scorer of
test papers and in others it has been upgraded to a general learning
consultant to students. Clearly this is an area in need of continuing
research because of the tremendous implications for the total instructional
setting.

Instructional designers who are preparing materials for use in
individualized systems have neither ignored the role of teachers nor
have they had it as their major concern. Their central purpose has been
to identify those objectives that are to be achieved and to design
instructional strategy and evaluation techniques that are employed.
Therefore new materials designed specifically for selfinstruction are
needed.

The third area of concern is the delivery of instruction. A delivery
system is defined as the integration of the instructor, students, and
the instructional materials in a given setting in order to bring about
the desired learning outcomes. 'Various individualized delivery systems
have been developed, and teachers have implemented methods for meeting
the needs of their students which reflect the instructor's preferred
styles of teaching. No one pattern has universally emerged as the best

and most appropriate one.

l46

Modular Instruction

The types of instructional materials which are typically used in
individualized instruction have come to be referred to as modules. Just
as there is no universally accepted definition of individualized instruction,
similarly there is no general definition of a module. An analogy may
help in this explanation of what constitutes a module. Consider the
technique known in the building industry as modular construction.

Various components of a building are constructed in a factory and shipped
to the construction site. These modules are then placed together in a
particular configuration that results in the construction of a new
building. Workers are still required to drive the nails and place the
screws and bolts which hold the entire structure together. They also
pour the foundation and add the finishing touches which make it a sound
and secure building.

Modular instruction may be considered in much the same way. A
module is a selfcontained or selfinstructional unit of instruction which
has an integrated theme, provides students with information needed to
acquire specified knowledge and skills, and serves as one component of a
total curriculum.

Most instructional designers would agree with the definition given
above. However they would differ on a number of the specific characteristics
of modules. For example, the length of time required for students to
study a module may vary from say one to fifteen hours. Some designers
will insist that a module of instruction should include at least two
alternative presentations of the instructional materials and preferably

two or more modes of presentation to accommodate individual differences.

147

In addition, some designers would argue that a module should be
strictly self-contained. That is, a student should be able to achieve all
the objectives which are stated in the module without interacting with
the teacher or other individuals. Other designers will specifically include
in the design of the module the participation of peers, teachers, and
outsiders in order to involve the student in a variety of interactive
activities. .

Designers even differ on whether students should be informed of the
major objectives for a module. Some designers insist that students should
receive precise statements of the objectives for a module while others
argue that objectives may be rewarded at a level more appropriate for
the student, or that objectives may be omitted all together.

Regardless of the issues listed above, most modules involve some type
of active participation by the student in terms of interacting with the
instructional materials rather than being a passive reader of the materials.
The student is asked to perform various types of learning tasks and receives
feedback on that performance. There is some type of testing strategy which
indicates to students if they have achieved mastery of the content, and
what they should do if they have not yet achieved mastery.

Based upon the description of prior paragraphs, how would you recognize
a module if you saw one? In its most simple form, a module might be a
typewritten statement to students which indicates what it is they are
about to learn and how they will be tested. It would provide printed
or typed instructional materials as well as some practice test items. A
self-test which might be used prior to taking a terminal test could also
be included.

The most complex module might contain all of the items listed above,
but also incorporate a number of alternative sets of materials from which

the student might choose and the most appropriate one for him or herself.

148

Alternative media forms such as an audiotape or a filmstrip might be included.
In addition, the student might go to a laboratory to conduct an experiment
or go outside the school to gather information.

Regardless of the simplicity or complexity of a module, it should be
validated, that is, it should be demonstrated that students learn from it-
that they can perform the skills as described in the objectives for the
module. Methods have been developed which are used to obtain information
from.students as a module is being developed to improve the quality of
the module. After the module has been completed, data is collected which
is used to demonstrate the extent to which the module is effective in

bringing about anticipated changes in student ability.

 

A. Judge the following statements as True or False
1. Most modules require active interaction by the student with
instructional materials.
2. Tests are an important part of a module.
3. Tests are used to determine where the student stands in relation
to other students.

4. The types of instructional materials which are typically used

in personalized instruction.have come to be.referred to as

modules;

B. In a sentence or two state what you understand a module to be:

Feedback: 1. True 2. True 3. False 4. True
B. A module is.a self-contained or self-instructional.unit of instruction
*which has an integrated theme, provides student with information needed to acquire

__§pecified knowledge and skills, and serves as one component of a total curriculum.

149

Quiz

The major trend over the past 10 years in the method of instruction
has been towards

a. Standardizing

B. Individualizing

c. 'discovery

d. none of the above.

Educators in the humanistic camp believe in

a. strong personal growth of the individual student
b. the total development of individual children

c. the importance of individual differences

d. all of the above

The M.S.U. Dietetic model is based on

a. humanistic philosophy

b. behavioral sciences approach

c. on both a and b

d. none of the above.

The educators who prefer the behavioral science approach believe also
a. in the systematic approach

b. in the precise statement of objectives

c. the effectiveness of programmed instruction

d. all of the above

Do you consider this statement true or false?

.Both the humanistic and the systems approach emphasize the signi-

ficance of individual differences and the necessity of providing

appropriate instruction to the student.

150

6. What is a module? Do you agree with this definition: a module
is a self-contained or self-instructional unit of instruction which
has an integrated theme, provides students with information needed

to acquire specific knowledge and skills and serves as one component

of a total curriculum.

Feedback:

1. (b) 2. (d) 3. (c) 4. (d) 5. False 6. Yes

151

The focus of this Manual is on learning. We will constantly
ask the question "how does the person learn?" This approach
differs markedly from the usual "how will I teach?"

Well then, how does the person learn? The answers are
provided in this Manual , starting with the next module on how

we process information .

152

Our Model

Given the need to develOp individualized instructional
materials and being acquainted with the systems concept, what
should we do next? One seemingly reasonable approach would be to
use an already existing module as a model . There are several prob-
lems associated with this approach. Any given module is designed
to teach a particular type of learning to a particular type of student.
What is needed is a more generalized model - one which will describe
the procedures for developing a module regardless of the type of
learner or the type of learning which is to occur.

One general model for designing instructional materials is
referred to as the systems approach model. It must be emphasized
that there is no single systems approach model for designing instruct-
ion. There are a number of models which bear the labs "systems
approach, " and all of them include most of the same basic compon-
ents. The systems approach model which will be presented includes
the major components which are included in other models.

The systems approach models are an outgrowth of over 20 years
of research into the learning process. Each component of the model
is based upon theoretical and/or research outcomes which demon-
strate the effectiveness of that component. The model brings to-
gether in one coherent whole many of the concepts which you may
have already encountered in a variety of educational situations.

The model is graphically presented on the next page. It includes
six interconnected boxes. The boxes (each step) refer to sets of pro-

cedures and techniques which are employed by the instructional des-

igner to design, produce, evaluate and revise an instructional module.

153

6980 2282 not moose 0:8an .3.m..2 ect.}; 8:9”.

 

 

 

 

to:n:_o>o
til...
QDOLU

 

 

.IZtm.
etc on oto.
co_uu:_o>o
. o>3orttou

 

 

 

towns—
goatumcoo

 

 

0 noum

m noum

_—Iu.u>uu J

 

aLI

 

 

 

 

 

 

 

 

 

moEouuao
u:.:nne_
on>_6c(

 

 

 

nquonso
imitated—
DZOOQW

 

 

 

 

n noam
upstage.
amouumOQ.

outposts.
anaemia.

nzooet _o>o_
mono—trues acute.
tozn>30rc mummy
"commo— coru to.tmu.to
Qo_>u0

v noum 0 Guam

N acum

.QOum

 

 

154

How We Process Information

Recent research on human learning has focused on how people think
rather than how they respond to stimuli (the old Pavlov's dog, stimulus
response idea). This recent research has generated a body of theory
that explains how people take in information and how they organize
information in memory. These activities of assimilating and arranging
information are known collectively as information processing.

By and large, researchers agree that human perception and human
memory impose rigorous organization on what is learned and on how it is
learned. Concepts are not stored randomly, but rather are related to
other similar concepts in clusters, which in turn are related to other
clusters of concepts. The whole forms a logical and often measurable
cognitive structure. In addition, it is generally agreed that cognitive
structures are changed when new concepts are learned, and also that they
in turn act upon those new concepts to make them.more congruent with
existing structures. What we learn changes what we know and what we
know changes what we learn.

The interesting question to ask now is: What is it that goes on
inside the student's head when he learns? What is the process of learning
like?

Comtemporary theories conceive of learning as a matter of
information processing. Stimulation from the learner's environment
affects his central nervous system by a series of processing stages.

The transformed information is stored in memory, and a final transformation
makes possible a performance that is evident to an external observer. A

'more-or—less standard model employed in these theories is onlnext.page:

 

.2 .m .wnmmo sonACIId 0...:me

\\\I/J

Aswan .mmonm nowhnn .nOnpopnumnH non mcncnmoq no maunucommm

 

 

 

155

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

m m
m m
n > o
:85: o“ m w
Iv 25» o m m n
. ml .505 m o z
:85: m m u m
Fllml m .2
257023 _II. 2
11 o
m m
_ . _
r 5.252% o >
_ x o ,Illv m
I mmzoemum u
a
a
o o e o . o o .

 

 

 

 

 

C

r mw.UZ<hmeXm H— —1._OIL.ZOQ w>.hDUmeL

 

156
(l) A.number of contemporary learning theories treat learning and
memory as informationeprocessing, deriving their constructs
from.computer science and linguistic theory, as well as from
psychology. Atkinson and Shiffrin, 1968; Linday and Norman
1972. We use Atkinson-Shiffrin as an example of the general
model.
This is the way the model works.
Stimulation from the learner's environment affects his receptors
and enters the nervous system via a sensory register. This is the structure
responsible for the initial perception of objects and events that the
learner sees, hears, or otherwise senses. The information is "coded"
in the sensory register, that is, it has the form of a patterned repre-
sentation of the original stimulation. The information remains in this
form for only the smallest fraction of a second.
Entering the short-term memory, the information is again coded,
this time into a.conceptual form. Thus, a figure like X becomes a
representation such as an "X", a figure like 2 becomes the concept "two".
(not the word two). Persistence in the short-term memory is relatively
brief, a matter of seconds. However, the information may be processed
by internal rehearsal and thus preserved in the short-term memory for
longer periods. Rehearsal may also play a part in another operation; if the
information is to be remembered, it is once again transformed and enters
the long-term memory, where it is stored for later recall. ‘Most theories
assume that storage in long-term memory is permanent and that later
failures to recall result from difficulties of "finding" the information.
It is important to note that the short-term and long-term memories
may not actually be different structures, but only different ways of
functioning of the same structure. Notice also that information that has

passed from.the short-term memory to the long-term memory may be retrieved

157

back to short-term memory. The latter is sometimes spoken of as the
"working memory" or the "conscious memory". When new learning depends
partly on the recall of something that has previously been learned,
this something must be retrieved from long-term memory and must re-
enter the short-term memory.

Information from.either short-term or long-term memory, when
retrieved, passes to a response generator, which has the function of
transforming the information into action. The neural "message" from
this structure activates the effectors (muscles), producing a performance
which affects the learner's environment. This action is what enables
the external observer to tell that the.stimalatinnfhasohad its expected
effect-that the "information has been processed", and the learner has
indeed learned.

A very important set of structures has yet to be described. These
are labeled executive control and expectancies. Signals from these
structures are presumed to activate and modify the flow of information.
For example, the learner has an expectancy of what he will be able to do
once he has learned, and this in turn may affect how an external stimulus
is perceived, how it is coded in memory, and how it is transformed into
action. Control processes originating in the executive control structure
may determine how the information is coded when it enters long-term memory
and how the search and retrieval are conducted for recall, among other things.
(References to various theoretical accounts are: Atkinson & Shiffrin, 1968;

Norman, 1970; Anderson & Bower,.l972; Lindsay & Norman, 1972).

158
Quiz

According to modern learning theory, stimulation from the learner's
environment affects his sense organs, and is transformed to patterns
of neural "information" which are held for very brief periods in this

form. What is the structure called that performs this function?

A learner has acquired and stored the skill of expressing ounces as
pounds and fractions of pounds. Some days later, he is required to
use this skill in comparing the contents of cans of coffee. In re-

calling the skill, what process is involved?.

In the example of 7, if what has been learned is stored in Long-term

Memory, to what structure does it fow, when recall occurs?

0......00.....0...0.00000000000000000000000000000000000COOOOOOOOOOOOO

This is the most widely accepted information processing model. Please

label the processes.

v v: ‘9 v v .«5

 

 

 

 

 

 

 

 

 

 

 

 

 

.H’

 

 

 

 

 

 

 

 

 

 

 

 

 

159

Feedback to Quiz

Sensory Register. The initial transformation of sense organ
stimulation leads to very brief (of the order of hundredths of a
second) neural effects, a kind of "registration" or pure "perception"
of this stimulation. To be further processed, the information must

be transformed and enter Short-term.Memory in.a "coded" form.

Retrieval. The skill Of expressing ounces as pounds and fractions

of pounds must be retrieved from Long-term Memory.

Short-term Memory. The process Of retrieval frequently results in
the stored skill being returned to the Short-term Memory, which is

sometimes called the "working memory".

 

F} I EXECUTIVE COSTROL PROCESSES; J

 

 

 

 

 

 

 

 

 

 

 

 

 
 
  
   
 

 

 

 

 

5 ‘W «v v v 4!
F
E F
N E RESPONSE
v ‘. c ~ I
L o GENERATOR
o R
s
N
M
E a ‘ 's LONGJERM
¥ E E R t
c N E SHORT- MEMORY
f g ? TERM
'3 "—5 5 3. ‘ MBXORY
R E
j s R
' .1

 

 

 

 

 

 

160

In recent years a great deal has been written about Competency
Based Education (CBE). Unfortunately there do not seem to be
.a_nx successful programs in Operation which are entirely competency
based. It seems that a CBE program is too complex and radical
to be fully implemented and accepted.

This Manual follows many concepts of CBE, however it
prefers to learn from realities Of the world and therefore picks
and chooses. A reasonably accurate label for this Manual is to
say that it is "mastery based". The two following modules will
acquaint you with this all important concept of mastery and

m astery learning .

Answer

16]

Pretest Mastererearning

In mastery learning, which of the following is (are) held
constant?

a) Time.

h) Instructional activities.
c) Achievement.

d) All Of the above.

e) None Of the above.

Which of the following may vary with each student in mastery
learning?

a) The time it takes to achieve mastery.

b) The number of activities undertaken to achieve mastery.
c) The number of post-assessments.

d) All Of the above.

e) None Of the above. ‘

One (or more) basic beliefs underlying mastery learning is (are):

a) Mbst students can be competent at almost anything if
sufficiently enticed by grades.

b) Mbst students can become competent at almost anything if
given enough time.

c) Most students can become competent at almost anything if
they have a high I.Q.

d) All of the above.

e) None Of the above.

In a competency-based program which one of the following is more
likely to happen than the others?

a) The raising of standards.
b) Lowering Of standards.
c) None Of the above.

In mastery learning you bring all students to a designated level
of performance. This statement is essentially:

a) True.
b) False.

How do you handle the concept of failure in mastery learning?

a) We fail only a very small percentage Of students.
b) We fail a relatively large percentage of students.
c) We fail only the normal amount of students.

d) None of the above.

10.

ll.

12.

162

In developing a mastery learning type program.which Of the
following would be considered an essential task?

a) Obtain objectives and state them clearly to students before
the pre—assessment. . .

b) Obtain Objectives and do not give them to the students.

c) Obtain Objectives and give them to students after the
instructional activities.

d) All of the above.

e) NOne Of the above.

An important concept in the building Of a mastery learning type
program is:

a) To hold achievement constant and vary the time.

b) TO make Objectives explicit to the students.

c) To see that students receive adequate entry skills in order
to tackle the program successfully.

d) All of the above.

e) None of the above.

For evaluation in a mastery learning type program, which of the
following is acceptable?

a) Test the student in reference to the published instructional
and expressive objectives. ,

b) Since students do not know what they have to learn, eliminate
tests.

c) Use the tests to separate the best students from the worst
students.

d) All of the above.

e) None Of the above.

If a student does not pass the post-assessment in a competency-based

program you would:

a) Drop him from the course.

b) Provide alternate routes of instruction.
c) Allow him to proceed to the next module.
d) All Of the above.

e) None of the above.

In mastery learning, ability to learn may be defined by:

a) The amount Of time it takes a student to learn.
b) The score on an intelligence test.

c) The grade he received in previous subjects.

d) All of the above.

a) None of the above.

Mastery learning places heavy emphasis on:

a) Entrance requirements.

b) Grading.

c) Keeping time constant for each student.
d) All Of the above.

e) NOne of the above.

163

Feedback:

1. e 5. a 9. a
2. d 6. d 10. b
3. b 7. a 11. a

I61!
MASTERY LEARNING

The most wasteful and destructive aspect of our present
educational system is the set of expectations about student learning
each teacher brings to the beginning of a new course or term . The
instructor expects a third of his pupils to learn well what is taught,
a third to learn less well, and a third to fail or just "get by". These
expectations are transmitted to the pupils through school grading
policies and practices and through the methods and materials Of
instruction. Students quickly learn to act in accordance with them,
and the final sorting through the grading process approximates the
teacher's original expectations. A pernicious self-fulfill ing
prOphecy has been created.

Such a system fixes the academic goals of teachers and
students. It reduces teachers' aspirations and students' desires
for further learning . Further, it systematically destroys the ego
and self-concept of a sizeable proportion Of students who are legally
required to attend school for ten to twelve years under conditions
which are repeatedly frustrating and humiliating. The costs of such
a system in reducing student Opportunities for further learning and
in alienating youth from both school and society are too great to be
borne for long .

Most students (perhaps over 90 per cent) ga_n master what we

teach . Our basic instructional task is to define what we mean by

165

mastery Of a subject and to discover methods and materials to
help the largest proportion of our students reach it.

--Benjamin S. Bloom
Human Characteristics
and School Learning,
McGraw Hill, N.Y., 1976.

I66

Introduction .

Currently the strongest movement is the development of
instructional programs to meet individual student needs. This is not
a new theme but it has only been in the last decade, with developments
in Technology and the systems approach, that such programs have been
implemented on any large scale basis.

The basic argument in favor of individualizing instruction
comes from a multitude of research and evaluation studies that suggest
student differences in interests, motivation, learning rate, goals, and
capacity for learning, among other things; and, therefore, group-based
instruction on a common curriculum is inappropriate to meet their
educational needs. In addition, present group instruction practices fail
to help the student to cultivate a sense of responsibility for his educational,
personal , and social development or to make realistic educational decisions
and choices about his future.

The trend toward individualization of instruction has resulted in
the development of a diverse collection of attractive alternative models.
In this paper I will discuss Mastery Learning and allude to Individually
Prescribed Instruction (IPI) and Program for Learning in Accordance
with Needs (PLAN). For descriptions of IPI see Glaser 1968,1970,

and for PLAN see Flanagan 1967,1971 .

167

The mastery learning concept was introduced to American
schools in the 1920's with the work of Washburne (1922) and others in
the format of the Winnetka Plan. The program flourished in the 1920's;
however, without the technology to sustain a successful program, interest
among developers and implementers steadily diminished (Block,1971).
According to Block (1971), mastery learning was revived in the form of
programmed instruction in the late 1950's in an attempt to provide students
with instructional materials that would allow them to move at their own
pace and receive constant feedback on their level of mastery. But
programmed instruction was not effective for all students, and so, in
an attempt to handle individual differences better, Bloom (1968) and his
students (Airasian, 1971; Block, 1971) improved on the standard programmed
instruction model by combining it with a model of school learning developed
by Carroll (1963, 1970). Carroll's model of school learning provided
the conceptual framework for more effective handling of individual differences
within an Objective-based curriculum . In brief, Carroll's model states that
the level of mastery reached by a student on any instructional task or school
Objective is a function of the time actually spent learning the material and
the amount of time he needs to master the material. The amount Of time

a student actually spends learning the material depends on two factors -

168

time allowed, and his perseverance. The amount of time needed by the
student is dependent on three factors - aptitude, quality of the instructional
materials, and his ability to understand the instructional materials.
Carroll goes on to explain how these five factors interact to effect student
success in school learning.

Since Bloom's original paper in 1968 describing mastery learning,
a considerable amount Of mastery learning research has been conducted,
and the results suggest that the mastery learning model can be easily and
inexpensively implemented in courses at any level of education and in a wide
range of content areas (Block, 1970). In particular, Block (1971) notes
that the best results have been obtained when the course requires either
minimal prior learning or previous learning, which all or almost all Of
the students possess. In addition, various research findings have shown
better results in courses when the content is highly structured and sequential
in nature.

The outstanding features of mastery learning appear to be that it
is easily implementable, does not require the use of a computer to manage
instruction, and is appropriate . for almost any content area. Also, if
mastery learning is carried out properly, previous research suggests that
students will achieve higher scores and have more interest and a better
attitude toward school.

The curriculum is organized into units of instruction defined by
homogeneous clusters of behavioral objectives. Initial instruction on the

objectives covered in the unit is group-based. In this respect, mastery

I69

learning is structurally different from I PI and Project PLAN. For each
unit, one or more criterion-referenced tests, called formative tests, are
used to assess mastery of the Objectives. These tests are administered
immediately following the completion of the group-based instruction.
Individual ization is handled via supplemental materials, feedback, and
corrective techniques, applied to students who fail to achieve the defined
level of mastery on the test items covering the unit Objectives . Following
the last unit of instruction in the course, a final test covering a represent-
ative sample of course objectives is administered, and the data used for
grading purposes.

In describing the mastery learning model, Mayo (1971) notes that:

1. Students are made aware of course and unit expectations, so

that they view learning as a cooperative rather unan as a
competitive vents re.

2 . Standards of mastery are set in advance for the students, and
grading is in terms Of absolute performance rather than
relative performance.

3. Short diagnostic tests are used at the end of each
instructional unit.

4. Additional learning is prescribed for those who do not
demonstrate unit mastery.

5. Additional time for learning is prescribed to students who

seem to need it.

In summary, it should be noted that many variations on the

basic mastery model, as originally proposed by Bloom (1968), are being

I70

implemented in schools. For example, different implementers tend to
vary in the extent to which feedback/correction procedures are avail able
and used (Block, 1971_, Kim, 1971).
Block (1971) notes that "To individualize instruction within the
context Of ordinary group-based instruction, mastery learning relies
heavily on the constant flow Of feedback information to teacher and learner(p.9). "
However, it would seem that there is substantially less testing in a mastery

learning program than in either IPI or Project PLAN.

171

As compared to IPI, there is no placement testing, and unit
pretesting and curriculum-em bedded testing are not emphasized. Unit
posttesting and final assessment represent the two major kinds Of testing
in the program . In the spirit of Scriven (1967), tests to achieve these
two purposes are called "formative" and "summative" tests, respectively.

It should be noted, however, that formative tests, or unit posttests as they
are called in IPI, are not used for grading . The student data derived from
a formative test is used exclusively for diagnosing learning difficulties.

A formative test, or alternately called a diagnostic-process test,
is a criterion-referenced test that is designed to cover the objectives over
a unit of instruction in the mastery learning program . It is used to determine
whether or not a student has mastered the material and to serve as a basis
for prescribing supplemental work in areas where the student is weak
(Airasian, 1971). It is expected also, that the test will reinforce the learning
of high-achieving students. Implementers of the mastery learning model
have set the passing standard anywhere from 75% to 100%. There is no set
number of items or format suggested to measure each Objective; in addition,
there is a suggestion that instructional decisions are made on the basis Of
responses to individual items .

The formative tests in mastery learning represent the key to
individualizing instruction since it is on the basis of the scores on these
tests that individualization of instruction can take place. Units are kept
small so that unit testing takes pl ace frequently in order to increase the

effectiveness of the individualization Of instruction component of the program .

172

Although it remains an unresolved problem, the matter of setting
mastery levels or cutting scores, by which students can be separated into
mastery and non-mastery states on the basis of their performance on test
items designed to measure Objectives included in the criterion-referenced
tests, has been more actively researched in the context Of the mastery
learning program than anywhere. In addition to the usual concern for
setting mastery levels high enough to guarantee that students will have
sufficient amounts of skill to begin the next segment of instruction, Block
(1970) has noted that, in mastery learning, the mastery level is set in a
way that will maximize interest in and attitude toward learning. Some
interesting controlled research studies have revealed that a mastery level
of about 80-85% is substantially better than a level th at is higher or lower.
Bl ock's results suggest that setting mastery levels high (95%) may be best
for cognitive learning but, in the long run, positive attitudes and interest in
the subject are less likely to develop. With a reduction in the mastery level
to 85%, there was a reduction in cognitive learning, but selected affective
outcomes were maximized . If the mastery level is set lower than 80-85%,
students do not usually have sufficient mastery of the skills to proceed
effectively with the instruction.

The prim ary purpose of the summative test in the mastery learning
model is to grade students on the basis of their achievement Of course
objectives. The items in the test are keyed to Objectives and are selected
to be representative of the total pool of course Objectives . A criterion-

referenced interpretation of the scores is recommended. Bloom (1971)

173

prOpOsed that cutting points be located on the ability continuum and that
grades be assigned on the basis of a student's position on the continuum
and not relative to other students in the course. A norm-referenced
interpretation of the scores is also possible (Popham, 1976).

A key part of the mastery learning program is the availability
of an extensive number of instructional methods for use by students who
fail to demonstrate mastery Of the Objectives covered on the formative
test. A formative test is administered at the end of the group-based
instruction on the unit objectives .

Among the alternative resources that are typically avail able
to the student are: small-group problem sessions, individual tutoring,
and programmed instruction, audiovisual methods, academic games and
puzzles, and reteaching.

The developers of the program have left the decision on the
appropriate instructional correctives to the student. It is expected that,
through experimentation with many Of the instructional correct ives, the
student will eventually learn which is "best" for him . This would seem to
be a very realistic solution to the. problem because of the shortage Of
ava'lable data on the apprOpriate matches between student characteristics

and instructional correctives.

17h

POST TEST ON MASTERY LEARNING

By the term "mastery level" we mean
A. the learner has perfected a particular task
B. the minimum accepted level Of learner achievement

C. the maximum measurable achievement level

0

none of the above

In a typical mastery based instructional unit, the learner will
A. always work by himself and not in a group
B. always work in a group

C. work in a group or by himself, depending on how the unit is
organized

In a typical mastery (or competency) based program, the learner
A. is given constant feedback in the form of quizzes or questions

B. is given a test where the scores indicate his level Of success

In mastery learning the resolution of a learning problem by a student
usually requires one Of the following measures:

A. more time for learning
B. different media or materials

C. diagnosis to determine what missing prerequisites, knowledge,
or skills he must first acquire to master the Objective

D. consultation with his teacher

E. all of the above

175

Does Carroll's Model Of School Learning (Mastery Learning) deal with
matters such as (a) perseverance, (b) aptitude, (c) ability to understand
instruction, (d) opportunity to learn?

A. Yes, all Of them

B. NO, none of them

C. Only (a) and (b)

D. Only (c) and (d)

Carroll's Model of School Learning (Mastery Learning)
A. ignores the question of quality of instruction, as it should

B. ignores the question of quality of instruction and therefore is not
very valid

C. takes into consideration the question of quality of instruction

D. quite firmly states that given enough time, the learner can overcome
shortcomings in the quality of instruction

In Carroll's Model for Mastery Learning, the independent variable achievement
has:

A. seven independent variables

B. no independent variables

C. ten independent variables

D. as many independent variables as there are learners

E. I don't really care

According to Carroll, the degree Of learning equals:
A. time allowed for the task

B. learner's aptitude

C. interest Of the learner

D. time actually spent
f

 

time needed

I76

 

9 . Do you think there is any difference between education
and training?
A. Yes
B. No
Feedback
1 . B 5. A
2. C 6. C
3. A 7. A
4. E 8. D
9. A question which always generates a lot of energy but no

"I ight" on the subject. Personally, I do think there is a
difference,as I heard a colleague once say: the trained
person knows the price Of a certain object, the educated
person knows its value.

I77

WE HAVE WAYS FOR YOU TO MASTER
Mastery Learning

Evidence shows that most instructors begin each new quarter
with the expectation that about a third of his students will
adequately learn what he has to teach. He expects about a third
of his students to fail or to just "get by." Finally, he expects
another third to learn a good deal of what he has to teach, but
not enough tO be regarded as "good students." This set of
expectations, supported by policies and practices in grading,
becomes transmitted to the students through the grading proce-
dures and through the methods and materials of instruction. The
system creates a self-fulfilling prophecy such that the final
sorting of students through the grading process becomes approx?

imately equivalent to the original expectations (Bloom 1977).

 

 

1. Most educational institutions employ grading practices such

that inevitably one third of their students will be labeled as

a. having failed or learned only enough to just "get by".
b. having learned only an average amount of what was taught.
c. having learned quite adequately what was taught.

d. all Of the above

178

This set of expectations, which fixes the academic goals Of
teachers and students is the most wasteful and destructive'
aspects of the present educational system. It reduces the
aspirations of both teachers and students; it reduces motivation
for learning in students; and it systematically destroys the ego

and self-concept Of a sizable group of students.

Most students (perhaps over 90 percent) can master what we
have to teach them, and it is the task of instruction to find
the means which will enable our students to master the subject
under consideration. Our basic task is to determine what we
mean by mastery of the subject and to search for the methods and
materials which will enable the largest proportion of our students

to attain such mastery.

 

 

2. Under Bloom's concept of "Learning for Mastery", with appropriate

learning conditions, real mastery of a subject can be achieved by

a. at least 90% Of all students
b. at least 752 of all students
c. at least 652 of all students

d. at least 50% Of all students

179

Part of our problems stem from the use of the bell curve for
assessment.. We have for so long used the normal curve in grading
students that we have come to believe in it. Our achievement measures
are designed to detect differences among our learners, even if the
differences are trivial in terms of the subject matter. ‘We then
distribute our grades in a normal fashion. In any group of students we
expect to have some small percent receive A grades. We are surprised
when.the percentage differs greatly from about 10 percent. We are also
prepared to fail an equal proportion Of students. Quite frequently this
failure is determined by the rank order of the students in the group
rather than by their failure to grasp the essential ideas Of the
course. Thus, we have become accustomed to classify students into
about five categories Of level of performance and to assign grades in
some relative fashion. It matters not that the failures of one year
performed at about the same level as the C students of another year.
Nor does it matter that the A students Of one school do about as well

as the F students of another school.

 

 

3. Using the 'normal' distribution curve as the basis for grading

students assures that

a. measured differences in achievement among students are
important and significant differences.

b. failure in a subject area is determined by the failure to
grasp the essential ideas of the course.

c. both a and b d. neither a nor b

180

Having become "conditioned" to the normal distribution, we set
grade policies in these terms and are horrified when some teacher
attempts to recommend a very different distribution of grades. Ad-
ministrators are constantly on the alert to control teachers who are
"too easy" or "tOO hard" in their grading. A teacher whose grade
distribution is normal will avoid difficulties with administrators.
But even more important, we find ways of convincing students that they
can only do C work or D work by our grading system and even by our
system of quiz and progress testing. Finally, we proceed in our
teaching as though only the minority of our students should be able to

learn what we have to teach.

 

 

4. Given students 'normally' distributed for aptitude in a subject, and

providing all exactly the same learning conditions will result

a. in a normal distribution of achievement.
b. in a + .70 or higher correlation between achievement
and aptitude.

c. both a and b d. neither a nor b

 

 

181
There is nothing sacred about the normal curve. It is the dis-
tribution most appropriate to chance and random activity. Education is
a purposeful activity and we seek to have the students learn what we
have to teach. If we are effective in Our instruction, the distri-
bution of achievement should be very different from the normal curve.
In fact, we may even insist that our educational efforts have been

unsuccessful to the extent to which our distribution of achievement

 

approximates the normal distribution.

Individual differences in learners is a fact that can be demon-
strated in many ways. That our students vary in many ways can never be
forgotten. That these variations must be reflected in learning stand-
ards and achievement criteria is more a reflection Of our policies and
practices rather than the necessities Of the case. Our basic task in
education is to find strategies which will take individual differences
into consideration but which will dO so in such a way as to promote the

fullest development of the individual.

.A learning strategy for mastery may be derived from the work Of
Carroll (1963), supported by the ideas of Morrison (1926), Bruner
(1966), Skinner (1954), Suppes (1966), Goodlad and Anderson (1959), and
Glaser (1968). In presenting these ideas we will refer to some Of the

research findings which bear on them.

182
Put in its most brief form the model proposed by Carroll (1963)

makes it clear that if the students are normally distributed with
respect to aptitude for some subject.(mathematics, science, literature,
history, etc.) and all the students are provided with exactly the gag;
instruction (same in terms Of amount of instruction,.quality of in-
struction, and time available for learning), the end result will be a
normal distribution on an appropriate measure of achievement. Further-
more, the relationship between aptitude and achievement will be re-
latively high (correlations of + .70 or higher are to be expected if
the aptitude and achievement measures are valid and reliable). Con-
versely, if the students are normally distributed with respect to
aptitude, but the kind and quality Of instruction and the amount of
time available for learning are made appropriate to the characteristics
and needs Of gash student, the majority of students may be expected to
achieve mastery of the subject. And, the relationship between aptitude

and achievement should approach zero.

 

 

5. Given students 'normally' distributed for aptitude in a subject
and providing learning;conditions appropriate to the needs of each

individual student will result in

 

a. the majority of the students achieving mastery in
the subject.

b. a correlation between achievement and aptitude
approaching zero.

c. both a and b d. neither a nor b

183
We have come to recognize that individuals do differ in their

aptitudes for particular kinds of learning and over the years we have
developed a large number of aptitude tests to measure these differ-
ences. In study after study we have found that aptitude tests are
relatively good predictors of achievement criteria (achievement tests
or teacher judgments). Thus, a good set of mathematic aptitude tests
given at the beginning of the year will correlate as high as + .70 with
the mathematics achievement tests given at the end Of the course in

algebra, or some other mathematics subject.

The use of aptitude tests for predictive purposes and the high
correlations between such tests and achievement criteria have led many
Of us to the view that high levels of achievement are possible only for
the most able students. From this, it is an easy step to some nOtion
of a causal connection between aptitude and.achievement. The simplest
notion of causality is that the students with high levels of aptitude
can learn the complex ideas of the subject while the students with low

levels of aptitude can learn only the simplest ideas of the subject.

 

 

6. The high correlation usually found between aptitude tests and

achievement tests indicate that

a. high achievement is possible only for the high
aptitude students.

b. students with low aptitude can learn only the simplest
ideas.

c. both a and b d. neither a nor b

18h

Carrol.sees aptitude as the amount Of time required by the learner
to attain mastery of a learning task. Implicit in this formulation is
the assumption that, given enough time, all students can conceivably
attain mastery of a learning task. If Carroll is right, then learning
mastery is theoretically available to all, if we can find the means for

helping each student.

One type of support for this view is to be found in the grade
norms for many standardized achievement tests. These norms demonstrate
that selected criterion scores achieved by the top students at one
grade level are achieved by the majority of students at a later grade
level. Further support is available in studies where students can
learn at their own rate. These studies show that although most student
eventually reach mastery on.each learning task, some students achieve
mastery much sooner than dO other others (Glaser, 1968; Atkinson,

1967).

Can all students learn a subject equally well? That is, can all
students master a learning task at a high level of complexity? As we
study aptitude distributions in relation to student performance we have
become convinced that there are differences between the extreme students
and the remainder of the population. At the top Of the aptitude dis-
tribution (1 percent to 5 percent) there are likely to be some students
who have a special talent for the subject. Such students are able to
learn and to use the subject with greater fluency than other students.
The student with special aptitudes for music or foreign languages can

learn these subjects in ways not available to most other students.

185
Whether this is a matter of native endowment or the effect of previous

training is not clear, although this must vary from subject to subject.

We believe (as does Carroll) that aptitudes are predictive of rate
of learning rather than the level (or complexity) of learning that is
possible. Thus,‘we are expressing the view that, given sufficient time
(and appropriate types of help), 95 percent of students (the top 5
percent + the next 90 percent) can learn a subject up to a high level
of mastery. We are convinced that.the grade of A as an index of mastery
of a subject can, under appropriate conditions, be achieved by up to 95

percent of the students in a class.

It is assumed that it will take some students more effort, time
and help to achieve this level than it will other students. For some
students the effort and help required may make it prohibitive. Thus,
to learn high school algebra to a point of mastery may require several

years for some students but only a fraction of a year for other students.

 

 

7. The fact that, on standardized achievanent tests, the top scores
achieved by only a few students at one grade level are achieved by

the majority at a later grade level, supports the view that

a. aptitude is the amount of time required to attain mastery
of a learning task.

b. given enough time, the majority of students can achieve
mastery of a learning task.

c. both a and b d. neither a nor b

186

Whether mastery learning is worth this great effort for the
students who may take several years is highly questionable. One basic
problem for a mastery learning strategy is to find ways of reducing the
amount of time required for the slower students to a point where it is
no longer a prohibitively long and difficult task for these less able

students.

We do not believe that aptitude for particular learning tasks
is completely stable. There is evidence (Bloom, 1964; Hunt, 1961) that
the aptitude for particular learning tasks may be modified by appro-
priate environmental conditions or learning experiences in the school
and the home. The major task of educational programs concerned with
learning to learn and general education should be to produce positive
changes in the students' basic aptitudes. It is likely that these
aptitudes can be most markedly affected during the early years in the
home and during the elementary years of school. Undoubtedly, however,

some changes can take place at later points in the individual's career.

However, even if marked changes are not made in the individual's
aptitudes, it is highly probable that more effective learning conditions
can reduce the amount of time required to learn a subject to mastery
for all students and especially for the students with lower aptitudes.
It is this problem which must be directly attacked by strategies for

‘mastery learning.

187

8. Which of these statements is a mistaken assumption?

Aptitudes are predictive of rate of learning rather than
level of learning.

Aptitudes for particular learning tasks are stable and
cannot be modified by the environment or learning ex-
periences.

There are real differences in aptitudes for particular
learning tasks at the extremes of the student population.
Each individual student can vary considerably in his
aptitudes for different learning tasks and subject

areas 0

188
One may start with the assumption that individual students need

very different types and qualities of instruction to achieve

mastery. That is, the same content and objectives of instruction may
be learned by different students as the result of very different types
of instruction. Carroll (1963) defines the "quality of instruction in
terms of the degree to which the presentation, explanation, and ordering
of elements of the task to be learned approach the optimum for a given

learner."

Much research is needed to determine how individual differences
in learners can be related to variations in the quality of instruction.
There is evidence that some students can learn quite well through
independent learning efforts while others need highly structured teach-
ing-learning stiuations (Congreve, 1965). It seems reasonable to
expect that some students will need more concrete illustrations and
explanations than will others; some students may need more examples to
get an idea than do others; some students may need more approval and
reinforcement than others; and some students may even need to have
several repetitions of the explanation while others may be able to get

it the first time.

 

 

9. One of the more pressing requirements of instructional research

today is

a. to find the one method, material, and curriculum that is
best for all students.

b. to define the qualities and kinds of instruction needed
by various types of learners. .

c. both a and b d. neither a nor b

p 189
‘If the student has difficulty in understanding the teacher's

instruction and/or the instructional material, he is likely to have
great difficulty in learning the subject. The ability to understand
instruction may be defined as the ability of the learner to understand
the nature of the task he is to learn and the procedures he is to

follow in the learning of the task.

Here is a point at which the student's abilities interact with
the instructional materials and the instructor's abilities in teaching.
For the student in our highly verbal schools it is likely that this
ability to understand instruction is primarily determined by verbal
ability and reading comprehension. These two measures of language
ability are significantly related to achievement in the majority of
subjects and they are highly related (+ .50 to + .60) to grade point
averages at the high school or college level. What this suggests is
that verbal ability (independent of specific aptitudes for each sub-
ject) determines some general ability to learn from teachers and in-

structional materials.

 

 

10. Giving students a choice of alternative instructional methods and

materials

a. probably will have the greatest payoff in the ability of
students to understand instruction.

b. should help students to become more independent and to
overcome feelings of defeatism and passivity about learning.

c. both a and b d. neither a nor b

190

Carroll defines perseverance as the time the learner is willing
to spend in learning. If a student needs to spend a certain amount of
time to master a particular task, and he spends less than this amount
in active learning, he is not likely to learn the task to the level of
mastery. Carroll attempts to differentiate between spending time on

learning and the amount of time the student is actively engaged in

learning.

- In our own research we are finding that the demands for per-
serverance may be sharply reduced if students are provided with in-
structional resources most.appropriate for them. Frequent feedback
accompanied by Specific help in instruction and material as needed can
reduce the time (and perseverance) required. Improvement in the quality
of instruction (or explanations and illustrations) may reduce the

amount of perseverance necessary for a given learning task.

There seems to be little reason to make learning so difficult
that only a small proportion of the students can persevere to mastery.
Endurance and unusual perseverance may be appropriate for long-distance
running -- they are not great virtues in their own right. The emphasis

should be on learning, not on vague ideas of discipline and endurance.

 

 

ll. Perseverance at a particular learning task will probably decrease

a. 'with frequent positive feedback on progress.
b. with frequent reward for correct performance.
c. 'with evidence of coming success at the task.
d. with repeated failure to achieve mastery of the task.

191

OPERATING PROCEDURES

The operating procedures we have used are intended to provide
detailed feedback to teachers and students and to provide specific
supplementary instructional resources as needed.' These procedures are
devised to insure mastery of each learning unit in such a way as to
reduce the time required while directly affecting both quality of
instruction and the ability of the student to understand the instruc-

tiOn.

Formative Evaluation. One useful operating procedure is to
break a course or subject into smaller units of learning. Such a
learning unit may correspond to a chapter in a textbook, a well-defined
content portion of a course or a particular time unit of the course. We
have tended to think of units as involving a week or two of learning

activity.

Using some of the ideas of Gagne (1965) and Bloom (1956) we
have attempted to analyze each unit into a number of elements ranging
from specific terms or facts, more complex and abstract ideas such as
concepts and principles, and relatively complex processes such as
application of principles and analysis of complex theoretical statements.
We believe, as does Gagne (1965) that these elements form a hierarchy

of learning tasks.

-192
We have then attempted to construct brief diagnostic-progress

tests which can be used to determine whether or not the student has
mastered the unit and what, if anything, the student must still do to -
master it. We have borrowed the term "Formative Evaluation" from

Scriven (1967) to refer to these diagnostic-progress tests.

Frequent formative evaluation tests pace the learning of students
at the appropriate time. The appropriate use Of these tests helps to
insure that each set of learning tasks is thoroughly mastered before

subsequent learning tasks are started.

 

 

12. Formative evaluation tests

a. are essentially diagnostic, self-study guides and, thus,
a part of the learning process.

b. are usually given at least six weeks apart to provide
general review.

c. are essentially comprehensive achievement tests.

d. none of the above.

193

Each formative test is administered after the completion of
the apprOpriate learning unit. While the frequency of these progress
tests may vary throughout the course, it is likely that some portions
of.the course —- especially the early sections of the course - may
need more frequent formative tests than later portions. Where some of
the learning units are basic and prerequisite for other units of the
course, the tests should be frequent enough to insure thorough mastery

of such learning material.

For those students who have thoroughly mastered the unit, the
formative tests should reinforce the learning and assure the student
that his present mode of learning and approach to study is adequate.
Since he will have a number of such tests, the student who consistently
demonstrates mastery should be able to reduce his anxiety about his

course achievement.

For students who lack mastery of a particular unit, the formative
tests should reveal the particular points of difficulty - the specific
questions they answer incorrectly and the particular ideas, skills, and
processes they still need to work on. It is most helpful when the
diagnosis shows the elements in a learning hierarchy that the student
still needs to learn. We have found that students respond best to the
diagnostic results when they are referred to particular instructional
materials or processes intended to help them correct their difficulties.
The diagnosis should be accompanied by a very specific prescription if

the students are to do anything about it.

13.

199

Which statement is correct?

a.

Students respond best to the diagnostic results of a
formative test when they are given a general remedial
prescription rather than a specific prescription.

For students who fail to master a particular unit of
material, the formative test should reinforce the learning
and assure the student that his present mode of learning
and approach to study is adequate.

both a and b d. neither a nor b

195

These formative tests may also provide feedback for the teacher
since they can be used to identify particular points in the instruction
that are in need of modification. The formative evaluation tests also
can serve as a means of quality control in future cycles of the course.
The performance of the students on each test may be compared with the
norms for previous years to insure that students are doing as well or
better. Such comparisons can also be used to insure that changes in
instruction or materials are not producing more error and difficulty

than was true in a previous cycle of the course.

 

 

14. Which statement is incorrect? Formative evaluation tests

la. should provide detailed feedback to the teacher on the
particular points of instruction that need modification.

b. should be graded to maintain high student motivation and
to determine student capabilities.

O. can serve as a means of quality control in future cycles
of a course.

d. provide detailed diagnosis and prescription of what is

yet to be done before mastery is complete.

 

Feedback:
1.d 2.a. 3.d .4.c 5.b 6.d
7.c 8.b 9.b 10.c 11.d 12.a

18.d 14.b

196
Post Test on Mastery Learning.

Directions: Select the best answer.

1.

Students can be motivated to expend even further effort in correcting

their own errors on formative tests, and thus increase their achievement

scores, if

a. they are given specific suggestions and instructions on the
formative test itself as to what they need to do.

b. they meet in small groups once a week to help each other.

c. both a and b ' d. neither a nor b

If scores on achievement tests are 'normally' distributed and

highly correlated with aptitude tests, it can be claimed that

a. the educational efforts have failed.

b. there is a causal relationship between aptitude and achievement.

c. both a and b d. neither a nor b

Advanced technological societies should have an educational system

that

a. emphasizes prediction and selection of the talented few,
rather than development of talent of many.

b. increases to the optimum the proportion of students that can

c. both a and b d. neither a nor b

C.

d.

197

be successful, a strategy for "Mastery Learning" should

make frequent use of diagnostic and prescriptive procedures.
depend heavily on individual tutoring.

make no attanpt to decrease the amount of time needed by
students to reach mastery.

be primarily concerned with providing each student all the

time he needs to reach mastery.

Intrinsic motivation is usually highest when

a.

each student is to be judged in terms of his relative

position to and in competition with his classmates.

standards of master and excellence are set apart from
interstudent competition, followed by instruction that
enables the majority to come up to those standards.

both a and b d. neither a nor b

Evaluation procedures are very important to the concept of "Mastery

Learning" because

a.

both teachers and students need to know what constitutes
master and what the criteria are for achieving it.

both teachers and students must have a continuous check on

,progress toward mastery.

both a and b d. neither a nor b

198

7. By the term "masterylevel" we mean

8.

b.

C.

the learner has perfected a particular task.
the minimum accepted level of learner achievement.
the maximum measurable achievement level.

none of the above

typical masterybased instructional unit, the learner will

always work by himself and not in a group.

.always work in a group.

work.in a group or by himself, depending on how the unit is

organized.

typical mastery (or competency) based program, the learner
is given constant feedback in the form of quizzes or questions.

is given a test where the scores indicate his level of success.

10. In mastenrlearning the resolution of a learning problem by a student

usually requires one of the following measures:

a. 'more time for learning.

b.

difficult media or materials.

diagnosis to determine what missing prerequisties, knowledge,
or skills he must first acquire to master the objective.
consultation with his teacher.

all of the above.

 

Expected responses:

1. c

6. c:

2. c 3. b 4. a. 5. b
7. b 8. c 9. a. 10. e

199

This Manual calls for a Systematic development of modules.

 

What is meant by such terms as: system, systematic, systems
approach? The following module, adapted from National Special

Media InstitJtes provides the truth about systems.

200

Self Evaluation Quiz on Systems

 

1 . Give a deﬁnition of a systems approach:

 

 

 

2. Which of the following is included in a systems approach to teaching?

A.

techniques for precisely speciﬁring the purpose
of teaching

selecting among alternative teaching strategies
ways to make teaching more individualized

ways for determining changes which would make
teaching strategies more effective and efficient

3 . A systems approach is mainly concerned with:

learning efficiency
administrative efficiency
teaching efficiency

concepts of management

4. If the system is a car then the suprasystem is:

the driver
gas
transport

road

201

Self Evaluation Quiz on Systems

 

1 . Give a definition of a systems approach:

 

 

 

2. Which of the following is included in a systems approach to teaching?

A.

techniques for precisely specifying the purpose
of teaching

selecting among alternative teaching strategies
ways to make teaching more individualized

ways for determining changes which would make
teaching strategies more effective and efficient

3 . A systems approach is mainly concerned with:

A.

B.

C.

D.

learning efficiency
administrative efficiency
teaching efficiency

concepts of management

4. If the system is a car then the suprasystem is:

the driver
gas
transport

road

202

A system is considered to be Open when

A.

B.

C.

there is a great deal of interaction between the system
and its environment

the system is incomplete

people are free to leave whenever they like

Which is/are true for all systems

A.

B.

C.

D.

all systems are purposeful
all systems use the same approach
all systems operate according to rules

all systems precisely

When we say that a system is synergetic, it is

A.

B.

C.

another way of saying that the whole is greater
than the sum of its parts

a straight forward input, throughput, output phenomena

capable of fixing its own problems

The science of communications and control through feedback

is called

A.

B.

recycl ing
dynamics

cybernetics

latent cont rol

ﬂB

FEEDBACK

A possible definition of a systems approach is:
"a problem solving process that organizes decision
making systematically so that the relevant factors
in a given problem are considered."

All, A, B, c and D.

A,B,C.

201+

'me'l‘ruthAboutSysters (l)

Youwillseethatwehavewaystoteachyouaboutsystemswhich
have proven useful to educators responsible for instructimal develOp-
nent. misprogramnelesuseofprogratm'edinstructim' ' teclmiques' .
'Ihesereqm regularrespmsesfrunyou,andprovideyouwithmnediate' '
Immledgeoftheoorrectnessofyomrospmses. Ifyouhavenotpre-
viwslyleamedthrmghtleprogramedinstructimnethod,younaybe
inclinedtolookaleadtotieanswerspmvided,vﬁ.thoutwritingyomom
answersfirst. 'I‘heevidenoeindicatesthatbynekingymromrespmse
.ﬂlodmigatﬂepmgranm,ymmderstmdingmdretentim
will be significantly greater. Good luck!

(1) Basedonaprogramdeveloped forNationalSpecialMediaInstitutes,
1972.

For a wider mderstandjng cn Instructicnal Systems see B.H. Benathy,
Instructimal Systems.

205

E'systensapproadi'cbem'teadst,butm1y“systemapproadaes', ch.
Asymmbemrmuisrearemyomplecdefinitimsofsystemavaﬂable,
huntsinply,asystuappmadiisaproblen-solvingproosssuutorganizes
dodsim—mﬂdrgsystaaticauy,sothatmerelataanoftrerele\mtfactors
inagivaiproblen,atthetinewlmtheynsedtoberelated. milethe
msystamamroadimdslsappeartobediffermt,treyalldepeadmths
smbasicamoepts,rules.andtedmiques.

Marshavefomdasystensapproaditobeaveryuseﬁzltool,moe
tteyhaveounetomderstamdmdapplythefewbasicomoepts,nnes,and
Wmapartiwlarsystensmdel.

Despiteallofthebally—hoothereisnoglgmalofthes
a . Butallofthendohaveinomnmbasicc

 

 

r ,andtechnigms.
systemapproach
excepts,rules

Gobadcmdmderunemrsinpleexplanatimofmatasystersapproad:

206

Mays,nonntbermidirodcismm\edover,ochkumeltmedto,
meislﬂcslytofmdacriticofeduatimmﬂofteadung. Asinthecase
ofreligimmdpolitic,everyuefeelsfreetospeaklikeanecpertwhen
thembjectofteadiingmm. Nomereallydisaqreesabmttheinportanos
ofreligim,politiu.oreducatim,hxttiereisgreatdifferaioeacrosstla
lmdabwttre'goockms'ofdedsiaisnndsinthenmeofmyoftmsethree.
Irhbolievothatmthreesufferfranthesaieamditims: lemrsinthese
fieldsseldmmalyse,systenatically,matitisthatstnndbedone. The
mdnsmforaowtplishjngtmirﬁmygoals,moftmomfused,and
Wtordeddnghowmﬂtkenettndmﬂeedinmlislﬁngttegoal
areeitherigmmd,imdequatelymrriedmt,orhi&ieiinsmeme'shead.
Aslmgastlnseomditiauprevail,treappearmoeofdodngmthingis
probablyasaoaptableasaganﬁnsaoomplishmt. Asystensapproadi
Wtoreplaoeﬂefondore,mderpimungmstofqmteadﬁngpractioe.
ummmgumxmmmmmmm
abaxtleaming.

A hicludesasetoftedmiquesforpreciselyspedfying
ﬂagmposssofmneadiim,forselecdngmigaltemativeteadungstrate-
gies,mdfordstemixungd1angesthatneedtobemdetomkemrteadiing
strategies more effective and efficient.

 

 

System approach

207

mmdeffidaicytmeataismyedimtors,partlybeause
ﬂuy'remtsnmeaboutlnwtomkeuisirtsadungmefficieit,ardparuy
frunagmuimaxioemtlutamnnistrativeeffideicyuaytakepreosdeios
merstixlmtnoeds. 'melatteroouldmﬁdasshappen,ofowrse,ard
Wisofadninistrativeefficiecymedstobeantroﬂed,hxtm
mmomosnudwithwmﬁwefﬁdm. misiderthe
mmeroftinesthatateaderfindsitmwsaiynorepeatapartimlar
instructim,ortlumnberoftinesastudenthastopractioeaparticular
www.masofmmmdemmﬂe
firsttrial,orstillhadn'tleamedaftermytrials. Mofmthe
Mumgraduatewithoutbeingablemreadadeqmtelyortome
basicaritlneticskillsinhiseverydaylife. Orthinkaboutthe
yumgsterulndidmderstardoarlmmthadoogoumaghtiesareactivity
againmdagain,eitlerbecausemyofhisslouerfenwstuimtsdidn't
pidcwtiemderstanding,orbecmmtheteadnrwasi'tmreofanof
tlnotrertinestlnstiﬂertshadbeensubjectedtothesamactivities.
'mat'sti'nkindofinefficiencythatissovastefulofwrmaders'md
stuients'tine.

Asystelsapproadigrsatlyincroasostheefficiencyoft
andl . Gimthatasystetsapproadiisatoolfors
L .andthatitisasvalidforsolvinginstructimalproblemas
myotlnr.letmaenunemoftlndnracteristicsmsarytoasystem

w.

 

 

teaching, burning
solving probl-ts

208

Asystenispartofalargeremrimt,thatcmandmyeffectthe

 

 

 

desiredfmctiniingofthesystan. 'mislarger issuetines
calledtlnsiprasysten. Formle.ifthedieteticsdeparm:tuerette
systenbeingomsidersd,thenthesuprasysoenorthe wouldbetl'e
miversity.
swim
m

Byttnsmtdm,ifdieteticedumtimistlesystenmderansi¢hratim,
thaiths .ortheenvimment,wmldbeA.D.A. Oriftheclassroan
isthesystenveareloddngat,thmttnsupnsystenoaﬂdbeﬂiesdnol.
mtl'notherhand.iftlnsystenversthssdnol,tlmthe
oouldbethssdiooldistrict. 'naeazprasystenformrsdiooldistrict,

 

tocanymreaaipleastepfurtmr,muldbetlnStateDeparmentof
mum.

 

 

when
sways-ta, «swimsuit

'nnpointthatisinportanttorumnberisttmtthsmatimof
mmdepeidsmuemmarestiﬁying. 'niesaneomditim
laoldsforMorm. Ifthecarwasthesysteuvevere

 

 

Widthtlmmofitssubeystenmor ,wouldbes'teering,
mtor,trmsnissim,andheatingsubsysteis. 'memjorpartstlutnaheup
asystentten,arecalledeither or .

augments

209

mummmer,thessating,&viossforpresmtinginfonnatim,air
mﬂitiaiing,studsntrespmsede\dmare or 7 of
trephysicalsystencalledtlaclassroan.

 

 

www.murtsiordermtiuportmt)
ammbeﬂorﬂ.depmdingmtmmmtofinteractim
with,andexd1angeofinfomimandmergywithits' envinxna'rt. China.
duingmnyyursofits'histmy,didmtpemitforeignerstoeiterits
borders,noritscitimtoleave. 'lbthebgreeitsleadersmraable.all
micatdmandtxaﬁmcutoff. 'n'esecnxlitirnsmdecmiaac
system. mm,mtlnotlnrhand,importedandexportedgoodstomst
ofthemajornatims. Emmanuomstmtlytradimidsasasiell,whethsr
ardiitectin'al.philosophical,militarial,orart.istic. knewouldbsomsidered

an system

 

 

closed
0931

Aclassroanmayalsobslabeledanopenorclosedsysten. 'lheself-
Wdassromthatdepeidsprimadlymﬂawoeswithinaclassroan
totheexclusimofmstoftheresmrminthesysten'semdmmentor

.isalleda/an systen.

 

 

 

3W
closed

Ofaxxrse,aclassrounmaybehsignodtotakeadmtageofextemal
resources. Poreenple,classmvmidiuphasizsindividualizedinstructim,
waddhavenocallmalloftmresourusavaﬂable,inordertoprovideths
myldndsmdmnbsnofecperimreqinredtomyoutsudistransgies.
mmmhumd system.

210

systemteﬂtocimgeandadapttnnewcxnditiaisand
requirmxtsmrereadilytimm systers.

 

 

open

OPE!
closed

mfortmately,thslargemjorityofourclassroalsfallintothe
static, or systanoategory. Becameofthestrmgdefensss built
whymﬂusym.mrhtineaﬁeffortisreq\nredmgetwideasadopted.
m,'bybsingaaareofdiaracteristicsofopeimdclossdsysneis,
teadnrsarebetterequippedtoidaitifymatmtbemtodangestatic
systeminto . systers.

 

 

closed
open

Allsystetsarem,mrormtﬂeymethesaueappm. may.
mmimedtoinsuresaiedssiredoutomeorm. 'lheyarecxisidered
mfultodaedegrestreycaneffectivelymdefficieitlyprodnetrat
Miredoutame. mfortmanely,nanysyste:sdomtclearlydefinetheir
desiredoutomes. 'matis,they&:mtstatetheiroutomessothatthey
canbsueasm'edwithanyprecisim. macatimismtorimsforits
mimstatemtsofpinpose. Redeemaofnostindustrialsysters
dspadsmtteirlcmingpredselyiuwtomtteirprochctmmdme
effideicywithmiditheproductwasprodimd.

mmmreqlnresthestatamtofmsmbleobjectiva.
medicatimwecallthese objectives. Wemﬂdallagreethat
itismxiieaiertopreciselychﬂmthemtomesofanmwbilepro-
ductimsystentlnnitmuldbsformedimtimalsystemhmmtdmsn't
mitlessworthdoing.

 

 

211

Barysystsmhasbiult-inlimitatimsandoptims. Usually,thel.i.mit:ations
malledmtraints,“tboptimsarealledmtrollablevariables. A
saddisooverybyreseardersisﬂatmteadersmhavebempracdcingfor
wars,myhavemlyalimitedhwledgeofﬂnmstraintstleyareoperating
mder.andtheoptimsthatareavailabletothen. Innanycasos,ithasbeen
famdtlntwhatappearedtobealimitatimor. .wassatethingover
michtluteadiercnuldmrtmmtrol.

 

 

omstraint

Ifateaci'erhasmd'nios,hxtmteadiaparticulargrade,ortousea
partiaalartectbook.ortoteadiaspecifiednmberoftnms,umallof
tlnsethingsmﬂdbecalled .However,ifshewereableto
dmsethegradetlntshemﬂdteadi,dnte3¢bookthatsremndise,ue
graingsystandntsramideredbest,ﬂnmtlutaremldteadhﬂien
thosevnuldbemlled .

 

 

axistraints
aritrollable variables , @tims

Inaprofessiaiwiereresom'oesarehardtocrneby,acleariomledgeof
thesystem and ismoessaryiftlntsadieristotake
mdnnadvantagsofherresmroesformadmig.

 

 

Wt: (order not important)
controllable variables, options

1;chde 'isthatatleasttm
altenutivenemsforaoomplistungadesirsdaztome,mstbemalyzedmd
asparodinnenmofeffsctiveressandeffidaicy. Fewtoadxerslmowhowto
“nudism. litastclonottxr'Isidermxmaroleawroprziuitae.iiI

212

tinyouisideritatall. hammoftierespectiveoostsam
buiefitsoftwoormre forsolvingminstmctimalproblen
haveanaaedtlnseteadnrspncticingsudianparism.togreatlyextmd
daeffectoftheirneagerrsso‘m.

 

 

approach
alternatives
Manoarpls,therearemmermsmtmdsofinstructimofwhididrillp
fieldeqaeriane,hcmme,progrmdumtim,project.recitatim,mle
play,arxlse:dnar,arebutafew. appossthatateadierhadsareobjectives
mtmmm’uwummaemw-W (this
leamingprogranvmldbeanmle). Rutarethecxmstraintsandoptims
thatyouneednoomsiderindioosingbeueeithesem ? appose
mmiérthetimvariable. Ifteaderpreseitatimtimislnnted,but
indepeidmtsmdytiueforstudmtsismt.tlmitmuldappearﬂuttla
self-instructimalprogrmisthebost . If,inaever.the
timorcostofproendngordevelopingu'eself-instrmtimalprogzmis
excessive,itmaybemrodesirablemdioosethslecture . Ch
ﬂnbasisofﬂiiskiMOftrade-offbetweaicnstmdbaiefit,theteadieris
abletodaoosethe audiprovidosthemsteffectiveleamingat
tlnleastoostofherlimitsdreoouroes.

 

 

alternatives
altemative
alternative
alternative
Bymyouare-aarethata mistsoffmctimsthat
mtivelym'itrihmeﬂnirgmmmrdtmdesired 'ofthe
system. 'nisoperaoorofaninstructimalsystenitheteadierireqlurescxn-
mmmmmmmuﬂmmaﬁmuy

and aoomrplishingthspmposesofthsmm. 'menedmisns

213

sotiptoprovidothisinformatimareoanedfeedbadcuedunisns.

 

 

systan
$533355”
Dyprovidingtheoperatoroftlnsystenwithinfonmtimabaztmwwelltm
firictimsarsbsingcarriedmt,“ mdmtisnnakesitpossible
toadjusttl'nfmctiaisfornmdnneffectimand . The
mdmiqmsforaoquiringinfomﬁmmuaeffectivmessofasysmmtim.
udformldngtiatinfmtimavaihblemtlateadermmmedsit,are
keyfactorsinmkingmtmtateadiingbleamingsysteuinproves. msofthe
MWWW,NMMW?MBWW
test. Mtimamamltiuxhofotiars,ixnltﬂing:ntingscales,deddists,
m,m.¢mmW.mmm,m
mjoctivedevices.mtmm:tim,standardizedinstrmmts. Feedbadthas
beamesoinportanttrutaunlenewscimcehasgrmwamndit.“
M. mfomally,cybemetiesisoalledthescienoeofmmicatims
aradomtmlthruxyhfsedbadc. Usefuliuprovarentofinstructimal
canmlyoomntimmghomtinmdapplicatimofueerpiridsldatawipported

WWoraperimtaEimIprovidedby .

 

 

W

officim

alternatives

feedbadc.cybernetics
ﬁninprovamtorrevisimofourinstrmtimalaltemativesisa

Wmﬂkimforthesystemapproach. mstsystatsinprovethrough

miveapprcrdnatims. Fewinstructimalsystembeginasperfectsystem,

madmiftheydidtlndungesinomstraintsardoptimsovertimmﬂd

mﬂmimerfoct. 'lbomtimnllyinprovs,asystanwstberevisedising

tinepiricaldataprovidedby . Wiwmoadmtimm

211-I

aningclosermdclosertoamdelofsmﬂmtleamingeffectivemssand
efficiency,aftereachr .

 

 

feedback
revision

Probablymeofthsbostaidstorevisingminstrmtimalsysteu.isa
Mmdojectives,mﬂnds,udcita1aformlishimﬂeobjecdws.
Mimeadioftiainstrmtiaulobjectivesuntbsmtdndwlthﬂnm
unenforbewungﬂieobjective,udﬂntestitemﬁordetemimnglnvmll
theobjectivemsnet. This annigobjecitves,uetlnds,and
criteriaprovideﬂnteadierwiﬂihardinfomatimabwtmidimnpmeitsare
mtﬁmctiamig‘ properly,andwhichmesare. The - "

mtchczntentheteadieranlsttdentmnre,intlninstnactim.
tlnttteyarebeingmmoosssful.

 

 

hatch
objectives-uatinds-criteria

'niesystareapproadirequirestlattlecriterimitetswestitmslrelate
specificallytotlebemvioralobjective,ntierthmaspedficmtmdor
teadungmaterial. Tendersoftenrelatetheix impospartimilar
paginabook,ortomataparticularpersmndghthavesaid,asevidmoe
thatthssmﬂeithasmuiredtl'edesiredbehavior. mismans.thatif
teadersdnossanewormnadeqcatemterialtoteaditlnsmobjective.
tieymttheirewritetheircriterimitsm. Shioethisisadifficult
mdtine-omsumngtask,itusmnygoesmdme.orisdaemdequately:
ortheteaderstidcswithpoornetmdsandmterialsmtreydonot
ravetinetochangethe irate.

 

 

criterim, tests

'criterim,tests

215

ﬁnally,systemaremig. misisanothervayofsaying
thattlewholeisgmatertlmtlnsunofits'parts. threspecdjically,
tiersareeffectsreoultingfrautreinteractimoftleompmentsofa
systemmidiaremtpredictablefranmigﬂnindividmlommrts.
unaffectsmaybedetrimeitalorbeieficialintemoftmplamnd
outounssofthosystan. Wmoftmuademreoftlﬁsplmm:
tlnmxomnbilethatmystedazslyavelopsmmdesuedvibratimathigh
speeds,asuspaxaimbridgemidioollapsesbecmaseofreciprocating
hammicsannedbyastrmgcmssudnd,orthecrad<thatsuidenly
appearsinﬂumuiWIqmllofadam. Anuﬂrstarﬁingthatminstrucdmal
systeniss .servestoalertteadiersmotryoutnew
systus,thatstfectsmtpredictedmyoowr. 'lhe recast-lists,
midiomparsmpxtwithqxtputshmﬂdbedaigiedtopomtupsynergetic
effects. ‘

 

 

Istheroaprooedurefordesigninginstructimwhidifollwsa
systanaticapproadi? Yes,wimwegettotI-emdeladvoatedintins
volumywwillfindthatournndelisbasedmthesystelsapproadi.

mthenaantine,ifywlookatthenextpage (Figure 31youwill
findabasicsystmticmdelofinstructimaldesign.

musicrndelisalsopreoentodwithwoaltematestrategies
(BundC). StrategyBreprssmtsaltematelearningexperieioesifthe
designerwishestousetlme. 'nusispartimlarlyusefuliftheleamer
didmtpedommﬂMmi-lebasicmdule.

Strategyciisestraddngasawaytoprovimaltemateleanﬁng
experimwmrheleannrssrcwagreatdealofheoerogermity.

216

WereHaveAll'nIEijectivesGa-ie?

mandiaPmmidifollow,smeofthefavoritephrasesofthe
1970'saremissing. Weseldanrefertotermsszasbelmrioralobjectives
orperfonnanoeobjectives. Myearsofeaperienoewithbehavioralobject-
iveshasaxndnoedusthattheyareofteiinadeqmteformiversitylevel
edmatim.

Deiavioralobjectivosaraimdeqmtsastheydonotindioatemy
oertainbotavimisraquired.mtgoalﬂeywﬂladiieveorthespecific
levelofinternalizatimormderstandingthatisbeingrequized. 'meir
valmcanbehrtherqmstiaedbytheirrelianoemtheusmlmetine
dasemtimofbehmrioralrespmsesasaneasureofassessmt. Webelieve
thatouraimarabettsrachievedbyspecifyimleanummtmsh Our
ratimalefollmzthisdiagranwmmustatzpartofmrratimale.

L

 

 

-r————o- LIAM

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

LEVEL: OE LEVEL: or LEVELS or mono-
EEELmo moEmmmuo moron OEVELomew'r
amnmEmzanou coo—EVALUA'nou eon-wouocscuaswe
too—oaomsunou 5 Eon-svumws é too-EKILLEo MOVEMEm
ion-VALumo g C.WLYSIB ‘ «meson ABILITIES
N
zoo-ammo :3 loo-APPLICATION ' too-RERcErwAL AEILmEs
A g zmownEuEusaou g zoo-same FUNDAMENTAL
I! .‘L‘ MOVEMENT
5 IMOWLEDOI a
M'lCTWE coonmve monomoron
11 ‘ 11 1
i‘ mm %
g A PERcE “3‘
c : PRICE THAI. =
mnuocvaL

 

 

 

 

 

———'—"' LEARNING

 

3
"wow:

 

ul.‘

Figure 3.—Pelatiomhips among learning
and the three taxonomies .

 

*

Iaamixgmtoaresaretheabilitiesorcapabilitiesthatbeoatepart f
theloanuerafterinsmntim. o

217

nitsrnalizatimphaseofleamingisqzrpzimryfocusofthe
leanﬁngprooess. 'misiswl-erethedesiredlemingoutoamoocur
“metered. 'misphaseischaracterizedbythedmigesthattake
plaoswithinthelearnerandwhichbecnmpartofthelearner. These
dungesrepreserttheleam'smﬁtivmaffectivemdpsydumtor
“anatominothermrdmthecapabilitiestlattkelearnerhas
acquiredwithmferewetouderstmﬁmgs,feelingsormvelentactivities.

ﬂammabilitiaoraqﬁzedattrihmomrepresmtmlevels
ofleamingthathaveoomzredandarebasedupmthemmtofintsmal—
izatimoffeelings,uderstmdingormvmtmpabilitythathastalm
place. Perhapstlnbestwaytointerpretthisintemalizatimoflsaming
isthrughﬂetmumssbymoaniwsm,mm1,e_t_§_gil§64hmd
Harrow(lS72). Eaciiofthesetaamiesportraysseveralhierarchical
levelsofcapabilitiesthatcanbeaoquiredbylearnersthraaghthe
leanungproosss. .

PigmeIﬂdepictsthereJatimshipsmagleamJngandthethree

taxmanies. Itcaiibsassmedthattliemxtomesofleannngsl-ioind
resaltinthemaisitimofapabilitiesmidicsnbsplaoedinto
levels 2.00-5.000ftheaffective tummy, 1.00—6.00 ofthecxagnitive

mm2.00-6.00oftmpsydmtortmuuuy.
Itaixaﬂdbenoosdthattlnleamingmtomesaretl'emdsfor

«hidiallmablinginstrmtionalactivitiesareperfonnsd. Sinoethe
learningmitmsareeids,theymybeviewedashavingthemly
intrinsicvalmtobefomdintheinstnactimalprm 'moselearning
Wmmprinepmposesofeducatim. Allothercmpmentsofths
instructimalprogran,mdiasttee1ablingstrategiesandevalmtim
ms,ammrelymforpro¢m:ghanmmorformlmting
Madamtofleamimoutomes.

218

Gagn(l974)statesitthisway: "iliatislearnedissmetinng
newthatrminsapartofthelearner. Salemdcalltheoeabilities
mtIprefa'tospeakoftluuascapabilities....Itistheseapabilities
thatomstitutethewtoumsotleamixy' (p. 3). GEmlmd(1974)supports
thisoaweptthmghhisemﬁmsismmderstmidingasbeingtheobjective
ofleanung.ratherthanmbehavioralmtomasmidiarethermises
madeaftermderstxidingoocurs. astanssthatbehavioral‘m'are
airplysuplesoftintypeoofperfmioetlmrepmentmdentanding'
(and-6). Qannstremgnizsthatttnleamingisneverpurelyaffective,
ormgnitive,orpsydmtor,hrttiutﬁestudentmaydevelopinantlmee
unassinzltaneously. Mediatorwstdioose.lu~ever,whidioftleoe
threeareasisofprimaryomoematanypartimlartimmmfomiladng
WWmﬁ/orobjectivutobeadiieved.

agardlessofapersm'sacaeptmoeornm-aooeptmoeofthshier-
arduicalnatmeoftlntmumies.itappearssafetpsaythatthsleaming
m,&inedastmabilitiesorcapabilitiestmtbeomepartofthe
leaner,slnuldanstimtettemetmcieswhidialeanierslmldadiieve.
Mummmstsdmtorsvdllagreethatompetenciescanbeplaoedinto
meormreoftlathreedmnins. 'nms,hamingwtomesslnﬂdrepresent
tinettaimtofspedficleanungintentsorompetmcieshmdtrese
shaﬂdbeouretingoalsotinmtim.

1I}:Icytowritinggoodcxametnicystateientsistodetenninefirst
tlalevelofleamingmtmthatisasizedforaspedficskillor
mmtmdtlmtostateitintemofaspedficgoaltobe
attuned. Gngnesuggeststhatveusestandardizedverbs.
mfoumgmsmuofgoalssemasmlesfortmidentiﬂ-
catimofspedficleamingmtomesormupeneiciestobsaduieved:

l. 'Bnloarnerwillstabethebasicmrtritimalmedsofa

 

*

Wenbedefixiedas'possessim’ ofrequiredlcmledgeskills,
attitiﬂesarxiabilities".

219

persmingoodtealth,vdtlmtreferringtommxtside
aerosofinformatim. Sewillmthiswitlnrterror.

2. 'Ihsleamervdllgaierateadietforapersmingoodhsalth
midiomtainsallthebasicmitritimalneedsforthatpersm.
‘Snwﬂldothiswitlnxtreferrimtomwtsidsmof

. infometimandwillmksmmstakes.
mtlnfirsteample,tinlearnsrmlneedtostateinformtim. This
isanaxtmofleanﬂngdﬁdamurescapabilitiuofrecall. Inthe
seomdeample,tlelearmrwillneedmadiet. misisan
mtomeofleannngvd'eratluleamermedscapabilitiestosyntlmize
appliablerulestoarriveattheendprnduct.
mmmammmmmmm
thataleameristoadiievemxithebeluvionlmthataredmmas
indicatorsofmoosssincnlpetenqattainmt. Hormlmifagoalis
establisiedforthsadiimtofaompetmcyattmapplicatimlevel,
Iml3.00oftlncngnitived:nain,thenthebelnviorala1tomndnsena
mindicatorofsnoessfulampeteicyadiievmtsmndalsorequirsa
raspmseattheapplieatimlevel. Ammamsm
waﬂdlilmdserequinalearningwtoamorrespmseatlevelioom
WW.
Mammatimshipeadstsbeneentheleanungminall
timeooftredanainsaoonrdingtothehierardiicalmrportrayedintle
mgnitive,affectivsmdpsydmtortammia. Inotherwords.thers
mmummmWMum
mmﬂnrupmaorleamingoutoumthatisusedtoevalmte
minadiie‘dngthedssiredleanung.
Youmaybeomfusedbyourmeofseveraltemssmhaslemum
m,cmpeteiciesandleanmigintentstomtmsmthing. For
www.myaretlnsmthingmdwinbeusedintsrdungeablyto
totaketlnplaoeofdnmfmliar,butnmitedtem"belnvioralobjective"

220

m Weprefertlnwidsrmioeptofleamirgoutocnestobehavioral
orperfomnm objectives. Byspodfying. (l)
wacanindicatewhycertainbehaviorismnred. Iearningoutomes
willalsospecﬂytheiZ) ofmtemalizatimormdsrstanding

 

(3) bmtrelymtlnusualmstimobservatim
ofbeluvioralrespmsesasaneasureofassesmt.

l. lemingmtonnns

2. level

3. leamjngqxtouuss
mammrumindanainsofleannm,theyareths(ll

and danains. mminisoxgmizedinto

(2) . mthsoognitivedmnainthshighesthierard'maooording

toaloan,is(3) . teammgisneverptn'elyaffective,or

cognitive. smumydevelopinallthreeareasm .

l. wgnitive OIL-understanding; affective or feeling: W or
m.

2. hiarardzieo

3. evaluatim

4. sinultarmly

Inamimaztcrnssaretmabilitiesoru) thatbeome
partoftlnlearner. Ieamingaztomessl'nildomstitutetheompeteicies
midialoarmrstauldadiieve:aolearningmsand(21 are
closelyrelatsd. teamingmtomesrepresentspedficleamingintentsor
(3) .

221

“WWW (orompetenciss).ywnsedno

determimtheu) ofloamingmrtcmethatisnemsaryfora
spedficsldll. Abigalevelskillrequiresahighm) of
loarningoutcrne.

Fin-tinnyournetl'ndsofassesmtstmldbeatthesaie (3)

astheleamingmwhidiymspedﬁed.
1. level

222

§pecifytng Learning Outcomes

A learning outcome specifies the skills, attitudes and abilities
you want learners to demonstrate as a result of instruction. A learning
outcome describes the intended w of instruction, and NOT the process
of instruction itself.

Specifying learning outcomes is important, otherwise there is no
sourd basis for the selection or design of instructional materials, content
or methods. If you don't know w you are going you will not be able
to say 131v you are going to get there. Instructors simply exist in
confusion and disarray of dneir own making uriess they can specify what
they want their students to accomplish as a result of their instruction.

Another important reason for specifying learning outcomes has to do
with finding out whether instruction has accomplished what you intended it
to do. Tests are the hurdles on our learning track and are supposed to
tell instructors and students alike whether they have been successful.

But unless learning outcomes are cl early and firmly fixed in the minds of
both parties ,. tests are at best misleading; at worst, they are irrelevant,
unfair or uninformative.

Another advantage of clearly defined learning outcomes is that they
provide students with the means to organize their own efforts (Towle, 1975;
Merrill, 1973). With clearly specified learning outcomes students are better

able to decide the activities which on their part will help them get to where
it is imartant for them to go.

The next pages are a guide which will help you to Mlearntng outcomes.

223

A learning outcome specifies the skills, attitudes and abilities
you want learners to demonstrate as a result of instruction. A learn?
ing outcome describes the intended result of instruction, and NOT the
process of instruction itself.

Specifying learning outcomes is important, otherwise there is no
sound basis for the selection or design of instructional materials,
content or methods. If you don't know where you are going you will not
be able to say Egg Y0“ are going to get there. Instructors simply
exist in confusion and disarray of their own making unless they can
specify what they want their students to accomplish as a result of
their instruction.

Another important reason for specifying learning outcomes has to
do with finding out whether instruction has accomplished what you
intended it to do. Tests are the hurdles on our learning track and are
supposed to tell instructors and students alike whether they have been
successful. But unless learning outcomes are clearly and firmly fixed
in the minds of both parties, tests are at best misleading; at worst,
they are irrelevant, unfair or uninformative.

Another advantage of clearly defined learning outcomes is that
they provide students with the means to organize their own efforts
(Towle, 1975; Merrill, 1973). With clearly specified learning outcomes
students are better able to decide the activities which on their part
will help them get to where it is important for them to go.

The next pages provide a guide which will help you to specify

learning outcomes.

22h
Learning Outcomes

Question 1:
The purpose for college curriculum is to
A. Lead students to gain certain skill, attitudes and abilities.
B. Determine how to teach or what materials to make available.

C. A guide to the instructor, formulated by the curriculum committee.

 

 

Answer:
A. The purpose of curriculum is to state learning outcomes in the areas
of skills, attitudes and abilities. (Skills, attitudes and abilities

are often referred to by the one term: competencies.)

 

Question 2:

The first step in writing learning outcomes is to determine
A. What you will do during class.

B. What the student will be able to do at the end of the course.

 

 

Answer:
The correct answer is B, as the purpose of the course is to lead

students to specified capabilities (competencies).

225

Question 3:
Pick the best stated objective (learning outcome) from this list:
A. The course will cover dietetic decision making.
B. The student will make correct dietetic decisions.
C. The instructor will analyze for the class the process of

making correct dietetic decisions.

 

 

Answer:

The expected answer is B. Although A and C may be necessary to bring
the student to the ability to make correct decisions, learning outcomes
(objectives) should be stated in terms of what the student will be able

to do after completing the course.

_— ——_— I
r

 

 

 

 

Question 4:
Suppose students should know acceptable standards of clinical
behavior, which of these learning outcomes best states this
A. The course will include 12 rules on clinical behavior.
B. The students will recognize examples of acceptable and

unacceptable clinical behavior.

 

 

Answer:

B is the better statement.

226

Question 5:
The previous example dealt with knowing acceptable clinical
behavior. Usually, just knowing is not sufficient; we require the
student to demonstrate favorable attitudes. Which of these learning
outcomes best states this
A. The student will develop a positive attitude to acceptable

clinical behaviors.

B. The student will.choose to behave in ways acceptable in

clinical situations.

 

 

Answer:
B is the expected answer because by choosing to behave in a certain

way the student is demonstrating the desired learning outcome.

227

Learning outcomes should be specified in terms of that which is to
happen to students themselves. The examples given have dealt with
students' knowledge, understanding, and attitudes to be gained. Object-
ives may also be stated in terms of skills or abilities to be gained by
students. The first principle, always, is the objectives must state the
ways in which students will change. They cannot properly be written in
terms of what the instructor will do or what the course will cover.

The next matter to consider in specifying course outcomes is
knowledge of results. How will you know’what has happened to the
students? How will you know'what they have learned?

Let us take another example. Say our course dealt with the Bill
of Rights, understand the differences between democracy and authoritarianism,
and appreciate democratic processes. One can not simply assume that,
as a result of their attending the course, the students gained this
knowledge, understanding, and appreciation. The instructor must attempt
to gather evidence of learning achieved.

The second step in specifying learning outcomes, then, is determina-
tion of attainment--demonstration that the desired change has taken
place.

Perhaps the most common method of determining student change in a
college course is the written examination. If the instructor chooses
to use this method, which of these statements would best be incorporated
in his objective?

A. The student will show that he knows the Bill of Rights.

B. The student will give evidence of his knowledge of the Bill

of Rights.

C. The student will respond to test items on the Bill of Rights.

228

 

Answer:
The best answer is C. It specifies that performance will be measured

by test items.

A

 

 

It is desirable to specify also the Eng_of test to be admin-
istered, for there are many forms of written examinations, each of
which may demand separate_skills. Pick from this list the response
that best states the objective.
A. The student will demonstrate his knowledge of the Bill of
Rights by responding to test items.
B. Given 10 multiple-choice questions, asking rights guaranteed in
the Bill of Rights, the student will select the correct
responses.

C. The student will write an examination on the Bill of Rights.

 

 

Answer:

The expected response is B. Answers A and C also specify examination
performance. However they do not state the type of exam as the number
of items necessary for the student to show that he knows the provisions

of the Bill of Rights.

229

Moving to our second earlier example, how can we tell when the
student "understands the difference between democracy and authori-
tarianism?" The instructor might ask that an essay demonstrating that
understanding be written. Which of these statements would then best be
incorporated in the objective?

A. The student will write 500-1000 word essay comparing and

contrasting democracy and authoritarianism,

B. The student will wirte a 500-1000 work essay on democracy
~and authroitarianimm in which he includes the following
points:

a) definition of terms;

b) three examples of each type of behavior.

 

 

Answer:
The correct response is B. It is not enough to specify an essay in

which the student "compares and contrasts." Comparing and contrasting
are general terms that mean many things to many people. The objective
should include approximate length of the paper expected and directions
regarding points to be included. Note that precise specification of

expected student action also lends precision to the instructor's exams

ination items.

230 .

Another type of test might be specified--one in which the student
demonstrates his understanding of the differences between democracy and
authoritarianism and the instructor so spared the chore of reaching a
mass of papers. In such an exam, the student is asked to identify,
from given statements of political events, those indicating domocratic
and those suggesting authoritarian behavior. In which of these ways
would the object then be stated?

VA. Given 10 paragraphs descriptive of political events, the
student will differentiate among them by marking those which
typify democratic and those which indicate authoritarian
behavior.

B. On a multiple choice examination of 10 items dealing with
democratic and authoritarian behavior, the student will select

the correct answers.

 

 

Answer:
The better answer is A, for there is little ambiguity as to the nature of
the test and the type of thinking which the student must apply to the

problem.

23]

we can set an attitudinal objective--"The student will appreciate
the American democratic process." How do we determine the extent to
‘which the student appreciates democracy? How do you write a test to
measure an attitude?

Knowledge, understanding, ang.appreciation can all be assessed by
observing student actions or products of those actions. If we cannot
measure student achievement in classroom exams, perhaps our students'
appreciation of democracy can be determined by out-of class behavior.

Suppose the instructor decided that students' appreciation of
democracy could be assessed by determining whether or not the students
voted when they became eligible; he could then set out to gather evidence
of their independent actions. How would such an objective be stated?

A. Eligible student will vote.

B. Eligible students will vote in the next general election.

C. Eligible students will vote many times during their lifetimes.

 

 

Answer:

The correct response is B. Although a general aim may be for the student
to vote many times during his life, such a statement is itself vague. It
is relatively easy for an instructor to determine if his students voted on

one election. Objectives must be written so that their attainment can be

assessed.

232

These are, then, the main principles to be followed in specifying
learning outcomes: Objectives must be stated in terms of student
learning and they must be stated in such a way that the instructor may
determine whether or not.learning has occurred.

Two additional matters must be considered before the task of
specifying instructional outcomes is complete. The first concerns the
conditions under which the learning is to be demonstrated; the circumr
stances surrounding the student's performance. Our earlier example had
the student moving toward responsible citizenship through-knowledge of
the Bill of Rights; he was demonstrating this knowledge on an examination.
But under what conditions was this examination to be administered?
Could the student use reference works? was it a "take-home" exam?

Pick the statement which restates the objective better.

A. Given lO multiple-choice questions dealing with recall

of provisions of the Bill of Rights, the student will select
the correct responses.

B. In an open-book examination, the student will select the

correct responses on 10 multiple-choice questions dealing

‘with recall of provisions of the Bill of Rights.

 

 

Answer:
B is a better objective because it communicates more information.
Conditions of performance in B indicate that the student will be

allowed to use a reference work while answering the questions.

233

In our third example, the student was to indicate his appreciation
of the democratic process by voting.

Which of these statements, added to the objective, indicates
conditions under which the performance is to occur?

Eligible students will vote in the next general election:

A. If they are registered.

B. Voluntarily.

C. For the candidates of their choice.

 

 

Answer:
The best response is B. "Voluntarily" suggests that the student has been

stimulated to act on his own.

23%

In all cases, objectives must specify the conditions under which
the performance will occur. These might be instances of voluntary or
mandatory behavior; in-class or out-of-class action; written or verbal
performance; whether or not reference works will be permitted, and so
forth.

One more specification and the objective is complete: The demon-
stration of learning, the conditions of performance and, last, the
criterion or standard. Setting the criterion simply involves a de-
cision as to the degree of accuracy which the instructor considers
adequate for the achievement of the particular objective. The standard
might range from 100%, on.ebjectives which are themselves prerequisite
to later objectives, down to a much lower level for complex, higher-
order behaviors.

Keep the following in mind:

1) What learning is to be achieved by the student, and
how will the learning be demonstrated?

2) Under what conditions is the student to demonstrate
his gained ability?

3) What standard is to be considered adequate for

achievement of the objective?

235

Here is a learning outcome:

Recognizes format of medical records.

Question:

Does the learning outcome include provision for the student to

demonstrate a particular attitude or ability?

Are the conditions under which the student's performance is to

occur noted?

Is a specific criterion or standard of performance given?

 

 

Answer:

The first requirement of a well specified learning outcome is satisfied;
the student will demonstrate ability by recognizing the format of medical
record. But you will agree that this is a very low level of skill. There
is no evidence of conditions or criterion. Such a statement gives no

direction to your students.

236
Question:
Mark your response for this example:

Verbally, without references, the student will state with

A. 100%
B. 75%
C. 50%

accuracy the three essential components in writing specific learning

outcomes e

 

 

Answer:

The correct response is A. A learning outcome that does not indicate
the student's performance, the conditions, and the degree of accuracy,

is not at all a specific learning outcome.

Consider this example:

Given a list of 10 basic terms commonly used in renal physiology,

the student will, without references, select the correct definition

with
A. 100%
B. 75%
C. 50%

accuracy from a list provided.

 

 

Answer:

The expected answer is A. If terms are basic to understanding the

course, the student should know all of them before being allowed to

continue.

237

Other objectives may well require less than 1002 performance.
For example:

In an open book examination, the student will write an essay

of 500-1000 words in which he selects and explains 4 examples

of authoritarian behavior demonstrated by Franklin Delano

Roosevelt during his first term. Essay to include definition

of authoritariansim, description of events, and rational for

Roosevelt"s action.

For this objective, the instructor may be satisified that the
student understands authoritarian behavior if he can find and explain
three examples--thus, 75% accuracy would suffice.

Criteria to be included, then, depend on the nature of the task
and its position in the sequence of tasks required for completion of
the course or curriculum. Objectives requiring abilities prerequisite
to the successful fulfillment of later objectives would carry higher

criterion standards. Terminal tasks often demand less than 100% per-

formance.

238

Now for the terminal task for this program. Here are the three
criteria which must be applied to all specific instructional objectives:
Provision must be made for the student to demonstrate a particular

attitude or ability.

The conditions under which the student's performance is to occur

are to be noted.

A criterion or standard of performance must be given.

239

The student will understand and be able to use terms and concepts

basic to the study of dietetics.

The objective includes provision for the student to demonstrate a

 

particular attitude or ability.

The conditions under which the student's performance is to occur

are noted.

A specific criterion or standard of performance is given.

 

 

This is an example of a general unit goal--a lead-in to specific
objectives. How will the student show he understand terms? By de-
fining them on an exam? By using them properly in his written papers
or in class discussions? Under what conditions? How many terms? With
‘what degree of accuracy?

As stated, the objective meets none of the criterion, but you can
make it meet all of them.

Your final task for this program is to write a specific, measurable
learning outcome which stems from the general goal before you. On the
following page restate the goal as an objective that meets all three

criteria.

21:0

The student will understand and be able to use terms and concepts

basic to clinical nutrition.

Demonstrate what?

 

 

 

Under what.conditions?

 

 

What criterion?

 

 

 

2M

Write True or False after the following statements:

1.

The majority of university level instruction is aimed at
the students' cognative domain.

 

 

 

 

 

2. The affective domain deals with attitudes, values,
dispositions, motivation and the broad area of
aesthetic concerns for feeling, beauty and form .

3. Thinking processes operate without feeling
processes.

4. All cognative processes have an affective
component.

5. Affective behavior can be inferred from a person's
approach and avoidance behavior.

Feedback

1 . Most people would say that this is the case (True).

2. True

3. False

4. True

5. True (usually)

2h2

SpecifyingAffective Learning Outcomes

The majority of University level instruction is aimed at the
students' coggitive domain. This includes skills such as remembering,
thinking logical and rational concerns and algorithmic truths. The
emphasis is on academic excellence, subject matter mastery, and the
learning of someone else's information.

While cognitive development is, undoubtedly, an important learn-
ing outcome, it is only part of the intellectual growth process. A
seriously neglected area is the affective domain which deals with
attitudes, values, dispositions, motivation and the broad area of
aesthetic concerns for feeling, beauty and form.

We often forget that thinking processes cannot operate without
feeling processes (Krathwohl 1964). Nearly all cognitive behaviors
have an affective component. One involves the other, and they cannot

be separated.

 

Quiz :

Instructors usually address the (1) domain in their

 

teaching. Remembering, intellectual skills and rational concerns are
examples of processes in the (2) domain. However, attitudes
and values belong to the (3) domain. Aesthetic concerns are

a development of the (4) domain. Thinking processes (can/cannot)

operate without feeling processes. Nearly all affective components

 

have a(n) (5) component; the two (can/cannot) be separated.
Answers:

(1) cognitive (4) affective

(2) -cognitive (5) cannot, cognitive

:(3) affective (6) cannot

2H3

It is possible to obtain affective learning outcomes by the use
of mainly the cognitive domain; and viceversa. The better the learner
feels about a subject, the more likely she is to learn more about it.
Then, the more she knows about the subject the more she appreciates it
and values it. Effective human development for the fully functioning
creative individual is based on a combination of both domains, cog-
nitive and affective (Williams 1969).

Piaget writes, "There is a close parrallel between the development
of affectivity and that of the intellectual functions, since these are
two indissociable aspects of every action".

Even though educators have for a long time talked about motivating
the learner, the building of positive self-concepts, attitudes and
values, is hardly ever planned as part of the instruciton.

There are many reasons for the neglect; one of them is compar-
ative ignorance of how to plan for affective teaching, how to bring it
about and how to evaluate it. In this short program we will address
this area.

Krathwohl (1964) has provided a taxonomy of the effective domain,

with five levels of hierarchy. The taxonomy is presented here:

244

Here is the taxonomy in more detail:

1.0

2.0

3.0

4.0

5.0

Receiving (Attending). Sensitivity to the existence of certain

phenomena and stimuli.

1.1 Awareness. Learner is conscious of stimuli.

1.2 Willingness to receive. Involves suspended judgement.

1.3 Controlled or selected attention. Differentiation of
stimulus.

Responding. Active attention to stimuli, e.g. compliance and

commitment to rules and practices.

2.1 Acquiescence in responding.

2.2 Willingness to respond.

2.3 Satisfaction in response.

Valuing. Consistent belief and attitude of worth held about
a phenomenon.

3.1 Acceptance of a value.

3.2 Preference for a value.

3.3 Commitment.

Organization. Organizing, interrelating, and analyzing different
relevant values.

4.1 Conceptualizing of a value.

4.2 Organization of a value system.

Characterization by a value or value concept. Behavior is
guided by values.

5.1 Generalized set.

5.2 Characterization.

The major categories of the affective domain are summarized above.

As was the case with the cognitive domain, a continuum is implied by the

sequencing of the categories. The Major characteristics of the affective

 

 

245

continuum are as follows: (1) increasingly emotional quality of
responses; (2) responses become more automatic as one progresses up
the continuum; (3) increasing willingness to attend to a specified
stimulus; and (4) developing integration of a value pattern at the
upper levels of the continuum. The overall organizing principle which
theoretically accounts for the affective phenomena in the process of
learning and growth is referred to as internalization. This term
generally refers to "the inner growth that occurs as the individual
becomes aware of and then adopts the attitudes, principles, codes, and
sanctions that become a part of him in forming value judgenents and in
guiding his conduct". The term internalization has much in common

with a term frequently used by social psychologists - socialization.

246

It would be too time consuming to analyze our affective learning
outcomes and to determine the exact hierarchy into which it fits on
Krathwohl's Taxonomy.

Instead we have divided the taxonomy roughly in two. The lower
levels cover interest and motivation, the higher levels cover attitude
and value.

First, we will discuss the lower levels which deal with interest
and motivation. When a learner likes or desires something, or some
activity, she engages in behavior which enables her to obtain these
objectives or participate in the activity. Engaging in such seeking
behaviors as in behaviors which make a given activity last longer are
evidences of interest or motivation. Your learning outcome requires
an interest or motivational strategy if the learner is to demonstrate:
increased persistence in working at some task; or more frequent par-

ticipation in some task.

 

 

Interest and motivation are clearly in the (1) domain.
They are the starting levels of the (2) domain; so (3)
i and (4) m are the required foundation before we

 

can move on to attitudes and values. A learner who shows increased
persistence and more frequently volunteers participation is showing

more (5) i and (6) m .

 

 

(l) affective
(2) affective
(3) interest
(4) motivation
(5) interest

(6) motivation

247

Affective behavior can be inferred from a person's approach and
avoidance behaviors. While many factors can complicate such choices,
the following oversimplified description should assist in planning
affective strategies.

When a person likes or desires a particular thing or event, he
makes an effort to obtain or seek after the thing that attracts him.
This type of behavior is called approach. On the other hand, when a
person does not like or fears a particular thing or event, he tries to
prevent contact with it. This type of behavior is called avoidance.
When a person really wants to do something, we say he is motivated; in
other words, he demonstrates an approach behavior. If a person who is
not forced in some way, persists in working at a task or goes back to
a particular kind of event again and again, we say he is interested in
this event; or in other words, he demonstrates an approach behavior.
An affective learning outcome identifies particular approach behaviors
from which we can infer interest or motivation in particular objects
or events.

When we attempt to promote interest or motivation, the desired
learning outcome should not be designed to have all students equally
'motivated or adopt common.interests. Rather, they are to modify the
negative extreme of behavior, to promote interest in something where
there was not interest before, to motivate in socially acceptable ways
where exisiting motives are disruptive or socially unacceptable, and
to empathize with the acceptable motives and interests of other.

One way to provide interest and commitment is to set up practice
conditions so that participating in a particular reinforcing event

depends on the learner's accomplishment of a specified learning activity.

248

Motivation or interest is increased when a learner successfully
engages in a particular task. The most successful procedure yet
identified for insuring involvement consists of establishing a contract
with each student whereby a certain amount of involvement with the

task results in an opportunity for activity the student finds rewarding

(Blaine and Merrill 1971).

 

A person's approach and avoidance behavior is an indication of

that person's (l) m . A person interested in some event or
task is displaying (2) a behavior, an indication that the
person is (3) m . Approach is clearly the opposite of (4)
a . One technique of increasing motivation consists of
establishing a (5) c .

(l) motivation
(2) approach
(3) ‘motivation
(4) avoidance

(5) contract

249

Does your learning outcome specify a change in values or at-
titudes? Attitudes and values are very complex sets of behavior which
include cognitive behaviors, psychomotor behaviors, as well as emotions
and feelings. Part of acquiring an attitude or value involves knowing
the concepts involved. There are three levels at which a student can
express attitude or value behavior. Level one is verbal expression
wherein he accepts a given attitude or value; level two is reported
behavior where he tells what he would do in a given situation; level
three is what he actually does in a given situation. All three levels
are probably necessary to affirm a student's attitude or value. Since
the first two levels can readily be faked, it is crucial during evaluation
that the student be unaware of which situations are to be used to
evaluate attitudes when attitudes are being observed.

On the next page, there are a number of evaluation strategies for
attitude or value learning outcome. Perhaps you would like to take
this short quiz before you turn over the page?

The most reliable evaluation for attitude and value behavior is

through . Verbal reports are not reliable because the student

 

will quite likely provide you with the answer you . Reported

behavior can be made more reliable as a measure by meeting certain

conditions; can you suggest some of these conditions?

 

250

Verbal Report: It is possible, but difficult, to obtain valid
measures of given attitude or value by verbal report. A measure which
merely asks the student whether or not he subscribes to a given attitude
or value is the most unreliable of all measures for several reasons.
First, asking the question tells the student he is being asked his
position relative to a given value. If he wants to pleaSe the questioner,
he will answer according to what he thinks the questioner wants,
rather than according to his feelings. Mere verbal expression does
little to measure how a student would act were he required to take a
position with respect to the attitude or value under question.

Reported Bahavior: Asking the student to state how he would act
in described situations can be a more reliable measurement, if the
following conditions are met. First, the student is led to believe
that the instrument is for some purpose other than to measure his
attitudes or values. Second, the student makes his choice for specific
situations rather than for abstract statements. Fourth, the descriptions
are worded without clues about the desired or anticipated response.

Observed Bahavior: Observing a student's choices without his
knowledge is the most reliable evaluation procedure for attitude and
value behavior. The following suggestions may assist in establishing
such observation conditions.

First, identify those situations which, if chose, would provide
the student the possibility to indicate his position relative to the
value or attitude under question. Be sure that the situations identified

are likely to occur with some frequency.

251

Second, be certain that a given situation presents the student a
choice where.alternatives would indicate different positions relative
to the value.

Third, be sure the student has a free choice. If, during the
presentation, the situation was described.and the alternatives iden-
tified relative to the value or attitude, or if the student is aware
that she is being observed, then there is no assurance of free choice.
Clues hidden in the situation itself may give some indication of the
preferred choice and constrain the decision.

Criteria: A single choice is not adequate to infer adoption of a
given.attitude of value. Several choices in several different situations
assure a more reliable inference.

It is usually inappropriate to award grades or other sanctions
for one value position over another. The purpose of observations
should be to evaluate the effectiveness of the experiences employed,

rather than to give awards to the students for conforming.

 

252

Perhaps the most effective way to teach values or attitudes is through
group (1) a or (2) s . In any case, the learner
should be given a variety of (3) e , because attitude change is

a slow process. We advocate three experiences in particular, they are:

 

 

 

(4) g p ; (S) s and (6) g d
(1) activities (4) group participation
(2) simulations (5) simulation

(3) experiences (6) group discussion

253

If your learning outcome specifies attitude change, then there
are practical strategies to accomplish this. You could provide
opportunities for the learner to participate in group activities or
simulations where accomplishing a desirable group goal depends on his
adopting (at least temporarily) the value system or attitudes of the
group. Provide the necessary congnitive information concerning the
attitude or value by using the apprOpriate cognitive strategies.

Cognitive Presentation: When teaching values and attitudes,

there is usually a considerable amount of cognitive understanding

necessary before.a student can rationalize a given value or
attitude positon. Consequently, teaching values frequently in-
volves teaching cognitive information.

Variety 2: Experiences: Attitude change is usually a slow pro-

cess involving a variety of experiences with the new attitude or

value, which is not difficult to realize considering the old
attitude may have been held for some time and reinforced on
numerous occasions. It is important, therefore, that each stu-
dent have a variety of experiences, each designed to promote
adoption of the new attitude or value.

1§3222.Participation: Perhaps the most successful way to change

an attitude or value is to put the student in a situation where

he is required to adopt the value or attitude in order to accome
plish a crucial group goal. This is the "act as if" phenomenon.

All of us have observed that a group of people holding very

different views suddenly adopt a common code when faced with a

crisis.

Simulation: A student's attitudes or values can be significantly

affected in carefully designed role playing or simulation situations.

254

These situations can cause the learner to experience negative
outcomes as a result of holding to his currently held values, or
positive outcomes as a result of temporarily adopting the new
value or attitude.
Gropp Discussion: While not as effective as presentations, group
participation or simulation discussions which help a student
examine the consistencies and inconsistencies of currently held
values, may help him change his attitude. This experience is
much more effective when used in conjunction with some of the
previously described procedures. .
A way to operationalize attitude change is to present the student
a statement of the attitude(s) or value(s) of concern. Recreate
situations illustrating the value or attitude, but avoid the specific
situations to be used in assessing whether or not he has adopted the
attitude or value.
Avoid Prompting Specific Behavioral Expression: Attitudes and
values require both cognitive expression and behavioral expres-
sion. It is desirable and necessary to verbalize-the value for
the student. It is also desirable for him to see behavioral
expression of the value. However, he must not be told the specific
behavioral expression which will be used to assess his attitude
or value change, because to do so may prompt him to respond after
remembering how he ought to act rather than on the basis of how
he feels.
Illustrate Attitude ggugglgg: Carefully designed presentations
can.have a significant effect in changing attitudes and values.
The procedures which make such presentations effective are those

which cause the student to carefully examine his existing values

255

and realize they are inconsistent with other values which have
even more importance to him. The procedure required is as fol-
lows: Identify those values or attitudes about which the student
has strong positive or negative feelings. During the presenta-
tion, demonstrate that the new attitude or value will preserve
conditions associated with the old positively held attitude or
prevent conditions associated with the old negatively held at-
titude.

Different Presentations: Because the same values are not held by
all individuals to the same degree, a very effective presentation
for one person may have little or no effect on another person.

It may be necessary, therefore, to prepare several presentations,
each based on different previously held values. Students could
then be grouped according to their previous values and shown the
appropriate presentation. If the student experiences a presenta-
tion based on an attitude be strongly opposes, he may be ad-
versely affected. However, in most cases, it is acceptable to

show all presentations to all students.

256

We have looked at the important basic concepts of
individualized instruction, systems and mastery learning.
Now we can start wiih constructing our own module! To
do this we will progress through each of the seven

components (or steps) in turn.

 

257

Pretest Step 1

1. What is meant by "learning outcomes"?

 

 

2. Do you agree with this statement:

"Learning outcome indicates what the instructor will do during

instruction".

3. When determining if a topic is suitable for your module, what

are the main questions to ask?

 

 

 

 

 

Check your answers on the next page. If you had all the answers, you may wish to

skip step 1 (after quickly skimming through it).

258

Pretest Step 1

Answers

3.

Learning outcomes are the abilities or capabilities that become

part of the learner after instruction.

No. A learning outcome indicates what the student will be able to
do after studying the unit.

a. Is there an actual need for instruction in this area?

b. Will the content change drastically or remain reasonably constant?

c. Is the content of reasonable length--not too short or too long?

d. Do students find difficulties in these areas?

 

 

 

 

 

 

 

 

 

 

259

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Step 1 Step 2 Step 3 Step 4 Step 5 Step 8
Develop F ti
Specify Analyze criterion Plan lesson: Construct orma ve
learning learning tests motivation lesson evaluation 1
outcomes outcomes - entry level tectnlques - one to one
- pretest media -small
- ”bedded crow
- post test - field
- attitude evaluation
Step 1
Recycle
Step 1. Determine Learning Outcomes*

Learnipg,0utcomes:

1. You will be able to identify learning outcomes which meet the

criteria for initiating the design of an effective module.

 

 

 

 

 

2. You will be able to state learning outcomes for a module of your

own choosing.

required for initiating the develOpment of a module.

 

The learning outcome will meet the major criteria

*Learning outcomes are the abilities or capabilities that become part
of the learner after instruction.

260

The learning outcome is a statement which indicates what it is that
the student will be able to do after completing a lesson. It should
describe the major culminating or synthesizing behavior which results
from studying the unit. It should not be inferred that this is the only
behavior the learner will know or be able to perform as a result of
studying the module, but that he will at least have this capability.
Learning outcome indicates what the student will be able to do after
studying the unit, and not what teachers will do during the unit.

Since a module is only one component of a total curriculum, it is
clear that the learning outcome is only a description of the terminal
abilities expected of a student after completing that particular module.
The different abilities and capabilities you expect from a student will
be specified in various modules as the curriculum builds toward higher
levels of knowledge, skills with regard to that particular module. The
learning outcome for one module may well be a description of skills a
student must perform as the curriculum builds toward higher levels of
knowledge, skills and understanding. This building process can be
achieved through the integration of several modules specifying learning
outcomes to interrelate various modules one to another.

The first consideration in identifying learning outcomes is determining
whether there is an actual need for instruction in a particular area.
The importance of this factor is not to simply avoid re-inventing the
‘wheel, but there is also a psychological consideration. As a module
developer, you will be investing a great deal of your own time, energy,
and creative talents in your materials. The need for a particular
instructional package can be highly motivating to the designer, and when

the going gets rough in the design process, this motivation becomes

important.

261

A concern when identifying learning outcomes is directly related to
the content itself. Is it content which will not change drastically
every several years? Does it have a somewhat logical structure to
facilitate development of instruction? Is the content a reasonable
length to be incorporated into a module? One way of answering these
questions is to determine the usual length of time required to teach a
particular topic using traditional teaching methods. The rule of thumb
advice is that to progress through a module will take at least twice as
much time as to progress through a conventional lesson.

Another factor in determining learning outcomes is the identification
of areas where students have had difficulty learning in the past.
Experienced teachers can identify particular concepts, principles, and
problem solving tasks which almost always prove to be difficult for
students; these difficult topics may be ideally suited for a module.

There are two other factors which should be taken into consideration
after identifying the learning outcomes. The first is to examine the
objective to identify quite precisely the type of capabilities the
student will demonstrate to indicate achievement of the goal. You may
find, for example, that they are primarily writing information which
they have memorized from the instructional materials. On the other
hand, they may be solving a very complex problem based upon concepts and
principles which they learned in the module. The nature of what the
students will be doing is determined primarily by the verb in the learning
outcome statement. Examine the verb and ask yourself whether that is
really an important capability; does that capability merit the hours

required to develop instruction for students to perform the objective?

262

For example, the verb §£g£g_usually calls for just memory, or low level
cognitive capability; as in "state the procedure for cleaning a meat
slicer". You may find that while your instructional goal meets many of
the criteria listed above, it simply does not appear to be an important
capability for students to have. Don't throw out the goal -start again.
Re-examine it and determine whether through rewording; particulary the
verb, you have a more valuable learning outcome. Remember, College
teachers are often accused of teaching at the low information or memory
recall level.

In order to assure yourself that the learning outcome does reflect
a meaningful statement of what a student will be doing, ask a colleague
for a description of what the student will be doing to fulfill the
instructional goal. These responses should indicate to you the clarity
of your statement. You may find that you obtain a much more detailed
description than you anticipated. However, this is not a problem unless
behaviors are described which you have not anticipated or cannot accept.
On the other hand the verb generate usually calls for higher order
cognitive capability, such as in "generate a POMR note". If the latter
is the case, then it is important that you reward the instructional goal
to reflect exactly what it is that you want the student to be able to
do.

You will need to identify the desired learning outcomes. At the
time of writing, there did not exist a national set of competencies for
entry level dietitians from which goals could be formulated. However,
with American Dietetic Association guidelines a number of Universities
have been hard at work to identify competencies required of an entry-
1evel Dietitian. An excellent source is the list of Dietetic competencies

developed at Michigan State University.

263

Some may prefer to carry out their own needs assessment. Such a
study would identify not only the goals but also the present level of
attainment of these goals. The discrepancy between the present status
and the desired goal becomes an identified need.

Other ways of identifying goals are by expert opinion, obvious

needs of learners, job or task analysis, and interest surveys.

264

Practice
The first step in building a module of instruction is to select and
state the desired learning outcomes. Several criteria can be used
to help you select a topic suitable for you.
1. Below is a list of statements of consideration for selection of
a learning outcome. Identify those which are important consider-
ations for selection.
stable content area
time required for designing instruction
vs. the importance of students possessing
that knowledge or skill.
area in which students have difficulty
learning.
few materials available on the topic
though, instruction is considered
important.
content area is fairly logical.
2. Make a tentative topic selection for an instructional module
which you would be interested in writing. To determine whether
you have the topics clearly in mind, state the learning outcomes

on paper.

II.

265

Wow that you have identified the learning outcome which meets the
selection criteria, you need to state it as clearly as possible.
Below is a list of considerations for writing learning outcomes.

Select those which are important considerations for writing instruc-

_ tional goals.

1.

Learning outcomes are clearly specified
Capabilities required of the student are
obvious in the statement.

Behavior in the goal can be measured.
Capabilities in the goal can be measured to
determine whether students have reached the
goal.

Learning outcomes are stated clearly with
a topic, intended capabilities, and any
limitations stated which will be imposed

on the outcome or the topic.

Approximate instructional time required
for students to reach the goal.
Approximate writing time you can devote

to writing and revising instruction.

266

Feedback

I.

II.

1. If you answer yes to all of the previous criteria, you are correct.

Each of these criteria is an important consideration in developing

an instructional goal.

2. With the goal written on paper, refer back to question 1 in the

same section. Evaluate your topic using each criterion statement.
a. Does your goal meet each criteria?
b. If it does not meet some criteria, can it be revised to do so?
c. If it does not meet a particular criterion and cannot be
revised to do so, you may want to write another instructional
goal and try again.
You may need help determining whether your goal meets some of the
criteria for topic selection such as need or interest. You might
discuss these issues relative to your goal with colleagues and
target students. Revise and rewrite your instructional goals as

needed to meet the above criteria.

If you believe that all the considerations in Section II of Practice
are inmortant, you are correct. You may check the clarity of your
goal by asking colleagues and intended students to verbally inter-
pret the instructional goal you have written. Do they interpret the
goal and the required behavior exactly as you intended? You may

need to revise. If your goal is too big for the instructional time
available (30 minutes, 1 hour, etc.) you may want to divide the

goal into its logical major parts, reword each part as an instructional
goal and then select the part most suited to your needs and time

constraints as the instructional goal for your materials. If you

267

goal is too small for the amount of time you desire, consider the
skills the student will need to enter your module and the skills
the student will be ready to learn as a result of completing your
module. By considering skills related to your goal in this fashion,
you can identify the appropriate instruction to include in a module
for a specified period of time. Of course you will want to revise
your instructional goal to include more skills or information as

required.

268

WRITING LEARNING OUTCOMES

Here is a summary and a short exercise to help you with writing learning
outcomes. (we go into the specifying of learning outcomes in more detail
later.)
Instruction outcomes have three elements:

1. the performance required by the student

2. the conditions under which the student must perform

3. the minimum.acceptance level of performance
In other words:

1. Do what?

2. Under what conditions?

3. At what performance level?

Here is a learning outcome:

Given an example of a dairy food, the learner will rank and
compare the nutrieint parts of that food, without error.

In this example, the learner will:

1. Do what?

 

 

2. under what conditions?

 

 

3. At what performance level?

 

 

we will get on fine if you wrote: (1) rank and compare the nutrient parts
of the given food

(2) given an example of dairy food

(3) without error

269

Now, write a learning outcome of your own choosing:

 

 

 

Look at your learning outcome and see if it has these parts:

1. Do what (performance)

 

 

2. Under what conditions

 

 

3. At what performance level (criterion)

 

 

If you are having problem in writing objectives, then refer to a number of
excellent guides available including:

Preparingilnstructional Objectives by Robert Mager (Palo Alto: Fearon
Publishers Inc.) 1962.

Important

It was mentioned earlier that it is necessary to be particularly careful
'with the verb in our objective -- e.b. identifies the various parts of
meat such as fat, loin, muscle-because the verb (identifies) will determine
the level of complexity of any given objective. For example, Gagne's
Domains of Learning_lists a hierarchy of objectives. On a following page
is Gagne's Domains of Learning. Notice the hierarchy under "intellectual
skill" - there are 5 levels, starting with the lowest level which is
discrimination and moving to the highest level which is higher order rule.
Notice also that Gagne suggests a verb to identify each domain. Later we
‘will take a much closer look at hierarchies of learning.

 

 

270

WWW

Humanleanu'ngisacarplexmtter. 'Iherearesomanytheories,
experts,neﬂndsaminavideasﬂ1attheycmfuseratlerthanclarify. To
catplicatematters,ead11eamerisdifferentandleamsinwaysmique
toher.

Itisnowonderthenthatnostinstnxctorscanpayvexylittle
attentimtoaﬂtheseeidnrtatimsandreverttoteadﬁnginwaysvmidi
closely resemble the way they were taught.

This clearly is not a satisfactory situatim. There are new ways
tomakeourteacrﬁngrmchmre effective.

'misshortuodulehasbeendesignedtohelpyoumderstandscre

importantccncepts. ‘mesecmceptsarevitalifwearetoinprove
learning.

 

271

'Ihemajorityofrnmmlearm'ngfalls intothreedanains. 'Iheyarethe
cognitive, affective and psychcnotor dunains.
leamingoutcmesspecifyleaniingwhichfallintooneofthese
danains.
W:
Oognitive- to pramte abilities in thought and mderstanding.
Affective - to pramte changes in attitude, feeling or auction.

Psycharotor - to prumte inprovenent in physical ormanipulative skills.

For which types of learning outccnes do instructors typically teach?

 

You were correct if you said "cognitive".

 

'misistmeeventhoughwelmowthatamajorportimofedncaticnal
failure is actually due to negative attitudes. Still, that is another story.
Inthisnndtﬂewewilldealwiththecognitivedanain. Laterwewillsee
how cognitive and affective learning can be integrated. The cognitive
danajndealswith . Anothertennfornental
abilityisjntellectualskills.

 

mental ability

 

272

Itispossibletotalkaboutthelevelofocmpleﬂtyofmygivm
leamingmrtoate. Satetinesthesearereferredtoashierarchies. For
the cognitive danain Bloan has published a hierarchy of cognitive learning
outocmes.

Ieamingoutoanesoanbedetenninedastothejxc . Often
wespecifytheartpleidtyofaleamingoutomebystatingwhereitfalls
intheH . Ah indicatesthatoognitivelearning

 

 

wtoarescanbeorderedastotheirdifficultyorc .

 

 

omplexity
hierarchy
hierarchy

crrrplexity

 

Bloom's Tammany lists a hierarchy of cognitive learning outoanes.
HereisabriefsmmaryofBlocm's Taxmany; notethatknowledgeisthe
lowest level and evaluatim is the highest.

Knowledge - to recall and recognize

Oarprehensim - to translate fruu one form to another

Applicaticn - to apply or use infometion in a new situation

Analysis-toaaﬁnesarethingomplexandbreakitdominto
itsparts

Synthesis-tomttogetherinformatiminamiqmornovel
wayto solve aproblem

EValuaticn - to make a judgment about satething in light of

sarecriteria

273

HowcanImakeitclearthatmyleanxingoutomebelmgstoaparticular
levelofocnplexity? Sinple-bytheverb. Ifymirlearrﬁngoutocmeuses
ﬂeverbMorliigthenymarespecifyingforleanﬁngmmCmes
atthelmowledgelevel. mthenextpageisalistofpossibleverbsand
the levels of complexity which they represent. The verb judge indicates
leamingoutoatesatthee level,andsodoverbssuchasappraise
andevaluate. Atthe synthesislevelthefirstverblistedisc .

 

 

evaluation

 

Totranslate francme formtoancthermeans operating atthe c
level according to Blocm's hierarchy. C is a (low/high) level

cognitive process.

 

 

At what level do most instructors teach? At the k level.
Atthek levelthestudentisnerelyregtﬁredtorecallor
recognize.

It would seem that at University level students should reach the
lexrelbytheendof this course. Atthe level

 

 

students are asked to make judgrrents basedcn criteria.

 

ocnprehension knovﬂedge
oarprehension knowledge
low evaluation

evaluation

 

27h

encodes
oumsapmo
anonme
onooco
voodoo
onoon
oma>on
osao>
onensoo
open
opmsﬁo>m
cadences
omonn

 

=3 ”.3ng

.onanmee
amends
oeﬁcamno
a: non
bemoan
aochancoo
aooaﬁoo
oacsumam
mundane
oumaneuom
:mwmmo
omoeonn
moan
omoesoo

«usage.

oudnomoaeo
ocdsmxo
o>Hom
madame
oceanono
huopcm>cw
«emcee
pounced
summaeo

ouwoﬁaano.

amonucoo
opmneoo
anon
unusahonxm
ovmdcoamo
omwmnnem

ovmaecmnouuao
enhance

€35.83
nmeHoc<

coumxm
norm
cannonom
mumnoeo
maoeemnaaw
coupomne
mauumsmho
oumuonnosmo
on:

hedges
madam
.aoneuopcw

a. 3 pacing

Have
sow>on
apogee
oumooH

huﬁocoow
amonexo
e398
oewewoooh
ocﬁnomoo
omoomao
oumamon
mamﬂmcmea

nowmcoeonmeoo

oceanoocn
madame
meme
Haoooe
peed
onooon
unseen

25%
oncogene.

 

 

 

 

 

 

 

 

 

 

 

meOOHDO

 

 

 

EHaHzooﬂ oﬁsﬁm E mm: mom 85> managed exam

 

275

Aseducetorsitisiuportantthatmspedfyourleamingmtome
inmaoftheiranpleadty. Weneedtoaskquestimslﬂce: Is
themgnitiveleveloftheskillsufficigxt? Wutisﬂncorrectlevel?
Byteadaingatthislml,willthestuientbeabletoperfomhigher
leveltukslater?

ﬂuegmstimsbecmecrmialwrmwereoognizethatahighlevel
cogaitivetaskmmlybeadaievedifleamdnghasbeenmrefullybuilt
mtoit. mereseardlervmohasccntrihroedgreetlytomrmderstmding
ofleemhgtﬁerardﬁesmdseqmoesislbbertm.

Gegnehasavelmedaruenrdlyof'leamingapebﬂities'whidxis
inmmyvayssindlartobloan'sm. WepreferGagne'sworkbecame
radealswithleanungwtcomewhidimakeshismdtmreoperatimal.
miswillbeocnencreapparentaswemalmg.

mthemctpageisaagne'snaminsofmarMngCapabilitjes.
Partiaalerlynotehisfiveartegoriesof Intellectual (cognitive) skills,
czﬂminatingatthehighestorder,thatofwgitivestrategy. 'lbreach
N.Gagmhasdesigmtedjusttmanverbtoindicateeachlevel
ortypeofcspability. Notethatinfomatim.identifiedbytheverb
&,®esmtmqualifyasmintellecmalskﬂl. 'meloweetintel-
leconlskillisd mdtheverbusedtoidentifythislevel
ofinhellectmlactivityisd .

Adisaiminatimrequinsteningourobjectapartfrunanother.
Witisaverybesickindofintellectualsldll.itisassmedto
havebeenlearnedearlyinlife. beverttaremybeaplaceforthis
lmnlofleamingifasmdmtwrwtseeﬂ'edistrinctim(i.e.discrinumte)
Wehrigmmddarkbarﬂarmmdermicrosowmwhidasmuldbe
iantifiedasacellwall.

 

 

276

GAGNE: DOMAINS OF LEARNING (CAPABILITIES)

 

Capability Verb Example
Attitude Chooses Chooses to eat "junk foods" for,
main meals.
Motor Skill Executes Executes the task of cleaning a
meat slicer.
Information States States orally foods considered as
' dairy foods.
Intellectural skill
l. Discrimination Discriminates Discriminates between chalk and
cheese.
2. Concrete
Concept Identifies Identifies the various parts of
meet such as fat, lean, muscle.
3. Defined »
Concept Classifies Classifies, by using a definition,
the concept "calorie."
4. Rule Demonstrates Demonstrates, by verbal explanation,
the function of electrolyltes.
5. Higher order Rule Generates, by taking into account
(Problem Solving) Generates patient variables, an acceptable
‘ renal diet.
6. Cognitive Strategy Originates Originates a solution to "dumping

syndrom" by applying principles
of teaching and learning.

 

Based on R.M. Gagne and L.J. Briggs, Principles of Instructional Design,
Holt, Rinehart and Winston Inc., N.Y., 1974. p.85.

277

£3.

Ammerofomceptsmidiinteractismemytodescribeaiﬂg.
Ibseeiittnleannerurhrstarndsapartianarmlemaskrerto
damstratemdimrkrstaidim. 'medemsuatimoflcmledgem
beverbel.

'n'eactimverbassociacedwitharuleisd ..By
dsmstratingthainteractimofammberofcmpamtancepts,tre
sensitisprovingthatslnmderstandsar .

 

 

HigtarOrderRﬂe (ProblsnSolving)

'niehighestintelleconlskillvdaideagneidmtifiesistlegm
orderrulewhidihealsoreferestoasprdalensolvmg. Berewereoognize
thatmidaofmrproblansolvingisdmethrmghtincmplexonminatim
ofsinplerrules. ﬁncapabilityofproblansolvingis,naturany,amjor
aimoftheeduoetiaulproass-weshmldgivepricritytoteadungtre
lemur'mthimtclearly'. Mmthelearnergeneratesasolutimto
aprdalan,slahasgeneratedahigherorderrule. Cmeequtly,theverb
mummmammmmmdtyo:
ttnintellectmlprooessisattheproblensolvingleve. Gegm'stenn

 

forproblemsolvingish o r . 'meverbmich
spedfieethatleemingisathigherorderruleisg . 'nae
learnerg asolutimtotheprtblanbymingamnberof

duplex: rules.

278

 

higherordernﬂe

 

279

Cogrﬁtive Strategy

Aspcialkindofintellecwalskiu,whichisinadditimatypeof
prtiwlarinportmceinproblmsolﬁng,iscallsdamitivestrategy.
untemcognitivesu'ategyapplieetothenriousskillsthatareused
bytleleamertommageﬂnpmsesofleanﬁng,raradaeringmdtlﬁrﬂc—
ing. Accgnitivestrategyismintemallyorgmizedsldlltmtselects
mdguidestheintenulpromsesinvolvedindefinhugandsolvingml
prtblans. Inothermds,itisasld.llbyneansofwhichtheleamer
mmgesheroxmttunldngbehavior. 'meverbwhid'ispecifiesamgvg
mew-

Vhenastuierrtoriginatesancvelsomumsmisoperatingatthe
highestlevelofinteuectualskillvmiwisc s .
'meverbtoindioeteoperatimatmgnitivestrategyleveliso .
Cognitivestrategyappliee’cothsvariaasskillstIatareusedbytl'e
leanertomanagetheproceseesofl r andt
Ac 3 ismintemallyorgmizedskillthatselects
madguidestbaintemlprooeesesirmlvedindefiningmdsolvingmvel
prdolm. '

 

cognitive strategy
originates '
leaning, rmdoering. trunking

 

280

Wehaveloohedatarangeoflnmczpabilitiescalldintellectml
skills. mummyofstatingthisisthatwehavelodcsdatleamjng
hierardziesinthecngnitivedmain.

Gagnedoesmtmmgnizeinformatimasacognitivesldllasi

 

reliesmnatcry. Gagae'sintellectualsldnsbeginwithd which
isspedfiedbytheverbd . Suchdiscriminatimisbemeen
cbjectpmpertieS-
Wmapersmisaskedtoihrtifyvaricuspartsofthenephrmm
amal,theskillisatc c level.
Whmtheanoeptismtataomcremlevelwteretheleanxercan
pointtoittheyaremmxgtcmrdsd c . Berethelearner

isaskedtoclassifythecmoeprtbymingadefinitimorbygivingm
explanatim. Definedanceptsareremgnizedbytheverbc .

Amrberofdefinedccncsptsaiduptoar . 'meleamerstws
masteryofttnrulebyd smathing,possiblybyaverbal
explanatim.

Wmapersmsynthesizesanmberofnﬂesandsolveeapmblen
ttmshehasgaieratedah o r .

m-additianlcategoryofintemallyorgmizedsldllsisc__
s undigovemthelearner'sbehaviorinleamingandthinking,
madtmsdetennimitsqmlitymdefficieacy.

 

cmgnitive strategy

 

 

28]

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Step 1 Step 2 Step 3 Step 4 Step 5 Step 6

Develop

Specify , Analyze criterion Plan lesson: Construct Formative

.4 learning ‘_ learning “3:; motivation lesson evaluation _,

outcanee ““00"“. - entry level technique - one to one
- pretest media -emell
«-enauwea oHND
- post test - field
- attitude evaluation

Step 1
| Recycle l

 

 

 

L

Step 2. Analyze the Learning Outcmmes
LearningTOutcomes

1. You will be able to identify and describe procedural, hierarchical

and combination approaches to instructional analysis.

2. You will be able to describe the relationship among the tasks which

are identified through an instructional analysis.

3. You will be able to apply instructional analysis techniques to

identify subskills required to attain specified learning outcomes.

4. - You will be able to describe entry behaviors and distinguish them

from general characteristics of students in a target population.

282

It has been traditional for the content of instructional materials,
typically textbooks, to be defined by experts who have developed a
structure for knowledge which makes up a particular discipline. Experienced
teachers, using these textbooks, often have varied the instructional
approach, sequence, or content as they proceeded to teach students on a
somewhat trial and error basis.

In recent years, research has been conducted in an effort to
identify more effective procedures for identifying the precise skills
and knowledge which should be included in instructional materials for
students to efficiently and effectively achieve an instructional goal.

For example, rather than defining a course on renal diets in terms of

three lectures and 80 pages of text, there has been an effort to identify
precisely what it is that students will be able to do when they complete
their studies on renal diets. The next step is to identify the subordi-
nate skills which are required for the student to achieve the instruc-
tional goal. Several procedures have been developed for identifying

major subordinate skills. This chapter will present several such approaches.

The particular approaches chosen for an instructional analysis will
depend upon the kind of learning required in the learning outcome. It
should be stressed that while researchers do not claim that the instructional
analysis approach is the only way to identify content which should be
included in a set of instructional materials, their data suggest that
the use of these approaches results in the identification of skills
which efficiently lead to the achievement of the instructional goal.

Two somewhat different approaches to instructional analysis will be

decribed first: the procedural approach and the hierarchical approach.

283

A third technique which is, in fact, a combination of these two, will
also be described. The method to be used is completely dependent upon

the type of behavior described in the learning outcomes.

Procedural Analysis

The simplest of the three instructional analysis approaches is the
procedural approach. This approach is used when the behavior to be
taught is essentially a sequence of behaviors which must be performed
one after another in sequence to achieve the instructional goal.

To do this the authors of Learnipg_§ystems Design advocate
that you first write a task description, then flow diagram the tasks and
then identify the concepts, principles, rules, facts, and skills to be
taught. Here is the process in more detail.

Your Task Description will identify the steps needed to carry out
the task. This description will help you to decide what to teach and
how to teach it. A task description.will also insure that all the
essential materials are taught, and at the same time, that no unnecessary
information will be included in the course.

Your Flow Diagram is a means of representing fixed sequence tasks
in a schematic or digramatic form; it is an aid to visualizing the
structure of a task. A flow diagram is a convenient method for describing
and clarifying relationships between actions, cues and feedback which
might otherwise be obscured or overlooked.

In flow diagraming, the steps in a task are represented by a set of

symbols. The shape of the symbol used depends on the function being

28h

performed. The use of flow diagrams for describing fixed sequence tasks

is relatively new and symbols have not been standardized; however it is

possible to use the symbols commonly employed in computer programing,

which are shown here.

 

 

 

 

 

f’_\}

 

The action or operation function.
Generally-requires an action such as

walks, finds, sorts, etc.

The decision function. Often identifies
feedback or cues and branches to

alternative action sequences.

The beginning or end of a task.

Connector to another point within the

flow diagram.

The information function. Making
information available, recording

information, etc.

285

 

In order to clarify how flow diagrams of fixed sequence tasks
are written and used, it will be helpful to recall, once again, the
example of the pencil sharpener. An oversimplified view of how this

task is performed might look like this:

 

Break

Pencil

 

 

 

 

‘

Find

 

Sharpener

 

 

 

 

 

‘ Sharpen

 

 

Actually, even such a simple task as sharpeneing a pencil has far
more steps than the above description suggests. It can be a valuable
exercise to trace.~ an elementary task of this kind from the beginning to

end.

 

Task

10.

286

Here is an example of a task description:
Say you wish the learners to master the following goal:
The learner can clean an electric food slicer following

the procedures presented in the instructional manual and observe

safety procedures.

Description:

Begin - with a recently used food slicer and the necessary cleaning
materials (cloth, hot water and cleaning compound, and spatula).
Is the motor off?

If yes, then go on to next step (#3)

If not, then turn switch to "off" position.

Unplug food slicer.

Are knife guards in place and slice adjustment is at zero?

If yes, then go on to next step (#5)

If not, put knife guards in place and/or slice adjustment at zero.
Remove carriage tray assembly unit.

Remove excess food from the base of the meat slicer.

Wipe entire outside of food slicer.

Clean both sides of the guage plates.

Remove upper and lower knife guard.

9a. Open lid of grinder cavity.

9b. Open guard lock. -

9c. Life upper guard straight up.

9d. Rotate lower guard to the stop position and lift up.

Clean blade.

lOa. Open blade as wide as possible by moving the slicing

adjustment to the higher number.

11.

12.

13.

14.

15.

16.

17.

18.

19.

287

lOb. Cover spatula with damp cloth.
lOc. Wipe blade on both sides rotating blade with the cloth-
covered spatula toward you.
10d. Clean crevices where blade is attached to the unit with
the spatula.
lOe. Wipe back part of the base close to the blade with spatula.
10f. Is sanitizing of blade required?
If no, then go on to next step (#11)
If yes, then:
-put sanitizer in water
-dampen cloth and wrap it on the spatula
-wipe blade with the clothrcovered spatula
wash upper and lower knife guards and dry.
Replace knife guards.
12a. Replace lower guard and rotate it to the bottom.
12b. Replace upper guard.
12c. Lock the guards into place.
wash carriage tray assembly unit.
Attach carriage unit and lock into place.
Move slice adjustment to O.
Tip food slicer on its side and wipe the bottom.and feet with
a damp cloth.
Plug in the food slicer.
Empty the water and store cloth, pan, and spatula.

End - with clean food slicer.

288

 

 

 

 

 

 

j
I 8 NO TURN SWITCH
MOTOR TO OFF
OFF POSITION
YES
UNPLUG
SLICER

 

 

 

 

 

 

 

 

 

 

 

REMOVE
EXCESS FOOD
FROM BASE

 

 

 

WIPE
ENTIRE
SLICER

 

 

l

 

 

CLEAN
CAGE
PLATE

 

 

 

REMOVE KNIFE
GU ARDS

(See Sub)
T

 

 

 

SUBROUTINE I

 

OPEN LID
OF

GRINDER CAVITY]

 

I

 

 

OPEN
LOCK
GUARD

 

 

I

IFT UPPER j

GUARD STRAIGHT

UP

 

I

 

ROTATE COVER
GUARD AND
LIFT

 

 

 

 

 

CLEAN BLADE
(SEE SUB-
(ROUTINE 2)

 

 

 

 

 

NASH AND DRY
KNIFE GUARDS

 

 

 

 

 

REPLACE
KNIFE GUARDS
(SEE SUD-
(ROUTINE 3);]

 

 

 

 

 

NASH
CARRIAGE TRA!
ASSEMBLY UNIT

 

 

 

 

 

ATTACH
CARRIAGE UNIT
AND LOCK INTO

PLACE

 

 

 

 

MOVE
SLICE ADJUST-
MENT TO 0

 

 

 

 

 

TIP SLICER
ON SIDE AND
CLEAN BOTTOM
AND FEET

 

 

 

I

 

 

 

 

 

 

ELADIWITH
SPATULA

 

 

 

 

CREVICES WITH
SPAIULA

 

 

 

WIPE RACK
PART OP BASE

 

 

 

 

 

COVER GUARDS
AND ROTAIE
TO BOTTOM

 

 

 

 

 

 

 

 

 

 

 

 

 

ROUTINE

 

 

PUT
SANITIZER
IN WATER

 

 

I

 

 

DAMPEN
CLOTH AND
COVER SPATULN

 

 

 

 

WIPE BLADE

 

 

 

PLUG - IN
SLICER

 

 

 

 

 

EMPTY WATER
AND STORE
MATERIALS

 

 

 

 

290

29]

Task Analysis
£953.22

carriage tray assembly unit
base of food slicer
guage plate
upper knife guard
lower knife guard
grinder cavity
guard lock
sanitize
sanitizer
Principles
Food particles, if not removed from the food slicer, will
spoil and may cause food poisoning when the slicer is used

again.

Water must be above a certain temperature (120 ) if it is to

be useful in removing grease and sticky food particles.

By opening the blade as wide as possible (setting the slicing
adjustment on the higher number), more of it is exposed for

cleaning.

When the slice adjustment is a O, the knife (blade) is full
guarded.
Rules

The motor must be turned off and unplugged while cleaning.

Do not touch the slicer blade with hands at any time - use the

cloth-covered spatula to manipulate the blade.

292

Attitudeglgstructional Analysis

The analysis of how to proceed to teach an attitude requires a different
approach. Not only must instructional designers consider and analyze the
information which must be presented, but they must also consider issues such
as: modeling or dramatization, credibility of source of the information,
knowledge of consequences of behavior, the importance or criticalness of
the issue to the audience, and the tone with which information and instruction
are presented. The environment or what is happening around the presentation
of information sometimes seems to have more affect on a person's attitude
toward the information presented than does the information itself. For
this reason, not only must the information be analyzed, but the instructional
situation, the method of presentation, the presenters, and the target
audience must also be carefully analyzed.

Here is an exercise. Your goal is for teenagers to develop a positive
attitude towards eating balanced meals. Your first step is to do an
analysis. While doing this analysis you may want to ask yourself, "Does
a person need to know the names of the categories of the four basic foods
(step 1) to be able to state why milk is classified into the dairy category
(step 2)"? Yes. This is a simple check to determine whether the relation-
ship is hierarchical or just logically ordered. You may find that a
combination analysis is required for this task.

An analysis is on page 2-22.

After you have done your analysis, describe how you would create an
instructional setting which would account for each of the factors listed
below.

II. Modeling:

III. Credibility:

IV. Consequences:

293

V. Importance:

VI. Tone:

Feedback
II. Modeling:
A. There are several alternatives here. The model(s) should be
someone with which teenagers can identify. Some might include:
1. Other students their age or slightly older.
2. Students they would like to resemble.
a. A popular boy or girl
b. An athlete
c. Someone handsome or pretty
d. Someone happy and successful
3. If your audience is mixed boys and girls, have you included models
for each? They will probably not both identify with a model from

one sex or the other.

III. Credibility:
A. Here again there would be several choices.

1. would your source be expected to know about the subject?

2. Who would teenagers consider knowledgeable about proper
health and diet? Perhaps an athletic coach, school nurse,
health teacher, local popular athlete who looks healthy
(”Here's how I got this wayi"), local physician, or a
member of the county health program would be credible to
teenagers.

3. Who would they want to see concerned about them and their

health?

IV.

Consequences:

A.

29’:

This is where the use of appropriate models is very important.

Here you may want to use several examples and several models.

Did you consider positive as well as negative consequences?

1.
a.
b.
Importance:

Some positive consequences might include: becoming
pretty or handsome, having plenty of energy, being
productive, feeling good, being healthy, having
attractive skin, being popular, being accepted, and
being the right size.

Some negative consequences might include the loss of
a good complexion, loss of popularity, loss of
acceptance, loss of energy, not participating in
activities or achieving goals, loss of teeth, over-

weight, underweight, etc.

Do you have presentations planned that will convey the message

”This is very important to your life and your well being. It is

important now and it is important to your future." Messages that

convey current importance will probably have more impact than

future importance.

Tone:

A.

Some questions you might consider to evaluate your ideas are:

l.
2.

3.

Do your presenters have the proper attitude?

Do they convey a message for fear, "You had better do this

or else.” Do they appear to be kind, concerned, and helping?'

Actually a combination of these tones may be effective for

this group.

Are your models pleased with themselves when eating properly

and disappointed in themselves when not eating properly?

295

Facts
Location of On-Off switch.

" " carriage tray assembly unit.

" " carriage locking mechanism.
base of the food slicer.
gauge plate.
" " knife guard lock.
upper and lower knife guards.
" " blade.
" " slicing adjustment.
grinder cavity.
" " crevices where blade is attached to the unit.
Skills
Removing and replacing carriage tray assembly unit.
Removing and replacing knife guards.

Cleaning and rotating blade with a cloth-covered spatula.

Tipping the food slicer on its side.

The procedural or task analysis is basically a set of procedures which,
in this case, results in a clean food slicer. Each step is
independent, and the steps could be taught separately from one another.

The technique of task analysis or procedural analysis is most
effective when applied to motor skills which might be found in a vo-
cational area such as learning to operate or maintain various kinds of
equipment.

Cognitive skills which fall into this category are almost always

"rule following“ types of behavior.

296
Hierarchical Analysis

The second approach to conducting an instructional analysis is
through the application of hierarchical analysis techniques. This
technique is generally applied to what Gagne identifies as intellectual
skills. Intellectual skills are those capabilities which the individual
has and uses to perform the thinking tasks.

Research conducted by Gagne and others, indicate that higher level
learning such as the application of principles and problem solving takes
place as a result of utilizing lower level intellectual skills such as the
integration of concepts, the ability to discriminate, and the ability to.
make verbal associations. Therefbre, it is necessary to identify the type
or level of learning which is being required in an instructional goal and
to use hierarchical techniques to identify subordinate skills which must
be achieved prior to achieving the goal.

How does the designer go about identifying the critical subordinate
skills a student must learn in order to achieve a higher level intellectual
skill? The process suggested by Gagne is one of asking the question, "What
does the student have to know or be able to do, so that simply given instruction
he could perform this task?" By asking this question, the designer can
identify one or more critical subordinate skills which will be required of
the learner prior to attempting instruction on the final task. After these
subordinate skills have been identified, the designer then asks the same
question with regard to each of these skills, namely, given a particular
subordinate skill what does the learner have to know or be able to do so
that simple given instruction the task could be accomplished? This will
result in the identification of one or more additional subordinate tasks.

If this process is continued one reaches a basic level of performance.

297

An hierarchical analysis is not devised on the basis of one attempt at
the process, or even two or three. It takes a number of attempts at identifying
the vertical subordinate skills and their interrelationships before you are
satisfied that you have identified all the relevant skills and have stated
them appropriately.

Unlike the procedural approach, it is almost impossible to know when
an appropriate and valid hierarchical analysis of an instructional goal
has been achieved. Several suggestions will be presented.for testing the
hierarchy at this point in the instructional development process. While
difficult to validate, the hierarchical approach is the best possible process
for identifying skills which should be included in an instructional program
for intellectual skills.

On the next page, you will find an example of a hierarchical
analysis. The desired learning outcome is:

When given subtraction problems involving two columns,
whole digit numbers with borrowing, the learner will be
able to do all such problems without any external help.

The learner will achieve at least 902 correct answers.

5.

298

 

13.

Subtract two-column,

whole digit numbers
with borrowing.

If

 

Subtract two columns of
whole digit numbers
without borrowing.

 

 

 

 

 

9.
Subtract a oneédigit
number from a two-
digit number. I

 

 
 

I

 
  

Subtract one column
whole di it numbers.

 

 

State of concept of sub
tracting one number

from another.

4. IState concept
f subtractin

3. Value associatedl———

 

 

2. Recognition of]
numbers 0-9 .

1. [Concept of numbers.]

 

 

12.

11.

 

Subtract a one digit number
from a smaller one digit
number to which 10 has been
added

 

 

I

 

 

Subtract 10 from the
left column of a two-
digit number and add
it to the right column

 

 

Inumber.

 

I

A State concept

of borrowing.J

 

Md 10 to any
one-digit number.

 

 

Hierarchical Analysis of Subtraction Goal

299

The structure of this analysis can be considered to explain why it
is classified as hierarchical. The purpose of the numbers on the boxes in
the instructional analysis is to facilitate the discussion of the order of
the skills and eventually relating them to the objectives. The numbers on
the boxes do not imply a sequence. Refer to box number three. Could you
teach the value associated with numbers 1 through 9 if students did not
know how to recognize numbers zero through 9 (box number 2)? Likewise,
teaching the recognition of symbols 0—9 as numbers (box number 2) would
be almost impossible if students did not know the concept of numbers (box
number 1). It would be improbable that students could be taught to accurately
perform the skills in any one step without first having knowledge and skills
related to the preceding step.

An example of hierarchical and non-hierarchical relationships can be
illustrated using boxes 5, 8, and 9.

It would be difficult to teach the concept of borrowing (box 9) if
students did not first have knowledge of subtraction, because subtraction
is a process used in borrowing. Subtraction a one-digit number from a two-
digit number (box 7) is also logical before students learn to subtract two
column whole digit numbers without borrowing (box 8). Students should
possess skills through step 7 before the concept of borrowing (box 9) is
attempted. Step 8, subtracting two-column whole digit numbers without
borrowing, is not subordinate to teaching the concept of borrowing (box 9),
but it is subordinate to subtracting two-column whole digit numbers with
borrowing (box 13). The skills in blocks 1, 2, and 3 are all subordinate
to the required for addition (box 10). This is true because the concepts
required in blocks 1, 2, and 3 are subordinate to both addition and sub-

traction skills.

300

A hierarchical analysis of this type not only illustrates skills,
concepts, and information which needs to be taught, but it also provides a
logical sequence for instruction.

Combination of Procedural and Hierarchical Approaches

There is a third approach to the instructional analysis process which
is a combination of both the procedural and hierarchical approaches. This
combination process can be seen most clearly when applied to a complex
psychomotor skill or a relatively complex linear chain of cognitive tasks.
Let us take the psychomotor example first.

Assume that you were going to teach a person to parallel-park a car.
This is a task which requires several intellectual skills including various
judgements as well as a set of motor skills for actually moving the automobile
into several specific locations. This task is represented on the next page.

You can see that the process has been broken down into four basic
components. The instructional goal is to parallel-park a car in a particular
position on a street. Since positioning a car represents a sequence of
movements which must be executed in order, it would appear that the appropri-
ate analysis is the procedural approach. However, for each step in the pro-
cedure, there is an intellectual skill as well as a physical skill which
must be learned. Therefore, the hierarchical approach has been applied to
identify the subordinate skills associated with each of the major skills.
In this example, the learner would have to learn the intellectual skill of
positioning the front bumper of their car with regard to the car on the right.
He/she would also have to learn the motor skill of bringing the car into
that position. A similar type of analysis could be made for each of the

other steps.

30]

I;

am mcﬁ>o
maﬁa: .cunu

Eoum ucwamuum

masons wcﬁanH

 

cunu
Eouw use up
oeuamoe HmHHmumm

 

A.E.m .mcwmo Scum nmuamm<v
wnaxumm Hmaamumm How mwmhamc< HdtpﬁuoouumeH coaumcﬁcﬁou

   

 

 

 

 

 

 

 

 

 

mensweuaose

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

cusp Hwaamnmm _

 

on mammc3
wcacmucwamuum

 

 

cowuﬁmom

 

 

puma one up
macaw wew>oz

 

 

names up we«

 

 

Isaac no case
mascomouee<

 

 

 

 

manoeuoua ecu mcausumxm

F! wcwxunn HoHHmumm mo

UZHMM<m AMAA<M<m

"NMDQNUomm <

 

coauwmoa we“
unnum.cﬁ poo
wea:0HuHmo

 

 

coauumaﬁn
nmumcwﬁmmo m Amamcmv momﬁm
mcaxomc maac3 weﬁcﬁmunﬂm modem toauomuﬁ omumcmﬁmmc
ucmﬁu ecu cu umu .emmmm 30H HmEacHE unnumco ca maoﬁcm>
we ucouw mcﬁ>oz um wcﬁxomm up wcwxomm m up wcaxom wcchHuﬁmom
_ IL.
_ _

cunu conceded you women you nocuo

Home: Essence, mcﬁxomc no mo mamas cues some Eouw cocoa

How coauﬁmoa comma 30H newsman Icwwam oumsuom Imﬁc Hmumuma

mensweuaueH waHsMauamuH wasseauameH maeseaocoee schemaucoee

 

 

302
Conducting Hierarchical Analyses
Scrne Practical Suggestions

- . o .

1 . Remember, a hierarchical analysis is only done
for intellectual skills. Skills such as inform-
ation learning are not sequenced in a hierarchy.

2. Start with some 6" x 4" cards. Write your
learning outcome on a card. “"""’ "

3 . Ask the question: what must the learner need
to know to have this learning outcome? Write
this prerequisite knowledge on another card
and place it immediately under the first. This
prerequisite skill is usually referred to as a
M-

4. Repeat this process. Soon you will find that
there is "branching" where two types of pre-
requisite intellectual skills are required .

5 . Keep building up this "map". If you do not
get an answelc to your question "what must the
student know to be able to do this?" ask
"what can go wrong or confbse the learner
at this point?"

6 . The statement you write on the card should
have an action verb such as "identifies" or
"generates", but otherwise do not worry
about following a "learning outcome
format" (that comes later).

7. Repeat this process until you have reached
intellectual skills which you think all your
learners already have from other or
previous courses.

8. Draw a "map" on "organization" of your cards.
Number all your cards, starting with the

first prerequisite or subskill you identified
and finish with your learning outcome.

An example of a hierarchical map is on the next page,

followed by an explanation.

 

 

303

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Step 1 Step 2 Step 3 Step 4 Step 5 Step 6
iDeveLoo
Specify NY“ criterion Plan lesson: Construct Formative
learning learning tests motivation lesson f evaluation ,_ ,
ouunrnee ouuxxnes ~¢rurylevel hmﬂvﬂques -onetc<mws
-pretsst truths ~1rnsﬂl
-ennbedded 99°00
- post test - field
-aﬂhhxie ‘ ewaluetkri
Step 7
I Ikecyche
Step 3.

Develop Criterion Referenced Tests

Learning Outcomes

. l. You will be able to identify the purposes for entry tests, pretests,

embedded tests and post-tests.
2. Given a variety of learning outcomes, you will be able to write
appropriate criterionpreferenced test items which reflect the

learning required of the learner as stated in the learning outcomes.

Criterionrreferenced tests are designed to measure explicit learning
outcomes. This type of testing is important to: (a) test and evaluate
students' progress, and (b) to provide information about the effectiveness
of the materials to the instructional designer. The results of criterion!
referenced tests indicate to the designer exactly how well students were
able to achieve. Thus, criterionpreferenced testing is a critical feature
of almost every instructional design model.

The term "criterion" is used because test items serve to determine
the adequacy of a student's performance in meeting the specified learning
outcome: success on these items determines whether a student has satisfied
the requirements for that instructional unit. The items serve as criterion

for making that decision.

304

Criterion-referenced: When we compare a measurement not with other
measurements, but with some objective standard, we are making a criterion-
referenced evaluation. If we measure our automobiles and say "None of

them runs,"

we have compared the state of the car with the criterion of
movement and judged that none meet that criterion.

The difference between the normrreferenced and criterion-referenced
methods of evaluations was beautifully illustrated a few years ago by
an example created by Dr. James Whipple. "The Coffee Pot" example it
was called, and it went like this. Imagine that an objective said that
a student is to be able to make a pot of coffee. He has all the necessary
tools and equipment at his disposal, and his task is to be able to make
a pot of coffee. A checklist of each of the steps in the process is
prepared, and the students performance scored both on a norm-referenced and

a criterion-referenced basis. Look at the Checklist for Making a Pot of

Coffee and the scores. Note the difference between the two methods of

scoring.
CHECKLIST FOR MAKING A POT OF COFFEE
Norm-Referenced Criterion-Referenced
Scoring Scoring

Disconnects coffee pot 10 X
Disassembles coffee pct 10 X
Cleans components and pot 10 X
Inspects components 10 X
Fills pot with water 10 X
Reassembles components 10 X
Fills basket with coffee 0 e
Reconnects coffee pot 10 X
Sets dial on cofee pot 10 X
Reports pot is parking

properly 10 X

Score 90% 02

305

Note that in a norm-referenced system a student can accumulate a
score of 90 percent, even though he failed to make a pot of coffee. A
criterion-referenced system, on the other hand, gives him a score of
zero. Though he acquired most of the skills, he did not accomplish the
objective. The objective asked that he be able to make a pot of coffee,
and he failed to demonstrate achievement of that objective. Therefore,
he is not certified as having achieved the objective.

In regard to instruction, criterion-referenced evaluation refers to
comparing a student's performance with a desired standard and judging
‘whether the student did or did not meet or exceed the standard. For
example, suppose we expect a student to be able to spell 60 percent of
the words in booklet 3 correctly, and he spells 56 percent of them
correctly. When we say that he has spelled 56 percent correctly, we are
describing the results of a measurement. Now there are two kinds of
evaluation that can follow:

Norm-referenced: "Gee, he is better than anyone else in the class.

Give him an A."

Criterion-Referenced: "He did not meet the criterion of 60 percent.

Give him.more instruction.

Though there are some uses for norm-referenced evaluation, the
concern here will be with criterion-referenced evaluation. When we want
to know whether an expectation (objective) or criterion has in fact been
achieved, only criterion-referenced procedures are appropriate.

There are basically four types of tests which the instructional

designer may utilize in a model.
i

306

The first is an entry test. This is'a criterion-referenced
test which is designed to measure skills which the designer has
identified as being critical to beginning the instructional materials.

The second type of test is a pretest. A pretest is criterion-
referenced to objectives which the designer intends to teach in the
module. If you consider a hierarchical instructional analysis, an
entry test measures all the skills which appear below the "line"
while a pretest measures all the skills which appear above the
"line".

The third and most common test used by the instructional
designer is the post-test. This criterion-referenced test is
parallel to and sometimes identical to the pretest. Like the
pretest, it measures learning taught in the instructional program.

The fourth type of test is an embedded test. This is not
necessarily a single test, but rather represents clusters of criterion-
referenced test items which are interspersed throughout the module.
These items are intended for practice by students prior to taking
the post—test. Sometimes embedded tests are ignored by instructional
designers and not included as a part of the instruction. This is
more often than not a design error. Embedded test items provide
learners with the opportunity to interact with material presented,
to evaluate themselves, and to receive feedback. Embedded test
items also provide the practice, repetition, and reinforcement
needed to help learners remember materials. By providing students
with the opportunity to interact with ideas presented in materials,
the designer has changed the learner's role from a passive reader

to an interactive participator.

307

How does one go about designing and developing a criterion-
referenced test?

Start by carefully evaluating your first (lowest level) enabling
objective. What does the verb specify? What is the learner asked to
learn? Your test item.must assess if this learning did take place.

Ensure that your test item does measure the learning described in
the enabling objective. For example, if the enabling objective requires
the learner to show understanding of the two concepts "acute renal
failure" and "uremic syndrome" by classifying_each (Gagne's term for
testing whether the learner knows the defined concepts). The test items
must ask the learner to show (a) what is meant by the term "acute renal
failure" and (b) what is meant by the term "uremic syndrome".

Here is a possible test item:

In no more than half a page of writing differentiate between

"acute renal failure" and "uremic syndrome"

You should then determine the answer which will meet the criterion (e.g.
90%). Such an answer could be:
"Acute renal failure includes acute cessation of kidney
function with the suppression of urine formation. Uremic
syndrome is a term used to describe either the critical phase
of progressive renal failure after 902 of kidney function has
been lost or to the critical phase of acute
renal failure due to suddent kidney shutdown."
You should also decide what extra grades to award to learners who
demonstrate learning over and above the master level. For example, if
the learner also states that the patient with "uremic syndrome" requires

renal dialyses, another 22 could be added to the grade.

308

The important point is to carefully note the learning described as indé
icated by the verb of the enabling objective. If the verb is to match,
to list, to select, or to describe, then you must provide a test which
allows a student to match, to list, to select, or to describe. Learning
outcomes in the cognitive or intellectual domain generally require paper
and pencil assessment items or items which call for a specific product
or performance. Generally, it is relatively easy to determine achievement
of a cognitive objective; either the student "knows" the appropriate
response or he doesn't. .Assessment in the effective or attitude domain
is not quite as simple. Affective objectives are generally concerned
'with the student's attitudes or preferences. As there may not be a way
to directly measure a student's attitude (e.g., whether he values a good
nutritional meal against fad food), items for effective objectives
generally require either that the student state his preferences or that
the instructor observe the student's behavior.

One of the major issues associated with criterion-referenced testing
is how’many items should be written for an objective; or asked in another
way, how many items do students need to answer correctly to be judged
successful on a particular objective? If students answer one item
correctly, can you assume they have achieved the objective, or if they
miss a single item, are you sure they have not mastered the concept?
Perhaps if you gave the students two items per objective and they answered
both correctly or missed both, you would have more confidence in your
decision as to whether they have mastered the objective. You can see
this argument could lead to writing many, many test items per objective.

There is no correct answer as to how many items you need. What little

309'

'research has been done in this area generally indicates that the more
'narrowly stated an objective is, the fewer items are required to be
assured you will make the correct judgment about students' performance.
If an ojbective is very global or hazy, it will require many more items
to determine whether students have mastered it.

As you design the instructional strategy for your unit, you may
find it is very valuable to have a number of criterion-referenced test
items which are parallel to the objectives. These can be divided into
the pretest/post-test, and inserted into the instruction for students to
use as practice. These embedded items help students to understand the
level of criterion performance which is expected of them and to determine

if they have sufficient understanding and skills to successfully take

the post-test.

A Pause for Evaluation of Your Desigg_

It is important at this stage in materials development to evaluate
the design you have created. The materials you have at this stage are
the framework for many hours of future development and conceptualization.
By determining whether flaws exist in your design and correcting errors
that are found to exist, many hours of less-than-satisfying developmental
work may be saved.

Exactly what is to be evaluated? The materials that should be
evaluated are: (a) the learning outcome, (b) the instructional analysis,
(c) the performance objectives, (d) the criterion-referenced test items,
and (e) the adequate selection and description of the target population.

Who should evaluate your design? There are several options here,

and the nature of your materials and time you have available for the

310

evaluation will be major factors in selecting the evaluators. Some

persons you might want to consider are:

1.

Content experts and/or instructional desiggers who can validate
(a) the need for such instruction, (b) the importance of the
behavior stated in the instructional goal, (c) the accurateness
of the subskills that were identified in the instructional
analysis, (d) the accurateness of the sequencing of subskills
in the instructional analysis, and (e) the relationship among
subskills identified in the instructional analysis, (f) the
parallelism between subskills in the instructional analysis

and enabling objectives, (g) the clarity of performance desired
and criteria established for the behavioral objectives, (h)

the parallelism between enabling objectives and test items,

and (i) the equality of multiple test items constructed to

measure performance on the same objective.

Peers could be asked to evaluate whether: (a) there is a need

for instruction identified, (b) it is an area in which their
students often experience learning difficulties, (c) whether
such instruction would be feasible in their classroon (equipment,
space, etc.), (d) any subskills have been omitted from the
instructional analysis, and (e) there is any deviation from

the required parallelism among the instructional goal, instruc-
tional analysis, enabling objectives and test items.

Target students could react to whether: (a) they would find
learning the material interesting, (b) they perceive a need

for the ability identified in the instructional goal, and (c)

'31]

they experience any difficulty understanding vocabulary and

required procedures when explained. Given time, the designer
may also choose to administer a sample pretest to determine

how target students can perform on the tasks without previous
instruction. This may provide an early indication of whether
an appropriate level of students have been identified as the
target group. If they can already perform most of the skills

or,if they are totally baffled, they may be the wrong group.

312

How to Organize Your Desigg

On this page is how you should organize your design.

page is an example.

Subskills

Subskill l

 

Subskill 2

 

Subskill n

 

Learning Outcome

Enabling_0bjectives

Enabling Objective 1

EnabligggObjective 2

EnablingﬁObjective n

Test

On the opposite

Item(s)

 

Test

Test

Test

Test

Test

Test

Test

Test

Test

Test

Test

Test

item

item

item

item

item

item

item

item

item

item

item

item

(a)*
(b)
(c)
(d)

(d)
(b)
(d)
(d)

(a)
(b)
(c)
(d)

 

*(a) Test item(s)
(b) Test item(s)
(c) Test item(s)
(d) Test item(s)

for entry test
for pretest

for post-test

for embedded test

313

Quiz on Criterion Referenced Testing

 

Here are some statements about criterion referenced testing.

Mark the item T if you believe it is a correct statement and F if you
believe it is an incorrect statement.

1 . A criterion referenced test is composed of items which
measure learning outcomes.

2 . The word criterion is synonymous with the word mastery
when used in a criterion referenced test.

3. Test items for criterion referenced tests are developed
directly from enabling objectives identified in the
instructional analysis .

4. Entry tests are developed to measure skills students should
possess before entering instruction.

5. Embedded tests provide students with practice and feedback
on their performance as they work through the module.

6. Pretests are used prior to instruction to indicate a
smdent's prior knowledge about what is to be taught.

‘

If you have marked all of the above as true then we are in complete agreement.

31h

Pre—Test for Step 4

Part A: Answer True or False.

 

 

 

 

 

 

 

 

1. University level students are highly motivated when they come to class.
2. You should not waste time and therefore, you should start with formal
instructional activities from the word go. .
3. You should spend a few minutes at the start of the module with activities
which will help the student to concentrate. .
4. To be able to motivate your students, you need to know them well because
individuals differ as to what motivates them. .
5. Motivation, then, is getting the learner's attention in such a way that
the learner has a better attitude toward the instruction to follow.
Part B
6. For the motivational part of the lesson, you may choose from a number
of activities. Can you state three activities designed to motivate
students? I
(1.
2.
3.
7- Can a human interest story be used as motivation?
8.

Do you consider the following statement as sufficient motivation?

"At the end of this session, there will be a quiz."

10.

Part

11.

12.

13.

14.

15.

315

Do you agree that for some students a good motivation is to tell

them, in lay terms, the learning outcome of the module? yes/no

 

Do you agree that it is motivational for the learner to be reminded
about knowledge which they have already acquired which will relate

directly to what they are about to learn? ’yes/no

 

When considering the sequence in which you will present information,

the simple rule is to start with lower level skills. true/false

 

How can you determine which skills are higher level and which skills

are lower level?

 

 

 

Educational psychologists urge that instruction be provided in small
"chunks". Students are encouraged to make responses at the end of each
"chunk" to see if they have learned or not. What do you consider is an

appropriate size for a "chunk"?

 

 

 

Do you agree that all knowledge should be taught in the same way, be it

concept, fact or principle? yes/no

 

How would you present a rule?

 

 

 

Part D

16.

316

Giving students constant practice with feedback each time you have taught

a sub obective is:

 

 

 

 

 

a. too time consuming
b. necessary
c. not often enough
d. practice should be left to the end of the lesson
17. Feedback is not essential to learning. true/false
18. What kind of reinforcement can you provide for adult learners?
19. Can the tests that you use during the module be also used at the end of
the module (in the post-test)? yes/no
20. When the learner does not respond correctly to an embedded test, then
the learner is encouraged to:
a. go on and try harder
b. told that one little mistake is nothing to worry about
c. go back and review that section again
d. start the module all over again
Part E
21. State the difference between testing and evaluating.

 

 

 

22.

23.

317

Criterion levels must be determined:

a. before a learner starts working through the module.

b. after a number of learners have worked through the module.

The criterion level will clearly show if the learner has failed or passed;

no further decisions need to be made. true/false

 

 

318

Answers

1. False (usually).

2. False. You should start with a motivational activity.

3. True. Some students need to be motivated while others may need to be
reassured and their anxiety level lowered.

4. Generally true.

5. True.

6. Here are three that come to my mind:
1. A well-considered human interest story.
2. A well~developed, interesting presentation, such as a film.
3. Telling students the relevance and need for the information to be

presented.

7. Yes.

8. No - bad.

9. Yes.

10. Yes.

11. True.

12. Look at your instructional analysis (or do an instructional analysis).

13. Usually, your sub objective is a suitable "chunk". If the "chunk"
seems to be big (more than, say, six minutes) it may be that your sub
objective should really be broken into further parts.

14. No.

15. One method, suggested by Gagne, is to present a variety of examples of
that rule through verbal stimuli.

16. Both (b) and (c) are the expected answers.

17. False. Feedback i§_essential to learning.

18. Usually a statement commending their performance.

319

Answers, continued.

19.

20.

21.

22.

23.

Yes.

The expected answer is (c) unless the mistake occurred at the start

of the module, in that case, the answer is (d).

Testing means giving a person a score, such as: You got 10 items right.
Evaluation means using the test results, then considering the criterion
level and then making a decision such as: The student did not reach the
criterion level and therefore, must get more instruction by proceeding
to the alternative module.

The expected answer is (a). Criterion levels must be determined before
learners start working through the modules.

While partly True, most educators would prefer False as the answer.
There need to be decisions on the evaluation of the performance, e.g., this
learner can skip Part A of the next module because the score of 100%
demonstrates that the learner already can master that part of the next

module (which is mainly revision).

 

320

Stoo’ Stet, 2 Stan 3 sun ‘ Ste” 5 Stop 0

 

 

 

 

 

 
 
   

 
   
  
 
 

 

  

 

 

 

 

 

 

 

 

 

 

 

Davion
Sooci'y Analyze criterion Pi an lesson Construct I Formative
100'"'"9 '99'"'“9 tests motivation lesson evaluation
r? outcomes V . outcomes .entry techniques .one to one
level‘ mgdla .smeil
.pretest group
.embedde .fleld
.post test eveluatio
.attitude
Step 7
4' ~Recycle IL

Step 4. Plan Lesson
Learning;0utcomes
1. You will be able to identify and describe the major components of
an instructional strategy.
2. You will be able to develop an instructional strategy for a set of

objectives for a particular group of learners.

In this step we plan our lesson and develop an instructional
strategy which will elicit stated learning outcomes from students.
Stating an instructional strategy is not simply a description of the
content which will be presented to the learner; it indicates what you
‘will do before you present that information, what students will do

'with.that information, and how it will be tested.

321

There are four major components to an instructional strategy:

1. Motivational activities

2. Information presentation

3. Student participation

4. Testing

We will look at each one of these components in some detail, because

the instructional designer must plan an approach to each of these steps.

'Motivational Activity

Prior to beginning the formal instructional activities, there are a
number of activities which you should consider. The first of these is
the motivation level of the learners who will use your instructional
‘module. You may assume that students will be assigned this module or
that you will be dealing with highly motivated adults and therefore
effort is not required on your part to establish a high motivation
level. You may be correct, however, you may wish to use some type of
special techniques such as an attractive color scheme, a cartoon, a
'human interest story, or some other approach to gain the attention of
the learners and "hook" them into your module. In order to do this
'properly, it takes a great deal of knowledge about the learners and what
‘wiJJ.in.fact hook them and what will turn them off.

Part of the motivational process may include showing students what
they will be able to do when they have completed the module. For certain
.leaxners it may be sufficient to state the learning outcome in the same
form that you have in your instructional design. For others, you may

wish to reward the learning outcome to lay terms so that students will

322

better understand what the module is about. There may be situations in
which the learning outcome is so remote from the students' present level
of understanding that it would be distracting and discouraging to be
told about it. In this rare case, you might consider the wisdom of
describing that type of learning outcome.

Another source of motivation may be to remind the learners about
knowledge which they have already acquired which will relate directly to
what they are about to learn. This can serve not only to inform the
learners that this is apprOpriate material for them but also serve as a

bridge into the instructional materials.

PresentinggInformation

 

One of the questions you will need to answer is: What is the
sequence which I should follow in presenting information to the student?
The most useful tool in determining the answer to this question is your
instructional analysis. If you have done a hierarchial analysis, then
you would begin with the lower level skills, that is, those just above
the line which separates the entry behaviors from.those skills which are
to be taught, and then progress up through the hierarchy. At no point
would you present information on a particular skill prior to having done
so for all related subordinate skills.

The instructional sequence for a task which is a procedure would of
course, logically be sequenced from the left or beginning point and
proceed to the right or terminal objective. If there are subordinate

capabilities for any of the major steps in the procedure, they would be

323

taught these prior to going on to the next major component. As an
aside, some psychologists have argued that with a learning task which
essentially is a linear string of tasks, it is more efficient to begin
with the final task and to work backwards, e.g., start by testing the
lemon pie. However, for beginning design efforts, an orderly progression
from the first step to the final step is the recommended approach.

The next question in your instructional strategy deals with the

size of "chun "

of material which you will provide in your module. The
two extremes in this regard are the linear programmed instruction approach
which tends to break all the information down into very small units and
requires constant responding by the student. At the other end of the
continuum is the conventional textbook in which a chapter is usually the
unit of information. A good way to determine the size of the chunk is

to limit it to one enabling objective.

The next step is determining exactly what information, concepts,
rules and principles need to be presented to the student. This is the
basic explanation of what the unit is "all about". The primary error in
this step is presenting too much information, much of which is not
related to the learning outcome. It is important not only to define any
new concepts but to explain their interrelationships with other con-
cepts. You will also need to determine the types and numbers of examples
which you will provide with each of the concepts in your module. Many
research studies have investigated how we learn concepts and how we use
examples and non-examples to accomplish that task. we know that learning

is facilitated by the use of examples and non-examples. These should be

included as a part of your instructional strategy.

324

As concepts are learned differently to say, principles there is a
chart at the end of this step which will help you to decide the most

appropriate teaching strategy.

Student Participation

One of the most powerful effects in the whole learning process is
that of practice with feedback. The learning process is greatly enhanced
by providing the student with activities which are directly relevant to
the objectives. Students should be provided an opportunity to practice
what you want them to be able to do. Not only should they be able to
practice, but they should be provided some type of feedback or information
about their performance. Feedback is sometimes referred to as "knowledge
of results". That is, students are told whether their answer is right
or wrong, or are shown a copy of the right answer from which they must
infer whether their answer is correct. Feedback may be provided also in
the form of reinforcement. Reinforcement for adult learners is typically
in terms of statements commending their performance. Young children
often respond to forms of reinforcement such as "happy face" or even the
opportunity to do some other activity.

It is important when considering the student participation component
of your module to relate that participation to the learning outcome
‘which you have stated. You may wish to use embedded criterion-
referenced test items which are parallel to those which will be used to
assess the student's final performance. Through practice and feedback
with these embedded items, students can determine whether they understand

the material and can keep moving ahead in the module or whether they

should go back and review certain concepts.

325

Testing

Part of your instructional strategy will involve the testing or
evaluation of the learner. The student participation activities dis-
cussed above are a part of that testing procedure in the sense that
students are assessing for themselves their level of competence as they
go through the module. You should also determine as part of your instruc-
tional strategy how the data from the pretest and post-test will be used
with regard to student performance. For example, if you were teaching a
topic in which some of the students had prior knowledge of the content,
you could use that information to branch or direct students directly to
those parts of the module which they had already mastered. This would
save them time by not repeating material which they already knew.

You will also be required to make decisions about the module with
regard to differing levels of performance on the post-test for the
module. Recommendations should be made to the teacher and student as to
‘what should be done as a result of particular outcomes. For example, if
students do not reach some predetermined level of performance such as
802 on the post-test, what should they do? Will they simply be branched
back to repeat certain parts of the module or will other materials be
made available to them? If students do reach the predetermined level of
mastery, should they continue with the next module or will there be

other enrichment activities available to them?

l.

326

Pretest--Step 5.

When trying to determine the most appropriate media to use, what

considerations should you take into account?

When teaching psychomotor skills state some of the appropriate media you

could use.

For the teaching of learning outcomes where problem solving is involved,

what media or methods would you suggest?

Often the best medium cannot be used for various practical considerations.

State some of these considerations.

Check your answers with those on the next page. If there is a high level of

agreement you may wish to merely skim.Step 5 and go on to Step 6.

327

Feedback

a. The type of learning involdved in the objective.

b. Availability of media.

c. Ability to design materials.

d. Cost effectiveness of materials.

e. Practical considerations such as flexibility, durability and
convenience of materials.

Moving pictures such as film or video; demonstration of the skill itself.

Simulation or role playing.

Availability of medium; cost; complexity of the medium; lack of expertise

to operate the medium.

328

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Steo l step 2 step 9 Step I Stew 6 Step e
-Devloo
Soecll’y Analyze criterion hien lesson‘ Construct Formative
ieernlng learning [9,... motlvntlon leeeon evaluation
4 outcomes ‘ outcomes . entry techniques .one to one
level_ media .emell
.oreteet group
.embedde .ileld
.ooet teet eveluetio
. attitude
Stew 7
AJI Recycle IL
Step 5.

Construct and Design Lesson

Learning.0utcomes

1. Given an instructional strategy, you will be able to describe
the procedures for develOping instructional materials.
2. You will be able to develop your own instructional materials

based on a given instructional strategy.

An important step, which is often overlooked in instructional
design efforts, is to determine if there are existing materials which
would fit your objectives. In some situations you will find there is a

prolifera of materials available, all of which are either superficial,

 

329

greatly detailed or not directed to the target population with which you
are interested. On the other hand, occasionally it is possible to
identify materials which will serve at least part of your needs in terms
of your overall package. When you consider the cost of developing a
video tape, or a slide tape, it is clearly worth the effort to spend
several hours examining existing materials to determine if they meet
your needs.

If no appropriate materials exist, you are in the instructional
design business, and you must make an additional decision with regard to
your instructional strategy. ‘What type of media will be employed for
those materials which you must develop? There are five important considerations
with regard to the selection of this medium.

Start by considering the type of learning involved in your objectives.
If, for example, you are teaching a psychomotor activity of some type,
it is important to use pictorial or moving visuals in order to demonstrate
criterion performance to students. If you are teaching students to make
auditory discriminations, then clearly some type of audio medium should
be employed as a part of your instruction. If the task is one of shaping
fine discriminations among concepts, then perhaps you wish to use a
medium which can supply feedback to students on their performance.

A second important consideration in media selection is the projected
availability g£_various media in the environment in which the instructional
package will be used. If the materials will be used in a learning
tresource center of a public school, community college, or university,
.then a whole array of media devices would probably be available. However,
if the package is designed for home study or use in a community center
where mediated equipment is not likely to be available, then you must
either develop a means of making that equipment available or limit

yourself to paper and pencil types of instructional materials. A related

330

to view again and again as needed. This would free the lecturer or specialist
to work with students in a discussion/problem solving atmosphere.

It would be difficult to illustrate the total process for developing a
set of instructional materials. The decisions you need to make will be based
on.the subject matter, the learning outcomes you have specified, your own
knowledge, and skills, your financial and facilities constraints, the time
required to develop and test the instructional package, your target population,
and the many, many other factors which must be dealt with as they arise.
Instead of trying to construct one example.that would illustrate all these
issues, perhaps a suggested step by step procedure may be more helpful. This
procedure is not being promoted as the only way to develop an instructional
package. In fact, you may find that you need to change the order of events
here and there as you go along. The procedure is provided here for consideration
as one way to order the events in the construction of an instructional package.

You have already completed many of these steps.

33'

Suggested Procedure for Developing ggzlnstructional Package

Step Activitl

1.

6.

7.

Select topic.

State learning outcome.

Perform an instructional analysis of the specified learning outcome.

Describe the target population as specifically as possible and identify

entry requirements.

write subskills and enabling objectives.

a.

C.

Develop criterion-referenced tests for the objectives

Determine whether your testing purpose will be to diagnose entry

requirement, to branch students, to evaluate students, to

. evaluate materials, or all of these.

Determine whether a test over expected entry requirements is necessary
or advisable.

Divide parallel questions written for each objective such that

some can be included on a pretest (if a pretest is advisable), some
can be used for embedded questions for practice and feedback during

instruction, and some can be included on the posttest.

Develop the instructional strategy.

Review the subskills included in the instructional analysis and
determine which should be included in instructional materials

and which should be included as expected entry behaviors.

Consider the target population relative to their attention span,

work and study habits, responsibility, and motivation. Then estimate
an average "best" time for each instructional activity for the group.
Consider the enabling objectives in the instructional analysis relative
to the type of learning and learning activity required for

each objective, and the logical sequence in which you believe objectives

should be presented.

10.

12.

332

d. Cluster objectives. Each cluster might have one or more objectives
depending on the difficulty of explanation or time required for
practice feedback. There are no rules for clustering, and you have
only the instructional analysis as a guide to develop the first set of
materials.

Survey the literature and subject matter experts to determine what

instructional materials are already available.

Consider how available materials might be adapted for use using a.manual

to guide students throught the materials.

.Determine whether new materials need to be designed. If yes, proceed to

11. If no, begin organizing and adapting available materials.

Given each cluster or set of instructional activities, make decisions about
relevant motivational materials, necessary review, presentation procedures,
student participation, practice, feedback, evaluation, and followbthrough
activities for each cluster or instructional activity.

For each cluster or instructional activity, consider the best medium

to present the materials, to monitor practice and feedback, to evaluate,

-and to pass student to the next instructional activity whether enrichment,

remediation, or the next activity in the sequence.

a. . Make decisions about the ideal media for the type of learning (stories
for attitude, sound tape for language pronunciation, printed materials
for spelling and punctuation, relief maps for contouring or geography,
etc.) '

b. The types of media required for a large set of materials may be
many and varied. Cost effectiveness of developing materials using
varied equipment and subskills should be considered.

c. Availability of materials, portability of materials, availability

and appropriateness of equipment should be considered.

14.

15.

16.

17.

333

d. Tentative decisions about the best medium or media combinations
should be made based on studying types of learning, activities
required, physical and technical constraints, finances and facilities
are but two possible problems.

Determine the format and presentation procedures for each objective or

cluster of objectives. Outline the materials if necessary (using

the behavioral objectives as a guide). Plan any general format or

presentation pattern you believe is necessary or would be effective.

write the instructional materials for each objective or cluster of
objectives in rough form. A.rough slide/tape may have the sound recorded
on a simple cassette recorder with indicators for changing visuals, and

the accompanying visuals can be magic marker stick figures on 3 by 5

.index cards. Printed, visual, or auditory materials in this very rough

form will allow you to check your sequence, flow of ideas, accuracy of
illustration of ideas, completeness, pace, etc. Make a rough set of
materials as complete as possible for each instructional activity.
Consider each completed instructional activity, or cluster of objectives,
for clarity and flow of ideas.. Does it go from easier to most difficult,
from the complete skill (behavioral objective for that activity) back to
the basic components or subskills, etc.?

Using one complete instructional activity, write the students' manual or
accompanying instructions to the students for that activity. This could
include the objectives, directions, and possibly motivational materials.
Using the materials developed in this first inexpensive, rough draft, you

are ready to begin evaluation activities.

33‘!

Pretest on Step 6

What is the purpose of formative evaluation?

 

 

What is the purpose of summative evaluation?

 

 

How rigorous should formative evaluation be?

a. Very rigorous, taking nearly as long as the design phase.
Usually involves a number of one-to-one situations, a small
group situation and a field test;

b. quite rigorous, taking about half as long as the design phase.
Usually involves working through the lesson with one or two
students;

c. not at all rigorous, usually involves giving the materials to
the class and observing their reactions.

List some of the ways of displaying data from your formative
evaluation.

 

 

335

Answers

The purpose of formative evaluation is to obtain data to increase
the efficiency and effectiveness of the lesson (in order to revise
the instructional materials).

The purpose of summative evaluation is to determine whether the
materials should be used in a particular setting or whether they
are as effective as claimed.

The expected answer is (a) very rigorous.

Some of the ways that data can be displayed includes tables,
graphs, histograms.

 

Step 1

Step 2

336

Step5

Step 6

 

 

 

 

 

 

 

Specify
learning
outcomes

 

 

 

Anal yze
learning
outcomes

 

 

 

 

Step 3 Step 4
Develop
criterion PM" “33°“:
t..t. motivation
.. pretest media
- ”bedded
- poet test
- attitude

 

 

 

 

 

Construct
lesson

 

 

 

eveluetion

 

 

 

 

 

 

Formative Evaluation of the Lesson

 

Recycle l
I I

 

Step 6

Be able to state the purpose of formative evaluation.

Be able to state the purpose of summative evaluation.

Be able to state the procedures of formative evaluation.

Be able to state various methods of collecting data for formative
evaluation.

 

 

337

Step 6

Formative Evaluation of the Lesson

Formative evaluation is the process used to obtain data to increase
the efficiency and effectiveness of the module. The emphasis in formative
evaluation is on the collection of data in order to revise the instructional
materials rather than to determine whether materials are of any value or
whether they are better than another set of materials. In essence, at
this point in the development process, every effort is made to make the
materials as effective as possible. When they reach a final version,
other people may collect data to determine whether the materials should
be used in a particular setting or whether they are as effective as is
claimed. This latter type of evaluation is often referred to as summa-
tive evaluation.

There are three stages of formative evaluation. The first is
referred to as one-to-one or clinical evaluation. In this initial phase
the designer works with individual students to obtain data to revise the

materials. The second stage of formative evaluation is a small group

 

evaluation. A group of 10-20 students, who are representative of the
target population, study the materials in an approximate "real life"
setting to collect the required data. The third stage of formative
evaluation is usually referred to as a field evaluation. The number of
students is not of particular consequence; usually 30 is sufficient. The
emphasis in the field evaluation is on the testing to the procedures
required for the installation of the instruction in as real a situation

as is possible. These three phases of formative evaluation will be

described in some detail.

F

.3

 

338

One-to-One-Evaluation

 

The first type of evaluation conducted by the designer following
the development of a draft set of instructional materials is a one-to-
one. The term "one-to—one" refers to the fact that, at this stage in the
evaluation, the designer selects two or more students who are typical of
the target population and literally sits at their side as each studies
the materials.

Only the test and instructional materials are used with the student.
The designer should pick at least one student from.the target population
who is slightly above average in general ability and at least one student
who is below average, and work on an individual basis with each of these
students.

The typical procedure in a one-to-one evaluation is to explain to
the student that you are designing a new set of instructional materials
and that you very much wish to have a student's reaction of the materials.
You should indicate that any mistakes which students might make are
probably due to deficiencies in the materials and not theirs. Encourage
the student to be relaxed and to talk about the materials. You should
not only have the student go through the materials but also take the
test(s) which are provided with the materials. You might also note the
amount of time which it takes a student to complete the material.

Small Group_Evaluation I

After the materials have been revised on the basis of information
obtained from the one-to-one evaluation, you should next select a group
of approximately 10 to 20 students. This number is not selected entirely
arbitrarily. If the number of students is less than 10, then the data
which you obtain will be suSpect in terms of whether you really have a

representative sample of the target population. One the other hand, if

339

you obtain data on many more than 20 students, you will find that you
have more information than you need, and that the data from additional
students will not provide you with a great deal of additional information
for careful evaluation and analysis in a small group setting.

The selection of students to participate in your small group trial
is a very important procedure. The students used to evaluate the
materials should be as representative of your total target population as
possible.

The basic procedures used in a small group evaluation are quite
different from those used in a one-to-one. The evaluator (or the
designer) begins by explaining that the materials are in the formative
stage of development and that it is necessary to obtain feedback on how
they may be improved. Having said this, the designer would administer
the materials in the manner they are intended to be used when they are
in final form. If an entry behavior test or a pretest are to be used,
they should be given. The students then study the instructional materials
and all resources which are available to them. They would take the
posttest at the completion of the materials. There should be very
little intervention in the process by the evaluator (designer). Only in
those cases when equipment does notwwork or a student has come to a
complete halt in the learning process and cannot continue, should the
designer work with the student to overcome the problem. The difficulty
and the solution should certainly be noted as part of the revision data.

An additional step in the small group formative evaluation is a
debriefing of each of the students, and perhaps the administration of an
attitude questionnaire. In the attitude questionnaire you might ask a
variety of questions about the instructional experience. These questions

could deal with the adequacy of the amount of time spent, the interest

3&0

the student had in the materials, the identification of areas that were
too difficult or too easy, and so on. The critical point in designing
an attitude questionnaire is to ask questions which will provide data
which can be used in the revision process. The type of question which
would be of interest but of very little value 13: "Did you like these
materials?" A very gross yes, no, or maybe response is very difficult
to interpret in terms of determining how to use that information in
module revision.

In the debriefing discussion with the student after the materials
have been finished, all types of questions can be asked about the pacing,
interest, and difficulty of the materials. By providing cues to the
student, a great deal of information about the relative effectiveness of
the materials and areas of misinterpretation may be obtained.

All the data from these various sources are summarized and decisions
are made as to how revisions are to be made.

Field Evaluation

In the final stage of formative evaluation, the designer attempts
to obtain a learning situation which is, or closely resembles, that
intended for the instructional materials.

You should select a group of approximately 30 individuals to par-
ticipate in your field trial. Again, the group(s) selected should be
selected in a manner to ensure that it is representative of the target
population for which the materials are intended.

Data Collection

 

There are some general guidelines that you should consider when
planning the procedures for any stage of formative evaluation. The most

important suggestion is that you gather all the data which you believe

3h]

will help you make decisions about improving instruction. Any data
which cannot stand up to this test, that is, that you can see how the
interpretation of the data could suggest ways in which materials might
be revised, should not be collected, because it will be of only limited
use to you, and may interfere with the collection of data you really
need.

The types of data you will probably want to collect would include
the following:

1. Test data collected in entry behaviors, pretests, posttests

and embedded tests. (The latter, embedded test data, is often
overlooked as a rich source of information about student per-
formance while they are engaged in the learning process).

2. Comments or notations made by students to you or marked on the
instructional materials about difficulties encountered at
particular points in the materials.

3. Attitude questionnaires and/or debriefing comments in which
students indicate their overall reactions to the instruction
and their perceptions of where difficulties lie with the
materials and instructional procedures in general.

The formative evaluation component in the instructional design
‘model is what differentiates this empirical procedure from a philosophical
or theoretical approach. Rather than speculating about the instructional
effectiveness of your materials, you.will be testing them out with
students. Therefore, you will want to do the best possible job of
collecting data that is reflective of the effectiveness of your materials.
There are several concerns the designer should keep in mind when planning

and implementing data collection procedures.

 

3h2

One concern in any evaluation of your materials is that technical
equipment is operating effectively. More than one designer has been
discouraged because when he tried a new set of instructional materials
on a particular piece of equipment, the equipment did not operate cor-
rectly. Therefore, the data from students was invalidated, and the
designer learned little more than that you need to have the audio-visual
equipment operating effectively to try out materials.

The area about which we know the least in the entire instructional
design, development and evaluation process is that of revising instruc-
tional materials. If you examine almost any instructional design model,
you will find major emphasis on the concept of formative evaluation,
namely, collecting data to determine the effectiveness of instructional
materials. The model will then indicate that after data have been col-
lected and summarized, you should revise the materials appropriately.
While a number of studies have indicated the benefit of revising instruc-
tional materials, the entire process has been primarily empirical rather
than theory based. In effect, we interpret the data in the most reasonable
way possible and then make changes which seem to be indicated by the
data.

There are two basic types of revisions which you will be considering
with your materials. The first changes need to be made in the
content or substance of the materials to make them more accurate or more
effective as a learning tool. The second type of change is related to
the procedures employed in the utilization of your materials.

There are many different ways in which the data collected in the
formative evaluation may be summarized to pinpoint areas of student
difficulties. Perhaps the most basic table is one which lists the entry
behavior scores, pretest scores, and the posttest scores for each student

who participates in the formative evaluation study.

343

Student Performance as a Z
of Total Possible Score

 

Student Entry Pretest Posttest
Number Behavior

1 9O 15 85

2 90 25 92

3 100 20 87

4 9O 10 g 98

5 6O 0 65

6 9O 10 82

7 95 15 87

8 100 20 93

 

More detailed analysis can be provided by displaying the performance
of each individual student in the formative evaluation for each objective.
Place an "X" in the box for a particular objective and a particular
student to indicate whether the student successfully achieved that
objective. Such a chart, an example of one follows, indicates the total
score for each student, and the percentage of students achieving each

objective.

3M.

Student Performance on the Posttest
(X indicates objective was achieved)

 

 

Objectives
Student 1 2 3 4 5 Percent
l X X X X 80
2 X X X X 80
3 X X X X X 100
4 X X X X 80
5 X X X X 80
6 X X X X 80
7 X X 40
8 X X X 60
100 75 75 75 50
Percent X=75

An important source of data is the embedded test item. Such data
helps to pinpoint the difficulties the learner is having as he/she works

through the lesson. A table can be constructed to show this data.

335

Another way to display data is through the use of various graphing
techniques. A graph may show the pretest and posttest performance for
each objective in the formative evaluation study. You might also want
to graph the amount of time required to complete the instructional
materials as well as amount of time required for the entry behaviors

test and posttest.

100

90

8O

70

Percentage 60

Performance 50
40 0 0 Posttest
30 X X Pretest

20

10

 

Objectives

Pretest/Posttest Graph Showing Student Performance

The best way to summarize data from an attitude questionnaire is to
simply indicate on a blank copy of the attitude questionnaire the percent
of students who choose each alternative to the various questions. If you
request open-ended, general responses from the students, these can be

summarized at the end of the questionnaire.

3A6

Self Evaluation on Step 6.

 

1. Explain in your own words the following terms in relation to
your module development:

a. Formative evaluation

 

 

b. Summative evaluation

 

 

2. State the major steps in formative evaluation

 

 

 

 

 

3. What types of data would you collect for your formative evaluation?

 

 

 

 

397

Given the following results, what conclusions would you make if the

criterion level is 85%?

 

Student Entry
Numbers Ability Pretest Posttest
l 90 30 85
2 95 25 96
3 100 90 98
4 90 20 9O
5 65 IO 50
6 100 20 75
7 9O 15 90

 

 

 

 

 

3&8

Feedback

a. Formative evaluation is the process used to obtain data to
increase the efficiency and effectiveness of the module.

b. Summative evaluation determines whether the module should be
used in a particular setting or whether it is as effective as

is claimed.

The one-to-one or clinical evaluation.
The small group evaluation, approximating real life setting.
Field evaluation to see if the module can be installed as part of

regular instructions.

Test data collected on entry abilities pretest, posttest and
embedded tests.
Comments or notations made by students.

Attitude questionnaires.

a) Student 5 did not meet the criteria, most likely due to inadequate
preparation--she did not have the required entry skills.

b) Student 6 has not used the criterion level. It would seem that
the particular module has caused problems; just where the problem

occurred may be revealed by a study of formative tests.

349

You may have already begun work on your module, which
is fine. If you have not then now is the time to make a start.
The next pages are a point by point summary on the development
of your module. It may be helpful to use the guide as a check

list.

350

Guide for Developing a Module

You are now ready to begin the design of your module. Upon completing
each component of the model you should have two products: the components
for your module and the description of how you develOped the module.

For example, when you have completed your instructional analysis, you

should note which method you used to derive the analysis and why.

For the present, you should identify your learning outcome and state it
in terms of some type of student behavior. A primary consideration in
the selection of the goal for your module is that you do not select a
topic area which will require many hours of instruction. A general
guideline is to select something which would normally require 45 minutes
of typical classroom time to teach. These time parameters are suggested
in order to limit the amount of work you will be required to do in
developing, evaluating, and revising the module as well as time required
to arrange for students to participate in the formative evaluation

process.

A second major consideration is the selection of a topic with which you

are already familiar but feel free to consult a content Specialist.

351

The third consideration is the availability of students to use your
materials. You might choose a topic which you know well, and could
teach in 45 minutes, but if you have no access to the target population
for whom the materials are intended, then you should change the topic

or the target population.

After choosing your instructional goal and stating it behaviorally, you
should choose the appropriate instructional analysis technique. Don't
become discouraged or "hung-up" on this step in the model. You may wish to
write subskills on 3 x 5 cards so they can be easily changed, added, omitted
or recordered. It will take time and thought to get an analysis with which

you will be satisfied.

The next step is to identify the required entry behaviors. These should be
derived directly from your instructional analysis and should be stated in
behavioral terms. You should also be explicit about your target population

in terms of their characteristics.

J1-

Next you should write the sub-objectives for your module. These should
be a clear indication of the relationship between the skills in the
instructional analysis and the objectives. You should use the same numbers

on the instructional analysis chart and the list of objectives to make the

relationship clear.

Then develop the items for the test which you will be using in your materials.
The items you produce should be sufficient for a pretest (if needed), embedded
tests and posttests. You may also want to develop an entry behavior test.

All items should be keyed to the chart, described above, which contains

the subordinate skills and objectives.

You are now ready to develop your instructional strategy. Use the five
categories that are described earlier in the program. Consider how you will
address each of the five critical components of the strategy. Avoid simply

listing concepts or skills that you will be teaching. Rather, you should

353

describe the type of preinstructional activities you will include in
order to get the students' attention and inform them of the objectives
and prerequisites for the module. You should then describe what you

will do for each of the other components of the strategy.

You are now ready to develop the first draft of your instructional module.
Clearly, it should be developed in correspondence with your instructional

strategy.

Commercial modules often include one or more mediated instructional
components. However, for your module it may not be possible to develop

a filmstrip, videotape or some other special form of mediated instruction.

Be careful to build into your module all the directions a student will

need to complete the instruction without any assistance form an instructor.

JJ1

Some modules require students to work in small groups to achieve some of

the objectives.

During the initial stages of development of the module, do not try to
produce a "final copy" appearance for the materials. Your first attempt
may be simply a long-hard draft which can be tried out with one or more

students in a one-to-one formative evaluation.

You will be required to carry out two types of evaluation. The first

is the one-to-one evaluation. You should try to select two or more

students who are representative of your target population. Sit with the
student as he goes through the text and your materials. Answer questions

as they arise and make note of your comments and observations. After

the initial one-to-one you may want to make some simple revision before
working with the second one-to-one student. Based on the performance

and observations of the first two students, you may want to make revisions and
have additional one-to-one evaluations, or you may be ready to prepare a

revision of your materials for the small group evaluation.

355

In the small group evaluation, attempt to have no fewer than eight nor

more than 20 students in the group.

In the small group evaluation, you will want to administer an attitude
questionnaire and discuss the materials with some of the students. You

may also want to have the students note the time they spend on the module.

After you have completed your small group evaluation, summarize your

data for presentation in your report.

Following this guide and checklist is a copy of the criterion referenced
instrument which will be used to evaluate your report on the development
of your instructional module. It is suggested that you use the headings

' on the evaluation instrument as headings within your report.

356

The items listed under each assignment are the components which the

instructor will evaluate and to which points will be assigned.

In your report, you should describe how you carried out each step. If

there were revisions made in a component, you should always include the
final version. You may wish to include the original version in order

to show what changes were made. The evaluator will first be looking for

the inclusion of each item in the report. Then the quality of the component
will be judged. The various revisions made by the author should be

described along with the rationale.

The final product is a report of your module development and a copy of the

module which was used in the small group evaluation.

357

Check List for Developingfa Module

Step

Activity

Select Topic.

State the learning outcome.

Perform an instructional analysis based on

the stated learning outcome.

Describe the target population as specifically as

possible and identify entry behaviors.

Write performance objectives and subobjectives.

Develop criterion-referenced tests for the objectives.

a.

Determine whether your testing purpose will be
to diagnose students' entry behavior, to branch
students, to evaluate students, to evaluate
materials or all of these.

Determine whether a test over expected entry
behaviors is necessary or advisable.

Divide parallel Questions written for each
objective such that some can be included on a
pretest (if a pretest is advisable), some can
be used for embedded questions for practice and
feedback during instruction and some can be

included on the posttest.

358

7. Develop the instructional strategy.

a.

Review the subskills included in the instruc-
tional analysis and determine which should be
included in instructional materials and which
should be included as expected entry behaviors.
Consider the target population relative to
their attention span, work and study habits,
responsibility and motivation. Then estimate
an average "best" time for each instructional
activity for the group.

Consider the objectives in the instructional
analysis relative to the type of learning and
learning activity required for each objective,
the general amount of time you believe would be
required to teach each objective and the logi-
cal sequency in which you believe objectives
should be presented.

Cluster objectives. Each cluster might have
one or more objectives depending on the diffi-
culty of explanation or time required for
practice and feedback. There are no rules for
clustering, and you have only the instructional
analysis as a guide to develop the first set of

materials.

Survey the literature and subject matter experts to

determine what instructional materials are already

available.

ll.

12.

)97

Consider how available materials might be adapted
for use using a manual to guide students through the
materials.

Determine whether new’materials need to be designed.

If yes, proceed to 11. If no, begin organizing and

adapting available materials.

Given each cluster or set of instructional activities,

make decisions about relevant motivational materials,

necessary review, presentation procedures, student
participation, practice, feedback, evaluation and
followbthrough activities for each cluster or instruc-
tional activity.

For each cluster or instructional activity, consider

the best medium to present the materials, to monitor

practice and feedback, to evaluate and to pass
students to the next instructional activity whether
enrichment, remediation or the next activity in the
sequence.

a. Make decisions about the ideal media for the
type of learning.

b. The types of media required for a large set of
materials may be many and varied. Cost effec-
tiveness of developing materials using varied
equipment and subskills should be considered.

c. Availability of materials, portability of
materials, availability and appropriateness of

equipment should be considered.

13.

14.

15.

360

d. Tentative decisions about the best medium or
media combinations should be made based on
studying types of learning, activities required,
physical and technical constraints, finances,
facilities and the target population.

Determine the format and presentation prodecures for

each objective or cluster of objectives. Plan

any general format or presentation pattern you

believe is necessary or would be effective.

write the instructional materials for each objective

or cluster of objectives in rough form. A rough

slide/tape may have the sound recorded on a simple
cassette recorder with indicators for changing
visuals, and the accompanying visuals can be magic

‘marker stick figures on 3 by 5 index cards. Printed,

visual or auditory materials in this very rough form

will allow you to check your sequence, flow of
ideas, accuracy of illustration of ideas, complete-
ness, pace, etc. Make a rough set of materials as
complete as possible for each instructional activity.

Consider each completed instructional activity, or

cluster or objectives, for clarity and flow of

ideas. Does it go from easiest to most difficult,

from the complete skill (behavioral objective for

the activity) back to the basic components or sub-

skills, etc.?

l6.

17.

18.

36]

Using one complete instructional activity, write the
student's manual or accompanying instructions to the
students for that activity. This could include the
objectives, directions and possibly motivational
materials.

Using the materials developed in this first inexpen—
sive, rough draft, you are ready to begin evaluation
activities.

You may either develop materials for the instructor's
manual as you go along, or you can take notes as you
develop and revise the instructional presentations
and activities. Using the notes, you can write the
instructor's guide. If you wait until the end to
complete the instructor's manual, you will surely
want to at least design the manual and determine
what types of information, suggestions, and tests
you will want to include in it. By designing the
manual early, you will know what information to
collect and procedures to note as you go along to be

included in the manual.

362

Formative Evaluation Activities

 

l. One-to-One Testing

Participation by students from the target population

1. Identify students that are typical of those you believe
will be found in the target population. (Include each
type of student that can be found in the target population).

2. Arrange for the student(s) to participate.

3. Discuss the process of a one-to-one test with the students.

4. Evaluate the test you have constructed to measure entry
behaviors. a. Can the student read the directions?

b. Does the student understand the problems

c. Does the student have the required prerequisite
skills?

5. Sit with the student while he or she goes through the
materials.

a. Instruct the student to write on the materials to
indicate where difficulty is encountered or to
verbally discuss ideas and problems.

b. If the student does not understand an example, try
another verbal example. Does this clarify the
issue? Note in writing the changes and suggestions
you make as you go through the materials.

c. If the student does not understand an explanation,
elaborate by adding information or changing the
order or presentation. Does this clarify the issue?

Note the changes you make in writing.

363

d. If the student appears bored or confused while going
through the materials, you may want to change the
presentation to include larger or smaller bits of
information before practice and feedback. Record
your ideas concerning the regrouping of materials as
you go along.

e. Keep notes on examples, illustrations, information
you add, and changes in sequence during the evalua-
tion process. Otherwise you may forget an important
decision or idea. Notetaking should be quick and in
rough form so the student is not distracted from the
materials. Even changes that seem trivial should be
included in a one-to-one evaluation report.

6. You may choose to test another student from the target
population before you make any changes or revisions in
you materials in order to verify that the changes are
necessary. However,if errors pointed out by your student
"consultant" are obvious, you may want to make revisions
before testing the next student both to save testing time
and to enable the next student to concentrate on other

problems that may exist in the materials.

B. Participation by subject matter experts

1. You should provide the expert with: (a) learning out-

comes, (b) the instructional analysis, (c) the intended

instruction, and (d) the tests. These materials should

)U'Y

be in very rough form, because major revisions could well
be the outcome of this one-to-one testing. You may want
to present your materials in the order described above.
You should be looking for verification of the (a) objec-
tive statements, (b) instructional analysis, (c) accuracy
and currency of the content, (d) appropriateness of the
instructional materials in vocabulary, interest, sequence,
chunk size and student participation activities, (e)
clarity and appropriateness of test items and test situations,
and (f) placement of this piece of instruction relative
to prior instruction and followbthrough instruction.

The number of subject matter experts you should approach
for assistance will vary with the complexity of the
information and skills covered in your materials. For
some instruction, one expert will be sufficient while for
others four may still seen inadequate. The nature of the
teaching task will dictate the number and type of expert

consultants you will need.

II. Small group testing

A. Participation by students from target population

Identify a small group of students who typify those in
your target population.

Arrange for a student sample to participate

a. Adequate time should be arranged for required testing

as well as instructional activities.

365

During the student's participation in the pretest, in-

struction and posttest, you may want to make notes about

suggestions for teachers who will use the materials

and/or about changes you want to make in the instruction

or procedures as a result of observing students interacting
with the materials.

Administer the test or required entry behaviors if one

is appropriate.

a. Check the directions, response patterns, and ques-
tions to ensure the wording is clear.

b. Instruct students to circle words they do not under-
stand and/or place a (XL) beside questions or di-
rections that are unclear.

c. Do not stop and discuss unclear items with students
during the test.

d. Record the time required for students to complete
the entry test.

Administer the pretest of skills to be taught during

instruction. This test and the test of required entry

behaviors could be combined into one test if desirable.

a. Have students circle any vocabulary which is unclear
to them.

b. Have students place a (7t) beside any directions,
questions or response requirements that are unclear
to them.

c. Have students write additional comments in the test
if they desire.

d. Record the time required for student to complete the

pretest.

e.

)uu

Do not discuss programs during the test with students.

Administer the instructional materials. Have the in-

structional setting as close to reality as possible with

all required equipment and materials present. Any in-

structional assistance required should also be present

during the trial.

a.

Train the needed instructional personnel to use the
materials in the intended manner.

Instruct students that you need their help in
evaluating the materials.

Have students sign their work so you can compare
their performance on the lesson with your expecta-
tions of their performance based on their entry
behaviors.

Instruct students to circle any unclear words and
place a ()t) beside any illustrations, examples, or
explanations that are unclear in the instruction.
Students should keep working through the materials
to the end without stopping for discussions.

Record the time required for students to complete
the instructional materials. Time required may be
distorted if students require instruction on unfam-

iliar equipment or procedures.

Administer the posttest

a.

Have students sign their posttest to enable comr
parisons with the pretest and embedded tests.

Have students circle any unclear vocabulary and
place a (x) beside any unclear directions, question

or response requirements.

367

c. Have students respond to as many items as they can

whether they are sure of the answer or whether they

are guessing. Often incorrect guesses can provide

clues to inadequate instruction. You may want them

to indicate which answers required guessing.

d. Record the time required for students to complete

the posttest.

8. Administer an attitude questionnaire to students and/or

instructors administering the materials.

a. You may want to ask questions like:

1)
2)

3)

4)

5)

6)

7)

8)

9)

10)

Wes the instruction too long or too short?
was the instruction too difficult or too easy?
Did you have problems with any sections or
parts of the instruction?

Were the cartoons or illustrations appropriate
or distracting?

Was the color interesting or distracting?
‘What did you like most?

What did you like least?

How would you change the instruction if you
could?

Did the tests measure the material that was
presented?

WOuld you prefer another medium?

9. Arrange for students to verbally discuss the pretest,

instruction and/or posttest with you or their teacher

after they have completed all the work.

)00

a. You may want to structure the discussion with
planned questions.

b. You may want to ask questions like, "would you
change the exercises in section X?" or, "Did you

like the example in section X?"

III. Field testing

A. Select an appropriate sample from the target population.

3.

Arrange for the selected group to try the materials.

a. Ensure there is an adequate number of students in
the group. Thirty is an often suggested number of
students to participate in a field trial.

b. Ensure that selected students are representative of
the range of abilities and skills of students in the
target population.

c. Ensure there are adequate personnel, facilities, and
equipment available for the trial.

Distribute the instructional materials as well as the

instructor's guide, if it is available, to the instructor

conducting the field test.

Discuss any instructions or special consideration which

may be needed if the instruction is out of context.

Stay away from the testing situation yourself as much as

possible.

Summarize the data you have collected. Summarized data

may include: (a) the report on the entry behavior test,

369

(b) the report on pre and posttest scores, (c) the report
on the time required for students to complete each test
used, (d) the report on the time required for students

to complete the instruction, (a) any remediation or
enrichment needs that become visible, (f) the report on
the attitude survey for students as well as from parti-

cipating instructors if possible.

370

The final part of this Manual is a criterion referenced
evaluation sheet, followed by rationale for the criteria.
The evaluation instrument is based on work done by

Dr. Walter Dick of Florida State University.

37]

moms EVALUATION

 

I.

II.

III.

V.

VII.

Instructional Cool

A.

rationale for module (criteria used for selecting

‘ mtmmm 3°“) 0 O O O O O O O O O O O O I I O

statementofinstructionslgosl . ... ......

Instructional Analysis

A.
I.
C.
D.

rationale for methodology selected . . . . . . . . .
completeness of diagram of subskills . . . . . . . .
description of relationship among subskills . . . .
clarification between subskills included as entry

behaviors and those included as skills to be learned
muhutumeeeeeeeeeeeeeeeee

Description of Target Population

A.

description of general characteristics of. target
population and implicstions these have for instruc-

- mm “a.mh O O O O O O O 0 O O O O O O O O O 0

Learning Outcomes

A.
I.

derivation from instructional analysis . . . . . . .
statement of the leaning outcome . . . . . . . . .

Criterion-Referenced Tests

A.
I.

C.

relationship of items to learning outcomes . . . . .
criterion level or mastery level of performance
clearlyidentified. . . . . .. . . . . . . . . .‘.
appropriate tests based upon materials and target
popultsion (pretests, posttest, embedded test, entry
behaviors test)

Instructional Strategy

A.
3.
C.
D.
3.
I.

preinstructional activities .. . . . . . . . . . . .
informationpresentstion. . . . . . . . . . . . . .
student participation activities . . . . . . . . . .
testing (covered in No. V) .

follow-through activities . . . . . . . . . . . . .
strategy congruent with materials . . . . . . . . .

Formative Evaluation

A.
B.
C.

description of one-to-one data collection procedures
results and revisions based upon one-to-one testing
description of small-group evaluation procedures
1. sample group characteristics . . . . . . . . . .
1. design of formative evaluation study . . . . . .
3. instruments and procedures used for data col-
1.:cm O I O O O O O O O O O O O O O O O O O O
4. dstssummsrysnddisplsy. . . . . . . . . . . .

Suggested Revisions

A.
3.
C.

u:.rn1. O O O O O O O O O O O O O O O O O O O O 0
tea ts O O I C O O O O O O O O O O O O O O O O O O O

pmc .dur.’ O O O O O O O O O O O I C O O I O I O C 0

TOTAL

Maxims

Points

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Criterion

Level

 

 

 

 

 

 

 

 

 

 

..IH

 

U

 

p

 

Ul

 

+

 

|~|~

II I'l.‘

 

 

 

In old A A_‘

73

Points

Awarded

llll ll

I.

II.

III.

372

MODULE EVALUATION

Instructional Goal

A.

The designer should clearly indicate the rationale for selecting the
topic for their instruction. The rationale should include such items
as their expertise in the content area, ability to provide the in-
struction within a reasonable amount of time, the availability of
students to try out the materials in the formative evaluation, and the
need for this type of instruction.

The goal statement should be a clear indication of the outcome of the
instruction. Although the goal does not need to be stated in precise
behavioral terms, it should reflect, in general, what students should

be able to do when they have completed the instruction.

Instructional Analysis

A.

C.

The designer should indicate which instructional analysis methodology

has been chosen and why it was selected.

The designer should include a complete instructional analysis chart that
lists all the subskills associated with the instructional goal. The sub-
skills should be stated in behavioral terms. Entry behaviors should be
included on the chart and clearly indicated as such.

The designer should describe the diagram previously described in B and
indicate the relationship of the subskills to each other.

The designer should describe the entry behaviors that have been identified
and explain the rationale for including them. For instruction that has no

relevant entry behaviors, students should explain why none are required.

Description of Target Population

A.

The designer should describe the general characteristics of the target
population for when their instruction is being prepared. The target

populations may vary from a particular classroom group to a national

373

category of students. Important aspects of this component are that the
target population is identified and that there is some statement of the
implications of the relationship between the general characteristics of

target students and the instruction to be provided for them.

IV. 'Performance Objectives

A.

The designer should include in the documentation report a chart that in-
dicates in one column the subskills and instructional goal as identified
in the instructional analysis and ins second column, the corresponding
performance objectives and terminal objective for the instructional unit.
Subskills and objectives should be numbered similarly in order to indicate
their relationship.

The objectives for the instruction should be stated using the three
components suggested by Mager. These include the conditions, performance,
and criteria. The alternative approach to writing objectives suggested
by Gagne and Briggs is also appropriate. Student should indicate which

format they have used for the objective statements.

V. Criterion-Referenced Tests

A.

The designer should provide a clear indication of the relationship between
their test items and their instructional objectives. This relationship can
be indicated by assigning the same numbers to test items as was used to
number the corresponding subskills and objectives, or, a chart can be
developed which indicates the item number of each performance objective and
the corresponding test items.

The designer should indicate the criterion used to judge the mastery level
of learners' performance relative to each of the objective and to the

total test.

After reviewing the test, the instructor should determine if there are an

adequate number of test items and whether the items are appropriate for

the target population and the content of the instruction.

VI.

37h

Instructional Strategy

A.

The designer should describe how their instruction will motivate learners,
inform them of what they are going to learn, and remind them of related
information which they already know.

The designer should describe how the content will be presented to learners.
This explanation should not simply be a running description of the specific
content, but rather an indication of how "chunk" size will be determined
and sequenced.

A description should be included of the activities provided to learners to
practice the behaviors taught in the instruction. The provision of feedback
for learners' performance should be described.

The evaluation of testing strategies was discussed previously in Section V.
Although the designer is not typically required to develop any remedial or
enrichment activities, a brief description of suggested ones should be
included.

After examining the instructional strategies described by the designer, the
instructor should examine the instructional unit itself to determine whether
the materials are consistent with the strategies described by students in

their documentation sections.

Formative Evaluation

A.

The designer should describe the preparation of instruction for the one-to-
one formative evaluation. This description should include the format of
materials to be evaluated, the characteristics and entry behaviors of
learners who participated in the evaluations, the criteria for the selection
of the learners, and the procedures followed in conducting the one-to-one
evaluation.

The designer should describe the results of the one-to-one evaluation,
particularly the major comments received from learners during the evaluation.

There should be a description of the types of revisions made in the materials

C.

375

and the.rationale for these revisions based on the results of the
one-to-one evaluation.

The designer should provide a complete description of the characteristics
and entry behaviors of learners who particpsted in the small-group
evaluation. There should also be a complete description of the pro-
cedures employed in implementing the evaluation. The description
should include the format of the materials, the procedures which were
used for testing, the instructions given to learners, and the feedback
received at the end of the instruction. There should be a description
of all instruments and procedures used to collect data. These des-
criptions can be used to document the effectiveness of the instruction
and to identify points in the materials that still need revision.
Students should summarise the data collected, including the display of
such items as pretest and posttest performance, performance on em-
bedded-test items, learning time, performance by objectives, and

learner attitudes as indicated on questionnaires.

VIII. Suggested Revisions

A.

C.

The designer should describe, based on the data and information gathered
during the small-group evaluation, the revisions which remain to be

made in their materials. The relationship between these revision
recommendations and the data obtained during the small-group evaluation
of the materials should be apparent.

The designer should indicate revisions that remain to be made in the test
instruments as a result of the data and information gathered during

the small-group evaluation.

The designer should indicate any instructional procedures that need to be
revised as a result of information gathered during the small-group
evaluation. They should describe any changes in procedures or ma-

terials needed if materials were used in a regular classroom setting.

376

Criteria for Evaluating Designer-Prepared Instructional Materials

 

Materials should include objectives, pretests/posttests, and the instruction.

2. The instruction should follow the instructional strategy established for the

unit as described in the documentation report. Some general guidelines:

nave adequate motivation and attention getting ideas been provided for

the content to be taught and the characteristics of the target learners?
lave learners been informed of necessary prerequisite knowledge or gkillg?
Have students been informed about what they will be learning to do (coupon
language objectives)?

Have ideas and concepts been presented in a logical sequence?

Is enough explanation provided for pupils to actually "learn" the ideas,
concepts, and skills?

Are accurate and adequate examples included of ideas, concepts, and skills
being taught?

Are there embedded tests to provide pupils with the opportunity to
practice skills?

Is adequate and accurate feedback provided to pupils responses to the

embedded tests?

3. Are materials legible, clear, and easy to follow?

 

It may be expected that the instructional materials will be prototype or first

drafts that are inexpensively produced. Drawings and illustrations should be

stick figures or magic marker drawings and materials should be inexpensively

duplicated. This ”draft" nature of the materials should not detract from the

students' evaluation on their materials. The important issue is that the materials

include the information on the right pages, and in the right order, so that it can

be tested with target learners, analyzed, and revised before being produced in a

more polished format.

377

Post Module Development Survey:

 

to the Designer

1 . Given the Opportunity would you design another module,
following this model?

 

 

 

2. How long did you spend on the development of this module?
(in hours)

 

 

 

3. If you were to do another module how long would that
take in relation to the first? (e.g. just as long, half as

long....)

 

 

4. Has the design of your module changed you in the way you
teach? (e.g. greatly, slightly, not at all . . . .)

 

 

5. Would you recommend this model to anyone else?

 

 

 

373

6. Was there an aspect of your design process which proved
frustrating because the program was unclear or deficient
in that area?

 

 

 

 

7. Can you recall a single most positive aspect about the program?

 

 

 

8 . Did you refer to any of the references given?
If you did, was it because of need or simply interest?

 

 

 

APPEN DIX E

Post Module Development Survey:

to the Designer

Post Module DeveIOpment Survey:

to the Designer

Given the opportunity would you design another module,
following this model?

 

 

 

How long did you spend on the development of this module?
(in hours)

 

 

 

If you were to do another module how long would that
take in relation to the first? (e.g. just as long, half as

long. ...)

 

 

Has the design of your module changed you in the way you
teach? (e.g. greatly, slightly, not at all . . . .)

 

 

Would you recommend this model to anyone else?

 

 

379

380

6. Was there an aspect of your design process which proved
frustrating because the program was unclear or deficient
in that area?

 

 

 

 

7. Can you recall a single most positive aspect about the program?

 

 

 

8 . Did you refer to any of the references given?
If you did, was it because of need or simply interest?

 

 

 

 

APPENDIX F

SUMMARY OF PROGRESS OF THE
DIETETIC UNDERGRADUATE RESOURCES
COMMITTEE (DURC)

The first major task of the committee was to deten’nine the
competencies required of an entry level dietitian; the task was
completed early in 1976. Next it was necessary to start the
process of sequencing competencies and translating competencies
into modules. At this stage considerable resistance to the compet-
ency based prograh began to surface from the faculty involved in
the preparation of dietitians at M.S.U . , mostly among those who
were not members of DURC.

To find out more about the opposition to the program, the
author and another graduate student spent the summer of 1976
interviewing faculty who opposed the introduction of CBE. The
general response from the faculty was distrust, and to some degree,
rejection, of a program of instruction which they perceived as
running counter to their own beliefs and methods of teaching. In
some cases the rejection was based on extersive knowledge of CBE,
in other cases (the majority) it was clearly because of insufficient
knowledge. Nearly everyone complained of not being consulted or
involved .

To remedy the communication gap, efforts were made to
involve the outside faculty in the program development by circulating
mimtes of DURC meetings, and circulating journals and articles.
The situation did not seem to improve. Those who were not know-

ledgeable of CBE were still not receiving the information they needed.

381

382

Worse, those who had extensive understanding of CBE found the
dissemination process alienating as they perceived it as a method
of persuasion, using a downward flow communication model, to
bring about dubious education practices. There seemed no place
in the process where they could express their concerns and bring
about some accomodation of views.

In March, 1977, to deal with this impasse, it was decided
that members of DURC should act as change agents and arrange
one to one meetings with resistant faculty. Prior to embarking on
these meetings a number of DURC members themselves expressed
a lack of sufficient knowledge about CBE to be able to conduct
interviews. To overcome this, it was agreed to hold a number of
training sessions for DURC members where the conceptual matters
of CBE would be discussed and processes of corsulting would be
practiced. After a number of training sessions members still
felt they lacked sufficient knowledge to carry out consulting. A
major concern expressed was the lack of concrete evidence of
the CBE concepts. For example, the concept "module" was
difficult to conceptualize without having concrete examples to give.
When two more meetings of DURC failed to provide the necessary
knowledge on CBE, the idea of working as consulting change agents

was dropped .

ER

"liliﬁlﬂﬂvﬂﬂlﬁlﬂlﬁﬂﬂlﬂiltjnjﬂﬁllll“