—
fl, -7..- .......... ,w'i
'b

A COST~ EFFECTIVE ANALYSIS OF SELECTED READING
PROGRAMS IN THE GRAND RAPIDS PUBLIC SCIBOLS

ThesIs for the Degree of Ph. D.
MICHIGAN STATE UNIVERSITY
JOAN MARIE WEBSITE
1972

 

  
   
   

  

L I B R A I“. Y
Michigan State
University

Intfi'w

This is to certify that the

thesis entitled

Cost Effective Analysis of Selected Reading
in the Grand Rapids Public Schools

presented by

Joan M. Webster

has been accepted towards fulfillment
of the requirements for

 

Pb- n - degree in Education

Major professor

DateMay 4 , 1972

 

0-7 639

dilate
we 2>

Wfi

N/c/b?

$1. a n‘
H);
1] ‘91 7 pa

MAR 0 8 2003

 

 

 

ABSTRACT

A COST-EFFECTIVE ANALYSIS OF SELECTED
READING PROGRAMS IN THE GRAND RAPIDS
PUBLIC SCHOOLS

BY

Joan Marie Webster

The Problem
The current challenge of accountability from the
taxpaying public requires proof of results. The results
might best be based upon a per-pupil cost of a learning
unit or grade gain skills acquired rather than solely on

the cost of maintaining children in school.

Purpose of the Study
The major purpose of the study was to determine a
cost-effectiveness relationship between inputs (in-
structional costs) and outputs (student achievement gains)
in six reading programs in the Grand Rapids Public Schools.
Subsequently, the design shown could establish a model upon
which school officials can make future decisions regarding

the use of resources available.

Joan Marie Webster

Student Population Sample

The studied programs operate within the Title I
area (inner city) identified as being eligible for com-
pensatory funds. A population sample was drawn from each
of the programs. The students have been identified as the
lowest performing students in reading in that grade level
in selected schools as verified by the school system's
previous spring testing scores. All students in the sample

are enrolled in grade levels 2, 3, 7, 8, and 9.

Procedure and Design

Due to the variety of programs, the different time
allotments for the reading activities and unequal class
sizes; the inputs (instructional costs) were reduced to a
common denominator of student minutes of exposure (SME) in
six cost categories that were defined as the most appropri-
ate definable variables that constitute the differences in
the programs studied. The six cost categories were
identified as staff salaries, both professional and
paraprofessional; start-up costs; added facility costs;
staff training costs; student and teacher incentive costs;
and costs for materials and supplies. A per-student annual
cost for the programs studied included three performance
contracting programs: Alpha II, Westinghouse Learning

Corporation (WLC),* Combined Motivation and Educational

 

*Now called Learning Unlimited.

Joan Marie Webster

Systems (CMES); Project Read--a publishing company's
"canned" program; Traditional Remedial Reading (TRR) and
a regular (control) school program. All other costs were
considered to be prorated equally across all programs.

A pre/post testing program using the Metropolitan
70's standardized achievement test measured the student
gains in the reading programs studied.

The statistical design deemed most appropriate for
examining the reading achievement gains was the analysis
of variance. Because of the unequal cell size, the
Scheffé post hoc procedure was applied.

To compute the cost of one-tenth (.1) student
achievement gain in reading and thereby establishing a
relationship between costs and gains (inputs to outputs),
the annual cost for reading was divided by the average one-
tenth grade gain.

Achievement Gains and Cost-
Effectiveness ResuIts

 

The unequal assignment of program treatments
prohibited any over-all analysis covering all of the data
at one time. Thus, three separate analyses were performed.

The elementary grades 2 and 3 were exposed to
Alpha II, WLC, Project Read, TRR, and a control program.
The middle grade 7 was exposed to Alpha II, CMES, TRR,
and a control group program. The middle grades 8 and 9

were exposed to Alpha II, CMES, and the control program.

Joan Marie Webster

 

 

 

Reading Treatment Pro ram Grades

and Mean Achievement Gain 2 3 7 8 9
Alpha II .4 .5 .5 1.0 .8
Westinghouse .6 .7

Project Read .7 .7

Traditional Remedial Reading .8 .8 .2

Combined Motivation and Education .7 1.1 .6
Control (Regular School) .7 .4 .4 .3 .4

The relationship of costs to gains (cost-
effectiveness) is summarized below for one-tenth (.1)

student grade gain:

Elementary Grades 2 and 3

 

Alpha II $ 22.51
Westinghouse Learning 11.82
Project Read 16.34
Traditional Remedial Reading 49.60
Control (Regular Program) 14.28

Middle School Grade 7

 

Alpha II $ 19.89
Combined Motivation and Education 16.23
Traditional Remedial Reading 274.75
Control . 23.82

Middle School Grades 8 and 9

 

Alpha II $ 10.47
Combined Motivation and Education 13.18
Control 25.24

Cost per one-tenth (.1) student achievement gain

in reading.

Joan Marie Webster

Major Conclusions

 

Elementary Grades 2 and 3

 

1.

All reading programs studied resulted in less than

one (1) year reading achievement gain.

There were no significant differences in grade
gain achievement among the programs within the two

studied grade levels.

Cost-effective analysis leads one to conclude that

TRR programs are the most costly.

The Westinghouse program is indicated to be the

least expensive per one-tenth (.1) grade gain.

Westinghouse, Project Read, and the control program
are the most cost-effective, producing the greatest
gains for less cost. All three are within a

$4.00 cost per one-tenth (.1) grade gain. However,
the control group was over 20 per cent more
expensive than WLC, Project Read was over 38 per

cent more expensive.

The overall mean gain for TRR was the highest but
the cost was considerably higher than the other
four programs, almost five times as much as the

lowest cost program.

Middle

Joan Marie Webster

School--Grade 7

 

1.

Middle

The TRR program had the highest cost per one-tenth
(.1) achievement gain (almost twelve times the

control group cost).

The CMES and Alpha II programs were the most cost-
effective, i.e., they provided the most learning

for the dollar.

The control group performed half as well as the

performance contracting programs.

Schools--Grades 8 and 9

 

1.

There was a significant difference between the
control group and both performance contracting

programs; Alpha II and CMES.

The control group was the most expensive group per
one-tenth (.1) grade gain with the Alpha II program
being only slightly over 41 per cent the cost of
the control group and the CMES program cost being

68 per cent of the control group.

Student achievement gains in Alpha II and CMES
were slightly above one (1) grade level, whereas
the control group only gained approximately one-

third as much.

Joan Marie Webster

Implications

 

The findings of this study seem to indicate that

certain implications for the reviewer's consideration must

be mentioned.

cations

l.

Herewith are presented several of these impli—

which should be given further deliberation:

The apparent implications created by the inter-
grouping of students in learning situations may
have some sociological impact. In the performance
contracting programs, there appeared to be an
effective social relationship established among
students for self-help and a help-one-another
attitude, i.e., one student tutoring another which
could have a further positive effect on better

achievement.

Students who could evaluate their progress in the
more structured and individualized programs appeared
to have an improved attitude towards themselves,

their peers and the school.

Student attendance in school did improve in the
performance contract and programmed instruction
programs which may have been a result of (a) better
student self-image, due to the student's perception
of his success, (b) the contract use obligation of

the contractor and staff which required a minimum

Joan Marie Webster

number of days attendance, and (c) or a greater
sensitivity of the school system for keeping more

accurate records.

Very serious consideration must be given the
feasibility of continuing the traditional remedial
reading programs as they are being carried on
today. There are implications in this study for

a more structured program. There are also impli-
cations for improving the methods of selecting
students and the sociological impact upon the
student who is chosen and identified with the
remedial program. The isolation factor of a
remedial reading setting may prove a hinderance

to the student's ability to learn.

The role of the building principal may need to be
redefined. The principal, in order to fulfill the
requirement of a two-way street of accountability
must have the autonomy to make decisions in
implementing learning programs which should
include day-to-day changes in program design as
well as adequate local latitude in financial
decisions affecting the immediate implementation

of program changes.

Joan Marie Webster

This study would indicate that schools have just
begun to develop criteria for performance objec-
tives of the education function. Schools will need
to more clearly define these objectives as well as
develop methods of evaluating the success or

failure in achieving these objectives.

A COST-EFFECTIVE ANALYSIS OF SELECTED
READING PROGRAMS IN THE GRAND RAPIDS
PUBLIC SCHOOLS
BY

\1. .\
N”
.I \{V

V
Joan Marie Webster

A THESIS

Submitted to
Michigan State University
in partial fulfillment of the requirements
for the degree of

DOCTOR OF PHILOSOPHY
Department of Elementary and Special Education

1972

© Copyright by
JOAN MARIE WEBSTER

1972

ACKNOWLEDGMENTS

If I were to acknowledge all those who contributed
directly and indirectly to this effort this would be the
longest section of all. I have chosen to be selective,
therefore, and give a special vote of appreciation to the
following:

The Administrative Staff of the Grand Rapids Public
Schools, namely, Mr. Phillip E. Runkel, Superintendent;
Dr. C. Robert Muth, Deputy Superintendent; Dr. Elmer H.
Vruggink, Assistant Superintendent for Instruction; and
Mr. Wallace Norgrove, Director of Special Programs. With-
out their continued support and cooperation, this study
would not have been possible.

In addition to these staff members, I especially
thank Mr. James Gallagher, Director of Budget and Finance,
who worked endlessly to assist me with the development
and computation of the cost data.

Dr. Stanley E. Hecker, School Finance Expert, for
his valuable consultations and advice in designing the
cost data model.

Dr. Leon Lessinger, Calloway Professor, for his
"long distance" enthusiastic suppdrt of my endeavors.

Mr. Charles Stalford, Project Manager of the
Office of Economic Opportunity; Remedial Performance
Incentive Project, for providing the testing and data
collection services.

Dr. Ruth Hill Useem, member of my doctoral com-
mittee, who not only provided me with sociological
insights but with a feminine model of a professional
educator.

Dr. Calhoun Collier, member of my doctoral com-

mittee, for his general support given to me throughout my
academic and professional career.

ii

Dr. William Durr, member of my doctoral com-
mittee, for sharing his valuable insights into the teach-
ing of reading and for providing growth opportunities for
me within the profession.

Dr. Vern Hicks, chairman of my doctoral committee,
for his seemingly infinite patience and strong support
throughout my program.

Dr. Norman P. Weinheimer, director of my doctoral
dissertation, former Superintendent of the Grand Rapids
Public Schools who had the courage to initiate most of
the programs studied in this thesis and who, as Michigan's
leading proponent of accountability in the public schools,
provided the idea and much of the incentive for the study.

And most of all, my husband, Dick, who somehow
managed to keep his composure during this period of
uncertainty. Without his love and encouragement, I would
not have completed this dissertation.

iii

TABLE OF CONTENTS

Page
LIST OF TABLES o o o o o o o o o o o o Vii
LIST OF CHARTS o o o o o o o o o o o 0 ix
Chapter

I. INTRODUCTION . . . . . . . . . . 1
Need for the Study . . . . . . . . 1
Nature of the Problem . . . . . . . 14
Purpose 0 O O O O O O O 18
Importance of the Study . . . . . . 20
Limitations and Assumptions . . . . . 21
Definition of Terms . . . . . . . 23
Summary . . . . . . . . . . . 25
II. SELECTED REVIEW OF LITERATURE . . . . . 26
Review of Literature . . . . . . . 26
Evaluation . . . . . . . . . . 59
Summary . . . . . . . . . . . 65

III. DEVELOPING AND FIELD TESTING
THE MODEL . . . . . . . . . . . 67
Introduction . . . . . . . . 67

Description of the District
to be Studied . . . . . . . . 68
Reading Programs Within the District . . 69
Student Body . . . . . . . . . . 73
Evaluation Instruments . . . . . . 74
Statistical Techniques . . . . . . 76
Program Variables . . . . . . . . 77
Sumary O I I O O O O O O O O 80

iv

Chapter Page

IV. ANALYSIS OF SELECTED CRITERIA . . . . . 82
Cost Categories . . . . . . . . . 82
Cost Category Summary . . . . . . . 85
Structure of the Model . . . . . . . 86
Program Description and Costs . . . . 87

Alpha II Learning Systems,

Performance Contracting

Project . . . . . . . . . . 89
Westinghouse Learning Corporation,

Performance Contracting

Project . . . . . . . . . . 99
Combined Motivation and Educational

Systems, Performance Contracting

Project . . . . . . . . . . 105
Project Read-—"Canned" Program . . . 110
Traditional Remedial Reading . . . . 116

Control Group--Reading . . . . . . 122

Reading Annual per Student Cost

Summary . . . . . . . . . . . 131
Per Program Student Achievement
Gains . . . . . . . . . . . . 133

Statistical Analyses of Gain Scores . . 133

Descriptive Analysis of Program Cost-
Effectiveness . . . . . . . . . 144

Summary . . . . . . . . . . . . 147

V. CONCLUSIONS, IMPLICATIONS AND
RECOMMENDATIONS . . . . . . . . . 148

Introduction . . . . . . . . . . 148
Review of Cost-Effectiveness Results . . 149
Interpretation of Results and
Conclusions . . . . . . . . . . 150
Specific Results of the Study . . . . 150
General Conclusions . . . . . . . . 153
Discussion of Findings . . . . . . . 155
Limitations of the Study . . . . . . 158
Suggestions for Future Research . . . . 161
Implications . . . . . . . . . . 162
Final Comments by the Researcher . . . 164
BIBLIOGRAPHY . . . . . . . . . . . . . 167

Page

APPENDICES
Appendix
A. Godmer Letter . . . . . . . . . . 172
B. Tyler Letter . . . . . . . . . . 173
C. Stalford Letter . . . . . . . . . 174
D. Reading Gain Score Distribution
Chart 0 O O I O O I O 0 O O O 175

vi

10.

LIST OF TABLES

Distribution of Reading Programs Across
Grade Levels . . . . . . . .

Statistical Design Employed in Analysis
of Achievement Gain Scores for Second
and Third Grades . . . . . . .

Mean Gain Scores by Cell for Second and
Third Grade Analysis . . . . .

Analysis of Variance on Pre and Post Reading
Achievement Gain Scores for Second and

Third Grade Programs . . . . .

The Gain Score Means for Programs Combined
Across Second and Third Grades for Use

in the Scheffé Post Hoc Procedures .

Scheffé Post Hoc Analyses on Second and
Third Grade Sample . . . . . .

Means for the Analysis of Reading Achieve-
ment Gain Scores for the Seventh Grade

sample I O O O O O O O O 0

Analysis of Variance Table for the Analysis
of Reading Achievement Test Gain Scores

for the Seventh Grade Sample . . .

Scheffé Post Hoc Analyses on Seventh
Grade Sample . . . . . . . .

Statistical Design Employed in Analysis

of Achievement Gain Scores for Eighth
and Ninth Grade Sample . . . . .

vii

Page

134

135

135

136

137

138

139

140

141

142

Table

11.

12.

13.

14.

15.

Page

Mean Gain Scores by Cell for Eighth and

Ninth Grade Sample . . . . . . . . 143
Analysis of Variance Table for the Analysis

of Reading Achievement Test Gain Scores

for the Eighth and Ninth Grade Sample . . 143
The Gain Score Means for Programs Combined

Across Eighth and Ninth Grades for Use

in the Scheffé Post Hoc Procedures . . . 144
Scheffé Post Hoc Analyses on the Eighth

and Ninth Grade Sample . . . . . . . 145
Annual Cost per One-Tenth Student

Achievement Gain . . . . . . . . . 146

viii

LIST OF CHARTS

Chart Page

I. Variable Cost Data--Alpha II Learning
Systems . . . . . . . . . . . . 98

II. Variable Cost Data--Westinghouse Learning
Corporation. . . . . . . . . . . 104

III. Variable Cost Data--Combined Motivation

and Education Systems . . . . . . . 109
IV. Variable Cost Data--Project Read . . . . 117
V. Traditional Remedial Reading Time Factor
Relationships . . . . . . . . . . 119
VI. Variable Cost Data--Traditional Remedial
Reading I O O O I O O O O I O O 123
VII. Variable Cost Data--Control Group . . . . 132

ix

CHAPTER I

INTRODUCTION

Man, being a social creature, has devised the
"community" as a social locality or environment in which
to live. Communities depend upon "public institutions"
for adhesiveness. Broadly defined, a public institution
is an established organization dedicated to a public
service. A public institution can be thought of as a
servant administering unto its master--the community.

The viability of the institution rests largely upon the
fiscal support of the community and this is significantly
catalyZed by the community‘s confidence caused by the
institution, indeed, realistically rendering a measureably
satisfactory service. If this confidence waivers,
assuredly one can expect fiscal deterioration of the
institution. Because the community and the institution
are so interdependent, the health of one is rather con-
tingent on the health of the other.

Public institutions have a fundamental and legal
obligation to be "accountable" to society. Formidable

pressures brought forth by the sixth and seventh decades

of the twentieth century vests "accountability" with
added meaning not only in the jargon of institutional
vocabulary, but is increasingly stressed within corporate
philosophies throughout the entire expanse of free
enterprise. Perhaps the emphasis on "accountability"
is precipitated by the economic effects of higher tax
burdens, rampant inflation and, now, a rather adamant
demand by the public that they "want their money's worth."
This cry for "our money's worth" is evidenced by
a lack of local support and understanding of education
in many Michigan communities as evidenced in a May 5,

1968, The Grand Rapids Press story which reported a

 

Michigan Educational Association observation. Of 120
school-operating millages voted thus far in Michigan in
1968, only 61 passed. George Taylor, Associated Press
Business writer, in a national survey, reported in The
Grand Rapids Press on April 23, 1968, that there are
signs all across the land that the American taxpayer is
becoming increasingly unwilling to vote "yes" on bond
issues and school budgets which could result in higher
taxes for him. He used as example the states of New
York, Oregon, Florida, California, and Kansas where
"voter resistance certainly is hardening in terms of
increasing number of rejections on bond and millage
issues." And even more recently John P. Corr, education

writer for The Philadelphia Inqgirer, wrote in a

 

December 3, 1970, story that the Pennsylvania State Board

of Education must find a way to make educators accountable

for the progress of their students and that a drastic

overhaul of the school financing system should be based

on a formula using definite educational goals and pro-

grams as a grant basis.

Mr.

Raymond L. Godmer,

School Management Services

Consultant, Michigan Department of Education, cites the

following statistics on voter reaction

ational millage and bond issues in the

in a letter to this writer dated March

1968-69
1969-70

1970-71 (through Jan.)

1968-69
1969-70
1970-71

Operational Issues

 

(through Jan.)

598 total 366
656 total 458
73 total 44

Bond Issues

 

122 total 41
112 total 42
48 total 19

to recent oper-

state of Michigan

12, 1971:

passed 232 defeated
passed 198 defeated
passed 29 defeated

passed 81 defeated
passed 70 defeated
passed 29 defeated

Even more recently, Dr. Norman P. Weinheimer,

Executive Director of the Michigan School Board Associ-

ation, issued a prepared statement of voting statistics

for that organization's monthly publication which tabu-

lated the June 14, 1971, Michigan millage election

results.

Dr.

Weinheimer's tabulations read as follows:

 

 

Operating Millages Additional Millages
Passed 82 districts Passed 43 districts
Defeated 19 districts Defeated 7 districts

Bond and "Building and Site"

 

Passed 4 districts
Defeated 7 districts
In analyzing these statistics, Dr. Weinheimer
concluded that one child out of four in Michigan faces
educational programs varying from no educational program
at all to seriously curtailed programs in the coming
school year.1
Most educators and responsible citizens are

hopeful that many of the crucial social issues can be
resolved through quality education. Accountability is
becoming a useful concept to revitalize the public's
faith in public school education. Dr. Leon Lessinger,
former commissioner for elementary and secondary edu-
cation in the United States Office of Education, now
Professor of Urban Education at Georgia State University,
defines the concept in this manner:

Accountability is the product of a process. At its

most basic level, it means that an agent, public or

private, entering into a contractural agreement to

perform a service will be answerable for performing

according to agreed-upon terms, within an established

time period, and with a stipulated use of resources

and performance standards. This definition of
accountability requires that the parties to the

 

1Norman P. Weinheimer, "From the Desk of the
Executive Director," Michigan School Board Journal (July,
1971), 10. ,

 

contract keep clear and complete records and that
this information be available for outside review.
It also suggests penalties and rewards, accounta-
bility without redress or incentive is mere
rhetoric!2

Mr. Robert J. Gravue enlarges upon this statement
when he says:

The concept of educational accountability is con-
cerned basically with techniques to guarantee a
certain level of student performance relative to
stated objectives and goals with an accompanying
efficient use of resources. The educational enter-
prise will face growing competition for resources,
with ecological, health and continuing national
defense demands among others being highlighted.

A criterion of effective fiscal support for public
education will be the degree of faith that the
citizenry has in the ability of school systems to
"give the people their money's worth."3

Lessinger, in a presentation at the Institute on
Independent Educational Accomplishment Auditing, November,
1969 stated:

. . . widespread disillusionment with the results
of American Education has generated intense and even
desperate efforts at school reform. . . .

Our schools must assume a revised commitment--
that every child shall learn. Such a commitment
includes the willingness to change a system which
does not work, and find one which does; to seek
causes of failure in the system and its personnel

 

2Leon Lessinger, "Engineering Accountability for
Results in Public Education," Phi Delta Kappan, LII, No. 4
(December, 1970), 127.

 

3Robert J. Gravue, "Accountability: Comments and
Questions," Educational Technology, XI, No. 1 (January,
1971), 34.

 

instead of focusing solely on students; in short,
to hold the school accountable for results in
terms of student learning rather than solely in
the use of resources.

Exactly two months later, the same idea was
reframed more specifically by Daniel P. Moynihan, Coun-
selor to the President:

For generations Americans were taught that the
best measures of how a school would perform were
inputs such as teachers' salaries or class size.
It 15 now definitely established that this is not
so. The relationship between such inputs and edu-
cational achievement is weak and uncertain, even
erratic.

Therefore, do what Coleman did: measure inputs.
If educational achievement, defined as reading
scores or whatever, is the desired output, measure
that. Moreover, measure it in such a way that the
public can have some idea of what it is getting
for its money, how its children are doing compared
to other children, and so forth.5

Although the concept of accountability in edu-
cation is not new, these statements and the call for
accountability from parents and legislators have resulted
in a new push in education; as Don Davies predicted in
December, 1969, "Accountability will soon replace rele-

. I 6
vance as 1n the '1n' word among educators."

 

4Leon M. Lessinger, "The General Concept of Inde-
pendent Accomplishment Auditing" (presented at the Insti-
tute on Independent Education Accomplishment Auditing,
Hotel Newporter, NeWport Beach, California, November 10-
12, 17-19, 1969). PP. 3-4.

5Daniel P. Moynihan, "To Solve a Problem, First
Define It," New York Times, January 12, 1970, p. 62.

 

6Don Davies, "The 'Relevance' of Accountability,"
The Journal of Teacher Education, XXI (Spring, 1970), 127.

 

President Nixon proposed in his January, 1970,
nationally televised veto of the $19.7 billion education
and labor appropriation bill, that he would prOpose "a
new and searching look at our American school system"
because, in terms of the money and efforts being put into
education in the United States, "we're entitled to get
more out in terms of better quality education."7

The President said in his Special Message to
Congress on Educational Reform, "We must stop congratu-
lating ourselves for Spending nearly as much money on
education as does the entire rest of the world--$65
billion a year on all levels--when we are not getting
as much as we should out of the dollars we spend."8 In
this March 3, 1970, message to Congress, the President
said:

. . . we derive another new concept: accountability.
School administrators and school teachers alike are
responsible for their performance, and it is in their
interest as well as in the interests of their pupils
that they be held accountable. Success should be
measured . . . by the results achieved in relation-

ship to the actual situation of the particular
school and the particular set of pupils.9

 

7"Transcript of the Nixon Speech," New York
Times, January 27, 1970, p. 24.

8"Excerpts from the President's Special Message
to Congress on Educational Reform," New York Times,
March 4, 1970, p. 28.

 

91bid.

Mr. Nixon reiterated the central thesis of the
"Coleman Report," The Equal Educational Opportunity Survey
of 1966, published by James Coleman of John Hopkins Uni-
versity when he said that the input-side of education--
more money for better pupil-teacher ratios, better build-
ings, better equipment-emay not make any substantial

difference."10

Accordingly, as with Dr. Coleman,
accountability in terms of performance has come to the
fore: has Don Davies' prediction been realized?

Donald Rumsfeld, former director of the Office
of Economic Opportunity, in a Speech titled "Experiments
in Education" and delivered in San Francisco on September
23, 1970, illustrates the public's concern for accounta-
bility in education when he read an open letter from
Washington, D.C. parents to the teachers' union which
was blocking implementation of the Clark Plan in the
District of Columbia's schools. The Clark Plan would
have provided a concentrated effort to improve reading
and math levels of students. It read:

AN OPEN LETTER TO THE TEACHERS' UNION FROM ANGRY
CITIZENS . . . We've supported you for pay raises.
We've supported you for better teaching conditions.
We've gone to bat for you before the School Board.
We've listened to both sides. We've been patient.
We've been understanding of your problems. We've

gone along with you. . . . Now it's time for you
to listen to US. And you better believe it.

 

10Robert B. Semple, Jr., "Nixon Proposes Re-

examination of Aid to Schools," New York Times, March 4,
1970, p. l.

 

We've had it up to here with your bickering and
squabbling with our School Board. We've had it up
to here with threats to torpedo a plan that has
such high promise of giving our kids the kind of
education they need--and haven't been getting-~that
the School Board voted for nine to one. We've had
it with a headline--grabbing union leader trying to
usurp the policy-making functions of our School
Board--a Board WE elected, by the way. We've had
it with being told that 250 odd members of your
union represent ALL of you, or even a majority.
We've had it with defiance by a few as an excuse
for damaging and maybe destroying our children's
chance for decent education and meaningful lives. . . .

About the Clark Plan. We elected a School Board
to make the decisions and set the policies. . .
Along came the recommendation to implement the Clark
Plan and the Board voted for it. That' 5 good enough
for us . . . isn't it good enough for YOU?

Let's get it straight. We've had it with seeing
our children's education washed down the drain in a
torrent of words. They're OUR children and we KNOW
what the school system has done for them so far.

And it's not enough. As far as we're concerned, the
Clark Plan is the best answer today, here and now,
for giving our children the education they need.

As far as we're concerned, the Clark Plan is IT.

And we expect you to get with it.11

Rumsfeld further states that the Office of Economic
Opportunity is in business to do something about poverty.
There can be no question about the link between poverty
and education. Nor can there be any question about the
inadequacy of existing efforts from the standpoint of the

12
poor.

Rumsfeld also said that if the O.E.O. has to step

on some toes of the educational institutions to get it

 

11Donald Rumsfeld, "Experiments in Education"
(Speech delivered in San Francisco, California, Septem-
ber 23, 1970), p. 14.

lzIbid.

10

solved, then so be it. If these educational institutions
have a knee jerk reaction and are too rigid to accept
criticism or change, then something has to be done to
help bring about the change. The school people's oppo-
sition makes them very much like most protectors of any
special interest. Teachers have a vested interest that
they are protecting rather than acting as an ethical
influence for the people that they theoretically are

to serve. They may view their job as to preserve the

status quo, but it is my job to help poor children.13

Mr. Rumsfeld continued his discourse on the
present failures of public school education and the tax-
payers' reluctance to continue supporting non-performance
producing educational programs when he said:

We have been told repeatedly by some that the
secret to a productive educational system lies in
the number of dollars we expend.

We have been told that shortcomings can be
corrected merely through the expenditure of addi-
tional funds. Such contentions have to be based
on assumptions:

--That reading levels are directly related to

teacher-pupil ratios.

--That increased per pupil expenditures affect
math achievement.

--That the quality of education is somehow
related to the attractiveness of the physical
plant.

--That what counts is the "quality of the school,"
not the quality of educational experience of
the individual child.

 

13Ibid., p. 15.

11

It is painfully clear that the massive influx
of more dollars, more classrooms, more teachers, and
more compensatory programs has fallen short of the
desired impact.

The poor have grown cynical-~they are fed up with
promises that are not kept, results which have not
materialized.

The non-poor--the taxpayers--have grown tired of
being told that every problem can be resolved if we
only spend more money--they are tired of paying for
programs which fail to produce the promised results.

The Office of Economic Opportunity is concerned
with the problems of the educational system serving
the poor. It has launched an experiment which will
Show potential for improving educational oppor-
tunities. It will test the concept of performance
contracting.

Under performance contracting, a school district
contracts with a specific group--a private firm
or its teachers' association, for example--to provide
instruction to children in the School district who
are not achieving at grade level norms. Contractors
are paid only to the extent that they are successful
in improving the educational skills of the children
they instruct. In the experiment, the contractors
will be paid only if they increase skills by more
than one grade level. They will make a profit only
if skills increase by 1.6 grade levels, nearly four
times the average now attained in schools serving
poor neighborhoods.

Within broad policy guidelines established by the
school board, the contractors are free to use a
variety of approaches to achieve their results. Some
are using computers and teaching machines. Others
are offering incentives to teachers and even to the
students themselves. But from the standpoint of
potential educational reform, the most significant
feature of this concept is accountability: no
success, no pay.

The experiment is under way in twenty-one school
districts. In each of the 18 districts one of Six
firms is under a performance based contract to teach
math and reading to a sample of youngsters in grades 1,
2, and 3 and 7, 8, and 9. In the additional three
districts, the teacher's organization or the school
district itself will be the contractor. Nearly 30,000
students will be a part of this experiment with approx-
imately 12,000 receiving the experimental services.
An independent, third-party organization is carrying
out the evaluation of the performance of the private

12

firms and a follow-up study of the students will
enable us to determine the impact of improvements
of basic skills on subsequent school performance.14
Performance contracting then, is one process for
which accountability may be an end product. This concept
is not new to public education. School managers (super-
intendents and business managers) have operated school
feeding, transportation, and maintenance programs on
performance contracts for years. It is the extension
of this idea into the realm of learning through a par-
ticular process that Lessinger identifies as performance
contracting.
AS defined by Lessinger, performance contracting
is an "educational engineering" process
. . . whereby a school contracts with private firms,
chosen competitively, to remove educational defi-
ciencies on a guaranteed performance basis or suffer
penalties. Without being told what program is to
be used, the contractor is encouraged to innovate
in a responsible manner. Upon successful demon-
stration, the contractor's program is adapted by
the school on a turnkey basis.15
One of the benefits of a performance contract to

a Local Education Authority is relating learning to cost.

Lessinger calls performance contracting the legal device

 

14Ibid.. pp. 4-6.

15Leon Lessinger, "White House Memo Defining Per-

formance Contracting," December, 1969.

13

upon which educational engineering or public
accountability depends.16

Dr. Raymond L. Boozer in his recent study con-
cluded that citizens are most likely to vote favorably
in future millage elections if the money appropriated is
used for special education programs, adult remedial and
trade programs, more individual instruction, smaller
classes for children, an extended use of school facili-
ties for all citizens, and for the construction of new
building facilities.l7

Therefore, the individual performance contracting
programs may be a device to render the accountability
upon which the voting public judges the effectiveness of
a school program.

Michigan's Governor William Milliken in his

State of the State message as reported in The Detroit

 

Free Press, January 15, 1971, urged state support for

 

performance contracting in the schools when he said per-
formance contracting experimentally under way in the

state introduces business management methods and profit

 

l6Leon Lessinger, Every Kid a Winner, Accounta-

bility in Education (New York: Simon and Schuster, 1970),
p.235.

 

 

17Raymond L. Boozer, "A Study of the Voting Publics
in Grand Rapids, Michigan, to Provide the Basis for Plan-
ning and Conducting Future Public School Operating Millage
Elections in the District" (unpublished Ph.D. thesis,
Michigan State University, 1969), p. 162.

14

motive into the schools with the private contractor
guaranteeing pupil progress as a condition of receiving
payment.

Dr. John W. Porter, Michigan State School Chief,
cited twenty-five major state education issues to be
resolved in the coming year. Among these was "Accounta-
bility and independent educational audits." Dr. Porter
and the Michigan State Board of Education also identified
and supported "performance contracting as a promising

educational practice."

Nature of the Problem

 

The challenge of public accountability through
performance contracting demands proof of results. The
results must be based upon a per-pupil cost of a learning
unit or grade gain skills acquired rather than solely on
the cost of maintaining children in school.

This is the nature of the problem. School sys-
tems identify school cost on the basis of how much they
spend for instruction; administration, attendance and
health; transportation; operation and maintenance;
capital outlay; fixed charges; service, transfers, and
18

contingency.

Lessinger states that:

 

18"The Board of Education of the City of Grand
Rapids, Michigan; Final Budget, 1970-71."

15

Our public elementary and secondary schools enroll

44 million students, employ 1.9 million teachers,

and Spend over $30 billion in tax funds annually.

We have all kinds of measurements of where that
money goes: we can pin down per—capita expenditures
in any school district in the country, state how
much any of them Spend for construction and interest
payments on borrowed money, and enumerate pupil-
teacher ratios until the sun goes down. But all

of these figures, useful as they are for some pur-
poses, refer exclusively to financial inputs rather
than educational outputs. We have virtually no
measurement of the results that our vast enterprise
yields. For example, we do not know what the average
cost of increasing a youngster's reading ability by
one year is: all we know is what it costs to keep
him seated for one year with a textbook and a teacher.
Many of the available indexes measure our competence
as financial managers; but how many of them evaluate
our effectiveness as educational managers? It would
make much more sense if we moved from the concept of
per-pupil cost to the concept of learning-unit cost,
and focused on the cost of Skill acquisition rather 19
than on the cost of maintaining children in schools.

Byron K. Love in his suggested Cost of Education
Model for K-12 Michigan districts, concludes that those
responsible for the educational program should be able to
set objectives, plan to fulfill those objectives by means
of a curriculum and instruction based upon it, budget
for what is planned and account for what happens. There
should be a complete set of information for each program.
This information should include the objectives of the
course, who taught it, who attended the class, the
nature of the instruction, materials used, location,
time scheduled, amount of money budgeted for it, and

the cost. With this set of information the program can

 

19Lessinger, op, cit., p. 11.

16

be meaningfully evaluated both educationally-—in terms
of how well it met the objectives (outputs)--and in terms

20 Love's

of resources (inputs) needed to implement.
final conclusion, then, is that better decisions regard-
ing program selection and Operation result when production
(output) is methodically related to objectives.

According to newly developed criteria, when
resources are limited, the setting of priorities is
essential. To provide a sound basis for establishing
such priorities, sets of information relating to the
programs are essential. This information is necessary
to calculate the required cost of performing instruction
with specified designs. A program could be educationally
effective but its high cost may render it indefensible.
However, with appropriate sets of information available,
a school board or school administrator can weigh the evi-
dence for continuing, modifying, or discontinuing a pro-
gram.

One of the most difficult, and often the most
neglected processes in school system management,
is communication upward, downward, and laterally
in the organization with the central purpose of
improving quality. With "programs" identified,

planned through staff involvement, budgeted for
with an understanding of what is economically

 

20Byron K. Love, "The Development of a Suggested

Instructional Programs, Cost Model for K-12 Districts of
the State of Michigan" (unpublished Ph.D. thesis, Michigan
State University, 1968).

l7

feasible, and accounted for in terms of all
resources committed to the objectives, there
is something to communicate about.21

To date, public information is exceedingly
limited on actual costs for performance contracts which
have been awarded recently or are pending. From current
literature, however, it appears that most of the districts
which have implemented or are planning to implement per-
formance contracting have utilized ESEA Title VIII (Ele-
mentary-Secondary Educational Act--Dropout Prevention)
funds. Because no additional monies are available under
Title VIII, many school districts have turned to Title I
(Compensatory Education) with the exception of the CEO
Remedial Performance Incentive Project.

At present, all performance contracting is done
in addition to costs of the present educational program,
and full state-aid expenditure per pupil is maintained
throughout the project; whether there may be a saving
in the future from changed staffing patterns is yet to
be determined. The awarding of performance contracts
on open bidding procedures may present the possibility
of real savings to the taxpayer. It is suggested by
many that performance contracting should not impose

further financial burden on the local taxpayer, unless

 

21"Program Planning, Budgeting and Accounting in
School System Operation" (a position paper by Allan R.
Lichtenberger, Chief, Terminology Compatibility Branch,
U.S. Office of Education, September 20, 1967), p. 4.

18

the private corporation can perform better than the
public schools have in the past. The taxpayer should
have the privilege of deciding whether the cost-benefits
of performance contracting over other more traditional
educational proqrams warrant such an arrangement.

The one basic problem of education today is that
a complete set of factual information for each performance
contracting program, a publisher's package program
("canned"), or traditional school programs defined in
terms of grade gains or learning units and relative costs

is not currently available in the public schools.

Purpose

The purpose of this study is to develop a cost-
effectiveness model which may identify and isolate the
most important factors contributing to the efficiency
with which resources are expended for instructional
activities in performance contracting programs, pub-
lisher's packaged programs, and traditional school pro-
grams. This model will be designed to determine the

following information:

A. Program identification including the name of
course, who attended it, course objectives,
design (methods and materials) and content,
length of time, type of school building and

classroom, and costs.

19

B. Grade gains or learning unit gains of students

enrolled in the courses.

C. Identification of costs per unit of student

achievement.

Dr. Elmer H. Vruggink in his recent study
recommends this type of study when he states that if
an administrator is to make decisions on the future of
programs in a local school system, he must have a variety

of information in the following areas:

1. Cost

2. Achievement and/or IQ as measured by standard
tests

3. Changing practices such as differentiated
staffing

4. Transiency
5. Curriculum improvements or teaching techniques
6. Parent involvement
7. Staff involvement
8. Teacher characteristics
9. Adult-pupil ratio
10. Teacher and administrator turnover
11. Student racial characteristics
12. Decentralization22

This research does not purport to be an in-depth
study of all these areas. Chapter III will review those
areas which will be considered as one pattern for a pos-

sible cost factor definition.

 

22Elmer H. Vruggink, "A Study of the Contribution

of Compensatory Programs in a Large Urban School System"
(unpublished Ph.D. thesis, Michigan State University,
1970) I p. 9.

20

Importance of the Study

 

The information in this study may provide the
local board of education and school administrators with
data to determine the degree of efficiency in which
resources are expended for instruction, the relationship
of factors contributing to the use of these resources,
and a basis for decision making regarding the future of
the programs studied.

The educational enterprise at present has two
key problems: a lack of sufficient resources and perti-
nent information for making decisions. Those who control
the resources, the taxpayers, show great reluctance to
increase the level of support of school programs proportion
to the rate of increasing costs and demands for service
without positive assurance of educational productivity.

The taxpayer who is demanding accountability,
perhaps has become knowledgeable not only in terms of
fiscal Spending but fiscal spending related to edu-
cational results.

According to Donald Rumsfeld, former director of
the Office of Economic Opportunity and President Nixon's
Cabinet member, the results of this information could
revolutionize public school education. He said, "his-
torically we have measured our concern by asking how

many millions we spend, how many textbooks we buy, how

21

many schools we build. But these are all inputs. They
don't tell us anything about the impact on human lives."23

If the present patterns of resources allocation
can be changed to report a relationship between costs
and benefits, and if school people allocate these

resources based on facts, public confidence can be

restored. The success of the student will tell the story.

Limitations and Assumptions

 

This study will be conducted within the school
district of Grand Rapids, Michigan, although background
will be drawn from other performance contracting projects
that currently exist throughout the schools of the nation.
Therefore, the applicability of the conclusions and
recommendations of this study to other districts may
be assessed in this light.

Secondly, although some reference will be made
to mathematics programs, all conclusions will be drawn
from the subject area of "reading."

Thirdly, the present practice of educational pro-
gram accounting is very broad and inclusive. For purposes
of this study, it will be broken down into five areas:

(a) instructional program, (b) personnel, (c) pupils,

(d) finances, and (e) facilities. The total information

 

23Jack Rosenthal, New York Times story, May 15,

 

1970.

22

system of each of these subsystems may provide a begin-
ning basis for answering the following types of questions
as recommended by Ronald Randall, Educational Turnkey
Systems, Inc.

1. How sensitive are summary costs to changes in
particular cost factors?

2. How sensitive are summary costs to changes in
the costs of resources consumed?

3. What constitutes the minimal description of an
educational medium or a method for conducting
instructional activities needed for economic
analysis?

4. What are economic trade-offs between constituent
resources consumed in instruction, subject to
overall budgetary limits?

5. What are the opportunity costs of certain
alternative management actions not taken?
Fourthly, although various aspects of the per-

formance contracting concept will be analyzed in greater
detail than the other more traditional programs studied
because it is a rather new concept in education. The
main emphasis of this study, however, will be in develop-
ing recommendations that will make available resources
more effective in terms of student performance.

A final assumption that is made is that performance
contracting may be a change strategy to foster greater
accountability which may benefit the students through
improved programs; benefit the teacher by providing a

different role in education; benefit school administrators

by providing solutions to financial and social crisis

 

24Ronald Randall, "The Cost-Ed Model," Educational
Turnkey Systems, Inc., October, 1970.

23

within the system and benefit the community by providing
sufficient data to assess the school program and further

to renew its efforts to support such programs.

Definition of Terms

 

The following terms are used repeatedly in the
study and have rather Specific definitions as they relate

to this study:

Behavior Modification.--The act of changing stu-

 

dent behavior through the use of social and non-social

reinforcers.

Comprehensive Evaluation.--The establishment of

 

systems of performance control based on the continuous
assessment of program operational and management process

and resultant products.

Contract.--A legal document between the per-

formance contractor and school system.

Contractor.--Educational technology firm selling

 

an educational or instructional service.

Cost—Effectiveness.--An analysis of unit learning

 

results obtained in relation to unit resources consumed
under alternative approaches to proqram operation, as a

determinant in continued program planning.

24

Hardware.-—Machinery such as tape recorders and

learning machines.

Learning_Center.--A performance contracting

 

project's student learning center room.

Line Item Budgeting.--Budgeting by specific

 

account items, i.e., administrative salaries, trans-

portation, etc.

Management Systems.--An adaption of systems

 

approach through such techniques as (MBO) Management by
Objectives or (PPBS) Program Planning and Budgeting Sys-

tems.

Performance Contract.--A procedure whereby private

 

industry contracts with a school district to guarantee a
certain amount of student achievement for a set amount of

money.

Performance Objectives.--The specifications of

 

program objectives in a comprehensive precise manner
which indicates measures and means for assessing the

degree of attainment of predetermined standards.

Reinforcing Events Area.--A classroom designed as

 

a "free" room where students can indulge in non-academic
events and fun such as pool playing, magazine reading,

etc. as a part of the reward and incentive system.

25

Software.-—Consumable materials such as programmed

instruction workbooks.

Student Incentives.—-Incentives given to stu-

 

dents for appropriate behavior or for completion of work.
In some programs the incentives are tied to achievement.
Incentives are tokens, points, praise, teacher attention,
stars, grades, etc. This can be exchanged for time in
the "free" room or other tangible rewards such as a

hamburger or taco.

Teacher Incentives.--Monetary incentives paid

 

to staff members based on student performance.

Turnkey.—-A process of turning a proven operation

over to a local school system.

Summary

Preliminary examination would indicate that a
need exists for more accountability by school system
decision-makers as to what the cost-effectiveness of
programs are. This study will attempt a design and
analysis of general educational reading programs using

a cost—effectiveness design.

CHAPTER II

SELECTED REVIEW OF LITERATURE

Review of Literature

 

School systems are being held more accountable
by the electorate while this same electorate is reluc-
tantly providing resources for the schools. Defeated
local bond issues may be a prime indicator of general
dissatisfaction with the schools. The taxpayer is ask-
ing "What am I getting for my money?" Traditionally,
the schools have answered in terms of the kinds of
facilities that exist, qualifications of the personnel
that are employed, and the size of the expenditures. The
public does not seem to accept this response in the
1970's. Today it appears that they want to know if
the students can read and compute, secure and hold a
job. This creates a new kind of accountability.

The direction of the accountability movement
centers around the concern to find ways to relate
dollars to outputs, i.e., the cost of a unit of edu-
cation of known quality and quantity in terms of

dollars expended. The traditional method of assessing

26

27

educational cost has been primarily based on the estab-
lishment of the relationship between dollars and edu-
cational input. These inputs have included such cost
factors as the number of students served per dollar
spent, the number of teachers and their dollar cost,
the dollar cost of buildings, and number of courses pro-
vided and their dollar cost. Most school systems can
provide this information easily with a little computation.
But what about production? What are we producing with
the measured inputs? Thus, the taxpayer is presenting
a demand to measure the quality of educational output
which he calls "accountability."

Accountability as defined by Lessinger is "doing
what you said you would do" or "proof of results."25
The basic idea of accountability in education is, accord-
ing to Stephen M. Barro, Rand Corporation economist,
"that school systems and schools, or more precisely,
the professional educators who operate them, should be
held responsible for educational outcomes for what

children learn."26

 

25Leon Lessinger, "Robbing Dr. Peter to Pay Paul:
Accounting for Our Stewardship of Public Education," Edu-
cational Technology, January, 1971, p. 13.

 

26Stephen M. Barro, "An Approach to Developing
Accountability Measures for the Public Schools," Phi
Delta Kappan, December, 1970, pp. 196-205.

 

28

This concept has been attacked by many educators,
including Mrs. Helen Bain, president of the NEA, who has
stated:

It is pure myth that classroom teachers can ever
be held accountable, with justice, under existing
conditions. The classroom teacher has either too
little control or no control over the factors
which might render accountability either feasible
or fa1r. 7

Robert Kane, Evaluation Consultant for the Cali-
fornia State Department of Education, disagrees with
Mrs. Bain when he suggests that the question of "fair-
ness" is academic; educators have always been held
accountable and the real question is whether we are
to continue to operate under a "One-Way Street Theory
of Accountability," or under a "Two-Way Street Theory of
Accountability."28

These two theories of accountability each require
that program evaluation be performed. The first, the
"One-Way Street Theory" requires evaluation for survival,
and encourages "evaluation for justification." Kane
agrees that the "Two-Way Street Theory" requires evalu-

ation for "justice" and encourages professionalism in

education. It also encourages the development and

 

27Helen Bain, "Self Governance Must Come First,
Then Accountability," Phi Delta Kappan, April, 1970,
p. 413.

 

28Robert M. Kane, "A System Approach, Accounta-
bility With Justice" (a report prepared for the California
State Department of Education, 1970).

29

conduct of programs designed to the greatest student
benefit, and not those designed to make schools look
good.

Kane, argues, that the public is not going to
continue to support programs which can only report "no
statistical difference." The public is no longer inter-
ested in programs that are based on data that says "the
teachers were enthusiastic, the kids enjoyed themselves,
and the parents liked the program." The public wants
hard data: student achievement related to cost. The
public will support programs which demonstrate student
learning at the same or less cost than programs which
do not demonstrate student learning."29

Kane cited the Gallup Second Annual Survey of
the Public's Attitude Toward the Public Schools as part
of his defense. He commented that this national survey
conducted by George Gallup indicates that a majority of
citizens have been quite willing to take the word of the
school board, the teachers, and administrators that the
schools are doing a good job. The community has looked

with pride on its school buildings and its winning

 

29Robert F. Kane, "Evaluation: What, Why, How"

(paper presented at the California Audio Visual Education
Association, Annual Conference, Fresno, California,
January 29 and 30, 1971), p. 5.

30

football teams. However, Gallup's Second Annual Survey
revealed that this method of judgment is in for a change.30

Gallup's survey revealed that most people would
like to know more objective data on student achievement.
In fact, it was reported in this study that a majority
of the persons interviewed would like to have national
tests administered in the local schools to measure stu-
dent progress and achievement and further to relate this
achievement to local costs as well as national costs and
national levels of achievement.31

Gallup concluded by stating that if schools hope
to avoid financial difficulties in the years ahead, they
have to respond to the public's demand for accountability
by informing the public of what they are trying to do,
if they have succeeded in doing it, and how much it
costs.32

To become educationally accountable, as apparently
is being currently mandated by the electorate, school

people must establish an objective basis for resource

allocation. A new rationality must be brought into the

 

3OIbid.

31George Gallup, "The Second Annual Survey of the
Public's Attitudes Toward the Public Schools" (a report
prepared by Gallup International, Princeton, N.J., Octo-
ber, 1970), pp. 1—4.

32Ibid., p. 4.

31

decision-making process. Educators do not currently
know how to accomplish this on an objective basis
because as Merlin G. Duncan, University of Alabama
professor of educational administration succinctly

says, "We have no real, valid, objective measures of

our outputs as they relate to our inputs."33

To facilitate educational program accountability,
Duncan says that it is imperative that school people pro-
vide some measure of the quality of educational outputs

and then relate these outputs to opportunity costs and

dollar costs per unit of output.34

Duncan recommends a system of accomplishing
educational program accountability as including the
following:

1. It should measure program effectiveness based
on stated real goal accomplishment in a time
frame.

2. It should report results on a multi-dimensional
format to the interested publics of the edu-
cational enterprise, both internal and external.

3. It should be a dynamic process that makes the
educational system more responsive to the needs
of society and its own clientele.

4. It should be related to comprehensive educational
planning and show that the programs generated
are economical in terms of opportunity costs.

5. The system by which accountability is satisfied
should also be flexible enough to provide input
to regenerate the system through constant evalu-
ation and feedback which serves as a guide to
program formulation, revision or termination.

 

33Merlin G. Duncan, "An Assessment of Accounta-
bility, The State of the Art," Educational Technology,
January, 1971, pp. 27-30.

 

34Ibid., p. 28.

32

6. It Should relate measurable educational goals,
to societal goals, and demonstrate the ability
to interface educational systems with other
public and private systems serving society.

The system suggested by Duncan demands an end to
the currently used line item--object of expenditure,
non-program related budgeting and reporting system. The
current system is the kind of budget process that serves
the purpose of administrators who are unable to make
difficult program decisions, thereby protecting useless
programs and activities, while questions of quality and
cost cannot be answered using this method. "The questions
can be answered," says Burton D. Friedman, Maryland State
Department of Education Project Director, "if the move-
ment to accountability is professionally based and
serves to improve the quality of education and the
effective utilization of scarce resources."36

L. Linton Deck, Jr., Assistant Superintendent,
Metropolitan Public Schools of Nashville, Tennessee
states:

If educational program accountability is to be

realistic, the criteria by which the schools will

be assessed should be based on a systems approach.

That is, school systems must systematically analyze
the variability of their input, systematically

 

3SIbid.

 

36Burton D. Friedman, quoted by Duncan, Ibid.,

p. 29.

33

develop operations to produce output in terms of
clearly understood objectives and assess output
for feedback to adjust the system.37
To systematically analyze inputs and relate them
to outputs, it appears that school systems must design
an evaluation system to provide the necessary information
to its decision-makers to insure that wise educational
decisions and resource allocations are made on the basis
of viable documented data. Stanley Krugar, USOE staff
member says, "such evaluation systems require that edu-
cational decisions be a result of a systematic exami-
nation of needs, objectives and the allocation or reallo-
cation of resources based upon some ordering of priori-
ties."38
If a systems approach recommended by Deck is
utilized, it can also be a means of designing an effec-
tive evaluation system. In fact, if Roger Kaufman's
(Professor of Education and Communication, Graduate
School of Human Behavior, U.S. International University,

San Diego, California) definition of a system as being

"the sum total of parts working independently and

 

37L. Linton Deck, Jr., "Accountability and the
Organizational Properties of Schools," Educational Tech-
nology, January, 1971, pp. 36-37.

 

38Statement expressed by Stanley Kruger, Michigan
Association of State and Federal Program Supervisors Con-
ference, Holiday Inn/North, Grand Rapids, Michigan, Feb-
ruary 10, 1971.

34

39

together to achieve a required outcome" is accepted, it

would seem that a complete educational program could not
be designed unless a system or process of evaluation be
an integral part of such a program.

James D. Finn has listed the main or basic pur-
poses of evaluation as:

1. Provide the implementors of a program with
information that will allow for needed modifi-
cations in program design to increase the
probability of the program achieving its
objectives.

2. Provide data to decision-makers regarding the
continuation of termination of programs.

3. Provide information as to the desirability of
adoption and/or expansion of the program beyond
its original sc0pe. As such, evaluation is a
basis prerequisite for effective program imple-
mentation and for dissemination and diffusion
of results of educational programs.40
Kaufman has expanded Finn's definition by design-

ing a six-step problem-solving model as a guideline for

the design of evaluation systems. This system incor-
porates the six steps of Problem Identification, Solution
Determination, Strategy Selection, Implementation, Determi-

nation of Performance Effectiveness and Revision.

 

39Roger A. Kaufman, "System Approaches to Edu-
cation: Discussion and Attempted Integration on Social
and Technological Change Implications for Education"
(report prepared for The Center for the Advanced Study
of Educational Administration, University of Oregon,
Eugene, Oregon, 1970), p. 135.

40James D. Finn, "Institutionalization of Edu-
cation," Educational Technology, December, 1969, p. 16.

41Roger A. Kaufman, "Accountability, A System
Approach and the Qualitative Improvement of Education--
An Attempted Integration," Educational Technology,
January, 1971, pp. 21-23.

 

35

This process model may have basic utility for
the design and accomplishment of effective and efficient
education. It suggests that a system of management for
defining and achieving educational accountability should
be included.

Another management system currently being con-
sidered by school people to accomplish educational pro-
gram accountability is Program, Planning, Budgeting Sys-
tems. PPBS was first applied in the context of national

defense.42

PPBS provides another means for answering
questions of education relative to "what do I give" and
"what do I get." It identifies the relationships between
product outcomes and costs for various alternative methods
and means.

Dorsey Baynham, editor of Planning of Educational
Change: PPBS, a monograph published by the Center for
Effecting Educational Change, defines PPBS as an intel-
ligent, orderly, and informative method of determining
what is actually being received for educational effort.43

Baynham expands this basic definition to read:

The process also answers to the names of systems
analysis or systems approach, and it represents

 

42E. L. Katzenbach, "Planning, Programming, Budget-
ing Systems: PPBS and Education" (report made to the New
England School Development Council, March, 1968).

43Dorsey Baynham, "PPBS and Several Good Reasons
It Shouldn't Scare You Off," American School Board
Journal (August, 1970), 27.

 

36

a procedure for budgeting school funds in terms
of the educational gains of students rather than
"this many teachers with those many textbooks
plus that many buses" and so on.44

Baynham states that:

True, the process is a bit complex and, given
the public's penchant for instant results, it may
face tough going in education. But it is a natural
for bringing reasonable order to public school
management because of its emphasis on product
(output) rather than on function (input).

PPBS is not simply a matter of implementation.
It's sort of a continuous flow over a multi-year
period that is accompanied by continuous feedback
and evaluation.45

Baynham says that it requires the greatest exper-
tise to link programming, accounting, and reporting,
with personnel management and systems analysis.46

It may be determined in the future that PPBS
could be an economic science having impact on negotiations,
bargaining, and compromise where choices must be made
among alternative approaches and among alternative levels
of a program by balancing needs against resources.

PPBS, thus, may become another strategy for
transferring the school budget from a line item, input
procedure, to a program—orientated, output procedure
with the measure of these inputs to output giving

emphasis to efficiency and initiating performance

budgeting.

 

44Ibid., p. 28. 4SIbid.

46Ibid.

37

Martin Buskin, education editor of Newsday
and president of the Education Writers Association,
says that this new management technique can enable
schoolmen to show taxpayers and the board just what

it costs to run a good reading program, a good high

school, a high-quality library.47

To further the concept of PPBS, R. E. Corrigan
has developed the following assumptions:

ASSUMPTIONS:

l. The school system exists to produce a set of
PRODUCTS--to achieve objectives expressed as
Specific changes in characteristics of LEARNERS.

2. The financial resources available to the school
system are less than equal the demands of the
system.

3. Objectives of a school system can theoretically
be achieved in a multitude of ways (PROGRAMS),
some of which are more effective and/or efficient.

4. Productivity of a schooI system can be enhanced
by organization of activities and services into
PROGRAMS specifically directed toward achieving
carefully defined GOALS.

5. Better decisions regarding program selection
and Operation result when the COSTS thereof are
considered on a long-term basis.

6. Better decisions regarding program selection
and operation result when PRODUCTION (output)
is methodically related to objectives.

 

 

 

These assumptions are an extension of the type
of thinking which has promoted many legislatures and

educators to move toward a Program Budgeting System from

 

47Martin Buskin, "PPBS Means Better Money Manage-
ment," School Management, November, 1969, p. 64.

 

48R. E. Corrigan, "The SAFE Planning, Programming,
Budgeting, Evaluation System: An Approach to Relevance
and Accountability in Education," R. E. Corrigan Associ-
ates, P.O. Box 5089, Anaheim, California, 92804, 1970,
p. 11. .

38

a Line Item Budget System, i.e., the perceived require-
ment to determine cost-effectiveness and cost benefit in
education; the requirement for being able to effectively
communicate to the public and the profession, what the
exact costs are for effectively teaching a child to read;
to do math, or to develop behaviors associated with good
citizenship.

As a further adaptation of PPBS Kane submits
that:

Educational programs must be developed in ways
other than by determining "How many bucks do we
have?"; and "how many programs can we implement
within the amount of bucks available?"; this has
the effect of placing the decision-making power
in the hands of the accountant, not the educator.
What is required are educational programs based
upon realistic performance objectives, derived
from the amount of bucks available, but based
upon high priority needs of learner! If we can
do this, and then evaluate our effectiveness in
producing learning, we will have finally developed
a legitimate basis for going to our publics and
requesting funds; a basis other than "we need more
money for better education." Wouldn't it be nice
if we could really identify how much money, for
how much better education! Wouldn't it be satis—
fying to be able to go to the public and prOpose,
with authority:

"Given Y amount of money, we guarantee that
90% of all youngsters having X characteristics,
will have a demonstrated reading level of 8 years
at the end of the 8th grade."

or

"Given Z amount of money, we guarantee 75% of
all youngsters having X characteristics will have
a demonstrated reading level of 8 years at the
end of the 8th grade."

we would then truly be in the position of being
involved in a "Two-Way Street" of Accountability;
we could then honestly cast the public in the role

39

of decision makers by making them decide "What Rate
Will You Be Willing to Live With And Support?"

I submit that what we have been doing has not

been good enough; There Must Be A Better Way.49

"The Better Way" cited by Kane is then, one of
program cost accounting. This cost data, once determined,
could be useful in planning and decision—making at all
levels of the educational enterprise. The major purposes
of accounting system are: (A) to provide control over
fiscal resources and (B) to provide relevant information
for planning, decision-making, and evaluation of programs.

It appears that program cost accounting, to be
most effective, should serve as one of many subsystems
of a general program accounting system.

Charles S. Benson poses the difficulty of the
problem of preparing a budget for such a system when he
says:

To prepare an ideal educational budget is difficult,
more difficult than the corresponding process in a
private business firm. The objectives of school
systems are multiple, and there is not absolute
agreement among parents, educators, and taxpayers
on the importance--or relative weighting-—of these
different objectives. More distressing, it is very
hard to measure closely the effects of given change
in school practice. It is thus, an imposing task

to choose rationally among alternative means to
accomplish stated ends.50

 

49Kane, op. cit., p. 16.

50Charles S. Benson, The Economics of Public Edu-

cation (Boston: Houghton Mifflin Company, 19617, p. 336.

 

40

However, educators must choose. The public
schools will undoubtedly continue to operate in an
economic system that has a scarcity of resources for
many years to come due to the increasing demand from
all other social and governmental services. The inner-
budget competition for those resources which are available
will increase in intensity as long as there are alternate
uses. The decisions ideally should be made to select
the minimum-cost mix of resources to achieve maximum
educational goals. Benson concludes, " . . . that a
necessary condition for sustained progress in education
is the accumulation of information on the particular
outcomes that result from specific combinations of
inputs."51

Thus, the chief advantages of the three budget
management techniques; systems analysis, PPBS, and program
cost accounting, may be that of initiating a cost-
effectiveness system that will relate a set of outputs
(progress toward stated objectives) and a specified set
of inputs (their related costs) in a manner that will
facilitate an evaluation study that will define the
processes that brought about the desired outcomes.

This relationship of inputs to outputs is often
called cost-effectiveness. The struggle to determine

degrees of cost-effectiveness in regard to educational

 

511bid., p. 361.

41

programs has been, and is likely to continue to be, filled
with obstacles just as it has been for business, industry,
and government for many of the same reasons. Benson
states,

. . . Possibly there will come a day when produc-
tivity measures will exist in education, and we
ought thus to see what kind of data are required in
order that such a tool of research can be made
available. We will want to examine the process of
weighing alternative costs to accomplish given,
specific ends. . . . (also) from the economists'
point of View, only as school systems go beyond the
measurement of productivity to study the contri-
bution of Specific inputs (various types of human
services and various types of physical goods) to
educational ends, i.e., to explore the interrelations
between inputs and outputs can productivity advance
in education be assured. . . .

. . . While a strong case exists a priority for
the statement that advance in productivity in the
industrialized nations cannot continue unless a
certain (somewhat undefined) level of schooling is
maintained, it is exceedingly difficult to relate a
given change in expenditure on public education
to a correSponding quantitative improvements in
economic productivity. (Further) it is sometimes
held that education is handicapped by not being
able to relate changes in dollar value to inputs
to some series of outputs. We do not have any
widely accepted index of productivity for the edu-
cation industry itself. It would be well if we
could measure the indirect contribution of education
of productivity, i.e., the effects of changes in
quality of schooling on changes in output in the
private economy say. But it would also be well
if we could measure the direct contribution of
changes in expenditures to change in outputs of
the education industry itself.5

There are many factors which are closely related

to changes in productivity which undoubtedly will have

 

521bid., p. 338.

42

to be defined and refined. Mills suggests the following
general list as being worthy of consideration in this
regard.

The quantity of quality of capital equipment
used.

The quality of (human) efforts input (This may
be a change in intensity or a change in average
degree of skill. Such a change in average skill
may result from a change in the competence of
individuals or groups or from a shift in the com-
position of the work force).

The ratio of effort input to productive instru-
ments used or to natural resources used (A change
in average productivity resulting from the play of
diminishing returns would be included in this
category).

The quality of natural resources or material
used.

The quantity of materials or intermediate
products used to produce a standard unit of final
product.

The amount of non-human power used or the
manner of its use.

The organization of productive units.

Working conditions. 53

The effectiveness of administration.

The above list is not as appropriate for edu-
cation as for business or industry, but it does point
out that there is a large number of factors involved and
an even larger number of possible combinations of factors
and relationships in the development of a cost-effective-
ness system which is designed to report productivity in
terms of student performance and effectiveness of dollar
expenditures.

Allan R. Lichtenberger, Chief, Terminology Com-

patibility Branch, U.S. Office of Education more

 

53C. Wright Mills, quoted by Benson, Ibid., p. 346.

43

specifically states that a school system which employs a

program of cost-effectiveness should expect the following:

*

Improved assessment of the efficiency of
allocation of education resources.

In plaIn words, this means tHat a better job can
be done of assigning staff, plant use, equipment
supplies and funds to accomplish what is believed
to be worth accomplishing.

A more continuous and consistent consideration
and review of educational objectives.

The very fact of planning, budgeting, and account-
ing for the comparatively small "units of
behavior" or "programs" geared to specific,
expected accomplishment virtually forces
attention to objectives.

Sharper and more consistent examination of
essential sequences of educational development.
Education is a highly sequential process. These
sequences both within and among "program" are
much more easily observed, studied, and managed
when there is a process of deliberate planning,
budgeting for, and accounting for each "program."
More effective communication through all levels
of management concerning processes and operatibns
as they relate to aCHievement offiobjectIVes.

One of the most dIfficult and often the most
neglected processes in school system management,
is communication upward, downward, and laterally
in the organization with the central purpose of
improving quality. With "programs" identified,
planned through staff involvement, budgeted for
with an understanding of what is economically
feasible, and accounted for in terms of all
resources committed to the objectives there is
something definite to communicate about.

Better understanding of how educational resources
and effOrt reIate to accompliShment.

If more money and more effort do not result in
more effective education, the educational enter—
prise is ironically unique. Still, education

has seldom shown clearly that more resources
result in improved accomplishment. With "pro-
grams" the opportunities to clarify this relation-
ship are at their best.

Disclosure of the kinds of educational development
foregone When resources are lIfiited.

Entirely too often the statement,IwWe have a good
school system," implies that the school system

is meeting more needs that it really is. Fre-
quently, too, a school system is criticized for

 

 

 

 

 

 

 

 

 

 

 

 

44

not providing certain kinds of educational
development when their provision is not possible
in terms of available resources. When "programs"
go through the process of planning, budgeting,
and accounting, the kinds of educational develop-
ment they make possible show with clarity the
kinds that are foregone. It is common sense to
make such exclusions evident.

* Better opportunity to set educational priorities.
Seldom does a school system have sufficient
resources to do all that needs to be done.
Setting of priorities is essential. "Programs"
bring these into focus, and the people who make
decisions have a much better basis for determining
orders of importance in View of resources
available.54

 

A cost—effectiveness design is thus seen to be a
part of the total process of program management. The
advantages noted above, however, will not be fully
realized unless program planning, program budgeting, and
program accounting are all implemented.

Lichtenberger simply defines the terms as follows:

The determination of what is to be done to
achieve the objectives of a "program" is program
planning.

The process of carefully estimating the cost of
doing what is to be done to achieve the objectives
of a "program" is program budgeting.

The recording of what is done in the operation
of a "program," of what is spent, and of what is
accomplished is program accountipg.55

 

 

The development and application of a cost-

effectiveness system which links learning to cost by

 

54"Program Planning, Budgeting, and Accounting in
School System Operation" (a position paper by Allen R.
.Lichtenberger, Chief, Terminology Compatibility Branch,
IJ.S. Office of Education, September 20, 1967), pp. 3-5.

551bid., p. 2.

45

coupling indicators of unit objectives achieved with
indicators of unit costs may provide one powerful tool
for a board of education to convince the electorate it
has accepted its obligation to make wise decisions about
prudent public policy and effective plans for education.

Using this rationale, it would appear that a
cost-effectiveness management system might satisfy the
public's demand for accountability. The question is,
"How can schools link learning to costs and thus become
accountable to the taxpayer?"

Leon Lessinger suggests that schools must develop
new educational approaches. Schools must define output
not as teaching done but as learning proven. And this
must be substantiated by data that shows educational gain
produced by specific sequences of teaching.

Lessinger continues by stating:

Once the output of schools is measured in proven
learning instead of resources allocated or teaching
done, the next step is to relate learning to costs.
In theory, we can easily compare the money spent and
the result achieved. We simply keep accounts of
costs of a specific teaching sequence and measure the
change in performance against a standardized evalu-
ation given before and after it. This is seldom
accurately done except in formal experiments. Now
we must extend this kind of calculation to cover wide

areas of teaching, especially the acquisition of
basic skills such as reading.

 

56Leon K. Lessinger, Every Kid a Winner, Accounta-
bility in Education (New York: Simon and Schuster, 1971),
p. 9.

 

57Ibid.

 

46

Charles Blaschke, President and Chairman of the
Board of Education, Turnkey Systems, Inc., a Washington,
D.C. management consulting firm, suggests the performance
contracting concept as one tool for school systems to
manage reform.58

Performance contracting is a procedure whereby
private industry contracts with a school district to
guarantee a certain amount of student achievement for a
set amount of money. This procedure is recognized by
some as the simplest form of cost-effectiveness available.

Blaschke believes that this concept can provide
a possible answer to the taxpayers' demand that schools
be held accountable for productivity in terms of student
performance and the effectiveness of dollar expenditures.
He claims that, "Nothing could regain the confidence of
the taxpayers more than a guaranteed result performance

"59

contract. Blaschke defines the guaranteed result

performance as a "program design in which a contractor

"60 The

achieves results or else he pays a penalty.
taxpayer can understand the rationale of this system
because he is probably in a business where he knows if

the product is no good, the irresponsible individuals

are fired, or the business goes into bankruptcy, but the

 

58Charles Blaschke, "Performance Contracting,"
IDEA, Winter Quarter, 1971, pp. 6-7.

591bid., p. 7. 6OIbid.

47

business does not keep getting more money year after
year. Too often the typical response for schools when
they have a failing product is to request additional
monies.

Charles Blaschke, who is also recognized as the
"father" of performance contracting, planned the Texar-
kana Dropout Prevention Program, the first performance
contract in the country awarded for the school year
1969-70.

In his introduction of Performance Contracting

 

in Education, Blaschke defines the Texarkana Project and

 

performance contracting as:

Performance contracting in public schools is a
reality. During the 1969-70 school year in
Texarkana schools, a private corporation, Dorsett
Educational Systems, Inc., is removing math and
reading deficiencies of approximately 400 students
on a guaranteed performance basis. The contractor
guaranteed an increase in student performance of
one grade level in either math or reading in a
specified number of classroom hours of instruction
for a Specified maximum cost. Payment is to be
based on whether the contractor meets his guarantee.
Initial testing of students has shown results
better than the regular school in less time and

at less cost.61

The following is Blaschke's description of the
Texarkana experience and its history which could be

repeated in many schools across the nation:

 

61Charles Blaschke, Performance Contracting in
Education: The Guaranteed Student Performance Approach
to Puinc Séhool System Reform (Champaign, 111.: Research
Press, EducatIonal Turnkey Systems, Inc., 1970), p. 2.

 

 

 

48

The Texarkana schools faced a variety of problems
during the 1968-69 school year. The average drOpout
rate per year for grades 7 through 12 in the poverty
area schools was fifteen percent. Twenty-seven per-
cent of students were participating in school pro-
grams for the disadvantaged. Students were two or
more grade levels deficient on the Iowa Test of
Basic Skills after four years of participation
of special programs. Eighty percent of federal
funds were going directly into salaries with only
twenty percent going into program enrichment.

The proposed programs were quite innovative,
certainly not "pre-packaged." For example, Dorsett
proposed utilizing for the first time in a large-
scale effort an audio-visual learning system which
had been under development for over twelve years.

At the same time, it would utilize software developed
by other corporations to insure that a total learning
was available to students with different learning
characteristics. Another corporation would have
utilized a new math program; a second corporation
proposed to combine three instructional management
systems and to convert existent Job Corps materials
to a new media format; a third corporation proposed
to develop a computer-based counseling system for

the Southwestern region, to be tested and validated
in the Texarkana school system.

All of the bidders, with the exception of one,
proposed to use self—paced instruction. To insure
efficient scheduling and administration through the
use of individualized self-paced instruction,
sophisticated instructional management systems were
part of each proposal. All recommended audio and
visual means of presentation. Three of the top
five proposals would have utilized the relatively
new concept of contingency management integrated
into the instructional program. It was obvious
from an evaluation of the prOposals that the com-
petitive process had caused the ten bidders to
enlist the resources of the nation's foremost edu-
cators in creating a program for Texarkana.62

 

The results of the Texarkana Project received a

troublesome jolt when the independent auditor's report

 

621bid., p. 12.

49

questioned the validity of the first year's results.63

The independent auditor questioned the gains in student
scores because they found some of the test questions had
been included verbatim in the instructional program.64
How much contamination occurred and what effect it really
had was not clear until Dr. Robert Kraner, President of
Epic Diversified Systems, Inc., completed the audit
report and stated that he agreed with internal evalu-
ator's report that progress checks, and laminated panels
of actual or closely related test items and the utili-
zation of these materials in a concentrated manner just
prior to an exit testing period were a gross violation
of the rationale underlying, the normative concept of
grade level achievement.65
Dr. Kraner, the Epic auditor, stated that his
agreement is based upon the two reported activities of:
l. Concentrating test items in the instructional
materials.

2. Intensively reviewing students over these
materials prior to exit testing.

 

63"Clouds and Controversy Over Texarkana,"
Nation's SChOOlS, LXXXVI, NO. 4 (October, 1970), 85.

 

64Ibid.

65Robert E. Kraner, Final Audit Report, Texarkana

 

Dropout Prevention Program (TuESOn, Afizona: Epic
Diversified Systems Corporation), p. 22.

 

50

There was a violation of the norm concept of grade-
level achievement which served as a basis for the
previous agreements between the Texarkana Dropout
Prevgption Program and Dorsett Educational Systems,
Inc.

Despite the auditor's report that the objective
of Texarkana Project, to reduce the dropout rate, had
been achieved, many educators remained skeptical.

Wendall Pierce, executive director of the Edu-
cation Commission of the States has stated that:

Significant opposition has come from organized
teachers' groups who argue that performance con-
tracting is a gimmick which diverts attention
from the real need for smaller classes, remedial
programs and better counseling and that learning
for rewards, as is done in some programs will
undermine proper motives. They continue to argue
that if performance contracting works, it will
cost more because the greater the achievement,
the larger the payment to the contractor. These
groups feel that if this money were paid for
their salaries, materials and reducing class
size, they could do as well.67

Performance contracting is not seen as the
panacea for determining cost-effectiveness nor even as
an alternative process for providing education by many
groups and individuals.

The strongest critic of the performance contract-
ing concept has been the American Federation of Teachers.

AFT President, David Sheldon, says that his concern is

the eventual growth of educational gimmicks which is an

 

66Ibid.

 

67Wendall H. Pierce, "New Directives for Edu-
cation," Compact, February, 1971, p. 2.

51

invasion of the responsibilities of teachers and grounds
for strikes.68 Mr. Sheldon resolved to sponsor a major
nationwide campaign to oppose performance contracting.
Dr. John M. Lumley, the National Education
Association's Executive Secretary for Government
Relations and Citizenship, said in a statement to the
Senate Appropriations Committee that the "NEA deplores
the performance contracting concept because it can
weaken the structure of the public school system and
can discredit the schools in the eyes of the public.69
Many supporters of the performance contracting
concept were quick to respond to the critics. Foremost,
among the supporters was Dr. John W. Porter, Michigan
State School Chief. Dr. Porter stated at the winter
workshop meeting of the High Scope Educational Research
Foundation that
. . . we must guarantee minimal student performance
in the primary and secondary schools. Someone has
to require schools to guarantee student performance
consistent with clearly spelled-out performance

objectives, irrespective of the educational mumbo-
jumbo related to learning theories.7O

 

68David Sheldon, Statement made at the AFT Con-
vention, August, 1970, Pittsburgh, Pennsylvania.

69"How Education Groups View Contracting,"
Nation's Schools, LXXXVI, No. 4 (October, 1970), 87.

 

70The Grand Rapids Press, "Porter Rips State Edu-
cators, Goals," December 3, 1970.

 

52

Dr. Porter continued to say that

. . . education groups react nervously when he
Speaks this way because we haven't been able to
transform the educational folklore to measurable
productivity. Today's questions focus on produc-
tivity-~the results obtained from the resources
used and the public schools are now being held
accountable for these results.71

"We must start to guarantee student performance,
and you don't guarantee student performance by insti-
tuting remedial reading, writing and arithmetic pro-
grams," continued Porter.

You institute guaranteed student performance by pro-
grams such as performance contracting where indi-
vidualized instruction occurs. As a society, we
profess to be interested in providing for the indi-
vidual differences of children, but the schools are
modeled after factories. AS a society, we talk and
talk about meeting the needs of children, but when
confronted with the concept of performance con-
tracting, some educators label it outside inter-
vention and profit peddling.72

The Office of Economic Opportunity evidencing
concern for the need that too little evaluation of such
programs was available, launched into a $6.5 million
year-long performance contracting experiment involving
28,000 students in twenty school districts in remedial
reading and mathematics.73

The CEO experiment was designed to test the per-

formance contracting concept in a variety of situations,

 

7lIbid. 721bid.

 

73Office of Economic Education, "Background: An
Experiment in Performance Contracting" (program des-
cription, July, 1970), p. l.

53

all carefully controlled and evaluated so that school
people might have some concrete data on such programs
before making the decision to enter into contracts with
technological firms who claim through the use of a
variety of techniques and educational hardware, to be
able to guarantee student results. The CEO felt that
many school systems across the nation have begun to
enter into performance contracts with firms, paying them
for grade gains achieved. While there is great promise
in this approach, there has been too little evaluation of
these programs and what has been evaluated has been small
and isolated. The CEO felt that the grade gain data
must be pinned down to a variety of student types and
other crucial variables and company costs must be clearly
isolated to determine cost effectiveness of the different
74
programs.

To give the reader a better understanding of the
size and scope, the following recap from the Nation's
Schools is herewith included:

With 6.5 million in federal money, the Office of
Economic Opportunity has launched a year-long per-
formance contracting experiment which this fall will
involve some 28,000 students in 21 school districts
in remedial reading and mathematics.

Of the 21 districts chosen from approximately
170 which had asked for funds, OEO will funnel
$5.6 million into performance contracts between

18 school districts and six education companies.
In the remaining three districts, OEO will Sign

 

74Ibid.

54

contracts for programs utilizing a traditional
educational framework, with local teacher groups,
for example, operating under incentive contracts
with CBC.

By employing these two types of contracts--
public and private--OEO hopes to learn after a
year whether the performance contracting approach
should be applied nationwide. Dr. John O. Wilson,
assistant director for planning, research and
evaluation at CEO, said the experiment would help
validate results in the first performance contract-
ing project in Texarkana, Arkansas.

Wilson expressed "concern" that the performance
contracting approach could lead to "teaching to
the test" since payment is geared to achievement
scores. In Texarkana, the contracting company,
Dorsett Educational Systems, Inc., was reported
to be preparing students for the tests, but the
company was denied making a direct coaching effort.

Wilson added that OEO'S experiment will contain
precautions to guard against the possibility of
teaching to the test. OEO will select three
standardized tests, which will be administered on
a random basis so that the company will not know
ahead of time which test will be given. Also,
75 per cent of the payment will be based on the
standardized test scores while 25 per cent will
rest on performance on reading and mathematics
tests (criterion reference tests).

The six private firms were selected on a com-
petitive basis from a group of about 30. DEC
said the firms chosen represent a "range of inno-
vative technics, including incentives to students,
teachers, parents, teaching machines and programmed
learning technics. . . .

The contracts between the education companies and
the 18 school districts will involve about 27,000
under achieving students in Grades 1—3 and 7-9.
Approximately 1,000 more students will be involved
in the three projects "to assess education incentive
system only," CEO said. OEO officials said the
performance contract would provide payments of
$110 per grade level increase in each of the skills
taught, adding that the companies would begin to
make a profit at 1.6 grade level increases. The
contracts provide a maximum payment ceiling of $220
per child per subject matter, or the fee at 2.3
grade level increases per year, according to CEO.

55

According to the CEO, the skills of each student
will be tested by the contractor at the beginning
of the experiment, periodically throughout the
school year, and at the end of the experiment.75

In addition to the CEO reading and math projects,
thirty-two other school systems were engaged in per-
formance contracting during the 1970-71 school year in
which similar results of productivity and cost-effective-
ness are yet to be measured. (OEO has summarily released
interim data on the project this spring but the total
data has not been released as of the completion of this
study.)

There seems to be common agreement among edu-
cators that the two basic skills that all children need
to master are reading and math. Since this thesis will
be delimited to a small part of the total educational
function as performed by schools, we will only concern
ourselves with only one segment, the reading portion
of the curriculum.

A brief description follows of the several pro-
grams that will be evaluated in this study on a cost-
effectiveness basis.

All three of the performance contracting programs,

the first to be listed herein, provide preservice staff

training, in—service, and continuous program monitoring.

 

75"OEO Announces $6.5 Million Contract Program)"
Nation's Schools, LXXXVI, No. 2 (August, 1970), 33.

56

Each program specifies specific behavioral objectives

and an evaluation design for measuring their attainment.

Alpha Learning Systems, Inc. Performance Con-

 

prggp.--Alpha Learning Systems, Inc. is based on the
premise that regular teachers, given the training and

tools of technology, can do as well as an outside,
hardware-oriented private industry. The company uses

all software materials, i.e., program materials currently
available on the Open market. The company does not pro-
duce any materials but blends and mixes a large reservoir
Of available materials according to the student's needs as
determined by the teacher and test results. Heavy emphasis
is placed on student and teacher incentives which are

used as motivational and reinforcement techniques.

Combined Motivational and Educational Systems Per-

 

formance Contract (CMES).--CMES is a highly structured

 

technological approach using a variety of machines and
programmed learning materials. The use of many para-
professionals has initiated the differentiated staffing
concept. An aid to developing self-concept is the
achievement motivation program (rap sessions). Student

incentives are utilized.

Westinghouse Learning Corporation Performance

 

Contract (WLC).--Westinghouse Learning Corporation utilizes

 

differentiated staffing. The program is based on a

57

diagnostic and prescriptive technique using a variety of
existing published materials: both software and hardware.
Student incentives are used as a contingency management

technique.

Reading in the Regular Program.—-The regular

 

reading program is taught by the classroom teacher who
uses an adOpted textbook as well as teacher selected
Optional materials. In this program, the fundamentals
Of reading are taught through traditional techniques.

It must be admitted there are many critics of the regular
reading program who allege that the public schools give
only lip service to individualized education based on a
needs assignment. If differences are considered, they
are met by dividing groups according to their abilities,
i.e., slow, normal, and fast. The overall goal Of the
regular classroom is "to teach the whole child Skills,
understandings and attitudes so he can take his rightful

76 Critics say this

place in our democratic society."
approach is not based on attainable—measurable goals.

The procedures used provide little or no pre-service for
the classroom teacher in the use Of the materials, except

by way Of previous training, little meaningful in-service

or monitoring of teacher performance; no systematic

 

76Language Arts Study Guide, Grand Rapids Public
Schools, 1960, p. I}

 

58

evaluation or measuring design to determine student per-
formance and little budget for teacher requests to supple-

ment her teaching.

Traditional Remedial Reading (TRR).--TO work with

 

students who have reading difficulties, the public schools
have developed a traditional type of remedial reading
program. This program is usually Operated by a reading
consultant who works with small groups of identified
remedial students for brief periods of a time on a once-
a-week schedule using a variety Of materials with limited

follow-through to adapt to the student's regular classwork.

Project Read (PR).--Other programs currently being

 

used in the public schools are "canned" programs, i.e.,
they are pre-packaged curricula produced by a book pub-
lisher which include tests, workbooks, etc. necessary
to teach reading. The programs are Often called program
learning. The programs usually consist of a number of
packaged workbooks designed tO be used in a sequential
manner by the classroom teacher. Project Read is such
a a prepackaged program. Within the format listed above,
the classroom teacher works on an individual basis only
with those students who need tutoring, all other students
work independently.

Because there are so many different kinds of pro-

grams in which reading is taught, the programs listed

59

above were chosen for this study because it appears to
incorporate most Of the commonly used techniques and

resources found in such programs.

Evaluation

 

As noted earlier, the next portion of this study
must address itself to the evaluation Of the effective-
ness of the approaches used in the aforementioned read-
ing programs.

One part of the effectiveness determination then
becomes a measurement of evaluation. Student evaluation
is often based on student performance on a nationally
normed commercially, available standardized achievement
tests administered at the beginning and end Of the
school year. The measure Of student achievement is the
grade gain accomplishment for a given period Of time.
Standardized tests usually assume an average student
will make one year's gain in one year Of schooling.

Some controversy over the utilization of stan-
dardized tests to measure gains is currently being voiced.
In researching this matter, Dr. Ralph W. Tyler, Director
Emeritus, Center of Advanced Studies in the Behavioral
Sciences was contacted by this writer. Dr. Tyler
responded in a letter dated May 7, 1971, stating that
the currently used tests are the best form of assessment

now available on the market. However, Dr. Tyler said

60

that to improve testing in terms of the actual learning
Of students, educators have a responsibility to encourage
reputable test makers to develop criterion-referenced
tests.

The application of cost-effectiveness logically
follows evaluation in that indicators of unit Objectives-
achievement, when coupled with indicators of unit costs,
may then provide a basis for cost-effectiveness analysis.

It is the intent of this study to develop a cost-
effectiveness design as one means of providing needed
information for the Grand Rapids Public Schools on per-
formance contracting programs, "canned" programs, and
other traditional school programs in the subject area Of
reading and which may be applicable to other subject
areas and to other school systems.

The cost-effectiveness model unit which will be
outlined in Chapter III was purposely designed to provide
a means of identifying and isolating the most probable
important factors contributing to the efficiency with
which resources are expanded for instructional activities
in the programs studied.

The model may be described as an example of one
of the "Specified tools and techniques" previously noted.
It is mainly concerned with Obtaining cost data and relat-
ing that data to the Objectives, evaluation, and dif-

ferentiating characteristics in the different kinds Of

61

reading programs (incentives, differentiated staffing,
timeframes, materials, etc.) of the programs. The
resultant information "package" related to output can
then be used as relevant data for planning, and decision-
making. The model was also concerned with analysis,

the breaking down and distribution of costs, as an
important prelude to synthesis of new and revised pro-
grams.

Stating the objectives for each program is a
difficult task. It is assumed that all instructional
programs have objectives and that they are not so
obscure, mysterious, or secret that they cannot be
expressed in written form. Stating the objectives in
behavioral terms may be the most difficult of all, but
it offers the important advantage of producing something
observable and testable. This tends to make evaluation
of progress made toward the objectives more measurable
and useful in terms of cost-effectiveness considerations.
Indeed, it would be almost impossible to measure cost-
effectiveness of programs for which the Objectives are
expressed as to know, to enjoy, to believe, to understand,
to grasp the significance of, and so forth. Mager has
defined the problems related to stating Objectives in
behavioral terms and concludes that:

l. A statement of instruction objectives is a

collection of words or symbols describing one
of your educational intents.

62

2. An objective will communicate your intent to
the degree you described what the learner will
be DOING when demonstrating his achievement and
how you will know when he is doing it.

3. TO describe terminal behavior (what the learner
will be DOING).

a. Identify and name the overall behavior act.

b. Define the important conditions under which
the behavior is to occur (givens or
restrictions, or both).

c. Define the criterion Of acceptable per-
formance.77

Performance objectives as defined by a recent
publication prepared by the Eric Diversified Systems,
Inc., President Dr. Robert E. Kraner, "performance
Objectives are precise descriptions of educational goals

in terms of desired behaviors outcomes, or material terms

which can be reliable, validly and objectively measured."78

Performance Objectives as defined by the U.S.
Office of Education include the following essential

elements:

1. The individual or group who is to perform in
a desired way.

2. The behavior to be demonstrated or product to be
developed. The behavior should be described
as an action or a product that can be observed.
Therefore, words such as understanding, appreci-
ation, thinking, enjoying, etc., should be
replaced with more precise behavioral terms.

3. The level of accomplishment of the minimum
acceptable level of performance. Research and
baseline data are important considerations in
setting the levels of performance.

 

77Robert F. Mager, Preparing Educational Objectives

(Palo Alto: Fearon Publishers, 1962), p. 53.

 

78Robert E. Kraner, "Developing and Writing
Behavioral Objectives," Educational Innovators Press,
Tucson, Arizona, p. 57.

and

63

4. Determining the units of performance measure-
ment and the means of measurement. For
instance, a statistically significant increase
in an academic skill is one way to identify
this element.

5. The conditions under which the performance is
expected to be measured may include a restruction
or limitation. A condition might also be a
supportive service or resource need to assist
the performance.79

James Lewis, Jr. describes behavioral Objectives
their relationship to education as:

The function Of education is to change or modify
behavior. Behavioral Objectives are designed to
indicate the desired behavior; therefore, they can
be defined as a collection of words, statements,
or symbols which describes an education intent.
Students learn to acquire knowledge, attitudes

and Skills. There are three domains Of learning,
i.e., the Cognitive (thinking), the Affective
(attitude) and the Psycho-Motor Skills. Behavior
results from thinking and learning. Therefore, in
order to write clearly stated Behavior Objectives,
the reader must be able to write these Objectives
realizing the seven Thought Processes and the
Thought Derivatives, so that the aims and goals

of the lesson are readily brought out and may be 80
adequately evaluated in terms of the expectations.

The process of evaluation regarding a large number

of objectives and reporting results would involve
nous records. For the purposes Of this model and
managerial decision-making regarding the programs
in the model, a more simplified expression of the

tives and evaluation must be used.

 

79USOE, ESEA Title VII Manual for Project

cants and Grantees (Washington, D.C.: Government
Office, April 24, 1969), p. 5.

80

volumi-
for
reported

objec-

Appli-
Printing

James Lewis, Jr., Administering the Individual-

 

ized Instruction Program (West Nyack, N.Y.: Parker PuB-

lishing CO., Inc., 1971), p. 79.

64

The "quality" of the program is simply an
expression of its effectiveness. The important question
regarding whether or not the class or activity is appro-
priate to the needs Of the pupils enrolled is involved
only peripherally. The model assumes that the Objectives
will be as appropriate as those who select them know how
to make them and that the task Of expressing the objec-
tives in summary form, evaluate the progress made toward
them by whatever means seem best, and compare the results
with the costs. Progress toward an appropriate Objective
may be of little educational value. However, that is the
basis Of another set of studies and is outside the limits
set for the model.

Instructional programs accounting is, in sum, a
system whereby what is to be done (Objectives) is care-
fully formulated, methods Of evaluation defined, resources
(personnel, facilities, supplies, and equipment) assigned
according to a plan and schedule, after which the program
is implemented, evaluation made, and costs determined.

A new cycle, based in part on the experiences with the
last cycle, is then planned for and implemented.

The implementation of the model will not fulfill
all of the many requirements of instructional programs
accounting--it has not been designed to do so. It will,
hopefully, serve to move a district closer to that goal.

Districts are presently performing many, perhaps most,

65

of the acts required to relate costs to programs. Imple-
mentation of the model should help the district take the
few remaining steps toward that goal. The next chapter
includes a description of the model and some suggestions

concerning its implementation.

Summary

The articles and studies reviewed in this section
generally support the need for cost-effectiveness studies
in the public schools. In the planning Of a cost—
effective study, it can be determined that inputs must
be related to outputs. Another way of specifying this
concept is to say that learning must be linked to cost.

It has been determined that the teaching of
reading in the public schools is a matter of priority.
Several different reading programs have been defined.

It may be concluded that these programs should be eval-
uated in terms of student achievement and cost per program.

To begin this process, program objectives have
to be defined. It must be assumed that the program
Objectives in this study shall be defined as certain goals
in reading skills and comprehension that must be achieved
by students. Although evaluation to determine how well
students achieve those goals is limited, we have to
accept the fact that there is no better way at this
point in time. The logical next step that follows is

to relate student achievement to program costs.

66

The establishment of a cost—effectiveness model
to relate student gains to program costs in several
types of reading approaches in the Grand Rapids Public

Schools is that logical next step: the purpose of this

study.

CHAPTER III

DEVELOPING AND FIELD TESTING THE MODEL

Introduction

 

This cost-effectiveness model has been developed to
provide a way of determining "what is spent for what is re-
ceived" for each program. This information can be used for
planning and budgeting as well as for cost analysis and
evaluation. The model suggests a way of identifying the
program which includes such information as the nature of
the program, who was effected, how many full—time certified
teachers were required for its implementation, how many
paraprofessionals were required, what methods and facili-
ties were used, what incentives were available to students
and staff, what additional staff training was required,
what start-up cost incurred, what material and supply
costs were incurred, and what Objectives were involved and
what progress was made toward those Objectives and reported
in terms of student grade gains.

The model is intended to be consistent with one
form of systems analysis. A general description Of systems

analysis has been defined as:

67

68

. . . inquiry to aid a decision—maker to choose a
course of action by systematically investigating
his proper Objectives, comparing quantitatively
where possibly the costs, effectiveness, and risks
associated with the alternative policies or strate-
gies for achieving them, and formulating additional
alternatives if those examined are found wanting.81

In addition to the systems analysis approach, this
model will attempt to relate the input factors (costs)

with the resulting outputs (student grade gains).

Description of the District to be Studied

 

The cost-effectiveness model will be field tested
in the Grand Rapids Public Schools located in Grand Rapids,
Michigan. The Grand Rapids Public School district is part
of the Kent County Intermediate district and is the larg-
est Of the nineteen districts Of which the Intermediate
district is composed. The Grand Rapids Public Schools
Operates a comprehensive educational program--prekinder-
garten through the Community Junior College.

The Grand Rapids Public Schools have a current en-
rollment of about 39,000 full-time students. Approximately
34,500 of these are in the kindergarten through twelfth
grade division, including Special Education and other
Special programs. Almost 4,500 full-time students attend
the Grand Rapids Junior College. The school system has

Operating units consisting of fifty-four elementary

 

81Richard A. Johnson; Fremond E. Kast; and James
E. Rosenzweig, The Theory and Management of Systems (New
York: McGraw-Hill Book CO., 1967), p. 144.

 

69

schools, four middle schools, five junior high schools,
four high schools, a school for the deaf, an orthopedic

school, and a school for the mentally handicapped.

ReadipgpProgpams Within the District

 

Several different reading programs Operate within
the school system. Appraisal of the cost-effectiveness
Of these several programs requires that the objectives of
these programs be stated, that evaluation of the progress
made toward these Objectives be measured and reported, and
that the costs of these programs also be determined and
reported. This model will relate the evaluation Of prog-
ress to costs. It will also accept the common objectives
that "all prOgrams are designed to teach students to read."

TO begin the presentation of the model and its

develOpment, a brief description Of the programs to be

studied will be made.

Regular Reading Program
(Elementary)

 

The regular reading program is designed to be
taught by the classroom teacher using an adopted textbook
plus teacher Optioned supplementary materials. For the
purposes of this study, we will limit this control group
to levels 2—3. Student minutes of exposure time to read-

ing may vary but will be reflected in the cost analysis.

70

Regular Reading Program
(SecondaryT

 

 

The regular reading prOgram in the secondary
schools is structured in the seventh grade by enrolling
each seventh level student for one 55-minute period of
reading per day for one semester. The eighth and ninth
level curriculum includes English and literature. Stu-
dents are enrolled for one 55-minute period per day per

year. Therefore, levels 7-9 will be studied.

Traditional Remedial Reading

 

This program is designed to remediate those stu-
dents who were not successful within the regular school
program. The program Operates in levels 2-9. For the
purposes of this study, we'll be concerned only with
grades 2, 3, and 7. This program traditionally is Oper-
ated by master teachers who function within the Specific
school buildings. Most teachers are assigned to more than
one building. The teachers use a wide variety of materials
and techniques based on their professional experiences.
Time allotments for student instruction vary but will be
analyzed in detail for cost-effectiveness. In addition to
remedial reading, it is assumed that the identified stu-
dents may receive some reading instruction in the regular

classroom.

71

Project Read

 

This program is a prepackaged program using a
publishing company's format known as the Sullivan mater-
ials published by Behavioral Research Laboratories. Ac-
cording to the program description, this material is de-
signed to lift a child from the status Of "non-reader" to
the status of "good" reader. This program is taught by
the regular classroom teacher who does not present original
material to the students but tutors them in their pro-
grammed instruction materials only when they require as-
sistance. A sixty-minute daily session is allocated in
each grade level, 1-6. We will study levels 2-3 in this
study.

Alpha Learning Systems, Inc.
Pprformance Contracting Project

 

 

The Alpha Program utilizes existing facilities,
software programmed materials, regular classroom teachers
who have been trained to diagnose and prescribe a cur—
riculum based on student needs, paraprofessionals as
teacher assistants, behavioral modification and contin—
gency management techniques of student incentives and
teacher incentives plus a student reinforcing events room.

This program is part Of the OEO Remedial Perform-
ance Incentive Program and services students in levels

1-3, and 7-9. The elementary grades meet for 55-minute

72

periods and the secondary groups for 45-minute periods.
This study will include grades 2, 3, 7, 8, and 9.
Westinghouse Learning Corporation
Performance Contracting Project

The Westinghouse Learning Corporation program
Operates in learning centers within a school building.
This environment utilizes motivational techniques, self-
instructional materials and an individually prescribed
curriculum designed to meet the learning needs of the
students. Students are given a diagnostic test upon
entering the program. The results of this test are
telexed to the Westinghouse Learning Corporation home
office where a prescription is written for the student.
A center manager (teacher) and paraprofessionals man the
center and assist the students with their work, using
both software and hardware materials. All students in
levels 1-6 within a school participate in the program but
for this study we are concerned with only those students
in levels 2-3. Time allotted for learning center work is

37.5 minutes per day.

Educational Systems Perform-
ance Contract Project

The CMES project is a motivation-centered instruc-
tional program that is hardware orientated. After stu-

dents are diagnosed, it is assumed that they are

73

appropriately placed in the curriculum where they work in
their individual carrel. An integral part Of the program
is the self-concept development A.M.S. (Achievement Moti-
vation Session). Students are reinforced for doing a
"good" job by being rewarded with "free" time in a games
room.

Students in grades 6-9 attend the MES center.
Students spend 45 minutes in the center. For the pur-
poses of this study, we will deal with students in
grades 7-9.

The three performance contracting projects men-
tioned above held two-week preservice workshops for the
staffs in addition to the regular school system orienta-
tion sessions to train them in the philOSOphy and mater-
ials of the program. This time spent in the workShOp will
be treated as an additional cost in the cost-effectiveness

model

Student Body

 

The studied programs Operate within the Title I
area (inner city) identified as being eligible for com—
pensatory education funds. A population sample was drawn
from each of the programs. The students have been identi-
fied as the lowest performing students in reading in that
grade level in selected schools as verified by the school

system previous spring testing scores.

74

Residents of the Title I area are defined and
designated by federal guidelines issued by the U.S. Office
of Education are low income families with the majority
being on public assistance. The universe of this study
includes students from black, Caucasian and Spanish-
speaking families. The criteria for compensatory educa-
tion funds as defined by the U.S.O.E. is used herein in
determining the population in this study. It is not with-
in the SCOpe of this study to differentiate between black,
Caucasian or Spanish-speaking students and their abilities
to perform toward established goals in reading. However,
all students were at least two years below grade level.

In order to reduce class size and instructional
time variables to a common denominator upon which to
allocate the variable costs all student time will be re-

ported in student minutes Of exposure.

Evaluation Instruments

 

Students have been identified to participate in
this study on the basis of their Title I residency and
school attendance and their low performance in reading
based on the 1969-71 school year spring tests.

Students studied represent grade levels 2, 3, 7,
8, and 9 and were enrolled in the reading program for the
entire year. All students were tested on a standardized,

nationally-known, commercially available achievement test.

75

They were post—tested on an alternate form Of the same
test (see Appendices).

To minimize the problem of "teaching the test" as
occurred in the Texarkana Project, all tests were scanned
(blinded) by National Testing Service, Lancaster, Pennsyl-
vania. All tests were administered and scored by a dis-
interested "outside" evaluator, Battelle Memorial Insti-
tute of Columbus, Ohio. The tests were identical for each
student on a given grade level regardless of the treatment
group, i.e., each grade level received a different stand-
ardized test. All tests used in this study were selected
by the Office of Economic Opportunity and concurred in
by the local school system. To insure added integrity,
attempting to determine the name of the test used was
considered a contractual violation.

All pre-tests were administered within the first
ten days of the 1970-71 school year while the post—tests
were administered within the last ten days Of the 1970-71
school year.

To further insure the credibility of the tests
used and because Of the students involved, historical
difficulties, test identifications and specifications
were outlined in the following memo to project directors
from the CEO:

This is to inform all parties, particularly

subcontractors, of the level of test used for
pre and post testing. As provided in clause 2.05.1

76

section D of the subcontract (TAC will make recom-
mendations to the Project Manager regarding the
appropriate test levels to be used. The Project
Manager shall determine the level of test to be
used), the following is a general description Of
the levels chosen.

In primary grades, the test levels were one
level below that level normally used and recommended
by the publisher. Example: Grade three was given
a Level II test or grade two a Level I test.

In the secondary grades, the level used was
Intermediate--grades 4, 5, and 6 for 7th grade and
Advanced--grades 7, 8, and 9 for 8th and 9th grades.

The rationale for test selection was that stu-
dents in this experiment were contractually required
to be below grade level. As a class, regardless of
their assigned grade level in school, they were func-
tioning below grade level.

The decision to administer the same level test,
alternate form, for the post as was given on the pre-
administration was to prevent distortions due to
imprecision of conversation between scores at one
test level and another, even within the same test
series.

We believe, due to the distortion involved in
different level tests, that provisions for dealing
with the test ceilings and the expected deficiencies
of the students as shown on the pre-tests, that our
action is reasonable.

Statistical Techniques

 

Using an analysis of gain scores (from pre to

post) achievement test scores, this study will be testing

for the difference on the gain scores for achievement to

determine if there is any significant difference between

programs. The total cost differential will be related to

the amount of gain to determine a total cost for the

student's score in an attempt to identify the critical

 

82Charles Stalford, Office of Economic Opportunity

memo, June 4, 1971.

77

variables that may have accounted for a student's achieve-

ment in the reading programs.

Program Variables

 

Traditionally, the instructional programs of the
Grand Rapids district as well as other districts repre—
sent the conversion Of the financial resources of the
district into professional and non-professional services,
facilities, and materials. These costs vary greatly from
subject to subject, level to level, program to program,
and building to building. Most districts used a line-
item system of accounting for the Operating costs of the
district. This system provides a measure of control but
is limited in budgeting and costing individual programs.

The accounting systems currently used in the
nation's school districts serve their basic purpose, that
of accounting for resources spent. The cost-effectiveness
system outlined below is not intended to supplant the
system but to supplement it and make it more valuable for
planning.

The cost-effectiveness model designed for this
study has accepted the basic objectives of the reading
programs to teach children to read and has attempted to
fulfill those objectives by means of an identified curric-
ulum with a Specified budget for the program for which an
attempt is being made to account for the results. A

complete set of line-item budgeting information is

78

currently available for each prOgram. In addition to the
budgetary factors, other program identification informa-
tion such as who were the students, the nature of the
curriculum, materials used, time scheduled, instructor

and facilities used, is also available. The nature Of the
problem is then to relate these inputs to the outputs,
i.e., the measured student grade gains.

To facilitate the distribution of costs of a
specific reading program and other instructional programs
included in the normal day Of astudent, six program var-
iables have been identified by Dr. Stanley Hecker, Depart-
ment of Administration and Higher Education, Michigan
State University, recognized educational cost analysis
expert in Michigan. He suggests the program "variables"
are salaries, staff training, Operating costs, incentives,
facilities, materials and supplies.83

Simply defined, these basic variables help to de--
fine the program parameters.

The descriptive factors as defined by Dr. Hecker,

and used in the construct Of this study are as follows:

Salaries
Salaries consist of that remuneration paid to the
professional and non-professional staff member who is

directly involved in reading programs.

 

83Stanley Hecker, consultation with writer,

April 29, 1971.

79

Staff Training

 

Staff training costs will report those costs spent
by the technological firm or school district for the purpose
Of pre-service training Of the staff in the Operation of the

specific programs studied.

Start-Up Costs

 

These costs represent district expenditures known
as start-up costs only. These include hardware supplies
such as teaching machines. The regularly commonly known
normal Operating costs will not be used because it is as-
sumed that differences in these costs would be negligible

in the different programs.

Incentives

 

Incentives costs are relegated to two categories:
staff and students. They encompass the incentive bonuses
paid to the staff and bonuses paid to students as reinforce-

ment techniques.

Facilities

 

Facility costs are computed for only those costs
incurred because of required remodeling or those costs for
additional space other than that which is required for a
regular classroom. It is a commonly accepted practice to
amortize such initial remodeling costs over a ten-year

period.

80

Materials and Supplies
Materials and supply costs include all of the soft-

ware and other consumable supplies used in the program.

Programs cost analysis using the six variable
dimensions listed above for the programs studied will be
computed on the basis of student minute Of exposure costs.

It was agreed for the purposes of this study,
costs of system-wide administration, custodial care,
transportation, student support services, facilities, and
fixed costs across the system, will be assumed as being
prorated equally among the reading programs studied, as
well as to other educational programs Operating within the
district. This assumption is based upon the recommenda-
tion of Dr. Stanley Hecker, who also recommended the omis-
sion of possible payments to the performance contracting
companies since this study is concerned with costs to the

school district to turnkey or replicate the program.8

Summary
In summary, it appears that this cost-effectiveness
model has the potential for providing information relevant
to decision—making regarding resource allocation. In the
case of performance contracting, it may also provide in-
formation for the school district as to the costs for
turnkeying the Operation, i.e., taking over that system

Of teaching reading.

 

84Ibid.

 

81

The study Of the Grand Rapids reading programs

using an analysis of variance statistical technique and

then relating the annual program cost factor variables to
the student grade gains, may result in an acceptable model
to make future cost-effective studies.

Providing this kind of information for program
planning is available in a local district and combining
this information with program objectives and evaluation,
educators may have relevant data to more effectively
compete with other societal and governmental units for
resources available.

Cost-effectiveness is identified in this model
as a comparison between the costs of a program (inputs)

and the benefits (outputs) resulting from the program.

CHAPTER IV

ANALYSIS OF SELECTED CRITERIA

In this chapter selected criteria are analyzed.
The first section deals with the establishment of the
cost factors affecting the Operating costs of selected
reading programs in the Grand Rapids Public Schools. The
second section deals with the analysis of grade gain data
of students in the studied programs. The third section
deals with a comparison to determine the cost-effectiveness

of the programs under study.

Cost Categories

 

Cost analysis is concerned with the determination
of fiscal resource requirements for the studied programs.
Within this analysis, six cost categories have been
identified by Dr. Stanley Hecker, school finance expert
at Michigan State University, as program variables and
parameters.85 They are staff salaries, costs for staff

training, "start-up" costs of new programs, facility

 

85Ibid.

82

83

costs, student and teacher incentive costs, and costs for
materials and supplies. Several other Operational costs
necessary for the comparison of all the programs were con-
sidered either as equal for each program or not considered
at all because the total amount of cost was insignificant
in relationship to other measurable costs. The descrip-
tive factors of the measured costs are as recommended by
Dr. Hecker and outlined in Chapter III. The development

and computation of these factors follows.

Salaries

The annual remuneration paid to certified teachers
and paraprofessionals directly involved in the reading
programs was obtained from the Grand Rapids Public Schools'
business office records. In the program analysis follow-
ing this section, staff time spent in reading activities
is prorated in proportion to the full teaching day activi—
ties Of the teachers, aides, etc.

It was recommended by Dr. Hecker that building
principal services, district-wide administration, custo-
dial services, etc. be omitted from this study as they
could be considered prorated equally for all programs in
the district.86 In other words, there was no more nor

less support services in the performance contracting or

prepackaged program than in the control programs.

 

861bid.

84

Staff Training

 

Extra staff pre-service and in-service costs have
been incurred by the school district or technological firm
in some instances. All such training time and costs have
been recorded as being in addition tO the regular working
hours and costs thereof. When training costs were assumed
by the outside company, costs were provided from company
records requested by the school district. Extra staff
training was done during the summer in workshops. Train-
ing Obtained during the regular school year was not at an
additional cost. This time was considered part of the
normal reading activities time and included in that time/

cost framework.

Start-Up Costs

 

District expenditures for such things as teaching
machines and other similar hardware items were considered
a part Of start-up costs. Usual normally used consumable
items such as chalk, paper, pencils, etc. were not con—
sidered to have a cost effect on the outcome of the
analysis.87 The costs for non-consumable items are

amortized over a three-year period as is common practice

in Grand Rapids.

 

87Ibid.

 

85

Incentives

 

Student incentives such as motivational rewards
are computed separately from teacher incentives. All

costs for incentives were obtained from company records.

Facilities

 

Facility costs are computed for those costs in-
curred because Of required remodeling such as installation
of carpeting, air-conditioning, carrels, etc., or those
costs for added classroom space due to special extra space
needs Of the specific program. These costs were Obtained
from the school district files and records. It is a
commonly accepted practice in Grand Rapids to amortize

such initial remodeling costs over a ten-year period.

Materials and Sppplies

 

The costs include all consumable software mater—
ials used in the programs such as program instruction

books, workbooks and unit teaching materials.

Cost Category Summary

 

The services and items listed above, required for
the educational program and identified in these cost
categories were to be considered as the basic cost ele-
ments necessary to implement and Operate the six reading
programs. This structure, then, provides the basic frame-
work for identifying the variable costs Of each program

studied.

86

Structure of the Model

 

The cost analysis model used here in this study
develOps a program description in terms of number of staff
members involved in reading activities, numbers of stu-
dents, and unit time elements and other resources re-
quired. Due to the variety of programs and different time
allotments for the reading activities in the several pro-
grams, it was necessary to determine a specific unit of
time as a common denominator to later develOp total cost
data. The student minutes of eXposure (SME) appeared to
be the most accurate basis to determine costs. It was
recommended by Professor Stanley Hecker and developed by
James J. Gallagher, Director of Budget and Finance for the
Grand Rapids Public Schools.88

The student minutes of eXposure (SME) is deter-
mined according tO the following formula: Number Of
Students (NS) multiplied by Instructional Time (T)
multiplied by 180 school days per year equals §tudent
Minutes Of Exposure (NS x T x 180 = SME).

All costs are determined on the basis of SME (stu-
dent minutes Of exposure) and computed to the sixth deci-

89

mal place as recommended by Mr. Gallagher. Mr. Gallagher

used the rationale that computing costs in the Grand

 

BBIpid. James J. Gallagher, consultation with
writer, May 13, 1971.

89Gallagher, pp. cit.

87

Rapids school district for 35,000 students with a total of
2,268,000,000 SME per year, a one numeric decimal of the

sixth place is equal to $2,268.

Program Description and Costs

 

For the purposes Of this study, reading instruc-
tion time is defined as the total time spent by staff
members in the reading activities of preparation, pre-
sentation (individual as well as group instruction),
evaluation, etc. Some staff members in several of the
programs had other teaching assignments which were pro-
rated to determine the time spent and costs incurred in
the program studied.

Following are brief descriptions of the six read-
ing programs, their instructional strategy, and the cost
development for each. The costs have been computed on the
student minutes of exposure basis in the six cost cate-
gories previously mentioned. However, after all computa-
tions are made, a final annual cost per student will be
generated to more easily evaluate the cost—effectiveness
of the programs studied. It is this resulting program
cost that will be the comparable determinant in relating
output (student achievement gains) to input (costs) or as

Lessinger says "Links learning to costs."90

 

90Lessinger, pp. cit., pp. 9—11.

88

The cost per unit of achievement will be reported
for grades 2, 3, 7, 8, and 9. The reader will note that
computed costs for several grade levels not studied in
depth in this research was necessary in order to develop
more accurate average costs for those programs being
studied. Specifically, grades 1 through 9 will be reported
in terms of cost per student minutes Of exposure; grades
1, 4, 5, and 6 will not be used in terms Of student per-
formance. This was done because in-service time, supplies,
etc., were part Of the total costs for all students in the
total reading program in the school. Since this study only
uses certain grade levels, the need for prorating the
actual costs for only those students in the programs
studied existed.

The decision was made after consultation with the
author's advisor and Michigan State University Research
and Evaluation staff director and statistician app to in-
clude grade one in the study, because of an inadequate
testing measure to determine first grade level student
achievement.91

The relationship between cost (input) and effec-
tiveness (output) will be reported for the following grade

levels as listed below for the six programs studied.

 

91Norman P. Weinheimer, Andrew Porter, and
Geoffry Yager, consultation with writer, March 3, 1972.

89

 

Reading Program Grade

 

Alpha Learning x x x x x
Westinghouse x x

Combined Motivation and Education x x x
Project Read x x

Traditional Remedial Reading x x x

Control (Regular School) x x x x x

 

The Alpha Learning Systems program cost data re-
ported below, follows the model used in determining costs
of staff, start—up, training, incentives, facilities, and
materials and supplies. Each of the other five programs
are developed according to this model. Narrative detail
will not be used unless deviation from the model format
is necessary. The deviation will then be explained.

Variable Cost Data Summary sheets follow each

program's cost development description.

ALPHA II LEARNING SYSTEMS, PERFORMANCE
CONTRACTING PROJECT
The Alpha contract learning project in reading and
math served 100 students in each of grade levels 1, 2, 3,
7, 8, and 9 for a total of 600 students. Seven teachers
and twelve aides were assigned to this project. In the
elementary grades (1, 2, 3), four teachers and six aides
taught 300 students reading and math for approximately

55 minutes in each subject. All elementary students were

90

assigned to project classroom for 55 minutes daily for
reading. Students were assigned on the basis of their
educational needs rather than by school-identified
classroom grade level. While student schedules were
flexible, all students met the daily time requirements.
For the purposes of this study, the reinforcing events

(R.E.) room is also considered a classroom area.

The 300 middle (7, 8, 9) school students were
assigned to a daily 45-minute period for reading. Three
teachers and six aides served them.

Fifty per cent Of the total staff (teachers and
aides) time was Spent in reading activities and 50 per
cent in mathematics activities.

The following was developed for the Alpha reading
prOgram using the SME formula, number of students (NS)

multiplied by instructional time (T) multiplied by 180
school days per year equals Student Minutes Of Exposure

(NS x T x 180 = SME).

 

Student Minutes of

Grade Level Exposure Reading

 

 

1 100 x 55 x 180 = 990,000
2 100 x 55 x 180 = 990,000
3 100 x 55 x 180 = 990,000
7 100 x 45 x 180 = 810,000
8 100 x 45 x 180 = 810,000
9 100 x 45 x 180 = 810,000

Total SME for Reading = 5,400,000

Total Program SME = 10,800,000

 

91

To determine accurate costs, it was necessary to
include the first grade level x student time (SME) as well
as the total project time Of 10,800,000 SME. Total SME is
the sum of both subjects (reading and mathematics) SME.

In the Alpha program, the six cost variables
of salaries, staff training, start—up costs, incentives,
facilities and materials and supplies are computed using

the total grade level SME as the common denominator.

Salary Costs

 

Salaries consist of that remuneration paid to pro-
fessional and non-professional staff members directly in-
volved in the programs. The salaries are totaled and
divided by the number of teachers to determine the average
salaries. Because there were an uneven number of teachers
for grades taught, salary cost statistics had to be con-
verted to "average" salaries for further development Of
the cost statistics. It was also argued by the chairman
of the author's committee that "averages" were more realis-
tic than attempting to prorate actual salaries Of individual
teachers from $8,200 in one classroom to $14,000 in
another. "Averages" tend to more effectively reflect a
regular school system situation in more realistically
establishing staff costs.

The average of the elementary teacher's salary in

the Alpha program is $9,437.50 for teaching reading and

92

 

Professional Staff Salary Data

 

Elementary Teacher Salary
A $11,750.00
B 9,800.00
C 8,000.00
D 8,200.00

 

$37,750.00 % 4 = $9,437.50

Middle Teacher Salary
E $ 8,000.00
F 8,000.00
G 8,400.00

 

$24,444.00 % 3 $8,133.33

 

mathematics. Because 50 per cent Of the project time is
spent in reading activities the average reading salary is
$4,718.75.

The average of the middle school teacher's salary
in the Alpha program for reading and mathematics is
$8,133.33. Average salary cost for teaching reading is
$4,066.00.

Paraprofessional salaries are also develOped on the
average for the elementary and middle school programs. The
average elementary paraprofessional salary cost is
$2,957.66. Fifty per cent of this cost is allocated to
reading. That cost is $1,478.00. The average middle
school paraprofessional salary cost is $2,676.00. The

cost allocated to reading is $1,388.00.

93

 

Paraprofessional Staff Salary Data

 

Staff Member Salary

$ 2,676.00
2,676.00
4,500.00
2,676.00
2,676.00
2,542.00

$17,746.00 % 6 = $2,957.66

”JUIUOCDB’

 

c $ 2,542.00
H 2,542.00
I 2,810.00
J 2,810.00
K 2,542.00
L 2,810.00

$16,056.00 4 6

 

$2,676.00

 

 

Summary of Staff Average Costs for Reading

 

Elementary
Professional $4,718.75
Paraprofessional 1,478.00

Middle School

Professional 4,066.00
Paraprofessional 1,388.00

 

To determine salary cost of instruction per minute
of student exposure, the average salary costs are divided
by the number of staff members, number of subjects taught,

and by the total grade level SME per subject.

94

The following chart represents these salary costs
(cost % number Of staff members % number Of subjects %

total SME = cost per SME):

 

Elementary Professional Salary (Grades 1, 2, 3)

$9,437.50 x 1-1/3 % 2 = $6,275.94 % 990,000 = $.006339

Elementary Paraprofessional Salary (Grades 1, 2, 3)

$2,957.66 X 2 3 2 = $295.66 % 990,000 = $.002987

Middle School Professional Salary (Grades 7, 8, 9)

$8,133.33 % 2 = $4,066.67 % 810,000 = $.005020

Middle School Paraprofessional Salary (Grades 7, 8, 9)

$2,676 x 2 f 2 = $2,676 % 810,000 = $.003303

 

Staff Training (Pre-Service
and In-Service

 

 

Staff training costs reflect those added costs
spent by the technological firm or school district for the
purpose of pre-service and in-service training of the staff
in the specific Operation of the Specific programs studied.

Alpha spent $1,688.00 for pre-service training.
Distributed over $10,800,000 SME (total project time), this
cost equals $.000156 per minute Of SME ($1,688.00 +

10,800,000 = $.000156).

95

Start-Up Costs

 

These costs represent district expenditures known
as start-up costs only. They include hardware supplies
such as teaching machines, etc. The regularly known
normal operating costs are not used because it has been
assumed that differences in these costs would be negligible
in the different prOgrams.92 There were no Alpha start-up

costs.

Incentives

 

Alpha student incentive costs totaled $4,946.50.
This cost divided by the total project SME (10,800,000)
equals $.000458 per student minute of exposure
($4,946.50 % 10,800,000 = $.000458).

Alpha staff incentive costs totaled $14,172.96.
This cost divided by the total project SME (10,800,000)
equals $.000386 per student minute of exposure $4,172.96 +

10,800,000 = $.000386).

Facilities

 

Facility costs are computed for only those costs
incurred because of required remodeling or those costs for
additional space other than that which is required for a
regular classroom. It is commonly accepted practice to
amortize such initial remodeling costs over a ten-year

period of time.

 

92Hecker, pp, cit.

96

Remodeling costs in the Alpha project were $217.70.
This cost divided by the total project SME and spread over
ten years equaled $.000002 ($217.70 % 10,800,000 t 10 =
$.000002).

The Alpha program for 600 students required the
equivalence of seven classrooms for instruction and three
additional classrooms for reinforcing events rooms. Assum-
ing that the regular class Size per room in Grand Rapids
is thirty, the 600 Alpha students would have required
twenty classrooms. Because students are only in the read-
ing program for approximately one-half of their instruc-
tional time, the required room space is ten classrooms.
Since ten rooms were used, it can be stated that no ad-

ditional costs for additional space was required.

Materials and Supplies

 

Materials and supplies costs include all Of the
software and other consumable items used in the program.

Alpha's cost for this item was $14,929.35. This
figure divided by total SME (10,800,000) equals $.001336
per student minute of exposure ($14,929.35 9 10,800,000 =
$.001336).

As was stated earlier, it was also agreed for pur-
poses Of this study, costs of system-wide administration,
custodial care, transportation, student support services,

facilities, and fixed costs across the system, are assumed

97

as being prorated equally among the programs studied, as
well as tO other programs Operating within the district.93
Variable Cost Data annual totals were computed
using specific salaries (see Variable Cost Data Chart 1).
The grade level totals equal the sum of the costs per
student minute of exposure in the six cost indexes: sal-
aries, staff training, start-up costs, facility costs,
student and teacher incentive costs, and material and
supplies costs multiplied by the SME for a particular
grade level. These figures were totaled and divided by
the number Of students in the project to determine the
annual cost per student. Thus, the annual cost per stu-
dent formula is the cost totals multiplied by the SME

divided by the number of students to determine the annual

cost per student.

 

Alpha Reading-—Comparative Cost Data per SME
(Cost x SME = Annual Cost)

 

 

 

 

Grade Level Total SME Cost Annual Cost
1 3.011664 $11,547.00

2 .011664 11,547.00

3 .011664 11,547.00

7 .010661 8,635.41

8 .010661 8,635.41

9 .010661 8,635.41
$60,547.23

93

Ibid.

 

98

 

 

 

 

 

ocwpuomnu>uooouco Ho>cq

H000H0.» H000a0.m H000«0.m H000AO.» vocaao.m vocaao.» v00a«0.m menace
0mm~00.m 0mma00.m 0mma00.» Omna00.m 0nma00.» 0nna00.m omnaoo.m wwwaddsm pad manauouux
0mv000.» 0mv000.m 0mv000.w mmc000.» 0mv000.w mmv000.w 0mv000.» ucupsum
00m000.m 00m000.» 00m000.m 00m000.m 00m000.» 00n000.m 00m000.m deceammowoum
”mo>aucoucn
~00000.m ~00000.m ~00000.» ~00000.» ~00000.» ~00000.» ~00000.m mowuwawuoh
0mH000.m 0m~000.m 0ma000.» 0ma000.» 0ma000.m 0ma000.m 0mH000.m ocacfiuua uuuum
"mumou douuuoum
m0mm00.» n)mm00.m m0mm00.» m0mm00.w n00~00.m 500~00.m h00~00.m HmcoAmmowoumuuucm
0~0m00.w 0~0m00.m 0~0m00.m 0~0m00.m mmm000.m 0mn000.m onm000.m Hmcodmmuuoum
”shadow uauauomm
mucoouuu an ouncw: ucapaum umm umou
nowadQSm one maowuoun:
mm.0~v.vH ucepsum
unscammououm
0m.0v0.v "noeaucuoau
om.mba.v nowadawoch
mcacaaus uuoum
05.nam mumoo dauuuoum
00.mmw.a
. . 00.0mm.mm 00.0mm.mm 00.0mm.Nm 00.0hv.am mm.0~v.~w 00.05v.am "umoo uumum guacammuuoumlcuom
NH N m m m m N quUm Aacofimm0uoumuaudm
. . 50.000.vm 50.000.vm no.000.vm mn.man.vm mn.0an.vm mn.0an.vm "umou uuuum Accodmmououm
s H H H m\aaa m\HnH n\ana wudum «occammououm
000.000.0a 000.0Hm 000.0H0 000.0H0 000.000 000.000 000.000 ousmomxm no
nuance: acouaum deuce
. . mv mv mv mm mm mm ousmomum
uo mousse: vampsum
000 00a 00a 00a 00a 00a 00a «acupaum u0 wanna:
uaauoa o m a n m a coauqauouen aeoeaum
uuenoum pad cued umou naduaaam

 

H BM‘ZU

usuuusm acacunoa HH ondH¢IIouaa umoo annuauu>

99

Reading prOgram cost % number Of students =
annual cost per student $60,547.23 € 600 = $100.91.

As previously stated, the following five other
programs (CMES, Westinghouse Learning Corp., Traditional
Remedial Reading, Project Read, and Control Schools) will
be reported using this same model. Detail or further ex-
planation will not be reported unless a deviation from

this model is necessary.

WESTINGHOUSE LEARNING CORPORATION,
PERFORMANCE CONTRACTING PROJECT
Elementary Program: Reading and Math
Grade Levels: 1—6
Professional Staff: 2

Paraprofessional Staff: 6

Students Enrolled

 

Grade Level

1 53
2 53
3 48
4 54
5 56
6 64

Total 328

Student Minutes of

 

Exposure (SME)

 

Grade Level

Staff Salaries

1

2

 

Professional

A

B

Paraprofessional

A

B

53

53

48

54

56

100

x 37.5

x 37.5

x 37.5

x 37.5

x 67.5

X

X

64 x 67.5 x

Total Reading

Total Program

$1

1,516.00

8,700.00

 

$2

$

0,216.00

2,542.00
2,542.00
2,542.00
2,542.00
2,542.00

2,542.00

 

$1

5,252.00

180

180

180

180

180

180

SME

SME

2

357,750
357,750
324,000
364,500
680,400

777,600

 

2,862,000

5,724,000

$10,108.00

6 = $2,542.00

101

As in the Alpha program, one-half of a staff
member's day is spent in reading activities, therefore,
50 per cent of that salary is allocated to reading.

Summary of Staff Averpge Costs
for Reading

 

 

Professional Salary

$10,108.00 $5,054.00
Paraprofessional Salary

$ 2,542.00 $1,271.00

Average of Specific Salary Costs
per Student Minute of Exposure

 

 

(Salary x Staff % Subjects e SME equals cost per SME)

Grade Level 1

 

Professional Salary
$10,108 x 1/3 % 2 = $1,684.66 % 357,750 = 5.004709
Paraprofessional Salary

$2,542 x 1 i 2 = $1,271 % 357,750 = $.003353

Grade Level 2

 

Professional Salary
$10,108 x 1/3 t 2 = $1,684.66 % 357,750 = $.00709
Paraprofessional Salary

$2,542 x 1 i 2 = $1,271 % 357,750 = $.003353

102

Grade Level 3

 

Professional Salary
$10,108 x 1/3 % 2 = $1,684.66

Paraprofessional Salary

$2,542 x 1 i 2 = $1,271 % 324,

Grade Level 4

 

Professional Salary
$10,108 x 1/3 t 2 = $1,684.66
Paraprofessional Salary

$2,524 x l % $1,271 % 364,500

Grade Level 5

 

Professional Salary
$10,108 x 1/3 % 2 = $1,684.66

Paraprofessional Salary

$2,542 x 1 i 2 = $1,271 % 680,

Grade Level 6

 

Professional Salary
$10,108 x 1/3 % 2 = $1,684.66
Paraprofessional Salary

$2,542 x 1 i 2 =

Staff Training

 

Costs e total SME = Cost per SME

$2,000 % 5,724,000 = $.000349

$1,271 % 777,

4 324,000 = 5.005200
000 = 5.003923

5 364,500 = $.004622
= 5.003487

6 680,400 = 5.002476
400 = $.OOl868

% 777,600 = 5.002166

600 = $.001635

103
Start-Up Costs

Cost % amortization time - total SME = Cost per SME

$8,000 % 3 % 5,724,000 = $.000466

Incentives (Student)
Cost % total SME = Cost per SME

$1,200 3 5,724,000 = $.000209

Facilities--Remode1ing

Cost 5 amortization time % total SME = Cost per SME

$10,500 i 10 % 5,724,000 = $.000183

Materials and Supplies
Cost é total SME = Cost per SME

57,000 % 5,724,000 = 5.001222

 

Westinghouse Reading Comparative Cost Data per SME
(See Chart II)

 

Grade Level Totals Annual Cost
1 5.010691 $ 3,824.71
2 .010691 3,824-71
3 .011552 3,742.85
4 .010538 3,841.10
5 .006773 4,608.35
6 .006230 4,844.45

 

$24,686.17

 

104

 

 

 

 

om~ooo.m Hasooo.» ammoao.» ~mmaao.» decode.» ”mooao.» nuance
mmaaoo.» NNNHno.m «waaoo.m -~Hoo.» -~aoo.m -~aoo.w nuaaddsm use Huauouqx
ooaooo.» mo~ooo.m momooo.m mo~ooo.» mo~ooo.m mo~ooo.u acoeaum
. . . . . . . . . . . . ascoamuououm
"60>AUCOOCH
nmaooo.m mmaooo.m mmaooo.m mmaooo.» nmaooo.m nmaooo.» manuaaauom
ovnooo.» uvmooo.m ovmooo.m memooo.» memooo.m Seneca.» acacaoue muoum
“numou mauuuoum
mmpaaOO.» momaoo.» smvmoo.m nmmmoo.m mmmnoo.m mmmnoo.» Hanoammououmruuuu
Scamooo.m osvaoo.m mmovoo.m oo~moo.m mesvoo.m oosvoo.u Hacoaumououu
“huaaum Owuauomm
>uoomuou up ouscaz ucopsum uom umoo
oo.ooo.sm moaammsm use uaaauouu:
oo.oo~.am ucoeaum
HGCmeMOWOum
umfi>wUCQOGH
unauaaaunm
oo.oom.o~w madcauua Luann
oo.ooo.~w mumoo mauuumum
. . oo.an~.dm oo.Ha~.Hm oo.Hn~.Hm oo.Hs~.Hm oo.Hn~.Hm oo.Hn~.Hm "umoo macaw HacoauuououEquaa
o H H H H H H uuoum Hocowmm0u0umlfludm
. . oo.vmo.am oo.«mo.~m oo.vmo.am oo.cvo.am oo.vwo.am oo.vmo.am ”umoo uuoum Hocoaumououm
N «\H «\H mxn «\H mxa nxa uuoum Haeoauuououa
ooo.v~h.m oom.~ss oov.omo oom.von ooo.vmm oms.smm omn.nmn cunnodxm no
nausea: ueaesum deuce
. . m.so m.so m.sm m.an m.em m.nn «unnodxm
MO m605CflZ ucovsum
mmm «0 mm cm me mm mm nucooaum uo nonssz
mAQUOF 0 m v n N H ¢OwuflENOucH “COflflum
uuonoum use even unoo uuauaaam

unaccemuuhuooouco Ho>oq

 

HH uudnu

noduauOQHOU ocwcudoq unaccocuuuoxnuauco umoo candaud>

105

Annual Reading Cost Per Pupil

 

Total cost % number Of students = Annual Reading Cost
per Student

$24,686.17 % 328 = $75.26

COMBINED MOTIVATION AND EDUCATIONAL SYSTEMS,
PERFORMANCE CONTRACTING PROJECT
Middle School Program: Reading and Mathematics

Grade Levels: 6-9

Professional Staff: 4
Paraprofessional Staff: 9
Students

Grade Level

6 190.5
7 124
8 120
9 50

Total 484.5

Student Minutes of Exppsure (SME)

 

Grade Level

6 190.5 X 45 X 180 = 1,543,050
7 124 x 45 x 180 = 1,004,400
8 120 X 45 X 180 = 972,000
9 50 X 45 X 180 = 405,000

 

Total per Subject 3,924,450

Total Program 7,848,900

106

Staff Salaries

 

 

Professional
A $11,600.00
B 8,000.00
C 9,000.00
D 9,400.00

$38,000.00 % 4 = $9,500.00

Paraprofessional
A $ 2,676.00
B 2,676.00
C 2,676.00
D 2,810.00
E 2,542.00
F 3,075.00
G 2,943.00
H 2,676.00
I 2,810.00

 

$24,884.00 % 9 = $2,764.88

Summary of Staff Average
Costs for Reading

 

 

Professional Salary $9,500.00 $4,750.00

Paraprofessional Salary $2,764.88 $1,382.44

Specific Salary Costs per
Student Minute of Exposure

 

 

(Salary x number of staff % number of subjects % SME =
Cost per SME)

107

Grade Level 6

 

Professional Salary
$9,500 % 2 = 4,750 i 1,543,050 = 3.003078
Paraprofessional Salary

$2,764.88 X 2.25 i 2 = 3,110.49 % 1,543,050 = $.002016

Grade Level 7

 

Professional Salary
$9,500 % 2 = 4,750 % 1,004,400 = 5.004729
Paraprofessional Salary

$3,110.49 x 2.25 i 2 % 1,004,400 = $.OO3097

Grade Level 8

 

Professional Salary
$9,500 % 2 = 4,750 S 972,000 = $.004886
Paraprofessional Salary

$3,110.49 X 2.25 i 2 % 972,000 = $.OO3200

Grade Level 9

 

Professional Salary
$9,500 % 2 = 4,750 % 405,000 = $.011728
Paraprofessional Salary

$3,110.49 X 2.25 i 2 S 405,000 = $.OO7680

Staff Training

 

Cost % total SME = Cost per SME

$1,500 % 7,848,900 = $.000191

108

Start-Up Costs

 

Cost % amortization time % total SME = Cost per SME

$51,000 % 3 = 17,000 % 7,848,900 = $.002165

Incentives (Student)

 

Cost % total SME = Cost per SME

$1,000 2 7,848,900 = $.000127

Facilities-~Remodeling

 

Cost % amortization time % total SME = Cost per SME

$9,500 % 10 % 7,848,900 = $.000121

Materials and Supplies

 

Cost % total SME = Cost per SME

$35,000 % 7,848,900 = $.004459

 

CMES Reading Comparative Cost Data per SME
(See Chart III)

 

Grade Level Total SME Cost Annual Cost Specific
6 $.012158 $18,760.40
7 $.014890 14,955.52
8 $.015150 14,725.80
9 .022024 8,919.72

 

$57,361.44

 

109

 

 

 

 

v~0-0.w omamao.» 0009a0.» mmHNHO.» nacuoe
$38.... $3.8.” $3.8.” $38..» 43396 05 3.235.
nm~000.w 5~H000.m hNHOOO.» n~H000.m unopsum
Hocoflmmuuoum
mo>aucoucH
HNH000.0 HNHOOO.» anaOOO.» a~H000.m moduaaauam
HOH000.m ~0H000.w a0a000.w HOH000.» mcwcwaua madam
00Hm00.w 00H~00.m 00HN00.m 00Hm00.m ”muaoo mouuuaum
000n00.w 00mm00.» >00n00.m 0a0m00.m moccammououmuoucm
0mnaa0.m 000v00.m m~nv00.m 0n0m00.m deceammououm
"Sucaum Owuwuodm
>uogouou an ouscwx ucopauw nod umou
00.000.mnm mouammnm pan maawuouut
00.0003“ m ucooSum
AucoflmmoNOum
"mo>wucoocu
00.00m.0 m mowuwawucm
00.00m.~ m acacwmua uuoum
00.000.Hmm mumou mouuueum
vq.~0m.am vv.~mn.aw vv.~mm.am vv.~mm.aw "umou muoum HucOAmmououmunucm
0 m~.~ mm.~ m~.~ m~.~ uuoum HucOMmmOuoumuuumm
. . 00.0mn.vm 00.0ms.vm 00.0me.vm 00.0mh.vw Humou uunum guacammououm
v A a H a uumum HacoAmmouOum
000.0v0.h 000.m0v 000.~no 00v.v00.a 0m0.mvm.a ouSmOme mo
mousse: unopaum Houoe
. . mv mv me me ouOuOme
mo mousse: uconsum
m.v0v 0m oma VNH m.00H mucopzum uo nonfisz
m m n 0

maouoa
younoum

 

onwpcoxuuuuooouou Ho>oq

noduoEMONcH unopsum
one dado uuou nuaueacm

 

maouuhm Aucowucuspm one sawue>auot penanSOUIIauco umou candwud>

HHH Hdeu

110

Annual Reading Cost
per Student

 

 

Total cost % number Of students = Annual reading cost
per student

$47,361.44 % 484.5 = $118.39

PROJECT READ--"CANNED" PROGRAM

Elementary Program: Reading
Grade Levels: 1—5
Professional Staff: 17

Paraprofessional Staff: 8

Students

Grade Level

1 66
2 58
3 95
4 83
5 _7_3

Total 375

Student Minutes of Exposure (SME)

 

Project Read teachers have documented that they
spend the required 60 minutes per day teaching reading.
The Grand Rapids Public Schools GREA Master Agreement
specifies 1,575 minutes of classroom exposure for elemen-
tary students in the total curriculum area. One day of

exposure of the professional or paraprofessional is equal

111

to 315 minutes. Reading time is 60 minutes Of this 315.
The relationship Of reading time to the total day's cur-
riculum is .190476 (60 % 315 = .190476). This factor will
be used as a determinant in allocating the costs for read-
ing.

Grade Level

1 66 x 60 x 180 = 712,800
2 58 x 60 x 180 = 626,400
3 95 x 60 x 180 = 1,026,000
4 83 x 60 x 180 = 896,400
5 73 x 60 x 180 = 788,400

 

Total 4,050,000

Staff Salaries

 

Grade Level 1

 

Professional
A $ 8,200.00
B 8,200.00
C 8,000.00

 

$24,400.00 % 3 = $8,133.33

Paraprofessional

B $ 2,542.00
C 3,075.00
D 2,810.00

 

$ 8,427.00 % 3 = $2,809.00

Grade Level 2
Professional
A
B
C

 

Paraprofessional
A
B

Grade Level 3

 

Professional
A

B
C
D

Paraprofessional
A
B

Grade Level 4
Professional
A

 

B
C
D

Paraprofessional
A
B

112

$ 7,800.00
9,800.00
10,900.00

 

$28,500.00

$ 3,075.00
2,676.00

 

$ 5,751.00

$ 8,400.00

8,400.00
10,300.00
10,300.00

 

$37,400.00

$ 2,676.00
2,542.00

 

$ 5,218.00

$ 8,700.00
11,600.00
11,600.00

8,200.00

 

$40,100.00

5 2,542.00
2,676.00

 

$ 5,218.00

3

2

$9,500.00

$2,875.50

$9,350.00

$2,609.00

$10,025.00

$2,609.00

113

Grade Level 5

 

Professional
A $ 7,800.00
B 8,200.00
C 9,008.00

 

$25,008.00 % 3 = $8,336.00

Summary Of Staff Average
Costs

 

Professional

Grade Level

1 $ 8,133.33
2 9,500.00
3 9,350.00
4 10,025.00
5 8,336.00
Paraprofessional
Grade Level

1 $ 2,809.00
2 2,875.00
3 2,609.00
4 2,609.00
5 -0-

The average of the specific professional and
paraprofessional salaries are computed by grade level in
Project Read because it is possible to determine that the
teachers and aides teach only students on that grade
level, whereas, in the performance contract programs,
salaries were computed on a total of grade levels served

since students were "in and out" of the program taking

114

other subjects in other classrooms utilizing the learning

center as Opposed to the self-contained classroom.

Average of the Specific
Salary Costspper SME

 

 

Grade Level 1

 

Professional

$8,133.33 x .190476 % 712,800 x 3 = $.006519
Paraprofessional

$2,809.00 x .190476 % 712,800 x 3 = $.002252

Grade Level 2

 

Professional
$9,500 x .190476 % 626,400 x 3 = 5.008664
Paraprofessional

$2,875.50 x .190476 % 626,400 X 2 = $.001748

Grade Level 3

 

Professional

$9,350 x .190476 5 1,026,000 x 4 = $.006940
Paraprofessional

$2,609 x .190476 e 1,026,000 x 2 = $.000969

Grade Level 4

 

Professional
$10,025 x .190476 % 896,400 x 4 = $.008520
Paraprofessional

$2,609 X .190476 % 896,400 X 2 = $.001180

115

Grade Leyel 5

 

Professional
$8,336 x .190476 - 788,400 x 3 = $.006039

Paraprofessional

Staff Training

 

NO costs

Start-Up

NO costs

Incentives

 

NO costs

Facilities

 

NO costs

Materials and Supplies

 

Software materials used are programmed for indi-
vidual student instruction. The school district pays the
technological firm $20 per student enrolled. There were
375 students enrolled in the Project Read Program in the
elementary school studied. Thus, the costs were $7,500.00
($20.00 x 375 = $7,500.00).

Costs % total SME = Cost per SME

$7,500 % 4,050,000 = $.001852

116

 

Project Read--Reading Comparative Cost per SME
(See Chart IV)

 

 

Grade Level Totals Annual Cost
1 $.010623 $ 7,562.07

2 .012264 7,682.17

3 .009770 10,024.02

4 .011561 10,363.28

5 .008791 6,930.82
$42,572.36

 

Annual Reading Cost
per Student

 

 

Total cost 4 number Of students = Annual cost per
student

$42,572.36 % 375 = $113.53

TRADI TIONAL REMEDIAL READI NG
Elementary and Middle School Program
Grade Levels: 1-7
Professional Staff: 16

Paraprofessional Staff: 0

Students

Grade Level Number
55
120
111
108
72
26

66
Total 558

\IO‘UT-bWNI"

117

 

amhmoo.

Nmmaoo.

ammooo.

Awmuaoo.

Nmmaoo.

omaaoo.
ONmmoo.

Ohhmoo.

ammaoo.

oomooo.
mvmwoo.

vaNHo.

ammdoo.

ovhnoo.
«$0000.

nNOOHo.

«modoo.

mmNNoo.
mamooo.

IHQUOB

«cassava can unannou-x
acousum
unscauuououm
"09>aucoucn
uoauaaauuu
acacauua uuauu
“auuoo maquuuu
Accowuuououmuuuam
HOSOHIQOuOBQ
"suuaam usuauonu

auooouuo an cased: uaouaum uoa u-ou

 

 

oom.hm

men

om.~mm.dm
n
ooo.omo.H
00

me

00.00vm
N

mm.mom.dw
v

00v.omm
00

n)

00.mmv»
N

m0.omb.dm
v
ooo.o«o.a
00

no

Hh.hvmm
N

Nm.mom.am
n
oov.oflo
am

am

vo.mmmm
n

o~.avm.dm
n

OOQ.NAh
co

wo

uoaaausm can .Hnauouux
uaouaum
Aegean-ououm

"00>«uaoucn

acauauaoam

uaaaauua «aqua

uuuoo nouuuaum

"uaoo madam assauuuououmucucm

«noun auscunOOuoumIcucm

"uaou uncum Hiccunuououm

uucum anneaoaououm
ousuonuu no

nousadx uccvsum H.009
nus-on!“

mo unusual ucdvsum

nucouavm uo Hon-:2

 

H1009

n

 

ocauaoxuusuomouao Hosea

acauqauoueu aeoeaum
can mung unoo nuance-m

 

ntflu POMbOflGII‘B‘G BmQU

DH hfl‘flU

NAM<H¢¢>

118

Student Minutes of E§posure (SME)

 

Student minutes of exposure per a given grade
level were computed by documenting the number Of students
exposed to all teachers teaching on that grade level and
multiplying that number by the number of minutes spent in
reading activities (see Chart V for computation).

Grade Level

510,840

2,303,640
1,779,660
1,132,560

821,700

353,340

627,480

Total 7,528,860

\lmU'I-waH

 

Traditional Remedial Reading
Staff SalarinOst Data

 

 

Professional Salaries

A $ 12,516.00
14,416.00
14,716.00
14,416.00
14,966.00
10,016.00
12,716.00
14,566.00
12,816.00
14,566.00
12,516.00
14,866.00
14,716.00
15,116.00

ZKIT'NCIHZBQ'TIHUOUJ

119

 

000 . mmm . h 0333: deuce occuo

OOhANw OOmNnHH Dogfish OQWMOMN ovmon HUG» u..—
cuaea: acousum

 

 

 

oovp~o ovnnmn
momv aoNo sumo moana onuN sea pom
cased: aconsum
omvn noon
5 “.3000?
m~on oom oou com com com com o Mono-es
”an an s coo no on snv mm a 66v mm a vow av o u I I 2 00:00.9
I I I I I I man no m oo~ as v mmv no a was mo as on“ vn m : none-on
I I I av «v a on" no a oNn we a u~m oo o mov Hm m m~ nu a a nose-op
u I I I I I com «o v" «vaa up a“ mov on o as 0 ~ ca“ Ha on u noses-a
I I u on on a ovn co m vvm no a 05v no u ~m~fi no an an on a n wagon-e
voN cod an N snv mm o emu an n ma mm a cu cu H I I I a mono-on
pad can vm a oHN vm v I I I I I - I I I mu ma a a acne-op
I I I I I I o- on s ohv on a mov oo 0 «He no a nma no va 0 nose-gr
I I I u n I own an m wan an v «Hun do“ «a man mu n so“ av s u acne-oh
I i I no pN n can ov v oov on ea ooa cod a mom so a u u I n unsavoa
u I I on on a an an a o~s co - van um va «an an a and on o a unnocoa
a I I I I I I I I cm” on m was vo o v~mv vs” o~ I I u o none-ca
I I I can on v on on a o~d on v ~mm on o nNo up nu an ad n n avenues
I u I I I I ow ca ~ I I I maau av” ma mama av nu cud on v < assuage
arm and. .53. Ba 3.2. .36. is 2:... .88 Ba I: .53 mum .5... .3m Ha I? .63 1 Ha 1.2. .3m
unnuo can a o n v n u a clu-

HOS

 

”33:3 Eh 39 as as gang

>3

120

Professional Salaries
0 $ 15,216.00
P 14,566.00

 

Total $227,706.00

There are no paraprofessionals employed in this program.

Due to the great variance in teaching schedules
and student minutes of exposure, as evidenced in Chart V,
it was necessary to determine the total per cent of ex-
posure time per grade level and determine its relationship
to the total SME. Teachers' total time (100%) is devoted
to remedial reading activities. As specified in Chart V,
total SME for all grade levels equals 7,528,860.

Factor development follows using the formula:
Grade level SME % total SME = Grade level salary alloca-
tion.

Grade Level

1 510,840 % 7,528,860 = .0678
2 2,303,640 % 7,528,860 = .3062
3 1,779,660 % 7,528,860 = .2363
4 1,132,560 % 7,528,860 = .1504
5 821,700 % 7,528,860 = .1091
6 353,340 % 7,528,860 = .0469
7 627,480 % 7,528,860 = .0833

121

Average of the Specific
Salary Costsgper SME

 

 

Grade Level Factor Salary Cost per SME
1 .0678 S 15,100.00 $.029559
2 .3062 68,193.00 .029602
3 .2363 52,625.00 .029570
4 .1504 33,495.00 .029575
5 .1091 24,297.00 .029569
6 .0469 ‘ 10,445.00 .029561
7 .0833 18,551.00 .029564

 

$222,706.00

Staff Training

 

No costs

Start-Up

No costs

Incentives

 

No costs

Facilities

 

No costs

Materials and Supplies

 

Costs % total SME = Cost per SME

$7,200 % 7,528,860 = $.000965

122

 

Traditional Remedial Reading Annual
Comparative Cost Data
(See Chart VI)

 

 

Grade Level Specific Salaries Annual Cost
1 $.030524 $ 15,592.88

2 .030567 70,415.36

3 .030535 54,341.92

4 .030540 34,588.38

5 .030534 25,089.79

6 .030526 10,786.06

7 .030529 19,156.34
$229,970.73

 

Annual Traditional Remedial
Reading Cost per Student

 

 

Total cost % number of students = Annual cost per
student

$229,970.73 % 558 = $412.13

CONTROL GROUP --READING
Control group reading activities consist of the

traditional methods used in the regular school.

Elementary and Middle School Program
Grade Levels: 1-3, 7-9
Professional Staff: 23

Paraprofessional Staff: 2.5

123

 

wNmOMO.

mwmooo.

memmo.

vanMO. ovano. nanno. homomo. eanno.

momooo. mooooo. moaooo. momooo. moaooo.

mommmo. ohmowo. Ohmmmo. «commo. Ommmwo.

.flvuoa

vowdamnm on: .Hvauouv:
aconsum
unscanuououm
“nobuuaoucn
aoauadvovm
unucavue uuvum
"nuaou naquvum
Hucowuuououmuouum
AndeanuouOum
"hudanm owuauoam

auooouqu an sauna: acousum uom avoo

 

 

oo.oo~.hw

OOIHmm.m~m oo.mvv.oam
wa n m
oom.mNm.h omv.h~o 00m.mmm

0mm mo om

oo.>m~.vmw oo.mmv.nnw oo.m~w.~mm oo.nmd.mow oo.oo«.maw
va ma va Va um
con.amm 0mm.n~a.a oo@.mhh.n ovo.n0n.u ov¢.o~m

.> uuonu men .cmflmoc EduoOua nocoauu Hoacw>vaA o» newouoouo mouud>u

Nb mow Add ONA mm

nowaunam cad macaw-an:
ucouaum
annuauaouOum
"uu>«ucoucn
nowuuauuom
nowadays uuuum
uuooo maquuum

”umoo uuuum auaovunouoaaIvuaa

uuvum auaOMnaououaI-uum

uumoo umdum «accusauuouh

uuuum ~1:0wunauoum
ousaomufl no

mouana: acouaum Hanna
«Managua

no nousca: uauoaum

nunvusum «0 non-52

 

m v n N A

 

HQUOB

venouumouhhooouuu Ho>0q

sowuqlu0ucu ucousum
can quad uuoo moan-aqm

 

UZHO‘HM 4<HOMIN¢ A4ZOHBHadflbtl‘H¢a FmQO nam<H¢4>

H> HI‘IU

124

Students
Grade Level Enrollment

123
121
125
364
350
270
Total 1,353

\OGDQUJNH

 

Staff Salaries (Average of
Specific Salary Deve10pment)

 

 

Grade Level 1

 

 

Professional

A $ 7,800.00

9,400.00

C 8,700.00

D 8,400.00

E 8,400.00

F 7,800.00

$50,500.00 % 6 = $8,416.60

Paraprofessional

A ,$ 3,075.00 (1/2 time) = $1,537.50

Grade Level 2

 

Professional
A $ 8,700.00
9,000.00
C 7,800.00
D 9,000.00
E 7,800.00
F 11,600.00

 

$43,900.00 % 6 = $8,983.33

Paraprofessional
A
B

Grade Level 3
Professional
A

 

MUCH!

Paraprofessional
A
B

Grade Level 7

 

Professional
A
B

Grade Level 8
Professional
A
B
C

 

Grade Level 9
Professional
A
B

 

125

$ 3,075.

2,626.

00
00

 

$ 5,751.

$ 8,000.
9,400.
10,900.
8,000.
11,600.

00

00
00
00
00
00

 

$47,900.

$ 2,943.
2,676

00

00

.00

 

$ 5,619

$14,416
11,516

.00

.00
.00

 

$25,932

$12,208.
8,700.
7,800.

.00

00
00
00

 

$28,708.

$11,900.
11,562.

00

00
00

 

$23,462.

00

2

$2,875.50

$9,580.00

$2,809.50

126

Student Minutes of Exposure (SME)

SME were computed by using the following formula:
SME = number of students x exposure time x 180 days. Be—
cause student time varies from classroom to classroom, it
was necessary to document each teacher's time report.
Example:

Grade Level 1

 

77 students for 60 minutes each and 46 students for
97 minutes each times 180 days = 639,360 SME
(77 x 60) = (46 x 97) x 180 = 1,634,760 SME.

Grade Level 2

 

(77 x 75) (44 x 110) x 180 = 1,910,700 SME

Grade Level 3

 

(85 x 80)

(40 x 60) x 180 = 1,656,000 SME

Grade Level 7

 

364 x 27.5 x 180 = 1,801,800 SME
Seventh grade students are exposed to reading
for one semester only, therefore, annual time

allocated was 27.5 minutes.

Grade Level 8

 

350 x 55 x 180

3,465,000 SME

Grade Level 9

 

270 x 55 x 180 2,673,000 SME

127

Total Reading SME

 

Grade Level
1,634,760
1,910,700
1,656,000
1,801,800
3,465,000
2,673,000
10,060,560 Total Reading SME

\DQQUJNH

 

Average of Specific
Salary Costs per SME

 

 

Computation formula:
Average salary x per cent daily time in program x
number of teachers % total SME = Cost per SME

Grade Level 1

 

Professional

$8,416.60 x .229629* x 6 = $11,596.17 %
1,634,760 = $.007093

Paraprofessional

$3,075 x .5* = $1,533.33 % 1,634,760 =
5.000940

(Two paraprofessionals spend one-fourth of
their time in reading activities.)

*Reading time computation:
315 daily minutes of exposure x 6 teachers =
1890 total daily minutes (60 minutes spent
by 4 teachers) = (97 minutes spent by 2
teachers) = 434 daily minutes of exposure to
reading.

434 % 1890 = .229692 daily per cent of time
spent by students in reading activities.

128

Grade Level 2

Professional

$8,933.33 x .235449* x 5.5 = $11,633.13 %
1,910,700 = $.006088

Paraprofessional
$2,875.50 % 1,910,700 = 3.001504

(Two aides spend 1/2 time in reading activi-
ties. This equals one full-time salary
allocation to reading.)

*Reading time computation:
315 x 6 = 1890 total daily minutes of ex-
posure (77 minutes Spent by 4 teachers) =
(110 minutes spent by 2 teachers) = 445 daily
minutes of reading exposure, 445 % 1890 =
.235449 daily per cent of time Spent by stu-

dents in reading exposure.

Grade Level 3

Professional

$9,580 x .22857l* x 4.5 = $9,853.69 %
1,656,000 = $.005950

Paraprofessional
$2,809.50 % 1,656,000 = $.001696

(One aide spends full time assisting teachers
in reading.)

*Reading time computation:
315 x 5 = 1575 total daily minutes of
exposure (80 minutes spent by 3 teachers) =
(60 minutes spent by 2 teachers) = 360 daily
minutes of reading exposure, 360 % 1575 =
.228571 daily per cent of time spent by
students in reading exposure.

129

Grade Level 7

 

Formula:

Specific salaries % total SME = Cost per SME
(There are no paraprofessionals assigned to the
secondary level.)

$25,932 % 1,801,800 = $.014392
Grade Level 8

 

 

$27,708 % 3,465,000 = $.008285
Grade Level 9
$23,462 % 2,673,000 = $.008777

Staff Training

 

No costs

Incentives

 

No costs

Facilities

 

No costs

Materials and Supplies

 

Cost % total SME = Cost per SME
The costs for materials and supplies represents

the school system's projected prorata costs. It was
apparent in seeking cost information that the school
system could not break out the exact cost per subject
matter, but costs have been develOped for this study
by the Board of Education Business Office by project-
ing actual costs for the year previous for the follow-

ing year (the year under study).

130

The following method of determining costs was
develOped by James J. Gallagher, Director of Budget
and Finance for the Grand Rapids Public Schools.

369 elementary students (grade levels 1-3) and
984 middle school students equals 1,796 students.

Class teaching supply costs per building (1969-70):

$12,078.50
3,974.57
3,310.04
$19,363.11

 

Using the 6-1/2 per cent increase per year, the
1970-71 instructional supplies cost can be pro-
jected to be $20,621.00 ($19,363.11 x .065 =
$20,621.00). One-fifth of this cost can be
allocated to reading, $20,621 x 1/5 = $4,124.50.

The per capita cost for textbooks is $14.00.

Using five subjects as the basis, reading supplies
represents one-fifth of the cost, or $14.00 x

1/5 = $2.80. This cost for 1,353 students at
$2.80 per student equals $3,788.40.

Totaling the instructional supplies cost and the
textbook costs, the total spent is $7,912.60
($4,124.20 + $3,788.40 = $7,912.60).

$7,912.60 total cost per subject % 1,796 students
equals $4.40 cost per student.

369 elementary students in grade levels 1-3 and
984 students in grade levels 7-9 equals 1,353
students in the research design.

$4.40 cost per student x 1,353 students equals
$5,953.20 cost for materials and supplies.

Cost % total SME = Cost per SME
$5,953.20 % 13,141,260 = 3.000453

131

 

Control School Reading Comparative Costs
(See Chart VII)

 

 

Grade Level Totals Annual Cost
1 $.008486 $ 13,872.57

2 .008045 15,371.58

3 .008099 13,411.94

7 .014845 26,727.72

8 .008739 30,277.17

9 .009230 24,671.77
$124,352.77

 

Annual Reading Cost per Student

 

Cost % student = Cost per student

$124,352.77 % 1,353 = $91.91

Reading Annual per Student
Cost Summary

 

 

 

 

Program Annual Cost
Alpha $100.91
Westinghouse 72.26
CMES 118.39
Project Read 113.53
Traditional Remedial Reading 412.13

Control 91.91

 

 

Ommmoo. mnbmoo. mvmvao. mmomoo. mvomoo. wmvmoo. namuoa

mmvooo. mmvooo. mmvooo. mmvooo. mmwooo. mmvooo. moaamQSm can maawuounz
ucoonum
HocOAmmououm
"mo>aucoucu
mowuuaauom
maficfiaue uuuum
"mumoo monuumum
. . . . . . wmofloo. ovmaoo. ovmooo. Hoseammououmnounm
hhhmoo. mmmmoo. mmmvao. ommmoo. mmoooo. mmonoo. acconmouOum
"xumHom aduwuomm

huomuumu an ounce: ucunaum uum umou

 

 

132

o~.mmm.mm moaammsm 0cm manauoun:
ucoosum
«uncamnuuoum

u m0>wUG¢USH

moaufiaaoom

wcacauua aunum

auuoo nquuuum

 

 

om.mom.~m mm.mnm.~m mm.mmm.am "umoo uuuum MaccammououmIouum
o.a o.~ m. uuaum aac0ammououmonumm
00.nov.mmm oo.m0h.p~m oo.~mm.mmm om.mmm.¢w vH.nmo.Ham ha.omm.a~w "umou wuaum HocOAnuowoum
mm N m m m.v m.m o uuuum Aucodmmowoum
oom.owo.oH ouo.mam.~ ooo.mov.m oom.Hom.H ooo.omo.a ooa.o~m.a ooa.vno.a ouamomxm uo
mousse: uconaum Hauoe
AGOAuouanloo new ovum uno«>oum oounuam«u¢>. ousuonxu
uo nausea: ucavsum
arm omn won mma ANA nNH mucuuaum no honaflz
m m h m w a
Houoe coAuoauOucH ucoosum
, mcflcmomuuxuooouou Hu>oq use sumo umou auwuoaom

 

ADOZU AQflBZOUII‘H‘O BmOU mdthm£>

HH> hfl‘lu

133

Per Program Student Achievement Gains

 

The second part of this chapter is intended to
indicate the results of the study of student achievement
gains which has previously been described. This section
is composed of two major sections: (1) a reporting of
the statistical analyses which compared the various read-
ing programs based on their respective mean gains in grade
level equivalencies, and (2) a descriptive analysis con-
cerning the cost-effectiveness analysis of each of the

various reading proqrams.

Statistical Analyses of Gain Scores

 

Pre and post reading achievement tests (see Appen-
dices for specific information) were administered by an
outside "evaluator," Battelle Memorial Institute, to each
pupil in each of six different reading programs in the
Grand Rapids Public Schools during 1970-71 school year.

As can be readily determined from Table 1, not every grade
level was exposed to the same treatments. The second and
third grades were exposed to five different treatments:
Alpha Program, Control, Traditional Remedial Reading,
Westinghouse Learning Corporation Systems, and finally,
Project Read. The seventh graders participated in four
different programs: Alpha, Control, Traditional Remedial
Reading, and CMES. A third group, the eighth and ninth
graders, were exposed to only three treatments: Alpha,

Control, and CMES. This unequal assignment of treatments

134

TABLE l.-—Distribution of Reading Programs Across Grade Levels
(Numbers of Students)

 

 

Traditional Pro'ect
Grade Control Remedial Alpha 3 Westinghouse CMES
. Read
Reading

2 73 24 75 41 28 --
3 69 31 81 54 24 ~-
7 71 22 81 —- -— 55
8 74 -- 71 -- -- 35
9 63 —- 65 —- -- 27

 

prohibited any overall analysis covering all of the data
at one time. Thus, three separate analyses of variance
were performed on the data: (1) second and third grades,
(2) seventh grade, and (3) eighth and ninth grades. In
each case, the dependent variable under investigation was
the gain from pre to post on a reading achievement test.
In each, the treatment factor was the independent variable
of major interest. In the second and third and eighth and
ninth grade studies, however, the grade factor was included
in the analysis to increase the precision of the tests for
treatment main effects.

Analysis on Second and
Third Graders

 

 

Table 2 illustrates the statistical design employed
in the analysis of the gain scores for second and third
graders. The cell means for each of the ten cells speci-

fied in the design are reported in Table 3. Examination of

135

TABLE 2.--Statistical Design Employed in Analysis of
Achievement Gain Scores for Second and Third

 

 

Grades
Traditional Pro'ect
Grade Alpha Control Remedial 3 Westinghouse
. Read
Reading
2 7585 7385 2485 4185 2885
3 8188 6933 318$ 548$ 245$

 

TABLE 3.--Mean Gain Scores by Cell for Second and Third
Grade Analysis

 

 

Traditional Pro'ect
Grade Alpha Control Remedial 3 Westinghouse
. Read
Reading
2 .4453 .7041 .7708 .6927 .5643
3 .5296 .3797 .8774 .6963 .6615

 

Table 3 indicates that each program at each grade level

did show a positive amount of average gain, although none

showed an average gain of one grade equivalent or more.
Table 4 presents the traditional analysis of

variance table for the second and third grade study. It

is clear from the table that there is neither a significant

grade by treatment interaction nor a significant main ef-

fect for grade. These two findings indicate that the two

grades are not responding in any significantly different

way from one another across the five treatments. There is,

136

in fact, some evidence that the use of the grade variable
did little, if anything, to increase the precision of the
test for the effects of the treatment variable.

TABLE 4.--Analysis of Variance on Pre and Post Reading

Achievement Gain Scores for Second and Third
Grade Programs

 

 

Source of Sums of Mean
Variance Squares d'f' Squares F Prob.
Program 5.7312 4 1.4328 4.7672 .0009
Grade .0234 1 .0234 .0777 .7805
Program x
Grade
Interaction 1.0836 4 .2709 .9015 .4629
Error 147.8952 492 .3006

 

Table 4 does indicate a significant program effect.
The means for the five treatments could only have been as
different as they were found to be in the given sample by
chance along 9 times out of 10,000. Thus, there are strong
indications that the means of the different programs are
significantly different.

Given that there were significant differences be-
tween programs, the next logical step in the analysis was
to investigate which programs were more or less effective
than others. With unequal cell size and the possibility

of multiple comparisons, the Scheffé post hoc technique was

137

94 The Scheffé pro-

selected for this part of the analysis.
cedure is one of several similar post hoc methods which
successfully control for "experimentwise" error rate. The
means across prOgrams were obtained by weighting the means
of the two grades using the numbers of subjects within

each grade level. These new means are given in Table 5.
The means from Table 5 were employed in the Scheffé post
hoc procedures. All post hoc techniques were tested with

a preset error level of .05 or 5 out of 100 possibilities
that a significant finding could have occurred by chance
alone. Table 6 shows the values of the desired sample
contracts and the calculated values needed for significance
under the Scheffé procedure. In order for the contrasts

to be significant at the .05 level, they must be greater
than the value calculated by the Scheffé technique.

TABLE 5.--The Gain Score Means for Programs Combined Across

Second and Third Grades for Use in the Scheffé
Post Hoc Procedures

 

Mean Gain in Reading

 

Program Achievement
Control .6437
Traditional Remedial Reading .8309
Alpha .4483
Project Read .6947
Westinghouse .6111

 

 

4Roger B. Kirk, Experimental Design: Procedures
for the Behavioral Sciences (Belmont, Ca1if.: Brooks Cole
Publishing Co., 1968), pp. 90-91.

 

 

138

TABLE 6.--Scheffé Post Hoc Analyses on Second and Third

Grade Sample

 

 

Sample Contrast Calcuéztfig Scheffé Signifggant
Y3 - 71 = .1954 .1961 Significant
Y2 - Y1 = .186 .269 n.s.
Y5 - fl - .0334 .271 n.s.
i4 - Y1 = .0502 .225 n.s.
_ i4 + X5
X1 --——§——- = .0134 .203 n.s.
2 - Y4 + Y5 = .1730 .270 n.s.
2 2
Y3 - Y2 = .3826 .265 Significant
x3 - x4 = .2464 .220 Significant

 

Note: i1 = Mean for Control

xl
u

2 Mean for Traditional Remedial Reading

Y3 = Mean for Alpha
Y4 = Mean for Project Read
is = Mean for Westinghouse Project

139

Examination of Table 6 reveals that the performance
of students enrolled in the Alpha program was significantly
lower than the performance of students in the Control
group, in the Traditional Remedial Reading program, and in
the Project Read program. There are no significant differ-
ences in a positive direction between any of the planned
and the control group.

Analysis on the Seventh
Grade Programs

 

 

As has been mentioned earlier, the seventh graders
were exposed to four reading programs during the 1970—71
school year. These four programs included Alpha, Control,
Traditional Remedial Reading, and CMES. A simple one-way
analysis of variance (ANOVA) was performed on the data
obtained from the seventh graders. The program means are
both contained in Table 7.

TABLE 7.--Means for the Analysis of Reading Achievement
Gain Scores for the Seventh Grade Sample

 

 

Traditional
Alpha Control Remedial -~—CMES
Reading
.5074 .3859 .1500 .7273

N=81 N=71 N=22 N=55

 

140

The results of analysis of variance for the seventh
grade is found in Table 8. The independent variable of
interest is, of course, programs, and the dependent vari-
able is, again, gain scores. There is a significant pro—
gram effect with a reported probability of less than .0328.
TABLE 8.-—Analysis of Variance Table for the Analysis of

Reading Achievement Test Gain Scores for the
Seventh Grade Sample

 

 

. Sums of Mean
Source of Variance Squares d.f. Squares F Prob.
Between (Proqram) 6.4326 3 2.1442 2.97 .0328
Within (Error) 162.4725 225 .7221

 

A set of Scheffé post hoc analyses were performed
to attempt to determine where the significant differences
between proqrams might be found. All simple contrasts
were performed and each was insignificant (see Table 9).
Thus, although there were significant differences between
programs, the differences present must have been of a more
complex nature. There is no guarantee that any further
investigation or more complex contrasts would result in a
significant finding, and, in addition, there is even less
chance that such a complex contrast would be interpretable
in any meaningful manner. In other words, the differences
might be attributed to teacher attitudes, amount of mater—

ials, carpeting, or other such undocumented items.

141

TABLE 9.--Scheffé Post Hoc Analyses on Seventh Grade
Sample

 

Calculated

 

 

 

 

Sample Contrast Scheffé Slggifiggnt
Value ‘
X1 - X2 = .1215 .3866 n.s
X3 - X2 = .2359 .5792 n.s
X4 - X2 = .3414 .4263 n.s.
Xl - X3 = .3574 .5706 n.s
X4 - X1 = .2199 .4145 n.s
X3 - X4 = .5773 .5988 n.s
Y + x + Y
i2 — l 33 4 = .0776 .3561 n.s.
i+§+§
2 - 1 2 4 = .3902 .5318 n.s
3 3
Note: Y1 = Mean for Alpha Program
Y2 = Mean for Control Program
Y3 = Mean for Traditional Remedial Reading Program
Y = Mean for C.M.E.S. Program

142

Analysis on the Eighth and
Ninth Grade Programs

 

 

Within the eighth and ninth grade sample, there
were three programs: Alpha, Control, and CMES. This
allowed for a 2 x 3 analysis of variance with two levels
of grade and three levels of program. The analysis of
variance employed tested the significance of the effects
for grade, for program, and for the interaction between
the two. Table 10 presents the statistical design em-
ployed on the reading achievement gain scores.

TABLE lO.--Statistical Design Employed in Analysis of

Achievement Gain Scores for Eighth and Ninth
Grade Sample

 

 

Grade Alpha Control CMES
8 7185 7485 3583
9 6585 6385 2783

 

Table 11 reports the means for each of the cells
as presented in the statistical design. An examination
of these means does point out that there were two cells
that did average over one year grade gain in reading (both
on the eighth grade level): one in the Alpha program, the

other in the CMES program.

143

TABLE ll.-—Mean Gain Scores by Cell for Eighth and Ninth
Grade Sample

 

 

Grade Alpha Control CMES
8 1.0760 .3081 1.1171
9 .8415 .4302 .6148

 

The analysis of variance table (Table 12) indicates
that there were no significant interactions between grade
and program. The same table reports a nonsignificant grade
effect. But again, as was found earlier, there is a signi-
ficant effect for program reported. The difference between
the program means obtained in this sample could have oc-
curred by chance alone only one time out of 10,000.

TABLE 12. Analysis of Variance Table for the Analysis of

Reading Achievement Test Gain Scores for the
Eighth and Ninth Grade Sample

 

 

. Sums of Mean
Source of Variance Squares d.f. Squares F Prob.
Program 27.3476 2 13.6738 10.91 .0001
Grade 1.5788 1 1.5788 1.260 .2625
Program x Grade
Interaction 4.6406 2 2.3203 1.852 .1586

Error 412.2370 329 1.2530

 

144

The Scheffé post hoc procedure was employed to
attempt to tease out the location of the differences
between programs. The means for programs were calculated
across grades by weighting with the sample sizes within

each grade level. These program means are presented in

Table 13.

TABLE l3.--The Gain Score Means for Programs Combined
Across Eighth and Ninth Grades for Use in
the Scheffé Post Hoc Procedures

 

 

PrOgram Mean Gain in Reading Achievement
Alpha .9640
Control Group .3642
CMES Program .8984

 

Table 14 presents the results of the Scheffé
analysis. Here, the tests indicate a significant differ-
ence between both the Alpha and the CMES proqrams and the
Control Group. There was, however, no difference between
the means for the two performance contracting programs.

Descriptive Analysis of Proqram
Cost-Effectiveness

 

 

The remaining section of this chapter is devoted
to the descriptive analysis of the respective cost-effec-

tiveness of each of the programs studied.

145

TABLE l4.--Scheffé Post Hoc Analyses on the Eighth and
Ninth Grade Sample

 

 

Calculated Scheffé Significant
Sample Contrast Value at .05
Y1 - Y* = .5998 .3312 significant
Y3 - Y2 = .5342 .4183 significant
X1 - X3 = .0656 .3728 n.s.

 

Note: 21 Mean for Alpha Program

XI
n

2 Mean for Control Program

XI

= Mean for CMES Program

A comparison of average mean gain in the reading
programs studied and the annual instructional costs of
those programs were computed in terms of costs per one-
tenth (.1) grade gain. This was done by the simple arith-
metical formula of dividing the annual average one-tenth
grade gain of the students in each grade level block
(grades 2, 3, 7, 8, 9) of the specific program under study
divided into the annual cost per student as established in
the first part of this chapter.

The following table is enclosed for easy analysis

of the data.

146

 

 

vm.m~m mH.mH m hv.oa m cflmw H. umm umou
~vm.m vmm.m ovm.m cflmu mpmuo nucwuumco
mvmm. vmmm. ovmm. cflmu cmmz
mIm mmemuo
mm.m~m mm.ma m mn.vnmw mm.mH m came H. 06m umoo
mmm.m mhm.n oom.a vuo.m cflmo momuu sucmuumco
mmmm. mama. coma. vaom. same :60:
a 06600
mm.va oo.mv a vm.6H m mm.HHm Hm.mm m 6360 H. 666 0660
emv.m mom.m avm.m HHH.m mmv.v came memno nucmuImco
amvm. momm. avmm. HHHG. mmvv. came :60:
MIN mmomuo
Hm.Hmm mm.mHHm mH.~va mm.maam mm.mnm Hm.OOHm unmesum umm umoo Hmscce
Houucoo mmzo .m.m.e 666m .6 use madam Emumoum

 

cflmw ucwEm>mH£o<

ucmosum augmelmco Hog umoo Hmscc¢I|.mH mamde

147

Summary

This chapter began with the rationale and develop-
ment of the student costs per year for the six reading pro-
grams under study. Per student instructional costs were
develOped.

The second part of the chapter dealt with an analy-
sis of the grade gain achievement records of the students
in the several programs to determine any significant dif-
ferences among the programs at the different grade levels.

The third portion of this chapter dealt with the.
equating of grade gains to costs of the programs under
study.

The next, and final chapter, summarizes the results
of the investigation along with conclusions, implications,

and recommendations for further studies.

CHAPTER V

CONCLUSIONS, IMPLICATIONS,

AND RECOMMENDATIONS

Introduction

 

The major purpose of the study was to determine a
cost-effectiveness relationship between inputs (in—
structional cost) and outputs (student achievement gains)
in six reading programs in the Grand Rapids Public Schools.
Subsequently, the design shown could establish a model
upon which school officials can make future decisions
regarding the use of resources available.

The inputs (instructional costs) were determined
on a basis of student minutes of exposure (SME) in six
cost categories that were defined as being the most
appropriate definable variables that constitute the
differences in costs among the programs studied. The six
cost categories were identified as staff salaries, both
professional and paraprofessional; start-up costs; added
facility costs; staff training costs, incentive costs,
and costs for materials and supplies. A per student

annual cost for the reading programs studied included

148

149

three performance contracting programs: Alpha II, Westing-
house Learning Corporation (WLC), Combined Motivation and
Education (CMES); Project Read--a publishing company's
(canned) program; Traditional Remedial Reading and a
regular (control) school program.

A pre/post testing program measured the student
achievement gains in the reading programs studied.

A relationship between costs and gains (inputs and
outputs) was established to compute the cost of one-tenth
year (.1) or one month student achievement gain in reading.

The relationship is summarized for easy reference
in this final chapter; conclusions are listed; implications
of the findings are discussed; limitations and suggestions
for further research are mentioned; and the chapter is

concluded with final comments by the researcher.

Review of Cost-Effectiveness Results

 

As evidence in Table 15, Chapter IV, the cost for
one-tenth (.1) student grade gain in the six reading
programs is:

Elementarnyrades 2 and 3

 

Alpha $ 22.51
Westinghouse Learning 11.82
Project Read 16.34
Traditional Remedial Reading 49.60

Control (Regular Program) 14.28

150

Middle School Grade 7

 

Alpha $ 19.89
Combined Motivation and Education

(CMES) 16.23
Traditional Remedial Reading 274.75
Control (Regular Program) 23.82

Middle School Grades 8 and 9

 

Alpha $ 10.47
Combined Motivation and Education

(CMES) 13.18
Control (Regular Program) 25.24

Interpretation of Results
and Conclusions

 

 

As previously stated, all grade levels were not
exposed to the same reading program treatment. Because of
this different grade and program treatment, it was
necessary to define three levels of reading instruction
within the design. They were (1) elementary grades [2 and
3], (2) seventh grade, and (3) eighth and ninth grades.
The following conclusions and implications of the previ—
ously listed data have also treated these three groupings

separately.

Specific Results of the Study

 

Early Elementary Grades
(2 and 3)

 

 

1. All the reading programs under study in the early
elementary grades resulted in less than one year

reading achievement grade gain.

151

There were no significant differences in grade
gain achievement among the five programs (Alpha,
WLC, Project Read, TRR, and Control), within the
two studied grade levels (second and third grades),
i.e., the second grades performed as well as the

third graders in a specific program.

There is a significant difference in reading
achievement of the several programs studied. The
performance of students enrolled in the Alpha
program were significantly lower than the students
in the control group program, Traditional Remedial
Reading program and Project Read Program. However,
there were no significant differences in a positive
direction between any of the planned programs and

the control group.

Cost—effective analysis leads one to conclude that
Traditional Remedial Reading programs are the most
costly (more than four times as costly as the
least expensive), with no significant positive
direction toward greater achievement except over

Alpha and control.

The Westinghouse program is indicated to be less
expensive per grade gain but it, too, has a mean
achievement gain level limitation of less than the
usual expected one year gain per one year of

instruction.

152

Seventh Grade Programs

 

1.

There is a significant program effect with the
achievement gain of the programs being determined
in this order from the lowest to highest--
Traditional Remedial Reading, control, Alpha,

CMES .

The Traditional Remedial Reading program had the
highest cost per achievement grade gain (almost
twelve times the control group cost) with the
Alpha and CMES programs less costly per gain than

the control group.

A comparison of learning gains with costs shows
that for this grade level the CMES program results

in the most cost-effective yield.

Eighth and Ninth Grade Programs

1.

There was a significant difference between the
control group and both Alpha and the CMES programs

in favor of the latter.

There was no significant difference between the

results of the eighth and ninth grades.

The control group was the most expensive group per
one-tenth grade gain with the Alpha program being
only slightly over 41 per cent the cost of the
control group and the CMES program cost being 68

per cent of the control group while each of the

153

latter two programs, Alpha and CMES, also having
a significant difference in achievement positively

in favor of the less expensive programs.

General Conclusions

 

Early Elementary Grades
(2 and 3)

 

 

The statistical analysis of each of the five early
elementary programs shows a positive amount of "average
gain" although none showed an average gain of one year
equivalent or more. However, the "means" for the five
programs indicate a significant difference between some of
the programs. When the grade gain is related to the cost,
the results suggests that Westinghouse, Project Read, and
the control reading programs are the most cost-effective,
i.e., producing the greatest gains for less cost. All
three are within a $4.00 cost per one-tenth grade gain.
However, the control group was over 20 per cent more
expensive than WLC, Project Read was over 38 per cent more
expensive. This would suggest that in addition to costs,
student needs and learning patterns might be the additional
determinants as to which program a school district may
choose. The overall mean gain for Traditional Remedial
Reading was considerably higher than the other four
programs. The cost is over four times as much per unit of
grade gain as the least costly program. This leads one

to very seriously question the feasibility and practicality

154

of continuing the current TRR program in this grade. The
WLC program was the most cost-effective program, i.e., the
least cost per the amount of grade gain. One could
speculate arithmetically that twice the amount of dollars
invested in the WLC program might result in twice the
student gain and still be only half the cost-effectiveness
of the Traditional Remedial Reading. However, the process
of student learning has not been proven to necessarily

follow in the same proportion as in the increased costs.

Middle School--Seventh Grade

 

The seventh grades were exposed to four reading
programs, Alpha, CMES, TRR, and control. In terms of the
mean gains there were significant differences between all
four programs. When these gains are related to the costs
in the cost-effectiveness design, it is clearly evident
that CMES performance contract program provides the most
learning for the dollar expended followed by the Alpha
performance contract program, the control group, and the
most expensive program being the TRR program. The control
group performed half as well as the performance contracting
programs under study. The Traditional Remedial Reading
Program appeared to be quite ineffective, both achievement
and cost wise. In other words, the two performance
contracting programs and control groups appear to be more
cost-effective than traditional remedial reading, which

was eleven times as costly as the next highest cost group.

155

TRR was seventeen times as costly as the most cost-

effective program, CMES.

Eighth and Ninth Grades

 

Three reading programs, Alpha, CMES, and control,
were studied for these two grade levels. An examination
of the reading achievement grade means points out that the
two eighth grade performance contracting reading programs;
Alpha and CMES, students achievement gains were slightly
above one grade level, whereas the control group only
gained approximately one-third as much. At the ninth
grade level, the same pattern existed but not as pronounced.
Applying the cost-effective formula to these student gains,
a school district decision-maker would be encouraged to
select one of the two performance contracting programs
over the control program. In any event, he would have

additional information from which to make decisions.

Discussion of Findings

 

The writer would like to mention and speculate on
the impact of some of the uncontrolled variables that may
have affected certain of the findings.

It has been common knowledge that performance
contracting has probably been advanced as an accountability
design. This movement has been improperly interpreted as
threatening to some teachers and teacher groups. Much

hostility had surrounded some aspects of this research

156

which was also experienced by others studying and evalu-

ating performance contracting. Some of the more critical

problems faced by this researcher are listed below.

1.

Governmental design. The Office of Economic

 

Opportunity (OEO) selected portions of the research
evaluation design and identified the student popu-
lation as an adjunct to the CEO performance
contracting project. While rigid design is
accepted practice for scientific research, one
questions the necessity for such rigidity in terms
of treatment of the program design. It was clearly
seen early in the research study that the Alpha
elementary (grades 1, 2, and 3) reading approach
was basically remedial in nature and may not have
been an appropriate treatment for the initial
learning process for very young children. The
overall study of the Alpha achievement gains
indicates better results and decreasing costs as
the grade and age level of the student rises

(e.g., second grade compared to eighth and ninth
grade). However, governmental contractual
constraints prevented the firm from making any
major program or curriculum changes during the

contractual period.

It appeared to this researcher that the elementary

control staff may have expended additional effort

 

157

in teaching reading during the period of time the

 

program was studied. Teachers may have been
motivated to compete with the other programs
studied. One wonders if such commitment will be

continuous.

The elementary Traditional Remedial Reading student

 

sample may be suspect because the performance
contracting programs and prepackaged program were
operating in the very lowest performing schools in
the district. Thus, the lowest producing students
were enrolled in these programs. The TRR groups
were also in low performing schools but did not
fully qualify nor were eligible for the other
programs. In other words, the TRR student chosen
may have been from a slightly higher potential
universe. In addition to the TRR program, it is
assumed that the TRR students received some
additional reading instruction in the regular

classroom which was not considered in this study.

All groups studied were aware that they were being
studied which could have provided a "Hawthorne"
effect or "halo" effect in varying degrees among
the programs depending upon the perception and

concerns of the staff and students involved.

158

5. The apparent failure of significant gains in

achievement of the TRR program at the seventh

 

 

grade level, clearly indicated that somewhere
between grade 3 and grade 7, the point of di-
minishing returns of the program may be met but
not determined due to the limitations of this

study.

6. The failure of a significant gain in achievement

of the eighth and ninth grade control groups

 

indicates that reading skill classes may be more

 

helpful for those students not performing at or
near grade level at that age group level, than the

traditional literature and English classes.

Limitations of the Study

 

It must be recognized that the initial year of
operation of the performance contracts studied may have
placed further limitations of success or failure due to
the newness of the concept, delays in "start-up," etc.

All of the other programs have been in existence for three
or more years in the schools studied.

There currently exists much controversy over the

appropriateness of the testing vehicles. The CEO con-

 

tractors in their "Joint Summary Statement,‘ question the

relevance of the data collected due to the selection of

159

the standardized tests which test general educational
achievement rather than reading skills as a separate
knowledge area.
The contractors state that:
General measures of education achievement do not
measure basic learning skills or basic knowledges;
either in toto or, which is more to the point, in
their separate subject area subtests; they measure
instead a wide range of highly complex skills in

somewhat cursory fashion.9

Testing conditions within the local school districts

 

including Grand Rapids were cited by the CEO Project
Directors as being impractical. They substantiated the
variation in conditions by citing inadequate pre-planning
time, an inability of some local districts to perform mass
pre-testing due to a lack of space, personnel, materials,
etc., within a ten day period, and the use of hurriedly
trained substitute teachers as testers as the three major
weaknesses of the testing program.97
Similarily the Grand Rapids performance contractors

also experienced the logistical problems of becoming

operational. James A. Mecklenburger, nationally recognized

 

95OEO Contractors, "Joint Summary Statement" (un-
published paper presented to CEO, March 15, 1972), pp. 7-9.

96Ibid.

97OEO Project Directors, "OEO Project Director's
Evaluation Chapter," submitted to the CEO, December 22,
1971, pp. 11-12.

160

expert on performance contracting states the OED experiment

 

design was inadequate due to a lack of adequate start-up

 

time. He says that:
While claiming to have conducted a rigorous experiment,
OEO actually began the experiment in a whirlwind
fashion, forcing companies to create and staff
programs, purchase materials, train teachers, and
refurbish facilities during the summer months.
Although one of the reasons for the experiment was
to provide comparative test data on companies'
instruction versus conventional instruction, con-
ventional instruction began in September, while at
many sites, contractors were not able to get their
programs going in full gear until mid-autumn or
later.

The evaluation design of the CEO and this study
which measured only student achievement grade gains, left
unmeasured many related areas such as the internal reform
brought about in the schools, the defining of a truly
individualized program of study for students, the organi-
zation of learning to be structured and sequenced and the
diagnostic and prescriptive methods of teaching children
to read.

There are further limitations which must be placed
upon the results of this study. The universe was limited
to certain reading programs in the Grand Rapids Public
Schools. Furthermore, the students studied were selected

from one section of the city with a previously defined

socio-economic background. One must be aware and most

 

98James A. Mecklenburger, "How OEO Failed Per—

formance Contracting," Nation's Schools (April, 1972),
31-32.

 

161

careful not to draw upon the conclusions suggested, herein,

to be of a universal nature.

Suggestions for Future Research

A careful generalization of this cost-effectiveness

model may be adopted to study any population of subjects

regardless of the similarity or dissimilarity to those

subjects studied in this research.

Variations of this research might include the

following types of studies:

1.

Conduct studies of student reading needs and relate
them to the behavioral objectives of a specific
program. If needs and objectives are closely
related, a better cost-effective model and program

may ultimately result.

Use the grade levels of 4, 5, and 6 to determine

if the cost-effectiveness pattern of performance
contract programs and TRR programs are more cost
effective from grades 2 to 9. There is a suspicion
that there is a transferance of cost-effectiveness
and achievement gain from one type program to

another between these grade levels.

Conduct similar studies using similar programs
with populations in the inner—city to determine

if socio-economic conditions have different effect

162

on the learning rate and ultimately cost-

effectiveness of programs.

Compare like programs in various school districts
to determine what effect geographical location
and/or other enviromental factors have on costs

as well as student achievement.

Conduct a second year study of the programs used

in this study, after program modifications have
been made, to determine if the changes can make a
significant difference in the student gain and thus

the cost—effectiveness of the program.

Consider an evaluation design that uses criterion
referenced tests based upon performance objectives
rather than standardized achievement tests for

measuring student gains.

Implications

 

The findings of this study seem to indicate that

certain implications for the reviewer's consideration must

be mentioned.

Herewith are presented several of these impli—

cations which should be given further deliberation:

1.

The apparent implications created by the inter-
grouping of students in learning situations may
have some sociological impact. In the performance

contracting programs, there appeared to be an

163

effective social relationship established among
students for self-help and a help-one-another
attitude, one student tutoring another which could
have a further positive effect on better achieve—

ment.

Students who could evaluate their progress in the
more structured and individualized programs
appeared to have an improved attitude towards

themselves, their peers, and the school.

Student attendance in school did improve in the
performance contract and programmed instruction
programs which may have been a result of (a) better
student self-image, due to the student's perception
of his success, (b) the contract obligation of the
contractor and staff which required a minimum
number of days attendance, and (c) or greater
sensitivity of the school system keeping more

accurate records.

Very serious consideration must be given the
feasibility of continuing the traditional remedial
reading programs as they are being carried on
today. There are implications in this study for

a more structured program. There are also impli-
cations for improving the methods of selecting
students and the sociological impact upon the

student who is chosen and identified with the

164

remedial program. The isolation factor of a
remedial reading setting may prove a hindrance to

the student's ability to learn.

5. The role of the building principal may need to be
redefined. The principal, in order to fulfill the
requirement of a two-way street of accountability
must have the autonomy to make decisions in
implementing learning programs which should include
day to day changes in program design as well as
adequate local latitude in financial decisions
affecting the immediate implementation of program

changes.

6. This study would indicate that schools have just
begun to develop criteria for performance objec-
tives of the education function. Schools will need
to more clearly define these objectives as well as
develop methods of evaluating the success or

failure in achieving these objectives.

Final Comments by the Researcher

 

When one completes a research study which has
required much effort over an extended period of time, with
its numerous delays and frustrations beyond the researcher's
control, it is extremely difficult to state in a few words

how one views her results; however, this will be attempted.

165

The author can only apologize for the one year
delay in reporting the final data. It had been hoped at
the onset that the results would have been forthcoming
immediately following the programs studied conclusion in
order that the results could be used in the decision-
making process for the local school district for the
current school year. This was not possible. Although the
study was researched during the 1970-71 school year, test
results were not available to the educational and scien-
tific communities from OEO, until late March, 1972. The
CEO has not yet released its own final report on that part
of its own experiment which was included in this study.

A June, 1972 date has been announced.

The author has attempted to isolate some of the
major costs factors in instructional programs in hopes of
providing a small step in developing a cost-effective
accountability model.

This accountability model is a part of a feedback
loop through which inputs are linked to outputs. This
feedback loop does seem to provide a means for establishing
responsibility between a school, a technological firm or
book publisher and the staff. Further refinement may also
provide a means for a "contract" for productivity and
cost-effectiveness which may also specify the conditions
under which each party will be responsible for a given

program in the future. If further refinement is possible,

166

responsible firms and responsible staffs may one day be
expected to accept greater responsibility in guaranteeing
results in some forms of learning. In the past, too often,
school districts have been sold materials and programs

with no one really accepting the responsibility for

student results. It is the opinion of the author that each
party involved must have an active interest in the students'
success or failure.

If this study will add one more small facet in our
quest for knowledge in the area of accountability, the
author will be exceedingly pleased. It is hoped this small
contribution may chart a path for developing a more
sophisticated cost-effective model for others to detail
and refine. Shortcuts may eventually be found which will
lead to the ultimate objective of educational quality

through cost-effectiveness data.

BIBLIOGRAPHY

BIBLIOGRAPHY

Bain, HEIGD. "Self Governance Must Come First, Then
Accountability." Phi Delta Kappan (April, 1970).

 

Barro, Stephan M. "An Approach to Developing Accountability
Measures for the Public Schools." Phi Delta
Kappan (December, 1970).

 

Baynham, Dorsey. "PPBS and Several Good Reasons It
Shouldn't Scare You Off." American School Board
Journal (August, 1970).

 

Benson, Charles S. The Economics of Public Education.
Boston: Houghton Mifflin Company, 1961.

 

Blaschke, Charles. "Performance Contracting." IDEA
(Winter Quarter, 1971).

. Performance Contracting_in Education: The
Guaranteed Student Performance Approach to Public
School System Reform. ’Educational Turnkey Systems,
Inc. Research Press, Champaign, Illinois, 1970.

 

 

 

"The Board of Education of the City of Grand Rapids,
Michigan: Final Budget, 1970-71."

Buskin, Martin. "PPBS Means Better Money Management."
School Management (November, 1969).

 

"Clouds and Controversey Over Texarkana." Nation's
Schools, LXXXVI, 4 (October. 1970).

Corrigan, R. E. "The SAFE Planning, Programming, Budgeting,
Evaluation System: An Approach to Relevance and
Accountability in Education." R. E. Corrigan
Associates, Anaheim, California, 1970.

Davies, Don. "The 'Relevance' of Accountability." The
Journal of Teacher Education (Spring, 1970).

 

167

168

Deck, L. Linton Jr. "Accountability and the Organizational
Properties of Schools." Educational Technology
(January, 1971).

 

Duncan, Merlin G. "An Assessment of Accountability, the
State of the Art." Educational Technology
(January, 1971).

 

Finn, James D. "Institutionalization of Education."
Educational Technology, IX, 12 (December, 1969).

 

Gallagher, James J. Consultation, May 13, 1971.

Gallup, George. The Second Annual Survey of the Public's
Attitudes Toward the Public Schools. A report
prepared by Gallup International, Princeton, N.J.:
October, 1970.

 

 

Gravue, Robert J. "Accountability: Comments and Questions."
Educational Technology, XI, 1 (January, 1971).

 

Hecker, Stanley. Consultation, April 29, 1971.

"How Education Groups View Contracting." Nation's Schools,
LXXXVI, 4 (October, 1970).

 

Johnson, Richard A.; Kast, Fremont E.; and Rosenzweig,
James E. The Theory and Manggement of Systems.
New York: McGraw-Hill Book Co., 1967.

Kane, Robert M. A System Approach, Accountability With
Justice. A report prepared fbr The California
State Department of Education, 1970.

Kane, Robert F. "Evaluation: What, Why, How." Paper
presented at the California Audio Visual Education
Assocation, Annual Conference, Fresno, Calif.,
Jan. 29 and 30.

Katzenbach, E. L. Planning, Programming, Budgeting
Systems: PPBS and Education. Report made to the
New England School Development Council, March,
1968.

 

 

Kaufman, Roger A. "Accountability, a System Approach and
the Qualitative Improvement of Education--An
Attempted Integration." Educational Technology
(January, 1971).

 

169

Kaufman, Roger A. System Approaches to Education:
Discussion and Attempted Integration on Social
andiTechnological Change Implications for Edu-
cation. Report prepared for The Center for the
Advanced Study of Educational Administration,
University of Oregon, Eugene, Oregon, 1970.

Kirk, Roger E. Experimental Design: Procedures for the
Behavioral Sciences. Belmont, Calif.: Brooks
Cole Publishing Co., 1968.

Kraner, Robert E. Developing and Writing Behavioral
Objectives. Tucson, Arizona: Educational
Innovators Press, 1971.

 

. Final Audit Rgport, Texarkana Dropout Pre-

vention Program. Tucson, Arizona: Epic Diversi-
fied Systems Corporation.

Krueger, Stanley. Statement expressed by speaker.
Michigan Association of State and Federal Program
Supervisors Conference, Holiday Inn North, Grand
Rapids, Michigan, February 10, 1971.

Language Arts Study Guide. Grand Rapids Public Schools,
1960.

Lessinger, Leon. "Engineering Accountability for Results
in Public Education." Phi Delta Kappan, LII, 4
(December, 1970).

. Every Kid a Winner, Accountability in Education.
Simon and Schuster, 1970.

. "The General Concept of Independent Accomplish-
ment Auditing." Presented at the Institute on
Independent Educational Accomplishment Auditing,
Hotel NeWporter, Newport Beach, Calif., Nov. 10-12,
17-19, 1969.

. "Robbing Dr. Peter to Pay Paul: Accounting for
Our Stewardship of Public Education." Educational
Technology (January, 1971).

 

 

 

. "White House Memo Defining Performance Con—
tracting." December, 1969.

 

Lewis, James Jr. Administering the Individualized
Instruction Program. West Nyack, N.Y.: Parker
Publishing Co., Inc., 1971.

 

170

Lichtenberger, Allan R. "Program Planning, Budgeting, and
Accounting in School System Operation." A position
paper by Allan R. Lichtenberger, Chief, Terminology
Compatibility Branch, U.S. Office of Education,
September 20, 1967.

Love, Byron K. "The Development of a Suggested In-
structional Programs Cost Model for K-12 Districts
of the State of Michigan." Unpublished doctoral
thesis, Michigan State University, 1968.

Mager, Robert F. Preparing Educational Objectives.
Palo Alto: Fearon Publishers, 1962.

 

Mecklenburger, James A. "How OEO Failed Performance
Contracting." Nation's Schools (April, 1972).

 

Moynihan, Daniel P. "To Solve a Problem, First Define It."
New York Times, January 12, 1970.

 

Nixon, Richard M. "Excerpts from the President's Special
Message to Congress on Educational Reform." New
York Times, March 4, 1970.

 

. "Transcript of the Nixon Speech." New York
Times, January 27, 1970.

Office of Economic Education. "Background: An Experiment
in Performance Contracting." Program description,
July, 1970.

"OEO Announces $6.5 Million Contract Program." Nation's
Schools, LXXXVI, 2 (August, 1970).

CEO Contractors. "Joint Summary Statement." Unpublished
paper presented to CEO, March 15, 1972.

CEO Project Directors. "OEO Project Director's Evaluation
Chapter." Submitted to the CEO, December 22, 1971.

Pierce, Wendall H. "New Directives for Education."
Compact (February, 1971).

Porter, Andrew, Consultation, March 3, 1971.

Porter, John W. "Porter Rips State Educators, Goats."
The Grand Rapids Press, December 3, 1970.

 

Randall, Ronald. The Cost-Ed Model. Educational Turnkey
Systems, Inc., October, 1970.

 

171

Rosenthal, Jack. New York Times story, May 15, 1970.

Semple, Robert B., Jr. "Nixon Proposes Re-examination of
Aid to Schools." New York Times, March 4, 1970.

 

Sheldon, David. Statement made at the AFT Convention,
Pittsburgh, Pennsylvania, August 21, 1970.

Stalford, Charles. Office of Economic Opportunity memo,
June 4, 1971.

USOE. ESEA Title VII Manual for Project Applicants and
Grantees. Washington, D.C.: Government Printing
Office, April 24, 1969.

Vruggink, Elmer H. "A Study of the Contribution of
Compensatory Programs in Large Urban School
System." Unpublished doctoral thesis, Michigan
State University, 1970.

Weinheimer, Norman P. Consultation, March 3, 1971.

. "From the Desk of the Executive Director."
Michigan School Board Journal (July, 1971).

Yager, Geoffrey. Consultation, March 3, 1971.

APPENDICES

APPENDIX A

GODMER LETTER

172
STATE OF MICHIGAN

Sgt-raj DEPARTMENT OF EDUCATION

lensing, Michigan 48902
STATE BOARD OF EDUCATION
EDWIN L. NOVAK. O.D.
JOHN W. PORTER President
Superintendent of MICHAEL J. DEEB
Public Instruction Vice President
DR. GORTON RIETHMILLER
Secretary
THOMAS J. BRENNAN
Treasurer
MARILYN JEAN KELLY
ANNETTA MILLER
DR. CHARLES E. MORTON
JAMES F. O'NEIL

(90V. WILLIAM G. MII.I.IKI:N

 

 

 

Ex-Olllcla
March 12, 1971
Miss Joan Webster
Turner School
959 Turner, N.W.
Grand Rapids, Michigan 49504
Dear Miss Webster:
Re: Operational Millage and Bond Issue
The following results indicate the voter reaction concerning
operational millage and bond issue by fiscal year.
Operational Issues
1968-69 598 total 366 passed 232 defeated
1969-70 656 " 458 " 198 "
1970—71 (Thru'Jan.) 73 " 44 " 29 "
30nd Issues
1968-69 122 total 41 passed 81 defeated
1969-70 112 " 42 " 7O "
l970-7l(Thru'Jan.) 48 " 19 " '29 "

Sincerely,

/,/'/ 1 i /
i/I/{éyQV/K’I 1(/ X ' 4' -%4/.:’ 4"” '4'

Raymond L. Godmer, Consultant
School Management Services

APPENDIX B

TYLER LETTER

173

S R I A SCIENCE RESEARCH ASSOCIATES. INC
A Subsodiary 01 IBM

259 East Erie Street
Chicago, lllinous 60611
(312) 944-7552

Cable SCIRESUS. Chicago

May 7, 1971

Joan M. Webster
Director
Contract Learning Project
Grand Rapids Public Schools
Grand Rapids, Michigan 4950l

Dear Dr. Webster:
Thank you for your letter of May 3.

There are probably many steps that can improve testing in terms of the actual
learning of students. One immediate step is to encourage reputable test
makers to develop criterion-referenced tests, assuring them that the schools
will use them. -

Another possibility is the construction of some tests of this type by a coup-
erative effort of several schools or school systems.

It was good to visit with you. I h0pe our paths will soon cross again.

Sincerely,

)

////' .
\ I. _ \ .

Ralph w. Tyler 3'
RNTzac

(Transcribed and mailed in Dr. Tyler's absence)

APPENDIX C

STALFORD LETTER

()I‘I’ICIC ()I“ I1Z(.I()N()M I(.I

OPPORTUNITY

April 14, 1972

Mrs. Joan Webster
7695 Cascade Road, S.E.

Grand Rapids, Michigan 49508

Dear Joan:

174

EXECUTIVE OFFICE OF THE PRESIDENT

WASHINGTON, D.C.

The tests used for evaluation in the Office of Economic
Opportunity Performance Contract Experiment are as follows:

In grade 1, the pretest was the Stanford
Early Achievement Test. The posttest
was the California Achievement Test,

level 1 Form B

In grades 2,3,7,8 and 9, the Metropolitan
Achievement Test, 1970 edition was used,
Form F for pre and Form G for post. The

levels were:

Grade
Grade
Grade
Grade
Grade

\OWNCDN

I hope this information meets

Sincerely,

,g'~.. ,Jl*fiu5
Charles Stalford
Project Manager

Primary I
Primary II
Intermediate
Advanced
Advanced

your needs.

Performance Contract Experiment
Experimental Research Division
Office of Planning, Research and Evaluation

20506

APPENDIX D

READING GAIN SCORE DISTRIBUTION CHART

Reading Gain Score Distribution 1049 Students

 

Student
Achievement
Gain

CMES WLC

TRR

Control

 

Alpha

 

Grade Level

 

 

Minus

17 26 19 1 10

7

l

9 22 15 14

1

175

11

11

1.0

19

19

8 19 41 25 15 9 12 16 13 12
75 81 81 71 65 71 57 67 73 64 23 29 22

2

55

35 27 28 42

53

 

II IIIIIIII IIIIIII IIIII III II III III II IIIIII III IIII I II

 

 

III-II-J‘l‘