t

 

 

LIBRARY
Michigan State
University

 

 

This is to certify that the

dissertation entitled
IDENTIFICATION OF PROBLEM VARIABLES

WHICH CREATE A TIME AND COST IMPACT

DURING LEVEL THREE INTERACTIVE VIDEODISC

COURSEWARE DEVELOPMENT
presented by

CLARA STEIER

has been accepted towards fulfillment
of the requirements for

Ph-D - degree in Educational. Systems

Development

A fiflmfl‘

Major professo

Date 242342 2

MS U is an Affirmative Action/Equal Opportunity Institution 0-12771

 

 

MSU

LIBRARIES
-—.

v

 

 

RETURNING MATERIALS:
Place in book drop to
remove this checkout from
your record. FINES will
be charged if book is
returned after the date
stamped below.

 

 

 

 

IDENTIFICATION OF PROBLEN VARIABLES
HHICH CREATE A TIHE AND COST IHPACT

DURING LEVEL THREE INTERACTIVE VIDEODISC COURSEUARE DEVELOPHENT

3!

Clara Steier

A DISSERTATION
Submitted to
Nichigan State University
in partial fulfillment of the requirements
for the degree of

DOCTOR OF PHILOSOPHY

Educational Systems Development

1987

Copyright by
CLARA SIBIBR

1987

ABSTRACT
IDENTIFICATION OF PROBLEH VARIABLES
WHICH CREATE A TINE AND COST INPACT
DURING LEVEL THREE INTERACTIVE VIDEODISC COURSEUARE DEVELOPHENT
3!

Clara Steier

Interactive videodisc (IVD), is the very latest instructional
technology. The medium is complex and requires a multi-disciplinary
team to achieve a reliable product. Training and knowledge about the
processes and procedures are primarily being generated through
trial-and-error in actual IVD projects and at great cost to both
contractors and client. Cost overruns appear to be the norm rather
than the'exception. Both developers and users are under increasing
pressure to produce more realistic cost estimates in order to ensure
that funds are available to complete the product. 'The means for
making adequate estimates are presently not available.

The major purpose of this study was to develop a baseline
knowledge of the problem variables. processes and interactions in the
analysis. design. development and production of Level 3 IVD for
training and/or education in industry and government which affected
the time and cost and contributed to the project cost overruns.
Twelve specific research questions were developed from the review of
literature. The questions focused on the time and cost effects and/or
extent of the effects of (a) knowledge required to make cost
estimates. (b) client IVD knowledge. (c) contractor agency size and

capabilities. (d) functions of team members. (e) team organization and

Clara Steier

personnel, (f) phase activities, is) phase most likely to cause an
effect, (h) effects of hardware. (1) effects of software. (3)
unintended and unplanned events. (k) unexpected costs and related
circumstances, and (l) unexpected time delays.

To explicate the problem variables. a case study methodology was
used. An interview guide containing 65 questions was developed to
guide the interview and help ensure that similar data were collected
from all of the cases in the study. One-on-one. audio-taped. guided
interviews were conducted with project managers, instructional
designers, information programmers and video producers for three cases
which had reported cost overruns in producing an IVD. Records,
documentation and reports were also examined where available to
further verify and cross check the data gathered during the interviews.

The findings of the study clearly show that time delays and cost
overrun factors are predominantly related to (a) the complexity of the
design. (b) hardware and software deficiencies. (c) client knowledge,

(d) contractor experience and (e) team functioning and communications.

DEDICATION

This dissertation is dedicated
to my Nether and Late Father

ACKNOWLEDGEHENTS

The writer wishes to express her appreciation to the many persons
who have contributed to the design. development, and execution of this
dissertation.

My thanks are expressed to Dr. Cass G. Gentry, chairman, and to
Drs. Bruce L. Niles, Peggy H. Riethmiller and Frederic R. Uickert for
their counsel and guidance during critical phases of this dissertation.

Particular thanks are expressed to Dr. Diana Bryan for the
unselfish gift of her time and professional guidance.

Special thanks go to the companies and their employees who
willingly shared their experiences. Without their cooperation and
time this dissertation would not have been possible.

Gratitude is also expressed to Eagle Technology, Inc., in
particular. Dr. Roy Perryman, for allowing a leave of absence and the
use of support services to make this research possible. I also wish
to thank Amelia Samuelson and Christine Galante for providing cheerful
word processing support.

Deepest appreciation and gratitude goes to my husband, Bill
Jorgensen. for his counsel, insightful suggestions, thought provoking
discussions, continual encouragement, unending patience and constant
love over the past four years and while this dissertation was in
progress. without his help and support I may not have done this. To

him, I can only offer myself and my love.

vi

TABLE OF CONTENTS

LIST OF TABLES .

LIST OF FIGURES

CHAPTER
I. PROBLEN

Introduction . .
Background of the Problem
Statement of the Problem .
Purpose of the Study .
Research Questions .
Assumptions .
Limitations of the Study .
Definition of Terms

Development Phases .

Research Hethodology .

IVD Terminology

IVD Levels .

General .
Significance of the Study

Organization of the Remainder of the Dissertation

Chapter Summary
II. REVIEW OF LITERATURE .

Interactive Videodisc System .
Problem Areas in IVD . . .
Costing Procedures for IVD .
Case Study and Field Research.
Chapter Summary. . . . . .

III. HETHODOLOGY.

Study Design . . .

Research Questions .

Sampling . . . .
Sample Selection .
Sample Case Description

PAGE

xi

H

OEDONO‘ODNH

20

20
24
29
35
39

41

41
43
44
44
46

TABLE OF CONTENTS (Continued)

CHAPTER

Data Collection Instrument .
Data Collection Procedures .
Pilot Interviews .
Interviews .
Case Record
Principles for Interviewing
Data Analysis
Chapter Summary.

IV. DESCRIPTION AND RESULTS OF INTERVIEWS.

Case Descriptions
Key Case Elements

Results
Research Question
Research Question
Research Question
Research Question
Research Question
Research Question
Research Question
Research Question
Research Question .
Research Question 10 .
Research Question 11 .
Research Question 12 .

Summary of Findings
Software . .
Hardware . .
Client Knowledge .

\DQNOU‘OWNH

Contractor Experience and Qualifications .

Design Phase . .
Team Nembers Roles and Functions .

V. SUHNARY, CONCLUSIONS AND RECONNENDATIONS .

Summary of the Study .
Purpose .
Nethodology .
Findings, Conclusions and Recommendations
Research Questions .
General Recommendations

BIBLIOGRAPHY .

viii

. 102
. 105
. 109
. 112
. 112
. 113
. 113
. 114
. 116
. 116

PAGE

46
48
48
48
49
50
52
S4

S6

56
58
61
61
69
72
75
79
84
89
91
96

. 118

. 118
. 118
. 118

119

. 123
. 133

. 135

TABLE OF CONTENTS (Continued)

PAEE
APPENDICES
A. PRODUCT DESCRIPTION/COST DECISION WORKSHEET . . . . . . 144
B. STUDY INTRODUCTION TO CLIENT . . . . . . . . . . . . . . 149

C. INTERVIEW GUIDE . . . . . . . . . . . . . . . . . . . . 150

ix

TABLES
Chapter IV

1 Case Description . .

2 Information Known Before and Learned After Contract
Which Affected Time and Cost .

3 Effect of Client Knowledge on Time and Cost in
Producing an IVD . . .

4 Factors of Contractor Size and Capabilities Related to
Time and Cost in Producing an IVD . . . . . .

5 Functions of Team Nembers During IVD Process .

6 Effects of Project Team Organization and Personnel IVD
Training on Time and Cost of an IVD Project .

7 Activities in a Phase Which Affect Time and Cost

8 Effect of Hardware on IVD Development Time and Cost .

9 Effect of Software on IVD Development Time and Cost .

10 Creation of Cost Overruns by Unintended and

LIST OF TABLES

Unplanned Events

11 Circumstances Under Which Unexpected Costs Occur
12 Circumstances Creating Unexpected Time Delays .

Chapter V

13 Findings, Conclusions and Recommendations .

PAGE

59

62

70

73
76

80
85
92
97

. 103
. 107
. 110

. 120

LIST OF FIGURES

FIGURE PAGE

 

1 Case 3 Client and Contractor Relationship . . . . . . 57

xi

CHAPTER I
PROBLEH

Introduction

Interactive videodisc (IVD) is the very latest instructional
technology which interfaces microprocessor responding or evaluating
devices with a videodisc player. The interactivity of this video is a
program style which allows the learner to proceed in a variety of ways
through the program and respond to the program material in a
substantive manner. The sequence and selection of material will vary
depending on the viewer's response. For example, in. a simulated
equipment repair task the learner may be asked to select and attach
test equipment through a touch screen mode. The student who makes a
mistake in any part of the performance could be branched to
remediation or to outcomes appropriate for the error mmde, while the
student who performs correctly would proceed in the logical sequence.
The visuals would be coded onto a videodisc and played back. by
commands given to a microcomputer.

To the uninitiated. an interactive videodisc system seems nothing
more than the marriage between a computer and video, with the video
being stored in a new format -- a flat laser disc. However, this is
not the case. Interactive videodisc systems must be accepted as a
unique technology. Walker (1979-80) writes that the strength of

videodisc lies in its flexibility in manner of presentation (most

2

important are motion and still frame capabilities). its tremendous
information storage capability when single framing is employed and
lastly. its rapid access to any playing point which is usually three
seconds or less. Walker also writes that when a videodisc is joined
with some form of computer data processing. it provides the lesson
designer with unparalleled resources for blending still and motion
pictures. sound. text and graphics for instructional purposes. Thus.
it goes without saying. the development of an interactive videodisc
system requires the combined skills of instructional designers.
computer programmers. and video technologists. DeBloois (1982). Floyd
(1980). Gayeski (1984-85). Hon (1980). and Worcester (1983). indicate
the challenge to each team member is to understand one or more media
and furthermore to understand their application and relationships in a
very unique manner which requires the modification of old skills and
processes and the acquisition of new skills and processes to achieve
reliable results. As the review of literature will show. at the
present time all of these skills and processes are being defined
through individual experimentation. which may be costing those who are
designing and developing training in this medium large losses in time
and resources which in turn translates to a dollar loss to the
contractor. Unfortunately. there has been little empirical research
on which to base decisions about IVD costing data and methodologies or

the development of adequate cost models.

Background of the Problfl
Since the use of interactive videodisc technology is fairly new.
coming to the forefront as a viable instructional system in the last

five years. little or no research has been done on the subject. The

3

few studies which do exist (Bright. 1983; Naxwell. 1985) look more at
the potential uses of the medium. The effectiveness is well
established. However. these studies allude to high cost factors in
terms of time and dollars required to develop highly interactive
videodisc training. A concern is expressed by both Bright and Naxwell
that unless a means is found to control and/or lower costs. the medium
may not be used to its fullest potential. Nerrill (1980) states the
concern more succinctly:

The videodisc cost picture offers both good and

bad news. The good news is that disc

replication costs are low. and videodisc player

costs are competitive. The bad news is that

production [including developmenti and

mastering costs are relatively high. (p. 93)
Nerrill continues:

Total costs for developing and producing

educational videodiscs are very difficult to

obtain .... In some cases. production. costs

are reported but development costs are

ignored. When data on specific costs are

available. we find that the costs vary widely

because of different development and production

procedures and different instructional

strategies. Very few* educational discs have

been produced to date. and the cost associated

with their production includes a lot of

trial-and-error learning costs. ,(p. 93)
Nerrill‘s sentiments are echoed by’ Gayeski (1984-85). Hon (1981),
Kirkpatrick and Kirkpatrick (1985). Kribs (1979-80). and Winslow
(1981).

In talking with videodisc developers at conferences and through
business contacts. an increasing number of people are venturing into
interactive videodisc production. Nany of these are even venturing in
at Level 3 which has the potential to be the most interactive. complex

and versatile of the videodisc technology. Level 3 marries the audio

4

and visual capabilities of the videodisc technology with the
flexibility and sophistication of an external computer. By
controlling the videodisc player with an external computer. more
sophisticated branching can be achieved. With this sophistication and
increased complexity. the ability to predict accurate time and cost
for development and production of interactive videodisc programs has
decreased. Nany complain that at the present time complete costing
models do not exist to assist the contractor in making a realistic
estimate or for the client to detemmine what funding should be
allocated for an IVD project (Gery. 1987; Nerrill, 1980; Parsloe et
al., 1983). As with any innovation. time is required to build a base
of experiences from which principles and concepts can be drawn and
applied to other efforts. Presently, a systematic approach to
deriving a set of variables which could be validated and provide the
knowledge base upon which to build a predictive cost model for Level 3

interactive videodisc projects has not been undertaken.

Statement of the Problem

There has been little substantive research on which to base the
development of adequate cost models for Level 3 IVD systems. It is
paradoxical that while many writers (Klotz. 1983; Kirkpatrick a
Kirkpatrick. 1985; Naxwell. 1985; Nerrill. 1980; Parsloe et al., 1983)
involved in IVD since the late 70's. have strongly advocated the need
for more and better costing data and methodologies. researchers seem
to have ignored the topic.

Level 3 IVD is a fairly new medium requiring a modification of
old skills and procedures and the acquisition of new skills. Training

and knowledge about the processes and procedures are primarily being

5

generated through trial-and-error in actual IVD projects. apparently
at a great cost to contractors (Allen a Erickson, 1986; DeBloois.
1982; Gayeski. 1984-85; Worcester, 1983).

IVD cases reported in the literature (Dennis. 1984; DeBloois.
1982; LaGow, 1979-80; Nanning et al., 1983; Platt 8. NcConville. 1982)
are usually very large projects which are atypical and which do not
have time and costs adequately documented for development and
production. Further, the documentation and description of the
courseware and software development is general and of little value to
the designer or programer.

The available literature on analyzing. designing, developing and
producing IVD is at a very general level saying more "what to do“
rather than “how to do.“ Further. where “how to" instructions are
given, the recomended procedures are more appropriately applied to
the simpler Levels 1 and 2 IVD of the Nebraska scale (Cambre. 1984;
Lindsey. 1984; Parsloe et al., 1983; Tennyson a Breuer. 1984).

Another aspect is that problems currently identified in Level 3
IVD cases which might provide assistance to the contractor or client
in planning are usually described too generally or grouped too broadly
(i.e.. programing) from which to draw any conclusions or gain
insight.

The central focus of the first research efforts in developing a
useful cost model for Level 3 IVD must be the development of baseline
information derived from problems occurring in current Level 3
projects.

Therefore. the problem statement of the current study might be

sumrized as follows: Now can a Level 3 IVD project be viewed.

6
planned and costed? A more fundamental question is: What specific
unknown variables unique to Level 3 IVD, which will affect time and
cost. are presently contributing to the cost overruns in current

contractual efforts?

Purpose of the78tudy

The major purpose of this study was to develop a baseline
knowledge of the problem variables. processes and interactions in the
analysis. design, development and production of Level 3 interactive
videodiscs for training and/or education in industry and government
which affected the final cost and time to produce the product. The
main objective was to collect and document empirical data about
variables within analysis, design. development and production which
affected Level 3 IVD time and cost contributing to cost overruns.
This baseline information is important for further study in the

development of costing models for interactive videodisc efforts.

Research Questions
The specific focus of the investigation was delineated in terms
of the following research questions:
1. What type of knowledge is needed and when does it have to be
available to make time or cost estimates for an IVD project?
2. To what extent does client IVD knowledge. as reported by the
contractor, have an effect on time and cost of an IVD project?
3. What effects. if any. do contractor agency size and
capability have on time and cost of an IVD project?
4. What functions does each team member perform in the process

of carrying out an IVD project?

10.

11.

12.

The

7
What effects. if any. do project team organization and
personnel IVD training and experience have on time and cost
of an IVD project?
What activities in a phase have a time and cost effect on
subsequent phases of an IVD project?
Is one phase more likely than any other to affect time and
cost of an IVD project?
What effects. if any. does IVD hardware have on time and cost
of an IVD project?
What effects. if any. does IVD software have on time and cost
of an IVD project?
To *what extent do unintended events and unplanned events
create a time and cost overrun in an IVD project?
When and under what circumstances do unexpected costs occur
in an IVD project?
When and under what circumstances do unexpected time delays

occur in an IVD project?

Assumptions

following assumptions were made in this study:

The study would generate presently unconsidered variables
which affect time and cost in IVD development efforts.

The case methodology applied would meet the objectives of
this study.

The subjects interviewed would cooperate and honestly
describe the activities and actions taken during the
analysis. design. development and production of the

interactive videodisc process.

The

BOEVO t0

1.

8
The bias of the researcher could be controlled through the
use of formal interview techniques and examination of
existing case documents.
The projects selected for study adequately represent Level 3
IVD technology.
There would be three IVD Level 3 projects at such a stage of

completion as to make them valid for study.

Lygitations ofithe Study

present study is subject to the following limitations that
narrow its focus:

In view of the situation in industry and the willingness of
people in industry to discuss their specific losses by item
but not necessarily reveal figures related to profit and
overhead costs. this study relates only to items of direct
costs and losses.

Because of the methodology employed (case study). the
outcomes of this study cannot be generalized to the
population from which these cases have been drawn. Instead.
the results of this study should inspire appropriate
follow-up research. focusing on specific hypotheses using
appropriate sampling methods.

Because of the time and cost involved in conducting case and
field research. only a small number of cases could be
studied. Therefore. -every possible type and approach to
Level 3 IVD may not have been sampled. As a result. it is

possible that some Level 3 IVD problem variables which may

section.

9

affect total project time and cost may not have been
identified.

The results of this study are only related to Level 3 IVD
and. therefore. may or may not include problem variables
which affect time and cost in Levels 1. 2 and 4.

This study is purposefully limited to cases involving losses
in final time and cost. and. therefore. may or may not
include all the problem variables which affect time and cost
in cases which ended the project on schedule and within the

overall dollar limit .

Definition of Terms

Definition of terms used in the study are presented in this

Develoment Phases

These represent the processes of instructional development.

There are generally four phases related to the development of IVD.

These are:

1.

2.

Analysis: A careful study of the desired behavior

required for an instructional program.
This analysis helps determine the cause of
performance problems. what must be done to
correct the performance. and the best
technique to provide the training (Iuppa.

1984. pp. 15-16).

Design: The process of writing objectives .

organizing the material into instructional

3.

4.

Development:

Production:

Research Nethodology

1.

2.

10

sequences. arranging the sequences into a
proper order. and defining the presentation
and testing formats (Iuppa. 1984. pp.
21-22). Also recognized as preproduction
in some of the literature.

The process of writing test items.
narrative elements and a description of the
pictorial elements in the program (Iuppa.
1984. p. 71). Also recognized as
preproduction in some of the literature.

The phase in which video and film footage
is shot. editing and preparation of the
tape or film for reproduction as an
interactive tape or disc is performed. and
the computer programs are written (Parsloe.
lioffos. 8. Epic team. 1983. Daynes. 1982).
Also recognized as post-production/

premastering in some of the literature.

Terms related to the research methods employed include:

Case Nethod/Study:

Case Record:

A recognized method of inquiry defined as
an “in—depth investigation of a given
social unit resulting in a complete. well
organized picture of that unit“ (Isaac a
Nichael. 1979).

The compilation of the voluminous data

organized into a comprehensive. primary

3.

‘0

Interview Guide:

Purposeful Sample:

IVD Terminology

11

resource package. It includes all of the
major data which will be used in doing the
case analysis and case study (Patton. 1980).
Outline of a set of issues or topical areas
that are to be explored. with each
respondent before interviewing begins. The
interviewer is required to adapt both the
wording and sequence of questions to
specific respondents in the context of the
actual interview (Patton. 1980. p. 198).

A strategy used when one wants to learn
something and come to understand something
about certain select cases without needing
to generalize to all cases. To» do
purposeful sampling. certain
pre-established information must. be known
about the case before it is studied

(Patton. 1980).

Special operational terms for this study generally accepted by

professionals using the IVD medium include:

1.

Authoring:

The preparation of a computer program.
often using an "author language“ or
“authoring system! that allows people
without formal training in computer
programming to prepare applications

programs for computer-based systems

2.

3.

‘0

Authoring Language:

Authoring System:

Courseware:

Frame:

12

(Parsloe et al., 1983. p. 248).

A high level computer program. itself often
based on a computer language like BASIC or
Pascal. that facilitates the preparation of
computer programs by reducing the number of
instructions involved and translating these
into a language resembling everyday English
(Parsloe et al., 1983. p. 249).

A collection of authoring programs that
allows users without formal computer
programming skills to prepare application
programs. often working in everyday
language. and ‘without the painstaking
detail of computer programming proper
(Parsloe et al., 1983. p. 249).

That part of an interactive video training
or teaching course comprising the video
program (content on disc or tape). and its
complementary . computer program(s).
including those generating text and/or
graphics (Parsloe et al., 1983. p. 253).

A single. complete picture in a video or
film recording. A video frame consists of
two interlaced fields of either 525 lines
(NTSC) or 625 lines (PAL/SECAN). running at
30 frames per second (fps) (NTSC) or 25 fps

(PAL/SECAH). Film runs at 24 fps (Niller a

6. Information
Programmer:

7. Instructional
Designer:

8. Interactive:

9. Interactivity:

10. Interactive Video:

11. Linear:

13
Sayers. 1986).
The person on an interactive video
production team who translates the work of
the instructional designer into a computer
program (Parsloe et al., 1983. p. 259).
The person on the interactive video team
who. given the aims and objectives of the
project. analyzes the content. of the
program and arranges that mmterial in a way
that can then be used by the information
programmer (Parsloe et al., 1983. p. 259).
Involving the active participation of the
user in directing the flow of the computer
or video program. The opposite of
interactive is linear (Parsloe et al.,
1983. p. 259).
A reciprocal dialogue between the user and
the system.(Niller a Sayers. 1986).
The convergence_ of video and computer
technology: a video program and a computer
program running in tandem under the control
of the person in front of the screen. In
interactive video. the user's actions.
choices and decisions genuinely affect the
way in which the program unfolds (Parsloe
et al., 1983. p. 259).

A video program which plays straight

12. Search:

13.

14.

15.

Software:

Still Frame (SF):

Still-Frame Audio
(SPA):

14
through from start to finish without
interruption (Parsloe et al., 1983. p. 262).
The facility in interactive video systems
to request a specific frame. identified by
its unique sequential reference number. and
then to instruct the player to move
directly to that frame. forwards or
backwards. from any other point on the same
side of the disc or tape (Parsloe et al.,
1983. p. 269).
The programs which actually make the
computer run. Software represents the
intellectual side of computer-based
technology. hardware the physical side
(Parsloe et al., 1983. p. 270).
Information recorded on a frame or track of
a videodisc that is intended to ‘be
retrieved and displayed as a single
motionless image. Playback is achieved by
repeating the play of the same track.
rather than going on to the next - as
opposed to a freeze-frame that stops the
action within a motion sequence. (Niller a
Sayers. 1986)
A method of digitally encoding and decoding
several seconds of voice-quality audio per

individual disc frame. resulting in a

15
potential for several hours of audio per
disc. By using a buffer to store the audio
information. a limited amount of audio may
be delivered to accompany a still-frame.
(Niller a Sayers. 1986)

16. Videodisc: A generic term describing a medium of video
information storage which uses thin
circular plates. usually primarily composed
of translucent plastic. on which video.
audio and various control signals are
encoded. usually along with spiral tracks.
Optical disc systems use a laser beam to
read the surface of the disc; they are so
far divided between reflective and
transmissive systems. Capacitance systems
employ a sensor or stylus; they are divided
between “grooved" and “grooveless” disc
systems. A magnetic videodisc is used in
broadcast television. and for information
storage (Parsloe et al., 1983. p. 273).

IVD Levels

The Nebraska Scale. devised by the Nebraska Videodisc

Design/Production Group to describe interactivity in videodisc

players. defines the following for levels of IVD:

1. Level 1: The first practical level of the Nebraska
scale. Level 1 represents the basic

features expected even on domestic

2. Level 2:

3. Level 3:

4. Level 4:

General

16

equipment: remote control. the “search“
facility. freeze frame. forward and reverse
motion. quick scan. slow motion and step
frame replay (Parsloe et al., 1983. p. 261).
The midpoint on the Nebraska scale. Level
2 player is one which. using only its own
onboard microprocessor. can offer multiple
choice. the branching facility and score
keeping (Parsloe et al., 1983. p. 261).
Effectively the top of the Nebraska scale.
Level 3 represents a video player.
industrial or domestic. linked to an
external computer (mainframe. mini or
micro). This level offers the greatest
versatility of any interactive
configuration (Parsloe et al., 1983.
p. 261).

In a practical interpretation. a complete
workstation comprising video and computer
equipment and furniture. This level still
represents conjecture into the future of
interactive technology. rather than actual

applications (Parsloe et al., 1983. p. 261).

Other terms used in the study include:

1. Client:

A person. company. or agency which

contracts with another company or agency

17

for the acquisition of training materials
and systems.

2. Contractor: A business. organization or person who
contracts to supply certain materials or do
certain work for a stipulated sum.

3. Direct Costs: Costs which are incurred in the performmnce
of the work to include labor. mmterial and
travel. Excluded are profit and overhead
percentages.

4. Project: A contracted training program development
and production effort using interactive
videodisc.

5. Training: Learning experiences or activities that are
designed to facilitate. strengthen or

improve performance on the job.

§ignificance ofgthe Study

As a review of the literature in the next chapter will show. the
systematic study of specific cost factors related to the development
and production of Level 3 IVD systems has been almost non existent.
While a Level 3 IVD system has powerful instructional capabilities
and features. the basic high costs of development and production for
such a high technology system. the learning by trial-and-error on the
part of the IVD developers. and the inability to forecast. predict
and control the cost of the finished product indicate an urgent need
to develop adequate cost models for use in planning and implementing
IVD projects. If potential IVD users in the future are not to be

discouraged. clients and contracting agencies need some hard facts

18
and knowledge about the variables affecting Level 3 IVD time and
costs. Such knowledge will help to optimize the quality and
efficiency of IVD projects and products.

The primary significance of the present study is its
contribution toward the building of an empirical knowledge base
concerning the variables which affect Level 3 IVD time and cost. The
development of this baseline data is of value to future researchers
in the development of costing models and cost efficient processes and
procedures for interactive videodisc efforts.

This study should also be of prime interest to IVD development
and production agencies of Level 3 IVD for training. In particular.
this study should provide valuable inputs and guidance for coating of
IVD projects for industry and government which. in turn. will help
reduce the dollar loss presently incurred by many contractors.

In addition. this study should be of interest to the client
tasked with assessing and judging proposed budgets for IVD projects.
The variables identified should provide guidance to assist the client
in the process of evaluating proposals and budgets for an expression
of adequate understanding and allowance of the potential pitfalls by
the contractor. Further. it should provide guidance in establishing
realistic funds for the execution of a project as well as indicate
the information a contracting agency requires to more accurately bid

a contract and prevent a dollar loss.

 

Organization of the Remainder of the Dissertgtiog

In Chapter II a review of selected literature and research
pertinent to this study is undertaken. Reference is made to the

finite body of research and literature concerning IVD systems.

19
problem areas in IVD systems. costing procedures for IVD and case
study and field research. New evidence and more recent research
applicable to the study are reviewed.

Chapter III is a description of the methodology of the study.
including the research design. the research sample. the
instrumentation. the data collection and recording procedures. and
the data analysis procedures.

The results of the investigation and a discussion of the results
are set forth in Chapter IV.

Chapter V sumarizes the study findings. and then presents the

conclusions. and recommendations.

Chapter Summary

Interactive video is a relatively new instructional technology
which shows great promise as an effective training medium. The most
severe problem facing the use of this technology is the difficulty in
estimating the cost of developing IVD courseware since the courseware
includes an interactive video disc and both computer operating and
instructional software. The objective of this study was to identify
empirically the problem variables which affect the time and cost of
producing Level 3 IVD courseware. and to create a baseline of
information from which to conduct future research toward the
development of Level 3 IVD cost models. Further. this study should
provide insight to both IVD developers and clients on cost elements

in IVD which may cause overruns.

CHAPTER II
REVIEW OF LITERATURE

This chapter. which is divided into four sections. presents (a) a
general review of the literature related to interactive videodisc
systems. (b) a review of the literature related to problem areas in
interactive videodisc systems. (c) a review of present time and
costing procedures for the generation of an interactive videodisc
system and (d) a review of the literature related to case study and

field research.

Interactive Videodisc System
There is a large body of literature that describes the
interactive videodisc technology. An interactive videodisc system is
defined by Parsloe et al. (1983) as a:
Video program which can be controlled by the
person who is using it. Usually. this means a
video program and a computer program running in
tandem. The computer program controls the
video program - and the person in front of the
screen controls them both. (p. l)
Naxwell (1985) defines the videodisc system as “a multimedia delivery
device which combines pictorial. audio and digital information“
(p. 16).
The interactive delivery system may employ either videodisc or

videotape under the control of a microprocessor either built right

into the video player. or as a separate computer. As explained by

20

21

Parsloe at al. (1983):
The computer program which controls the video
program can actually be encoded on a videodisc
or loaded temporarily into a videodisc player
through its remote control keypad. or stored on
the magnetic tape or floppy disk of an external
computer. (p. 2)

A main feature of interactive videodisc systems is the quick
access time of getting from one frame to another. One side of a disc
can store up to 54.000 frames of information. A frame is a single
complete picture in a video or film recording. A video frame consists
of two interlaced fields of either 525 lines (NTSC) or 625 lines.
(PAL/SECAN). running at 30 frames per second (fps) (NTSC) or 25 fps
(PAL/SECAN). In a laser disc player. the longest access time to any
frame is a matter of seconds. which is usually under three seconds.
while in a videotape player. access can take several minutes. Other
features of IVD include the ability to hold a frame in place without
loss of picture quality for hours and frames may be viewed as "stills“
or in sequence as motion. In addition. with the stereo/audio
capability. it is possible to give the user a choice of narrator track
as in a dual language program.

There are two major videodisc systems: optical and capacitive.
Within each of these major types are variations. The systems are
incompatible because players differ in the way information is picked
up from the surface of the disc for playback. Optical players are
produced as optical-reflective and optical-transmissive.
Optical-reflective players use lasers to write (record) and read
(playback) a videodisc. Light reflected off the disc's surface is

converted to electrical signals from which video and audio signals are

derived. The optical-transmissive players use a translucent disc. so

22

the reading laser passes straight through the disc allowing the disc
to be read on both sides consecutively simply by repositioning the
laser beam. The capacitive system players are also of two types.
Both systems use a pickup stylus or sensor to transmit recorded video
and audio information. However. the two systems have incompatible
formats. The Radio Corporation of America (RCA) uses a grooved disc
with the stylus riding on the disc. The Japanese Victor Company (JVC)
is essentially a grooveless system which has the stylus gliding across
the surface of the disc without mmking actual contact. It is
generally felt that this reflective-laser system is the most durable
since the essential information is recorded well beneath the tough
protective coating on the disc. The RCA capacitive system is viewed
as the least durable since discs are worn by the stylus and easily
damaged by scratches. marks and dust. Further. the capacitive systems
lack the same flexibility of speed. freeze frame and audio quality of
the laser discs.

Interactivity of videodisc has been categorized by the Nebraska
Videodisc Design/Production Group (Parsloe et al., 1983). It is a
five point scale which generally can be applied to both videodisc and
videotape. Level 0 indicates a home use system and is generally not
seen as having any interactivity. Level 1 contains: remote control.
a search facility. freeze frame. forward and reverse motion. quick
scan. slow motion and step frame replay. It is the simplest level of
interactivity.

Level 2. the midpoint of the scale. offers the first true
interactivity. The equipment contains a small computer in the

videoplayer which allows for branching away from the main body of the

23
program into loops or tangents. Additional features to Level 2 are
scorekeeping and multiple choice. The instructional program cannot be
changed in any way in Level 2 because the video. audio and computer
programs are mastered on the disc; therefore. once the information is
recorded on the disc it is impossible to make changes without
re-mastering the disc. This is a serious disadvantage if changes are
required. The hardware consists of a videodisc player. its monitor
and keypad. Level 2 usage is mostly in comercial application such as
point-of-sale (Parsloe et al., 1983).

Level 3. the subject of this study. ”represents a video player of
any kind linked to an external computer-mainframe. mini or micro“
(Parsloe et al., 1983. p. 17). This level offers the greatest
versatility. A more sophisticated level of branching may be used and
the shelf life of a disc may be increased by careful planning. The
video material can be used with any number of programs and the
computer programs are easily changed. Furthermore. the computer can
generate text and graphics which can be used to enhance or supplement
the material recorded on video. Level 3 is used in full scale
simulation where an added computer is required to coordinate responses
from disparate parts. In a more modest manner. it is employed in
teaching and training where computer-generated graphics and text are
required (Parsloe. 1983).

Level 4 is defined as the workstation comprising the video and
computer equipment. and the furniture. It also is seen as the
“speculative level of technology as yet unattained by existing
equipment.“ (Parsloe et al., 1983. p. 17). Parsloe's speculative

level includes enhancements in hardware as well as the theory of use

24
such as artificial intelligence. Dyane 5. Butler (1984) also define

Level 4 as “... a theoretical domain in which all things are possible"

(p. 13).

Problem Areas in IVD

There has been a paucity of research on problem areas in IVD
systems. At the present time. only one major study (Klotz. 1983) has
been conducted which systematically investigated the problems and
complications involved in producing interactive videodisc systems. In
this study. Klotz (1983) conducted a survey of people who had attended
an interactive videodisc symposium or workshop at the University of
Nebraska ETV Center. The greatest areas of difficulty in producing an
IVD were in computer programing. shooting. scripting. editing. and
single frame edits. When Klotz reviewed the degree of difficulty in
relation to levels of design. the data suggested the more
sophisticated the level. the more likely problems will occur. This is
logical since programing sophistication increases by level. The
particular problems identified were in editing and single frame edits
for Levels 2 and 3 because of the nature of editing for videodisc and
the complexity of the program material in various projects.

Time was reported as being most heavily impacted by scripting.
single frame edits. and computer programing. Other problems related
to time were: turn around time for disc mastering. and delays in
hardware or promised new features in hardware by the manufacturer
(Klotz. 1983).

Additional problems identified by Klotz referred to:

l. The client's lack of knowledge and inability to explain all

the possibilities and problems.

25

2. Assembling a multi-disciplinary team that could comunicate

and work together.

3. Deciding what to put on disc and what to put on the computer.

4. Planning enough at the outset of the project to keep the

details coordinated.

The solution to the problems encountered usually were to put more
time and money toward the effort and continue finding solutions
through trial-and-error. Klotz did not collect any data on costing
though many comments by his respondents reflected a high concern about
the cost. Nany of his respondents reported their companies have made
a conitment to IVD and presently view the high cost and
non-profitability of the medium as an investment in the future.

A review of other published literature revealed that some authors
tend to oversimplify. For example. Clement (1981) wrote:

Videodisc producers generally agree that
creating a videodisc progrmm is relatively
easy. Organizations with the skills to produce
a videotape or film should have little problem
making a videodisc. Authoring systems now
available greatly aid the process of creating
program interactivity. (p. 13)

Although Clement states that the production may be relatively
easy. it isn't clear what Level of production he is defining. Nor can
we judge what he. determines to ‘be adequate “skills to produce a
videodisc.“ Such writings are often misleading to the uninitiated in
IVD. Further. it is unclear how many producers agree with him. It
appears from Winslow (1981). Gayeski (1984-85). Gayeski and Williams
(1984). Floyd (1980). Etherington (1981). and Hon (1983) that. the
opposite is true. They write of the complexity of IVD. not so much in

terms of the hardware involved but the program content and programming

26
required for sophisticated branching.

Etherington (1981) describes the complexity of IVD and decision
making as follows:

Options is the only word that comes closest to
describing what it is like to produce for
videodisc. Assuming there is a micro-computer
coupled with the disc player. options may be
left open from the shoot all the way to viewing
the disc. It is like working with a piece of
putty that can be molded to your specific need
at any step along the way. The layout of the
program breaks down into various segments which
can be individually accessible. or can be
combined in any order to conform to the desired
presentation. Add some new menus with a
character generator. or enter a new series of
commands into the micro-computer. and the whole
structure of the program can change. (p. 37)

Parsloe et al., (1983) elaborate on problem areas of
flowcharting. documentation and record keeping. and scripting. New
personnel to IVD are frequently cautioned to speak with experienced
people in the process to avoid costly errors. This researcher knows
of an instance where a government agency undertook the production of a
videodisc without prior experience. After identification of the
tasks. they proceeded to shooting of each task in a sequence. Seven
weeks were spent in shooting and fifteen months in editing before the
project was aborted and started over. The lack of planning and record
keeping made editing of the disc impossible. There was no immediate
means of tracking pictures. a lot of needless duplication in shooting
occurred and shooting time increased dramatically. Unfortunately.
while this approach may have worked for linear video. interactive
video requires a different type of planning.

Another difficulty addressed in the literature refers to the

establishment of a production team. Parsloe et a1. (1983) explain:

27

The size of the production team. like that of
the budget. ought to reflect the scale and
complexity of the project. Not that it always
will -- getting the numbers right is a problem
in jobs of all kind Anyone who works in
[the] field knows of jobs that went wrong -- of
very small companies producing monumental work.
of teams so large that talented people sat
idle. of projects so mismanaged that the full
weight of the project fell on one person ..... '
(p. 205).

Besides the team size problem. team composition in terms of talent and
skill is another major factor. Professionals with CAI or video
experience have old skills which need to be modified and new skills
which need to be learned (Gayeski. 1984-85; Floyd. 1980; Non. 1980 and
Parsloe et al., 1983). The extent of the problem is explained by
Parsloe et al. (1983):

Teamwork is a combination of talent. hard work
and good planning. and is a function of
management and personality as well as
experience and training. Interactive video is
still on such a growth curve that there are not
enough people with qualifications such as
formal training and practical experience to
meet the demand. This may be the case for a
few years yet [creating] a danger of some
clients' falling into the hands of production
companies who can't deliver the goods. (p. 205)

Parsloe goes on to say:
From the production company's point of view ...
Those people with genuine interactive
production experience have the pick of jobs

(and salaries) and the rest have to fall back
on a combination of relevppt egperience and a

willingness to learn the trade. (p. 205)

The problem of team members is greater on small projects since
people have to develop all around experience rather than become
specialized. It is difficult. if not impossible. to find the right

combination of skills.

28
The types of personnel on the team typically are (Parsloe et al.,
1983):

a System Analyst or Problem Analyst c the architect of the
program. A person who sees how something is done or ought to
be done and then designs and implements new and better ways
of proceeding.

0 Instructional Designer = the person who knows how to arrange
and present the information in a way that its audience will
enjoy and understand.

0 Information Programer = translates the work of the
instructional designer into computing terms.

0 Audio Visual Production . employs a typical video production
team of a producer. director. production assistant. subject
matter expert. art director and possibly a script writer.

a Project Nanagement = involves a production secretary tasked
with the administrative work and the project manager
responsible for all administrative and material
considerations.

A final problem addressed in the literature is the cost factor.

To date. no clear guides exist to tell what an interactive videodisc
program will cost. While the literature on coating is sparse. the
authors (Bright. 1983; Floyd. 1980; Gayeski. 1984-85; Kirkpatrick 8.
Kirkpatrick. 1985; Klotz. 1983; Kribs. 1979-80; Naxwell. 1985;
Nerrill. 1980; Parsloe et al., 1983) agree that there is a real need
for better definition of the cost to make IVD a more viable medium.
When looking at the medium there is a difference of opinion on what

drives the cost. Parsloe et a1. (1983) say:

29

The cost of the project will reflect both
production expenses which do not differ greatly
from those of conventional linear video. and
costs run up in a design stage which is
considerably longer and more complex than the
traditional medium.

The time taken to select and develop delivery

and authoring systems. to research a project

and to analyze its component parts. to prepare

draft flowcharts. scripts. storyboards.

computer programs and documentation. can

account for more than half of the production

schedule before a foot of video is shot.

(p. 221)
Nerrill (1980) concurs that production costs are the same as for
standard film or video tape. Bright (1983). on the other hand.
indicates the cost of actually pressing a disc is prohibitive. along
with the necessity for large staffs of people with varying skills.
Winslow (1981) cements on the hardware associated with IVD as very
costly:

This kind of videodisc application begins with

a player for “$2.500 to $3.000 then adds $1.500

and up for mastering and replication of each

30-minute program side -- and all of this is

before we get down to the really serious.

above-the—line costs involved in designing.

scripting. computerizing. testing. validating.

and installing a given instruction. training.

and information program. (p. 38)

While cost and control of cost is a major issue. the procedures

for costing an IVD project is seen as an even greater problem as

discussed below.

Costipg Procedures form
The preparation of a proposal with detailed cost figures to cover
the span of a project presents a challenge given the varying degrees
of content as well as design complexities which can be pursued. To

date. no research has been done on cost models which directly relate

30

to interactive videodisc. While some use costing models for computer
aided instruction (CAI) and join them with cost models for linear
video production as a base. numerous cost items are not accounted
for. Further. many of the cost models in CAI are calculated on some
comon rule of thumb such as “100 to 200 hours of development time for
every hour of instruction developed“ (Kearsley 1983. p. 56). Of
course. this general rule is then modified by Kearsley with the
explanation. “The amount of time required (and hence costs) will vary
considerably with the type of C81 involved. the capabilities of the
system and the experience of the personnel“ (p. 56). Gary (1987).
argues that the concept of hours of development time per hours of CBT
is invalid “because in extensively branched. learner-controlled. or
inquiry-based instructional programs. we can't define 'an hour of CBT
[Computer Based Training)” (p. 178). Like Kearsley. Gery says time
and cost depend on situational variables which must be determined on a
case-by-case basis. Gery further writes that the ratios quoted of
25:1. 150:1 or 400:1 are usually unsubstantiated industry averages.
Translated into dollars the ratio figure for CAI development are
quoted anywhere from $10 per minute to $600 per minute. This provides
little guidance for CAI development much less for IVD development. To
this base calculation must be added the cost of hardware and software
acquisition and if the project entails implementation. the operational
costs. as well as expenses associated with personnel and facilities
(Kearsley. 1983).

In a 1985 Society for Applied Learning Technology (SALT) workshop
on CAI. Research and Planning. Inc.. provided CAI costing worksheets

with relative values (See Appendix A). Gery (1987) in her work has

31

identified and summarized all of the general CBT variables referred to
above into four major categories: courseware. technical. human and
other. Within these she has identified more discrete variables which
affect time and cost. These variables are then plotted on a scatter
diagram from low to high in terms of development time and development
cost. Then. defining an hour of CBT as“...a course that is linear in
nature. includes conditional feedback. and restricts the use of
conditional branching to review segments“ (p. 188). Gery shows a
development graph of 85-150 development hours per CBT hour as low cost
and time. 150-300 as medium cost and time. and 300+ as high cost. The
cost of the project is then estimated based on where the preponderance
of the variables fall for each project. Adjustments would have to be
made to the hours if the CBT being developed was more interactive and
complex than the definition outline. Gery readily admits. to estimate
accurately using this model. you would have to be a highly experienced
and seasoned CBT developer.

Nest of the other CAI literature refers to cost benefit models
(Heath. 8. Orlich. 1977; Kearsley. 1982 and 1984; Kielt &. Spitzer.
1977). which do not provide the guidance required by developers. or
acquisition models for existing software (Pressman & Rosenbloom. 1984).

Production costs in interactive videodisc systems are often based
on linear video costs. Whitney (1980) cited the following costs:

0 Edited material on 16mm costs - $500-$700 for premastering

a Nastering costs equal about $1,500

a A computer program added to the disc adds another $500-$1.000

0 Cost of the disc is $10 ($5 per side).

Since Whitney does not elaborate on the length. details. level

32
and number of discs for the program. one can not use the data as a
formula for coating IVD projects.
Daynes (1984). when questioned on the use of costing projects by
cost per running minute. indicated the following figures:

For a computer-controlled interactive videodisc
with every available option. including a touch
screen. computer graphic overlay. still frame
audio. and a very high level of production
value. the production cost could run as high as
$10,000 per linear minute. For a 30-minute
disc that is a pretty expensive program.
However. costs. in my experience. average
around $3.500 per linear minute. This includes
both design and production costs. (p. 32)

Kulvicki (1985). at a SALT workshop for interactive videodisc.

disseminated a costing equation for the mastering of a disc:

 

 

Cost/Hr. =

f ; a Rsrnsscnum(neoplasm-19g Cost + Cost

.5 e 1 - 1.8F + 1 SF Narket Size Per
100 360 Disc

SF . number of single frames
SFC a single frame production costs (Bennion estimates 810)
HP = number of motion frames
NFC s motion frame production costs (Bennion estimates 3.20)
360 = number of still frames for one hour of instruction (10 sec .1
frame)

1 SF a percentage of disc space containing still frame instructions

No further explanation is provided on application and usage.
Kribs (1979-80) provides costing information on the videodisc delivery
system by capability. A consumer model with passive audio-visual
programs. some programmed instruction. and without an authoring
feature is estimated at $700. The range than extends to a full video
resolution color graphics with NTSC color overlay which can be used
for interactive equipment simulation. This version allows for full
video resolution color graphics to be developed and modified without

disc mastery. The price estimated is 320.000. Into these figures

33
must be added the cost of development and production.

Nerrill (1980) breaks the costing into development and
production. mastering and replication. and equipment. Nerrill sees
the basic cost of producing master sequences as ranging from 81.000 to
$4.000 per minute. The production of still frames using original art
and revisions will cost $100 per frame. These figures do not include
the front-end instructional design costs. If the videodisc is to be
controlled by’ a microcomputer. the production costs *would require
computer programming. The standard costs for programming PLATO
courseware extend from $50 to $200 per hour of instruction depending
on the skill of the author (Nerrill. 1980). Kalba (1978) reported
that different authors estimate the development and production costs
as ranging from $5,000 to $700,000. The low and estimates involve
assembling materials from. existing sources with. minimal front-end
costs. while the latter includes expensive original motion sequences.
Thus. we can conclude it is extremely difficult to estimate the
development and production costs with the large number of variables.

Nerrill (1980) found that mastering and replication costs for IVD
by Disco Vision Associates cost 83.500 for two-sided video disc with
30 minutes per side. The more discs produced. the lower the overall
price. i.e.. 3000 copies could cost 833.000 or $11 per disc.

Equipment costs vary with the sophistication and complexity.
Nerrill estimates costs from $500 to $3.000 per unit. These prices
are consistent with figures given by Winslow (1981). Kirkpatrick and
Kirkpatrick (1985). Kribs (1979-80). and Daynes and Butler (1984).

Parsloe et a1. (1983) provide more guidance on the elements which

make up the cost of an IVD project. They divide the cost and

34

labor into three phases. Phase I involves only people. This phase
covers all aspects of the research and design. It is labor-intensive
and considered to be the most difficult to estimate. All team.members
may become involved in this phase. Phase II requires cost and labor
estimates to cover people. facilities. equipment. stock. goods.
royalties. incidentals such as travel. in—house administrative items
and production costs. This phase requires the listing of every single
activity in the production schedule. citing the individuals whose work
is needed. the time allotted to each task and all the required
facilities equipment and supplies. Phase III is the information
programing. where it all comes together. In this phase. the fine
detailing in the design of both individual computer screens and the
branching pattern as a whole take place. Once completed. instructions
to the computer are input and then linked to the video player. The
last aspect is testing the product. In this third phase. the delivery
system equipment and entire team of people are again included in the
cost. Parsloe et al. indicate this is also “a labor-intensive phase.
but easier to budget than Phase I since the team now have a reasonable
idea of the general size and scope of the work involved“ (p. 228).

Several aspects surface in the Parsloe. et al. model. First. no
guidance is given on how to cost design variables or to judge time.
The guidance: is merely that of content. categories. Secondly. the
authors presume coating will be accomplished in phases. thus providing
insight to management on cost variables and a more accurate assessment
of time as the project progresses. However. most projects for
industny and government often require the submission of a firm-fixed

price for the total project before any work is begun. Such costing is

35
prone to severe errors in judgement because it is often undertaken
without benefit of a content analysis or a design specification which

could provide some insight.

Case Study and Field Research
Case study as developed by anthropologists and sociologists is a

qualitative method of research which is well documented in the
literature. Franklin a Osborne (1971) show that the early literature
reflects a conflict between qualitative and quantitative research by
citing such works as Lundgren's C_a;e study .vs. statistical methods
done in 1941. In that particular work. Lundgren wrote that each
method has a purpose and may supplement or contribute to the other.
While the debates still linger on today over quantitative vs.
qualitative. it has come to be an accepted fact that both types of
research are important. often complementing each other in a research
design (Campbell. 1979; Isaac & Hichael. 1979; Patton. 1980; Reichardt
8. Cook. 1979). Campbell (1979). a recognized guru of quantitative
research. goes to great lengths to correct his excesses against case
studies. He indicates that qualitative research. like quantitative
research. has weaknesses but often times quantitative findings can
only be explained by qualitative research. Specifically. Campbell
cites research where results in quantitative research cannot be
replicated in subsequent research. It is often only through
qualitative study that the confounding variables can be learned to
provide a premise for further quantitative research. He stresses that
quantitative research often generates what is happening without being
able to explain why. while qualitative research may help to generate

the theory or explanation behind the phenomenon. Campbell cites

36

qualitative research as the common-sense knowing upon ‘which
quantitative research should build and go beyond. Campbell suggests
the use of both qualitative and quantitative methods to clearly study
a phenomenon. ‘Where the results of the qualitative research. are
“contrary to the quantitative results. the quantitative results should
be regarded as suspect until the reasons for the discrepancy are well
understood“ (p. 52).

While researchers concur that both quantitative and qualitative
methods combined make for a stronger research design. Reichardt and
Cook (1979) admit this is often difficult since it makes a study more
costly. requires more time to do and most researchers are not
sufficiently trained in both methods. Thus. one can conclude that the
driving force for selecting a research method should be the research
question. and a research methodology should be selected which can
adequately answer it.

The case study approach is defined as “... an intensive. detailed
analysis and description of a single organism. institution. or
phenomenon in the context of its environment” (Anderson. Ball. Hurphy
& Associates. 1979. p. 46). Isaac and Nichael (1979) describe it in a
similar manner. '[an] in-depth investigation of a given social unit
resulting in a complete. well-organized picture of that unit“
(p. 20). Franklin and Osborne (1971) approach the definition a bit
differently:

The case study ... is not a specific
technique. Rather. it is a method of
organizing data for the purpose of analyzing
the life of a social unit ... [using] both the
historical and functional approach in an effort

to see the social unit as an integrated whole.
(p. 184)

37

Franklin and Osborne define the historical approach as the use of
existing documents. records. life histories and statistical records
which may provide critical information. The functional approach
involves the collection of data through observations. interviews.
questionnaires. and surveys. Essentially. the functional approach
generates all of the information for interpretation while the
historical approach interprets existing information to explain a
phenomenon.

All of the literature agrees that the case study approach allows
for the gathering of information about a select unit to define some
characteristics about that unit of interest to the researcher. Isaac
and Nichael (1979) indicate the purpose of the study will determine if
all or only selected segments of a unit will be studied.

Host of the researchers agree the greatest benefit from the case
study is the determination of “important variables. processes. and
interactions that deserve more extensive attention" (Isaac 8. Nichael.
1979. p. 20). Glaser and Straus (1970) and Patton (1980) concur
indicating case studies are a valid means of research to derive
concepts and interrelationships to form a set of hypothesis for a
given area. Foreman (1971) sees the use of case study for the
purposes of illustration. concept and hypothesis development.
hypothesis testing. prediction or postdiction. methodological testing
or refinement. He clearly points out the use of the case study is
best in new fields of research with little prior accumulated data or
when the research problem demands conceptualization.

Weaknesses of the case study method center on the lack of

representativeness and consequently the inability to generalize the

38

findings to other units (Campbell. 1979; Isaac E Hichael. 1979).
While this is an accepted weakness by this researcher. it also is
believed if the purpose of the study is to generate findings so that
others might generate hypotheses for further study. the inability to
generalize is not a negative factor. A second weakness discussed by
Campbell (1979). Isaac and Nichael (1979) and Patton (1980) is the
vulnerability to subjective biases. Huch of this bias can be
controlled through sampling. and the procedural methods adopted for
data collection and data analysis (Foreman. 1971; Patton. 1980; Whyte
1982).

By establishing prior criteria for sampling. a researcher's bias
can be circumvented or at the very least minimized. To do this. the
statement of the problem must be clear and the research questions
outlined. Sampling may be random if generalization to a larger
population is desired. Purposeful sampling "is used as a strategy
when one wants to learn something and come to understand something
about certain select cases without needing to generalize to all such
cases“ (Patton. 1980. p. 100). This is generally the sampling
procedure which is used for the development of baseline data.

Bias during data collection may be controlled through the use of
interview guides (Patton. 1980; Whyte. 1982). The guides ensure
similar data is collected from all respondents while allowing for a
free flow of information. An additional method for control of bias
during data collection is to cross-check information with more than
one respondent or with other researchers (Patton. 1980; Foreman. 1971;
Whyte. 1982). In addition. all qualitative researchers indicated bias

can be controlled through the use of research interviewing techniques

39

such as the use of a tape recorder to prevent researcher
misinterpretation or forgetting of key information; the use of
open-ended questioning; and conscientiously refraining from judging
the responses which might lead the respondent (Patton. 1980; Whyte.
1982). Finally. by writing notes and compiling data as soon after an
interview as possible. selective recording of data will be
circumvented and a more accurate record will be generated.

Bias during analysis can be controlled in several ways. Foreman
(1971) suggests controlling bias by doing a comparison across cases.
To conduct such comparisons between cases as well as within cases
Patton (1980) and Whyte (1982) construct and reconstruct patterns.
categories and typologies to ensure that the best fit to the data is
achieved. To check the validity of the data analysis. an analysis of
the case data can be conducted by independent researchers (Foreman.
1971) and the results compared for agreement. Patton (1980)
recounends checking the validity of the data analysis by having the
people described in the data analysis react to what is described.
Patton suggests from this reaction. there is “a great deal which can
be learned about accuracy. fairness. and validity” (p. 338).

The entire issue of bias can best be sumed up through Campbell's
(1979) discussions with Becker. Becker found that invariably the
researcher undertaking an intensive case study ends up finding out

that his prior beliefs and theories were wrong.

Chapter Sumary

A search of the literature shows that there is a paucity of
reliable cost models which apply to IVD development and production

projects. The study by Klotz clearly shows there are problems in IVD

40
development and production. Further. writings of Floyd. Gayeski.
Kribs. Parsloe et a1. and others indicate considerable trial-and-error
experimentation with the medium.

Present writings are too general to provide the guidance required
to avoid the present trial-and-error approach. A better understanding
of the requirements to produce a Level 3 IVD will help a client to
more accurately fund and the contracting agency to more accurately
cost a project. This in turn will better assure that clients will
receive the product for which they contracted and that the contracting
agency will not suffer a dollar loss in producing that product.

The case study approach is cited in the literature as a viable
qualitative research methodology to derive important variables.
processes. and interactions. The results of qualitative research are
then subjected to more indepth study using other research methods.
The case method is particularly good for study in areas where little
prior data exists and there is a need for conceptualization. such as

found in the coating of interactive videodisc.

CHAPTER III

HETHODOLOGY

As indicated previously. the study was designed to collect
baseline data on the variables associated with time and cost overruns
in Level 3 interactive videodisc (IVD) development and production. In
this chapter a description is provided of the study design. sampling
plan. data collection instrument. data collection procedures.

preparation of data. and the data analysis.

Study Desigp
This study used a case method approach to derive the key problem
variables which affect time and cost in Level 3 IVD projects for
industry and government. The case method is recognized as a
legitimate method of inquiry to pioneer new ground and uncover
variables which deserve further research. Anderson. Ball. Nurphy and
Associates (1975) define a case study as “... an intensive. detailed
analysis and description of a single organism. institution. or
phenomenon in the context of its environment“ (p. 36). Isaac and
Nichael (1979) define a case study as an:
In-depth investigation of a given social unit
resulting in a complete. well-organized picture
of that unit. Depending upon the purpose. the
scope of the study may encompass an entire life
cycle or only a selected segment; it may

concentrate on specific factors or take in the
totality of elements and events. (p. 20)

41

42

Isaac and Nichael further explain:

Because they [case studies] are intensive. they

bring to light the important variables.

processes. and interactions that deserve more

extensive attention. (p. 20)
Glaser and Straus (1970) and Patton (1980) indicated case studies are
a valid means of research to derive concepts and interrelationships to
form a set of hypotheses for a given area.

Since the literature to date is sparse and general on the
processes and procedures used in all phases of IVD development. and
little research on costing IVD is published to date. the case method
provided the most viable means to build a bank of baseline data to
meet the objective of this study. answer the established research
questions and provide data for the establishment of hypotheses in
future research. Data from three Level 3 IVD projects for industry
and government were collected and analyzed. Each case selected met
the criteria established and outlined in the sampling plan.

The data were collected by the researcher through in-person
guided interviews and examination of project documentation and
products. Complete cases were built by interviewing various project
personnel from the contracting agency so as to obtain the perspectives
and facts from the project manager. instructional designer.
information programmer and video producer. On some teams a single
individual filled the requirements for more than one position on a
team. ‘while in other cases several individuals performed discreet
tasks of a single position. In some situations. one position was held
by several different people at varying stages of the project because

of expertise or because of staff turnover. In each case. enough

personnel were interviewed to adequately cover all the project

43
aspects. The study focused on the problems or processes which created
a dollar loss to the. contractor and only contractor personnel were
interviewed.

All subjects and agencies in this study' were provided
confidentiality. Each case has been coded by numbers and by letters
for the subcontractors within a case 'where necessary. All other
identifying information has been screened and/or coded in an

appropriate manner.

Research Questions

The specific focus of the investigation was delineated in terms

of the following research questions:

1. What type of knowledge is needed and when does it have to be
available to make time or cost estimates for an IVD project?

2. To what extent does client IVD knowledge. as reported by the
contractor. have an effect on time and cost of an IVD project?

3. What effects. if any. do contractor agency size and
capability have on time and cost of an IVD project?

4. What functions does each team member perform in the process
of carrying out an IVD project?

5. What effects. if any. does project team organization. and
personnel IVD training and experience have on time and cost
of an IVD project?

6. What activities in a phase have a time and cost effect on
subsequent phases of an IVD project?

7. Is one phase more likely than any other to affect time and

cost of an IVD project?

44

8. What effects. if any. does IVD hardware have on time and cost
of an IVD project?

9. What effects. if any. does IVD software have on time and cost
of an IVD project?

10. To what extent do unintended events and unplanned events
create a time and cost overrun in an IVD project?

11. When and under what circumstances do unexpected costs occur
in an IVD project?

12. When and under what circumstances do unexpected time delays

occur in an IVD project?

Sampling
Sample Selectiop

The final study sample consisted of three cases. All cases were
selected by the researcher from a list of nine Level 3 IVD projects
secured through contacts of the researcher. contacts of the Editor of
Videodisc Nopitor and contacts of the Director of the Society for
Applied Learning Technology (SALT). All projects selected represented
purposeful samples (Patton. 1980) so as to ensure that the objectives
of the study would be met.

For purposeful sampling and the conduct of basic research. as few
as one case may constitute a legitimate study. A minimum of three
cases was established to cover possible variability due to content.
staffing. hardware and project size. The established minimum of three
cases allowed for a means of comparison and enough data from which to
derive more definitive variables. Of the nine projects identified.

eight projects met the following established criteria:

45

1. A Level 3 IVD training project as defined by the Nebraska
Scale.

2. A completed or nearly completed project so data could be
collected from the analysis. design. development. and
production phases.

3. A contracted training development project for government or
private industry.

4. A fixed price contract which experienced a cost overrun.

5. The availability of more than one team member for
interviewing. covering the major positions of project
manager. instructional designer. programmer and video
producer.

After the initial phone contacts to verify the projects against
the criteria. two of the eight companies were unable to participate
because of litigation or an inability to get approval from company
management. The remaining six projects were accepted as valid cases
for the study.

During data collection interviews. it became evident that three
projects were actually a part of one large project for a client and a
fourth company was a subcontractor to one of the three within this
same project. Although there ‘were differences identified between
these four companies in such aspects as project management. team
composition and contractor organization. the client established
methods. procedures and equipment to be used by all the contractors
producing discs. To avoid bias. the data from the three projects were

treated as one case for purposes of this study. The net result was a

total of three cases for the study.

46
Sample Case Descriptiop

Case 1 was undertaken by a large state university in 1985 to
produce six double sided Level 3 IVDs with accompanying print
materials for a large firm in private industry. This project was a
conversion of an existing audio-tape and print program to interactive
videodisc. The contract required mainly development and production
phase work.

Case 2 was a contractual effort undertaken by a private training
consultant and development firm for a branch of the military. The
effort was begun in 1984 and is presently being readied for delivery.
This project. was for the development of maintenance training for
highly sophisticated equipment. All phases from analysis through
final production were a part of the contractual effort.

Case 3 was undertaken by several private training development
firms to produce eleven Level 3 single-sided IVDs for a manufacturing
company to teach maintenance and repair of their product. Five of the
discs were conversion efforts from Level 2 IVDs. The remaining six
were new discs. The project had three separate contractors.
Contractor A produced four single-sided discs. Contractor 8 produced
one single-sided disc and Contractor C produced six new single-sided
discs plus the computer managed instructional (CHI) system which
supported all of the discs in the project. Each contractor within

this case took the disc from design through development and production.

Qgta Collection Instrument
The literature was reviewed and an interview guide. to include
demographic data. was developed. The interview guide approach as

defined by Patton (1980) “involves outlining a set of issues that are

47

to be explored with each respondent before interviewing begins“ (p.
198). Both Patton (1980) and Whyte (1982) agree that the issues are
not covered in any particular order or with any standard wording. The
actual questions to elicit the information are generated during the
time of the interview. The guide merely serves as a checklist to
ensure that similar information is collected from all the people
interviewed. Whyte (1982) explains best the reason for the guided
interview:

In research we want the informant [respondent]

to talk about things of vital interest to him.

but we also need his co-operation in covering

matters of importance to the researcher. though

possibly of little interest to the informant

[respondent]. (p. 112)

The interview guide approach was used in this study to help
ensure the systematic collection of similar data across cases and from
individual subjects within cases; establish an initial organization of
content for analysis purposes; provide topical areas within which to
explore. probe and ask questions that would elucidate and illuminate a
particular subject; and keep the interviews focused. The interview
guide included the collection of direct case related content.
demographic information such as experience and training of team
members. and contractor organization size. age. description. and
experience in the medium.

The final 9-page interview guide included an orientation to the
subject regarding the purpose of the study. an assurance of
confidentiality. and an elicitation of the project description to set
interviewees at ease. Sixty-five questions were developed and grouped

under the following topics: (a) General Information. (b) Analysis

Phase. (c) Design Phase. (d) Development Phase. (e) Production Phase.

48
and (f) Budgeting Phase. The interview guide used in this study is

presented in Appendix C.

Data Collectiop Procedures

 

Data for this study were collected through audio taped in-person
guided interviews. follow-up telephone interviews. and examination of
documentation and products made available by the subjects of this
study. Documents provided varied between cases. Some of the types of
documents included Requests for Proposals (RFPs). proposals.
instructional and system specifications. trade-off analyses and
monthly progress reports. In several instances. correspondence and
personal progress logs were provided. The study was conducted over a
four month time period extending from November 1986 to Narch 1987.

A list of potential cases for study was obtained as outlined in
the sampling plan. The identified contracting agencies were contacted
by phone to enlist their cooperation and determine whether the case
met the established sample criteria. If it did. an appointment was
made for a personal interview on-site. Each phone contact was
followed with a letter and explanatory sheet (Appendix B) about the
research study. confirming the appointment and outlining the time
requirements.

Pilptggpterview

A pilot interview was conducted with one of the sample cases
using the interview guide. Since no substantive changes were required
to the interview guide or overall interview process. the pilot
interview remained as a part of the sample in this study.

Interviews

On—site. audio taped interviews of contractor personnel using the

49

Interview Guide were conducted. It was calculated that from eight to
eighteen hours of interviewing time were required per case depending
on the size. complexity and number of personnel on the team of the
contracting agency. Each initial interview lasted from two to three
hours. The interview' notes and content. were reviewed immediately
following each interview to expand notes into) more comprehensive
detail. Thank you letters to agencies participating in the study were
sent.

Case Record

A case record. or compilation of data. was developed for each
case by transcribing interview tapes. organizing field. notes. and
editing redundancies and ordering the content. Once this was
accomplished. follow-up interviews were conducted for content
clarification and collection of additional data overlooked at the
first interview; One project manager. instructional designer and
programmer were contacted for additional information in each case.

As expected. using this data collection methodology. more
information was available from some individuals than from others.
These differences were of concern to this researcher since one cannot
always be certain how the findings are influenced by these qualitative
differences in the depth and breadth of information received. Patton
(1980) explains that for the conduct of basic research one is
attempting to understand the whole so differences such as these are
not as much of an issue. However. the differences were minimized as
much as possible through the application of interview principles. use

of the interview guide and conduct of follow-up interviews before the

analysis began.

50

PriqciplLes for I_nperviewit_:g

To ensure the success of the interviews. the principles of

interviewing as outlined by Patton (1980) were applied by the

interviewer in this study. These principles include:

1.

Use of open-ended questions. The use of open-ended questions
avoids a predetermined response. Open-ended questions allow the
respondent to answer in their own terms taking whatever direction
they want in order to represent what they want to say. It is
known that a series of closed questions is likely to lead the
interviewee to giving what they expect the interviewer wants to
hear. making the data unreliable.

Use of singular questions. Asking a single question at a time
will avoid confusing the interviewee and will help ensure a clear
and complete response. In addition. it *will avoid problems
interpreting the response during the analysis process.

Use of peutral questi_o:_|_s_. Patton defines neutrality as meaning
“... that the person being interviewed can tell. me anything
without engendering either my favor or disfavor with regard to
the content of their response“ (p. 231). To do this. Patton
suggests developing rapport by conveying empathy and
understanding without judgment. Both neutrality and rapport can
be handled by asking illustrative questions which show the
interviewer that no response will be a surprise.

Use of clear questions. To ask clear questions. the interviewer
must use mutually understood terminology and labels. This
includes using language and terminology that is understandable

and part of the frame of reference of the person being

51
interviewed. Questions using the respondent‘s language are
more likely to be clear to the respondent. Patton further
suggests using prefatory and transitory statements to help
the interviewee focus on the subject of immediate interest to
the interviewer.

5. Haintain control of the ipterview. Time is precious and can be
used most effectively by knowing the interview guide. asking the
right questions to get the desired answers. and giving
appropriate verbal and non-verbal feedback to the person being
interviewed. To accomplish this requires the interviewer to
listen carefully to make sure the response received provides the
answer to the question asked.

Beyond the direct. questioning principles. Patton (1980) makes
some suggestions for recording interviews. The following as
abstracted from Patton were applied in this study:

a An explanation was given to each interviewee as to why the

interviews were being taped.

0 Notes were jotted down in key phrases and lists of major
points were made to help in formulating questions. pacing the
interview and capturing the interviewee language.

a Recordings were conducted by:

- Checking the working condition of equipment prior to and
again at the beginning of an interview.

- Selecting a quiet area for the interview.

- Turning the recorder off during irrelevant discussions.

- Labeling all tapes properly with the name of the

interviewee. date. company and position. Code numbers and

52
letters were assigned to each case before analysis began.

0 Expanding notes after the interview by recording impressions
and adding information observed during the interview.

0 Checking each tape inediately after the session to make
certain the recorder functioned properly during the recording
so that notes could be made from recall if necessary. There
were no recording failures either during recording or

subsequent transcription of the tapes.

Data Analysis

The following general procedures to analyze the case data
collected were used. This researcher would like to caution the
reviewer at this point. that. case» method analysis procedures are
driven by the resulting content from the interviews. Patterns.
categories and themes were experimented with until the best fit
evolved from the data. Essentially. the following rules or guides
were established by this researcher for the analysis:

a The data in the analysis were to be tied to behavioral facts

which are more easily observed and verified.

e The facts gleened from the data had to be verifiable through a

cross check with other people on the project.

a Researcher derived facts from the data had to be verifiable

against facts derived by a second researcher.

a Two dimensional matrices would be used to develop an

understanding of the data and to present the data.

0 Examples were to be kept at a minimum for presentation of the

data to avoid a lengthy and cumbersome report which would be

difficult for the reader to follow and understand.

53

Case records were built for each case to serve as the major
source of documentation. Each case was constructed in accordance with
the phases of analysis. design. development and production. Within
these phases the data were organized by chronological order of events
and by team members to better determine when problem variables
occurred and which team.member(s) were involved.

Each case was examined and problem areas highlighted. Problem
areas were further examined to determine whether a cause was evident
or whether the problem identified was a symptom. For example. if a
programer indicated major time was lost because of bugs in the
authoring or programing language. the interview data were further
scrutinized to support. refute. or identify a different cause. The
real reason for time loss could have been the programmer's lack of
familiarity with the language. equipment incompatibility. or one of
many other possibilities. These variables were listed as primary or
secondary and categorized into phases in accordance with the
established effects.

An attempt was made to develop matrices and differing comparison
processes to cross check information. and identify variables which
might not be evident in any one type of classification or category
system. The exact type of matrices and comparisons were determined by
the data collected. The most suitable fit for the data appeared to be
in accordance with the IVD process phases. team position and research
questions. This allowed for some comparison and assessment of the
similarities and uniqueness of each case. Data matrices were analyzed
to arrive at a preliminary set of findings.

Another researcher. familiar with IVD and schooled in

54
instructional development. was asked to review the raw transcripts and
suggest possible matrices for the data. Since this second researcher
arrived at the same types of matrices as the author of this research.
copies of the author developed matrices were provided to the
researcher for analysis. The findings and conclusions of the
researcher were compared to those of this study's author. Differences
were discussed and resolved either by the collection of more
supporting data for the finding or the item was dropped if sufficient

evidence could not be found. The result was a final set of findings.

Chapter Sumary

The Case Study research methodology used to abstract the
variables which any cause Level 3 IVD project time and cost overruns
has been described in this chapter. The methodology involved a
researcher developed interview guide and audio-taped. in-person
interviews with project personnel on each of three cases.

Three cases were selected for the study in accordance with
preestablished criteria. All interviews were conducted applying the
interview principles and taping procedures outlined in Patton (1980).
Personnel holding the key positions of project manager. information
programmer. instructional designer and video producer were interviewed
for each case.

The data analysis involved several attempts at development of
matrices and comparison procedures built from the interview
transcripts until a good fit was achieved. The data were best
analyzed by phases and team position. Validity in the case data
analysis was achieved through the use of another researcher who

independently reviewed the interview transcripts. suggested matrices

55
and comparisons and derived findings. Where differences occurred in
findings between this author and the validating researcher. additional

data were collected to either support or refute the finding in

question.

CHAPTER IV

DESCRIPTION AND RESULTS

In this chapter. the results from analyses of the data for three
interactive videodisc Level 3 projects for industry and government are
presented and discussed. This chapter is divided into three
sections: (a) case descriptions. (b) results. and (c) sumary of
findings. The results section is further subdivided in accordance

with the research questions of the study.

gq§e_gescriptiop§

Three cases were selected for study. Case 1 and Case 2 were
individual contracting agencies. each with a client for whom they
produced a product. Host of the work required to produce the product
was performed by personnel within the contracting agency. Facilities
and/or services consisted of hired vendor services for audio
recording. video production and editing. However. each agency
remained very closely involved. frequently performing the work
themselves using the hired facilities and equipment. Case 3 consists
of three contracting agencies working on a single project for one
client. The individual contractors each had subcontractors for the
work. The relationship of the client. contractors and subcontractors

within the Case 3 project are as diagramed in Figure 1.

56

57

958%.»: 38258 e8 .56 n 88 .F maze:

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

_§__e.a__¥§_ airs... .....i
Z 1.. H H... I
:fii .. 7%.. i:

m 1 IR? s H
+

 

_ #2540 ;

 

58

The amount of project work performed by each major contractor
within Case 3 varied greatly. Contractor A subcontracted all of the
major activities. acting more in a role of administrator and procurer
of materials. In contrast. Contractor C performed most of the effort
internally. contracting for added personnel and facilities in video
production to supplement their in-house capability. Contractor B had
some in-house production capabilities. subcontracting mainly for
writers and programmers. As in Cases 1 and 2. the individual
contractors (A. B. and C) in Case 3 were required to go outside of
their organizations for audio recording and tape editing facilities.
The equipment and facilities required to perform audio recording. tape
editing and video lighting and shooting are highly specialized and
often are not a part of a general training contracting agency.

Key Case Elements

The basic elements of each case are presented in Table 1.
Because of the nature of IVD. the size of the project cannot be judged
by the number of discs produced or by the timeline within which it was
produced. The amount of disc instructional content varies by nature
of the format. For example. both motion and audio require large
amounts of disc space in proportion to the instruction presented.

Case 1 used audio extensively and no motion. Case 3 used both
audio and motion extensively. while Case 2 used almost no motion and
no audio. The product in Case 2 is composed of still frames and
computer generated graphics. Hotion is simulated through the
programming of the still frames. thus allowing for more instructional
content. In addition to the effect of presentation format on the

disc. one must note the differences in the amount of disc space used.

59

 

 

 

 

 

 

 

 

 

 

 

 

 

TABLE 1. Case Description
1 n s Cqso 1 Caso 2 Case 3
Contractor A Dnivorsity Collogo of Nilitary analysis and Private industry training
Education Department training firm. software dovolopmont fir-a with
onginooring. technical subcontracting to produc-
onginooring sorvicos tion and programming I
___ hqqqos
Cliont Equipment manufacturer and Department of Dofonso (DoD) Auto manufacturor
training dovolopqg qqqpqqiqq ainin a one __

Pnrpouo Convert filmatrips. audio- Dovolop s Lovol 3 IVD for Convort Lovol 2 IVD to
tape and print materials equipment maintonanco. Lovol 3.
to Lovol 3 Train SNEs in IVD dovolop- Produco now programs.

most and production.
Develop an authoring lan-
’ naugo fog limited qutiqg.

Product 4 doublo sidod discs: 1 doublo sidod disc: 00 ll singlo sided discs (5 .
still frame. audio. naintonanco tasks. vory lovol 2 convorsions. 4 -
graphics. studont work small amount of motion now discs): Computor
Journal. or installation soquoncos. still frame. so Hanagod Instruction pack-
guido audio. graphics. CH1 pack- ago. mostly notion. mini-

ngo. suthor/oditing pack- mal still framo. audio.
ago. student workbook. graphics

author/oditor documenta—

tign,

Timolino 10 months Approximately 42 months 4 - 9 months variod by
(1905-lgggl (3 1/2 your!) (less-12!); c 5- 986 .

Estimatod lnstruc- 50 Era of IVD Piguros woro not svailablo. Porty-fivo minutos to l 172 I

tional Tins 50 hrs of C81 udthout video Estimatos aro botwoon 30-50 hours per disc :

hrs.

Phases Ninimal dosign and dovolop- Analysis. Design. Dovolop- Design. Dovolopmont.

n c o P uct
Staff Project Ngr. (financial) Project Ngr./Inst. Dovolopor Throo soparsto contractors

Training Iiuipnonf

Configuration

rune

last. Dosignor/Producor
Inst . Dosignor/Progrs-or
Studont Graphic Artists (0)
Student Programmers (S)

(2)
Project Ngr./Progrsmmors (2)
Analyst/Editor (l)
Programmers (3)
Subject Nattor Exports (SEE)
(3 - 4) at a time

loto: Staff changod
throughout the project.
Usually 3 - 4 contractor
porsonnol worn on tho
project at a given tins.

 

touch scroon
keyboard

gqptrsctog A
Project Ngr.

Subcontractor A-l
Project ng.IDosignor
3 Programmers
l Dosignoerritor
Subcontractor A-2
1 Producor/Diroctor.
support staff

gqntgacto; !
Project Ngr.IDosignorl

Producor
Subcontractor B-l
Writer/Dosignor
Subcontractor 0-2
3 Programmers

t actor C
Project Ngr
Producer/Diroctor plus 5
pooplo
lust. Dosignors/Writors
(3 1/2 - 5)
Editors (2)

 

Sony View 2000
light pen
bar codo roador

gpyboarq (QQQtogs oplu)

 

Contractor Pur-

All providod by cliont

 

Proapqggqgo (5)
Sony on 2000

touch scroon
light pen

 

 

 

3 systems. 3 softwuro

4 systems. 2 software pack-

 

 

 

chssod Equipment A packagos agoa. and graphics soft-

[qftwaro wars

[inql goggwqpo PC Pilot CA as ic ttico Lunona r c

Initial Cost Sid 3100.000 - 8130.000 8450.000 - 8550.000 Conbinod total of all 3 con-
o tr tors - 7

Porcontago of Cost 15% - 20! 40% - 451 351 - 401

 

 

 

 

 

Cll - Conputor-Sasod instruction

60

Case 1 did not use all of the disc space, and Case 3 discs each
contained the same parts of the Computer Hanaged Instruction (CHI)
program at the beginning of each disc. thus requiring less individual
production and programing. Further. each disc in Case 3 was a one
sided disc while the other two cases produced double sided discs.

Timelines in each of the cases *were compressed or expanded
through variations in the number and amount of staff assigned to the
effort. Staff members did not spend 100 percent of their time on the
project in any of the cases. The number and type of staff varied by
project phase and criticality of the deadline. Video producers and
directors usually only joined a team during the end of the development
and early portion of production phases. Each case used the services
of only one programer at the onset of the project. This usually
increased to three or more programers near the latter half of the
production phase of each project. Case 3 had a great variance of time
spent between the contractors on the project. Contractor C required a
full nine months to produce the new discs, develop the CHI package,
and produce the design guidelines for the other contractors.
Contractor A essentially developed and produced the four discs in four
months. Finally. the timelines in the table reflect the calendar time
spent from contract initiation of the project to delivery of the
discs. These timelines include large delays due to changes of
equipment. software or instructional content in some cases.

Equipment and software purchases were required in Cases 2 and 3.
These costs ranged from $10,000 to 514.000 per system with software
and required peripherals. The acquisition of equipment had not been

planned for in the original budgets and constituted a capital

61
investment for each of the companies.

The percentage of cost estimates over the initial bid included
the actual costs itemized in the financial records. plus the
tremendous amount of additional labor above the 40 hour week worked by
professionals which was not compensated or accounted for in the record
system. The overtime labor estimates were mostly self estimates by
interviewees except for free lance writers who maintained careful

records.

Results

The results of the data analysis are presented for each of 12
research questions addressed by the study. Data judged relevant to
each research question has been organized and presented in matrices
appropriate to the question asked. A brief discussion follows each
question. This organization should facilitate the drawing of
conclusions regarding the problem variables that affect the time and
cost for producing an IVD.

Research Question 1: What DIE of knowledge is mded and when does

it h_ave to be available to make time or cost estimates for an IVD

project?

To study this question. the individual case data were reviewed to

 

determine what the contractor knew and understood the requirements of
the project to be when the initial bid was developed. This perception
was then contrasted with what the contractor learned after the project
was underway. This information was provided by project managers and
instructional designers and corroborated by other project staff. As
can be seen in Table 2. each contractor either lacked critical

information or assumed certain elements which were not clearly

62

 

 

.oon no. onoo
uncanoonnn oennnooo nonnoao nnnoeo unenuo
.nxu on» sun: euonunonoo on on one
soon nonnnno oonoono on nnononnnooo man
no noonnneon on on on: oonnuonn nonnnnno
.nnonno no nonnouon no: nonunu
unannooo venunmnon unonnnn vnn oononounu
.oononnnnnnnon
«no noooo non vnv oonnn nnnooa
.onnueo noononn one oonnnno
onennmnn sen-no vnn vennnunnn on:
onouonnanoo noun» nun anemone nonnonnvnoou
.oonnu unneooo
nonononnon on on on: noon: .neo
on: unouonaoou nonnouonm none ouoonnonn
.oouoonne on ounnunnoo
no nonsense nonanon unom nonn oonnnonnnn
.onnnno non-Innnonn anon
unnonno .oeunueo unonnnoo onnnanoo noon
.oonnnnoo nuouonnnoo non non
nonam Ionono men n on: nouono to; noon
.nnnnn no
omnnn onsenonn on on on; noon: .ooono
3 none.— aonnm non 3o onnnnoo annnnnno
.oonnnnuoo
nnau nounonn noon nonnnoon no nnoonn
.veonnonon nouoonunoo on on non-anon:
.oonnooon onoon anon. nuns»
.ennnuovnnn
oooonn oononnonno non nn noon;
no neon; non—nanny ono noun-none unou

.nnnonnoannono noonnn on venomonon nonn

=ennnAo noonon ononoo nonnno—ovon non-Anna:

.unom unannouoooo non nnanono
noon oonon noxonn oonnonon on on:
nononnnnoo .oonnouon non unannnnoo ononuu
.ooonnonn
on nnenno ha veaonnnon non nommnnnonn
.unonnno
mononu—m onomnnon onoooo eon no nonunnno
eoonoon eonn onnn-o ononanoo onnAH
.uoenonn
nnnnoo monono non» noon on enigma anon
monn nonnao one-annoe no no noonooo unonau
.oono anon onono n eon on nonnnooo
on on on: mannono nan .onnononnooen
nonnnonn onn mnnnonn no eono on»
non ooono onnnono nonoonoo nonnnn oonnnuo-
.onnnnnon nonnoauno oennnun
monn ennnao non—noon oonaunon nnnoev
canon-noon unonuo onn nunne~nuoo noon
.o. oo..ono
on non nouonnunoo new .nonnu nan no
on nononnnm oononnnnn nnonno anoono no.
.venonnno non»
oennuonnennn nnn noun-on anon one: ouooh
.uoononn ennnononn non nonnononuno none
noun-no non on: gonna n...» on nonnnuooon

.nnnao

noonennonno onnno—unn n nonnnoon noenonn
.nno—oonn unannnooe nonoono ononnno—n

(noono on unnonooon non nonconoonn non:
.onooon on on on; non nonnonnuo—o

one: nonnnnon nonoeo oouooounn nnonuu
.unonnnnuo

onnnononn can: onoaoono nun~nnon~ou
.uonenvenno non nonnono

nnnnnnno eonononno oonnoonn onnnannh
.oAnnon «nu nnan onom no nonnaoo no

oennnao on non o~ooo onnaonnn nonn-
.unos nonnoooonn

one non ounnnnnon noon ononono on» nano
.onoo nonnouon one—do

eon-on: on nuooo nonnnoe ononno nnnon
.uonnu
ononm anon onoon anon on own» ononno
onosn oeuonooann nonnnuen oonou one-n

nununnu noon nonunnonon onnnu onn onooeu

.ounnwnnoonn nonnn onvno nonnnnno
.nnonno no nonnnuon oonnnoen nnnooo

unonnno non canon onnonnnu nonnonnnn

nonuonnunoo nonnn
nonnnon nonunnnonna

 

.onouonnnnoo nonno non

nonon nonooo eonnonn on u nononnnooo
.nnonnoo nonuunnooo

onnnanoo onn onnnnnnn ennoonn on anon
.nnnnneno no unnnnooooo one: mean

noono no unannoo on» neon onouonnunou

.ooaononnn nnonuo on on one: o-Iu
.nnnmnoueooo

non non-annuo nonnnnn on one—«u

.onnnom ennn on on oonnnmnu
.ooono non onneno on non noononoo

unoo an none.— ov on a non n onononnnnoo
.nnouonnunoo nonno one no neon can u

nonconnnoo no nonouoooo on on nonuooo use
.ounn~nnon ononnnn

.aann non: .oonno. nn nos-n nnononno
.oooo on on coon. nn noon; no.

(nonno sane .nanouun one—o nonnom non om

.onnnnnon o oonunnA nouns-noon anonuu

.mnuono nan. noon connnnnooo noonau
.onouonnn

-noo no noon no anon on on no: noononn

.nwnnonn one umononu
.nnnoonn unnonnoonnonn non non—on

on» no one non nnnnnnnu unonno eonoonn
.enonoon noononn oeoon on non

unenuo non noncono nonnnoo on oonoonn
.nonnonno noononn on» non

non-annonn omnnnunnn o nonnnnn on nnonau
.nnonnouoooo ononeonooo

non oneuouo noon nononnn ou noonau
.nonnonoono onn
unannononoo .o .nnnooo .n .ononannn .n

"no-non canon on noon on on on; once
.ooonno no ooonoono no on

on: neonnn unn non nanonnnm on» no anon
.nuonn eonnneunnnn ono anon-onun oouuono

on. unnoooon on on.nnooo noon o noun-on-
.eon on

onnnnennm mononuoonnnnn enannnnon on
.non on: cannons

none: on: nonnoonnonn non.) non nnononne-

.nuonu oonnnennnnn non nonuoonuonn nun

.nnononn ennnononn

.onnu o.nonoonnnnn
no no nxu onn onnennno no nonon noononn
.unnnnnnoe ennnononn
.oo-onnnnn noonno onnmunoo onn unannnoo-
.oonnnoon onnn ononuo ononn
anyone .ounnanoon nonon ononn unnnnnno
.unnnon ooono
n on nonnoonoo on on anon nno oonononn-
.oeonoaaen nonunm non unnnennouon om
.ounou unnunnn
no I: soon 328a 2. So .83...
.mono-o nnnno‘nno nonnnnnooo unonuu
.unom nnnoon nononnonoo on no eunnna
.onnu on
....» onooo on. sons. ..onnooannn no
nannonn onunonunn non cannon n nononnnh

nonconnnnoo ononna

 

 

 

nxumnu

u ammo

 

 

qmumuw

 

nnonu nonnnnnonou
mumuuml.:u .

 

umOU VG“ 05MB OUUUOWH< £UM£3 HUMHUGOU Hmum< VOQHGUA flaw whowwm 3306M fiOMUGEHOHfiH

.Nnumnwmana

63
stipulated in the request for proposals (RPP) or the final contract.

In all three cases. the contractors received one or two page
Request for Proposals (RPP) or’ a single page letter requesting a
proposal for a contract. Bach contractor. or primary contractor as in
Case 3. had met with the client prior to contracting and were provided
some general information and guidance upon which the contractual bid
was made. As shown in Table 2 under “Information Known Before
Contract“. each contractor accepted information which when compared to
“Information Learned After Contracting” does not always match. It
appears some information was inaccuracte (e.g.. design features). some
verbal agreements were not met by the client and new client
requirements surfaced which had an effect on the cost of each project
either in time and/or direct costs.

In Case 1. two pieces of information proved unworkable and
greatly affected costs. The first. was related to the authoring
language. About two months into the project. the contractor
discovered that: (a) the client-designated authoring language could
not support the client-designated design features. and (b) the amount
of code required by the authoring language could not be accommodated
in the nmnmry space available. This led to the selection of another
language which in turn required the reprograming of templates to be
used by the programsers. In addition. the original cost estimates
were based on student labor using a fairly simple authoring language.
The language change required the progranuners to program in a more
complex language which slowed their speed and increased labor costs.

The second and perhaps most costly knowledge factor in Case 1 was

the requirement for unplanned audio recording studio time because the

64

original audio tapes that were to be provided could not be located.
The production tapes made available had background music which is a
problem for the stop and go format of the video disc. About S5000 was
spent in audio studio time to edit the music to fade up when
initialized and fade down after the words. This audio editing in turn
created a requirement for check discs valued at a cost of about S3500
- S4000 to confirm the accuracy of interactive program encoding.

Information unknown at the time of contracting in Case 1
concerned the requirement for a demo piece with some special effects.
iiaving learned of this requirement after contracting and hoping to
produce good relations. the contractor agreed to produce the demo
within the contract before inquiring about cost. The demo created an
additional expense of S4000.

Finally. in Case 1 the compressed timeline for the size of the
project lacked tolerance for any delays. An error in shipping the
disc caused a need for reshipment by Federal Express at a cost of S300
- S500. To an already strained budget. this became a big expense.
Equipment failures and client requested changes for given design
features throughout the project also added time and labor costs to the
project. P

Case 2 was different from the other cases. The project extended
over several years. covered all phases of the process. was designed to
teach complex and detailed maintenance skills. and was coated in three
phases. As shown in Table 2. the three bid phases were to be for in)
analysis. (b) design. and (c) development and production. According
to the literature. this should have allowed for a. more accurate

costing. however. it appears that the size of the project. the client

65
changes in labor and equipment support. the change in equipment and
software. and client designated design features are the items which
most directly affected the time and cost estimate in each phase of the
videodisc development.

The Phase I analysis (Case 2) resulted in a total of 80
maintenance tasks. This was far greater than had been expected by the
contractor. The sheer volume and complexity of the tasks for the
program affected the amount of labor and equipment required to
complete each phase. The expected client manpower was usually
under-supplied from original estimates or withdrawn. leaving the
contractor with the need to supply the additional labor to complete
the project. This became particularly costly to the contractor during
the development and production phases of the contract. Actual costs
could not be provided since much of the labor was completed by the
contractor's professional staff working long hours to complete the
effort. This overtime labor is non-compensated labor.

Since Case 2 was bid in phases. the equipment reconnendations
made in Phase I were accepted by the client. Because of delays in
government procurement. the contractor purchased one of the designated
systems. plus related software. to facilitate some of the automated
design and development work. The government did not procure the
requisite systems as expected. so added labor occurred later when
manual processes were used for design and development work. The
contractor-acquired equipment had to be reconfigured with a cache hard
disc to accomodate the necessary lines of code.

During the Phase III production effort in Case 2. the government

switched from the Sony SRO-70 system to the Sony View 2000 system.

66

Again. the government could not procure the equipment in a timely
fashion and the contractor"was forced to purchase two systems at
S10.000 to S14.000 per system and lease a third system to complete the
project. The equipment change required reprogramming since the Sony
SliC operating system was different from the Sony View operating
system. The net effect of the equipment change was new equipment
costs. considerable time delays and added labor costs.

In addition to the equipment changes. the software used in the
Case 2 project was changed twice. The first software change was made
to support client requested features which were only provided to the
contractor after the programing was well underway. A version of
PASCAL. still under development. was selected to accommodate the
features requested. This language change caused delays. reprogramming
costs and labor hours for debugging of the developmental software to
make it perform as required. A second software change was required
when the software supplier withdrew the support of the developmental
PASCAL software. Again. delays and extensive reprogramming costs were
incurred along with the acquisition costs of another software. These
costs. although mmch larger due to the volume of reprogramming. were
similar to those experienced in the software change in Case 1.

Case 3 includes three separate contractors and is a project which
‘was preceded by a research phase. Contractors A and. C had been
involved in some initial work with the client using prototype
equipment. The research work with the prototype system was intended
to lay the groundwork for the project contained in this study.
iiowever. all of the equipment and programing were lost when Sony

recalled the prototypes. The case as presented here does not account

67
for the early research since several of the contractors and
subcontractors of that phase were either dismissed by the client for
poor performance or the companies failed financially and the data
could not be collected. The case as presented here includes the team
configuration as given in Figure 1 of the Case Descriptions.

The contracts in Case 3 were much less definitive with the client
than in either Cases 1 or 2. As shown in Table 2. each contractor was
aware that they were a member of a team of companies and aware that
the client expected team cooperation and similar discs from each
contractor. However. each one was caught with added costs created by
having to. make client directed changes to their discs to produce
products similar to the products of the other contractors. The design
guide. which was to be used by all contractors. arrived too late to be
of assistance. Further. the document did not anticipate all of the
difficulties which would be encountered in a conversion effort. For
example. the placement of icons could not be as specified since they
often covered critical portions of the visuals of the pro-existing
video which had to be used. Henu treatments. user interactions. color
choices and graphics package choices were some of the problems not
addressed by the design guide. Design problems such as these caused
individual contractor delays and reprogramming costs.

Again. as in Case 2. Case 3 contractors were required to purchase
the equipment and software at a cost of S10.000 to Sl3.000 per
system. The equipment and operating system were newly released by the
manufacturer and caused time delays to resolve problems. The lack of
adequate printed specifications and/or explanations caused many of the

subcontracted programming companies to spend labor hours in

68
experimentation to resolve these equipment and software problemm.
Some contractors delayed most of their work efforts until the software
arrived. Where designs had been created. changes were required to
more readily fit the lack of remaining programming time.

The disc nestering was one of the first performed by Sony and had
to be redone several times for each contractor. Problems such as
frame numbers being off or sections of the tapes being dropped were
common. These probleme incurred added travel to disc production
facilities and caused production time delays.

The expectation of what was available of the original work. such
as the original print scripts and the ease of using existing Level 2
motion. caught the contractors in a bind in Case 3 as it did in Case
1. Additional labor was required to correct original print scripts.
produce still frames and still frame audio. The labor for correcting
the scripts was required before the conversion could be designed. The
still frame and still frame audio was undertaken to allow the
development of a more interactive program.

In summary. in all three cases. key information available at the
time of contract award was both inadequate and in some cases
inaccurate. Basing cost estimate bids on the available information.
without some type of written stipulations in the contract about the
assumptions and client responsibility. seemed to create rather large
unexpected costs to the contractor when the assumptions proved
incorrect or could not be executed. The result was rather significant

cost overruns.

69

Research mestion 2: To what extent does client IVD knowledgeI as
reported by the contractorI have an effect on time and cost of an IVD
project?

As reported by program managers. instructional designers. and
information programmers. the client's knowledge about IVD is judged by
the clients actions and requests of. or directives to. the
contractor. Table 3 itemizes by case the client actions. requests or
directives which either did or could have had an effect on the time
and cost of the project.

Overall. the client knowledge about IVD as reported by the
contractor was generally low. Some serious difficulties were averted
through education of the client at contracting. In Case 1 the client
was helped to understand that the existing artwork was inadequate for
the video medium and would create problems in programming the text for
each screen. Both Cases 1 and 2 successfully negotiated the client to
accept a longer timeline. In addition. Case 2 convinced the client to
proceed contractually by phases. Typically. specifications by the
client on program design. features and requirements were either never
provided as in Case 2. or made well into the project as in Case 3
after the client became aware of the possibilities.

It appears that the greatest cost impact was felt by contractors
in situations where the client had materials or products which they
wanted used in the IVD. In both Cases 1 and 3 the contractors were
restricted to using existing audio tapes. In Case 3 the client did
come to understand that the audio had to be re-recorded. In Case 1.
expensive studio time was taken to adjust the background music. This
expense was equal to or greater than that which may have been required

for new recordings. In both instances. the client seemed to lack a

70

noon-n noooon Senna-n - non

 

.nnouonnn

noon nan noonon noon nounmao n ooennon

on nnonno on» no nonoonno oononnon

nonooo ooonnon on oonnnooo non»
unnnnmnnonoe eononenoe vno nnnnmnnnonnoi

.onnon nan no unn ooonooonuon

naunonnoon .nnno—noononnn eonoonn on
ononn nun oonnnn nunno unooon on nonnoon-

.oonnnoonnoo

no nnnnmnnnennen eonononno non—noon
eonon onnno on» nonn unannoo no onnnnu

.ooonooonnnoooo noonno on»

noon on onnnnennn nononnnonnooo no neon
nnnnmonoo omnn can oononnnno nonnnun-

.oonnnnen nonooo non unoonn
on nonnouon .non u an nnnnmnnnonno-
.nnnmnnnoe on» no onononnonno onlm

no nonnnnon on» no venoooonn nonvnoon

innono can unnnnnnoaoun on cool ononno
noonnoo on oonnnooo so: nnn'nnnonnen

.onnnon

n - n no nonoo .ooooononooo nonononon

nnnu onnnnoo on» nonoonn on nonnon

noon on .onnnnoo on» onnnm on nononnu
noon no venononn non oeononn ones nun»

.onnon «nu n noon

noononn .ooonononoo onno noonooo noooo
n anon one on. non ooonnn no oouonnnnoo

 

.nmnnmnumunn.
nnnnmnnnonnon man u on non—noon noonuon
ooonh .oononnon oonoonoon nnonno
ooonnon on nonnn .u.n on nonnnno on:
onnnnnnn nonnoannn on» no. onn oonnunon
non nuns nonnnnnn annoon no. .oononnon
venoonoen nan no anon no noonno enu
man no. son oono on» canon. noonno can
nonnn noon on on on: oonnnno nonnmonnonn
.nnnnon ooono on anon
nnn no noonnn oomn oonoonn on nonunnnnoo
.nooo nonnn noononn
onoo noon ooo... - ooo.on noon. oooo
.ooono noose no nonnoooonn ono nno
noon onn no noon ononn an.) noonn onoon
non on onooo. onoo. on on non. nononoo
onnnon ennn on onnoonnoo
one unnnn ooh .onnnon nno on noon
enonn non nonoonn noonnoo on nononnnnoo

 

 

.oooo on.
-nooonoooo non nonoonn oo- ono noon on.
nononoon on» onenonn on o.nononnnnoo
nonno on» canon nan nonnnooo nnnnnnn
none monn nennunon nonoeo convene.
.ononn
no unnonooen-on no nonnann nonuon
non nononnnoon no anon nouun non nuns:
.noenonn on» onnn nannon «one
coon onononnnnoo on nonoonn one: nonnm
ounnnennn nonunnnonnooo no unannonnooe
no nonon on eonononon onnnn cannons-

.ennnn unnmmnnn
-ooo on» ooon anon ooonoonoooonoo non
nonuo on neon oenonnen nonooo oonnoonn
.nnennoo nnnnnnno unseen non one nun»
.nnonnoo
eonnnnnnnnm on» nn veonennenne nun
non onom noenonn on» on vennnnon om-m
.nonnnonnnn
unnono non wean onn ennuoon no one.
on non canon no onneo non) noon non
on. on nnooono nonnonono no none on.
(nonen noononnonnn non anon nun» nonnnonn
.onunon
on.- no onno noonooo noooo noon-noon

.mummmmun11‘
nonooo nonoonoen nan unonnoo non
So not... loon- oonoooooo ... noonooon
.nnnonn
on onoonno nannmnnnonn onnnoono nanoseco-
.oonnnno
nonooo noonon: non on annoon nonoonoo
nan monn nononoonoo nannnnno n enn-
.nnnnn enn no venouneoo nonnnnnn oennne
(non nononnoo one on nononnnnoo nonoononn
.nnoo on» no nonnn
one oonn nan nnnnon noon. on non onom
nonnm ounnnonnm nnnnn nonnonnn nonn
ounnonn noono on nonoonunoo unannoun-
..nnonm onoonn
noon. non Jenna. nannn onnnn nannonno eon
.ouoom o u o no annuomnn n oesoennsn

nnonnnn .oeonnonnnn
.oooooon no .ooonno

ononunn .ooonnoonnn
.ouoonon- one no

 

ammo

 

u anew

 

anus-u

 

  

 

Q>H no unmoseonn on once one

mane no ewoonzomx ucomno no noonnm

.MH nun—mywnr

71
clear understanding of the effect of using existing linear audio tapes
for a non-linear video disc medium. The contractor. on the other
hand. seems to have assumed the client understood the technical
aspects of the request. and/or did not understand it themselves. at
least initially

In all three cases. the client began making design changes during
the programing stage. which is the final stage of the project. These
suggestions or changes usually followed demonstrations of the product
or other IVD project products which helped the client begin to realize
the potential of the medium. However. what the client failed to
realize in most cases was the amount of reprograming required to
achieve the new effects or features desired. In addition. design
changes at this stage have a ripple effect. A change in one part of
software or courseware may cause a required change in numerous other
parts of the software or courseware. Since no formal agreements were
established when the instructional design and programing designs were
“locked in.” clients were able to make design changes up to the point
of delivery for the field tests with resultant eleventh hour time and
cost impacts on the projects.

Case I experienced added difficulties when the client specified
an authoring system and some design features. The client was unaware
at the time of contracting that the designated software system could
not support the design features. The contractor learned the desired
design features. critical to achieving the product. took precedence
over using the client specified authoring system. This had not been
the impression given at the time of contract award. The late switch

to another language lost time and added cost in reprogramming of

72
templates and program code.

Another aspect of client knowledge that affected Case 2 was the
qualifications and knowledge of the subject matter experts (SHEs).
while the 838s were not expected by the contractor to be knowledgeable
or experienced in IVD. the contractor did expect them to have some
experience in designing visual media for instruction and to be experts
on the equipment maintenance tasks. Since the SHEs were neither
visually oriented nor experts on the equipment. it took longer than
planned to create the storyboards. Secondly. reprograming had to
occur to correct content and create more accurate visual transitions
between frames of content during the production phase.

In summary. IVD project time and costs appeared to be more
directly affected by the client IVD knowledge level in projects where
conversions were being, made of existing, materials. as occurred in
Cases 1 and 3. Further. all of the cases experienced reprogramming
costs as the client grew more knowledgeable about the potential of the
medium and requested instructional and programming design changes. It

seems these changes are an inevitable part of the process;

Research Questiop 3: what effects. if 39!; do contractor agency size
and capability have on time and cost of an IVD project?

The type of contractor agency varies considerably between cases.

 

 

 

as shown in Table 4. Case 1 is an academic institution geared toward
the education of professionals who will be employed by agencies such
as those described in Cases 2 and 3. In an academic institution. a
contract is held by a relatively small nucleus of people and heavily
staffed by graduate student labor which costs far less than the

professionals hired by private industry (Cases 2 and 3). Typically.

73

 

nonnenonn non:

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

non-Innnonn
onomnnom
nonnenoon oonn nonnnoo onon
non-Innnonn envooo cannon-n: nonn
noonnneo ,nnnvnouon onvnn annnnon .nnomnnouo
none—on anon nnonnoo nonno enon vno convene—o nononnnoon no nonnnoa
nonononon ononn can nonngnnn ,nonnnovonn ooonn .nnnnenonm eon:
nnnmnnnuo .nnomnnnuo onommnnnonn .nnonooono enu non
noon on can no onnnu onnmnnom noon-non nonnnoo on nae: on neon
noon. oeono nn noonnnoo ononnnm n>n -nnsuo nonunonn onn oeono oounnon nonnononm non:
non nnonnno nonnono unomnonnnon on» no nan vnn noonnnne nonuannn menu—on onnn vnn nonnnnoonn ononn
Hononnnvon -vonn ooonn ono cannon .nouonnno .nnunoonn n noonnonn nonnnn unnnnooon vnn ononno ononn cannons-
ononnn: non—n: n>n unconnoonuonn song: noon-non nonnoooonn ooonn ouonnno anon-no non unnnnnnnev unn noun-now noonennn
ononn
nenuo ono
unannoo nonom
on nomzmn
onnonnn
noon-n noose-m
unonnno nnno
-oo ennunono nonnoooonnon
won and onoan
unonnno nnnUnna unnnneoonnh
nonnoaoonn ,nnonnnoeh ono nonnn—nou unnuuoo onnn oennmn;
onnnmnnnonn none—noonunnn "on nonnon nnonnnm non-oeuonn anon can oongnono nnnmnnnoe oovnn
nnonnno nononnnn nnen unonnnm nonommnnnonnn onoonnnnu nnnnnnom ononmnnnonn
nan onn coon» \noonoonn ononnnnoannn onoonnnnu nounnnoon onenn nan on nonnn uneven. nnnnvnnu
onnnnnooa man. vno oennn noonnnn ouonnonn none—nun nun—nononnonh once:
nnnnnnnh onnnnnoon ononnn: ononnnoon Innsunnnnu ones-noon monounnnnu nnonnnm
onnunnm nononnnnnnonn non—nanny onenonooon onn onenonoom Hone—nunnnnnn onononooon oo- onenn non unannounnnonn vno nun—um eonosnnn
onnom .mnon n non-one on
nunnneo anon oennnoon onnnuo one nun: ounnonoooo
-nno ono nun ennnu one» on cannon. onn onoononn .nononono
.oumwo no on wo no noon onnmnwoo u n nonn one-nonnonn noon
-nn no anon .nn nun noon nnnn onn .o.n noon-none neomnen
-noo unanno- -nn. neon: anonnno noonu nun) unnnoonnnno
.onoonn noon non-nu onenn nounnnoen non .oneennnno ennmnnoo nonnnonnonno
unoneoev eonnh nnnnunnnnna oeunnnnnonm .nnnnnonn non-no nonnm nonnh ounomnnnnov unmeonon nennoonen nnnnmou
. -.- . can «no
-nnnonsn nonunonne neauo annm ononono
noononn on ooooon annm non-unnnoo
8n - on“ 2 - on So - o2 2n - m2 2: ... nnoo. Soot-Soon o - o ooononoan no onion
lummmmmmmlmummo
nnuononn .onnemnno onnnnnnn vnn
one—non noun: noon-non noononom unnnoonnnnn noonnnoow nunnonn Anon non nnonnnn no onnnm on-
nonnnunnnnmoo nonnnnnonnnn unannoHeoeo ennnunom ooononnnnnn eonnonn on non-nononoo
nnnnounn: nonnnonnnan non-none on non nonooo onn annoooon non nonnnnoo nnnnonnnh
"non nnnoon noon-m non—nannu manna—nnno ono o nnnnn unnonnnh .onounoonn
unnmnoneooo annonuoanunnn ,nonnoononn vnn "an—m nnonnoo ono onnnonoonv monono none—nunnnonn
unnnnnnh vno nonnunnonn nnnooo unanono annnoennnno vnn nonnnnnn nnnnnnnu non nnnnnnnn nonnnnnuonn unmevnun nonnnnunou no onnh
u n n
n nonu u soon u soon oenoon
D>H cm wdwuavoum aw umoo vcw OEMH ou vmuuHmm mwfiumamnmmwo van unwm heuumuudou mo mHOuomh .¢ MAQ<H

74
contracts are undertaken for generation of research and experience for
graduate students rather than for profits as in Cases 2 and 3.

Case 2 is distinctive from Cases 1 and 3 in that the client is
the government, specifically the Department of Defense (DoD).
Govermment contracting requires more accounting and administrative
support than appears to be evident in either Cases 1 and 3. Further.
this is the only case which has a highly qualified staff of software
programmers and technical engineers for support within the company.

Case 3 is nude up of three primary contracting agencies
specializing in the development of very different types of training
for private industry. Contractor A is noted for technical
publications, Contractor B is predominantly production of print and
film. while Contractor C has a broader range of training development
experience. All three contractors develop training for manufacturing
in the auto industry. Companies B and C have provided supervisory,
management and other auto related training development services.

As is evident from Table 4. all of the cases in this study
required capabilities external to the organization. Usually, video
and audio production and editing services were hired or
subcontracted. The Case 3 primary contractors also required
programming services. Contractor A. unlike any other contractor. had
to subcontract all of the services to complete the project. Also in
Case 3. all of the primary contractors relied on writers external to
the organizations.

Disc mastering and all the facility requirements for editing of
the tapes were external requirements across all cases. This is normal

since the state of the technology is so new that only a few companies

75
presently have the capabilities for mastering discs. Because of the
technology, disc mastering costs will continue to be external
requirements by all contractors for IVD.

The number of internal staff identified in Table 4 in each case
did not appear to be significant since both large and small staffs
required external services. Case 1 required artists and used student
programmers. Case 2 did not have a producer/director or video expert
available to the team during a major portion of the project. In Case
3, Contractor A required all of the services to be subcontracted to
produce the product while Contractors 3 and C required programers
experienced in Lattice C.

From the available data, it appeared that company size and
capability did not have an effect on time and cost since all of the
required professional services could be secured externally. All of
the contractor agencies in this study performed general training
services, hiring specialized professional services where and when
required, for a set fee. Hiring specialized professionals may be more
cost efficient to the total operation, particularly where a large
capital investment would be required for equipment and facilities such
as required in video production and audio recording.

Research Question 4: what function does each team nmmber perform in
ghg_process of carr in out an IVD ro ect?

To answer this research question. personnel assigned to the
projects of each case studied were interviewed to determine what roles
they had in the IVD process. The results of this analysis are shown
in Table 5.

The project manager was either the financial manager (Case 1) or

.oeooo an. on voouaooo an coco-.Ieun can cannon yea-so om
.ouon ma» venumndan non-ml 0:» zone) a« Coco one on munch mnOAlsl "nub-u

 

 

 

 

 

«an oemnuaou n— “N. ooonuaou _

76

coo-unloh noaouoa

-——‘

.N. own-on aooonuoouu-on

conglouaq menace: _

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

“ _
_ augmoouoou Inuuoon .H. noonuoou _ .u. conga-somed- non-H
. ocean yoga .u. one-oonuoum ouclouod vo- oouonomoh oesopoa _ aw. uoouoou yellouuoum
_ sonueaooun coon» nonopon .w. onooonooum .~. apogemonn no anonymoun own-oeuoun none-luonon
_ .n. uquum
_ .n. unannou-u genome:
_ .n. .oeqoonaosm uoa.vu
an. ocean uooaxoopnuoo no
.n. sun-co .n. .n..~.o¢ essence
_ oncogene-I madame: .n. nuance» Ann analogue» ouomoouaoo
_ 8:33.: 83» .2 888:3“ __ .2 385.3: 88.3....
mam. cannon
_ ooonuaou «one: coll-unoum _ announce” _ _
_ .n. nan-eunouoxm connooou _ unannouo: oouunu: _ .n. own-on cacao-nooo
_ an. non-Iouuoun nounoo: _ ova-oosuoum _ .n. unannood acouoou gonna:
_ mmwm my _ sun-on _ cocoouom Honuoou
_ nonmeuuounxm ounoeoo- . unannouox oouunua _ ocean uoohxoounuoonoo _
_ AN. non-lounoun . nonuonnom _ gonna-om ou-xouuoou _
. .N. conuoaoonm coon» _ ova-onhuoum . uuonomoun .~n onouuood once-ounmuou ocean-on
_ souuoou hang-so _ nonuoou mama-mo _ Houaoou nuns-so Houuoou nunnosa wooonuoauuoon
_ A." a «6:38: 3 m. s=QH¢.SJ 2 M 2 5:25 an: ... H2:
_ an. coalescent mango-on. _ .A. use-ouooo: dunno-onn _ an. noomouooo: uonuooonn _ a.n. uooleuooo: menus-on. .uu: uoenoun
. _ _
_ 43338.. sum—$2.8 o «348 .3594 finial“...
mmououm Q>H ”sauna Pawnee: Ewan. no «define—Sm .m mqmfla

77

was a multi-skilled person (Cases 2 and 3). participating in all
phases of the work. Instructional designers were the most active
people in all three cases. With few' exceptions, they‘ were most
heavily involved in the project courseware design. which included
sequencing of content, flowcharting. writing and developing objective
and test items, scripting, storyboarding. and outlining student flow
and screen conventions. They also performed the specific role of
quality control for all phases in all three Cases. In Case 2 the
instructional designer *was also responsible for the conduct of a
requirements and task analysis to determine content and design. as
well as participating in the video production and programming
activities. In both Cases 2 and 3, instructional designers assisted
the progranmers in redesign of the courseware to make it fit within
software or hardware parameters.

The role of information programmer was not always confined to the
production phase although this is the phase in which all software
programming was done. In Case 2, the third and fourth project
managers were experienced programers. Further, they were involved
with all phases of the project. The third project. manager also
assisted in training subject matter experts in flowcharting. of the
task to be trained during the analysis phase. In addition this
manager was responsible for system selection. development of automated
data processes and record keeping for storyboarding. During the
analysis and design phase. he began a study of the programming
language and began developing routines and the computer program
design. In Case 1, the lead programmer was involved early enough in

the project to develop software templates and routines in the design

78
phase to be used by less experienced programmers as production tools.

The role of video producer was usually defined much as it is for
linear video. Often the producer was not involved in the process
until the preproduction activities began. However. in Case 3. one of
the primary contractors and manager of the conversion of one disc was
also an experienced producer. Because the staff was small. this same
mmnager also acted as the instructional designer, wrote scripts, and
developed storyboards, objectives and test items.

Writers in Case 3 not only did scripting, but many were involved
in content analysis and design. Further, some were required to work
closely with programmers during the production phase to clarify
scripts, change the graphic complexity and in general facilitate the
programming process. One writer coded touch points and. performed
other editorial type activities for’ the programmer using routines
written for the purpose.

Only Case 3. Contractor A used an editor in a traditional manner
to fulfill all the documentation requirements as specified by the
client. No other contractors used a designated editor.

In summary, team members for . IVD projects fulfilled
multi-dimensional roles in all three cases. The most versatile were
the instructional designers and mmnager/programmers. It appeared that
it was a requirement by all except the producer to participate in all
phases throughout the project to ensure a successful product within

the constraints of the timeline.

79

Research estion 5: Hhat effects if an do ro ect team

organizationI and personnel IVD training and exgrience have on time
and cost of an IVD project?

To answer this question, the contractor team in each case was
examined by review of the size. content and organization of each.
Then the training and experience of the individuals in key roles were
examined on each team. Both intrateam and interteam connunications
were examined to determine patterns and unique characteristics.
Finally. problems from the work effort were examined to determine
those which emanated from one of the above factors. Table 6
sumarizes the results of this effort.

Table 6 shows each contractor found it necessary to use
instructional designers and programmers. Case 2 used an instructional
designer as an editor to sequence training or establish interactivity
requirements of the program. Case 2 also made extensive use of
Subject hatter Experts. Case 1 had a need for several artists to redo
original art work and Case 3 used artists and writers for the major
portion of the production effort. A large portion of the writing for
Case 2 was done by Sth.

Training and experience varied greatly across projects. Case 1
had the most degreed and experienced project management group. Case 2
experienced the widest range of degrees among project management
personnel. Case 2 also changed project managers most often. Case 3
had the largest group of team members and most overall experience in
media production related to. but different from. Level 3 IVD. Case 2
had the least experienced staff for any role on the team.

Comunication patterns varied widely in flow and methodology.

Case 1 used one—on-one. group meetings. a file folder system and a

8C)

Amman uxoz mo coocwuaoov

 

 

 

oonuuocoo oocno on nooooo doom _ _
ooo o coo emu coo .oooooo ooooonoooc _
mooonuuonuoon moom «n coo ouzm noonuo
.oooonnoooo non oo no: non-lonuono
oooo ouoc omoo nun: oooo—oo nonaomoo .
on oouooc ooo~ooooo o coo commonoooo ooo
ocoo

“
.oooonoomoo non omen .coooonuooo _
nooonuoooco o» ooounoo oo nun: _
.ooounoc ooonuoo no couonoooo nnoum _
oonuooocu on oooooc I .oo: aeonoun .
u nonconuooo _
oonuuocooo oocno coo oooonoo nouoomoo _ onnnuoonoo on oooonooooo coo nonmmooo
on ooooooc coo nooooouoo Nun coo nnm _ -ouo ouoo—oomoo coo oooonoo nouommoo
mbn on oooonuoooo coo .umo: oooonoon _ on oooonoc ooouool coo ouommoououn oak
_
_
.
_
_
_
.
_
_

 

 

 

 

 

 

 

on on u uooocounsooounoon\.nuu noonoun .oonu
n oouooouoou -oonuoon moon-oono on oooonnoouo coo
n noooo oo uoo a>n omoo ooouoc ooouoom oonuoooco oo coo nouncu .uoonooo
coo nnoum coo ooaoounnxoooocoum «no .nonmmoooooo coo oonooc _ one oonooc cocnoooo nononoou mom—on:
oonoon .nuoon monono mono—on» on oooonooooo nun: _ .ouo-oo on ooooooo manoe-
on oooooc coo nounomxoooonooo uvc nonuooo monouo o oom oooooo: goonoom uooa _ ooa non-moooooo on oeoonooooo noocoum
oooo—oo oouoomoo on noon! coo .oooonooooo onouaoo on omououo nonuoouoo
_ ooooonn on oouooc o coo oommooooun «no nonmmoooooo oonooouoo coo abs omoo coo on» no ooo oo nononoou coo ouoocouu
_ oonooa .oooonoa nouonmou on an n nouooo: uoonoum .nbn on one:-
.uuoou on oouuoc o co. .uo: uoonoun ~-o .oooon goo-mouoooan oocno on ouoo -uoom omoo coo oeoonuoooo oonooouoo coo
.oonuooocu nooonuooop on oooooc o coo -.ooooo coo can on non . nooooo: uoonoon nooocoon .nnu on oooonuoouo oonooou
uoonno unooooo oou no uooooom noonoon noooom uoo coo ouon .omn on .ooon- nouooona non on oooamom
o nouoonuoou o—ounooo. can on o: . oooooo: uoonoon uoo .omu on can u oooooo: uoonoon look no cononouh
ouommmmu
noon one. no ooo com nouuoouoooooo soon uoouooo couocnuoo - an»
oonoo _ oououomou coo ooonuooo coooaoooc coo
uonooooo oxo ou— coo oooo coo .uooooom _ onomooosoe oou commooooom . oommooooun oououomou coo ooonuoou
o co: oouuoouoooooo coo nouuoouooo noon = nunonuoooouon non ooonuoou cooouoooc . ooommonoooo uoocouo
noonuo on» no commoooooo cooo coo oo—uooouoon noono no noomooooox . oommonoooo coed
coonooo u aeonooo on oooooooo oouunomoo cooonoooc coo oooo ouoc cocooom ooancn oommooooon -
noon. no oooomom duo on aoounooooo mononueo oooocoon - nouooonn-ou
o oo couuo uououoonooom ooomunouxoooacuox uoonooo on» conooo: - oooooo: noonoun noncom ooooonn - noooool uoonouo oonuounoonuo nooooouom
.oounmmxu
ooommonuoun m
ooounumxouooonooc nooonuooouoou mnn _
nnouo n oono oouoooncsoooocoun
oooooom uoonoon
u nouooouoou
ooommonoono n
an. oouoouuoooooa o
_ canon-ooxoosona
= nun oouooouoooooa o
m oouooouooo
moon oooocouoxuoounoocsuooooom uoonoon
.uu-
.ann. ooouoouaoooooo a ooun - youooouooo _
nnoao conuoooooo coo nouoooncxoooocoun
~-o oouuooaooooou o
sounoascoaon.oc o ..o«. .oooaeu
ouommouoono n nouuo: noonoomxooonunooooh oooon and
uooonooc\.oo: noonoun ooommooooom ooooh onommoooonnxouoocoun
—-o nouooouoooooa o nouncusuonnooo nononooh ouonuno onooonoxouoocou»
o oouoonuoou moon oooooom uoonoon an. .oo: uoonoonsoommooooon .oo: uoonoonsoooonooo nooonuooonoon
.«o .a. nomlooooumxnooonoon nooonuoooaoou
.no. ooouoouuoooooo om» ooao o uooouoou .oo: monsoonxooooonooo uooonuooouoon oooocounxoooonooo dooonaooouoou nnoum
”nomuw u oooo u ooou oooo

 

 

 

 

moowoumzli.
Q>H on no uooo can many an couonom.mwwowoua n>H noncommom can mowuouwommwo aooH uoowoum no uuomwmu .o mqm<y

 

81.

 

.eueoeuon

eon noel on eoooeoo mongol no .cnu

ee oooe oenuoneeon Ionoenn ennneooe

coo nonmmenoenoon non neeoxom—u onooo
coeeee oeoonnoooo mom a done; no need

.econ on unenno omnunoee no couee:

no eneeo noem,o. connoeon oooeouoeoeo
nun: eoenueonoommee noeueooe no one;

.eoeneoe

oneoe ueo cnee: oneu—uonmeo oeooooo

ooneeon conneoc on» you on eoonuoon
o:o nneou nonuooeon no cocoo eneueonuoeu

.uonmlencenoon ooem

coneeee conoooe nnuooooonn one: eoene

-nooc coo neuoo: e:o nnoeon.eoeneeoe

nee-coon noon no oeeeooo nnoeoe nnoe
oo—oom oeneneoc ocem oeeenone oouunmmeu

e—cmd mlonn wanna s «nu

 

 

.ooe e nemom non
ee... eon e. on. on u. soon. on eeou
menoeno eon oonUnn:on coo nonnnnceo
enee on-mn noon. eo- eeeu eon cocoon
enemmenoenn cocoonneoeo nnoe .ononnoooc
eno nouno .ooeoe no age oen coo oceu
couoooon goo eonnn no eonnenonn oeo ueo
cnc nemoenoeno ceeoonneoooon .oonceu

uoooonnno nnoo connouon nnemom coon-n4
.onezeenoeo
no ooneocon coo non-menoeno non enoeo
ennoo noel none-en .eeooenn ooneoc
encol nononeo e on onooce naneoeno ueo
cnc noon .enoolem moon eon no oooonnoo

-oo o>n n none; no need eon no oeeeooo
.cooneooee nae
on oneoono coo.- oeeo eoenueenoeone
coo eonecooeno nonuneoon noononnnc

coo moon neoconaoeu-ooon~o no on:
noooeenoo
noonno no coooconn nonunn: coo ecneeo
-nneue on eonooeuenooeeo. no oeoeooo

one: no econ coeeeu oooonnoonoo. noonuu
commenuenoon oo

ea coo oeno: ocoo noonennnoon cooocenn
.ooeeon onecoooe coo ueeo no ooccnn
-noee oonne moon no eoenuecoemmooen
.leoo a: Ionn canon eloe o2: noel

-ooooem noon: no ooec one: eoenuonuenom

 

ooeeoueeoee nnooo nun: ooenoeo
-nommmee neoneneeeo monsoon noneonnnnc
coo nooo ueo .nnome oe: neooenuoeo one-
euoocenn
oooueneoee no. on nneeeoeoo ueo
.eneonoeomeo one: no.» ooeeeoo coonenue
ee: eeenoeonoammee neoeenuoeoneoon
oneooonooeo on oenuom
onenon oooonnoenc on ooeenooe oouunoleo

.emo—oeno
ne oenooneeon nomnen eo ee: onooh
.uno: nnoeeoooooo oooeono
on em: on Ieeo e... oe coeeon enemne
ueo one: uoemooeoem noon: noooeno
noee coo noonno no eooemoonoe neonon
enommenneno ooocouo
ne oeo eon ooeonnooen oo eoonoeen
coo eooenomou ono:onoe nonoeoc en ce-
oooo—neooo couoen one-menueno
nooceue oooeeeo menoonn ea noooen meek
.enec none-eon coo eon no ooecon
oo «no on coo - eoooooooe encoo coo coo
oneoe en coeo one: onoomoo emonn coon:
.nnnec. nem
-menoenn coon ooee no ooooono no you e
connecon enemmenoeno no one: ooooenooeoou
eoooceno nnoou
econ ea coo coo oenooononon no eaec
neooeu noeoon: coone: one-menoeno nenoeoa

eneooen no neonnn

 

.unenno «eoooenooeoxmuuMImexmm one:
ueo cnc noemoneoom noooo noonno coo
neueenuoee oooeou ooeo mongol oenenuoc
coo ooenuonueooo one cnc goo-oceoem noon:
nu—nnoen nooooenuoeu oo coeem
one: oeeeoeo nooooonnon .moeo mum noonn
oune oe ooec one: enno:
coon one: eoenueonoamoeo nun nononon
.nnnnnoen emee
on coone: one ooone econ coe nooooeneo
neuoenooeo ooo:ooo noeuoee nnneo

ooonuoem oeenn

noom noneeo cocenoono eon-coooe neoconn
eon-egoceouom on

eoooeoo noon on coee oe: cneeo onnonnoo o
eooeeeem oeeon

on enocnen ounn no monene e ee: onooh
enemmenoenn nooceue ooe:noo non-nee

monoeno coe oenuoeneuon emee ee: onooH

Din-Quadh

ooennoonoemmeu

 

 

mama

 

Noma

 

demo

 

Ac. uoeov

 

oooeen

.0 mqm<H

82

bulletin board. Case 2 was mostly word of mouth but the project was
heavily influenced by verbal agreements made at the upper management
level which were not always passed along or were not evaluated by
technical people. These agreements appear to have had a serious cost
effect by influencing time and changing requirements. The majority of
Case 2's communication between client SHEs and contractor team members
was over a long distance. usually by phone. and was relatively
infrequent.

Case 3 communications were complex because they involved required
discourse between contractors, contractors and consultants, and
contractors and clients. All communicated directly with the hardware
and software manufacturers. All of the communications ‘were
accomplished through scheduled meetings, telephone and one—on-one
discussions. There were different levels of conxnunication between
groups such as upper' management. analyst/designer and programmers.
Frequently the communications between groups did not include
information which was pertinent to the tasks or efforts of another
group. Because of the organizational nature of this project.
intercontractor communications and discussions were critical to
exchange required information for product development. However.
because they also had to continue to be competitors. the information
was often cryptic and limited. This often caused lost time and labor
to either correct or add what had not been shared during earlier
communications.

The lack of experienced programers was a major time and cost
factor in Cases 1 and 2. which caused an undetermined amount of effort

in Case 1 and at least 5 months of reprograming in Case 2. Time was

83
lost in Case 1 because of misunderstandings, changes of methodology
not cousunicated to the cadre of programers. and programer
inexperience with the language used. Templates and routines developed
by experienced programrs were used by less experienced programers
in all three cases to assist editors and lessen experienced
progranxners time requirements.

Client supplied team members for Case 2 were not experienced in
IVD. and could not be managed by the contractor. Several man days
were lost because they did not spend the agreed upon hours on the
project. Further, differences in the client's internal reporting
procedures and organizational structure caused problems for the SilBs
unrelated to the IVD project. These problems did delay the design and
development efforts. The lack of IVD experience by the initial client
SHEs for the analysis and design caused repeated redesign and
reprograming throughout the project.

The Case 2 effort was further complicated by the absence of the
project manager/experienced programer for long periods of time. He
worked in another office of the company which was located in another
city and typically visited the work site either at the client's
location or at the contractor's project location in two week
intervals. During the remainder of the time, comunications were
conducted and directions were given to all team members by phone.
This appears to have allowed the efforts of the client SHEs to slow
and appears to have been part of the cause for the inefficient coding
by the programer to go unobserved until well into the programing
effort.

Verbal agreements between client and contractor management in

84
Case 2 caused additional unexpected work by changing the design
requirements after major portions of the work had been completed.

In Case 3 the committee approach to problem solving led to
frequent changes of direction and reversals of previous decisions.
The fact that team members were also competitors in Case 3 led to
minimal comunications about technical accomplishments and the need
for each contractor to develop its own routines for the same problem.

In sumary. team member experience was usually minimal at all
levels of nmnagement and throughout the team. Inexperienced managers
appears to have led to poor planning and lack of insight about
potential long range problems. This in turn led to long hours of work
to redo or modify the products. The lack of training or experience on
the part of the designers or programers led to poorly designed IVD
sequences or inefficient use of code, both of which led to rework and
extra cost and time. Comunication breakdowns caused problems and

extra cost in time and labor in all three cases.

Research Questiop 6: Hhat activities in a phase have a time and cost
effect on subsequent phases of an IVD project?

To examine the issue of what activities in a phase affect costs
in subsequent phases. the case data were reviewed to identify cost and
time overruns. These overruns were then traced back to the activity
and phase which appeared to be the probable cause for the problems
identified. The results of this analysis are shown in Table 7.

The data seemed to suggest that the lack of clearly defined
client expectations and specifications at the time of contracting was
a significant time and cost factor. All three cases experienced

changes or added requirements directly traceable to some contracting

85

Amman uxwz :o coacmumouv

 

 

.eueoonn eon ooooooo on connocon one: oonooono
no you c .uenc cocne onoooc ooo oo uoouooo

eon—n oooc on conoue .- oooc

 

 

 

 

 

 

 

 

 

 

 

oe ce'onoonoon oo on coo
own-en eon :o nonooouon no Ionuonn no nnnenuoenonon 4 332—3 onenono non—Bloc
non-nae: oonmnuem 3 nm :o co... ouneoc ono:oenoou oenuoocenm ,cocoouoo noemonnovon oonooenu
nonuoeoeo eon no onemooo ee: no ooneec eon coooe
unocoo n.ocnc noouno noonnu oueounom .ooeeon
none” on onneoc woo—onne on on eeoooou onnooc oonuoocono ooooooo onnooc
ooneeo noes—eon ou euoonnu coumlono eoenc oo—nonneoolon Boeloonoeoo oo—Ienneno o Hooonuoonueon
enonnooo nonnoc couoooono
no coon—oooc. neeoo one: enoomoooc onnnuoom
coo-no eocoooo
coo con-ea no: nueneno ooo: conenoonue uoo oonneuoomoooc noonooueu
uo—uoonuoeu one: onemoenooo eon non coee oo on eoo—ooonnnoeou oennuoceno noenno noel on nonIonoonoeu
ecnoeen own no eeon e on coon-Ben eeeoooonn
noooo: £32553. 2.» ouelouoe on no:
noeuoo eon coe ecnooon elo- en eeouuo no one." o
noel—oueoon on counooen Joel—nave a oneBneo on euoeoo eon.
couomouoo
eon-en oo on no: oenuenoelouoc couenon .- ecneeonnoum
oonnooconn oooc no: one» non-om sou nonne couocoo oeo
\noeloouoneo one: ouoooo non-oono nunonuoonooon o ecnooonnoum
eoonuoonnoo noooooo nae—Ho
no'nononoc sin e no couocnnoo ueo one: ecnoeonnouu oennuocono onoce'ooeo on no—lonnonoen
ooonnu eon 5n: onucouoon oooo noo coo ooneoc coooono
on—ooa lenoenn nouo'oo .- ooneoc neoenuoonueon oonunn: c ooze-Goon ooou neooou ooou oonIonnonn u ooeu
eoooo ouoonno cooeo
eon-en eonooocono noozo oo noonne coooe nonnooo neon oonuuoconn econ on em: encoue o—coo
eooon
ooonn coonm e coo oonueocono noon-nee no one
con oono: eooen oonnoocono coon-nee one on noenno $73.33 on em: nonunco
uuoonuoaxonneea no connouoo .onoennoeeoo oooeno coooe neononno oonooocono oncoe a neon 32...: 32.1
uoSn on on Iooene
oonoon nonnoouoe onino o Ionn enemnnee no on eoo o coooonn ooou
noonuoonnoou connouon oona: eonouoon onneec cennnuooe uoonau oonnoocgn nonoon Sm neon senionoenn
oo—neoconoxnoeloonoeoc nonnoc oonuooconn ooeu olec e
nonnoonuooo oonc aoc o non noolennocon e cocce noennu \noelnonoeon non nonIonoono a oonnoocenm n oeeu
nuaou H.338.— nmacu 533nm anon—Goo Gonna noun: noun. o .58 nmou
no nus:

umoo can mafia noomm< suns: woman m an owmuw>muo<

3:5 In mac—E

.n mAm<H

86

oennoenonon cooeoe: noueoooo I won
emenn nuns» n no

 

 

—
_

 

 

 

 

 

noemoeueeoo eoennoonnc nonmmonneno _
nooneoa connenoc ocneenn neo cnc enonnee o ecnoeonneum
oenuuoceno nonnoc
ooneeo onoennone ocem one: ooennoennnoeoe non o nnu
eoooooaoe
ooneoo nooonn ne coo no no son: nonenneenouon ooom eon
_
oooc nnoonnoc
no noeeono eon eoeeono on anon. so» eenneenn
eon» ne oooeme
nee-noneooa onoeeeeeon e on cooennomoooe eo nee cneeo ne
nooneoo one:unee on» non noun-en eon eonneoc neoennoonuon
nee-onnooon eon coeuonecoo ueo cnc
ooune a ooonno eon eon: noel nee cnc enoonn:
ooneeo geneon connec eaoelennecon eon coooeoo
noonueennoeo neonuo ea» c onoenneoeoe one: eocneo noonnu
oenoooceno nonnoc oenoono
oonuoecenn nee one: eoononooeon .cocooo ee: encoe :om
ooneeo none—noenouon ocneeno en n» onn:
nenenuooo counonnonon ee: oennes .nn none; no oenenoooeu
_
eoonuoom
oonooenuoeo _ oenuoencneee nnooo: oeeo en geomonnecon neonnu
. .e.» _
. non-Ionueno no noeeme onoooeeeon e on coeonooe _
ooneeo _ oo on no ono:nnee eon non eons-on eeo onneoo .
. .
.nnmmqunnomamamn .11 .nmmqmummmqmmmn _
no sad-n

connoeeonn

eonnueeonn

oonnuaeono

oonoeocenn

noomoenoeeo

neon..o

eonuoaeono

emooon and

 

non-monoenn oo—nec
one-monoeno nun: one: on
connecon enoooneocsenoonn:

eoonueonnnoooe n-u noel on
con-Ionnenoon o eon-econ neon

eooooocoe
nooeoo onen'n en 'noenooo

omnn no nooeme onooeeeeon
e ononn: cooennooeoeo oo
n.ocnooo eonneoc nonnenoo
oeoeooo mono-enveno nonnoc
noouoeo oenoeo no oonoocon

oenoeooeoooec
coo eon—nee ne ounn:on

on» non: emonoeno en oec
menoenoon a onenc noneemo-

ocem
one: ooeneneoc :on onoo:
onoiel moon ne noonnee
e no oennenooeonnon Jun:
one—noel nonoenuoeonnoonuo

 

n mmou

 

econoaeonn
noon

IUHII In uncut

eooooeooe
nooenuoenueen no ooneoco-

Ae.saoov

.n ”wqadwnr

87

decision or information. Case 1 had to produce a demo disc, Case 2
did not have clear design specifications. and Case 3 lacked the design
guides. The effects rippled through the design phase, the scripting
during development, and continued on through the final programing
during production. It would seem that some of the changes were a
result of not obtaining client sign-off and/or the guidelines at the
early stages of the project, as in the situation of the design
document and CH1 software.

The overall design aspects also were changed throughout the
projects (Cases 2 and 3) either to circumvent software deficiencies,
lack of computer storage. or a lack of available time to complete the
effort and make the pre-established delivery date. Host often the
designs were too complex to be achieved with the software being used
or would have required far more programming time than was available to
complete the job. Both Cases 1 and 2 made a software change to
overcome some of the difficulties. However, this did not alleviate
all of the problems in Case 2. Both Cases 2 and 3 continued to make
design changes during the final project phase. In Case 3. the design
also had to be altered to alleviate the overly choppy affect created
by excessive still frame (SF) insertions in linear video to provide
for interactivity.

Storyboards and scripts produced during design/development.
proved to be inadequate, incorrect or incomplete and affected the
activities 'of the programers during production in both Cases 2 and
3. Each of the entries in Table 7 which were traced back to the
development phase as the phase of probable cause related to a problem

or situation produced by the storyboards or scripts.

88

Still frame audio (SPA) decisions made during the design phase
translated into added costs during development and production.
Hatching of SPA tracks proved to be a particularly difficult problem
in Case 3. while the use of poor quality production tapes with a mixed
track affected costs in Case 1.

Case 2 had unique problems with cost and time because they did
not meintain. a current instructional design document. requirements
continued to change. and graphics storage became’ a problem" The
instructional design documents became obsolete when changes in
hardware and software were mede. These changes eliminated an
electronic storyboard capability designed at the start of the project
to facilitate the trecking of changes. Without this capability. the
project team did not continue to keep the documentation current. It
appeared that subsequent changes in project personnel for both the
contractor and client and the lack of current documentation left the
contractor *without adequate means to stop client change requests.
These requests were a result of the clients gain of IVD knowledge.
The knowledge gain generated corresponding changes in expectation for
their own IVD program.

Too many meetings and/or lack of meetings surfaced frequently as
a cost factor in all three cases. as was shown earlier when the
comunications patterns were discussed in Research Question 5. Table
6. In Case 3. the client required weekly meetings with all three
prime contractors as well as appropriate representatives from each
sub-contractor's staff. Heetings typically extended over a half day.
All interviewees in this study viewed the meetings as excessive and

costly for the outcomes achieved. It was not uncommon to have design

89
decisions made during a meeting of programmers and programming
decisions mede during a nesting of instructional designers throughout
all phases of the project. ultimately affecting the amount of labor
expended to correct poor decisions.

It appeared from all the analysis that the activities involved in
designing the program and developing the storyboards had the greatest
effect on activities. in subsequent phases. particularly the
programming aspect of the production phase which in turn affected time
and costs.

Research gestion 7: Is one phase more likely than any other to
affect time and cost of an IVD ro ect?

To determine which phase if any is more likely to affect time and
cost in an IVD project, case data were examined in a similar manner as
for the Research Question 6.

In looking at Table 7, presented under Research Question 6. one
can clearly see that time and cost effects are evident predominantly
during the production aspect of the project. However, when examining
effects for probable cause. it is apparent that the activities of the
design phase are most likely to generate the increase in production
costs. Out of twenty two entries on Table 7, fourteen of the entries
are identified under the “Phase of Probable Cause“ column as being
either all or partially related to activities in the Design Phase.
Both contracting and development phases have been identified five
times each as being the phase where the problem was originated.

To determine the greatest effect on time and dollars. one has to
examine the time and dollars expended in relation to the cause phase.

In Table 7, Case 1, the greatest cost ($8,000-39.000) was incurred for

90

audio studio time and check discs. This was a direct result of the
contracting arrangements. Case 2 incurred extensive programing and
reprograming costs. These costs were somewhat evenly induced by
incomplete contracting specifications. design specifications left
somewhat open and inadequate documentation processes throughout the
project. These probleme created an increase of about 20-25t. in
reprogramming time and labor.

In addition to the information presented in Table 7. several time
and cost overruns were caused by factors which were not a result of
previous activity. These costs appear to be directly related to
equipment and software problems. The problems were usually evident
during production and were due to unexpected and unplanned
activities. Case 2 experienced a software change created by the
dropping of menufacturer support for HT+ Pascal. This change required
extensive reprogramming of the work completed up to the time of the
change. Cases 2 and 3 each incurred heavy costs in equipment
acquisition which were to have been client costs. Further. all of the
equipment used was new on the market. Each project spent large
amounts of time debugging the systems and resolving manufacturer
problems.

From the information in Table 7. it appears that the Design Phase
has the greatest effect on time and cost in terms of labor. The
complexity of the design seems to be the central issue. The
Production Phase evidences the cost overruns created by Design.
However. equipment and software problems as outlined above and
discussed in gxeater depth in the remaining research questions appear

to have an equal effect on time and cost overruns.

91

Cases 2 and 3 incurred repair costs for light pens, and XEBEC
disc failures. Similar types of difficulties were incurred in Case
1. While no exact dollar figures were available. Case 2 estimated a
loss of about one to two man months for resolution of equipment
problems. Case 3 incurred an equal or greater amount.

Research Qpestion B: What effects. if any, does IVD hardware have on
time and cost of an interactive videodisc project?

The data from each of the three cases were reviewed to determine
what kind of equipment related problems occurred during the conduct of
the project. This information was then examined on an event by event
basis to identify whether or not there was a time or cost related
effect on the project as a result of each problem.

As shown in Table 8. there were problems conunon to all three
cases. Equipment just out of prototype stage was a source of
difficulty in all ceses. The new equipment appeared to be the first
of its kind for each nenufacturer. Contractors in many cases could
not explain what the problems were for purposes of this study since
the equipment was frequently shipped to the manufacturer for repair.
For each problem. the manufacturer took from several days to several
weeks to resolve the issue.

hanufacturer related problems ranged from a limited number of
available machines for acquisition by contractors (Case 2) to
situations where light pen and cursor misalignment produced student
evaluations which were so improbable they were disregarded. This
latter problem, caused by size of touch points (Case 3). forced
programmers to space answers to test questions on the screen so

accidental choices would not be made. This problem was never resolved

92

Amman uxoz co venomucoov

 

 

.ono neon use cneeo .oene nueou
.:on> noon oe oon
no. one no so on ease eennnoesoxoonnnn:
.ooooe oneneue oooc nonnioo conning
.oeonue eon oe noneo emee
no no oouem nee cnc oeo nounn coo eneenoo
.onuenno one: no connen eoeo noon;
..oennoenouon coo ooo ooe:noo
neuoc ee: oeoeo. eecno oenuom oe oennoe
unonon oem noun" unenooe oeo cneeo monena
.nonnomeo
eon anemone nee cnc onexcneo noem
.monb noon
on oonneoeno Ionn noelnneoo coooeoo

.conno oe neo enoceon
noneeneoo neonuoo .eenonnen ooo noon;
"emouoeno o:eou no oonuonoeon on oeoeoe
non mono coneeoo nnoennoc nonno lonene
mono :oo nonoooooc anon. ee: nononeenooe:
.eoonoeem :oo onnm emouoeno
coneeooooe oenee on mean no: nonoooenooe:
.uonmmenneno
on nonsennnnc coeeeo emonene oocne
ooe:ooo eeoeoooeu conuenooe on eoeoononnno
.oeen one:
oooc ne enoueeno oennooceno nonnoc coo
menoenn ne oene nen uneme eon ee: mouena
.eemnu
n-e coouemnenen oo o» coo coe eaec
neon .onoenuon noo ee: eenc cneo canon
.oonmmenoonoon
coeoeo mooenm :onn oo nonnooeo :om
.ono:oneo ne non-menoenoon coooeo monenm

.ennooon non anon on coeo

coe eononnen Ieuene en oec noel—no.3 counmna

.oeonooem eon on oon ueo cneeo
coe xenon eon one: no cooeneooe onenc neeeA
.ooenene nonnoc coone: onenc noeoa
.one:anee eooeoo no one:
no evenc nonooee one on moon coeoeo enoem

-onnooon ooeneue cooeeseonooono ooh .coonlnn

one: oeeoe oneneue nononooo coe oenc noeea
omonoeno oonoec

Imam.“ concede“

 

 

 

eeo noeno

oeonee ooeen

eeuene no.» no.» .neenn.
louene ennneoeno noom mnemmwenn

 

 

 

 

nuaF

 

 

eon» on o.-ooo eon. eeoeoo oeneooee noenno coo oo- eoen. eonoonono neenn on one: econooeo , onne,eeenoonn
_
nnemem n «an en cocoeoen _
gene one: cneeo _
nocoen nonconoou neenooo _
ooenom oooeh
oom noon;
cneeo eunonenc
anennoemxoonoeem neonoem neueo _
oenc neooasnonoomeo. mono noem .neonnn oeenee ocean _
neonoem neuee noemnneneno eenc noeoa sin n
85 nee... .8926...» neon 2.32.: eon-no fin co... e3. ...
N oomu Alumna. momma"

 

oooc coo mane nomEno~o>on Q>H no onoacnem no noonnm

. w M1329

93

 

oncmo elonn noon” a on»

 

 

.nmocneuon coo nnnco n on onoeoe
on encone eon no enoc nun coo neee
noeenn connooon .ecemon oo en eoenc
nonnnooon onm neon neo coco: eoenc oooou
_ .oenee
_ on enoc nonoeee oeen monoeno eoon
..noea eon oe oon nee cnoe: monomenneno
...on .noxoo eon e. as oooc ..a nonnno
umeo coo nonnnn: emee .olnn ooeo on a
.neenno 2: no cennncoo
oo on coo eoooooel noon nno .emeno
ueno eon non ooeoe onenene connooon
eon one eeeo neo coc nonunion ooh u

.oeneeon en enoeooom e
econcooo oeen oono: monoono e no: on

.neneo oeoen ooonoe oocne eon coo _
neenoe eon oonel neo cnc one:cneo ooh u _
.monoeno eon oeneo
ne moon eooooo en emnn noeonmnenoc
non coneoee .coeo eo on conenoeon oo
en coo eoeo nonon eon eoeeo emee on u _
.one: oeonoononoo non oooooooe no _
menoeoc en nooom coo emnn neee enon
.eoooooooe oeonem connoc oeo noonn
eon coo eocne eon ooe:neo nnnennoe
unonoo nneoooe neo cnoe: onemcnoo o
.monoeno eoon ooneoeeon neon em: _
I: no no... e 2.2: .23 8-2 ee _
nen nonoo ee: nonnomeo eon coo .nnoo _
_ nonoooee oo-no oe noneo no: onemcneo .
eon eeooooo ee: enoh .nu ooonnenn .
oooonoen non-monoeno eon non nonnooeo .
eon nneonee neo cone: enemcneo noon . _
_
o _
.

 

 

"omonoenn
conoenoooo nenoeeo coo ono:cnoo noe
.noocoo no non no emonenm :eo»

noon :eo noo en oeeo en enooooonnneo _

.noemmenenoc neonn
nonnoc ocee nonoomoe ne oenc eon onn:

coneon oo neo cneeo eocoo oeeoone: noeconm

.conneoon

noncoo coeonoon eo en coo eoeo noonn
.monoeno ooeo onn: nneeeooeo one:
enneo coneeoon coe emonoeno oeneeon en
:eno nnconooooooo one:ne ee: nononoen
nooel ooh .nneoon noncoe non-mono
-enoon non enenoc oooeoeo oenoeon on
eooe: no enoc nonoooe oeen eoennonee

“a“ flrnfifiuwoouhflnmewfiwuuuué
.nennn moon
ooooc en coo noon eeoeeoo enenonoen
-ooom eon onn: oeneeon en enoc nenoeoe
oeen emonoenn conooooooo .nneeonoc

onenoo ceneonnoo neo one: eoonooom :eo ooh

.nooneno eon ec en omnn coo
noemonooo ne neeo eon nonenen .menoeno
eon onecemmeooe en conneneon ee: oooc
canon o .ooecon on on coo coo neon
one: one: .noeneno eon ne enoemonoooon
eon non nnome eon ee: enon noo .nnnm

en cocoeono ee: nnemom monenm moon ooh
.omnn eon no oeonnoo o no

oonoen ooe noonn coo nneo ouo non ooecon

eo en coo eoenneoon .emnnneee ne enoeo
cow noeoe eono onemnnoe eon lenoenoon
en eonoem eenn oeen moon ooh .emnn on
nooeo noon on oooc one:nneo one men-eno
-on on cooo e coeooo nook .noemooooe
nonooonnoo non enoooeomeo eoennonon coo
enonnomeo .oooneno nonneneoe on eooeoe
ooocoeneenneo e coo monenm :eo» eon

 

.nenonoenooom
eon onn: ooeoo eon oe emnn nooeool
ne non e conoocon coo emnn ooeo omnno:ec ne
enoc ennn coooee noo nnnooouonnon cooomoeo
noon .ooec oo cneeo one: eo .ooon no: noel
nonooo enon onno: .nononeenooem eon on ooeo
2.2.?"th z... aflurwéranvsfis
.ooenc eon ooe on
cennncel enoneno noncoe ooecon one: eeenc
onno: coeoonneooo one: oneneo .noneno oooc
noeen eon on one: on neo oeonc coeoee emonoeno
oneo .eoenneonnneooo no nee conoceno one:
emoe coo onenene nonnoc coone: eoenc noeen omen
.nooneno eon on emnn
nommenoenm ennoo eonoem oonon noeoe coeoeo
enon. .nnelem nono'oo eon on 32: neon
coeo oono: nonnn no oenonoe e on oneonoen
oooneonoe oo menn nnnoe e eeeeo oooo on no:
nnoeen eon. .Ionlono o no ee: oneneno onec
nonoomeo .eoonelooooono coo eoeenneo cne:
nen conene oo en coo eonooeno oeoeooo .oeno
.ooenc onem enoceno en oeooono cocce coooou
enoh .ooeoe oooc ne nen e no oeon .encoe
eon onn: conooeo .enoelonnooon enooenn ooh
.emnn ne eooe: «no u no enoc no noeoe
no eoen e nen eonoem oenon non oe noe: enoh
.monoenn oonooel eon oeneoon en connooon ee:
ooe: non nec ooe noneoeo .connoe one: noon

 

 

 

 

_
_ oeo: .nononeenooem eon en :eo one: eoonoeno emoooenn eon
one ceeoee monone :onn noem eon en eooeou . en oouoom eon menn one:cneo no eooeoo ooh _ nee: .neoennenooo nnnon neo one: eeonoeom_:om no noonno
nxooeo n. n ommu _ o emuw mum-on

Ac.u:oov .w mom¢s

94
by the manufacturer.

Other problems which caused direct interface with manufacturers
included debugging machines when they behaved in a way the
manufacturer‘s engineers "did not know they would”. as in the case of
the system not supporting the light pen interaction for motion
(Case 3). Hardware components were not as reliable as initially
expected by the client. e.g.. new XBBEC hard discs caused many days
of work to reformat and recover lost data (Case 2). The XEBEC discs
had to be returned to the manufacturer for repair. causing an
equipment shortage for production. As cited in Table 8. each verbal
interaction with the manufacturer caused at least one day of down time
on the affected computer. A similar loss of time was expressed by
programers in Cases 1 and 3.

Hemory space in the computers was a universal problem. Each
contractor solved the problem differently. In Case 1. a shift was
made from an authoring system to a lower order language which required
less operational space. In Case 2, the memory was expanded through
the use of a cache disc in one situation and in another situation a 10
megabyte hard disk was added. Inefficient programing was also
corrected and made more efficient to reduce the code to fit the
available machine memory. In Case 3. both client's and the
contractor's machines had to have the memory increased to accomodate
the program. These were all direct costs in machine purchase or labor.

There were also many unique problems for each case. Laser discs
caused additional costs in Case 1. Incorrect storage caused some
check discs to warp. other discs separated and still others were too

thick and caused a shaving build-up in the disc player which jamed

95

the players. The discs had to be remade. The contractor learned the
blank discs had not been manufactured to specifications. which is a
separate issue from the disc content mastering process in which a
master disc is made from which all subsequent discs are cut. Finally,
in Case 1. the aspect ratio (3 x 4) of the video caused almost all
original artwork supplied by the client to be redone. This did not
affect labor hours, but project time was extended.

In Case 2, as shown in Table 8. a change from one kind of system
to another had a significant effect. causing five months of
reprograming time. The incompatibility of the operating systems
required a reentry of data.

Still Frame Audio (SPA) was a significant problem in Case 3 which
came to be noticed on playback of the check discs. An investigation
showed that not enough precision had been used in recording the master
tapes and the audio track. Contractor personnel had to be sent to the
recording studios to carefully monitor the entire production of the
check disc. Two to three days of labor per disc were incurred, as
well as related travel costs.

Light pens caused delays and problems in Cases 2 and 3 when they
did not operate or were erratic. Pens were taped with electrical tape
to cut static electricity or were returned to Sony for repair.
Another light pen problem arose in Case 3 when the interactivity could
not be supported on motion video sequences. The problem was caused by
a delay between the appearance of the cue and the machine's ability to
process the students response from the light pen. Resolution of this
problem entailed more production and programing. Still frames were

required between motion sequences to achieve interactivity. A final

96
significant problem arose in Case 3 when the programming or compiling
done on the IBH PC/AT would not run on the Sony View. These problems
took several hundred hours to resolve.

Two basic generalizations can be drawn from the data related to
hardware. First, all three cases used newly issued machines or models
which appears to be the basis for many unexpected problems and costly
delays or labor expenditures. Secondly. each case experienced a
requirement by the client to accomodate a machine with a memory
smaller than needed for the program eventually produced. The data
showed other troublesome problems, but these appeared to be a result
of normal mechine breakdown or incompatibility that is not unusual in

the computer production environment.

Qpestion 9: What effect. if any, does IVD softwaregppve on time and
cost of an IVD project?

To determine the answer to this question, several people from

 

 

each case were interviewed to determine the effect of software
problems on their project. The problems were documented on an event
by event basis and described in detail. The impact of each problem
was then examined to determine if more time or labor hours costs were
incurred for each problem than expected, and the totals were compared
to budgeted time or labor hours for each project.

It was not unusual to document several weeks or months of
additional time because of software problems. Problems ranged from a
failure of the software to support project requirements (Cases 1 and
2) to nuisance issues such as repeated recoding of transitions from
CHI to courseware (Case 3). Each project had major software

problems. As can be seen from Table 9. frequent software changes and

97

 

Amman nxoz oo coaonnooov

 

nononoeoeo noonne eon onn: nonneol oeoo nonne enoeo
nenoeoe on eonoonl nenonoe lenn oeon no .noenln neeo
neoonnoeo e no: onooo coo elnn eoeen ne .ooecon ooh

.onen nonool eon oe noelooenn

nonenn nnoon onoeoo on elnn eon coo .nn ec on nooooe
oe nonconn .eennomnoe on nonnneeen connocon nneoel on
loon nonnoeo ne cooneon enon eon oe eononono nonnnon

e.noenno en ooc eelnn noel ooecon one: eoeen

.eooen:

.onon nonool
eon oe conennono coo cooocenn one: noon
noon oeneoonoo ee one: connooon eononeno _

--------

 

.ecee nonool m-o oeen eoeoooen
nonnco no noelonnooon oenneonnenonee coeeenoon

.eononeno onn: ooene eoemoo no conoooon ee connee
on on .ennn econ goon ocean. one: eennn .onnoeno

.one: ne eooe: nenonoe oeen
cool-onoennoo .eeoc cneo eon oe non on no ooecon
on eeoeen on on een none-ennono neon... eon no no.

.ooenenn noon nen connooon one: eoennooon
«no noeoo .oennooon non enoc eonon en onoeo e:n
lenn oeen eeenneoon onnn: on coeo ooooooen nonomoeoo

.ocee oenenoeoeo e onon: on elnn oonlnoennon
ne enec «no n coeoee a: on on: noeeen.lenn oooeou

.oceeon on olnn no eonool oum oeen enoh .ene:nnee
one no cool-onoennon coeooe «nouns on one lenn oooeou

.connooon non-menoenn no enoc neneeon
.ooec .nconne non-lenoenn nneonlnnonn on ooc noon-n

elee no ee: on: neoeen en eonn nonnn lenn oooeou .

 

..ommzfiflan n
nncoonnn oeo onol ecol oo en cocooo _
noonno eon no connocon lonoenn nenncn

.oonn eon
lenn eooononn ooeool en coeo eo: eoelon
.oeene ne non e cocooo eonnn eenonenc

.oceo nneoeoooooo no oeoneonlnno a eoonn

oceo on ooene no oenneocen ne con-neoee
econ .oenc cneo oe non on no econ on com

.oeene on- een no oeen econ oon'onoenn

.ooeoooen
nonoloeee on oonnnn: oo on coo noonenn
eon no conoooon eoeoneoon noneene nenoeom

.noea no connennoe eo on oooeo nee no: on:
oeoeoeo oooeen a: on on: neueen lenn oooeou

.eno neeeen no oeoenoe nooneoe en

oenn:e e connooen eooh .lonene nonnonone
moo-mu on one lenn eeooeoe coeoen :eo,

noon on on-omm noon lenn eoonoeol ne oooeou

.eoonnoen
one mnennon neo cneeo eonn nonnn oeoeo
:eo on: neoeen on eonn nennn lenn onoeou

 

.el n no omen-enn o oeeonoo n e eoeo nooneoe
n on on ooe men eeeenneh w» o no: .oeonue

eon oe eonneloooonne eon oneneeo en coeo one: eoeenneo

 

 

mmdmflfidmflmflmflufil

—
_
_
_
_
_
_
_
_
n
_
_
.
_
_
_
_
.
_
.
.
.
.
.
.
_
.
.
_
_
_
_
.
_
_
_
_
n
_
.
.

-‘—-

.noewomnMNMn noe

coooenon eoooooe conoe: noonno .oenenooo
oennoonooe o no: coo nonloeoeo elnn ee:
oeonoe eon oe eunnolooeonne ne noelouenn

.ooone nnelol oool eeueo
oeen lonene nonneonoe eon oeoeooo eoeo oen

nonnn on lonene nonneonoe lenn oooeou

 

ununnasm

 

 

 

noqm,coe many newsmano>on a>n oo onosnnom no noonnm

 

ooe-on
.u. mononoa n eoou
_
_
_
_
_
..onnoeno. ...-an .no nee... .neenn.
no nee..e .nnee.
.ceee non. ..oo noon. .nennnsn. n ...u
no
.... : scroon.
nonnonnee gonna... neenno .nennnon. n e..o
unnnummomnnmm «xhnmmmwnuun
.n annoy

98

Awwom uxoz :o coacmucoov

 

 

 

lull—LII! HOE II,
'3 ~53. .onoc nononoo nono onnooon on onaoo nool
noon nonunion on ononnnc oon no cooooo olonoono oon.

.olnn noon no

onoc 2... cooooo :5. J: noon .o. 23 on ononnnc
oon o3: ononnoolooon oo 3 use coonou oononon onon.
.oonouoon :oo oloo no ooouoonco ooon on oononon noaoo
o on .0. oonunoa no oononon ooo Ionn conoooo noon.

.oononon noonnoo 2.5 2:332: nonoonoooo coo noon
on: non-nonooo coo olonoono no o oo onoooono oou
nonnonooonc olnooaoc no onoc Hon o co u oono:
connou-on no: no oononon noon: :5 coo Ion-no ooo

 

.coonlnonocoo noon
on.» mo non o noon oooo coo ooonooon nool one: one
. Coon nnon oo: nonoonnon col: oononon oou no

on no—Ionnonn coo non-nu ooooo : uoo noo: Ionono

eon nonou on onooon cooooo nunoooo ououo on oononon

o none—3o coo .3: o on connono nanoooo noon oo

-oo nonooo nnoo one: ooooon o on ooononen nonlonnonn

 
 

.onuooon on
nooon coo olnu no nooolo coon-nonecoo oo connouon coo
oonno connoooo =5. ..3 oooo ouooonoo nonlnnona
Solo o cooooo noo .Ionoonn nno o noo oo: onon.
.353: 25 5—: oonnuoooou no onooou o oo: onoon.
.HE on unoo coo oonooo eon on :0 Ionn nonnooo no
nonooonn 25 no cooooo oo: no: :5 oo:. lonoono eon.

.olnn .o-
oon no noooo on ooo noon—n coo oonuonnoooc oncoo
eon no.- on noon oon: nonono: noon. one: 3.8.. noon—

.connooon Zoos noonono no unooo
nonnonoc .oonool nuo noon .0. ounuooa oo noZnIoo
.2 oo: cnooo :onn noon on ooonooon Ionoau nonoononoo

6:38 on
onoc none-o. ooou oonc oocno o no elonn nonnn 2: oo
onn no noncnooou con: 2 oo: no noonno oo: :5
no oooc eon. .onoonnooc non Ion-ens nunnnno noncon-

o nonoononoc no anon on .connooon one: olouoono onn

oo:-noon oo: noonnc no oononon
coo o o ooooo ocoo nonunion olouoonn co:

.nounnloo
non ononnnc onn: ononuoolouon oo: 3
cooon coo o oo:»!— no :33: cooooou

.Iononn
:o; noon non no: no oononon noon: con—ooo-

.no—Ionnono no you o noon
coo oc on cnoo nnon ono: oocon coo ooononon

.olonnono 2: 6:: oo: noo cone: :8

.328 nonno onooono nooonn .ooo oncoo
o3: connooon nnuuennoo nonoo nonoou ooonon

.u 8.31—
onto ononnogu noo :onn noon on non—goo

.ononnonioo uoo one: oouonn
oncoo coo oocnn no ooonoon ooo ooooooo
Ionoono o oo: nonnn oncoo olonn :nnn

 

2.30am 55. no nbunnn

 

gash

 

 

an. 338»

 

no.u:o°v

cooonoooo
n ooou
Ia

. m Hannah.

99

 

.cocnooon noo connno noonnoo onooo olnnnooo nool
cooooo ononnooo onoononno on ooono oooon oon nonocon

.ooooon ooo on :oo onoon on oono no noc
onnno oo oono .coooono noon coonoo oono ooonoocono no _
onoc nonoooo cooooo ooe-on on nonon on Ionn onoooo ooh _

 

.conooooooc noo oo: cooo
noo onooconn oo ooono oonn .noool coo oono no noo
oon noon ooooooo on cooo nooo- .noocono nonnooo
onno o nonoooc on oono o:o oo oooc oo: nonnnn:

coo oooooooooooo onooonn no non o coooconn noonn:

.ooonn:on noon on ooc oooo .o:u

-onooon oo noon oo oon:o noon oonc nonnn .connooo noo
oo: olnnnooo .olnn no ooonn oo: onolnnoo on .ooonn

uonnnoon nnu oon nool on oounnn:on oo on coo onoon noon

 

 

 

 

 

.nooono oon oo c onnooo oooon
ooo: cocnooon ono: oonooon oooeoonno

coo swxn oooo onol oo: nonoooo noouoc
.ooono ouoon onoooo nonn non coonnooc one:
olonnono oonooononon .coonnnooco noon
ooon nonnon oo: ooonoo oo oono oooou noon

.nonon us on oooc oonononn
:- 38 3 o... .26 3:. :eo-oooc oo-
oo: oonoo us ooooooo oonooonn no nonnooo
non oono.— on Ionn ooolon on onoooo o... co:

.cooocono ncoonno oonooonn no ooo
conoononno ooonnonnnoon ono:nnoo coo oonH

633.. n6 2: no
ooooooo co:onno one: one:ooo anon o:n oo
coo ooonnooou no on cooonnnoon one: ouooh

 

nllaoo-M nah no Hmummm

 

Blow-m...—

 

g

 

no.uaoov

cooonnooo
n Coco
undo

. m mqmdh

100
problems caused by lack of space or programing problems unique to
each software were the source of a major portion of the additional
expenses.

In Case 1. the authoring language had to be discarded and
replaced by a low level programing language. The reason for the
change was twofold. First. the initially selected authoring language
was too large to fit the available computer memory and leave enough
work space for programing the course content. Second. individual
alphanumerics had to be sensed from the keyboard during instructional
lessons. The authoring system could only sense words, not individual
key strokes.

Case 2 also had to change languages. not once but twice. As can
be seen in Table 9, Pilot Plus. Pascal HT+. and HS Pascal were all
used during the contractual effort. In addition. a change of
equipment during the project changed the operating systems from CPR to
HS—DOS. necessitating yet another reprogramming effort and a switch to
a second version of Pascal HT+ which was still under development. It
can easily be seen that an additional five months to a year of
programming time was added to the project simply because of changes in
software. An additional effect in Case 2 was the need to reprogram
months of work done by an inefficient programmer to make the code fit
on the storage disc. Further, a programmer able to write code in an
assembler language was needed to program functions required by the
client into the program. This took additional time. up to six months,
because HS Pascal did not perform as required. The total effect of
software problems on Case 2 was at least 12-18 months of additional

project time and corresponding labor of two programers and an

101
instructional designer/editor.

Although Case 3 problems were more numerous, the total impact was
not as great as on the other projects. Incompatibility of software
with compilers and drivers caused significant down time. The need for
more storage space for graphics than was available required time to
find an alternative media method. The solution was to produce an
overlay on the video track of the disc. In addition, there were
timing problems with cues and audio tracks. lengths of audio tracks
did not Imtch video sequences, and the touch areas on the screen were
much larger than expected. These were time consuming problems. but
not of the same magnitude of those experienced in Case 2. The most
significant problem in Case 3 occurred when Lattice C was not
compatible with the Sony View compiler and delayed the start of the
project by several mmnths. This caused a time schedule problem, but
did not affect overall cost. However. the frequency and number of
problems did add up to additional costs which contributed
significantly to the cost overrun.

From these data. it appeared that software problems were easily
the most significant factor affecting cost or time on an IVD project.
Lack of planning or experience led to selection of an inadequate
language for the effort which was frequently only discovered well into
the project. This usually required a language switch and conversion
effort leading to a major cost overrun in reprograming. Also. as
seen in Case 2. software suddenly lost manufacturer support or was no
longer viable because of equipment or other external changes.
However. as was seen in Case 3. the same language throughout a project

did not necessarily predict a problem free project.

102

Research Question 10: To what extent do unintended events and

un lanned events create a time and cost overrun in an IVD ro ect?

To answer this question. it was necessary to first define
unintended and unplanned events. Two criteria were established.
First. such an event was not part of the normal instructional design
process. This criterion separated “unintended and unplanned“ events
from those in the process which should have been foreseen but which
were not due to lack of foresight or experience. An example of an
event which should have been expected was the necessity to reformat
graphics which were not in a video format.

The second criterion for defining unintended and unplanned events
was that such events were caused by something out of the control of
the project team. For example, a misapplication of software would be
something which should have been controlled by the team. but
malfunctions of the computer would be an unplanned event.

To identify these events, the data from all cases were reviewed
for time and cost overruns. The associated activities which appeared
to have caused the overruns were examined and entered into Table 10 if
they met either of the two criteria.

Table 10 describes these events and their effect on both cost and
time on the project. Review of the table reveals that. with few
exceptions. the major effects were technical in nature. These
involved either the IVD machine, the software. or the manufacturing of
the video discs. The only exceptions were improper shipping and
client escorting requirements during visits (Case 1) which took away
from time actively spent producing the product. Case 1 also had a
unique problem. The videodiscs are composed of layers. Four discs

began to separate which made them thicker and caused problems in the

103

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

. eyed
“ ye seen on canoe eeeyoe memo» one on eaeaeue
_ eay no eye». gooey :eo yoeuee oy oeummeyuoyu
. .eyemeeymoy one ecu-leyuoyu ovey uey use nouyuyxey sees-o yuan .eeouyeuuuuoeue
_ .emuy .eyoyuve eye oy euev neyeyee one meuaoym on» eye: yeyayoeuaoem eay eu coy-ouoeu een» yeuyea noel ye:
_ y.yemmeyuoyu you eyeNNov veeemoay ueyeyem uueuv oy modem uo eyeoa veyooea ueyeeea .Nyueuyoeyeumu you veee .eeeye meson eeeyoe uo emu»
_
_ .38... 8 3 .... .8... 2:. is... a:
_ .mennoyu emu eeoeueuv oy euev Neyeeee one no con-amey on you mason an .eeoo uuyoeyyoo
_ ..ueuv emu ooey oy yeyayoeuemem oy yeoo use .eoeuv :eo you cue: ones Neyeeem you ye: eoeuv yeyeem on» no mum on» uo neusou
_
_ .eyemeeyeoo emu eyeooyy
_ .yue: euay oy eeo ye: emoa .eayeol Nu cneeo menu eyouea menu you sue: on we: yoyoeyyeou
_ .uemueun coo-yum uo veeueeu eyeeh N uooy yoenoyy oak .emeyeue new uo yea-mm veyumun e eye: eyesh
_
_ .ueNyeeo ouaueyu on: menu yeAyey .oeuc emu oy
_ eoeea use one» yea-em mo menu you .IeAy mono-ea
_ oy we: yoyyeyyaoo een .yyenoym ea» cu veyueoey
_ .eyeuuov voyamma ueyeeea .ueuyuve young-ya emu Nae eyeoommoyoe oy nueoee euyen yo-
. .umuyeyeel eye umuyuve you emu» oueeya wee omen neuyeyeel you eueo Nemouyuvqc ee:.meyeNe aha emu mo yeyagmoy eey uo Nyuoeyeu n eeeu
_ Jay .9 so.
_ on» yeeo uuymm eeuyye you cue ooe-muaue name—Nu
_ .emoo eye; oy ye: ymeuuo .eyeeyeueee yueAy veyoenuey yeuvoa oak .eye:
_ emu yen: o0 oy eyoyuve eye eye-layuoym .mouuemuouyyey use—no uo seen on eye neeyaoo ea» uo neuyuve ve- umumleyuoyu myu: one:
_ uo yon-N you ye: emuy eyyye eey uo eyeou you-cu enyeom ueyeeee noon ooonoyy emu eye emeyeum :e—y use» on» eueyao oy veeyue yeeuuu
_
_ .ooo.eONu oy ooo.emuuu .Nyyeetey .aanuml ye: one-amen smug-yeuo :eo ea» eyeue
. .eemuayem :ee uo «you eeuy .eyeoo neeeyy eyes yeeyeoo uo eayeom m .eyemeeyeoy uo neummeyuoyyey eu mean-eey yuan .meyeum :eum noon
see yoned you eyeauov gee-mom» ueyeeeu eyuymey see scum-eyuoymey uo amuse: any _ oy asset» use» moyu yeemmumue cone-mo yeeuau
.eye:yuoe veuyeyeoy asses .ueoeey
ou Nye-eeeem eye: enoc neyeyem .m: on «I oy oemm-yuoyyey ea ou we; eyemyuoe yuenoym
yeah um nee-em moyu ou oy meyuoyy you. .uu usuyyomyee commoye yeemcoyu enema-en one eye
.ueuuuemev nee eoueyeemoo you emuu yemmeyuoyu uyeeeoy e neuyoueeeo amen. eye: once can ..s: nee-em you yyoumee yuemy veyyoyo yeeuuu N eeeu
. . .you...\uumlmmmlmmlummlumdmmm
yoyyeyyeoo yea .veoeemye eye:.eyeoa emuyyeeo yemmeyuoyy seen up oooc ea annoy eye: on .yueue Nyeee eo neuuyouee veyumvey yeeuHo
.veeuoeey umuea ye: .yoeuv oecue emu neueevoyy ovum:
_ .eeom menaoyu emu euua: Negev uo euev ueyeeem eeeuaoem yueay cameo oy we: yeyeyyeueeem you:
.emeueoym yen-«y seemey see you‘ve: emu mu venu-
_ you enemy-me one yes: .eueeveeyem eleven 1m-
_ .eeeuayem .ooeuv on» venue-yo. eye: yueun .eeuayem ea» mu eoeeyeeny
emu eo aye: oy yoaea .yeomooym you. one may oy veuuuool one eeyuemey eye: “ecu-yam emu oeuveeyue .yeuouay you. eay nause-
ua eeyom em: yeoo “emooey em on men eyeua eeeuaoel emu euua: een-new ye: yoenoyu .veyeyemee eyenen one see seen-em emeo eyeuc ooe—m
.yuae see .cocooo-a y. el.» aoee .yeyeyoeueeem em» up
eeoeueum oy eyeom yoneu - yeoo «emu-u: eeumoem emu eoeuyey on one. a oy ya soon voyeumey on on see eye veeouyyeeunem emeyeue emu
.eouyeoou yyeyyou en» es eouyyee eyeyyyu
.yyeuo uo so no commune ea oy men eoyua .eyeye umoy: emu
yeemmuaeey you eyeanov veyveem ueyeeeu .eoyuv you neuyue: enoc N a. eyuuuo me on eeeuo oeouy uo eeyoa yeee ueeumm n eeeo
i.. .3385 jyyoyeyeoyuflyy a all
mucm>m vmflfimfimdb flaw vwvaouflwab ha mcbhuw>o umoo mo flamumwko .OH mqm<fi

10¢
laser player machines. The increased thickness created disc wear.
Shavings from the discs jamed the players and the wear made the discs
opaque. The programers had to ”guess“ at visuals until the new discs
were produced and received. Project delay and reprograming were the
results of this problem.

lachine malfunctions were a general rule across all three cases.
In Cases 1 and 3, a shortage of manufactured machines caused repair or
replacement to be delayed. This in turn, resulted in project delays
of several days.

Other unplanned events related to the video discs themselves.
This occurred in Cases 1 and 3. Either equipment malfunction (Case 1)
or procedures in mastering still frame audio (Case 3) caused the
problems. and the discs had to be redone by the manufacturer.

Case 3 experienced two other unexpected problems. Both problems
related to machine capabilities. In one situation. the computer of
the IVD system did not have the capacity to handle both the authoring
language and the required graphics. This resulted in production of
graphics and their transfer as visuals to the master tape rather than
digitized for storage in the computer. Secondly, Case 3 experienced
problems when the touch areas on the screen were larger than specified
by the manufacturer and more than one intended touch point on the
visuals lay in the same area sensed by the light pen. This problem
required redesign and reprograming to ensure touch points had more
separation on the screen. Graphics were redone to enlarge diagrams on
the screen. and test answers were reformatted by resequencing answers

when more than one response was correct to avoid an overlay of the

touch points.

105

The unintended/unplanned event problems in Case 2 were almost
totally related to the changes in hardware and software initiated by
either the client or the manufacturer of the system. Software changes
were two-fold. The dropping of support by the manufacturer for Pascal
HT+ caused several days to change to Pascal NS. The change in
hardware caused a change in operational language and resulted in
several months of reprograming. This hardware change from the SEC-70
to the Sony View 2000 was significant in cost ($150,000 - $200,000)
because of a need to totally reprogram to use the new operating system
and then to reenter project data. The final problem in Case 2 was
related to machines, but resulted from the client's lack of
responsiveness to help solve the problem. The client had agreed to
provide part of the programing and editing team, as well as
equipment. However. the computers and players did not arrive until
late in the project and were improperly configured when they did
arrive. The contractor had to do the work. causing cost overruns, and
had to delay delivery of the courseware, comensurately adding several
months to the contract performance time.

In sumary. the unexpected and unintended events which most often
occurred were related to machine delivery or failures and disc
manufacturing problems. These problems can be extremely costly and
delay projects by several months. as seen in both Cases 2 and 3.
Research Question 11: when and under what circumstances do
unex ected costs occur in an IVD ro ect?

To answer this research question required an examination of the
data with respect to those costs which were either overlooked in

budgeting for a project or for which cost was underestimated in the

106
budget. The data were examined for cost overruns and the probable
cause of the overrun was identified. The results of this examination
are described in Table 11. Two types of cost overruns were
identified: direct costs (machine purchase or shipping) and labor
costs for unexpected hours beyond those budgeted.

lost of the problems described in Table 11 concerned the
production phase of the effort. Closer examination showed the
greatest percentage of the problemm ‘were related to hardware and
software. In all three cases. many of the hardware problems resulted
from new equipment malfunctions or unexpected limitations. Beyond
this. there did not seem to be a pattern across all three cases except
for a language change. All cases changed programming languages.
though the Case 3 language change occurred prior to the point where
this study began examination of the case. However. the effects of the
change were still felt by two of the primary contractors in Case 3.

The unexpected costs in Case 1 resulted from higher than
anticipated travel costs, audio recording, production to provide
higher quality sound effects and a change in authoring languages.
Travel costs were for a variety of activities, mainly to attend more
project review meetings than anticipated and to work with the
recording studios to produce ithe audio tracks. The change from
authoring system to language resulted in a 501 increase in programmer
wages. The audio production costs were high because the sound effects
were very poor in the original media sound track. Further, there was
background music which had to be diminished in the videodisc sound
track because of the segmentation of the learning sequences which used

only portions of the sound track. Case 1 also experienced high

107

 

.el—u ye'ey-oym uo enev 7N use eye:yuoe uo eeesuyom
.33". yea:
3 eav ye; ye.— eayyeyo 89.3» 23 uo eueyeeuyem e-yeu sees-u

.leunoym
e5 eeuoeey oy veyuevey eye: '3 yeleyuoyn uo euee: ueyeeem

.yeeeea yeeo ooo.eNe - eoo.myu .eleyaoye «my eye ecu-lee-
voymey .uauyueyyoy louyeullayov 23 Se 3 emu layyeeo ueou

.uyo: eouyoavoyy eyol see eluy eyeleyeoyu

..n may
see 539 ouveye .eluy eouyesvoym uo eyeon neyee e veeaeu

Joe—.oym e5 uo one e5 :2. eueyuem euzoeyu
euaefi. 3 we: .eye:uuoe :5 e5 :2: eunuuemloy yo: eye: eyunmeyu

.vaOlm tum
you 2.- .yeeleyuauey eyuuueyeyeu of you ecu-eye ou eyeeovee yeea: lyee

uo one 2: ye nee-:— vee Eonenev en 3 men eleyu :uym .eIeuaoym ul—
ue 2.8.. 2.23: uo eeaeoea eeyeeovee ooe: yeeez :33: eyes on you 3:3
:5. .eneseun vee yeevaye e5 eee:yen nyu-uyueyeyeu veyuavey leyuoym emu.

.ayuoym ueeouuuoyyeeu e5 1:: an:
:3: you Zoo: :0 e5. .uueaoym e5 mo eyeayo :e n.— ee: you yoyyeyu
:eoynse emo na gnome-em ee: ewe—vol 3.6. louyueyuelu ceueeel yeyeiou

6:: en 3:3 euoeyy neayee egy uo eyes see ye;
e5 ee elee e5 you eye: eumuyom .eyeuo u: Nee!— eny no teen en 3:3
52: S Heel.— e:» lo eyol ee: eye: .weyou eu venoueeev ea 3 we: en»

.9: e5 you ee_. 3 venuemee leny yeueou. uooy 0 eBay-.—
.eeouuoaau veyueuey e5 uo nlll lyouyem yo: Buoy ones-een oeuyofile
lay eeleyea .0 e331. .eue: sen amuleyuoym e 3 eueegy ou vesse-

.I: 2.3: e
elevey eeeuoym e5 wee youyeyyeou e5 na see—eyes: em on sea eeeuyueauey

eo—yyevoym ezu. .33 oo:- ea» 3.3 2.3.3 5 eleuaoym ueyeyee veeeeu «nu

n eeeu

 

.Ieyuoym loueyeeloy e euuy: 3 ll: ylleyuoym enev can

.leey uo n
n3 3 een youueyylou 68.3» =er ee: eleyene :2. e5 uo
ueou .393 3 8: eleyuoym N you oeu'yuoym uo efieol a-..

.euneZe-eee eyel—yee eluu.

. eyeeevee Ieyuoymey you ewes:- eyeyeoueeu
may veeoye aye: 3 eluy ye'eyuoym vee youuoe uo enee ueyeeem

.eluy ye'eyuoym uo eyeoa 3.3

3 veeeuey yeyauyeuaeel men: we en u: on out Ne em onu venues

.leu ne ul—yeyemo mu e-aeay e ue eeaeyen
uyouue aluIeyuoymey yonel e me o delay euau. .yeymyueumlel eyegyen
ole yeeuuy e5 nn 93:25 ee: uyenoym 2:2... yells—eve uo eueeau

  

.eouyyevoym va- :3 ev uo use you eaneuueee you eye: ea yeeuuu

.leuuolvoym

nee: eyouen veyoeyyoy en o» yea Se eeefl. 65...! you yeeyeou e5

3 seen me: :34: eeuyeyauyeeu 95.3 9.3- one nn eouyevuuee ueeuu emu.
1:33". of no 3:2...- eye: eyyemee yeyyel yuenaee uo emooyu ueyeeen

.youyeyyeoy no eyes on e» we: yo lumen—oeey you ee: Ieuaoym .e lo
eloe lu .eleuyuczuey yuegy 5:: 23.... e5 lo eyaoa nmel veyulvey lo:
nmuoeey one .veysvoyyeu yeyuu een: eIeNnoym nlel sea Ieyene neon :2. e5.

 

 

.GN—hhobc “.90 on n\.n 6000‘

.330 Nepeyy money—=5 eeay yea-=-

.eye:oo veyeeea
ueye e ee: yeou .veueuvx uol ee: nyenSev Smey nn 9:23»

.e 3 alleyuoym
e5 net—€59 nuyeee .veyuauey eye: eyeoa ye'e uoym eyol nae:

2.3: 25 em 3 yes: one ouvaye .33: us :8

.eeovey en 3 veveel see yoom eye: euyeuue ...—see All:

.3: eyoyueyyeoy eey loyu Neeeyy toy—av: Son 6::
uo use 32. eye: eye—yell see eyeevuae usueyey veveeu eouvaue eouyyevoyy

.veuvemee yol el:
eeeym-n neyeven uo eel an: ear .veyoemee ye.» enamel el: alums—an

.eueaueen .3353 en: yon: oy eueeay oy ny eeoec ee: an .3on
eyed-loo nHeo .eeuoyyenea oy yyeey you 339 :3: oeuyoayle neeuuuyo een.

:nyeyy veeoe ye; oo:- ea» ... ouei eay ..eueul; oy uluuuve ueuueme veeeeu
eeou. eufl. .yuei veeoynuoea 5—: Some ee: 3 veyyou ee: yoyyeyyeoy elk

 

hum—duh u...— 8 Unnnu 58

J33. eye: aye—.oym 25 you see: on on eye: nous: emu-ya veno- Nee—:8

 

 

 

mg no

 

ysouo eumoo vmuuoaxmaa nomnz nova: mmoamueabuyflo

 

— eeeo
lav

.NH mqm<y

108
shipping costs because an express service had to be used as a mode of
shipping.

The unexpected costs in Case 2 stemmed mainly from the newness of
the. IVD System and changes in systems and software. As has been
mentioned before. the change of software was necessitated by an
equipment change and by the withdrawl of support by the software
manufacturer. The equipment change was initiated by the client and
resulted in several mmnths of reprogramming. Because of a change in
the operating language. the new IVD System had many hardware and
software malfunctions to be resolved. The issue of subject matter
experts affecting cost is not clear until it is recognized that they
were also to be part of the design and production team. Their absence
forced the contractor to provide more labor hours to complete the job.

Almost all of the unexpected costs in Case 3 were caused by
software problems or changes. A change from an authoring system to
Lattice C caused more programming time and additional costs for labor
than anticipated by the primary contractor. SPA was a big cost impact
due to improper production techniques at the studios, audio on the
original Level II discs and the need to develop SE at the end of every
linear sequence to provide interactivity. The inaccurate discs were
caused by the production engineer's lack of precision in laying down
the tracks. Because the aim of the project was to convert Level II to
Level III IVD, it was assumed the Level II tracks were sufficient.
They were not and audio had to be redone. The lack of ability of the
IVD system to accommodate interactivity with linear sequences caused a
need to intersperse still frames between linear sequences to be used

for interactive courseware. This need caused several days of

109
production and programing.

The CHI developed for Case 3 caused two problems. First, the CHI
would not interact with the instructional program without some
software design. The CHI restricted the programing. As an example,
tests had been designed with a large number of test questions and with
more than one answer to several questions. The CHI program was
structured to only accept 25 questions per test and one correct answer
per question. This structure cost several thousand dollars for
programers to redo the software and courseware. In the second
problem. the graphics package had to be switched because of
incompatibility with CHI. The Lumena graphics package was used for
linkage with the CHI. This change resulted in several hours of extra
programing and about 25 hours of overtime to reprogram the existing
graphics.

In sumary, a review of Table 11 showed that all three cases
experienced unexpected costs due to a software change. However, the
remaining unexpected costs across the three cases were not very
consistent except that the majority were related to hardware and
software problems or malfunctions.
mestion 12: When and under what circumstances do unexpected time
dela s occur in an IVD ro ect?

To determine the answer to this question data from the three
cases were examined to identify when time delays occurred and the
circumstances which caused the delays. and then the effect of the
circumstances was quantified in time. The results of this analysis
are presented in Table 12.

A review of Table 12 showed that the production phase was when

110

 

.eeueeao euel oy eneo aeyeeem

.enev nuN
.lenaoym

aoee you euee: ueyeeee on enee neyeeee ueoA
.eluy yelleyuoym uo euee: enue

Hueyeeo .yeou eye: euavol yen enee neyeeem
.eyeey eeueeeey use eyemyuoe ovey on me:

.neaov 23.3. N

.eluu yelleyeoye uo eayeol mue

.nldlv uo lull: nuN

 

.eneo e:u
.eluy alulleyuoym eeyoemee lemy yeulon veeleo

.eyoyye yoeyyoo oy eyeoa
eluuyeeo neel ole oeueueyy you enev Neyeeem

.eoeeyeleeyuy mole you neuee enev m on ma

.enev nnN no eouyueeoym eeneuem

 

 

«c .Ifinnnmanuununmmucmuuu+

4
_
_
_
_
.
_
_
.
_
.

V nnnnn ammanmmummeummnumnhuu“mmuumummmmmmUImmmmuwmmmcumnuummmMHQOl :m-c+

 

.eeueyo: use yelyou ymuyoe oy eeeeeso neel voyeeeoey yeeuuu
.eleyu yoeyyoy one no «nm em» you oy eeeo ee: eoueuoeym
aeeoee you .emey yeyeel eay eo o:ov yam nuyeeyyooeu ee: «um
.ueulsemoy
eluy nyee eye: .leaaoym emu eeuoeey eeay see .yu eeuoe oy eoeyem
yueyyou ea» aloe uoeyy .leneoym e ee: eyes» yeyeyoeueeel esy eoeue
noon on enueo eeolm oak ..eusa. venuemeeee een eeeunuel neom :eo law
..eeen you eeouyeeau mN uo eyeey eavee: ngeo cueo: yu ooe eueu
..yeyloo ene:ne yoe ee: yu .eeueeylou yeelel leey Heyeeee e5 uo eeo .

 

 

_
.eye:eeyeoo one eueel emu uo eouyevuaee eo eavyeeo ee: yeeuno .
.yeueon one» on _
yoenoym ulueyou .eluu uo eeou emu you me euel ou yeueou eye: on men _
yoyueyyeou ooe .eyeu eayeol oN veeuyye ycelmuaue yueph .eeulleyeoym .

see eouyoevoym eay uo eyyem eeouyee ayu: one: oy veeyue we: ueeunu
.euueev ucuyav :eou uol

eye: soda: see .veyoeyyoo ea oy een noun: eyoyye aeyoveyoy ueyeeee
semen .eouyyayyeeu eyecuney oy eu venueo eyyemye.u«yunuowwunuum

voyueuey eye: yeyeyueuaeel ayu: enueo eeoam noel ooe eeoueno enemne

.uyo: leyu elem eumoem ymeu eyue eouyumcoym ue vyeeeuun

.eluy e ye aye: oy eyelleyuoym :eu
e name memeune yeAy ooe eleyene men uo eyeglse aeyuluu eye: eyegp

.eeyyemye

een» esan yoaum Ayu: ye:o~e new eye: eyelleyeoyy yeeeeye een .eeum
youum ou leyene ueuyoayae leyu veuoeao ee: eeeeeeeu neulleyuoym ean
.yeeleoenyey you yeyoyeeuolel _
emu on deep yeee em on we: use ueuyyo: eoye vane: yeelmueoe emu _
.yyeyyeoo neuyueu emu uo yyem _

uol eye: nous: eeouueuuuueeme elee oy neuvyouye me eeel elo eeea
ey venee see emey elem neueeme e veeoee nea» yeAy vevuoee yeeune emu

 

hounnu

d -—-—

ne eemoueeee ee: eye:yuoe numuu eouyoeyyeeu yeeleeeeen yeyamlou een _

China-.00
one u. use-ecu...e
Chin-NICO
.ay yo eouyuaeoye

louyumeoym eel luueem

eouyeevoye neuyem

eouyeeuyee
ee- eoyyoeeoye

eouyueeoym neuyem

eouyecuuee yeevoyn
mouyoeveym use

+ nnnnn yummnmmMeeUnue men-

mu you em
\eeue uo eouyoeooyy
eyeeeeyeoo

emu uo louuoeveym

eyemeeymoe emu

uo young-coym eluyem
eyemeeyeou eay

uo eouyyleoym uluylm

emey yeyeel
uo eeuyoeeoym eeuyem

 

 

mac-damnauauu

A

m

eeeo-

a eeeu

 

 

mnmuoa mafia wouoomxmcb wmwummyo moucmyeaauywm

.NH

 

mania.

111

most time delays occured for all three cases. Case 1 experienced
delays with master tape and disc production and programming of
courseware. Case 2 experienced production delays because of equipment
malfunction or late delivery of client equipment. Case 3's production
delays were due to three separate issues: (a) limited capability of a
CHI package which prevented implementation of several courseware
features, (b) malfunctions in new machinery, and (c) a lack of editing
precision where audio was not always begun on the exact frame number
necessary to match the visual to which it referred.

There ‘were only two other instances where time delays were
experienced due to unexpected circumstances. The first was during
courseware validation or formative evaluation to check the correct
sequence of course steps when the subject matter experts discovered
errors in the content of the program (Case 2). Time had to be taken
to correct these errors. The second significant delay was due to the
unexpected emphasis by the client on script format and appearance
(Case 3), when the script appearance had not been considered a
significant issue by the contractor.

Table 12 also displays a series of circumstances under which time
delays were experienced. Each case seemed to have its own set of
issues. A review of these circumstances revealed that each case had
equipment problems. Other significant issues which have not been
mentioned above were: a change from an authoring system to an
authoring language which slowed down inexperienced programers (Case
1), overdue responses from client on product reviews (Case 2). and
weather caused delays (Case 1).

The three cases demonstrated that unexpected problems with

112
hardware, programing software and client-instigated changes in the
courseware or documentation are the most often experienced causes of

unexpected delays.

Summer: of Findings

There are several areas which appear to have created the largest
negative effect on the cost of design, development and production of
IVD courseware. These areas are:

e Software

e Hardware

e Client knowledge

e Contractor experience and qualifications
e Design

e Team functioning

Overall the data showed that time delays ranged from about 2
months in Case 1. to 4 mmnths in Case 3 and one and one half years in
Case 2. The total cost effects ranged from about 201. in Case 1 to
about 491 in Case 2 and about 401 in Case 3. One area. organization
size and capability. did not appear to adversely affect the time and
cost of the project.

Software

The largest cost effect in all three cases stemmed from software
problems. Cases 1 and 2 had selected or been given software which did
not support the courseware design. Changing to software which did
support the design caused significant changes in cost due to the need
to reprogram the courseware. Case 2 experienced two additional
changes, once to accommodate the changing equipment which had

incompatible operating languages and a final change due to the sudden

113
non-support of the software by the manufacturer. The switch in
operating languages was more extensive (5-6 months) than the final
conversion effort.

Graphics software packages proved incompatible in both Cases 2
and 3. Case 3 experienced the most extensive reprogramming of
graphics in order to match the courseware to the CHI developed.
Hardware

Hardware problems appeared to be directly related to the fact the
equipment was newly released from the prototype state. As with any
new technology, there were reliability problems. In all three cases,
the project manager and/or programmers spent many hours and lost from
days to weeks attempting to resolve the problems by telephone. This
did not always prove successful. On several occasions the equipment
or parts of the equipment such as the XBBBC hard disc. light pens and
disc player had to be shipped back to the manufacturer. In other
situations, the programmers received little or no help from the
manufacturer and had to 'work around the problem. such as in the
redesign of the instruction to avoid the overlap of touch areas,
reconfiguring the equipment to use a faster compiler, or contacting
other users for advice based on experience.

Equipment acquisition costs (Cases 2 & 3) and delays in receiving
equipment by the client (Case 2) caused large overruns in material
costs and added labor to accomplish the work.

Defective discs or incorrectly mastered discs also affected the
cost through delays. travel and remastering costs.

Client Knowledge
The level of client knowledge appeared to play a key role in the

11A

escalation of costs during the project. Client specification of
equipment, software, design features and use of existing materials
often proved unworkable or costly to the contractor. In some
instances equipment and/or software had to be changed to support the
design features specified by the client. In addition. in all cases
where the client directed the contractor to use pre-existing materials
such as audio tapes and video. a cost increase resulted and the
quality of the final product was often limited.

There was evidence to show the client's knowledge grew throughout
the project, causing, a change in their expectations of"what they
wanted the product to do. The changing expectations led to the
changing of the design and the addition of special features which in
turn caused a need to reprogram the courseware and/or created a need
for additional programming to meet the new requirements.

Finally, the client's lack of knowledge and understanding of the
complexity of the medium and requirements to accomplish the task
surfaced in unrealistic timelines. Extensive overtime was required by
contractor professional staff to meet the unrealistic delivery dates.
Contractor Experience gnd Qualifications

Contractor experience and qualifications had a varied effect on
the cost of the project. It appears that the contractor's desire to
acquire experience in IVD often led them to bid lower than they might
have had they had extensive IVD experience. Further. the contractors
prepared bids based on general information and did not always
recognize the need to investigate details. such as evaluation of the
audio tapes to determine what would be required to make use of them.

Contractor inexperience and lack of training in Level. 3 IVD

115

manifested itself in extra costs in two major areas: design and
programming. In each case. design of courseware was too complex for
the software, and programers had to spend extra time in redesign
meetings with designers or change to different languages to meet the
design requirement. Inexperience also caused problems in
programming. In Case 2, an inexperienced programmer worked for
several mmnths creating code which was very inefficient and exceeded
the available computer memory. This code had to be reprogrammed over
a period of weeks. Case 1 had to develop short software templates and
routines for less experienced programmers to use as tools. Further,
the switch from a relatively simple authoring system to an authoring
language increased the programming time since the programmers were not
skilled programmers by profession in Case 1.

Another area which showed evidence of contractor inexperience was
the undue desire to please the client, agreeing to make changes and
willingness to incorporate new requirements without assessing the cost
effects of the requests.

Poorly designed comunication methodology also had a negative
effect on each project. Exact costs could not be discerned. However,
all three cases had to reprogram courseware because of slow
communications. miscommunications. and management decisions which were
not transmitted in a timely manner to the team workers. Case 3 tried
to solve communications problems by using group meetings. However,
with the number of companies involved. large amounts of time were
consumed making simple decisions. and frequent changes were made in
courseware design to make each product look similar to the other

products.

116
Design Phase

The data show that the activities of the Design Phase have the
greatest effect on all other activities and phases. The Design Phase
establishes the instructional flow. instructional treatment. level of
interactivity. visual conventions and the management structure. In
this study, the level of complexity put into the design often
outstripped the abilities of hardware and software selected. This
adversely affected the programing aspect and caused software
changes. In addition, equipment frequently had to be reconfigured to
provide the added memory space required for the code to achieve the
complex design.

Hethods selected for storyboarding and scripting were often
inadequate and did not provide the detail required by the
programmers. In Cases 2 and 3, the instructional designer had to
simplify the design during the programming stage either to save time
so the delivery dates could be met or because the software couldn't
support the feature. Again, in both Cases 2 and 3. the instructional
designer had to spend many unplanned hours with the editor in the
video production and with the programmer to ensure an accurate product
was achieved as well as to facilitate efficient production work.

Team Hember Rolesggnd Fungtiogg

Personnel assigned to the team typically performed multiple
roles. Some team members had a broad range of experience and could
adequately fill the requirements of project. manager. instructional
designer. producer and director; others. however, performed these
mmltiple functions because of the pressure of schedules. Performance

of multiple roles and functions by team members affected the projects

117
both positively and negatively. In. a positive sense. it. fostered
communication between team members and helped ensure that members were
operating with the same understanding of what was to be achieved. In
a negative sense, mistakes were made because of inexperience in
performing the task which required time to redo, such as in the case
of some of the writers, instructional designers and programmers.

The management role was often performed by the instructional
designer. video producer/director. or a programmer. There did not
appear to be evidence to show one type was better than another. what
did affect a project budget was the retention of staff members for the
duration of the project and the communication flow between team
members. The changing of staff, either management or team members,
appears to have had an adverse effect. This was especially evident in
Case 2. Adequate documentation was unavailable and each of the four
managers had a different perspective on the design. purpose and
requirements of the program. This created undue changes, delays and
labor costs.

Overall, the way in which the team members comunicated and
transmitted critical information and decisions appeared to be as much
of a factor as the role they performed. The data showed that team
members did not perform an isolated task in a single phase and then
leave the project. Designers and programmers in particular ‘were

required throughout all phases of the effort.

CHAPTER V
SUHNARY, CONCLUSIONS AND RECONHENDATIONS

This concluding chapter has two major sections: (1) a summary of
the study. and (2) findings, conclusions and recommendations for

future research.

Summary of the Study

Purpose
The purpose of this study ‘was to investigate and develop a

baseline knowledge of problem variables which appear to cause time and
cost overruns in the development of Level 3 interactive videodiscs.
Twelve specific research questions were used to explore the effects of
problems associated with such variables as contractor knowledge,
client knowledge, contractor agency size and capability. team member
functions, project team organization, personnel IVD training and
experience, IVD phase activities, hardware, software. unintended and
unplanned events. unexpected costs, and unexpected time delays on
final cost and time to produce the product.
Hethodology

The case method approach was used in the study to derive the key
problem variables which affect time and cost in Level 3 IVD projects.
From. a potential list of 8 cases, three cases ‘were selected for

inclusion in the study. Each case was a Level 3 IVD project to

118

119
develop training for government or private industry which was either
completed or near cmmpletion and which had incurred a cost overrun.

The cases were projects produced by a large educational
institution. a private firm doing only training analysis and
development work for the Department of Defense. and a composite of
firms doing training and development work for a single manufacturer.
The size of the projects varied greatly. Only Case 2 performed all
phases from analysis through production. Case 1 was predominantly
development and production. while Case 3 covered design through
production.

The time delays extended from about two mmnths on one project to
one and one half years on_another. The cost overruns incurred varied
from about 201 in Case 1. to about 401 in Case 3 and about 451 in
Case 2.

Data from the three Level 3 IVD projects for industry and
government were collected and analyzed. The data collection included
in-person guided interviews with project managers. instructional
designers, information programmers, and video producers. Script
writers or graphics programmers were included in the interview process
where required. Data were also collected by examination of project

documentation and products for each case.

Findings, Conclusions and Recommendations
The conclusions of the study are presented in Table 13 with the

related research questions, findings and recomendations for future
research. The remainder of this section discusses the conclusions and

recommendations for each research question.

120

Awwmm uxmz ecu :o vmacmucoov

 

 

.euevol :ouyeouoolloo ooe oouyeeuoeuyo
leey m>u e-uyoeuue eluy ooe yoeuo

a gone; you eeeeao aeolyoueeev Neoouy
nuoyyeou fine a. eyeueoel ooe eyeuov
uoyo .eyelleyuoym .eyeouueov aeoouu

uoayyeou no veyuouey eeuooeyemlou lo-

.yyouue

nyeeeeoeoo: yoeeeym on e:e_eey uoeou
loyu you ecu-eye one .oouyeouoolloo uo
eeouu yeeuu nuuyoeou .eyeolel leey uo
money euouyuol any oo eyeyuoeoooo yeal

ooe oeueovey
cu eyoo: voooe use Aye: oooc nayemoym
[mu on veyuoeoy eoeuyeuuoolloo yoom

.eoyeyuem
nouyeyuoolloo venueuue yu eeoeuen
yoeuyomlu ee: oouyeuueeuyo leoy uoenoym

.eleyuoym ooey ou yeeu ooe eluy
veove veeoeo .eyelleyuoym uo nnueuoemee

uyuenoym m>u oe uo yeou
ooe eluy oo eye: eooeu
uyemee ooe nououeyy n>u
Neomoeyem vol .louyeNu
uoeuyo leeu uyeaoym eeov

 

 

 

 

 

-_=ul eay aeuuneoee oy eeuvoue yooveou leou yoolmoueneu n>u ee uo oouyeuuoeuyc .uououeyy vol eooeuyomee uo aye; .noe uu .eyoeuue yen: .m
.yuonoyo gueo uo eeeegm
Noyeeee ou veeuoyou eleuen eyeyuya
.eyelleyuoym eye:yuoe
ogy oe—e eye: eyeouueoo Heoouyooyue
uou aloe .eyoowueeo neoouyyoyyeou uo
nyuoemeo on» ou veuyo: eyolleyuoym eye:
-yuoe ooe eyeooooym elem .oouyooou ooe
cozy eyol oelyouyem eyeolel leey ueo:
.leeu m>u eyuy
-uouue oe euel oy yeolol leeu auee no .oeeuu oouyueom leoy Heyeoou ego ovu— .=uueeo meoouyooyue nyoenoym m>u me use
voyuavey euuuue eyueomleo emu eoulyeuev -uoo eoouyyoou lyeuyem ou veyuooey eu -ou yo oouyoovoym oevue .moulleyuoym no—nyyeo uo eeeooym egy
oy uselmoaeeeo emu you eyeeleyuooey yenlel leey seem .ooyeeouueo nuyeeuo ee :uoe eoouyooou yesyo any uo ooe cu on lyouyem yenlel leeu
Huuue ooe .-ueuyoe eoouyee one nvoum uoo eu yomlel leey aoee uo euoy as» oouuuue eu nun-:eo yeueoel uuenoym can noee eeov eoouuyoou yea: .e
.yoeuuo emu ou uosuoym you uo .uueye "nele e
ueoo nueyeeo ea» oo yoyoeyyooo one no oy oeyuouey yo venueu neny eouyu nuoenoym m>u
nouyoeyuoooeoe uo eyyeuue emu oo nvoye -uunemeu evueoym oy yyouuo emu uo eoouy oe uo yeoy ooe eluy me
e uooeoou .yoeeeney naneouyeuyeye .eluy ooe eueeo uyoo eloe yoeyyooonoe on we: eyouueyy eye: nyuuuoeoeo ooe eeue
ea ou enmlee gueooe euyeu e eeoyye uueyuo so yoeuue ye one: oy yeomme -ooo ueo: .NHele oy euyeu loyu veuyee neoeue youueyuooo om
yoyoeu nyuuuaemeu ooe eeue emu nmoum yo: oo nyuuuaemeo voe enue nuoeu< eeuuuNunemeu ooe euue nyoone yoyoeyuoou .noe uu .eyueuue yea: .n
.ouueeo oeyueoo
emu uyommoe yo: vuo yueyyooo egg
.mnu ysome euco~:oou uo een-ye nnyee nouyov yo oy youym eye:
eyeooeoeou uo eeoeoen ouueuueoyoo -oyea yoxooe eye:yuoe vouuuoeme yoeuau nyyenoym
on nel eoouyeyuuuoeme voyeoouueo uoeuuu m>~ oe uo ueoo toe eluu
.eeuoego oo uueuue oe eye: .you
.eoel ea :eo eyoeleeyue Neeyoeyyooy .uoulleyuoymey ooe ouueeooy eeueoeuee leyuoym you ueeouey emu ou mew noun: no yaooy emu na veuyomey
uuue_ueey yeay oe nouuoeyyooo eyouea ooe ey ooe veeeeyoou eaveu:ooy yueAy eoouyeyoemee uoouuu oemoegu ouch .yuen ee .euoe~:eou emu yoeuuu
uoeuuy emu e euove eu :oa uo nooum ee oeyyoyoo eyeeoaey euoego uoeusu -oym ezu nouyoe :eyu eave~:oou uoeuuu eeov yoeyye yes: oh .N
.veyyouuu eye: eooyye-o nuoefloym m>u so you eeyel
.eeeam yoenoym yo uoenoym e noun-ea ooe .uouyeoo e eyoyoe you uoeuuuuuoe yeoo .veeo ea oy eaeuyeyel nouueuye -uuee yeoy yo lluy eyel
mouyueyyoou ou youym yoeunu emu loyu yo: ee: oo—yelyouou ezy uo nueyooue ooe eunenueee uueye .eye:oye£ .eye:yuoe ou eaneuueee on ou one:
oouyelyouou meyuouey nouyyeyu-e you see aymev oak .yoeyyoo on oy eyeemme .yoeyooy emu yooae oouyelyouou ueye u. eooo oo:: one veveeo
eyoveuoym yo uevel e uo ueelmoueeev emu eunenueee oouyelyouou uo omnu oak loyu nyee oo veeen eye: eeyeluuee ueou eu euceu:0ou uo emny yea: .u

 

 

soyeeeeu eyouon yon

 

 

 

euluvmun

 

Boa-...fl

 

MCOMUWUCUEEOUUM Ufim mGOMmDHU=OU

.mwamesma

 

.mfi mqm<fi

121

Amwmm yxwz co oysmmymoov

 

.eye:yuoe
oeeuu ayu: eyeool oyueeo mega—yooyyeou
a>u oleuu uo_y=yeee you may—muey

yeoo ooe eluy eay nooye .euolee
eyeyyooyome nyueouyeuyeye e youea

.eyoeleyuouey

ouueeo yeoo_yoayyeou emu uo euoey

e oy eyeyey neay ee eoouyeyouuuooo
.eye:vyea. eye:yuoe eeuyueuue nu—yeeou

.mhu you eye:yuoe uo eeyoyeeu
ouueeo eyeuymoyome yeol eay eoylyeyem

.oyyeev leyyoym eyeoellouye

ey oeuoeao ea oy eea eye:yuoe eay

oea: uoylleyuoym you veyyoyou eye eooy
-yeeo yeoo ooe eluy eay “soyyeyo yeoo
ooe eluy e e eeyo you eeov eye:yuoe oak

eye:yuoe eouauel ayu:
euayyemlooou eye: eeyeyoeo euyameyu

ooe eloeyealoy ooy ee: eye:yuoe can
oeyeeoaey yoouuo eay eeyoyeeu uo
woulleyyoym :oyue yoo onoc: eye:yuom
yoooee eye:yuoe

eay na eye:yuoe eay uo yyommse oo-
eye:yuoe :eo

you vole veemeo eluaeau nyeoyaue:
"vevouoou eyoeao you eoeu
.uo_lleyuoymey eyeymlou yeolue uoyeoeo
yueaoym lay nouyso eou:y yo eooe
oeeoeao ee: eye:yuoe eay .eeeo auee on

nyuenoym

m>u me uo yeoo coo lluy
mo eyea eye:yuoe m>u eeoo
.noe uu .eyoeuuo yea:

 

 

eye:oyea oy eoouyeooelloyey youyel ue
eeeomyom you eye:oyea gnu you eeyoyeeu
ou—eeo yoeyyomlu yeol eay nu—yoeou

.ayo: yeelmeyeeev

a>u eyoyou oo yoeuue eay eeyloae
layyyouae oouyoyveym e moueeeo ooe
nuonooaoey :eo uo eyoeuyele eay eoyleyn

.leuaoyo eay eyuoeey

oy mHea uo eyyooe nuoo lay on any:
yeyeyoeuooel eay .yuenoym e ou oeeo eye
eeouaoel oerNeeeo numeo uu .eye:oyea
ayu: eleuaoym eooyeloo eye: eyeah

 

 

.yuonoyo eay oy yoyym yeofi
n eeeu ou ooe yuenoym eay youyov N eeeu
o. oeuoeuyeoye eye: eeuoeao eouaoe:

.eoou leyuoym
oeyuouey Nue eyeoolloyoe oy yoenoym
aoee ou uuele ooy ee: nyolel loyaye:

.eleyaoym eeyoeey oy eneo ye e
aooy eo also a. ooe .veyuoeey eye:
eyeyoyueuooel ayu: eoouyoeyeyo— eeouu

 

 

.oeeyeeey nyymu you
eye: ayua: eleuaoyo ayu: oeyvoye e eey
aye ou oeyeeyyeo eye: eeouaoel :e-

nyoenoym

mp— oe uo yeoo ooe eluy
oo eeea eye:eyea m>u eeoo
.noe uu .eyoeuue yea:

 

.eyoeuue
eluy ole yeoy eoulyeyeo oy veyvaye
ea eeyoyeeu euyeeo ueeoyyueyyeeu yeah

.oeyoeye
ea eyeool on eeo ueulleyuoym yeah

.yeou ooe elyy you oeyvsye ole memo
uueyeo ea nyyyyyoeyeyoy uo eueool yeah

mouyov 1«.«11|<« eye
uoylleyyoym ooe ooyyoovoym oevuy eay
ayoa eyoue .yoAlH uo elyey ou yyenoym
uo eeeam nuyeoo yeol eay ey eeeam
louyooooym eay .yeee:o= .veyuouey eluy
soulllyuoym oy eyluey nuyyey—v eeyayeeu
ooe nyueeumloo leyuoym eaH .veouuev
eye eeyoyeeu ooe leyuoyo eay uo
nyueeumloo eay yeay ouyeeo eay ou ey yu
eoeue yeoo ooe eluy oo yoeuue yeeyeeyu
eay eeea oy eyeeooe eeean ouueeo eah

 

.eeeam ooyyyovoym lay my noulleyueym
you veyuouey yeoo ooe el—y uo yooole
eay oeyyeuue eeeam oyueev lay you
'yov eooo o-ueeo eay uo nyyeeumleo eaH

.eeuooeaye eluy yell oy
eeeeam ooyyooooym eay uoyyov oeuuyvol
ea oy oea ye yel ea yoo oyooo eeyuo
.eeeam ouueeo eay youyoo .ouyeeo Neoouy
vyoyyeou eay ou eoel eoouyeoyuuoeom

 

uyyenoym
a>u ee uo yeoy ooe eluy
yoeuue oy yeayo nee oeay
nneyyu eyol eeeam eye on

 

.nyueuyueyeylu
yo .yeooe y eyu_e eyy yo .oyyeoyyyee
can He Imuvlum umoo val '—u Heavy—00

 

.yoyoleaonem naauua oy enuy
-.ouoy nuauya loyu lo—yyeyyeou uo euoey
eay you eyoeleyuaoey eeyyoeyeyoy eay
aeuuaeyee onoc: aoua: eyeool ooyeeen

 

.eeeeam yoeoueeaoe ou oeyusoey elyy ooe
yeoo eAy oo yeuyoeouuou yeol eay eye
yyenoym may uo eeuyuyuyye ouueeo ear

 

 

 

aoye l eyoyon yon

 

eouyoo ‘ ea oy
vea leyuoym eay ooe yeeooye eay oy you
-yoeyyeuo eleyea ny—yuyoeyeyoy aool ooh

.yuenoyo eay you eeeeau
eye:yuoe eay na oeaeuymloyoe ea yoo
vasoo ouueea eay eeyoeyeou ueyeeee on

.yeeoao

yo euooeaoe ouayu: volley-eye ea

you ouooo ouyeeo eye:eey=oo eeyyoeyeyou
eay uo eloe yeay vooou eyelleyuoym

uyoeneym emu ee uo e eeam
yoeoueeaae lo yoeuue yeoo
vol eluy e e ea eam
e a. eeuyueuyoe yea:

 

 

yum—voun

 

.4 I auye on

Av.ymoov

.MH uAm<H

122

 

 

.cu val .0 .0 mac—ymlmo
ayyeeeeu you ole—yeooellooel ee elem

.yoyoeu
neueo eyaeooeeey e eyeoollouoe oy
oeyyeyyeooo ea oyooae eeyooeaoe yoenoyn
.yoenoym emu n nope; eeuoloo e ou
oeyleuyemee ea uyu: eneueo veyuemeeoa

.oeueev ooe oeloeao yoeunu
eye:oyea yo
eye:yuoe oy eeyoeau coy-unyeou yoeuuu

.oeyyooey
ee: soulleyuoymey oea: eneueo oeeoeo
eeuyuoouuuuv noylleyuoym eye:yuom

eneueo
yuenoym oeeoeo eooyyyoouyel eye:oye=

28?... o:
lo a. ylooo enlglo lluy
oeyyemeeol oo eeooeyeloy
-yuo yea: yeooo ooe oo:: .Nu

 

.ou one .o .a eeouyeeoo
aoyeeeeu you eoouyevoelloyeu ee elem

.eyeoo oeyoemeeoo uyele nueeyyeyey
eyeoelloooe oy yuyoa ea oyooae eyeuoon

.oeuoooyyou eye
eleyene eye:yuoe ooe eye:oyea voyage-ea
n—meo yea: yoooo eyeoo oeyoeoeeom

.oeyeeooa oeay yeyeyy eyol yooy eoyvoye
oeoue ooe ouooe ye ooueueyeooe eeo—u

.eyeou nouomuae oeyoemee oeay yea-u:

.oeyeyooa oeay eyooa
uo_lleyuoym eyol aoey eye:yuoe :e-

.oeeyoeey eleuaoym yen
oy eyeoo ooe en eo veeoeo eleyene :e-

 

.uo.lleyuoym Neouu oy yeyym
yean yoeyooo uo ooey oeeaeo eyyemee
yeyyel yoenase uo emooyu yleyeuuum

.yuenoym eay uo yeoo veeuey
aoya: oeyyeoou eye: eeyoeao eye:yuom

uyyenoym

emu no l— yooyo eyeoo

veyyemeeo: om eeyoeyelou
-yuu yea: yeoco ooe yea: .yu

 

.elayyyoune
eo—neluy eu oevoyoou ea oy oyeooeye
e ae—uaeyee oey ayua: eyeeee veneeno

 

eyeuymoymme nuueyuyeuyeye e nouem

.lAy—youue yeyyeoo e

my oeoouoey ea eey aqua: eouee yeyyeoo
yuye—yeey e eeuyeo oy eyoeee ooe:-um
-oo ooe oeooeyouoo oy eoo oeyysoou
yeoo uo eeeyoeoyem eay nooye .eumlee
eyymoymme nuyeouyeuyeye e youea

 

.eeyoeyeou eeeay eyeool
ulooue oy oooc-yo ea vuooae ee—eoeaye
ooe eyeuoon .yeoo ooe elyy oo yyeuue

eeuyeeeo e eeea auya: eye:yuoe ooe
eye:oyea oy oeyeyey eyoeee oeooeyooo
ooe oeooeyoyoo uo yealoo e eye eyeah

.eoooey ea oy oea .oeyeyeel nayeeyyoo
yoo ee: ouooe eleyu uuuym .eleyene
m>u uo yealoo oeyulum e eye: eyeah

.m—ea oeyuemee e.yoeuao uo oeuu
ou aye: eyol oo oy yeyyeyyooe oeeoeo
eleyene e.yoeuyy uo nyeeyyev eyea

.eeeoeoeu eo—yeyemo
cu euoeay eyeoollouye oy soulleyuoymey
veeoey yoeuuy na yelloyooe o. euoeau

.eye:yuoe
e— euoeao e oeeoeo ooe yeyoyyeuooel
eay na oeomoyo ee: yyommse eye:yuom

.veyeumey
ea oy oea ooe .eoyaoel eay ou oeueleo
eye: ooe oeyeyemee eyeneu yeuvoev—m

.veoeumey
ea oy oea ooe oono—yooouuel eleyenm

 

 

ayyeeeed eyoyon you

c 1 I

 

e v I

 

DIE—VB“-

 

nyoenoym m>u me
a. loyyeeo yeoy ooe elyy

 

no—om ov yleyel yea: oh .0”
eoouyee c aoyeeeem

Ao.ucoov .mH money

123

Research Questiopg

Research Question 1: ‘Hhat type of knowledge is needed and when does
it have to be available to make time or cost
estimates for an IVD project?

As stated in Table 13. the data in all three cases clearly showed
that information about the instructional content, software, hardware.
staff and existing materials was generally made available before
contracting and was used by contractors in the development of project
cost estimates. However, once the contract was underway. added
information about these same aspects became apparent and could be tied
directly to time delays and added costs. From the data one can
conclude that the type of information being made available was
appropriate. However, the depth and accuracy of the information was
insufficient to make accurate cost estimates at the time of
contracting. Further. as shown in Research Question 2. the client
knowledge about IVD was often insufficient for the client to know when
they were providing incomplete and inaccurate information. Therefore,
it is recomended that research be conducted to develop models or
procedures for extracting required information from the client prior
to contracting and costing a project or project phase. To develop a
model would require the identification of initial elements in each of
the aspects of instructional content, software, hardware, staff and

existing materials to be used. which have a time and cost affect on

the project.

12s
Research Question 2: To what extent does client IVD knowledge. as

reported by the contractor. have an effect on
time and cost of an IVD project?

Essentially, there were two effects of client knowledge as
observed by contractors. First. the clients knowledge about IVD
tended to increase during the project and their expectations of how
the product should look and operate changed accordingly in all three
cases. Each client added or changed previously accepted design
features during the project. Secondly. each client specified
hardware, software and/or design features which did not support the
desired design. The software was either too inflexible or required
too much time to achieve the effect. The hardware lacked required
features, critical memory space, or operated too slowly to be
efficient instruction. The design features specified proved
unworkable or became distracting to the instructional process.

The client change requests occurred as knowledge increased which
in turn caused extensive redesign and reprograming efforts on each
contract. Further. the client-delineated specifications may have been
unrealistic because of the client's lack of critical knowledge about
the capabilities of Level 3 IVD and the processes and procedures
required to develop the product. These findings are similar to the
findings by Klotz (1983) in a survey of problems related to IVD.

To ensure that expectations remain realistic and that client
specifications are executable. studies on how to educate the client
before contracting so that realistic contractual agreements can be
made appears critical. Focusing on present client self-education
methods which changed expectations and on modes of discerning the

client‘s knowledge level about IVD would provide valuable information

125

toward establishing how to educate and the amount of education to
provide to the client before contracting. Ultimately. client
education would minimize false assumptions and provide the client with
information to use in specifying their requirements to contractors.
Until such studies are completed, some contractual contingencies are
required to minimize the impact of these client changes on contractors.
Research Question 3: ‘Uhat effects. if any does contractor agency

size and capability have on time and cost of an

IVD project?

As stated in Table 13. contractor agency size and capabilities
varied from large to small. Contractors appeared to use
subcontractors, consultants and specialized free lance personnel. such
as writers, to secure the necessary capabilities to complete the job.
Regardless of the approach used. contractors had identified the
requirements at the time of contracting and often had made the
necessary agreements prior to completing the timeline and cost
estimate for the project. From the data, one can conclude that agency
size and capability did not have an effect on time and cost estimates
of a Level 3 IVD project.

It is recomended that this conclusion regarding the effect of
agency size and capabilities on IVD development time» and cost be
verified by conducting a study across a large enough sample to be
statistically relevant. An added study could be conducted to discern
whether IVD projects cost more or less in relationship to the amount

of subcontracting activity. This information would assist clients in

selecting the most cost effective and efficient contractor.

126

Research Question A: What functions does each team member perform in
the process of carrying out an IVD project?

The data collected showed team members performing in more than
one role. In each case studied, the project manager performed in one
or more of the team positions of instructional designer. programmer
and video producer. Personnel with instructional design training most
often serve as the project manager. Other team members also performed
multiple duties or activities typically defined for another position,
e.g. programmer. It was not unusual to have instructional designers
and writers doing programming. A few programmers did do some
instructional design. Thus. it can be concluded that the individual
roles of team members were not clearly delineated. Each temm member
could very likely be required to perform functions which they are not
necessarily skilled or trained to perform. This undefined role may be
a direct factor of the complexity and interrelationship of the medium
requiring individuals to possess a composite of skills.

Research is required that focuses on the various activities and
skill requirements for IVD development which would determine the
composite skills required by each team member to make an effective IVD
team. This research is necessary to bridge the gap between the highly
specialized person and the more broadly skilled person whose knowledge
and expertise can be wedded to other critical areas involved in IVD

development.

Research Question 5: "but effects. if any, do project team
organization and personnel IVD training and
experience have on time and cost of an. IVD
project?

As stated in Table 13. each case showed effects traceable to the

inadequate experience and/or training in IVD development. The

127

seriousness of the problem was most evident among programmers. While
all programmers had formal training, many did not have enough
experience programming with the selected language and developing
efficient program code. This experience was critical in helping
programers to stay within the boundaries of the machine memory and
maximize the operational speed of the final program. The overall team
organization affected the communication patterns and medes of
communication. Absentee and unavailable key personnel. communication
by phone and/or written notices tended to result in missed
information, misinterpreted information and poorly executed work which
had to be redone. This problem is similar to the findings by Klotz
(1983). It can be concluded that the organization of an IVD
development team mmst focus on the multiple as well as diverse roles
of team members, establish clear lines of communication, and provide
for frequent product reviews to prevent unnecessary effort.

Research is required that would establish the minimum
comeetencies required by instructional designers. programmers,
producers and managers in all instructional development phases of a
Level 3 IVD project. The establishment of minimum skill requirements
would help minimize the effects of inexperienced or untrained team
personnel. Further, a study of the skill requirements for IVD
development would help derive cost efficient and time effective IVD
team organization and communication models. Such models are required
to facilitate the interaction of diverse staff, such as instructional

designers and programmers, and ensure the transmission of accurate and

complete information.

128

Research Question 6: What activities in a phase have a time and cost
effect on subsequent phases of an IVD project?

As reported in Table 13, the case data indicated that the single
activity which generated subsequent time and cost problems is related
to the design phase. Programmers had to request that designs be
modified in order to accomplish them within schedule and/or budget.
In some instances. the design could not be executed with the available
software. Finally, some designs proved ineffective and counter to
good instruction when executed. Either the content had to be
redesigned. or software had to be changed to support the design. In
any event. delays occurred and added costs were incurred to rectify
the deficiencies. Klotz (1983) also found that the more sophisticated
the level of design the more likely problems would occur during
programming. Thus. it can be concluded that the design developed for
the IVD appears to control the time and cost in the development. and
production phases.

Further research is required to develop models which would
establish the interactivity requirements for the range of instruction
from highly cognitive to highly psychomotor. The theoretic literature
to date has not focused on specific design models for IVD. Typically.
models applied are drawn for print. linear mediums or CBI which do not
adequately address the capabilities and features of IVD. Time and
cost studies on the application of the different models of
interactivity are essential once medals are developed. Such studies
would yield more precise costing parameters and would prove useful in

establishing budgets.

129

Research Question 7: Is one phase more likely than any other to
affect time and cost of an IVD project?

As discussed in the previous research question. the activities in
the design phase appeared to have had the greatest effect on
subsequent activities. Essentially. the instructional design either
had to be redone or modified to fit within existing time schedules.
It also was evident that the more complex the design, the more time
and cost increased during production. The speed of executing a
complex design was also related to the flexibility and ease of using
the selected software. Clearly. from the data one can conclude that
the design phase has the greatest effect on time and cost. However,
the costs are incurred in executing the design. This makes the
production phase the most costly phase of a project. The programing
aspect of the production phase was the most difficult time and cost to
discern and/or estimate.

The resulting recomendations are somewhat like those stated for
Question 6. Research is required to develop and study models of
interactivity for time and cost. Related to this research is the
study of design features to determine cost and time effects. Hany of
the treatments, such as light pen. touch points, and instructional
strategy. require extensive time to program. It would be useful when
costing to have some clear estimates of programing costs for these
and similar features. Further. this study showed that the efficiency
of the programer code affected time and cost. Studies need to be
made of programer design models to identify efficient models for
programing IVD. These studies could extend to identifying links

between the instructional design and the computer program design.

130

Research Question 8: what effects. if any, does IVD hardware have on
time and cost of an IVD project?

Table 13 summarizes the problems linked to hardware. These
problems emanated from three separate issues. First. newly developed
machines contained problems which could only be resolved in
consultation with or by the manufacturer. Secondly, machine memory
space was too small to accommodate the required program code in each
case. Finally, some cases experienced equipment changes during the
project which caused extensive reprogramming and extensive time
delays. It is comon for any type of technology to have problems.
Further. it is common to expect newly released hardware to require a
debugging phase. with the technology changing rapidly, one can
conclude that there will continue to be numerous problems with
hardware and that resolution of the problems will require interaction
with the manufacturer. Therefore, it is important that research be
conducted which will examine the emerging new technology and develop a
prediction algorithm showing the effect on future IVD development. In
addition, research studies conducted which would identify the most
important design features for IVD hardware for purposes of making
recomendations to hardware manufacturers would be useful. To date
the technology is an eclectic gathering of diverse pieces forced into
service to produce and play instructional IVDs. While some
manufacturers have begun to treat the technology as a single unit,
input is required which would make the medium mere usable and perhaps

more cost efficient for instruction and training requirements.

131

Research Question 9: that effects, if any, does IVD software have on
time and cost of an IVD project?

Significant time delays and cost increases were evident in each
project during the programing stage. The time and cost increases
directly relate to the changes made in the software packages used for
the project. Each case changed languages at least once. One case
changed languages three times. Graphics packages were changed in all
cases. As reported in Table 13. the reasons for the software changes
were external to the programer or the software itself. The
difficulties occurred when the software was incompatible with the
design, too cumbersome for accomodation in the available memory.
incompatible with the machine operating language, and lacked
manufacturer support. The actual functioning of the software did not
create time and cost overruns. Rather, time and cost overruns were
incurred for programing when a complete software had to be changed or
when the software selected was inappropriate for the job. Existing
programed material could not be converted in most cases and had to be
reprogramed.

Clearly. research is required to help identify the most
appropriate design features of software for IVD development. To date,
most of the software being used for IVD was created for some other
purpose. As a result, critical features for programing instruction
on IVD are either lacking or are too cumbersome to be used.
Additional research to identify effective software (hardware)
configurations as they relate to a range of instructional design
features would be useful to those selecting software. Finally. once
IVD design models are developed. time studies using a variety of the

software would provide valuable data for people in a position to

132

select efficient, as well as effective software.

Research Question 10: To what extent do unintended events and
unplanned events create a time and cost overrun
on an IVD project?

Unintended and unplanned events were occurrences outside of the
normal instructional design process which had a time and cost effect.
These time and cost items were out of the control of the project
manager or personnel. It is significant that each project experienced
some unplanned/unintended events. Problems comon to all projects
were system malfunctioning and the limited availability of systems for
the project. Other problems related to disc separations. changes in
equipment by the client, manufacturer non-support of software and
delayed delivery of production equipment. Hanufacturer delays in
equipment delivery was also identified by Klotz (1983). These
unplanned/unintended events occurred frequently enough and caused
large enough time delays and cost effects to be significant.
Unplanned/unintended events related to hardware and software ‘will
continue to occur. Budgets and schedules should be planned to
accomodate these events. The difficulty lies in predicting a time
and cost percentage since the problems vary and so will the effects on
time and cost vary.

Studies of the percentage of time and percentage of cost created
on projects by unplanned/unintended events would be useful data for
establishing realistic values to insert into time and cost algorithms
used for time and cost budgeting. By using a statistically
appropriate sample, one could also establish some prediction levels of

the likelihood of certain types of problems occurring with

corresponding time and cost percentages.

133

Research Question 11: when and under what circumstances do
unexpected costs occur on an IVD project?

A series of unrelated, unexpected costs were encountered on each
project. These costs were due to software changes, redo of invalid
content. new systems replacing old systems, unexpectedly long
programing times. shipping costs and added travel. Like unplanned
events, unexpected costs will always occur on a project. Again, the
type of problem and the effect on time and dollars is unpredictable.
Data are required to provide percentages of costs which occur due to
unexpected problems. Similar studies as outlined in Research
Questions 8, 9. and 10 would provide the required data to develop more
predictive cost percentages.

Research Question 12: when and under what circumstances do
unexpected time’delays occur in IVD projects?

Time delays occurred across all projects. Hany of the delays
were attributable to the client's change of hardware, courseware
design and/or software. Time was consistently affected by changing of
graphics packages which generated a need to reprogram. Further, each
project experienced time delays because of equipment malfunctioning.
Delay varied from (one day to several weeks. The more scarce the
equipment. the greater the project delays. Technology. particularly
new and emerging technology, will continue to increase time on Level 3
IVD projects. It is important to conduct research which would help
delineate the percentage of time. Such data would be valuable for
establishing realistic timelines. The conduct of research similar to
that listed in Research Questions 8. 9, and 10 would provide the data

necessary to establish time guidelines.

134

General Recommendations

After carefully reviewing the data and results of this study it
appears valuable data could be obtained by conducting another study,
using the case methodology. of Level 3 IVD projects with a fixed price
contract which did not incur a time or cost overrun. A comparison
with the data of this study would help to identify differences between
projects incurring cost overruns and those completing work within
budget. Further, if similar problems do occur, valuable data could be
collected on how the problem was handled to avoid the negative time

and/or cost effects.

BIBLIOGRAPHY

13S

BIBLIOGRAPHY

Active Learning Systems. (1985). Final report: Interactive cablg_
television. (Contract No. 400-B4-OO6). Washington. D.C.: National
Institute of Education. (ERIC ‘Document Reproduction. Service No.
ED257451).

Allen, 8.8. (1986). Interactivating linear video. Journal of
gpgpgter-Based Instruction. 13(4). 107-112.

Allen, R. J. 5 Allen, S. H. (1983). Computer-assisted videodisc.
Performance and Instruction, 22(9), 28-31.

Allen, B.S. & Erickson, D.H. (1986). Training Interactive Videodisc
Designers. Journal of Instructional Development. 2(2), 19-28.

An interview with Eric Nowlin of Vidmax. (1983). Performance and
InstructionI 22(9). 11-13.

Anderson. S. 3.. Ball. 8., Hurphy. R. T. a Associates. (1975).
Encyclopedia of educational evaluation. San Francisco: Jossey Bass.

Andrews, A. E.. Pistone, R. A., Roman, J. 3.. Stoddard, H. L. (1986).
Design of Level III interactive videodisc training: Exploiting the
power of the technology to enhance learning. Proceedings of theggth
Interservice/ Industry Training Eguipgent Conference, 1, 145-150.

Awotua-Efebo, E. B. (1984). Guidelines for cost-effective training
product development. Educational Technology, 23(3), 42-44.

Bailey, H. H. (1980. November). The computer interactive videodisc.
Filmmakers Honthly, pp. 11-28.

Barker, B. (1985, October). Haintaining and/or expandipgfcurriculug_
offeringLin small and rural—high school_s. Paper presented at the
Technology-Based Curriculum Systems Teleconference. Dallas, TX.

Bejar. I. (1982). Videodisc in education: Integrating the computer
and communications technology. Byte. 1(6). 78-104.

Bennion. J. L. (1985). Authoring interactive videodisc courseware.
In H. DeBloois (Ed.). Creating courseware for
gicrocogputer/videodisc learning systems. Englewood Cliffs, NJ:
Educational Technology.

Bennion. J. L. a Schneider. E. U. (1975). Interactive videodisc
systems for education. Journal_of the Society,of Hotion Picture and

Television Engineers (SHPTE). §g(12), 949-953.

136

Bork, A. (1978-1979). Stand-alone computer systems: Our educational
future. Journal of Educational Technology Systems, 1(3), 201-207.

Bright. L. K. (1983, November). lateragtive videodisc learning

systems: The promise of new access to counseling services. Paper
presented at the meeting of NCACES Creative Leadership and Skill

Development Conference, Chicago, IL.

Branson, R. R. a Forster, R. H. (1979). Educational applications
research and videodisk technology. Journal of Educational

Technology Systems, §(3), 241-262.

Bunderson, C. V. (1983). A survey of videodisc projects at HICAT.
Performance and Instruction. 22(9). 24-25.

Bunderson, C. V. (1979-80). Instructional strategies for videodisc
courseware: The HcGraw Hill disc. Journal of Educational Systems,
§(3), 207-210.

Bunderson, C. V., Hoekema. J., Hon, D.. Uilson, R. L., Uorchester, C.,
a Woodward, H. (1983). Interactive video: An elephant in search of
a definition. Performance and Instruction, aa(9), 4-5.

Burgess, R. G. (Ed.). (1982). Field research: a sourcebook and
field manual. London: George Allen a Unwin.

Burgess, R. G. (Ed.). (1984). The research process ia educational
settiags: Tea_gase studies. New York: Falmer Press.

Butler, D. H. (1981). Five caveats for videodisc in training.
Instructional Innovator. 29(2). 16-18.

Caldwell, R. H. (1981). Computers and curriculum - promises and
problems. Mnology and Educat% Proceedimgs of Ae Nationafil
Conference on Technology and Education. Washington, D.C.:
Institute for Educational Leadership.

Campbell, D. T. (1979). “Degrees of freedom“ and the case study. In
T. D. Cook & C. S. Reichardt. (Eds.). Qualitative and quantitatlve
methods ia__avaluatioaL research. (Vol. 1.) (pp. 49-67). Beverly
Hills: Sage Publications.

Cambre, H. A. (1984). Note on new technology: Interactive video.
Instructional Innovator. 22(6), 24-25.

Caplovitz, D. (1983). The stages of social research. New York: John
Niley a Sons.

Clark. D. J. (1984). How do interactive videodiscs rate against other
media. Instructional Innovator, 22(6). 12-16.

Clement. F. (1981). Oh dad, poor dad. mem's bought the wrong
videodisc and I'm so sad. Instructional Innovator, 29(2), 12-15.

137

Cook, T. D. & Reichardt, C. S. (Eds.). (1979). Qualitative agd
quantitative methods in evaluatioa researca. (Vol. 1). Beverly
Hills: Sage Publications.

Crawford, C. (1982). The Atari tutorial. Part 10: Human
engineering. Byte. 1(6). 302-318.

Currier. R. L. (1983). Interactive videodisc learning systems. High
Technology. 3(11). 51-59.

Daynes. R. (1984). Innovator interview: Rod Daynes - Interactive
Technologies Corporation. Instructional Innovator. g2(6), 30-32.

Daynes, R. (1982). Experimenting with videodisc: Programs for this
emerging medium require new attitudes toward packaging,
organization, and style. Instructional Innovator, 21(2), 24, 25. 44.

Daynes. R. (1982). The videodisc interfacing primer. Byte, 1(6),
48-59.

Daynes, R. a Butler, B. (Eds.) (1984). lae videodisc book: A guide
apd directory. NY: John Wiley & Sons, Inc.

DeBloois, H. L. (Ed.) (1982). Videodisc/Hicrocomputer Coursewagg
Design. Englewood Cliffs, NJ: Educational Publications.

DeChenne, J. A. & Evans, R. (1982). Simulating medical emergencies.
lnstructional Innovator, 21(1), 23.

Dennis, V. E. (1984). High tech training at Arthur Anderson and
Company. Instructional Innovator, 22(3), 6-12.

Etherington, J. (1981). Producing for videodiscs: Strange new
world. E&ITV, l§(3). 35-37.

Filstead. W. J. (1970). Qualitative methodology. Chicago: Harkham
Publishing.

Floyd, 8. (1980). Designing interactive video programs. Training and
Development, lg(12), 73-77.

Foreman, P. B. (1971). The theory of case studies. In B. J. Franklin
5. H. W. Osborne. (Eds.). Research methods: ylssues and insights
(pp. 187-205). Belmont, CA: Wadsworth.

Fuller, R.G. (1987). Doing interactive videodisc projects.

Proceedings of the 1987 Coafereage on Technology in Training and
Education. (pp. 533-541). Arlington, VA: American Defense

Preparedness Association.

Gallup, D. A. (1977). Determining the cost-effectiveness of
Instructional Technology. Educational Technology, ll(2), 34-37.

138

Gayeski, D. H. (1983). Corporate sad instructional video: Design and
production. Englewood Cliffs. New Jersey: Prentice-Hall.

Gayeski, D. H. (1984-85). Interactive video: Integrating design
“levels“ and hardware “levels.“ Journal of Educational Technology
Systems. ll(3), 145-151.

Gayeski, D. H. 5 Williams, D. V. (1985). lateractiveggedia.
Englewood Cliffs. NJ: Prentice Hall.

Gayeski, D. H. &.Williams, D. V. (1984). Interactive video in higher
education. In 0. Zuber-Skerritt (Ed.), Video in higher educagion
(pp. 64-74). New York: Nichols.

Gery, G. (1986). How long will it really take: Estimating CBT
development costs. §(7). 31-37.

Gery, G. (1987). Haking CBT happen. Boston: Weingarten Publications.

Gindele, J. F. & Gindele. J. G. (1984). Interactive videodisc
technology and its implications for education. Technological
Horizons in Education. lg(1). 93-97.

Glaser, B. J. a Strauss, A. L. (1970). Discovery of substantive
theory: A basic strategy underlying qualitative research. In W. J.
Filstead (Ed.). Qualitative Hethodology (pp. 288-304). Chicago:
Harkham.

Haynes. B. (1984). Video production technology in Australian higher
education. In 0. Zuber-Skerritt (Ed.), Video in higher education
(pp. 75-83). New York: Nichols.

Heath, 8. P. a Orlich. D. C. (1977). Determining costs of educational
technology: An exploratory review and analysis. Educational

Technology, ll(2). 26-33.

Hoekema. J. (1983). Interactive videodisc: A new architecture.
Performance and Instruction, ag(9), 6-9.

Holmgren, J. E. at al (1979-1980). Effectiveness of army training
extension course lessons on videodisk. Journal of Educatioaal

Technology Systems. §(3), 263-274.

Hord, E. V. (1984). Guidelines for designing computer-assisted
instruction. Instructional Innovator. ga(1), 19-23.

Hon. D. (1980). The videodisc, microcomputer and the satellite.
Training and Development Journal, la(12), 28-34.

Hon. D. (1981. April). Toward real-world applications: Part II - The
budget. Videodisc News, pp. 4-6.

139

Hon, D. (1983). The promise of interactive video. Performance and
Instruction, 23(9), 21-23.

Interactive video on a shoestring. (1984). Instructional Innovator,
22(6). 29, 40.

Isaac, S. & Williams, H. B. (1979). Handbook ialresearch and
evaluation. San Diego, CA: Edits.

Isailovic, J. (1985). Videodisc sad optical memory systems.
Englewood Cliffs, NJ: Prentice-Hall.

Iuppa, H. V. (1984). A practical guide to interactive video. White
Plains. NY: Knowledge Industry Publications.

Kalba, R. R. (1978, November). Reproduction, distribution, and
utilization costs for programable videodisc technology. Phase I
Task Report subaitted to tae Nebraska Educational Television
Network. Cambridge, HA: Kalba Bowen Associates.

Rearsley, G. (1982). Costs; beaefits and productivity in training
systems. Reading, HA: Addison-Wesley.

Rearsley, G. (1983). Computer-based training: A guide to selection
and implementation. Reading, HA: Addison-Wesley.

Kearsley, G. (1984). Training and technology: A handbook for HRD
professionals. Reading. HA: Addison-Wesley.

Kemph, J. (1981). Videodisk comes to school. Videodisk/Teleteaa,
l(2), 94-97.

Kielt, J. P. a Spitzer. D. R. (1977). Costing educational
technology: Some promising approaches. Educational Tecagglogy,
l1(3), 42-45.

Kirkpatrick, L. D. a Rirkpatrick, D.S. (1985). The physics teachers
and the videodisc. Physics Teacaaa, aa(7), 413-418.

Rlotz, J. T. (1983). A study of the major problems encoaatered in the

productioa; of videodiscs. Unpublished master's thesis. Ithaca
College, Ithaca, NY.

 

Rribs, H. D. (1979-80). Authoring techniques for interactive video
systems. Journal of Educational_Systeaa, E(3), 211-219.

Kulvicki. D.H. (1985, February). lye iateragtive videodisc concepts
workshop. Preconference workshop at the Third Conference on
Interactive Instruction Delivery for The Society for Applied
Learning Technology. Orlando. FL.

LaGow, R. L. (1979-80). Instructional development related to an

intelligent videodisc. Journal of Educational Systems. E(3).
231-239.

140

L'Allie, J. J. (1983). An opportunity to shape a new technology.
Performapce and lnstructioa, ga(9), 3-4.

Lazarsfeld. Paul F. (1972). Qualitative analysis: Historical and
criticallessaya. Boston: Allyn & Bacon.

Leveridge. L. I. (1979-80). Experience in educational design for
interactive videodisk and quadrasync presentations. Jouraal of

Educational Technology Systems, E(3), 221-230.

Lindsey, J. (1984). The challenge of designing for interactive
video. Instructional Innovator, 22(6), 17-19.

Lokker. S. J. (1985). When to use and when not to use video. In C.
Degen (Ed.). Understanding and using video (Pp. l3-23). New York:
Longman.

Hanning, D. T., Balson, P., Ebner, D. G.. a Brooks. F. R. (1983).
Student acceptance of videodisk-based programs for paramedical
training. l,HLE. Journal, ll(3). 105-108.

Haxwell, D. P. (1985). Utilization of interactive videodisc training
in business and industry (Doctoral dissertation, Kansas State
University, 1985). Dissertation Abstracts lateraational, 46, O6-A,
p. 1604.

HcRae, J. L. (1979). Economic analysis of videodisc technology in the
delivery of exported training packages. Proceediags of the
conference on Interactive Videodiscland Hedia Storage Teganology in
Educatiga aag Training. Warrenton, VA: Society for Applied
Learning Technology.

Herrill. P. (1980). Education and training applications of videodisc
technology. In E. Sigel. et al (Eds.), Videodiscs: The technology,
age applgzations ad the future. (pp. 69-101). White Plains, NY:
Knowledge Industry.

Herrill. P. F. & Bennion. J. (1979). Videodisc technology in
education: The current scene. NSPI Journal, lE(9), 18-26.

Hiller, R. L. a Sayers, J. H. (Eds.). (1986). Videodisc aad related
technologies: A glossaay of teraa. Falls Church, VA: Future
Systems. Inc.

Holnar. A. R. (1979-80). Intelligent videodisc and the learning
activity. Journal ongducational Systems. §(1). 31-40.

Parsloe, E., Hoffos. S. a EPIC TEAH (Eds.) (1983). lateractive
video. Wilmslow. Cheshire, U.K.: Sigma Technical Press.

Patton. H. Q. (1980). Qualitative evaluatioalmethods. Beverly
Hills: Sage Publications.

141

Pfannkuch, R. (1974). Characteristics of videodisc systems. Journal
of the SHPTE. §§(7). 585-587.

Pieper. W. J., Richardson. J. J., Harmon, K. R., Keller, R. A. &
Hassey. R. H. (1984). lnteractive graphics simulator: Desigay
developaant. llaad effectiveness/cost evalpatiaa. (Report No.
AFHRL-TR-84-38). Brooks Air Force Base. TX: Air Force Human
Resources Laboratory. (ERIC Document Reproduction Services No. ED
253211).

Platt. W. A. &.HcConville, J. R. (1982). Use of interactive training
devices for military training. Viewpoiats in Teaching and Learning,
§E(3), 105-116. -

Pollack. R. A. a Stillwell. P. (1983). Interactive video directory.
Performance andalpatructiaa.lg;(9). 36-51.

Pressman, I. & Rosenbloom. B. (1983-84). CAI: Its cost and its
role. Journal ongducational Technology Systems. lg(3), 183-208.

Research & Planning. Inc. (1985, February). 9E1 course developaant.
Preconference workshop at the Third conference on Interactive
Instruction Delivery for The Society for Applied Learning
Technology, Orlando, FL.

Schneider, E. W. (1976). Videodisc, or the individualization of
instructional television. Educational Technology, l§(5), 53-58.

Schwartz, E. (1985). Eye eduaators' handbool to lateaaative
videodisc. ‘Washington, DC: Association for Educational

Communications a Technology.

Sigel, E., Schubin. H.. Herrill. P., Kenneth. C.S., Rusche, J. &
Horder, A. (Eds.). (1980). Videodiscs: The technologyI the
applicatioas sad the future. White Plains, NY: Knowledge Industry.

Sheppeck. H. & Cohen. S. L. (1985). Put a dollar value on training.
Training and Development Journal, 22(11),-59-62.

Simcoe. D. E. (1983). Interactive video today. Instructional
Innovator, gE(8). 12-13.

Smith. R.C. (1985, February). Elements of designing an interactive
videodisc program. Preconference workshop of the Third Conference

on Interactive Instruction Delivery for’ The Society for .Applied
Learning Technolosl. Orlando, FL.

Stembler. W.A. a Pond, G. (1987). Costing drivers for interactive
courseware. Proceedings all the 1987 Coaf_ereace on Technology in
Training and Education. (pp. 533-541). Arlington. VA: American
Defense Preparedness Association.

142

Stenhouse. L. (1984). Library access, library use. and user education
in academic sixth forms: an autobiographical account. In R. G.
Burgess (Ed.). The research process ia educationa_l settings: 'Lea
aase studies (PP. 211-233). New York: Falmer Press.

Tennyson. R. D. R Breuer, K. (1984). Cognitive design guidelines for
using video and computer technology in course development. In 0.
Zuber-Skerritt (Ed.). Video in higher educati_oa (pp. 26-63). New
York: Nichols.

Thomas, W. (1981). Interactive Video. Instructional Innovator,
29(2) p 19-20. “e

Thomas. W. (1980). Shoot the works with videobased training.
Training and Developaent Journal, lg(12), 83-86.

Thompson-Hanning, D., Balson. P., Ebner, D. G., R Brooks. F. R.
(1983). Student acceptance of videodisk-based programs for
paramedical training. l.H.E. Journal. __(ll), 105-108.

Walker, C. L. (1979-80). Computer controlled videodisc/videoplayer
system. Journalgof Educational Systeaa, E(3), 201-206.

Walton. J. (1973). Standardized case comparison: Observations on
method in comparative sociology. In H. Armer R A. D. Grimshaw
(Eds.). Comparative social researgz Hetaodological problems and
strategies (pp. 173-191). New York: Wiley and Sons.

Webb, B. (1982). The art of note-taking. In R. G. Burgess (Ed.).
Flald researca: a sourcebookglapd field manual (pp. 195-199).
London: George Allen R Unwin.

Weinstein, S. H. (1983). adoption of interactive videodiscs for
training. Unpublished Hanuscript.

Whyte, W. F. (1982). Interviewing in field research. In R. G.
Burgess (Ed.). Fiald Research: .A sourcebook and field manual (pp.
111-122). London: George Allen and Unwin.

Wicat Systems, Inc. (1982). Interactive videodisc design and
production. Byte. 1(6), 56-59.

Wilson, L. (1983). Interactive video: A step toward natural
learning. Performance and Instruction. ga(9), 32-33.

Wilson, R. (1983). Interactive video: What makes it work?
Perforaance and Instruction. aa(9), 26-27.

Winslow, K. (1981). Videodisc systems: A retrospective. ERITV.
13(3), 38-39.

Woodward, H. (1983). Talents and tolerance: Writing interactive
video. Performance and Instruction, ga(9). 17-20.

143

Worchester, C. (183). Interactive video - A new video. Performance
and Instruction. 22(9), 14-16.

Wright, E.E. (1985, February). Level III videodisc project task
analysis. Preconference workshop at the Third Conference on
Interactive Instruction Delivery for The Society for Applied
Learning Technolosy. Orlando, FL.

Zwaneveld. E. H. (1974). An audiovisual producer/user's view of
videodisc technology. Journal of SHPTE. §§(7). 583-585.

Zuber-Skerritt, 0. (Ed.). (1984). Video in higher educatioa. New
York: Nichols.

APPENDICES

APPENDIX A

PRODUCT DESCRIPTION/COST DECISION WORKSHEETS

144

PRODUCT DESCRIPTION/COB!‘DICIBIOI‘IORKSBBETS

I. CBT.Approacb

 

 

 

 

 

.CDIRIEtIRiltisl .Cnltlz .Intll
Drill a Practice 83.000
Tutorial 4.000
Simulation 7.000
Game 8.000
CBT APPROACH TOTAL I

 

(MAXIHUH I $22,000)

II. Instructional Design

Characteristics Costs Intel

1. Interactivity: (learner interacts with CBT e.g. through
questions. responses or other activities)

 

 

highly interactive $20,000
moderately interactive 14,000
minimally interactive ' 8,000

 

2. Prompta:

 

every question 3.000

 

key questions only 1,000

3. Help Function:

extensive 1.000

 

moderate 500

 

‘n.b. Cost figures not based on actual dollar value, meant to
indicate relative value only.

away Research & Planning. inc. 115 First Street Cambridge, MA (617) 847-5061
IHhtfihuumnmi

145
II. Instructional Design (can't)

shanctcristics nests: metal -
4. ‘ Summaries:

frequent 82.000

 

 

occasional 1.000

5. Learner able to make
comments . 10.000

 

6. Learner able to skip
ahead without answering 4.000

 

7. Feedback:

Feedback presented for wrong

 

 

answer responses 1,000
Feedback presented for correct

answer responses 1.000
Answer judging 3.000

 

Records kept on student
performance 1.000

 

8. Learner Control

1. Learner able to move

 

 

 

 

around within course 3.000
2. Learner to answer correctly

before moving on 500
3. Learner allowed out of

question after 3 tries 500
4. A main menu describing

options to learner 500
5. Help menu 500

 

9. Learner adaptation (learners
name incorporated into
feedback) 1.000

 

 

msraoc'r 10m. pas 1cm mom .

 

(HAHINDH I 376.500)

mass By Research a Planning, Inc. us are Some: Cambridge. MA (617) 547-5061
.Aflngflbnmlymi

146

Ill.‘6rapbics/8creen Design

 

 

shamnriatics costs: fatal
1. Split screen:

(horizontal or vertical) 1.000
2. Bound:

- frequent - 3.000

- occasional 1.000

 

3. Animation:

 

 

 

 

 

 

 

 

 

- frequent 8,000
- occasional 4.000
4. Pace:
' f83t 5:000
- medium 3.000
- slow 1,000
5. Color:
2 colors 3.000
ea. additional color 1.000
GRAPHICS/SCREEN DESIGN TOTAL -
(MAXIMUM - $29,000 with 2
colors)

ems By Research A Planning. Inc. 313 First Street Cambridge. MA (617) 347-5061
.Aflnchbnmuval

147

IV. technical Issues

 

 

 

 

 

 

 

 

Wins mats: fetal
One disc course (approximately
1 hour of instruct 410.000
Self-booting (program automatically
loads from operat ng system) 15.000
booting-up instructions in
documentation 3.000
mmtiblssithnillmm
IBM PC (or compatible) 5.000
020 Professional 5.000
Apple IIB 3.000
Wang PC 3.000
TECHNICAL ISSUES TOTAL -

 

(MAXIMUM O $44.000)

OlflfiBy Research & Planning, Inc. 313 First Street Cambridge. MA (617) 347-3061
ldhnflmueznmd.

148

BUM! IORR.B!.'1'

mm: m

 

I. CBT Approach 8

II. Instructional Design

 

III. Graphics/Screen Design

 

IV. Technical Issues

 

 

TOTAL FOR ONE-HOUR COURSE $

 

(HAXIHUM-8171.SOO)

FOR EACH ADDITIONAL DISC ADD 40‘ OF TOTAL 8

 

(HOUR OP INSTRUCT ION)

 

GRAND TOTAL 8

 

0(985 By Research 8 Planning, Inc. 213 First Street Cambridge. MA (617) “7°50“
l“hfi¢hueune¢

APPENDIX B

STUDY INTRODUCTION TO CLIENT

149

LEVEL 3 INTERACTIVE VIDEODISC STUDY

This study is being undertaken in an effort to generate baseline
knowledge of the problem variables, processes and interactions in the
analysis, design, development and production of Level 3 Interactive
Videodisc programs for training in industry and government. which
affect the final cost and time to produce the product. Below you will
find further information which will outline your role as a participant
in this study.

OBJECTIVE: To collect empirical facts about Level 3 IVD
development and production time and cost factors which

create COSL overruns.

PARTICIPANTS: Companies who have produced Level 3 IVD programs for
training in industry and/or government on a contract
basis.

PROCESS: This study is built around the in-depth analysis of
projects (cases) which have been completed and which
have incurred a time and/or dollar loss. To collect
the information necessary for the analysis, the
following steps need to be performed:

0 In-person interviews will be conducted with staff
who planned. analyzed, designed, developed and
produced the disc. The roles performed may include
project director. instructional designer, programer
and video producer.

Each interview will take from one to two hours
depending on the complexity of the project and the
activities which generated problems in the process.
The source of all information will be kept
confidential.

o The initial interviews may be followed by telephone
interviews to clarify and expand information which
may either not have been clear from the initial
interviews or overlooked during the interviews.

I wish to thank all participants in advance for your time and
cooperation. I will be happy to share the final findings with all
participants in this study. If you are interested in the findings.
please don't hesitate to contact me at:

Clara Steier

1041 Shaffer Trail
Oviedo, FL 32765
305/365-2603

APPENDIX C

INTERVIEW GUIDE

150
APPENDIX C

INTERVIEW GUIDE

IVD COST AND TIRE FACTOR STUDY

The following interview guide was adhered to only as a checklist.
since the recommended procedure for collecting data for the case study
method was to permit 'the interviewee to control the conversation.
Thus, logical sequence was not necessarily followed. It. was the
responsibility of the interviewer to make sure that all areas of
interest were covered. Accuracy was checked through reviewing audio
tapes of the session. The areas listed below did not occur in all
projects. The study focused on the problem areas in each project.
Provisions were made for follow-up phone calls to clarify data and
collect missed data.

A. INTERVIEW RECORD IDENTIFICATION DATA (These data will be
recorded verbally and in writing on a label on each tape)

Date:

 

Company :

 

Interviewee:

 

Position on Team:

 

Telephone: 5 1

Address:

 

 

B. INTRODUCTION OF NYSELF AND THE STUDY TO INCLUDE (Project
Director or company contact)

1. Elicit cooperation.

2. Provide confidentiality guarantee by presenting of copy of
the introductory letter to the study (This letter may have
preceded in-person interview in many cases).

3. Ask permission to see products and documentation where
appropriate.

151

ELICIT A DESCRIPTION OF THE PROJECT TO SET INTERVIENEE AT EASE
(All participants)

4. Description of client to include type of organization and
client knowledge and experience in interactive video.

5. Size of project.

6. Content type.

7. Equipment type and amount.

8. Degree of project completeness (up to implementation)

9. Obtain an overall view of the interviewee's assessment of the
problematic areas (e.g.. design, scripting. or programming).

10. Determine interviewee terminology and definitions.

GENERAL INFORMATION (Project Manager)

11. Descriptive information about the contracting company for the
development of interactive videodiscs:

b.

Company Name:

 

Company Size (number of people and skill types)

 

 

Company Description:

 

Organization (Generalists or specialists in Departments

 

Experience in interactive videodisc:

(1) Number of projects
(2) Levels (1. 2. 3 or 4)
(3) C1ient(s) (same or different)

 

 

 

If previous disc experience:

(1) Bow similar are completed projects?
(2) what is the team composition for the projects (same
or different)?

152

g. Staffing:
(1) what type of credentials do you require?
(2) How much individual experience do team mmmbers have
in interactive video? or in related areas as CAI?
linear video?

h. Staff Training:

(1) Nhat kind of training was provided to staff working
on the project?

(2) Now and by whom was the training conducted? when?

Professional meetings and workshops
- In-house by an employee or an outside firm

12. How often did you meet with the client and for what purposes?

13. What kind. if any. decisions were made by phone with:

a. the client
b. production houses
c. other

ANALYSIS PHASE (All subjects)

14. How would you define analysis?

15.

16.

17.

18.

19.

20.

Describe your role in the analysis phase?
a. How did you begin the work on this project?
(1) Determine needs
(2) Set goals
(3) Collect and analyze tasks
(4) Select hardware or software or both

What type and number of client meetings were held during and
after completion of the analysis?

Can you describe how you conducted your task analysis?

Who did the analysis? How many people were required to
complete the analysis?

what prior experience did you have doing analysis for
interactive video?

when and how did you select the hardware? software?

153
21. What criteria did you use to select the hardware?

22. When did you contact the mastering facility and for what?

a. Input devices e. Added features
- joysticks - range of color
- mouse - operation
- touch screen - cost
- tablet or bit pad - availability
- voice actuation
- graphic overlays f. Predetermined by client
- movement recognition
- light pen g. Maintenance or repair
- bar code reader
- keyboard

b. User controls
- scan
- stop
- fast
- slow
- freeze

c. Ease of Use
- language
- controls
d. Visual clarity

- resolution
- other

23. Staffing:

a. What people did you have to interact with during this
phase?

(1) Skills and knowledge (designers. develOpers,
producers. programmers. other)

b. Can you describe the interactions between team.members?
(l) Kind
(2) Amount
P. DESIGN PHASE (All subjects)

24. Now would you define the design phase?

25.

26.

27.

28.

29.

30.

31.

32.

154
Describe your role during the design phase?

a. How did you begin your work in this phase?

(1) Select re-usable film. audio, graphics
(2) Establish objectives

(3) Establish performance measures

(4) Deve10p management plan

(5) Develop formats

What type and number of client meetings were held during and
after completion of the design work?

Now many people were involved in doing the design work?

What prior experience did you have in design for interactive
videodisc?

Staffing:

a. What people did you have to interact with during this
phase?

(1) Skills and knowledge (designers. developers,
producers, programmers. other)

b. Can you describe the interactions between team members?

(1) Kind
(2) Amount

Describe the processes and procedures you used to:

Select re-usable film, audio. graphics
Establish objectives

Establish performance measures

Develop management plan

Develop formats

(DO-00"

how was the design affected by the analysis work?

What decisions made in analysis affected the instructional
features used in this design phase?

a. hardware
b. Software
c. Content

DEVELOPNENT PHASE (All subjects)

33.

Now do you define the development phase?

34.

35.

36.

37.

38.

39.

40.

155

Describe your role during the development phase?

How did you conduct the work of the development phase?

a. Outline the content, interactivity and use of still frames
b. Draft script and storyboard

c. Develop display formats

d. Reviews of scripts. storyboards for content, style,
feasibility

e. Design and develop the computer program
f. Conduct formative try-outs

g. Plan effects, animation

b. Other

What type and number of client meetings were held during and
after completion of the development work?

What prior experience did you have for doing development work
in interactive videodisc?

How was the development work affected by the:

a. Analysis phase?
b. Design phase?

Staffing:

a. What people did you have to interact with during this
phase?

(1) Skills and knowledge
b. Can you describe the interactions between team members?

(1) Kind
(2) Amount

How was the development work affected by:
a. Hardware

b. Software
c. Content

156

41. Can you describe the procedures you follow which were
different from. any other instructional medium. development

work?

a. Flowcharting
b. Scripting

c. Storyboarding
d. Other

PRODUCTION PHASE (All subjects)

42. Can you give me your definition of production?

43. Can you describe your role in the production phase?

44. How would you describe your procedures and activities during
production? Typical production activities include:

b.

Preparation of production list, final script
Coding

Obtaining masters of existing materials used (audio,
motion, stills, etc.)

Contracting talent and music if needed

Layout of disc

Planning production

Produce, review/approve video text, frames, animation
Produce and review/approve art, photos, print materials
Shoot location video

Shoot studio video. record, edit

Review and approve audio and video

Preliminary edit

Hake edit master tape (with effects)

Review/approve master tape, print materials

Nave master tape made into disc

Review and approve disc-coded tape

45.

46.

47.

‘8.

49.

50.

51.

157

q. Review returned Check Disc

r. Write computer program

8. Test and debug program

t. Calculate frame numbers from master tape. enter program
u. Enter final frame numbers in program

v. Debug and refine program

What parts of the production were done:

a. In—house?
b. With external sources?

What prior experience did you have in doing interactive
videodisc production? other related experience?
Staffing:

(a) What people did you have to interact with during this
phase?

(1) Skills and knowledge (designers. developers,
programmers. producers, other)

b. Can you describe the interactions between team members?

(1) Kind
(2) Amount

How was the production phase affected by decisions and
activities of the:

a. Analysis phase
b. Design phase
c. Development phase

flow was the production work affected by:
a Hardware

b. Software

c. Content

d. Other

What type and number of client meetings were held during and
after completion of the production work?

Can you describe the product revision process which you used,
if any?

52.

53.

158

How did you control the quality and performance of the
mastering house?

Describe the processes which are unique to interactive
videodisc in the production phase?

BUDGETING (Project Director, ID. Programmer. Video Producer)

54.

55.

56.

57.

58.

59.

What was the original cost bid without overhead and profit
dollars? (If willing to reveal).

What was the final (actual) cost? (If willing to reveal).
If subject is unwilling to reveal actual figures:

a. Can you tell me how much over and under you were in cost
by percentage (i.e.. 51, 101. etc.)?

b. Can you tell me how much over or under you were in cost
by percentage in each phase?

What data were provided by the client contracting for the
interactive videodisc?

a. Can you describe the type of information provided?
b. Sufficient level of information?

c. What interaction was there between contractor and client
before the proposed bid?

d. What parameters and restrictions were placed by the
client in:

(a) Hardware

(b) Content

(c) Time

(d) Instructional features
(e) Software

(f) Other

Who established the project cost? and how did you go about
the process?

a. Entire interactive videodisc team
b. Individual

c. Contracts department

d. Other

What IVD software and equipment did you have in-house when
you decided to bid on the project?

60.

61.

62.

63.

64.

65.

159

How were in-house materials used?

What, if any, types of information (equipment/hardware!
software) were investigated before the bidding or contracting?

How did you select and contract staff for the project?
What types of individual skills and resources were allocated

for acquisition after award of the contract? In what amounts
or numbers?

a. Staff

b. Equipment
c. Software
d. Facilities

What kind of assumptions did you make. if any, when you bid
the project?

Can you describe the modifications you would make to your
costing of future contracts for Level 3 interactive videodisc?

END OF INTERVIEW GUIDE