gnu“

I

p

:5 a
3.3% Von.
. 5%an

.

as V.

< .rh

(A ’-
a
F .
z.
.. .
a.

a
5.1.3:.
am.“

£3,

a,” :32... ~ “if.

 

 

2!. .
(E. r

.v
3.

‘ 1...... I; {a

v4 2| . i

n! 1.. .
...0:.:..~.. Er...
49:13:15.... .
a! {it . u .A
.K::‘ .
vi‘lvt-Hvl‘
V1

THE? S

HIGAN

willlllmlllillll‘ill‘lllll

3 1 93 01770 43

This is to certify that the
dissertation entitled

A RELATIONAL DATABASE APPROACH TO SHOP FLOOR
INFORMATION MANAGEMENT AND PERFORMANCE EVALUATION

presented by

Robert J. Marsh

has been accepted towards fulﬁllment
of the requirements for

Ph.D. Business Administration

 

 

 

 

 

degree in
Major pr essor d ‘
’ ‘i
Date 2. / l 11 q
MSUI'J an Afﬁrmative Action/Equal Opportunity Institution 0-12771

 

LIBRARY
Michigan State
University

 

 

 

 

PLACE IN RETURN BOX to remove this checkout from your record.
TO AVOID FINE retum on or before date due.
MAY BE RECALLED with earlier due date if requested.

 

DATE DUE

DATE DUE

DATE DUE

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

1“ WM“

A RELATIONAL DATABASE APPROACH TO SHOP FLOOR INFORMATION
MANAGEMENT AND PERFORMANCE EVALUATION

By

Robert James Marsh

A DISSERTATION
Submitted to
Michigan State University
in partial fulfillment of the requirements
for the degree of

DOCTOR OF PHILOSOPHY

Department of Marketing and Supply Chain Management

1999

cfl

eff
dif

Shc

ABSTRACT

A RELATIONAL DATABASE APPROACH TO SHOP FLOOR INFORMATION
MANAGEMENT AND PERFORMANCE EVAULATION

By

Robert James Marsh

Current shop floor performance reporting procedures, which stress aggregate and
mean measures, do not provide an adequate assessment of shop ﬂoor performance. Such
existing measures prevent an in depth evaluation of operations and normally do not allow
the identiﬁcation and detailed description of complex shop ﬂoor phenomena. This
research addressed this problem by utilizing non-traditional data modeling and database
management techniques to provide managers with a more descriptive set of shop ﬂoor
performance measures. Various conceptual modeling methods from the Information
Systems ﬁeld were used in this research as was an actual implementation of a relational
database of historical, disaggregated shop ﬂoor operational data. Both traditional
aggregate and non-traditional disaggregate performance measures and information were
derived from the database and presented to information users for evaluation. The
performance of the users was evaluated to determine which type of information was more
effective in the identiﬁcation of underlying shop ﬂoor problems and conditions.

It was found that the conceptual and implemented modeling methods were quite
effective in providing a more descriptive dataset. However, the effectiveness of the
different information types appeared to be dependent on the nature of the problem on the

shop floor.

 

M
62

(kt

bet
the

[Ogi

atL

Ray

Jill a

tth

ACKNOWLEDGMENTS

I would like to acknowledge the efforts of my committee in helping me to
complete this work. I was fortunate to have four outstanding members, two in my ﬁeld
of Operations Management and two in Accounting/Information Systems. My chair, Dr.
Steven Melnyk and another member, Dr. William McCarthy, were instrumental in taking
my initial somewhat fuzzy idea and helping to form it into something researchable. This
included many meetings and many cups of coffee several years ago, particularly with Dr.
McCarthy, while this idea was taking shape. Another member of my committee, Dr.
Gary Ragatz, provided additional guidance in the early phases of this work, helping me to
decide what was meaningful to Operations Management researchers and what was not.
The ﬁnal member of my committee, Dr. Severin Grabski, provided an excellent link
between the Accounting/Information Systems and Operations ﬁelds. He was able to see
the relevance of my tapic to both ﬁelds and was really was the “glue” that brought them
together. My thanks go to all of them.

I would also like to acknowledge the support provided by my current department
at Lake Superior State University and in particular that provided by my Dean, Professor
Ray Adams. Their encouragement throughout this process is greatly appreciated.

Last, but certainly not least, I would like to offer my thanks and love to my wife
Jill and our sons Michael, David and Christopher. Their patience and understanding
throughout this process helped me to have the proper perspective on what is really

important. We all look forward to enjoying our post-dissertation life together.

iii

TABLE OF CONTENTS

TABLE OF FIGURES ............................................................................ viii
1.0 Introduction and Overview ..................................................................... 1
1.1 Introduction .................. ' .............................................................. l
1.2 Problem Statement ........................................................................ 2
1.3 Discussion of Research .................................................................. 3
1.4 Methods and Key Results ................................................................ 5
1.5 Conclusions and Organization ........................................................... 6
1.5 .1 Organization of the Remaining Dissertation ..................................... 6

2.0 Literature Review ............................................................................ 7
2.1 Overview ................................................................................... 7
2.2 Background. ............................................................................... 7
2.3 Information Systems in Manufacturing ................................................ 8
2.3.1 Data Structure, Management and Processing. .' ................................ 10

2.4 Data Management in the Systems/Accounting Field ................................ 11
2.5 Semantic Data Models .................................................................. 13
2.5.1 Manufacturing Information Systems and Semantic Modeling ............... 14

2. 5. 2 Database Modeling in Simulation ............................................... 16

2. 5.3 Semantic Data Modeling Summary ............................................. 18

2.6 Simulated Shop Floor Operational Factors .......... , ................................ 19
2.6.1 Bottleneck Resources ............................................................. 19
2.6.2 Irregular order or job releases ................................................... 21
2.6.3 Due Date Assignment. .............................................. ‘ .............. 23
2.6.4 Different Job Types ............................................................... 25
2.6.5 Dispatch Rules .................................................................... 26
2.6.6 Operational Factors Summary ................................................... 27

2.7 Performance Evaluation of the Shop Floor .......................................... 27
2.7.1 Shop Floor Financial Performance Measurement ............................. 28
2.7.2 Non-Aggregate Data Collection and Reporting ................................ 30

2.8 Summary ................................................................................. 30
3.0 Research Questions and Design Overview ................................................. 36
3 . 1 Chapter Overview ....................................................................... 36
3.2 Research Questions and Hypotheses ................................................. 37
3.2.1 Discussion of Research Questions .............................................. 37
3.2.2 Theoretical Basis for Questions .................................................... 38
3.3 Hypotheses and Propositions ......................................................... '39
3.3.1 Proposition #1 ..................................................................... 39
3.3.2 Proposition #2 ..................................................................... 40

iv

3.3.3 Hypothesis #1 ..................................................................... 41

3.4 Discussion of the Research Methodology ............................................ 42
3.4.1 Development of Useful Performance Measures ............................... 42
3.4.2 Semantic and Simulation Modeling ............................... p .............. 42
3 4 4 Clarity of Information Presented ................................................ 43

3.5 Conclusion ............................................................................... 44

4.0 Research Methodology and Design .......................................................... 45

4.1 Chapter Overview ....................................................................... 45

4.2 Determination of Performance Measures and Information ......................... 46
4.2.1 Initial List .......................................................................... 47
4. 2. 2 ExpertOpinion .................................................................... 47

4.2.2.1 Interview Structure ......................................................... 47

4. 3 Model Building .......................................................................... 48
4.3.1 Levels 1 and 2 ..................................................................... 50
4. 3. 2 Level 3 ............................................................................. 50

4.3.2.1 General Shop Description .................................................. 51
4. 3. 2. 2 Model Validation and Veriﬁcation ........................................ 52
4. 3. 2. 3 Sh0p Floor Conditions ..................................................... 52
4. 3. 2. 4 Conditions Summary ....................................................... 53
4.3.2.5 Base Case Operational Parameters ........................................ 53

4. 3. 3 Data Model Design ............................................................. 54
4. 3. 2. 8 Entity Types ................................................................. 57
4. 3. 4 Level 4 ............................................................................. 58

4.4 Simulation Model Operation and Data Collection (Phase 3) ....................... 62

4.41 DataGeneratron .......... 62

4.4.1. 1 Simulation Operation ....................................................... 64
4. 4. 2 Data Collection and Database Population ...................................... 65
4. 4. 3 Measurement Materialization .................................................... 68
4.4.3.1 ‘ JOBS Entity Measures Materialization ................................... 69
4. 4. 3. 3 Additional Entities and Relationships ..................................... 69
4. 4. 4 Hypothesis and Proposition Evaluation ........................................ 73
4. 4. 5 Phase 3 Outcome .................................................................. 73

4.5 Model and Performance Measurement Evaluation .................................. 74

4.5.1 Lab Session Instrument .......................................................... 74
4. 5. 1. 1 Aggregate Group ............................................................ 75
4.5.1.2 Disaggregate Group ........................................................ 76
4. 5. l. 3 Condition Identiﬁcation .................................................... 76
4.5.1.4 Identiﬁcation Process ....................................................... 77
4.5.1.5 Clarity of Information Presentation and Training ....................... 77

4.5.2 Instrument Validation and Pilot Sessions ...................................... 78

4.5.3 Laboratory Experimental Sessions .............................................. 79
4.5.3.1 Recruitment and Incentitives ............................................... 79
4. 5. 3. 2 Lab Experiment Structure .................................................. 80
4. 5. 3. 3 Lab Sessions’ Experimental Factors ...................................... 81
4. 5.3.4 Lab Experiment Performance Measures .................................. 82

4. 5. 3. 4.1 Gap ......................................................................... .82

4.6 Data Analysis ............................................................................ 83

4.6.1 A Priori Analysis (Phase 4) ...................................................... 83

4.6.2 Post Hoc Analysis ................................................................ 84
4.6.2.1 Distributional Analyses ..................................................... 85
4.6.3 Clarity of Training and Information Presentation ............................. 85
4.6.4 Phase 4 Outcome .................................................................. 86
4.7 Potential err' 'tations of the Research Design ........................................ 86
4.7.1 Signiﬁcance Level (__) ......................................................... 86
4.7.2 External Validity and the Generahzab' ility of Results ......................... 87
4.8 Summary ................................................................................. 89
5.0 Experimental Analysis and Results .......................................................... 92
5 . 1 Chapter Overview ....................................................................... 92
5.2 Determination of Performance Measures ............................................. 93
5.2.1 Expert Selected Measures ........................................................ 93

5 . 3 Development of Semantically Based Simulation and Database Models .......... 95
5.3.1 Expected Results .................................................................. 95
5.3.2 Measurement Derivation Results (PRP- 1) ..................................... 96
5.3.3 Derivation of Experts’ Useful Measmes (PRP-2) ............................ 98

5 .4 Laboratory Evaluation of Performance Measures and Information (Phase 4). . . 99
5.4.1 Sample Description ............................................................... 99
5.4.2 Assumptions of Analysis of Covariance ...................................... 100
5.4.3 Lab Analysis Results ............................................................ 102
5.4. 4 Differences rn Distributions ......................................................... 107
5.4.5 Post Hoc Analysis ............................................................... 108
5.4.5.1 Time ......................................................................... 109
5.4.5.2 Conﬁdence .................................................................. 113
5.4.6 Speciﬁc Measures’ Usefulness per Condition ................................ 115
5.4.7 Clarity of Information Presentation ............................................ 119
5.4.8 Summary of Results ............................................................. 123
5.5 Discussion of Results ................................................................. 124
5.5.1 Development of a Semantically Based Simulation Model ................... 124
5.5.2 Derivation of Measurements and Information ................................ 126
5.5.3 Participants’ Identiﬁcation of Shop Floor Conditions (HM) ............... 126
5.5.3.1 Covariates ................................................................... 129
5.5.4 Participants’ Judgement of Measures’ Usefulness ........................... 130
5.5.5 Clarity of Information Presenwd ............................................... 132
5.6 Possible Confounding Factors ....................................................... 133
5.6.1 Sample Size ....................................................................... 133
5.6.2 Learning Effect ................................................................... 134
5.6.3 Use of Student Participants ..................................................... 134
5.6.4 Fatigue Effects ................................................................... 135
5.6.5 Motivation ........................................................................ 136
5.7 Results Discussion Summary ........................................................ 136
6.0 Conclusions and Future Duec' tions ..................................................... 140
6. 1 Chapter Overview ...................................................................... 140
6.2 Semantic Data Modeling with Simulation Models ................................. 141
6. 3 Condition Identiﬁcation ............................................................... 142

6.4 Usefulness an

65 Futurt Rescar
6.5.1 Manager
6.5.2 Non Stat
6.5.3 Dispatch
65.4 L'scrs'P

6‘6 Sumury .....

mam .........
.tDPBDDl A ..........
APPENDIX B ...........
PPBDIX C ...........

APPBDDI D ...........

6.4 Usefulness and Clarity of Measures ................................................. 143
6.5 Future Research ........................................................................ 144
6.5 1 Managers’ Perception of Measures’ Usefulness ............................. 144
6.5.2 Non Stationary Conditions ..................................................... 146
6.5.3 Dispatch Rules ................................................................... 148

6.5 4 Users’ Process of Interpreting Information ................................... 149

6.6 Summary ................................................................................ 151
REFERENCES .................................................................................... 153
APPENDIX A ...................................................................................... 160
APPENDIX B ...................................................................................... 179
APPENDIX C ...................................................................................... 189
APPENDIX D ...................................................................................... 193

inn 1: Manufacnr
FrrtZ: Database .\
rm 3; ln‘ormatic
Frrt 4: Phases of l
Fart 5: Determina:
ﬁrst 6: Model and
Patti: Database I
Errt 8: Shop Fioo
Egan k Data Modt
Frat 10: Shop Flo
Egrt ll: JOB Gen
1:.th 12: Sarnplc C
Egrt 13: Relations
Fat 14: Sample r
7.5m 15: Data Ger
7‘1th 16: Simulate
Err: 17: Cmrion
F1$3118 Measm]
F433! 19: Data “0
{‘55: mm- Database

. “In: .

TABLE OF FIGURES

Figure 1: Manufacturing Planning and Control Hierarchy ....................................... 4
Figure 2: Database Modeling Literature Summary .............................................. 32
Figtne 3: Information Managrnent and Performance Measurement Literature Summary.. 34
Figure 4: Phases of Proposed Research .......................................................... 46
Figure 5: Determination of Performance Measures and Information (Phase 1) ............. 46
Figure 6: Model and Database Development (Phase 2) ......................................... 48
Figure 7: Database Development Process (adapted ﬁ'om McCarthy, 1979) ................. 50
Figure 8: Shop Floor Resources, Events and Agents .......................................... 54
Figm'e 9: Data Model of Simulated Shop Floor (Primary Keys Noted) ...................... 56
Figure 10: Shop Floor Entity Types .............................................................. 57
Figure 11: JOB Generalization Hierarchy ........................................................ 57
Figure 12: Sample OPERATION Records ....................................................... 59
Figure 13: Relational Database Tables ............................................................ 60
Figure 14: Sample Database Tables ............................................................... 61
Figure 15: Data Generation and Capture (Phase 3) ............................................. 62
Figure 16: Simulated Shop Floor Model with Data Collection Points ........................ 63
Figure 17: Creation of Main Entities’ and Relationships’ Data by Simulation Model ...... 67
Figure 18: Measurement and Entity Summary .................................................. 68
Figure 19: Data Model with Implementation Additions ........................................ 71
Figure 20: Database Tables for Additional and Revised Entities and Relationships ........ 73
Figure 21: Performance Measures Evaluation (Phase 4) ....................................... 74
Figure 22: Information Derived from Aggregate Dataset ....................................... 75
Figure 23: Information Derived from Disaggregate Dataset ................................... 76
Figme 24: Experimental Session Outline ........................................................ 80
Figure 25: Analysis of Covariance Factors and Covariates .................................... 81
Figure 26: Summary of Lab and Field Research Results ...................................... 88
Figure 27: Research Phases Summary ........................................................... 91
Figure 28: Expert Survey Summary .............................................................. 94
Figure 29: Comparison of Conventional (A g) and Materialized (Disagg) Measures ...... 97
Figure 30: Sample Characteristics ............................................................... 100
Figure 31: Analysis Abbreviations ............................................................... 102
Figure 32: Experimental Results- Mean (Standard Deviation) Values ....................... 102
Figure 33: Repeated Measures Gap’ AN COVA ............................................... 104
Figme 34: Gap’ AN COVAs for Shop Floor Conditions ...................................... 105
Figure 35: Kolmogorov-Smirnov Test Results for Condition Identiﬁcation. Maximum
difference (signiﬁcance level) .............................................................. 107
Figure 36: Chi Square Test Coefﬁcients for Correct/Incorrect Distributions .............. 108

ﬁgrt 37: COHﬁdch‘
1:338:wa 3
ﬁgrt 39: Tire AXC
Egrt «IO. Rtpcated I
Egrt 41: Confidenc
Frrt 42: Summary
Egrt 43: Expected 1
5th 44: AVOVA 1

level, ‘ indicate
Err: 45: Significar
Egrt #6: Mapping c
.3.th 47: Expected ;
igrt 48: Interprtta
Frrt ~19: 511mm
Egrt 50: Work Ctr
Fin: 5]: Process T
Fgrt 52: Job Comp
Err 531TSYS am
{fit 54: TARDY (
n'E'ui“'55:'l'AR1)y C
it: 56-. Pmion of
£333 57: Daily Av:

Figure 37: Conﬁdence and Time Results— Mean (Standard Deviation) Values ............. 109

Figure 38: Repeated Measures ANCOVA for Time ........................................... 110
Figure 39: Time AN COVAs for Shop Floor Conditions ...................................... 111
Figure 40: Repeawd Measures AN COVA Results for Conﬁdence .......................... 113
Figure 41: Conﬁdence AN COVAs for Shop Floor Conditions .............................. 114
Figme 42: Summary of Most Highly Ranked Measure for Nature of Responses ......... 118
Figure 43: Expected Results- Clarity of Information Presented .............................. 121
Figure 44: AN OVA Results for Follow Up Questions. Means, F value and (signiﬁcance
level, * indicates p < 0.10) ................................................................. 122
Figure 45: Signiﬁcant (or = 0.10) Covariates ................................................... 129
Figure 46: Mapping of Experimental Design ................................................... 131
Figure 47: Expected and Actual Results Summary ............................................ 139
Figure 48: Interpretation Process Steps ......................................................... 149
Figure 49: Summary of Possible Future Research Topics ................................... 151
Figure 50: Work Center Routing Look up Table ............................................... 162
Figure 51: Process Time Look up Table ........................................................ 163
Figure 52: Job Completion 'Iime ('Iime_out) Query/Calculation in Database .............. 166
Figme 53: TSYS and LATE Calculations within Spreadsheet ............................... 167
Figme 54: TARDY Calculation within Spreadsheet ........................................... 168
Figure 55: TARDY Calculation in Database .................................................... 169
Figure 56: Portion of JOBS-WC Relationship Table .......................................... 170
Figure 57: Daily Average Queue Length Calculation in DAYS-WC ......................... 171
Figure 58: Number of Jobs Processed per Day Calculation in DAYS-WC ................. 172
Figure 59: Usefulness Rankings for Correct Identiﬁcation Using Disaggregate Data- Mean,
(Standard Deviation) and Number Responding .......................................... 173
Figure 60: Usefulness Rankings for Incorrect Identiﬁcation Using Disaggregate Data-
Mean (Standard Deviation) and Number Responding ................................... 174
Figure 61: Usefulness Rankings for Correct Identiﬁcation Using Aggregate Data- Mean
(Standard Deviation) and Number Responding .......................................... 176
Figure 62: Usefulness Rankings for Incorrect Identiﬁcation Using Aggregate Data- Mean
(Standard Deviation) and Number Responding .......................................... 177

1.1 Introduction

1
1

lenient identifying an.

 

Despite the trier.

al what impact that strut
pmhrly rue sith one
ﬁu- computer simulatio

nit orient nature of sh

 

 

 

shhion environment '
Wily 0n the evaluatio:
Emblems, rather than

terriers, has not CXplo

”this Wrong With the st.

1.0 Introduction and Overview

1.1 Introduction

Despite the extensive use of information in manufacturing, very little research has
been done identifying and evaluating the actual structure of information on the shop floor
and what impact that structure may have on managerial decision making. This is
particularly true with one of the most commonly used methods of researching the shop
floor- computer simulation. Such evaluation has not been feasible in the past, due largely
to the current nature of shop floor data and information management techniques within a
simulation environment. Traditionally, simulation-based research has been focused
primarily on the evaluation of the effects of certain decision nrles and operational conditions
and problems, rather than on the actual diagnosis and identiﬁcation of such problems.
Such a priori speciﬁcation of problems and conditions, while providing insights to
researchers, has not explored the issue facing many practitioners, namely how to determine
what is wrong with the shop ﬂoor, or in other words, how to identify the underlying .
problems that are causing poor performance.

In an attempt to establish a more descriptive information framework, one that
allows the identiﬁcation of speciﬁc operational problems, this dissertation proposes
combining elements of traditional Operations Management (OM) research, i.e., a simulated
shop floor, with modeling and analysis techniques of the Information Systems (IS) ﬁeld to
explicitly evaluate manufacturing’s information structure. In doing so, this work is
designed to link the two areas of research and practice in a new and unique way. It is
intended to show, by combining these two areas on both a theoretical and implementation
level, how improved cross functional integration may be achievable.

To aid in establishing a linkage between OM and IS work, this dissertation
proposes using an information system based on a relational database to exploit shop ﬂoor

data While relational databases themselves have been widely used in actual manufacturing

mags, their theoredca
irritating literanrr
use of previoas researc
McCarty, 1982; Colar
denaturing to any n
tubes: modeling, are
data and intonation th

Thus this resea:
Wing but also the '
“PM? 3‘ lht actual I
their the Slmcmrc it
has gained

In addition to e:
its work also meOSC:
333%?!an This is d
ire al'aﬂr'tblc to them
“ﬂaring Shop ﬂoor P
ism“ is 5‘11?qu

u‘; l ' , .

1‘2
PIOlilcm Slater:
Cumin

Shon n

 

settings, their theoretical implementation has not been signiﬁcantly explored in the
manufacturing literature. Moreover, the speciﬁc approach being proposed, which makes
use of previous research in the accounting information systems ﬁeld (McCarthy, 1979;
McCarthy, 1982; Colantoni, Manes and Whinston, 1971), has not been explored in
manufacturing to any meaningful extent. Elements of IS research, such as conceptual
database modeling, are used in this research to develop a method of managing shop ﬂoor
data and information that may result in improved evaluation of the shop ﬂoor.

Thus this research is examining not only the issue of performance measurement and
reporting, but also the way in which information is handled on the shop floor. It looks
explicitly at the actual means by which shop ﬂoor data can be structured and managed and
whether the structure itself has an impact on both the performance measures used and the
insights gained.

In addition to exploring the enhancement of shop floor information management,
this work also proposes to employ actual information users to evaluate the beneﬁts of such
management. This is done with a controlled laboratory experiment in which users/subjects
have available to them diﬁ‘erent types of shop ﬂoor information and are given the task of
evaluating shop ﬂoor performance and identifying problems in its operation. The users’
performance is subsequently evaluated to determine if information type has an effect on
problem identiﬁcation.

1.2 Problem Statement

Current shop ﬂoor performance reporting procedtnes, which stress aggregate and
mean measures, do not provide an adequate assessment of shop ﬂoor performance. Such
existing measures prevent an in depth evaluation of operations and normally do not allow
the identiﬁcation and detailed description of complex shop ﬂoor phenomena. This
deﬁciency is also evidenced in one of the most common methods of shop floor research,

simulation modeling. As a result, it is difﬁth to use simulation to identify and diagnosis

Mammal”
mmhmﬁo
uwmmm

mm level decisi

1.3 Discussion of RC

this March is 1
$0150“ data and its $1
rndaed emironment u
nitration mnagement
rains will be genera!
him types (to be dis
more laboratory 5:
1.3.1 Inportancc of the

It is critical that n
910;: floor. At its essenc

humming Planning

speciﬁc operational conditions that may be causing shop ﬂoor performance to degrade.
That is, actual interactions on the shop floor can go unnoticed. This emphasis on aggregate
and mean point measures also prevents simulation from being widely used as a managerial
or enterprise level decisiOn tool.

1.3 Discussion of Research

This research is investigating the actual structure, management and processing of
shop floor data and its subsequent presentation as potentially useful information. A
simulated environment will be utilized to evalute different methods of shop ﬂoor data and
information management and will act as the vehicle by which operational performance
measures will be generabd. The resulting measures and information, of primarily two
different types (to be discussed later), will then be evaluated by actual information users in
a controlled laboratory setting.
1.3.1 Importance of the Research

It is critical that researchers and practitioners have a thorough understanding of the
shop ﬂoor. At its essence, shop ﬂoor control is the foundation upon which the

Manufacturing Planning and Control Hierarchy is based, as indicated in ﬁgure 1.

 

AGGREGATE!
PRODUCTION
PLANNING

1

MASTER
PRODUCTION
SCHEDULING

1

MATERIALS .
REQUIREMENTS
PLANNING

/ \

SHOP FLOOR
TROL PURCHASING

 

 

 

 

 

 

 

 

 

 

 

 

 

Figure 1: Manufacnning Planning and Control Hierarchy

However, unlike the upper levels of the hierarchy, such as Aggregate Planning or
Master Production Scheduling, shop ﬂoor control is very detailed, requiring information
and decisions on a daily or hourly basis (Stevenson, 1999). Aggregate and mean measures
of capacity and performance are not sufﬁcient at such a detailed level, although shop ﬂoor
managers and researchers have traditionally had to use such measures solely within an OM
environment to evaluate performance (Blackstone, er. al., 1982).

This research is attempting to increase the descriptive power of shop ﬂoor
performance measures by incorporating certain rs methods into what traditionally has been
an OM area. By doing so, such performance measures may provide a level of detail
commensurate with that of the nature of shop ﬂoor operations and allow additional insight
of those operations. In addition, the combination of IS and OM techniques being proposed
provide a substantial extension of previous work towards the development of a new
methodology of information management and evaluation, as will be reviewed in the next
chapter.

This research has elements of interest to various ﬁelds of research and practice.

The simulation model used is based very closely on an actual shop ﬂoor that was studied

M a period of tint

{sot Gupta 4). The '
train to the use of s
retard IS managem
h: 051 frld. allowin,
ripe-tart HtlilOdOlOi
research nukes use of
enti'onment Rather I
research sill allow am
M attempt to deterr.
”ii “iii this drawn 5:
Tim-hm to evaluate

tin organizational B

over a period of time and incorporates features not normally included in simulation models
(see Chapter 4). The development of such a detailed empirically based model acts to lend
validity to the use of simulation models. In addition, this research will utilize a relatively
standard IS management tool, the relational database, in ways not previously explored in
the OM ﬁeld. allowing both OM and IS researchers the opportunity to observe how their
respective methodologies can work together to create and use information. Finally, this
research makes use of a methodology not normally associated with OM, the laboratory
environment. Rather than statistically analyze a factorial simulation experiment, this
research will allow actual information users to evaluate two different types of information
in an attempt to determine which is more effective in identifying shop floor problems. Not
only will this demonstrate how managers may use information, but it also will allow OM
researchers to evaluate the utility of the lab experiment, a methodology normally associated
with Organizational Behavior or Industrial Organizational Psychology (OB/IO Psych).

1.4 Methods and Key Results

As described earlier, this research proposes to combine a variety of methodologies
in an attempt to develop more descriptive and meaningful performance measures. A
literature review will be conducted as will interviews with two shop ﬂoor experts to
develop an initial listing of both traditional and non-traditional measures. Modeling
techniques from the IS ﬁeld will then used in conjunction with a simulation model to collect
a larger, more disaggregated dataset from the model’s runs. This dataset will in turn be
converwd into performance measures, some traditional, some non-traditional, with the use
of a relational database. A laboratory experiment will then be conducted to evaluate the
usefulness of the various types of measures.

On the key question of the measures’ utility, several interesting ﬁndings are
reported. It was found that information type did have a signiﬁcant effect in identifying _

some conditions. It appears that users evaluating the effect of more dynamic and changing

unions can beneﬁt 1
nations the informal

rthscussodinlaters

1.5 Conclusions ar

As discussed, I
hfnrnanon sufficient
lh'ough the combinat
Ique and useful app
lit multidisciplinary
manly the OM field
151 Olganization c

Chapter 2 sun
an, “m Special tn
litters the Itsearch .
mm“? to the pm
ixilﬁl‘i'ﬂenta] design I
Math are PRScnter

has - .
“m Wuonsfc

conditions can beneﬁt from the use of more disaggregate measures, while for other
conditions the information type made no signiﬁcant difference. The ramiﬁcations of this

are discussed in later sections.

1.5 Conclusions and Organization

As discussed, this work is attempting to address the issues of shop ﬂoor
information sufﬁciency and utility through the use of several different methodologies.
Through the combination of OM, IS and 03/10 Psych methods, it is hoped to present a
unique and useful approach to shop floor data management and performance reporting.
The mum-disciplinary nature of this work is such that the results may have application in
not only the OM ﬁeld, but other functional areas of a typical enterprise.
1.5.1 Organization of the Remaining Dissertation

Chapter 2 surveys the literature of information management in both the OM and IS
ﬁelds, with special emphasis on data and database modeling techniques. Chapter 3
presents the research questions and hypotheses and discusses them further in terms of their
relationship to the problem statement. Chapter 4 lays out in some detail the research
experimental design methodology and the proposed data analysis. The results of the
research are presenwd and discussed in Chapter 5 and Chapter 6 concludes this work and

suggests directions for funne research.

2,1 Ovenicw

'lhis chapter
deals with the uses
trends by which i
entronrnents. L'pc
:ndestandably, use
statue. The vast
this in the lS field
les Conceptual
3rd OM ﬁelds are ¢
k3? element of this

VariOus tra
release and due (lat
M “‘3‘ Conch:
33:-'3 in {ht ldb p0;
“Elation 0f [ht s1
and meTHIalion.

3.2 BaCkgIoun.

2 . 0 Literature Review

2.1 Overview

This chapter synthesizes previous work done in both the OM and IS ﬁelds that
deals with the uses and structure of information. This review initially discusses the overall
methods by which information has been approached and managed in manufacturing
environments. Upon such a review, it will be seen that while data and information are,
understandably, used extensively in OM, there is very little discussion of their actual
structure. The vast majority of work dealing with data and information structure has been
done in the IS ﬁeld, which has developed a solid theoretical foundation for research.
Various conceptual and practical applications of data management techniques in both the IS
and OM ﬁelds are discussed, including the use of relational databases, which will become a
key element of this research.

Various traditional shop ﬂoor operational conditions, such as dispatch, order
release and due date assignment rules, are reviewed to establish their importance in the OM
ﬁeld. These conditions will be among those evaluated and identiﬁed by actual information
users in the lab portion of this research. Finally, a review is made of performance
evaluation of the shop ﬂoor, discussing the use of both tactical and non-tactical measures

and information.

2.2 Background

The increasing importance of information in manufacturing has been notiwd by past
OM researchers. Skinner maintained that a factory is “75% information handling and only
25% a materials transforming system” (1969, p. 63). He further maintained that this
relationship has led to a more critical role in the organization for information systems and
information specialists. Hollander, Denna and Cherrington (1995) also suggested a more

core role for the information system within a manufacturing organization with their

suggestion thm manufa
micron of business 1
stern lhat is, even 1
arming events and :
management is becorm
its baton: necessary
Keeling and Kurstedt
and ranaged that is as

However, dcsp
than which 1m nor
narration managcm
tubad a Central role it
mi“ sirrlulzition- i
mm of data to so
“Win or 0ijb tar din
alike cumulative val-

itii‘al'tstorcdhaVt a V

suggestion that manufacturing, along with the entire organization, needs to be viewed as a
collection of business processes that are closely supported by the company’s information
system. That is, even physical processes should be described primarily as information
generating events and should be recorded as such. In fact, manufacturing information
management is becoming so important in today’s business environment, some feel, that it
has become necessary to deﬁne and evaluate “information-oriented” performance (Hill,
Koelling and Kurstedt, 1993), in which it is the actual quality of the information produced
and managed that is assessed

However, despite this growing recognition of information’s importance, one
element which has not seen much attention in the OM literature is the speaﬁc treatment of
information management on the shop ﬂoor. While discussed peripherally, information has
not had a central role in the literanne. In the most commonly used method of shop ﬂoor '
analysis- simulation- information management has been restricted to the very limited
collectionofdatatosupportperformance measures suchas themeanorvarianceoftimein
system or of job tardiness. And since means and variances can be derived from only'two
or three cumulative values, very limited data are stored during the simulation run and those
that are stored have a very simple structure, designed to support the relatively simple
measures being used. There has been very little attention paid to developing more
extensive or theoretically based data structures for the shop floor; it has been considered
sufﬁcient merely to collect and analyze data to report certain aggregate performance

11363811138.

2.3 Information Systems in Manufacturing

In order to accumme gauge performance, Kaplan and Norton (1992) maintained
that it is necessary to properly manage manufacturing information. Indeed, an
unresponsive information system can be the “Achilles heel” of performance measurement

(p.75). They maintained that for a shop ﬂoor information system to be effective in aiding

prim evalu;
ms but Other f um
thtI‘v'HIIOil, little it
stem 0' how the 1
there is very limiter
treated or, more
attrition of the ll
Mpcrformancc v

In one worlt
Mh(1981),we
D it shop ﬂoor m:
ii339%“ bCIween th
Minfitment Comp
System and (5) Cor.
“when S)’Stcm
“Wide infC’FIIIatic

Hm ht “019d the

performance evaluation, it should be integramd with not only the shop ﬂoor operational
areas but other functional areas of the company as well. However, despite this
observation, little work has been done to address the issue of the shop ﬂoor’s information
system or how the system may act as an element of performance evaluation. In addition,
there is very limited work detailing the areas to which the shop ﬂoor could or should be
integrawd or, more importantly, how that integration could be made. Without even a
recognition of the importance of other areas, little progress could be made in integrating
their performance with that of the shop floor.

In one work, however, Hax, while supporting the call for more integrative OM
research (1981), went on to identify the speciﬁc areas of the ﬁrm that should be of interest
to the shop ﬂoor manager (and vice versa) and established a framework to create the
linkages between them. The areas he identiﬁed were (1) Planning System, (2)
Management Control System, (3) Organizational Structure, (4) Evaluation and Reward
System and (5) Communication and Information System. Hax’s speciﬁc inclusion of the
information system was signiﬁcant as was his description of the system as one that should
“provide information with varying degrees of detail to all those managers...” (p. 576).
Hence, he noted the key importance of the information system as an integrating tool and as
one that could provide various views and presentations of generawd data, depending on the
user’s needs.

A survey conducted by Gupta (1994) also pointed out the importance of an
integrative shop ﬂoor information system, while discussing some reasons for its lack of
implementation. He concluded that the primary reason for ineffective manufacturing
information management was the lack of coordination among different departments- a point
in general agreement with previous researchers’ views (Hax, 1981). One reason for this,
Gupta felt, is that in many organizations the information systems are viewed strictly as
Operational or data processing tools and not as strategic weapons. The MIS department

was felt to spend most of its time in an accounting mode and, despite the technology at its

45130531, did net u
and Clark (1981) 2
in mm so much U
mzufxnning inf <
Although data retri
bow how to trans
tendered This d
dffcrent definition
353’; and as “data t'!
Denna and Cherrin
Ins salt: is net n.
WSW before m

disposal, did not understand manufacturing or the shop ﬂoor. In an earlier work, Groff
and Clark (1981) also addressed the issue of the ﬁrm’s technology and concluded that it
was not so much the lack of technology that was impeding the implementation of
manufacturing information systems, but rather the lack of a theoretical framework.
Although data retrieval may be quite advanwd, OM researchers and practitioners need to
know how to transfonn data into information so that it can support the problems being
considered. This distinction between data and information was further made by way of
different deﬁnitions of information as “biased data”. (Hill, Koelling and Kurstedt, 1993, p.
380) and as “data that has meaning to the receiver— the information customer” (Hollander,
Denna and Cherrington, 1995, p. 10). Thus, researchers have recognized that data for
data’s sake is not necessarily useful; it must have some organization or structure and be
processed before meaningful information can be derived
2.3.1 Data Structure, Management and Processing

The speciﬁc issue of shop ﬂoor data structure as it relates to information retrieval
has been discussed only sparingly in the OM literature. Bhimani and Bromwich (1991)
suggested that for Just In Time (JIT) systems, a more comprehensive means of data
collection is needed, one which they denoted as “non-cumulative” data storage. Their
argument was that it is important to collect data on all events and activities, both value and
non-value added, thus allowing the calculation of total manufacturing costs. However, no
suggestions were made as to how exactly to structure this “non-cumulative” information
system. Son (1990) presented something similar by recognizing that there are many more
contributors to manufacturing cost than just labor productivity and that these various cost
categories must be separamd out. The use of a computerized database was recommended
as a means of capturing all of the disparate data categories. Although both of these works
hinwd at a different means of managing manufacturing information, they suggeswd no

speciﬁc structure for the data nor did they provide a theoretical framework to do so.

10

While only lirm
tisisnottosay that I)"
in relatively aggregate

nanntharinnnnre:

 

 

 

COECCiiOIl was supporter
him should not col
54). This is a caution ag
Murmurs Howe
intimation sysrem shou
What at Odds with u
Wm. Such targeted
treble infmmdon (H

Thus tht issue of

While only limited research on shop ﬂoor or manufacturing data structure exists,
this is not to say that typical simulated job shop data have no structure. Their structure is
just relatively aggregate in nature and designed around certain pre-speciﬁed performance
measures that in turn restrict data collection. On the one hand, this limiwd approach to data
collection was supported by Anderson, Chervany and Narasimhan (1979) who maintained
that “you should not collect data if you do not know what you should be measuring” (p.
54). This is a caution against collecting data in other than a pre-speciﬁed manner to support
certain measures. However, Melnyk er a1. (1985) commented that a proper shop ﬂoor
information system should be an objective record of events, without interpretation. This is
somewhat at odds with the practice of collecting data only according to pre-speciﬁed
measures. Such targeted collection implies an interpretative bias, i.e., raw data begins to
resemble information (Hill, Koelling and Kurstedt, 1993).

Thus the issue of an exploitable yet objective and useful data structure, while
mentioned in the literature, seems to be repeatedly skirted. Research has looked at the
various uses of shop ﬂoor data, either directly or indirectly, but has not addressed their

speciﬁc structure or organization.

2.4 Data Management in the Systems/Accounting Field

While data and information structure have not received much attention in the OM
literature, they have been treated in IS theory and research. Much of this work has come
from the accounting ﬁeld as accountants have generally been considered the custodians of
an organization’s information store (Johnson and Kaplan, 1987). This work forms the
basis for much of the study of information in the operations ﬁeld, and thus is discussed in
this review.

Goetz (1939) ﬁrst proposed the idea of maintaining enterprise information in what
be dubbed the “Basic Historical Record,” which was to be an uncontaminated record of

occurrences or transactions indicating what the enterprise obtained and surrendered.

11

Gcctz’s goal “5 3 ﬂ
£5328"le f0“ 5
casement to W use
combinations of data
mputﬂiwd databx
the total?! of a non-
(1985) and as one lhi
tifmt forms) [0 di
or separate transacti
suggesting that the ac

Colantoni, M.
attend-hm! data 5
information in a funct
nth event or transact
trans of Searth and n
was: Each Itcon
”ten for an Operation
This. the data structu

Ptt'haps the m
m Siting: was mad:~

IRDB‘
l COdd emisio

Goetz’s goal was a ﬂexible system that stored data in their most primitive and
disaggregated form so that they could be reclassiﬁed or reaggregated into the form most
convenient to the user. He states that “problem after problem may require special
combinations of data” (p. 154), which is a very accurate depiction of a modem
computerized database compiling data on demand for various users. This is very similar to
the concept of a non-interpretive database of information as discussed by Melnyk et at.
(1985) and as one that can provide information with varying degrees of detail (i.e. in "'
different forms) to different managers (Hax, 1981). Each record stored would correspond i
to a separate transaction or event. This work was the ﬁrst reported instance of a researcher
suggesting that the actual structure or method of storage of the data merits study by itself.
Colantoni, Manes and Whinston (1971) extended the idea of a disaggregated events
or record-based data store a step further by laying out a method of capturing event
information in a functioning database. They suggested a means of coding the data so that
each event or transaction could be identiﬁed as a binary record. Further, they presented a
means of search and retrieval for the data that could be implemented in a computeriwd
database. Each record would have various characteristics, such as name, date and hours
spent for an operation. The database they suggested was hierarchical or tree-like in nature.
Thus, the data structme proposed by Goetz (1939) was becoming somewhat more reﬁned
Perhaps the most signiﬁcant contribution to the idea of events-based disaggregated
data storage was made by Codd (1970) with his development of the relational database
(RDB). Codd envisioned the RDB as a means of sharing information in very large
organizations or across large data banks. The RDB concept differed from previous
databases in that it stressed the independence of the stored data from any applications
thereof. Moreover, data were broken down into separate tables in the RDB, each
representing a different entity (an event, resource, person, etc.) with very speciﬁc local
information. The database became very segmented And, by basing RDB logic on set V
theory, Codd provided a sound theoretical basis for the idea of data disaggregation. In

12

 

essence, he providc
it: could be access
Goetz’s (1939) and
herniation sysrern
approxh to data ma
{19837 for an objecr
positing a needed 1
storage and use of d
animation manage
4313 in separate and
WM the tools b;

2.5 Semantic Da

The concept:
“535 data could be
ﬁre a had to mode
M be compiled at
mil known as da
am ”‘3‘ can be r

F' 112), nhis, a data

Mum“ 0f interr

ma“ model

1992).

. “We da
mm“ 0f (1

essence, he provided a justiﬁcation for and the means of highly disaggregamd data storage
that could be accessed by multiple users in an organization. This was a realization of both
Goetz’s (1939) and Colantoni, Manes and Whinston’s (1971) goal that in an “events”
information system the user determines the level of aggregation of the data. Moreover, this
approach to data management is very much in line with the perspectives of Melnyk et al.
(1985) for an objective recording of shop information and Groff and Clark (1981) as
providing a needed theoretical basis for manufacturing information management. The
storage and use of disaggregawd data has been called the “database approach” to
information management (Dunn and McCarthy, 1997). Codd’s RDB design, by storing
data in separate and independent relations, laid the foundation for this approach and
provided the tools by which the data could be exploited

2.5 Semantic Data Models

The concepts of disaggregated data and the RDB provided valuable techniques by
which data could be struCtured, manipulated and exploited However, with these new tools
came a need to model and depict actual situations so that the data associated with them
could be compiled and used in a “database” manner. For this, the IS ﬁeld has turned to a
concept known as data modeling. A data model has been described by Navathe as “a set of
concepts that can be used to describe the structure of and operations on a database” (1992,
p. 112). Thus, a data model provides the structure by which disaggregated data on the
phenomenon of interest can be captured, stored, interrelated and manipulated Often,
building a data model is a precursor to an RDB implementation (Batini, Ceri and Navathe,
1992). i

A semantic data model enhances the structtne of the stored data with the additional
characteristics of (1) expressiveness— the various data and relationships describe the
phenomena of interest and can be distinguished from each other, (2) simplicity— the user .
can readily understand it; (3) realism— objects in the model closely correspond to real-world

13

momma and (4) Y
[Navathe 1992; Du:
by summing to cap
relied events and re
intentory orders wo

The most cor
hhnonship (ER) rr
capture real world er
tsaset of entities th
Willing entirely v
““1115an a
mom (Brodie, 1

McCarthy (1

Warm or an c

phenomena and (4) relevance— the actual value added activities or processes are modeled
(N avathe, 1992; Dunn and McCarthy, 1997). Semantic modeling extends data modeling
by attempting to capture reality more accurately. For example, in the case of a job shop, all
related events and resources, such as order releases, employees’ involvement, vendors and
inventory orders would be included along with normal machining operations.

The most commonly cited wchnique to depict data models has been Chen’s Entity-
Relationship (ER) modeling (1976). ER modeling has a very semantic basis in that it can
capture real world'entities and their interrelationships. Chen essentially viewed the world '
as a set of entities that were interrelated The methodology allows models to be built while
remaining entirely within the relational framework. Additionally, such models are very
visually appealing and clear and relatively easy to understand with only a few key basic
concepts (Brodie, 1984).

McCarthy (1979) extended semantic modeling into the accounting ﬁeld with his
development of an enterprise model using ER techniques. Within his model, he ﬁrst
identiﬁed very clearly all entities and relationships of interest in an enterprise. By capturing
all events or transactions, he demonstrated how “standard” accounting reports could be
generated from a disaggregate dataset. This work was expanded by McCarthy (1982) with
his development of the REA framework (Resource-Event-Agent). This framework
recognizes that each transaction (Event) has two Other related entities- a Resorn'ce, which is
added to or subtracted from, and an Agent, the facilitator of the event. Thus he provided a
more explicit categorization of an enterprise’s total entities and relationships and was able to
model and capture all value adding and their related paying events.

2.5.1 Manufacturing Information Systems and Semantic Modeling

There is a limited amount of work describing semantic modeling and the database
approach to information management in the manufacturing ﬁeld. most of it in the
production planning area. Lee and Fu (1985) modeled a production planning process
wherein they decomposed a machining environment by establishing interrelationships

14

 

brooch parts and P
inlet down by t}?
Wing and Walker ('
houses is clearly
tuning among varit
disaggregated the 0;
and pans to identify
disaggregated Opera
amss similar to
é53'5’Eftgatod level,
times or tools a
Chi and Navathc, 1
ML but Pfosidet
Nmkm
rah-mural He St.‘
4a a ”10%th “

between parts and processes or operations. Moreover, the parts and processes were further
broken down by types so that speciﬁc parts could be linked to speciﬁc operation types.
Wang and Walker (1989) also developed a model where the relationship between parts and
processes is clearly speciﬁed They extended the model to include sequence dependent
routing among various processes or operations. Through semantic modeling, they
disaggregated the operational data to establish a three way linkage between machines, tools
and parts to identify a unique instance of an operational event. Shooshtarian et al. (1993)
disaggregated operational data on tools and machines into several types and subtypes using
a process similar to Smith and Smith’s (1977) generalization hierarchy. At the
disaggregated level, they were able to show a strong semantic link between tools and
processes or tools and workpieces, even going so far as to indicate cardinalities (Batini,
Ceri and Navathe, 1992). Shooshtarian er al. alluded to a RDB implementation of this
model, but provided no details.

Nandakumar (1990) extended semantic modeling into the Bill of Materials (BOM)
environment. He suggested that basic part data storage can be done within one entity that
has a relationship with imelf within the database to represent parent-component
relationships. He combined an RDB containing this disaggregated data with a FORTRAN
BOM processor to perform MRP calculations. Marsh and Vickery (1995) extended this
work by incorporating the time varying aspects of MRP within a semantic model. This
allowed the complete implementation of the MRP environment within an RDB, as opposed
to relying on a companion third generation language processor. In addition, their semantic
model explored the inclusion of other functional areas in the MRP implementation, such as
vendors/suppliers and customers. It was found that by modeling the MRP planning system
semantically and in such a disaggregated manner, linking, for example, parts or
subassemblies to vendors could be done in a relatively straightforward manner.

McCarthy’s REA framework (1982), as discussed earlier, established a conceptual
data modeling approach to a value added chain of activities, a concept which was later

15

 

extended more cap“
ml of a manufac‘.
pp: adding events
ads-reed this conce
adridual activities
addedactivities. Th
the REA framework
thrive way of allo
matting manufa
W" level in
hitches was reconsr
ﬁnish and Marsh (
m that allowed

Wilma (19
thong "cm‘basod

extended more explicitly by Geerts and McCarthy (1995). In the latter work, a semantic
model of a manufacturing organization was proposed that allowed for the capture of all
value adding events associated with production. Grabski and Marsh (1994) further
advanced this concept to a “discretized” continuous manufacturing process and showed that
individual activities within a complete process chain can be separately modeled as value
added activities. They semantically modeled each stage of the continuous process within
the REA framework. This “micro” approach to data modeling was shown to be an
effective way of allowing for the collection of a detailed and disaggregated dataset
representing manufacturing information. From this database, non-manufacturing or
“enterprise” level information, such as cost summaries, exception reports and operational
histories was reconstructed. Although relatively simple information was made available,
Grabski and Marsh (1994) showed that it was the semantic structtn‘e of the model and
process that allowed for this presentation. -

Maybury (1995) discussed the usefulness of semantic and relational linkages
among events-based data in producing meaningful summary information. He also ‘
presented speciﬁc methods of events summarization including aggregation- the combining
of several similar events into one. Also emphasized was the distinction between events’
frequency distribution and their distribution or occurences over time. These distinctions
could be made more eﬁ‘ectively, he felt, by proper modeling of events’ datasets. Basta
(1995) discussed the use of very large and very disaggregated datasets in presenting
managerial information, although more from an application standpoint. He maintained the
importance of properly structuring the resulting database to allow for useful information
summarization.

2.5.2 Database Modeling in Simulation

Although some of the previous work in manufacturing data modeling has dealt with

actual or simulated processes (notably Shooshtarian er al., 1992; Grabski and Marsh, '

1994), most has been on the conceptual or exploratory level, with very little work done in

16

 

ms of implementati

has reported some wo
databms and simulate

    
 
  

Yancey (1987 l
to manage data created
targeted and designe '
and subsequent reporti.
minuously time persi
ﬁll that such complehc
mid could lmride in
cmimminent

Hill» wmhner
mitten. to 510R en
313mm and Charact
5‘11, it W0uld fatilitatc
Wh may have $01
allll'iation dealt with tl

C“RC-no and S.
as“, ‘0 Mp dCﬁnc
melt“! enabling it

Winthkltsear

@3612!ch data Slol

terms of implementation or data generation and capture. However, the simulation literature
has reported some work in establishing a relationship between conceptual data models,
databases and simulation models.

Yancey (1987) was perhaps the ﬁrst to report on the advantages of using databases
to manage data creawd by simulation. He maintained that a database that was closely
integrated and designed with a simulation model would provide for detailed data storage
and subsequent reporting of various types of information, including discrete or
continuously time persistent data or the tracking of events over time. More importantly, he
felt that such comprehensive storage of the vast amounts of data created by a simulation
model could provide insights into why certain behaviors happen within a modeled
environment.

Hitz, Wenhner and Oren (1993) suggested a broader use of the database within
simulation— to store entire models to enhance their reusability. If many different models’
parameters and characteristics as well as their semantic relationships could be stored, they
felt, it would facilitate researchers’ selection of models for future research. While this
approach may have some merit, most of the previous research regarding databases and
simulation dealt with the database as a repository of generated data for a speciﬁc model.

Centeno and Standridge (1993) developed a general framework for the use of a
database to help deﬁne a simulated environment. They suggested that the database can be
an eﬂicient enabling technology for the design of the simulation model, much as is being
proposed in this research. The relational database model was cited in particular as being a
very effective data storage and manipulation medium.

A work that perhaps extended database and semantic modeling the furthest into the
ﬁeld of simulation was that of Roberts (1991). He suggested that a semantically based ER
model could act essentially as a template for an RDB that could receive data generated by a
simulation model. The database then could respond to virtually any query regarding the ’

model’s historical operation. He compared the calculation of mean and variance, which are

17

 

thinly simplc 10
complete dataset. 51
significant in that he
simulation model at
example and descrit
datasets would look
Koh, de Souza and l
liich used an RDB
MW in which
of step floor m
model lbcn had acce
”Manet could l

relatively simple to compute during a model run, with what may be available from a
complete dataset, such as correlations, plots, histograms and the like. His work is
signiﬁcant in that he speciﬁcally encouraged the use of semantics to better design the
simulation model and the resulting collecting database. Roberts presented a simple queuing
example and described how it could be semantically (ER) modeled and what the resulting
datasets would look like, however, no actual implementation was done. A later work by
Koh, de Souza and Ho (1995) implemented a method dubbed “direct database simulation”
which used an RDB in conjunction with a simulation model. An experiment was
performed in which a simulation model was linked to an RDB that acwd as the repository
of shop floor parameters such as work center and part family information. The simulation
model then had access to certain database information during operation. They found that
performance could be improved by making this dataset available. However, no explicit
mention was made of the database modeling technique (i.e., semantics or otherwise) and
the collected data was not used for performance evaluation.
2.5.3 Semantic Data Modeling Summary

The IS ﬁeld has produced some useful tools for compiling and analyzing data and
information, with semantics and RDBs forming the core of this methodology.
Manufacturing researchers recently have come to realize the beneﬁts of semantic modeling
and, while there are instances of some practical implementations, particularly of RDBs
(Knight, 1994), there is little conceptual or theoretical work reported in the literature. What
research there is has predominantly been in the simulation literature, where the notion of
semantic modeling has been introduced, but no implemented models are discussed.

Prior to examining how simulation and database theory can be combined, it is
necessary to ﬁrst survey issues commonly studied using simulation in the past job shop

literature.

18

2.6 Sinmlated S!

Traditionall)
shop ﬂoor with the i
am by research
Some of those issue
discussed in the foll
2.6.1 BOttleneck R

A bottleneck
Such a resource ma)
Ghosh and Ragatz‘ 1
31° machine constrai
mm and practit
wham- There an
it Statiormy bOttlen

n“ 5’“ one
Within: that Constraj
high {ht shOp is g
' dew does no: It
Sh0p_

A “‘0? cxpen

2.6 Simulated Shop Floor Operational Factors

Traditionally, simulation modeling has been used to evaluate the performance of a
shop ﬂoor with the introduction of various experimental factors. These factors have been
deemed by researchers and/or practitioners in the past to be relevant issues to explore.
Some of those issues and factors, particularly as they relate to the proposed research, are
discussed in the following sections.

2.6.1 Bottleneck Resomees

A bottleneck resource is one that restricts the overall ﬂow through the shop ﬂoor.
Such a resource may be machines and/or labor (Trelevan, 1987; 1989) or tools (Melnyk,
Ghosh and Ragatz, 1989; Ghosh, Melnyk and Ragatz, 1992). In the proposed research,
the machine constraint will be studied. Studies of bottlenecks have appeared in both the
academic and practitioner literature and are recognized as a relevant problem for production
schedulers. There are two primary types of bottlenecks or constrained machine resources-
the stationary bottleneck and the moving bottleneck.

The ﬁrst type, the stationary bottleneck, is the simplest to describe, consisting of a
machine that constrains the overall ﬂow through the shop. The aggregate demand or flow
through the shop is greater than the bottleneck resource can accommodate. A stationary
bottleneck does not move, that is, a set machine remains the overriding constraint in the
shop.

A shop experiencing a moving bottleneck always has one machine at a time that is
acting as a brake on aggregate throughput, but the problem machine is not the same all the
time. One work center will be the bottleneck initially, and another will be at a later time.
This may be caused by the nature of the jobs in the shop over the course of a certain time
period such as a month, or it may be caused by rotating maintenance schedules.

Resource constraints have been researched quite extensively in the OM literature.
One of the ﬁrst works in this area was by Harty (1969) in which he established that it was
critical to identify bottlenecks and to schedule the shop so as not to overload them. He

19

malted that bottle
reliable to work on
of scheduling around
trdttloadcd. they in:
ﬁltration and to assn
Hans but is one the
discussed the idea of
reading to the seve
WINK thus expl:
Lingayat et. t
muiatod Shep ﬂoor
Wally. in order
m“ mtChémS'uns I
“it most effecrivc j

30316ka m

maintained that bottleneck resources must be kept free so that they would always be
available to work on incoming jobs. Roy and Meikle (1995) also discussed the importance
of scheduling around the bottleneck, but rather than emphasize keeping the bottlenecks
underloamd, they maintained that it was important to keep them fully loaded with a high
utilization and to assure that they are not starved for jobs. This approach differed from
Harty’s but is one that was to become increasingly prevalent. Roy and Meikle also
discussed the idea of a resource based model, in which the work centers were ranked
according to the severity of their constraint or by how much they were acting as a
bottleneck, thus explicitly recognizing the bottleneck as a key element of the shop ﬂoor.

Lingayat er. al. (1995) investigated the presence of bottleneck resources in
simulated shop ﬂoor settings. They based their models on actual systems studied
empirically, in order to gain useful insights ofa practical nature. They found that order
release mechansims based on maintaining a high load through a shop’s bottleneck resource
were most effective in scheduling jobs in the shop.

Bottleneck management came more into the public eye with the publication of
works by Goldratt (1990; 1992), extolling his belief that all policies on the shop floor must
be subordinated to the bottlenecks. There were, he felt, two major areas of the shop ﬂoor-
the constraints and everything else. The non-constrained portion of the shop cannot and
should not produce more than the constraint can handle. Although Goldratt maintained that
identiﬁcation of the bottleneck is the ﬁrst step in evaluating a shop floor’s performance, he
emphasized that this may not be an easy task. To demonstrate this, he provided the
example of his ﬁctitious characters in The Goal (1992) laboriously sifting through their
information system before the bottleneck constraint was found.

Plenert ( 1993) also emphasized the importance of ﬁnding and utilizing the
bottlenecks in a manufacturing environment and, more signiﬁcantly, discussed how they
may not always be stationary, referring to a moving bottleneck in which the consu'aint

shifts through the plant. Certain analytical methods of dealing with stationary bottlenecks

20

l-

 

 

letornc Malay

rich for Whatever th.

 

arson her/idem '
In a SW of a

 

based managemot W
trhlt1994; 1995*
lObs and thus became E
tildematld for m‘ ”SC
M due dates on the F
2.6.2 1mm” order
lob mica-965 ‘0 I
a mute with ‘1 Sig
mm jobs so that 0
that 1994). law ‘1
probing how ‘0 dc“
Although mention is m
M thatiObS’ am"
he ckleloPCd algorith
trillion with a tthr
mate input interaniVi

“It
tau '
atttlrlngfacill'tyar

tiattract '
annals. Th

: r
M. TCVICW
and TC

become immediately more complex when the bottleneck is found to be moving through the
shop, for whatever the reason. This, he maintained, is much more prevalent in actual
shops than is evident by the literature.

In a study of an actual job shop on which the simulation model in this research is
based. management was often dealing with resource constraints, or bottlenecks (Marsh and
Melnyk, 1994; 1995). One work center in particular was on the routing for a great many
jobs and thus became a bottleneck. Even three shifts per day of operation could not satisfy
all demand for the resource. Thus, schedulers had to be aware of this work center and base
their due dates on the presence of a substantial queue in front of it.

2.6.2 Irregular order or job releases

Job releases to the shop floor is an area that has received a great deal of attention in
the literature with a signiﬁcant portion ofthis work being determining the way in which to
introduce jobs so that overall shop performance is maximimd (Law, McComas and
Vincent, 1994). Law and Kelton devote an entire chapter (1991, p. 325-419) to
prescribing how to develop input distributions for jobs entering a simulated shop.
Although mention is made of empirically based input distributions, most often it is
proposed that jobs’ arrivals be modeled with theoretical distributions. Some researchers
have developed algorithms and software packages that will approximate an empirical
distribution with a theoretical one (Law and Vincent, 1993), aimed at establishing more
accurate input interarrival times.

More recently, research has attempted to decouple jobs’ entrance to the
manufactming facility and their subsequent release to the shop ﬂoor by the use of order
release mechanisms. This entailed the creation of an additional area of the shop ﬂoor- the
order review and release pool (Melnyk and Ragatz, 1989). As jobs are planned and
generawd, they ﬁrst ﬂow to this pool before being released onto the shop floor. The way
orders are ultimately released to the Shop ﬂoor has been the primary focus of the research in
this area.

21

Wlsner (1993-

 

tiudes and dCSCt'lpti\
mentally all of this
separating job genera.
mechanisms provide
are released to the Her
1995), it was reported
times to he shop t1
tamed and are first
matting, before the)
Muhammﬂq
0rd“ Elease ’ 5
f3“ Of Other more p0
(1936) W that m;

hill: ignoring Order re

Wisner (1992) identiﬁed several broad areas in the order release literature: (1) case
studies and descriptive work; (2) analytical studies; and (3) simulation based studies. In
essentially all of this work, the key element of order release is identiﬁed as that of
separating job generation from job introduction to the shop ﬂoor. That is, order release
mwhanisms provide a buffer between customer orders and the rate at which those orders
are released to the ﬂoor. In fact, in a recent empirically based work (Marsh and Melnyk,
1995), it was reported that shop ﬂoor management and personnel are concerned just with
releases to the shop floor and not really at all with the actual generation of jobs. Jobs are
generated and are ﬁrst processed by different departments, such as engineering and
estimating, before they are given to the shop ﬂoor. Thus the order release pool is present
and, as far as shop ﬂoor management is concerned, is the source of jobs.

Order release’s practical importance may have been overlooked in recent research in
favor of other more popular operational factors. In a survey of practitioners Melnyk er. al.
(1986) reported that managers felt that academic research overemphasized dispatching rules
while ignoring order release mechanisms, which, according to the managers, have a much
more substantial impact on shop floor operations. It was felt that, at a minimum, the two
should be studied together. Other work examined the interplay between order releases and
dispatching and found that not only was there a signiﬁcant interaction between the two
factors, but that the proper choice of an order release mechanism could greatly simplify the
choice of dispatching rule (Ragatz and Mabert, 1988). Melnyk and Ragatz (1989) also
reviewed order release mechanisms and similarly concluded that, used properly, they can
reduce total sh0p work load, balance the work load across the various work centers and can
have a signiﬁcant impact on a job’s total time in system, although such impact may not
always be beneﬁcial.

In two empirically based works by Marsh and Melnyk (1994; 1995) orders in a
simulated shop floor were released to the shop floor according to policy set by shop ﬂoor
management. Depending on the type of job, order release varies by shift and can be

22

 

relatively snDOth and
battles of orders (as t
pursuance than if ti:
2.6.3 Dr: Date Assi,

lntheliteratulr

 

has been due date perfl
(Blackstone er. al., 195
has been a large body (
{minis of due date it
mm 1934). that is,
Siblhhlng 3 ﬁarnewc
{1939) mated the lit
“lighten. the due (12
W“ bdﬂg set iﬂttmg
“556”. 1984). due da.
“Changing SbOp C01
Wm algonthmic r
51m
. One mm” S
Died W‘ Itlated to ll
:0? (1934) disc
_ HR in
3% it 01- (1::th-cr
‘ 4) 1n

who!) Custoi

Nd a en
a . n dUC
‘1th W

m asSignec

dam “the;

m
”Wilda:

relatively smooth and continuous, or can be in fairly large batches. The release of large
batches of orders (as many as 60 jobs simultaneously) may cause signﬁcantly different
performance than if those same jobs were introduced in a smoother manner.
2.6.3 Due Date Assignment

In the literature, one recurring measurement of simulated shop ﬂoor performance
has been due date performance, normally represented by mean job lateness or tardiness
(Blackstone er. al., 1982). Coupled with this focus on due date performance, however,
has been a large body of research dealing with actual due date management. The primary
emphasis of due date management has been the assignment of due dates to jobs (Ragatz and
Mabert, 1984), that is, how best can due dates be assigned to assure realistic operation. In
establishing a framework for the study of due date relawd problems, Cheng and Gupta
(1989) partioned the literature into three areas: (1) exogenous versus endogenous due date
assignment- the due dates are assigned externally to and out of the control of the shop
versus being set internally; (2) static versus dynamic due date assignment (Ragatz and
Mabert, 1984)- due dates are ﬁxed upon job introducdon versus being allowed to vary
with changing shop conditions; and (3) analytical versus simulation based research— an
optimized algorithmic method versus signiﬁcance testing of several different experimental
factors. '

One particular stream of due date research that spans both analytical and simulation
based work related to this research is that of managing jobs in an assembly shop. Ragatz
and Mabert (1984) discussed the difﬁculty of coordinating the due dates of various
components in an assembly shop. This problem was also discussed and analyzed by
Bagchi et. al. (1994) in an analytical examination of a single machine shop that explicitly
studied multi-job customer orders. Two different due date assignment methods were
studied- a common due date was given to all jobs in a single customer order, and separate
due dates were assigned to each job. An optimal solution was developed for the case of the

common due date when each order’s jobs were processed contiguously. However, Bagchi

23

er. al. called for mom
Pmt'Cd too difﬁcult f

Cheng (1988
bath of n jobs in a 5
solution, but it had 1i
Goodwin and Goodv
assembly shop but w
monies In their rr
assembly levels with
S5?le t0 the next 1e
feral} Gomlmcnts w
its m that r
55% For exampk, c
ﬁzrnsions, GOOdwi

er. al. called for more research in the case of the individual due dates as that problem
proved too difﬁcult for an analytical solution.

Cheng (1988; 1989) looked at the problem of assigning common due dates to a
batch of n jobs in a shop with m parallel machines. It was possible to develop an optimal
solution, but it had limiwd application due to the shop’s relatively simple structure.
Goodwin and Goodwin (1982) also performed a study of due date assignment in an
assembly shop but with a much more explicit examination of different products and product
structures. In their model there were 14 different product structures with four different
assembly levels with assembly dependency, that is, sub-assemblies could not be made and
shipped to the next level until all of their components were completed A common due date
for all components was set, although it was not entirely clear exactly how this was done. It
was determined that not all results of a traditional job shop generalized to an assembly
shop. For example, due date based dispatching rules outperformed SPT on most
dimensions. Goodwin and Goodwin called for more research in the multi-level assembly
shop to help ascertain why such results were not generalimble, with the suggestion that
more detailed data, such as that of a distributional nature, be examined.

Adam er. al. (1993) also studied a multi-level assembly shop in which jobs were
divided into “segments” which have to be assembled together after processing is complete.
A segment is represenmd by a series of operations. The same assembly dependency as
proposed by Goodwin and Goodwin (1982) was used in this study. Three different
methods of internally (endogenously) assigning due dates were used, both traditional and
dynamic, with the dynamic methods relying on things such as job characteristics and the
state of the shop at the time of assignment. Jobs introduced were of various types, some
with segments, some with sub-assemblies and some with sub-subassemblies. Nine
diﬂ'erent job structures were studied. Adam er. al. found that there were signiﬁcant
differences between the performance of the different due date assignment methods and that

24

that diftelem wet
canplexity affeCted '
1.6.4 Different lob

Since the cone
anyone, some more
its: research

In the literature
0?: way they have be:
ﬁlers and Treleven (1
Ste1’ {hit hAd val-yin g
mm random food
500 robs, each had to
ﬁance: betwcm Ll
gm W118 SUpen‘.
maimed ﬂow-she

Jobs also Carl
W and UAW (19
km mm“! Custorr
313mm): am= W

ills -
m mmuCCl

Whhmdy. 1

these differences were a function of the different job structlues. Thus the job structure and
complexity affecwd the choice of due date assignment rule.
2.6.4 Diﬁ‘erent Job Types

Since the concept of a job type can, if only by its generic name, be almost anything
toanyone,somemoreprecisionis warrantedtofurtherdeﬁneajobtypeinboththisand
past research.

In the literature, jobs have been segregated into types based on various criteria.
One way they have been so separated is by the nature of their routing and operations.
Elvers and Treleven (1982) performed a simulation study comparing the performance of a
shop that had varying mixes of jobs of different types. One type, job-shop jobs, were
assigned random routing among between two and six work centers. The other type, ﬂow-
shop jobs, each had four operations, all in the same sequence. They found signiﬁcant
differences between the two types when they were mixed, with the ﬂow-shop jobs in
general having superior performance. In fact, of three different job type mixes, the one that
emphasized ﬂow-shop jobs, while retaining some job-shop jobs, was the best performer.

Jobs also can be typed by different priorities, as has been done in the literature.
Ashby and Uzsoy (1995) developed a study based on the assumption that different jobs
have different customers and, hence, potentially different priorities. Diﬁ'erent order release
mhanisms were used based on jobs’ different priorities. In this study, four main job
types were introduced— standard, non-standard, priority and rush. The mix of jobs was
varied in the study. It was found that the choice of order release mechanism had a
signiﬁcant effect on shop performance, with the due date (hence priority) based release
methods having the best performance. This line of study was also pursued by Malhotra et.
al. (1994) who stated that “not all jobs on the shop ﬂoor are of equal strategic importance to
the ﬁrm” (p. 713). They also recognized that different customers have different priorities
that are usually dictated by the customer. Thus it became necessary to deal with a multi- .
class priority system within a job shop. Like Ashby and Uzsoy (1995) this study included

25

tr mix of lob ”Vs a
have a Signiﬁcamt eff:

ramps m‘ m'
Hut, and 1M (1995).
ftlowed by a ﬂow Shh

 

to? and thcu ‘0 the f
mach WP" “50' d
tort center. That is. ‘
hymen“ at the ser-
3?“ was nor an expc
tic.
2.65 Baruch Rm“
Dupatch rules
relatively early work
ﬁned priority or dist
npuhmlcs as £11059
Mlle the Order in

”thahons ofone o

the mix of job types as a factor in the study. The job type mix (by priority) was found to
have a signiﬁcant effect on due date performance.

Perhaps the most explicit treatment found of different job types was developed by
Huq and Huq (1995). The model in this study was a hybrid shop, with a job shop being
followed by a flow shop; followed in the sense that jobs were first routed through the job
shop and then to the ﬂow shop. Job types were defined by their routing, which was ﬁxed
by each type. Also, different job types were assigned different processing times at each
week center. That is, it was assumed that jobs of different types had different machining
requirements at the same work centers. Despite such explicit treatment of types, the mix of
types was not an experimental factor in the study nor are any results broken out by job
type.
2.6.5 Dispatch Rules

Dispatch rules are perhaps the most heavily researched area in the OM literature. In
a relatively early work, Gere (1966) made an attempt to distinguish between what he
termed priority or dispatch rules, scheduling rules, and hetn'istics. He deﬁned priority or
dispatch rules as those that simply assign values to jobs in a queue and which in turn
determine the order in which the jobs leave the queue. Scheduling rules were seen as
combinations of one or more priority rules while heuristics were rules of thumb. In one
review, Panwalkar and Iskander (1977) proposed a further classiﬁcation of scheduling
rules into simple priority rules, combinations of priority rules, weighted priority indexes
and heuristics and other rules. In their review they list and describe 35 simple priority rules
as well as over 60 other types. Their resulting literature review included approximately 36
works that referenced use of one or more scheduling rules. Among those ﬁgured
prominently were SI (shortest imminent operation, also known as SPT), DD (earliest due
date), FIFO (ﬁrst in ﬁrst out) and 8-2 or S-l/OP (variants on least slack or ratio of slack).

Blackstone er. al. (1982) performed an equally exhaustive review of dispatching

rules, primarily as used in simulated environments. Throughout the review, some common

26

 

marching rules cmr
wring Shep floor
imludcd SI, DD, lczu
Blacksmnt er. al. evar
mu time in system,
llrcrt has bee.
flutter, the conclusi

Blather): er. al. (198

 

“135d 00 the measure
WHO: in this measur
Eel-cs: due date tends
m“ And. slack be
15.6 Mortal Fa:

The: CXiStS a s
W “00! Control. ,
mm“) Mung w:

dispatching rules emerged as not only the most researched, but as the most effective in
improving shop floor performance. Among those listed as the overall best performers
included SI, DD, least slack and least slack per operation, critical ratio and FIFO.
Blackstone er. al. evaulated performance based on the traditional aggregate measures of
mean time in system, mean tardiness and mean lateness.

There has been additional work dealing with dispatching rules in the past 17 years.
However, the conclusions that have been reached are startlingly similar to those of
Blackstone er. al. (1982). SI or SPT continues to be the overall best performer when
judged on the measure of mean time in system. It does, however, result in considerable
variance in this measure due to its tendency to hold onto very long process time jobs.
Earliest due date tends to outperform other rules when judged by mean job tardiness or
lateness. And, slack based rules (including critical ratio) are also strong contenders.
2.6.6 Operational Factors Summary

There exists a substantial body of literature discussing the commonly studied areas
of shop floor control. As has been seen, the predominant method of study has been
simulation modeling with the various problems and conditions being the factors of
structured experiments. Throughout this work, with few exceptions, the research
methodology has been relatively traditional and the performance measures aggregate and
tactical in nature.

2.7 Performance Evaluation of the Shop Floor

The most common performance measures for simulawd shop ﬂoor studies have
traditionally been very local and very tactically oriented. In Blackstone et. al.’s survey
work (1982), they restricted themselves to reporting on the results of only three main
measures- time in system, lateness and tardiness. These measures do nor require a great
deal of data collection nor do they require data from areas outside of the traditional shop
floor. While other measures have been reported, the majority are very local and aggregate.

27

 

Another survey piec
scheduling rules and
mi: up predominzm

Amicw ofs

 

Emu: (Trelevan, l

mnﬂowtimtandn

 

rtspahttly, 42% and

m the next most (:01

Mind review of [he s

hr and mean latcncs

Widely.

2.7.1 Shop Floor Fir
Althmgh tactic

Another survey piece (Panwalker and Iskander, 1977) reporwd on a great number of
scheduling rules and numerous performance measures. However, those measures were
made up predominantly of such aggregate tactical measure as time in system and lateness.

A review of several other survey articles dealing with the dual resource contrained
literature (Trelevan, 1989) and order review/release works (W isner, 1992), revealed that
mean flow time and mean lateness were the most commonly used measures, appearing in,
respectively, 42% and 33% of the studies. Variance of ﬂow time and percent of jobs late
were the next most common, being reported 28% and 25% of the time, respectively. A
limited review of the scheduling literature revealed a very similar trend, with mean ﬂow
time and mean lateness being reported in 44% and 78% of the articles surveyed,
respectively.

2.7.1 Shop Floor Financial Performance Measurement

Although tactical and operational measures are by far the most common, there are
instances of utilizing ﬁnancial or cost based performance measurements in shop ﬂoor
simulation studies. Despite the inclusion of costs, however, most such measures remain
relatively local and aggregate in nature.

One of the original works (Rowe, 1960) used work in process (WIP) values for
both dispatching decisions and measuring performance. Rowe found that by dispatching
jobs according to their WIP value, it was possible to reduce overall inventory costs. This
dispatching technique was similar to other local methods in that it looked only at jobs in
individual queues. Anorher work (Hershauer and Ebert, 1973) denoted a response variable
to be minimized as the weighted sum of various traditionally non-cost (i.e., aggregate and
tactical) peformance measures. Each such measure was assigned a weighting coefﬁcient so
that the resulting sum could be represented in cost (dollars). The authors readily admitwd
that the choice of coiefﬁcients was difﬁcult and somewhat arbitrary and provided little

rationale for their choices thereof. In both of these works, the ultimate measure of

28

WW was C0:

 

wry rich dataset
Other work (
hatred a fairly sta:
value In general it t
that MW Ih°
tales were unployed

htscdonthevaluead.

 

Finally that of the 12

01M Itsearth
m m Momma
{1983) discussed the 1
5mm“ and Prabht
Pmttss time, With hi i
“this used the rem
jobs ﬁm With the lovt
funIllatiorr

performance was cost, but the measmes were highly aggregated and did not incorporate a
very rich dataset.

Other work (Hoffman and Scudder, 1983; Scudder and Hoffman, 1987) has
evaluated a fairly standard job shop using cost based performance measures, primarily WIP
value. In general it was found that WIP value acted as a more accurate indicator of overall
shop performance than traditional measures, particularly when value based dispatching
rules were employed. In both of these related works, WIP was calculated in the aggregate,
based on the value added to each job as it was processed in the shop, with value being
primarily that of the labor added.

Other researchers have explored the idea of value based dispatching in conjunction
with cost performance measures. Srivastava and Prabhu (1993) and Wilson and Mardis
(1983) discussed the same value-based variant of SPT dispatching, denoted VSP'I‘ by
Srivastava and Prabhu, who dispatched jobs according to the ratio of job value to imminent
process time, with higher value and shorter jobs having the higher priorities. Wilson and
Mardis used the reverse ratio, that is, imminent process time to job value, and dispatched
jobs ﬁrst with the lowest ratios. Both studies’ models tracbd WIP value by the following
formulation:

WIP = Flown'me x Job Value x Carry Rate (%)

Jobs’ WIP values are accumulated as the model runs and reported at the conclusion
of the run. Both authors discussed the importance of properly valuing each job, but
provide little guidance as to how it was done in their studies. In both cases, WIP is
measuring aggregate shop performance.

A different approach to the problem of evaluating a shop ﬁnancially was suggested
by Scudder and Smith-Daniels (1991). In this work, they incorporated a method to
calculate each job’s Net Present Value (NPV). It was found that NPV was a better

indicator of shop performance than simply mean WIP value, particularly when value based

29

Wm rule
mm), it is

While f1
models, they ha
mung. WT
ﬁlial???” 0’
2.7.2 Non-Ag

While t
degree, ﬁnancl
tth aggrt gate
trample, Melt.
a the results t
process). The;
“her than sin
been vario
“useful when t}

mt‘ses (i.e.,

dispatching rules are utilized. However, with short run simulation models (less than one or
two years), it is questionable how applicable NPV is as a measure.

While ﬁnancial measurements have been used in conjunction with simulation
models, they have still been based on, for the most part, aggregate data collection and
reporting. There have been very few if any attempts to collect ﬁnancial or cost data in a
disaggregate or events-based manner.

2.7.2 Non-Aggregate Data Collection and Reporting

While there is a strong tradition of reporting aggregate tactical and, to a lesser
degree, ﬁnancial measures in simulation, some researchers have presented arguments that
such aggregate mean or variance measures may not be as insightful as others. For
example, Melnyk er. al. (1995) used the method of survival analysis to analyze and report
on the results of a simulation study of a service operation (i.e., a Red Cross blood donation
process). They employed survival analysis to study of the distribution of simulation data,
rather than simply batch means and variances. Thus, differences in distributions of data
between various conditions were presented and studied. This was found to be particularly
useful when the data were not normally distributed, as required by more traditional mean
analyses (i.e., ANOVA).

2.8 Summary

Although past OM researchers have discussed the importance of information and
data management, there has been little work done on its explicit treatment in the
manufacturing ﬁeld. As has been shown, the information systems ﬁeld has produced some
meaningful and efﬁcient methods of modeling and implementing data management systems
(i.e., semantics, ER models and the RDB) that have been carried over into the OM ﬁeld
somewhat, but not to a large degree.

Recent work in the simulation, engineering and information systems literature

indicates a strong interest in semantic modeling to aid in simulation design, information

30

mutation and per?

wqhng systems ETC

ttrluation. Conseqm

 

hformrtion have nor
ape of shop ﬂoor in.
uses have never been _
wndit'ons.

There are certs
mhonnrnts that run

otOM and IS literatun

 

me of these Speciﬁc
he following Chapters
° The explicit
infOl'mation

presentation and performance evaluation. However, very few implementations of actual
working systems are reported and none deal explicitly with enhancing performance
evaluation. Consequently, this lack of implementations has also meant that actual users of
information have not been given the opportunity, in a research setting, to test different
types of shop ﬂoor information and to evaluate their relative utility. That is, information
users have never been evaluawd based on their identiﬁcation of shop ﬂoor problems or
conditions.

There are certain areas of information management within manufacturing
environments that require additional research to be fully incorporated into the current body
of OM and IS literature. As shown in ﬁgures 2 and 3, this research proposes to address
some of these speciﬁc issues not explored before. Among those that will be presenwd in
the following chapters are:

0 The explicit use of a relational database to manage simulated shop ﬂoor

information, including an actual implementation;

0 The materialization of shop ﬂoor information and performance measures from
actual (simulawd) disaggregated data in a relational database. These measures
will then be used for actual managerial decision making;

0 The development and use of disaggregate-based shop ﬂoor performance
measmes, such as time series and distributional data.

0 The evaluation, in a lab setting, of information users' performance in evaluating
shop ﬂoor performance and operation based on the type of information
provided to them.

31

l9?

teammate. h

mm:
Value Chain

Implemmtahon
Stmulation

Dt’IzCollettion

ﬁt:
PM

Mm’ l' .

@E

 

Colantoni
et. al.
197 l

1970

1976

McCarthy
1979,1982

Lee & Fu
1985

Wang &
Walker
1989

Shoosh.
et. al.
1991

 

Disaggregated
Storage

 

Datarnanipulate,
“.139

X

 

Darabme

>e><><§§

 

Conceptual
Examples

><><

 

Accounting

 

Value Chain

><><

 

OM

 

Implementation

XX

 

Simulation

 

Semantic

 

Data Collection
&

 

Use for PM

 

PM
Materialization

 

 

PM Evaluation

 

 

 

 

 

 

 

 

 

Figure 2: Database Modeling Literature Summary

32

 

 

Nardalmrnar
1989, 1990

Vickery
1995

Yancey
1987

1991

K,S&

1995

This
Work

 

Wanted
me

 

ﬂ

Data manipulate,

 

Database

 

Conceptual
Examples

 

Accounting

 

Value Chain

 

OM

 

Implementation

><><

 

Simulation

><><><

 

Semantic

NXNX

XX

XX

 

Data Collection
eat—SEES"

 

Use for PM

 

PM
I! 'l' .

 

PM Evaluation

 

 

 

 

 

 

 

X XX ><><><><><

 

Figure 2 (cont’d)

33

 

 

 

 

 

 

 

Amitagedt.
Atkinson
1990

Howell &
Stoucy
1988

cud.
1991

Bruns &
McKinnon
1992

Bhimani &
Bromwich
1991

Son
1990

 

Assam verms
Wm

 

Deﬁne “useful”
Eonnance measure

 

Operational vs.
Financial

 

J IT. ﬂexible
manufmg'

 

Graphical
Presentation

 

Distributional

 

Simulation based
m

 

Lab Experiment

 

 

 

 

 

 

 

 

 

Figure 3: Information Managment and Performance Measmement Literature Summary

34

l l

Aggregate vet

Dcﬁnthtscfu.

 

 

Operational V:
Francial

Jl'l. ﬂexible
‘ Mttfattunno

 

Presentation

 

 

 

 

Anderson, er. Melnyk er.
al. 1979 al. 1985

1983

Melnyk
er. al. This
1995 Work

 

Aggregate versus
' Info

X X

X

 

Deﬁne “useful"
performance measure

I X

 

Operational vs.
Financial

 

JIT, ﬂexible
manufacturing

 

Graphical
Presentation

 

Distributional

 

Simulation based

 

testing .
Lab Experiment

 

 

 

 

 

><
NNXX

 

Figure 3 (cont’d)

35

 

 

3.(

     
 
 

3.1 Chapter Ovc.

AS shown in
eplcred theortticall}
me insmnces were ’
techniques has nor be
floor setting.

One of the p
llflm‘tfion rnartagern
Stttiﬁcally. can a ser
fumbping an enh

Another issuc
ﬁttl, is the “material
(m dim measur
d‘S‘i'll’tivtt'.’ Such cla
my “abated

“hit; some tr

3.0 Research Questions and Design Overview

3.1 ' Chapter Overview

As shown in the preceding literature review, data and information structure, while
explored dramatically in the IS ﬁeld, has had limited treatment in the OM ﬁeld. Although
some instances were identiﬁed in the literature, the use of data and information modeling
techniques has not been explored and implemented extensively in a manufacturing or shop
ﬂoor setting. '

One of the purposes of this research is to determine whether IS based data and
information management techniques can be applied to a simulated shop floor setting.
Speciﬁcally, can a semantic data model for the shop floor be developed and act as the basis
for developing an enhanced simulation model?

Another issue discussed at some length in the IS literature, and one examined in this
work. is the “materialization” of performance measures and information. Is it possible to
derive desired measures from a disaggregated database and are such measures more
descriptive? Such claims have been made by McCarthy (1982), but have not been
speciﬁcally evaluated in the OM ﬁeld.

While some topics being explored in this work have been studied by past
researchers (see ﬁgures 2 and 3), they have not been brought together as a whole in any
single work. While relational databases have been proposed for an enterprise’s
disaggregated data storage vehicle, their speciﬁc conceptual modeling for a simulation
setting has not been discussed extensively. Moreover, no such implementations have been
reported. Further, the concept of disaggregate versus aggregate information and measures
has been discussed in both the IS and OM literature, as has the use of more descriptive
measurements, but, again, in isolation from each other and without any repormd

implementations. Finally, the actual evaluation by information users of such measures,

36

 

M16 @3de
team, parri

3.2 Research QU
To evaluate
({dtSilOIlS an propos

l) Cananc

 

whether materialized from actual operations or from a simulation model, has not been
studied before, particularly in a laboratory setting.

3.2 Research Questions and Hypotheses

To evaluate the issues discussed in the previous section, the following research

questions are proposed.

1) Can an events-based model of a shop ﬂoor completely and accurately capture
disaggregated shop ﬂoor data in a manner that allows for the efﬁcient transfer
of that data into an events-based database for information storage and
presentation?

2) To what extent can a database populated with disaggregated data be more
useful in completely describing shop ﬂoor performance and in identifying
operational problems that may not be detectable with traditional aggregate
measurements?

3.2.1 Discussion of Research Questions

Research Question #1 proposes a non-conventional conceptual approach to shop
ﬂoor data capturing and reporting. Speciﬁcally, it is asking if a structured data model
based on disaggregated entities can be superimposed on a simulation model to produce a
meaningful dataset. Also, it asks whether these data can be efﬁciently transferred to a
database that can in turn present the data as usable information. In evaluating this question,
several factors will bespeciﬁcally examined and tested. One overall issue to be explored is
the technical feasibility of linking two such models. Can a data model be developed that
can be meaningfully translawd into or superimposed upon a simulation model? One way
of evaluating this speciﬁc issue is to determine if useful measures can be derived from a
dataset generated by a database enhanced simulation model. Another issue is the
consistency of measlnes materialized from a disaggregate dataset and those derived in a

conventional manner in a simulation model. In other words, when considering the same

37

MW Mu
rsul’ﬁ? These spa

hitting section-

 

tial]

 

l

 

performance measures, such as time in system. do the two methods produce similar
results? These speciﬁc issues are addressed by the hypotheses and propositions in the
following section.

Research Question #2 looks at the issue of usefulness. Once collected and
reporwd, does the information available from the proposed disaggregated data collection
serve a useful purpose? That is, can it help users of the information identify why
performance is either superior or inferior? Can information be generated that is more
insightful in terms of its ability to explain and identify the underlying operational problems
and behaviors that may be leading to inferior performance? This question will attempt to
address whether or not such information is relevant for users of the data. The results of a
controlled laboratory experiment in which users are given disaggregate or aggregate based

measures and information will be used to evaluate this question.

3.2.2 Theoretical Basis for Questions

The research problem being explored is the lack of descriptive and insightful
performance measures and information from current simulation models. The asserted lack
of such information restricts simulation’s uses to research without making it widely
available as a managerial decision tool.

This research is based on the theoretical assertion that more descriptive and more
comprehensive performance information will lead to better shop floor performance
evaluation. Speciﬁcally, the effectiveness and efﬁciency of users’ evaluation of the causes
of shop ﬂoor performance problems will be tested based on the information they have
available. In general, the comparative types of information users will have available in this

research are:
Traditional Altemate
Aggregate, mean Distributional
Cross sectional, snap shot Historical time series

38

Those users
t the that”? being F
pgfamance and cor

The semantic
as the means by whi
till be testing this ti
section
3.3 Hypotheses 2

Some of the 1
Statistical analysis, a
being WM. the
Mom of the rese
Wally tcsmd H

3'3'1 PmPOSition #

PRP‘II The

ma.
been

his meOSldl
1‘1:meth of m I
mum appmachl
model to “Port "Edititl
|

:03 .
ltnhonal [mans i

i .

‘ﬁdadOn
and proorot

l
l’iéscsﬂngits aCCUFaI
t

 

Those users who have the alternate, more descriptive information should, according
to the theory being proposed. be able to more efficiently and effectively evaluate shOp ﬂoor
performance and correctly identify the underlying causes of any performance degradation.

The semantic database approach to information collection and generation is offered
as the means by which to provide better performance measurement. Thus, this research
will be testing this theory according to the hypotheses and propositions in the following

section.

3.3 Hypotheses and Propositions

Some of the research questions of this study do not lend themselves to formal
statistical analysis, while others do. For this reason, borh hypotheses and propositions are
being presented, the former being more suitable for testing. Propositions are expected
directions of the research, but will only be evaluated and discussed, not formally or
statistically tested. Hypotheses, on the other hand, will be tested statistically for

signiﬁcance.
3.3.1 Proposition #1

PRP- 1: The means of conventionally (non-semantically) derived performance
measures are comparable to the means of the same measures that have
been derived from a semantically based disaggregated dataset.

This proposition addresses one of the issues raised by research question #1, that of
the compatibility of the proposed enhanced approach to simulation modeling and the
conventional approach. It will examine the ability of an enhanced or expanded simulation
model to report traditional tactical measures that are consistent with those derived by
conventional means, i.e., through built-in functions of the simulation software. It is a
validation and proof of concept of the method of measurement materialization and a means

of assessing its accuracy.

39

fusibih']
Emma
3“ deter
the he
2%!
DOI ohm
h djSCl

This issue of consistency of two differently derived measurement sets has not been
previously explored in the literature, thus there is no past basis for making a priori
expectations of the ﬁndings for this particular proposition. Researchers have generally
poinwd to disaggregate data storage as producing more descriptive information than
traditional measures, rather than simply providing an alternate method of deriving
traditional measures (Centeno and Standridge, 1993; Yancy, 1987 ; Roberts, 1991).
Although the materialization process within an RDB may be quite complex, since the
disaggregate data and the aggregate measures will be of the same simulation run, it is very
reasonable to assume that the results will be very similar, if not nearly identical.

Support for the proposition would indicate that the proposed methodology is
compatible, at least in this instance, with that already being done within conventional
simulation software and models. Lack of support for the proposition could indicate that the
proposed method of data management may not provide a means of creating usable
performance measmes, or that the conventional method of measurement derivation, directly
within the simulation software, is inaccurate.

3.3.2 Proposition #2

PRP-2: It is possible to materialize useful measures through the use of a database
enhanced simulation model and its resulting disaggregated dataset.

This proposition is evaluating, in qualitative terms only, the question of the overall
feasibility of the approach suggested by research question #1. It addresses the actual
“materializability” of useful performance measures. The manner in which such measmes
are determined to be useful involves several interviews and questionnaires and is discussed
in the next chapter. The proposition supports the position that a disaggregated dataset from
a semantically enhanwd model can provide a means of producing useful measures that are
not obtainable by conventional means. Based on work such as that of Yancey (1987), who

ﬁrst discussed designing semantic data models in conjunction with simulation models, and

40

Roberts (1991), who introduced the idea of using a database of simulated data to
materialize useful measures, a reasonable a priori expectation is that it is feasible to
materialize useful measures from a disaggregate dataset. In fact, Roberts suggested some
speciﬁc types of enhanced measures, such as time series plots and histograms, that will be
utilized in this research. Thus, expectations are such that this proposition will be
supported. Lack of support for this proposition, as with proposition #1, would indicate that
the proposed method of data management may not provide a means of creating usable
alternate performance measures.
3.3.3 Hypothesis #1

H04 Information and measures materialized from a semantically based historical

dataset do not result in signiﬁcantly different performance by users in

identifying and describing shop ﬂoor operational problems and conditions,
as opposed to using traditional aggregate tactical measures.

This hypothesis provides a direct means of evaluating research question #2. In
testing this hypothesis, the issue of the usefulness of the proposed measures and
information outputs will be examined. Participants in a lab experiment will be given the
opportunity to use the actual outputs- either traditional aggregate tactical measures or the
information and measures derived from a disaggregate dataset. The success of the various
groups in identifying and describing the underlying operational conditions will be evaluated
statistically to determine if access to either set of measures results in signiﬁcantly different
(greater or lesser) success.

The null hypothesis holds that there will be no difference in performance between
groups using two different types of measures. It is expected that this null will be rejected.
In fact, the speciﬁc theory being tested in this research, that a disaggregate, semantic-based
approach to modeling yields more useful information, postulates that the performance of
the group with the disaggregate-based measures will be signiﬁcantly superior. Works cited

41

 

prttiously in this Sl
snchasMchh)’ l
madlsaggltgalc

can be reasonably}
ledingtna rejectior
3.4 Discussion 0

Several diffe

 

ﬁlmed in more dc
his time.
3.4.1 Development
Pmposition it
brieﬂy describes hou
it“? ﬂoor echns an
7m WOW ages
hirer cam their ‘
film the bash for de‘
Myst: of thc Cchr
3'4'2 Sammie and
Research Qu.
itcocm so to 3pc
{3137301111 dambasc an

Elms. To begin;

previously in this section (Yancey, 1987; Roberts, 1991) along with others in the IS ﬁeld,
such as McCarthy (1982) and Maybury (1995) point to the expectation that measures based
on a disaggregate dataset are both more useful and more descriptive. Thus, such measures
can be reasonably expecwd, a priori, to improve the performance of those using them,
leading to a rejection ofthe null.

3.4 Discussion of the Research Methodology

Several different methodologies are employed throughout this research and each is
discussed in more detail in the following chapters. However, an overview is presented at
3.4.1 Development of Useful Performance Measures

Proposition #2 makes reference to “useful” information and the ensuing discussion
brieﬂy describes how such usefulness is determined. In the ﬁrst phase of the research, two
shop ﬂoor experts are contacted and interviewed as to their opinions of various measures,
both traditional aggregate and disaggregate. In addition, a questionnaire is provided to
further capture their opinions. These Opinions, in conjunction with a literature review,
form the basis for developing a list of measures for future evaluation. No statistical
analysis of the experts’ responses is made.
3.4.2 Semantic and Simulation Modeling

Research Question #1 asks whether a semantic data model and a simulation model
can coexist, so to speak, to allow for the generation of data that can be utilized in a
relational database and whether the database can be used to materialize performance
measures. To begin addressing this issue, ﬁrst, using Entity Relationship (ER) drawings,
a detailed data model of the Shep floor was developed. This formed the basis of the
relational database that stored the inforamtion and also dictated, to a certain extent, the
design of the simulation model. Next, the actual simulation model was developed,

incorporating many u'aditional features but also some additional ones to allow for the

42

 

collation of a larg:
march question #3
(with a database) dtl
3.4.3 Usefulness .
A laborator .
patieipants were girl
shop ﬂoor problems
idifftl’tlll conditions,
primary analysis too
the inclusion of oova
£50395. Chi Square .
in“ “31572 the It

 

R5315 0f the lab exp.
Witness was also c
hind most useful w:

This is the on
mm“ "“3 deﬁne
mm“. M will 1
mm dimmions, s

mimetinmnm

collection of a large disaggregate dataset. To evaluate the models’ output and address
research question #1, a comparison of the means of conventionally and non-conventionally
(with a database) derived measures was made.

3.4.3 Usefulness of Aggregate versus Disaggregate Measures

A laboratory experiment was performed in which two independent groups of
participants were given different measurement types and asked to identify the underlying
shop floor problems and conditions. Since each participant was asked to evaluate ﬁve
different conditions, a repeated measures Analysis of Covariance (AN COVA) was the
primary analysis tool to test the a priori hypothesis (Hypothesis #1). This analysis allowed
the inclusion of covariates while comparing the mean performance of the two independent
groups. Chi Square and Kolmogorov-Smimov tests are used as post hoc analysis tools to
further analyze the nature of the lab participants’ responses. In addition to testing the
results of the lab experiment, the participants’ judgement of the various measures’
usefulness was also evaluated, although not statistically. The measures that each group
found most useful were identiﬁed and compared with expectations.

This is the only section ,of the research in which independent and dependent
variables were defined. The dependent variable is the performance of the two groups of
participants. As will be discussed in the next chapter, this performance was judged on
several dimensions, such as accuracy, efﬁciency and conﬁdence. The independent variable
is the type of information and measures being used, speciﬁcally aggregate and
disaggregate. In addition, several demographic covariates were also analyzed.

3.4.4 Clarity of Information Presenwd

The ﬁnal phase of analysis involves the evaluation of the lab participants’ opinion
of the clarity of the information presented This is done to determine if there is a signiﬁcant
difference between the two groups’ feelings regarding the clarity, adequacy and usefulness

of the information given to them and the manner in which they were trained to use the

43

 

   
   
 

The initial r
with IS and OM da:
ﬁniteness of dif‘;
primarily with quail
culuated in a rigorc
“Eh W ItSlllt
this itself to more
thally if using
{mm}? evaluate

Additional
ti bod; the differcn
Muted This all
m and if Cither ;
her, thus 3110M:

information. Analysis of Variance and Kolmogorov-S mirnov tests were performed to

statistically evaluate the groups’ differences.

3.5 Conclusion

The initial research problem was addressed by two research questions, one dealing
with IS and OM data management techniques and the other with the actual utility and
effectiveness of different types of performance information. The ﬁrst question deals
primarily with qualitative and conceptual issues, hence its related propositions are not
evaluated in a rigorous statistical sense. Rather general evaluations are made and compared
with expecred results. Research question #2 goes more to the heart of the research and
lends itself to more in depth quantitative analysis. Its accompanying hypothesis, #1, asks
speciﬁcally if using either of two types of measures allows users to more effectively or
emciently evaluate shop ﬂoor conditions and is tested much more rigorously.

Additional analyses address the users’ (lab participants’) perceptions and opinions
of both the different measures’ relative usefulness and the manner in which they were
presented. This allows us to evaluate if those measures expected to be helpful actually
were and if either group felt that the information could have been presented differently or

better, thus allowing some insights into the structure of the experiment itself.

4.1 Chapter 0v

 

lhis chapel

plied as to which p

covers how certain c

teams are used ll
described This incll
this caseit augmentl
“hop ﬂoor’s seman
mating and collec
disused iDCllxﬁng 1

“it mmainin g seetior

 

WWW insmm
Willa] condin'o
it iUtstions being ac‘
WWW Perform

There 313 font

5hr fOHom'ng SCCtic

4.0 Research Methodology and Design

4. 1 Chapter Overview

This chapter outlines the four main phases of the research design. The ﬁrst section
covers how certain experts in the ﬁeld of OM and shop ﬂoor control were contacted and
polled as to which performance measures and information they thought were useful. These
measures are used in a later phase of the research. Next, the model building process is
described. This includes a discussion of information systems methodology and how, in
this case, it augments traditional simulation modeling. Also covered is the development of
a shop ﬂoa’s semantic data model and resulting relational database that forms the basis for
generating and collecting data from the simulation model. Next, the shop being modeled is
discussed, including the various parameters and experimental conditions that are explored.
The remaining sections of this chapter discuss the development, validation and use of an
experimental instrument within the laboratory setting. This includes a discussion of the
experimental conditions that will be evaluated, the design of the lab sessions themselves,
- the questions being addressed by the instrument and the method by which the lab
participants’ performance will be judged.

There are four primary phases to this research, as outlined in ﬁgure 4 and discussed

in the following sections, which numbers are indicated in the ﬁgure.

45

 

4.2 Du:

tmphcal an.
ﬁsh: 5.

 

 

 

 

Phase 1 Phase 2 , Phase 3 7 Phase 4

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Phase 1 Literature review; poll shop ﬂoor and Operations Management experts;
obtain opinions on “usef ” measures.

Phase 2 Design and build a semantically based data model and accompanying
relational database for the shop ﬂoor; develop a discrete events
simulation model of the ﬂoor as well.

Phase 3 Validate, verify the simulation model; run it to generate and collect
aggrigate and disagge_gate datasets; populate relational database.

Phase 4 Conduct controlled laboratory experiment allowing participants to judge
effectiveness, efﬁciency, usefulness of various measure types.

 

 

 

 

Figure 4: Phases of Proposed Research

4.2 Determination of Performance Measures and Information
The performance information was developed for this research with a combination of
errrpirical and literature research, with the assistance of OM experts, as described in

ﬁgure 5.

 

 

 

 

Initial Ust T Experts'Oplnion -— Ranked Ust ._....

 

 

 

 

 

 

 

 

 

 

Figure 5: Determination of Performance Measures and Information (Phase 1)

The goal of this research phase was to provide a reasonable basis for the measures
to be used in later phases, speciﬁcally those to be presented to laboratory experiment
participants in Phase 4. To that end, traditional simulation performance measures, as
identiﬁed in the literature, were augmented by additional measures obtainable from
disaggregated data storage. All measures were then reviewed by two OM experts prior to
ﬁnal compilation and presentation in the lab setting. This was done principally for added

46

 

insights to
literature-t
4.2.1 Itli'

A 1
measures,
operationa
These incl
amused ll

15 OFPOSCI
4.2.2 B

fields of C

whites

‘35 neces

noChIqutle

minute

insights to the literature review and to provide some additional relevance to the initial
literature-based performance list.
4.2.1 Initial List

A list of traditionally used aggregate and ﬁnancial shop ﬂoor performance
measures, as identiﬁed in section 2.7, was compiled. To the list were added other
operational measures that could be materialized from a disaggregated shop ﬂoor dataset.
These included measures and information such as distribution of time in system, as

opposed to the aggregate meastn'e of mean time in system, and time series of queue length,

as opposed to mean queue length
4.2.2 Expert Opinion

After the initial list of measures and information was drawn up, two experts in the
ﬁelds of OM and Shop Floor Control were contacted and asked to respond to the listed
measures.

Only two persons were interviewed and surveyed, for several reasons. First, it
was necessary to have several in depth discussions of the project and the data management
techniques with those persons prior to their completing the survey. In addition, it was
important to have persons who were indeed expert and who had a wide array of experience
in manufacturing environments. In limiting the sample to those in this category, only two
persons were chosen. This allowed an examination of the results of the survey in some
depth, although not as much breadth as is traditionally done. The limiwd size of the sample
did, however, effectively eliminate any statistical power, making formal analysis of the
survey results impossible.

4.2.2.1 Interview Structure

The two experts were interviewed, one in person, the other on the telephone. The
beginning portion of the interview was devoted to discussing the research in general terms.

Both aggregate and disaggregate data and measures were discussed along with the possible

47

advanta
the use:
he sm
left witl

consist:

betweei
the sub j
the sun
At the e
return it

faxedtc

item]
4.3

1“can.
Die Ste;

advantages and disadvantages of each. Care was taken to keep the discussions focused on
the uses of the information and why some types were felt to be more useful than others.
The structure of the interview was patterned closely on the survey instrument, which was
left with them after the discussion. This enabled the interview to proceed in an orderly and
consistent manner.

The primary researcher was the only interviewer with both subjects and spent
between 40 and 60 minutes with each person. During this time, notes were taken regarding
the subject’s comments and suggestions as to what constituted a “useful” measure. Also,
the survey instrument was discussed so that it was clear what was required of the subjects.
At the end of the interview, the subjects were requested to ﬁll out the survey form and
return it to the researcher. In the case of the telephone interview, the survey form had been
faxed to him prior to the conversation. The survey form consisted of ranking various
meastues and information sets in terms of their managerial usefulness. See Appendix B for

a complete copy of the survey form.

4.3 Model Building

The second phase of this research involved developing the simulation model and the
accompanying relational database and then linking the two within the simulation model.
The steps entailed are as shown in ﬁgure 6.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Data Model RDB Design, Semantically Based
Design ‘ Implementation . ' Smulatron ‘-—>
Model
Simulation Shop floor
Model Design —» Conditions
Included

 

 

 

 

 

 

Figtue 6: Model and Database Development (Phase 2)

48

The simul:
data model of the
previously studier
McCarthy (1979)
noiels are not on
And while the sir
genera: the nece

It was foe
it topdown app]
Also, his hmw
lilowing the m
MCCathy's frarr
Wt has substar
Denna er. al. (19
discussed Addi
mum Mccat
discuss“ in the

The simulation model and database were designed concurrently based on a semantic
data model of the shop ﬂoor and on the actual workings and design of a shop ﬂoor
previously studied (Marsh and Melnyk, 1995). In doing so, the methodology suggested by
McCarthy (1979) and shown in ﬁgure 7 was used. It is important to note that the two
models are not only built concurrently, but are also used in conjunction with each other.
And, while the simulation model is in many ways fairly traditional, it has been altered to
generate the necessary information for evaluation, as will be discussed in section 4.4.1.

It was found that McCarthy’s work (1979) provided a reasonable framework due to
its top-down approach to system modeling and its focus on a conceptual and visual model.
Also, his framework described the various levels of model building in general terms,
allowing the modeler to deﬁne each as his/her needs dictate. Like Chen’s (1976),
McCarthy’s framework in this work used ER drawings to help visualize a system. Further
work has substantiated this approach, as discussed in section 2.4, particularly that of
Denna er. al. (1993), in which several successful implementations of this approach are
discussed. Additional work by Marsh (1995) and Grabski and Marsh (1994) have
extended McCarthy’s methodology and shown its applicability to the OM ﬁeld, as

discussed in the literature review sections 2.5.1-2.5.3.

49

@ w , A umacrunm ENTERPRISE

 

 

 

 

 

 

 

 

 

 

 

 

 

Cabana-yam ) wt NICO“
WWW SIMILATDNMGIL
dthoSyIt-m W’
W 1%on
1mm MWEWNW
J MWWMW
www.momsm Jmmcpummmbm
. EWJW
muff“
Gavan-woman
v v 4
mountain-m 0"“ Wm neutron“, “name
NHL-Hm“ Mandolin-cures
ianmwm) MW)

Figure 7: Database Development Process (adapted from McCarthy, 1979)

4.3.1 Levels 1 and2

The ﬁrst step in the implementation of the proposed shop floor information system,
described by Levels 1 and 2 in ﬁgure 7, was considering the speciﬁc environment to be
studied. In this research, the general aspect of reath of interest is the manufacturing
enterprise, with an in-depth look at the shop floor.
4.3.2 Level 3

Once the environment had been identiﬁed, the next step was to build a conceptual
and operational model of the system. This step actually entailed the building of two
separate models— the semantic data model and the discrete events simulation model. As it

was important to capture the operational and tactical features of the shop ﬂoor environment,

50

the ﬁrst P3”
m augment
operation, as
trained and
presened in
parameters a
4.3.2.1
Willi
patterned afu
fora major a
on7 differet
Ctnters coma
SimPliﬁed to
Operated rim:
thnight shit
[hm Plums:
“15mm; (2
impecﬁon pr
taken to this r
itﬁtter. The c
0f tht
Pits-r Ms:
W Center p
hand]: jobs [h
the term; hav

lengths of the:

the ﬁrst part of this step was to build a relatively traditional simulation model. This model
was augmented with the ability to generate and capture disaggregate historical data during
Operation, as discussed in more detail in section 4.4.1. It is this generated data that will be
analyzed and evaluated in the further phases of the research. The simulation model will be
presented in general terms in the following sections, with a more in-depth discussion of its
parameters and operation in Appendix A, sections A.l - A.3.
4.3.2.1 General Shop Description

While the simulation model is that of a hypothetical shop ﬂoor, it is strongly
patterned after an actual environment previously studied— the machine/support tool room
for a major automobile manufacturer (Marsh and Melnyk, 1995). The actual shop consists
of 27 different work centers (lathe, boring mill, welding, heat treating, etc.) with most
centers containing from three to ten machines. For this research, however, the shop will be
simpliﬁed to six work centers, comprising those that are the most frequenwd. The shop is
operated three shifts everyday, although stafﬁng levels are lower during the afternoon and
midnight shifts and on weekends. There is a centralized customer service area that serves
three purposes: (1) to provide an entry to the shop for those jobs being carried in by
customers; (2) to provide a central inter-operation queue area; and (3) to provide ﬁnal
inspection prior to the jobs’ exit from the shap. All jobs, once done at a work center, are
taken to this area for inspection and holding until an operator is available at the next work
center. The customer service area is also where material orders are placed and received.

Of the six work centers, form are conventional in that they process jobs according to
preset process times and have an accompanying queue that can be measured. The ﬁfth
work center is a batch heat treating station that does not have a queue per se, but does
handle jobs that flow through it according to preset processing times. Various machines in
the center have queues in which jobs wait until a batch load accumulates, however, the

lengths of these queues are not reported The sixth work center is simply ﬁnal inspection

51

and is thc last
center and cnu
processing tin
4.3.2.2

To val
management c
Ibeduling w:
addition, as th
mm of numl
features of the
machine worl
immune.

To at

M1 and {Dec

and is the last station for all jobs in the shop. There is never any waiting for this work
center and entry to the center is equivalent to leaving the shop. This center has a zero
processing time.

4.3.2.2 Model Validation and Veriﬁcation

To validate the simulation model, it was discussed with the actual shop ﬂoor
management during its original construction. Interarrival times, routings and employee
scheduling were conﬁrmed with management as they were built into the model. In
addition, as the model was run, its output was compared with that of the actual shop in
terms of number and type of jobs and overall shop load. It compared favorably with many
features of the actual shop. Moreover, many general features of the shop, such as multi-
machine work centers, queues and order release pools, are similar to those seen in the
literature.

To aid in verification, the model was built in modules, such as the order/release
pool and~mechanism, work centers, assembly and inventory areas. Each module was
tested and veriﬁed individually with numerous tracer to assure its proper operation. As the
modules were assembled into the ﬁnal model, additional testing was done to assure that all
the components operated well together.
4.3.2.3 Shop Floor conditions

Ultimately, this research evaluates the impact of various types of information on
shop ﬂoor decision making and problem identiﬁcation by examining the effect of the type
of information on users’ assessment and identification of problems. To this end, a series
of shop floor problems and conditions as identiﬁed in the literature (section 2.6) and
substantiated by experts (section 4.2) were developed and operationalized in the simulation
model. The conditions are discussed more thoroughly with the detailed shop and model
description in Appendix A, but are summarized in the next section.

52

 

 

These com

to pmtnt interact
inmcﬁorrs that Vt
imparts. This
“nightfomaxd n
“S 10 Est the id:
Muted, Such c
That is, the actua
lam; being dcr
113an ‘0 lab 1
conditions.

4.3.2.4 Conditions Summary

The six conditions being introduced to the model in this research are as follows:

Varying dispatch rules

Varying due date assignment rules
Irregular job or order release
Stationary bottleneck

Moving bottleneck

99939!"

These conditions were deliberately inserted into the simulation model one at a time,
to prevent interactive effects. It was felt that more than one condition at a time would create
interactions that would excessively complicate the identiﬁcation for the experimental
participants. This approach also allowed the analysis of the performance data in a more
straightforward manner, as discussed in section 4.6.1. Since the objective of the research
was to test me identiﬁability of such conditions, depending on the type of information
presented, such conditions were not teswd as experimental factors in the traditional sense.
That is, the acrual performance of the shop'floor was not evaluated with one or more
factors being determined as the “best.” Rather, performance results of the model were
presented to lab experiment participants so that they could attempt to identify the
conditions.

These conditions do not constitute an exhaustive list, but are illustrative of the
general category of problems seen by shop ﬂoor managers, who would be the main users
of the performance information being evaluated in this research.
4.3.2.5 Base Case Operational Parameters

A base case was established that represented no conditions being inserted into the
model. The following summarizes the operational parameters for the model’s base case:

1. N o stationary or moving bottlenecks

2. Smooth job release to the ﬂoor

53

3. Indiv
4. “Hist
5. First
6. Jobr
7. Nojt
8. Capa

4.3.3 Data Mo

Cancun:
the shop ﬂoor w
andAgcnts (RE
iIllfffltlatiortship
shown in ft gum

“Writes the

N

315m:

WORK C
1088
RAW MA

 

Individual due date assignment for projects’ jobs

“Historical” mix of job types (more projects)

First come ﬁrst served (FCFS) dispatch rule

Job routing established at outset; 10 work centers maximum; repeats allowed
No job preempting (although priorities varied)

Capacity utilization range: 87—89%

9°89???

4.3.3 DataModelDecign

Concurrent with the development of the simulation model, a semantic data model of
the shop ﬂoor was designed. During this process, all of the relevant Resources, Events
and Agents (REA) involved in the shop ﬂoor were identiﬁed, as were their key
interrelationships. This is in accordance with McCarthy’s (1979) proposed methodology
shown in ﬁgure 7 and discussed in a subsequent work (McCarthy, 1982). Figure 8

summarizes the various entities of the shop ﬂoor.

 

 

 

 

Readiness Events Agents

WORK cams OPERATTON remover;

JOBS RAW MAT’L RECEIPT RAW MAT’L SOURCE
RAW MAT’L RAW MAT’L ORDER
MULTI-JOB ORDER

 

Figure 8: Shop Floor Resources, Events and Agents

Resources are those entities that represent assets and that may be altered during the
operation of the system. For example, JOBS represents an asset to which value is added as
it is routed through the shop. Similarly, RAW MATERIAL represents a store that may be
decremented (inventory). Events represent speciﬁc instances or occurrences in the system.
OPERATTON refers to the actual work being done to a particular job at a particular work _
center. Each operation is a different event. RAW MATERIAL RECEIPT represents the

54

event of actually
implementing the

OPERATION cw
I'

In a tradit:
decremented rcso
tsourcc and dccrI
enhancement of 3'
identiﬁed as cithel
mdancc with Ll

After idcn
Wcrt noted The 5|
mmhetiology (197
tables mOdcling th
i“filtration to ace

is Colantoni, Man

Shep being mOdCIt'

 

event of actually receiving a shipment of a speciﬁc order. Agents are those responsible for
implementing the events. The primary agent- the EMPLOYEE- is responsible for the
OPERATTON event that adds value to JOBS.

In a traditional REA model, each Event is related to an incremented and
decremented resource. In the case of this shop, an OPERATTON will increment the JOBS
resource and decrement the Employee labor-hours pool. Thus one resource pays for the
enhancement of another. It should be mad that, while the entities in this model were
identiﬁed as either Resources, Events or Agents, the data model was not developed in strict
accordance with the REA framework (McCarthy, 1982), as will be apparent.

After identiﬁcation of the various entities in ﬁgure 8, their key interrelationships
were noted. The shop floor’s data model was then developed using Chen’s ER
methodology (1976). Developing such a model allowed the creation of a system of data
tables modeling the shOp ﬂoor and avoided the prespeciﬁcation of which data or
information to accumulate. This is in keeping with events based accounting theorists such
as Colantoni, Manes and Whinston (1971) and McCarthy (1979). The ER model for the
shop being modeled is shown in ﬁgm'e 9.

55

 

 

OZiIOP<c W10

 

 

Figum 9.

 

o“

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

/
EMPLOYEE
I /
11-
c.» «m
o CSEMP ‘
/
3' it; (1.1) (“’1“).
g N RM (LN) (1.1) RM
é RECEIPT 0 ORDER
8 (M) (0.?!) ‘W 3""
OPERATION: >— SEQUENCE M 45”
> (MI) 9‘
1
SOURCE
JOB ’
__I_V,i J 2’" 1 ,
caoa omoe wuoa ’
(1. t) (1.1) (1.1) (1.1)
(0.") (0.") to.» (1,", .«p’
CSEMP 2 WI ’
PROCESS at I (M) (1.1)
w t m
“'08 3' MM nut 69‘
l 7 I
:3; BCHMAN
a" l J
2
EMPLOYEE
/

 

 

 

Figure 9: Data Model of Simulated Shop Floor (Primary Keys Noted)

56

4.3.2.8 Entity Types
Within several of the entities liswd in ﬁgure 8 and shown in ﬁgure 9, different types
were identiﬁed, as shown in ﬁgure 10.

 

Rem—m

JOB PROJECT
WALK-IN
CRIB
QUICK RESPONSE

 

MULTT-J OB ORDER PROJECT
WALK-IN

EMPLOYEES OPERATOR
CUSTOMER SERVICE CLERK (CSEMP)
PROCESS PLANNER '

BENCI-IMAN (BCHMAN)
ESTIMATOR (EST EMP)

 

 

 

Figure 10: Shop Floor Entity Types

These various types are represented by generalization hierarchies (Smith and Smith,
1977) in ﬁgure 9. For example, Jobs consists of four different types, as shown more

clearly in ﬁgure 11.

 

 

 

 

 

 

 

 

CJOB ORJOB WIJOB PJOB

 

 

 

 

 

 

 

 

 

 

 

 

Figure 11: JOB Generalization Hierarchy

57

 

Also not
SERVICE. 01’
ont- OPERATO
apparent with
presentation and
4.3.4 [ml 4

Level 4 i
built simulation n
comprise essentia
march however
tablet in Level 3.
03d“ Storage bu
march. a rclatio

Also note that, although Employee shows ﬁve different types (CUSTOMER
SERVICE, OPERATOR, BENCHMAN, PROCESS PLANNER and ESTIMATOR), only
one- OPERATOR- is used in any meaningful way in this research. This will become
apparent with the discussion of the experimental results and was done to keep the
presentation and analysis of information manageable.

4.3.4 Level 4

Level 4 is the outcome reporting portion of the modeling process. Traditionally
built simulation models repm't various aggregate and primarily tactical measures, which
comprise essentially all of the data captured by the model. The approach proposed in this
research however, has available the more complete detailed data represented by the ER
tables in Level 3. In his original methodology, McCarthy (1979) did not specify a method
of data storage but rather left that to the reader’s choice of database systems. In this
research, a relational database was chosen as the means of data storage and processing. It
is within the database that all shop floor data from the simulation model runs were
transferred and where both traditional measures and more enhanced and descriptive
measures and information were derived. The ER model in ﬁgure 9 formed the basis of the
relational database implementation. Figure 13 shows the complete relational database tables
of the model.

It should be noted that the tables shown in ﬁgure 13 do not include all relevant
posted foreign keys as would normally be expected for one-to-many relationships. Only in
several tables (OPERATION, PJOB, WUOB) are foreign keys shown (Emp#, Proj#,
Proj#, respectively). This is done so that the tables more closely resemble those actually
derived from the simulation model. Also noteworthy is the presence of the table
SEQUENCE and the use of its key, Seq#, in the OPERATTON and JOB-WC tables. This
entity is somewhat non-operational in nature, but necessary to distinguish those records in

OPERATTON or J OB-WC that repeat work centers, since such repeats are allowed Thus,

58

illob #101 pr
mords woult
ﬁght 12.

if Job #101 proceeds to WC #2 as its ﬁrst step and later again as its third step, these two
records would be identiﬁed in OPERATTON as shown in the ﬁrst and third row of

ﬁgure 12.

 

Figure 12: Sample OPERATTON Records

Thus, the triple key lam-mm is the primary key for each record.

Figure 14 shows the key database tables that were used in the research with some
actual data from the simulation runs. The method of data collection and table population
will be discussed in the next section.

4.3.4 Phase 2 Outcome

This phase of the research builds upon the ﬁrst, the development of a list of useful
and meaningful measures, and develops the models by which such measures can be
formulated and creawd. The exercise of developing the semantic data model allowed for
the unambiguous deﬁnition of each relevant entity of the shop ﬂoor and, as a result, the
building of an accurate simulation model and one that will be able to generate the
disaggregate dataset that will be required in further phases of this research.

59

JOB (ML
00
P301
Wt
QRJ
MLITI JC
PRC
W1 l
we (m
OPERAK
SEQUENt
JOB-WC (
OPERATI
SOURCE
RM ORDI
RM RECE
with
CS
OF]
M
£83

 

\Bc

 

 

JOB (1912i. Job_Type, J ob _DueDate, Total_PT, Time_in)
CJOB m
PJOB am, Proj#*)
WIJOB (Joni, Proj#*)
QRJOB (19bit)
MULTI JOB (Emjﬂ, N um_Jobs, PTtotal, Time_In, Emp#*)
PROJECT (Proﬁt, Num_Jobs)
WI (Emit, Num_Jobs)
WC m Name, Num_machines)
OPERATOR-WC W, m
SEQUENCE (Emit)
JOB-WC (12bit. m Seat, Time_Into_Queue)
OPERATION (lleit. m Seat, Process_Time, Emp#*, Time_in)
SOURCE (Scum Name, Address)
RM ORDER (Eli, Source#, Job#, Time)
RM RECEIPT (m, Time)
EMPLOYEE W. Name, Address, SSN, PayRate)
CS EMP (Elm!!!)
OPERATOR (EM)
PROCESS PLANNER (Empﬁ)
EST EMP (EM)
BCHMAN (Empﬁ)

 

Figure 13: Relational Database Tables

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

JOBS MULTIJOB

mg Job_Ty Job_DD Total_PT Time_in ' Num_Jobs PT__Total Time_in
1 2 1335.39 333.50 0-04 1 1 123.56 0.00
2 4 25.23 500.50 1.24 2 1 355.55 2.56
3 3 722.00 320.00 2.00] 3 1 1256.30 3.34
4 2 1393.66 312.50 2.54 4 1 23.56 7.63
5 3 1032.94 516.50 2.94] 5 1 1256.35 3.96
6 2 335.10 327.00 3.34] 6 1 2567.30 11.23
7 4 101.12 163.50 5.12| 7 1 763.50 12.54
3 4 30.40 132.00 6.4d 3 1 673.50 20.13
9 3 726.72 499.50 6.721

0.103 PJOB ' PRQECT

3

5

9

12

13

20

59

61

OPERATION

1 6.0

10 3.0

13 30.0

14 1.0

11 ‘ 3.0

14 1.5

14 2.5
JOB-WC

m m 5m Time-

intoQ

1 6 1 17.09

10 3 1 17.09

13 6 1 17.14

14 s 1 17.19

11 6 1 17.21

14 4 2 13.19

14 5 3 19.69

 

 

 

 

 

 

Figure 14: Sample Database Tables

61

4.4 Si

later anal)

figure 15.

4.4.1 Dz

lcannes, i'
relational.
Speciﬁed]
This was t
the Simula
through m
ijn 111
record (ﬁg
Sl01911 in m
&kah
111111115 p(
Qt be Cap 1
“Wing 1
“it
pmcess lim
”0111 96mm

4.4 Simulation Model Operation and Data Collection (Phase 3)
The third phase of the research uses the simulation model to generate the data that is
later analyzed by the laboratory participants. The overall parts of this phase are shown in

 

 

 

 

 

ﬁgure 15.
Run Simulation Model Data Collection Measures/Reports
Generate/Capture * Populate Materialization *"
Data Database

 

 

 

 

 

 

 

 

Figure 15: Data Generation and Capture (Phase 3)

4.4.1 Data Generation

Although the simulation model used in this research has many conventional
features, its main purpose was to generate data for later analysis. The data shown in the
relational database tables in ﬁgure 14 were obtained from the simulation model, which was
speciﬁcallydesignedsothatthedatarequiredbythe semanticmodelcouldbecaptured.
This was done primarily by attaching attributes to the various items, such as JOBS, within
the simulation model and then reading the values of those attributes as the items move
through the shop. For example, the attributes Job#, Job_Type, J ob_DueDate, Total_PT,
Time_in were assigned when the job was introduced to the shop, and make up the JOB
record (ﬁgure 13). The work centers to which the job is routed also are assigned, but
stored in an internal array until such time that they are needed and recorded (in JOB-WC).
So while JOB-WC represents a routing table whose data are set at the beginning of a job’s
run, it is populated only drning the run, so that other attributes such as Time_Into_Queue
can be captured. Storing data initially in an array rather than as attributes avoided
exceeding the simulation software’s limitation of 15-20 attributes per item (Job).

When a job is processed at a work center, the job number (J ob#) is read as is the
process time for that speciﬁc work center (Process_Time), actual clock time (Time_in),
work center number (W C#) and employee number (Emp#), among others. These data are

62

mended i1
all operatit
OPERATI
mm of the
while OPE
difference
while OPE
operation.
data collec

recorded in the OPERATTON record. This allows for a complete disaggregated record of
all operations to be captured. Note the subtle distinction between JOB-WC and
OPERATION: not all jobs that enter a work center’s queue will be processed during the
run of the model. Thus, JOB-WC captures all jobs routed to a particular work center,
while OPERATTON captures only those that leave the queue. This can be seen by the
difference in the “time” attribute for each entity: JOB-WC captures Time_Into_Queue,
while OPERATTON is concerned with Time_In and Process_Time at the work center
operation. Figure 16 is a schematic representation of the model showing the approximate

 

 

 

 

 

 

 

 

 

 

 

 

datacollectionpoints.
' ----- "I
I Source '
o. _____ | _.-.l_-,
l"J""’ I Employoo'
IRMOrder: .. ..... 1
I
nl m
l"""""‘
1RMRocolptl
I ----- -1
|WorkContorl
Inn—cpl..--
__ l
11.4
W
— I—--l---. .---l---.
I Job-WC : 1 Operation l

 

Figure 16: Simulated Shop Floor Model with Data Collection Points

63

arrays tha
maicl's r1
naterializ
software.
equipped ‘
system 7.1
Expressm
sane hard
4.4.1.1
Be
c1perimen
531156. 501
Inthhtnisr.
rtSit-211th it-

My stat

The information generated in the simulation model was stored internally in text ﬁle
arrays that could be transferred to the relational database for further processing after the
model’s run. It is within the database that the chosen performance measures were
materialized and reported. The simulation model was written in ExtendTM (1995) simulation
software, version 3.2 PPC, and was run on a 75 Mhz Power Macintosh 7200 computer
equipped with 32 MB of RAM and 500 MB of hard drive space. Macintosh operating
system 7.5.3 was used. The database software chosen in this research was Helix
ExpressTM (1994), version 3.5, developed by Helix Technologies, Inc, running on the
same hardware platform.

4.4.1.1 Simulation Operation

Because of the simulation model’s primary role as a data generator, the
experimental factors in the model were not analywd per .se in the traditional statistical
sense. Some factors, speciﬁcally the moving bottleneck and the irregular order release
mechanism, were, by design, non-stable in nature. In fact, one of the purposes of this
research was to test the usefulness of disaggregate data in identifying conditions when non-
steady state behavior may be more common. For this reason, steady state was not achieved
in all cases nor was it a prerequisite for data collection. But, attainment of steady state was
attempted to avoid introducing a potential confounding factor into the research.

To avoid start up conditions as much as possible, the model was allowed an
initialization period after which data were collected for the equivalent of approximately 100
days of operation for each condition. This time period, which is approximately three
months or one quarter of a year, was felt to be adequately representative of a period of
operation a manager may review in an actual manufacturing setting. Only one run of the
model per condition was made due to the nature of the measures being collected. Since
large disaggregated datasets (long tables) were being generated in each run and those data

were used to create not only point estimates (means, variances) but also distributional and

1
ﬂat
1

I’.

gene:

3139
4.4.3

and l
JOBS
tithi:
the ct
data 1

hen-o:
Ronni

Job’s '

e) at
As shc
the or},

W :

“roar;

We kg

time series information and graphs, the concept of taking the means of multiple runs did not
apply in this case. However, common random numbers were used to assure that the
general shop parameters were unchanged across all of the conditions. This included the
type and ﬁequency of jobs generated, interarrival rates and process times.

4.4.2 Data Collection and Database Population

Although data were collected for most of the entities and tables shown in ﬁgures 9
and 13, the primary sources of data for analysis were just two entities and one relationship:
JOBS, OPERATTON and JOB-WC. The method by which these tables were populated
within the simulation model for transfer to the database is outlined in ﬁgure 17. Note that
the circled numbers 1, 2 and 3 in ﬁgure 17 refer also to the points in the model at which
data were collected. as shown in ﬁgure 16.

As jobs enter the shop/model, data are generated and captured in the JOBS table,
hence rthD in ﬁgures 16 and 17. Also shown below the Job attributes in ﬁgure 17 is the
Routing Array, which in addition to storing routing information, is used to calculate each
Job’s total process time (Total_PT').

As Jobs enter a work center’s queue, data are again generated and captured in step
Q) at JOB-WC. Since this is a relationship table, it uses the primary keys of other entities.
As shown, the attribute Job# comes from the JOBS entity, while WC# comes from one of
the other entities, WORK CENTER The latter entity is unchanging for the purposes of
this model, although additional work centers could be added to the database as needed in an
actual situation. Seq# is the key of SEQUENCE, a non-operational entity whose purpose
is to allow for repeat visits of Jobs, as discussed in the previous section. The only non-key
attribute in JOB-WC is Time_into_Queue, which is generated by the model’s internal
clock.

The third primary entity involved in data collection, OPERATION, has the same
triple key as JOB-WC, but captures different information at step 6). Process_Time is the
actual time the Job spends at the particular work center and is obtained ﬁom the Job’s

65

Routing
urclun
Ernpi‘l
The en

OPERI

the sho

Routing Array. Emp# (employee number) is also captured at this time from the other
unchanging entity, EMPLOYEE. The Job “grabs” the ﬁrst available employee (a unique
Emp#), availability being determined both by shift and by employee idleness at the time.
The employee pool in the simulation model is captured by the many-to-many relationship,
OPERATOR-WC, in the data model.

Thus steps (D, Q) and 6) indicate not only the progression of each Job through
the sh0p (ﬁgure 16), but also the method by which data are created and captured, while
maintaining their interrelatedness (ﬁgure 17), in accordance with the data model.

gasp no...

_ L _
518.: 85 E «can as Tau... 01518.: ﬂaw as _Eq.. 518.: 818.. _E1.o.£ 2518.. Ha...
a

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

‘ or T¥ascs>_§ _.E

.52 8.53.
28.3 512.: l _.I_
28.9 518.:

 

 

 

ego co £2.93 .35
1 £382.. 28.3 3301312.: .8548. 28125 8..
as :4. .86. .8» at o; .o E38133»
. 252 83 A11 oEoz\\1..o.s 4/ 3.3.3.... 8:18..
are 32.4\ i 3; 32. 1. 1.3583 32.

 

 

 

 

mgodzm 205.53 553 Vac; 0.5.8... 80..

_ O _ _ O _ cmEmo _ \./ _ 1
36:5 2055.6 8..
— 1 1 \ _ van—03 L1 < — 1

f _ a

88
mozmamm

 

 

 

 

 

 

     

 

 

 

 

6

Figure 17: Creation of Main Entities' and Relationship's Data by Simulation Model

67

4.4.3

013411

relation
pruluee
he and
llicintc

umut

attribute

Writer

.111: will

4.4.3 Measurement Materialization

The disaggregate measures produced for the lab experiment were all distributional
or graphic in nature. Deriving these measures/mformation sets required the use of bOth the
relational database and a spreadsheet. Various entities were queried within the database to
produce the values required. These values were then “dumped” from the database as a text
ﬁle and reloaded into a spreadsheet, which was Excel version 5.0 for the Power
Macintosh. This was done to take advantage of Excel’s superior graphics and
computational capabilities.

Asummaryofthemeasures thatwerematerializedﬁomthedatabaseandthe

attributes and entities necessary for them are shown in ﬁgure 18.

 

 

 

 

 

 

MeaSure Attributes Enti /Relationshi
Time in system Time_in JOBS
Time_out OPERATTONS
Lateness Job_DueDate JOBS
Time_out OPERATIONS
Tardiness J ob_DueDate JOBS
Time_out OPERATIONS
DD Allowance JobDD, Time_in JOBS
Qreue Length Q_length J OBS-WC
Jobs process/day Time_in JOBS-WC
Time_out OPERATIONS

 

 

 

 

 

Figure 18: Measurement and Entity Summary

Due to the importance of the JOBS entity and the Time_out attribute in the
management of the simulation data, as evidenced by their prominence in ﬁgure 18, their
role will be discussed in general terms in the following section. For a more in—depth
description of the method by which the Helix® database and Excel derived the disaggregate

measures, please refer to Appendix A, sections A.9-A.11.

68

4.4.3

thtjo
Value
ithTy
4111111

11151

drill-'13.:

4,113.;

4.4.3.1 JOBS Entity Measures Materialimtion

The JOBS table consists of the following attributes:

Job# IJob=Iype |Job=DD |Total_fPT |Trme_in
I | I |

In addition to these attributes, another value was calculated within the database,
Time Completed, or Time_out. This was the time that the Job completed its last operation,
and was derived by querying the OPERATTON entity within the database.

As can be seen in ﬁgures 13 and 17, the OPERATTON entity contains the attributes
Time_in, which is the time at which the Job enters a work center, and Process_Time,
indicating the elapsed time the Job will spend in the work center. Thus, to determine the
time of completion of each Operation, the following simple addition is required:

Time_out = Time_in + Process_Time

Of interest is the Time_out of the ﬁnal Operation of the job, which indicates when
the job is complewd. To retrieve this value, OPERATION is queried for the maximum
value of Time_out for each job. This corresponds to the time that Job is complewd. The
query for this value originates in the JOBS relation. Once this is done, the following
attributes/values are available in the JOBS relation and are converwd to a text ﬁle for

transfer to Excel:

J ob# [Job___Type |Job=DD ITotalrgI‘ |T”rme_in Time_out
I | I I

Note that Time_out is not a stored attribute of JOBS, but rather is procedurally

 

derived, as just described
4.4.3.3 Additional Entities and Relationships

As discussed in Appendix A, there were some additional entities and relationships

built into the database that did not appear in the data model of ﬁgme 9. These were put in

69

11111111
aniDl

ttw it!

the database for ease of implementation and computation. These additional entities are RUN
and DAYS and the relationship is DAYS-WC. Figure 19 shows the placement of these
new items relative to the original data model with their names italicized.

7O

 

 

 

 

 

 

 

 

Figrre 10

 

 

SEQUENCE

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Figure 19: Data Model with Implementation Additions

71

relatil

esser
RD
vith

lClll

int

As can be observed from ﬁgure 19, the addition of the new entities and
relationships changes the nature of the data model somewhat. OPERATTON and JOB-WC
now have quadruple keys, rather than merely triple keys. The new key for both of these

tables is as shown'below:

Run# is necessary to track the proper data within the database, although it is not
essential to the conceptual model of the shop ﬂoor system. Also note that, as a result of the
RUN entity, there is a new relationship, JOB-RUN. This relationship is set up as a table
within the database and it is within this relationship that all Job related information is
actually stored. Thus, the Jobs record appears as the following:

Rani Job_Type |Job=DD |Total=PT |Tirm_in
I I I | I

This allows querying and compiling of the correct Job data for the run (condition)
in question. OPERATTON and JOB-WC are similarly revised.
AnOther new relationship, DAYS-WC, due to the addition of the DAYS entity, is

 

also shown in ﬁgure 19. This relationship also uses Run# as part of its key, for the same
reason as those of the other entities. The use of this triple key can be seen in the
query/calculation for average daily queue lengths in ﬁgure 67 of Appendix A. It is within
this relationship that per day data regarding queue lengths and work center activity are
compiled. Figure 20 shows the relational database tables for the new entities and
relationships and those directly affected. Additional attributes to existing tables are shown

in italics.

72

 

OPERATTON (1912i, m Seat, M Process_Time, Emp#*, Time_in)
JOB-WC (but, m, 3m, Rm Time_into_Queue)

DAYS (Dam

RUN (Emit)

DAYS-WC (12% m. Rant)

JOB-RUNngﬁ, M Job_Type, Job _DueDate, Total_PT, Time_in)

 

 

 

Figure 20: Database Tables for Additional and Revised Entities and Relationships

4.4.4 Hypothesis and Proposition Evaluation

This portion (Phase 3) of the research considers proposition #1 and #2.
Proposition #1 is evaluating the differences between two different sets of performance
measures while PTOposition #2 merely looks at the possibility of materializing certain
measures “through the use of a database enhanced simulation model and its resulting
disaggregated dataset.” Given the nature of proposition #1 and the use of only one run of
the simulation model to generate data, formal statistical testing was not done. Measures
derived conventionally (aggregate measures ﬁom the simulation model) were compared to
those derived ﬁom a disaggregate dataset, but their differences were not tested statistically.
Similarly, for observational purposes, those measures deemed useful by the experts in
Phase 1 were materialized from the database. This exercise endeavored to answer the
question “Can the measures so desired be created for use?” The expected result of this
question is represented by Proposition #2.
4.4.5 Phase 3 Outcome

Phase 3 exploits the semantic and simulation models developed in Phase 2 and
implements them to produce the measures neemd by the lab participants in the next phase.
This phase includes the actual running of the simulation model and the generation of the
disaggregate dataset. In addition, it includes the transfer of the data from the simulation

73

theclttt‘
selectio'
11111111
611111
4.5

111111

45111

11111111

11110

“I
the

model to the relational database and the materialization of the disaggregate measures in both
the database and the spreadsheet. Certain implementation issues are resolved, such as the
selection of the proper software environment in which to perform the materializations
(database or spreadsheet) and the addition of several entities and relationships to the
original semantic model.
4.5 Model and Performance Measurement Evaluation

This ﬁnal portion of the research involved the use of laboratory participants to
evaluate performance measures and information of both types (traditional, aggregate and
disaggregate—based) to determine if one allows users to more readily identify shop ﬂoor
problems and conditions. The overall steps are shown in ﬁgure 21:

Pilot Sessions Training
$292233; Instrument ' Stnrctured Data Analysis
Validation Lab Experiment

Figure 21: Performance Measures Evaluation (Phase 4)

 

 

 

 

 

 

 

 

 

 

 

 

 

The populated relational database was used to create reports with various
information and measures to evaluate the simulated shOp’s performance. After the creation
of the performance information, it was provided to participants in a controlled laboratory
experiment for their evaluation. The results of this experiment were then used to analyze
the quality of certain information in identifying the inserted shop ﬂoor conditions.

4.5.1 Lab Session Instrument

After data from all simulation runs was collecwd in the relational database, it was
compiled into two general forms. Information presented in the traditional aggregate form
was prepared as was information derived from the disaggregate dataset. The two
instruments, copies of which are shown in Appendix D, were presented to the two groups
of experimental participants. The following sections describe the various parts of the

instrument— the conditions, measures and questions.

74

4.5.1

4.5.1.1 Aggregate Group
Figure 22 summarizes the aggregate measures provided to this group of

 

 

 

 

participants.

Means: Provided

Time in System Mean, standard deviation
Job Lateness Mean, standard deviation
Job Tardiness Mean, standard deviation
Due Date Allowance Mean, standard deviation
Queue Lengths (by WC) Mean, standard deviation
Machine Utilization Mean percentage

 

Figure 22: Information Derived from Aggregate Dataset

The following describes how the above measures were deﬁned and calculawd:
Wm: Measured as the difference between the time a job enters the shop
ﬂoor, including the order release pool, to the time it exits its ﬁnal inspection center.
It includes work center queue waiting time, if any. Measured in hours.
W: The amount of time a job varies from its assigned due date. Lateness
may be negative for early jobs, or positive for jobs actually behind schedule.
Measured in hours.

W: The amount of time a job is actually behind schedule. Jobs ahead
of schedule are not counted in mean tardiness. Measured in hours.
W: The amomt of time a job is allowed to ﬁnish. This is
assigned when the job enters the shop and represents the total time that job is
expected to be on the shop ﬂoor, including queue time. The time a job enters the
shop plus its due date allowance equals its due date. Measured in hours.
W: Measured by the number of jobs waiting. There is a separate value

for each work center with a measurable queue.

75

W: Represents the proportion of time the work centers are busy
with operations. Shown as percentage.
4.5.1.2 Disaggregate Group

Figm'e 23 summarizes the disaggregate measures provided to this group of

 

 

participants.

Measure Provmd'

Time in System Distribution
Lateness, all jobs Distribution
Lateness, project jobs Distribution
Tardiness Distribution
Due Date Allowance Distribution
Queue Lengths (by WC) Time series
Jobs Processed per day Distribution

 

 

 

Figure 23: Information Derived from Disaggregate Dataset

Most measures above are derived in the same manner as those in the previous
section, but their manner of presentation is different. Time in system, lateness (all jobs and
project jobs), tardiness and due date allowance are presented as histograms so that the
participants can see their actual distribution. Queue lengths are shown in a time series so
that the participant may see the change in lengths over time for all work centers. The ﬁnal
measure, jobs processed per day, is also shown as a histogram and is a distribution of the
number of jobs processed per day of operation. For an in-depth description of how these
measures were created, please refer to Appendix A.

4.5 .1. 3 Condition Identiﬁcation

Various questions were asked of the participants on the instruments, the ﬁrst one,
question 1, being the actual identiﬁcation of the inserted condition. Six possible conditions
were listed, plus the possibility of no condition having been inserted into the base case.

76

Then

the le

condi

infor

13h)

que
lath
3.5.

The participants were asked to provide a ranking for each possible condition, with a “1”
ranking indicating that which they felt was the most likely condition and “7” representing
the least likely. Thus this question entailed assigning a rank to each possible condition.

Question 2 asked the participants to indicate their degree of conﬁdence that the
condition identiﬁed as most likely (“1” ranking in question 1) is correct. Reponses could
vary ﬁom 0 to 100% conﬁdence.

Question 3 asked the participants to indicate, through a rank ordering, which
information sets were used in answering questions 1 and 2. The participants indicated a
rank ordering of the measures, for only those actually used.

In addition to providing answers to these questions, the participant was to indicate
the starting and ending time for identifying each condition. Starting time represented when
he/she started to review the information and measures; ending time was after question 3
was answered.

Questions 1 and 2 above and the reported starting and ending times formed the
basis for the lab sessions’ performance measures, as described in greater detail in
section 4.5.3.4
14.5.1.4 Identiﬁcation Process

There were additional questions asked of the participants that attempted to determine
why certain information was more useful, how the information was used and what
additional information, if any, would have been useful in identifying the conditions. These
questions and their answers, however, were not included in the analysis for this work but
rather may form the basis for future research.

4.5.1.5 Clarity of Information Presentation and Training

Each participant was asked to provide answers to eight follow up questions after the
experimental session ended. These questions were asked in part to determine if any

difference in performance of the two groups was due to inadequate training or lack of

77

understanding of the shop and its conditions. Also, the follow up questions addressed the
issue of adequacy of the information and the nature of its presentation (tabular versus
graphical). Answers were given as numbers on a scale from one to seven indicating strong
agreement to strong disagreement with a given statement. Five of the questions dealt with
how clear the information was presented and how useable it was. Three questions dealt
with how descriptive the information was or how useful it was in understanding the name
of the shop floor and its conditions. The questions were the same for both information
type groups, with one minor exception to be discussed later. Appendix D includes a
complete copy of this questionaire.

4.5.2 Instrument Validation and Pilot Sessions

After preparation of the instruments, a validity check was performed using four
Michigan State University Operations Management doctoral students. These students
attended a training session in which the modeled shop ﬂoor was described and the
performance measures and information were explained. The students were free to
comment on the content and presentation of the experimental protocol. The purpose of this
trial session was to check the training presentation for completeness and to obtain
preliminary fwdback prior to the actual sessions.

After the trial training session, one Ph.D. student worked through the actual
condition identiﬁcation exercises prepared for the experiment participants. The primary
researcher was in the room at all times during this exercise to answer questions and clear up
any confusion. The purpose of this session was to identify any ambiguity in the
instruments and to determine if any conditions were undetectable. This session provided
valuable feedback into the presentation of the material and also revealed several useful
points of clariﬁcation within the instruments. After some discussion, it was agrwd that
none of the conditions were completely intractable and that the instrument could proceed to
the next phase. A

78

4.1

W1

Ch

4.5.3 Laboratory Experimental Sessions

Participants for the experimental sessions came from two different sources. Some
were recruiwd from various classes in Michigan State University’s (MSU) Eli Broad
Graduate School of Management. Other participants were from an MBA level Management
Information Systems class at Lake Superior State University (LSSU), where the primary
researcher teaches. Participation of the MSU students was voluntary and had the potential
of being rewarded according to a compensation scheme to be described the next section.
Participants from LSSU were students in the researcher’s class and took part in the
experiment as an in-class assignment. Approximately 23 of the total participants were
MSU students, while 1/3 were from LSSU.
4.5.3.1 Recruitment and Incentitives

Students at MSU were approached in their classes, with the permission of the
instructor, and read the approximate text as shown in Appendix C. The voluntary nature of
the experiment was stressed throughout the 5- 10 minute presentation. Students were given
the opportunity to ask questions of the experiment both during the presentation and
afterward via Email or telephone with the researcher. Students from one MBA level class
were informed by their instructor they would be awarded nominal participation points in the
course in return for volunwering.

Participants in the experimental sessions were given the opportunity to win various
prizes, based on performance. For two of the three performance measures for the sessions
(Gap and Time, section 4.5.3.4), participants in each group were ranked separately. In
Other words, there was a different “curve” for each of the two groups. The top performers
in each group were awarded 11 points and the bottom performers 5 points.

After the sessions were completed. each participant earned a number of tickets, or
chances to win, equal to the number of points he/she earned. These tickets were placed in a
bowl and three winners’ names were drawn by a Marketing and Supply Chain Management

79

department secretary. The ﬁrst name drawn won a new “mountain” bicycle, worth
approximately $600. The second, third and fourth names chosen won cash prizes of $200,
$100 and $50, respectively. These prizes were fully explained to the students during the
recruitment presentation. Thus, all participants had a chance to win the prizes, although
those with better performance had a higher probability of winning.

Six separate experimental sessions were conducted- three with traditional aggregate
measures (ﬁgure 22) and three with alternate, disaggregate-based information (ﬁgtne 23).
There were a total of 53 participants, roughly evenly divided between the two
information/measure types. More details on the composition of the sample are given in
section 5.4.1.
4.5.3.2 Lab Experiment Structure

Each experimental session, regardless of the information type, was conducted as
outlined in ﬁgure 24.

 

 

1. Introduction and presentation of objectives.
2. Description of shop floor environment.
3 . Description of possible operational conditions.
Total time 41 -3), approximately 45 minutes
4. Break
5 . Presentation of example condition identiﬁcation with measures.

6. Example of additional condition- participants solve. General discussion
and presentation of correct answer afterwards.
Total time (5-6), approximatelL30 minutes
7 . Participants work on four conditions on their own. Randomly assigned
order.
Total time for entire sessioril -7), 2 to 2.5 hours

Figure 24: Experimental Session Outline

 

 

 

 

The purpose of the training session and the ﬁrst two condition identiﬁcations (steps
1-5) was to familiarize the participants not only with the environment, but also with the
process of identiﬁcation. Working through an example for them (step 5) and then

80

discus:
501m;

work 2

const
whit}
Thus.

lone

pair.

discussing their work after one task (Step 6) helped to introduce them to the problem
solving techniques necessary for this task. By the time they moved to step 7, their own
work and identiﬁcation, they were all equally prepared.
4.5.3.3 Lab Sessions’ Experimental Factors
The primary factor in this experiment is the type of information being presented. In
considering Research Question #2 (section 3.2), the main point of interest is the extent to
which one type of information is “more useful . . . in identifying operational problems.”
Thus, performance based on two levels of one factor- information type- will be compared.
In addition to the information type data, other facts, such as participants’ age, sex,
grade point average and previous manufacturing experience was also collected. This was
done so that the effect, if any, of such demographic factors on the participants’
performance could be evaluamd. Figure 25 summarizes the main factors and covariates of

the analysis.

 

Main effect:
Information Type (AgEgate vs. Disaggregate- 2 levels)
Covariates:
As:
Sex
Grade point average (GPA)
Manufacturing experience
Year in college
Figure 25: Analysis of Covariance Factors and Covariates

 

 

 

 

Age, sex, year in college and GPA were self reported. Year in college was given a
value from one to ﬁve for the years freshman (1) through graduate (5). Since all graduate
students were either in the ﬁrst or second year of an MBA program, no further distinction
was made. Manufacturing experience was based on a listing of manufacturing relawd jobs
or experiences the participant reported having. For those participants with non-zero

experience, their descriptions of their experience were reviewed by two MSU Ph.D.

81

students (other than the primary researcher) and given a ranking from one to ﬁve. The
rankings assigned by the two reviewing students were then averaged.
4.5.3.4 Lab Experiment Performance Measures

The performance of the participants in the laboratory was gauged using three
different metrics. ‘
4.5.3.4.1 Gap

Gap is deﬁned as the ranking given to the correct condition less one. For example,
if the correct actual condition was a stationary bottleneck and the participant provided a
ranking of “3” for that condition, his/her gap for this question would be 3-1=2. In this
instance, the participant’s #1 rank was something other than the actual condition, so the
choice was incorrect, resulting in a non zero Gap. If, on the other hand, the participant
gave the stationary bottleneck a ranking of “l” in this question, he/she would have made
the correct choice, judging the actual condition to be the most likely condition (#1 rank).
This would result in a Gap of l-l=0. Thus, a lower gap indicates a more accurate
response. By capturing Gap in this manner, not only are “right” and “wrong” answers
apparent (Gap = 0, Gap > 0, respectively), but the results are in a variable scale, greatly
facilitating statistical analysis.
4.5.3.4.2 Conﬁdence

Conﬁdence was given by the participant in the answer to question 2 on the
instrument. This measure ranged from 0 to 100% and was not objective, but rather was the
participant’s own opinion of his/her performance. Since this is also a variable scale,
statistical analysis was facilitated.
4.5.3.4.3 Time

Time, or efﬁciency, was indicated by the time taken for the participant to analyze
the information provided and provide answers to questions 1-3. Participants were asked to

indicate their beginning and ending time for each condition identiﬁcation and their total time

82

was determined by subtracting one from the other. Participants were instructed to use one
timepiece for this consistently, since it was not as important to note actual clock time as it
was to calculate elapsed time. Times were always given by the participants to the nearest
minute and thus this measure has some inherent imprecision.

It isrecognized that this is a selfreporwdmeasure, but the alternative was tohave
the researcher physically receive each participant’s complewd instrument and note the time
himself. It was felt that this would have entailed excessive walking around the lab room
and shuﬂling papers and would be too distractive for the participants in return for little

improvement, if any, in accuracy.

4.6 Data Analysis

The data analysis methodologies that address Propositions #1 and #2 are relatively
straightforward and do not require any additional discussion beyond that already provided
in chapter 3. However, the analysis proposed for Phase 4 (Hypothesis #2) is more in-
depth and requires more justiﬁcation, which is provided in the following sections.

4.6.1 A Priori Analysis (Phase 4)

Phase 4 is at the core of this research in that it is evaluating information users’
performance in utilizing the two information types to identify shop ﬂoor problems and
conditions. In evaluating the results of Phase 4’s experiment, several different analyses are
made. First, the demographics of the two different information type groups and the two
different schools from which participants were drawn were compared. This difference in
means was evaluated with t tests and/or Analysis of Variance (ANOVA) and was done to
determine if there were any differences between the groups that may have contributed to
different performance.

Next, performance in the laboratory was evaluated. Since all participants examined
ﬁve conditions, a repeated measures Analysis of Covariance (AN COVA) was the ﬁrst
analysis done to evaluate the a priori hypothesis (11,“). Each separate performance

83

measure (Gap, Conﬁdence, Time) was the repeated measure. AN COVA was chosen over
ANOVA since it allows for comparison of mean performance but also allows for the
inclusion of covariates (Keppel, 1991).

A signiﬁcant main effect of Type (between subjects), indicates that the type of
information (aggregate vs. disaggregate) had an effect on the identiﬁcation of the condition,
but no one condition was more identiﬁable than any other, regardless of information type
used. If there were only a signiﬁcant main effect for Gap (within subjects), this would
indicate that the condition being evaluated makes a difference in identiﬁcation, but such
difference was not the result of the information type. One or more conditions would be
more identiﬁable than the others, but it would be so regardless of information type used.

A more interesting result is a signiﬁcant interactive effect between information type
and Gap (within subjects). This indicates that some conditions (DR, DD, etc.) may be
more identiﬁable than Others, depending on the information type used. This result would
dictate further post hoc analysis, as discussed brieﬂy in the next section.

4.6.2 Post Hoc Analysis

A signiﬁcant Information Type x Gap interaction warrants additional analysis. An
AN COVA for each individual condition (DR, DD, etc.) would be made for the speciﬁc
performance measure for which signiﬁcance was discovered. The results found such a
signiﬁcant interaction for Gap and Time only. Further analyses determined for which
conditions there exists a signiﬁcant difference in participants performance.

Although there were three dependent variables (Gap, Conﬁdence and Time), the
results for each variable were analyzed separately, rather than combining them into an
analysis such as Multivariate Analysis of Covariance (MAN COVA). This was done
primarily to keep the analyses congruent with the original research questions. Although
data were gathered on three performance measures, only one of them, Gap, which deals
with the accuracy of condition identiﬁcation, is of paramount interest. This goes back to I

research question #2 which states: “To what extent can a database populated with

84

(A

disaggregated data be more useful in completely describing shop floor performance and in
identiﬁing operational problems that may not be detectable with traditional aggregate
measurements?” The question deals with issues of usefulness and, particularly,
identiﬁcation. The interaction, if any, between the various dependent variables was not an
issue addressed by the research questions, thus the inclusion of a MANCOVA did not seem
appropriate-

4.6.2.1 Distributional Analyses

While the ANmVA for Gap evaluates the differences in mean performance
between the information type groups, it does not provide any indication of how many
correct (Gap = 0) versus incorrect (Gap > 0) identiﬁcations were made by each group.
Determining this may be useful in that one group, say the disaggregate, may have a lower
overall mean gap (er < rtA), while the aggregate group may in fact have more correct
answers. To test this, further post hoc analysis was performed, by way of a Chi square
analysis, to examine if the distributions of correct and incorrect identiﬁcations for each
information group were signiﬁcantly different from each other and from what would be
expecwd randomly.

In addition to the mean and right/wrong performance, the underlying pattern of
responses of the two groups was also of interest to determine if any differences in means
can be attributed to differences in the actual distributions of data. For this comparison, a
Kolgoromov-Smirnov (K-S) test was performed.

4.6.3 Clarity of Training and Information Presentation

The follow up questions asked of the participants (section 4.5.1.5) attempwd to
address the impact, if any, of the presentation of training and information on performance.
Since responses to the questions consist of a scores on a seven point Likert scale of

agreement/disagreement, comparison of mean responses is appropriate.

85

To evaluate the difference in mean responses of the two groups, Analysis of
Variance (ANOVA) was used. Further, to determine if there was a signiﬁcant difference in
the underlying pattern or distribution of responses of the two groups, K—S test was
performed.

4.6.4 Phase 4 Outcome

Phase 4 deals with the determination of the actual usefulness of the different
information types. This usefulness is directly measured by the lab participants’
performance in identifying various operational conditions and by the participants’ ranking
of the various measures used. Thus, this phase of the research acts as a test of the
conceptual data model and the operationalimd simulation model and database developed in
previous phases. After analysis of the lab’s results, it was possible to identify for which
conditions a particular information type provided an advantage in identiﬁcation.

4.7 Potential Limitations of the Research Design

While this work does address the research questions and applies the appropriate
statistical analysis, there are some inherent limitations to its design. These are discussed
further in this section.

4.7.1 Signiﬁcance Level (or)

Cascio and Zedeck (1983) presented several different options to develop the
appropriate power for statistical analysis in research. Two of them, setting one’s sample
size and evaluating prior probabilities, are not suitable in this research. Sample size was
established by the available pool of volunteers for the lab experiment and, by the time
statistical analysis began, was ﬁnalized. Prior probabilities refer to previous researchers
evaluating these same issues in this manner, which has not been done.

When one’s sample size is set and is somewhat limited (the case in this research)
and if the research is of an exploratory nature, Cascio and Zedeck (1983) suggest
increasing the level of probability that will indicate signiﬁcance. Traditionally, the level of

86

choice in such research becomes 0.10, rather than the u’aditional 0.05. Since this research
is exploring issues not previously studied and is basically of a theory testing nature, a level
of 0.10 will be used. This has been shown to be effective in other exploratory research
(Magnan, 1994) and acts to identify effects that may be too weak to appear when the
threshold is 0.05, thus encouraging additional research along the same lines.

4.7 .2 External Validity and the Generalizability of Results

This research is using a laboratory experiment methodology to gain insights into the
way information users in actual settings may behave and perform The cavet “may” is
stressed because it is not a certainty that all results obtained in the lab can be directly
generalized to the real world and, in fact, such generalization may not even be desirable.

This issue of lab-to-ﬁeld generalizability has been the topic of much discussion in
the organizational behavior and indusuial organizational psychology literature. Locke
(1986) discussed at some length the concept of ecological representativeness, that is, how
similarthelabsettingistoarealwm'dsettingandhowimportantthatsimilaritymaybeto
generalizability. He concluded that achieving complete similarity is impossible and, in fact,
unnecessary. What is important, he felt, is that the essential features are similar. These
may amount, for example, to merely the lab subjects’ willingness to try for a certain goal,
as they did in this research. In addition, attempting to make a lab setting identical to a ﬁeld
setting may limit generalizability only to those real world settings that are the same as the
carefully contrived lab setting.

Berkowitz and Donnerstein (1982) also maintain that exact ecological
representativeness is neither achievable nor desirable, maintaining that the meaning the
subjects assign to their situation may be more important to making an experiment’s results
generalizable than'the sample’s demographic representativeness or the setting’s realism
What is important, they maintain, is to use the lab to establish what may happen, as
opposed to what will happen. Signiﬁcant results in the lab should be taken as an indication
of what can happen in a ﬁeld setting, not necessarily of what will happen. This point is

87

also stressed by Mook (1983) who maintains that direct generalizability should not always
be a goal. The laboratory, he states, should be a tool to help us explore the interactions of
various carefully controlled variables to test theory and predictions, not make them.
Despite the deemphasis among researchers of the direct generalizability of lab
results to a ﬁeld setting, there is ample evidence that results of lab and ﬁeld research are
quite similar in several different areas of research. Figure 26 (summarized from Locke,
1986) shows the percentage of the two different types of studies (lab and ﬁeld) in which

 

 

 

 

signiﬁcant results were found:
Percentage of Percentage of
Lab Studies Field Studies
Result Si ' cant Si ' cant
Expectancy theory predicted 80% 90%
job choice
Rater training on accuracy of 74% 80%
rformance a 'sals
Challenging goals lead to 90% 90%
higher performance
Relationship between job 100% 96%
scope and job satisfaction

 

 

 

 

 

Km 26: Summary of Lab and Field Research Results

It can be seen that not only are signiﬁcant results found with both methodologies
but, more importantly, the percentage of ﬁndings across the two methodologies are very
similarly high, indicating a great deal of congruence between what is found in the ﬁeld and
the laboratory.

Concerns have also been voiced that using exclusively students in lab research
limits generalizability. However, Locke maintains that focusing on the differences between
student lab subjects and real world employees ignores many similarities between the two
groups, namely that they are both human beings (1986). Moreover, many employees were
once students and likewise most students will one day be employees (it is hoped). In the
case of this research, at least one-third of the subjects (primarily those from LSSU) were
currently employed. And, although the tasks in the lab and real world settings may be

88

different, they have characteristics in common, such as skill requirements, repetitions and
consequences (rewards).

Since this research is attempting to address several speciﬁc questions and to test the
theoretical assertion that disaggregate information provides more useful information than
aggregate measures, the proposed lab setting seems appropriate. The primary interest is in
what does happen in the lab and how that indicates what may happen in the real world.
Using students to perform carefully deﬁned tasks should, according to previous
researchers (Locke, 1986; Mook, 1983; Fromkin and Streufert, 1976), provide a
reasonable basis for generalizing to what is achievable with actual employees in a real

world setting.

4.8 Summary -
The original research problem, that of the inadequacy of current simulation data and

information management techniques, is being evaluated in the context of a data model-
enhanwd simulation model. Such evaluation has included the development of a conceptual
data model of a shOp ﬂoor, which can act as a template for not only an RDB, but also a
simulation model. Such model development allowed us to specify and capture the
necessary data that can populate an RDB and that can be used to generate performance
measures not available from conventional simulation models. The actual usefulness of
such measures is being determined by participants who, in a controlled lab setting, have the
opportunity to use one of two information types, with their performance compared.

Thus the methodology proposed has a solid basis in conceptual and theoretical
concepts from the IS ﬁeld, and includes some more traditional aspects of the OM ﬁeld.
Although the ﬁrst three phases of the research do not lend themselves to rigorous testing,
the results will be instructive in that they will point to ways in which the data/simulation
modeling mproach may be improved. The fourth phase’s results will be statistically
analyzed and will indicate if information type is a signiﬁcant factor in shop ﬂoor

89

performance evaluation. A graphic outline of the four research phases is shown in ﬁgure
27.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Determination of
Pattormance Phase 1
Measures
l Initial List Experts' Opinion Ranked List
Simulation and Data
Model Building Phase 2
Data Model RDB Design. 4 Smflﬁca'ly Based
' I ementation Simulation "
Desrgn » mpl M ed el
* Simulation SWE'OO'
Model De ' COHdItIOl‘IS
[ 8'9" I lnchrded
Performance Measures
Materialization Phase 3
Run Simulation Model Data Collection ’ Measures/Reports
Generate/Camus ‘ Populate Materialization _”
Data > Database >
Performance Measure
Comparison/Analysis Phase 4
Instrument Pilot Sessions Training
Preparation Instrument Structured "F Data Analysis
f Validation f Lab Experiment

 

 

 

 

 

 

 

 

Figure 27: Research Phases Summary

91

 

5.0 Experimental Analysis and Results

5.1 Chapter Overview

The preceding chapters introduced the research problem, the setting for the
problem, its rationale and justiﬁcation and the research methodology used. This chapter
presents the data, summarizes the analysis, presents the major ﬁndings generated by the
analysis and discusses the ﬁndings.

This chapter is organized into six major sections, including this introductory
section. The second section (5.2) validates the use of non-conventional performance
measures by brieﬂy summarizing the major observations and results gained ﬁom the
interview of expert practitioners. This section provides evidence for the diﬁ'erences
between the aggregated and disaggregated measures that are subsequently evaluated. The
third section (5.3) focuses on feasibility by addressing the question of whether it is
possible to generate a set of meaningful disaggregated measrnes. These measures are
critical since they form the main input to the laboratory experiment.

The fourth section (5.4) deals with the issue of effectiveness as measured by the
ability of the laboratory participants to identify underlying shop ﬂoor conditions given
aggregated or disaggregated measures. This section begins by examining the data for two
potential somces of confounding, those being due to differences in the participant groups
and those due to the distributional form of the laboratory dataset. Differences in
performance means are then evaluated using a repeated measures AN COVA. Also, several
nonparametric tests are performed, including a Kolgoromov-Smirnov (K-S) test of the
underlying distribution and a Chi square analysis of the difference in the distributions of
correct and incorrect identiﬁcations.

The ﬁnal portion of the fourth section deals with the issues of efﬁciency and user
acceptability, as judged by the secondary performance measures of Time and Conﬁdence,
respectively. A repeated measures ANCOVA is performed to compare the mean

92

performance of each group as is as a K-S test to examine the distributions of the responses.
In addition, the ﬁfth section also contains an analysis and summary of ftuther user
acceptability issues, including the participants’ ranbd usefulness of the measures used and
the quality of the data presentation.

The ﬁfth and ﬁnal section of this chapter discusses the results. The more
interesting results are emphasimd and are weighed against expectations and evaluaed in
terms of what they may be revealing about the research questions and hypotheses.
Inferences are drawn and explanations offered for the results. Finally, directions for future

research are presented.

5.2 Determination of Performance Measures

This section veriﬁes tle need or attractiveness of disaggregated measures.
Interviews with two shop floor control experts were conducted. A survey instrument (see
Appendix B) was then sent to these two experts for assistance in determining the measures
and information to be used in the lab experiment. Both experts have more than 30 years
experience and are designated as CPIM‘. The survey consisted of listings of several
measmc/information types with qrestions as to their usefulness. The measure types were
‘ (1) aggregate- traditional simulation based measures; (2) ﬁnancial measures; and (3) non-
conventional measures- information that could be derived ﬁ'om more detailed,
disaggregated datasets.
5.2.1 Expert Selected Measures

The positions of the experts on the relative importance of the measures are
summarized in Figure 28. The experts were given a nine by ﬁve matrix where the nine
rows represented various measures drawn from the literature, and the ﬁve columns
represented dimensions of usefulness. These dimensions were: (1) Consistency with
Corporate Goals; (2) Good Snapshot of Ongoing Operation; (3) Enhances Monitoring; (4)

 

‘ Certiﬁed in Production and Inventory Management by the American Production and Inventory Control
Society (APICS), a nationally recognized association of production management professionals.

93

Problem Identiﬁcation and (5) Problem Diagnosis. For each cell of the matrix, the experts
were asked to assess the usefulness of the measures for that particular dimension on a l - 5
scale, with “1” being not useful at all and “5” being very useful. The values were then
averaged over the ﬁve dimensions to determine the relative ranking for each expert, which
was calculated on a l - 9 scale, with 1.0 representing the most useful measure. Ties in
ranking were broken by averaging. For example, expert A believed that mean queue length
and per job measures were of equal immrtance, so both were assigned an 8.5 value.

 

Figure 28: Expert Survey Summary

* ﬁnancial measures consised of those identiﬁed as important by Goldratt (1990)
and period shop net proﬁt and per job cost and revenue.

“With the exception of A’s top ranking of mean lateness, the experts, for the most
part, gave the traditionally top raed simulation measures- mean time in system, mean
lateness and mean tardiness (Blackstone et. al., 1982)- only average rankings. These
measures were deemed less useful than detailed information on shop floor conditions.
Another conventional measure- mean queue lengths— received very mixed marks, with A
believing that it was the least important of the measures presented (rank = 8.5), and B that
it was almost the most important (rank = 2.0). Consistent with the position taken by
Goldratt (1990), both experts ranked per job measures as the least useful. According to

94

these experts, the most important measures captured information not reported in traditional
simulation research, such as distribution of queue lengths and bottleneck identiﬁcation.
While these ﬁndings are based on a very limited sample, they do provide overall support
for the presence and use of non-conventional disaggregated measures and for the position

that they meet deﬁnite needs experienced in sh0p ﬂoor performance evaluation.

5 .3 Development of Semantically Based Simulation and Database Models

This section focuses on the feasibility of the semantic data modeling approach to
simulation information management. It includes the development of the models and the
materialization of the measures and also explores the issues of the practicality and
reasonableness of this approach. Speciﬁcally, it addresses the ﬁrst research question and
the ﬁrst two propositions:

RQ- 1 . Can an events-based model of a shop ﬂoor completely and accurately

capture disaggregated shop floor data in a manner that allows for the

efficient transfer of that data into an events-based database for information
storage and presentation? '

PRP- 1: The means of conventionally (non-semantically) derived performance
measures are comparable to the means of the Same measures that have been
derived from a semantically based disaggregated dataset. ‘

PRP-2: It is possible to materialize useful measures through the use of a database
enhanced simulation model and its resulting disaggregated dataset.

5.3.1 Expected Results
It was expected that the actual value of a performance measure would be

independent of the method of derivation, indicating support for PRP-l. It was also
expected that a semantic database approach to data management would be able to materialize
measures determined to be useful by shop ﬂoor experts, indicating support for PRP-2. '

95

5.3.2 Measurement Derivation Results (PRP-l)

Research Question #2 addresses the issue of feasibility, that is, will a database
approach to shop floor information management produce useful results? In developing the
model and accompanying dataset, it was found to be feasible to the extent that data could be
generated and transferred to a relational database. However, PRP-l provides a more
speciﬁc evaluation of feasibility by examining whether performance measures created in a
conventional manner by a simulation model are similar to those created from a
disaggregaed dataset. '

In comparing results from the two forms of information collection, it is important to
note that steady state conditions were not achievable under some shop ﬂoor conditions.
Given the objectives of this study, achieving steady state was not a prerequisite to data
collection, nor was it required for data analysis. However, where possible, efforts were
made to eliminate start up conditions and to achieve steady state. For each shop ﬂoor
condition only one run of the model was made. This was consistent with the purpose of
the simulation model, to generate data for laboratory evaluation rather than statistical
analysis.

Figure 29 shows the three most commonly used mean aggregate measures
(Blackstone et. al., 1982): time in system (TSYS), lateness (LATE) and tardiness
(TARDY), under two different scenarios- as derived conventionally from the simulation
model (Agg) and as materialized from the disaggregate dataset (Disagg). These results are
summarized by condition. The ﬁrst column of each section of ﬁgure 29 indicates the
speciﬁc condition present (DR, DD, etc.) and the time at which the measures were
captured. For all conditions, tie ﬁnal set of observations was taken at approximately 2800
hours. The next set of columns represent each of the three measures taken, TSYS, LATE
and TARDY. In the ﬁnal row of each section, discrepancy indicates the percentage by
which the measure as derived from the disaggregate dataset differs from the measure
derived conventionally, as of 2800 hours.

96

Rule (DR)

 

Due Date (DD)

 

Irregular Release
IR

 

Stationary Bottlenck
S B

 

Moving Bottleneck
MB

 

Figure 29: Comparison of Conventional (Ag) and Materialized (Disagg) Measures

97

In all cases, the ﬁnal discrepancy is very small (< 0.24%). Consequently, it is
reasonable to conclude that there is no practical difference between the differently derived
measures, thus supporting PRP-l. The results summarized in ﬁgure 29 are consistent with
what was observed when studying the actual shop on which the simulation model was
based The mean due date allowance for the jobs in the model was 2091 hours
(approximately 87 days). Although this is high, it must be noted that 50% of the jobs were
Project jobs that had a due date allowance of between 15 and 165 days. In addition,
approximately 38% of the jobs were either Walk-in or Crib jobs, which had due date
allowances of between 15 and 60 days. Only 12% of the jobs, those designated Quick
Response, had relatively short allowances, mostly one day. For all the model’s jobs
combined, the due date allowances are relatively uniformly distributed across their range (1
to 165 days). Under this job mix assumption, one that had been conﬁrmed by discussions
with the shop’s management, these allowances were reasonable and consistent with how
they were assigned on a day to day basis.

While the actual mean total processing time of the jobs was 312 hours (13 days),
more than 50% of the jobs had processing times less than 10 days, while the maximum
processing time was approximately 40 days. Thus the substantial negative lateness (i.e.,
earliness) of the jobs was expected and is reasonable. Discussions with management
indicated that, consistent with these results, the shop was able to complete most of its jobs
early, with the exception of the quick response jobs, which made up a relatively small
proportion of the total. This further supports the model’s being a realistic representation of
the shop ﬂoor being studied. It should also be noted that the multiplier between actual
work content (processing time) and due date allowance is approximately 7.0, which is

consistent with previous work (Christy and Kanet, 1988; Weeks and Fryer, 1977).

5.3.3 Derivation of Experts’ Useful Measures (PRP-Z)
Of the experts’ two most highly ranked non-conventional measures, bottleneck

identiﬁcation and distribution of queue lengths, only the latter was actually derived. The

98

former, bottleneck identiﬁcation, is not a meastue per se, and as such it was left as a task
for the laboratory phase of this research. The measures actually derived and used in the
laboratory experiment were primarily distributional in nature, and consiswd of more than
just queue lengths. However, the derivation one of the experts’ highly ranked measures as
well as other similar measures, offers support to PRP-2.

5 .4 Laboratory Evaluation of Performance Measmes and Information (Phase 4)

This phase of the research required participants to utilize different information sets
to identify the conditions present in a shop floor environment. It dealt with the issues of
quality and utility and addressed Research Question #2 and Hypothesis #1:

RQ—2. To what extent can a database populated with disaggregated data be more .

useful in completely describing shop ﬂoor performance and in identifying

operational problems that may not be dewctable with traditional aggregate
measurements?

110.1 Information and measures materialized from a semantically based historical
dataset do not result in signiﬁcantly different performance by users in
identifying and describing shop ﬂoor operational problems and conditions,
as opposed to using traditional aggregate tactical measures.

As shown above, the null hypothesis maintained that there would be no differences
between the performance of users of aggregate versus disaggregate information. It was

expected that users of the disaggregate information would signiﬁcantly outperform those

using aggregate measures.

5.4.1 Sample Description

The sample of participants consisted of students at Michigan State University
(MSU) and Lake Superior State University (LSSU). Figure 30 summarizes the
characteristics of the overall sample by treatment group. Although the participants were

assigned randomly to the two groups, a t test (a = 0.10) indicated a signiﬁcant difference
in age between the two information type groups.

The sample was analyzed to determine if there were any signiﬁcant differences in
the demographics of the participants between the two schools. T-tests were performed and
no signiﬁcant differences in any of the demographics of the two schools’ students were
found. However, since there still could be diﬂ'erences between the populations at the two

universities, School was used as a factor in all analyses.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Total Male Female Mean Mean Mean Mean Year
(n) (n) Age GPA Experience in College
(year)
Total Sample 53 31 22 28.1 3.53 0.8 4.7
LSSU 16 8 8 30.5 3.46 0.38 4.7
MSU 37 23 14 26.9 3.57 0.93 4.6
Mate 26 16 10 25.8* 3.42 0.94 4.5
LSSU 5 4 1 38.8 3.75 1.30 5.0
MSU 21 12 9 22.7 3.32 0.85 4.4
Aggregate 27 15 12 302* 3.64 0.54 0.48
LSSU 11 4 '1 35.7 3.74 0.87 5 .0
MSU 16 11 5 27.4 3.66 0.31 4.7
Figure 30: Sample Characteristics

5.4.2 Assumptions of Analysis of Covariance

Analysis of Covariance embodies certain assumptions, among them the presence of
normally distributed data and of homogeneity of variance between the sample groups. Of
these assumptions, the homogeneity of variance is the more critical (Neter and Wasserman,
1974; Keppel, 1991).

A repeated measures ANCOVA test was done on the original data to see what

factors and/or covariates were significant, regardless of adherence to the assumptions, and

100

to generate descriptive statistics and residuals, which were used in a Levene’s test (1960)
for homogeneity of variance. The AN COVA showed that Gap interacted signiﬁcantly with
Information Type and Age. Levene’s test was then performed on the residuals of the
ANCOVA to see if the variances among the testing groups were equal.

The Levene test yielded signiﬁcant results (p < 0.002), indicating that the variances
were signiﬁcantly different between the two information types. Therefore, it became
necessary to transform the data to bring the variances into equality.

Several transformations were attempted in accordance with the guidelines
recommended by Neter and Wasserman (1974). Since many of the Gap values were zero
(accurate identiﬁcation), a reciprocal or logarithm transformation was infeasible.
Ultimately, using the results generated by the Cox-Box Power u'ansformation (performed
with SPSS, version 8.0 for Windows), the following transformation was determined to be

effective in correcting the problem:

Y=)6+YY

This transformation brought the variances into equality (p < 0.263; Levene’s test), although
a K-S test with Lilliefors on the transformed data revealed that the assumption of normalcy
was still not met (p < 0.05), similar to the original data. However, since the key
assumption of homogeneity of variance was satisﬁed, it was felt that the lesser important
non-normalcy was not a serious deﬁciency. The transformed data were then analyzed
using ANCOVA. The term Gap’ is used to denote transformed data.

101

5.4.3 Lab Analysis Results
For all AN COVAs in this and the following sections, the abbreviations in ﬁgure 31

 

apply.
DR Dispatch Rule Variation
1]) Due Date Assignment Change
IR Irregular Job Release
SB Stationary Bottleneck
MB Moving Bottleneck
Drsagg Disaggregate Data Measures Group
Agg Aggregate, Traditional Measures Group
LSSU Lake Superior State University Students
MSU Michigan State University Students

 

 

 

Figure 31: Analysis Abbreviations

The means and standard deviations of the original untranﬁ'onned Gap for each of
the shop floor conditions are presented in ﬁgure 32. These values will be referred to later

and are presented to demonstrate the differences in performance between the two groups.

. 9 Experimental GAP ’
Condition Disa A
DR

 

 

 

 

 

 

 

 

 

(1.42) (0.56)
no (92;) (ﬁg
IR 238 ml—
(1.64) (1.36)
SB 0.46 0.44
(1.24) (1.26)
MB 6.96 .
(1.54) (1.90)

 

 

Figure 32: Experimental Results- Mean (Standard Deviation) Values

The repeated measmes AN COVA results for Gap’ are presented in table 33. No ,

main effects were found, although Year in School (Yr) was a signiﬁcant covariate (p <

102

0.078). However, signiﬁcant interactions were found for Gap’ x Type (p < 0.011) and
Gap’ x Age (p < 0035). Consequently, individual AN COVAs were performed for each
shop ﬂoor condition. These are presented in panels A through F in table 34.

103

Between Subjects

 

 

 

Source SS DF MS F P

Type 0.000 1 0.000 0.002 0.964
School 0.062 1 0.062 0.347 0.560

Type x School 0.073 1 0.07 3 0.406 0.528

Age 0.209 1 0.209 1.161 0.289

GPA 0.036 1 0.036 0.201 0.657

Yr 0.595 1 0.595 3.303 0.07 8*

Exp 0.001 1 0.001 0.004 0.951

Error 6.129 34 0.180

Within Subjects

Source SS DF MS F P
Gap’ 0.451 4 0.1 13 0.763 0.551
Gap’ x Type 2.314 4 0.579 3.914 0.005"
Gap’ x School 0.121 4 0.030 0.205 0.935
Gap’ x Type x School 0.915 4 0.229 1.547 0.192
Gap’ x Age 1.752 4 0.438 2.964 0.035"I
Gap’ x GPA 0.122 4 0.030 0.206 0.935
Gap’ x Yr 0.319 4 0.080 0.540 0.706
Gap’ x Exp 0.571 4 0.143 0.966 0.428
Error 20.102 136 0 148

 

Figure 33: Repeated Measures Gap’ ANCOVA
* Indicates signiﬁcance at a = 0.10

104

m1 A, Dispatch Rule (DR)

 

 

 

 

 

 

 

 

_SLource SS DF MS F P
Type 0.849 1 0.819 4.879 0034*
School 0.006 1 0.006 0.033 0.856
Type x School 0.356 1 0.356 2.046 0.162
Age 0.274 1 0.274 1.576 0.218
GPA 0.000 1 0.000 0.001 0.978
Yr 0. 109 1 0.109 0.624 0.435
Exp 0.014 1 0.014 0.082 0.776
Error 5.915 34 0.174
R2 0.338

Panel B, Due Date (DD)
Source 88 DF MS
Type 0.326 1 0.326 1.468 0.234
School 0.001 1 0.001 0.004 0.949
Type x School 0.429 1 0.429 1.933 0.173
Age 0.026 1 0.026 0.1 17 0.735
GPA 0.016 1 0.016 0.073 0.789
Yr 0.015 1 0.015 0.067 0.797
Exp 0.000 1 0.000 0.000 0.991
Error 7.555 34 0.222
R2 0.074

Panel C, Irregular Job Release (IR)
Source . 88 DF MS F P
Type 0.105 1 0.105 1.893 0.178
School 0.001 1 0.001 0.010 0.922
Type x School 0.001 1 0.001 0.017 0.898
Age 0.103 1 0.103 1.853 0.182
GPA 0.096 1 0.096 1.719 0.199
Yr 0.152 1 0.152 2.736 0.107
Exp 0.384 1 0.384 6.911 0013*
Error 1.891 34 0.056
R 0.270

Panel D, Stationary Bottleneck (SB) .
Sauce 8 S DF MS F P
Type 0.109 1 0.109 1.374 0.249
School 0.053 1 0.053 0.663 0.421
Type x School 0.005 1 0.005 0.067 0.797
Age 1.537 1 1.537 19.369 0.000*
GPA 0.002 1 0.002 0.027 0.871
Yr 0.363 1 0.363 4.578 0040*
Exp 0.040 1 0.040 0.508 0.481
Error 2.698 34 0.079
R2 0.425

Figure 34: Gap’ AN COVAs for Shop Floor Conditions

* Indicates signiﬁcant at a. = 0.10

105

Panel E, Moving Bonleneck (MB)

 

 

Source SS DF MS F P
Type 0.925 1 0.925 3.849 0058*
School 0.124 1 0.124 0.5 15 0.478
Type x School 0.196 1 0.196 0.816 0.373
Age 0.022 1 0.022 0.090 0.766
GPA 0.044 1 0.044 0.182 0.673
Yr 0.276 1 0.276 1.148 0.292
Exp 0.133 1 0.133 0.552 0.463
Error 8.173 34 0.240
R2 0.145

Figure 34 (continued)

Information type was signiﬁcant for the dispatch rule (p < 0.034) and the moving
bottleneck (p < 0.058) shop ﬂoor conditions. As shown in ﬁgure 31, identiﬁcation was
better (Gap = 0.35 versus 1.39, original data) for those participants who used the aggregate
measures for the DR condition, whereas participants who used disaggregate measmes
performed better (Gap = 0.96 versus 2.18, original data) for the MB condition.
Information type was not a signiﬁcant factor when participants were presented with the
shop ﬂoor conditions of due date assignment method (DD), irregular job release (IR) and
the stationary bottleneck (SB). Manufacturing Experience (Exp, p < 0.013) was a
signiﬁcant covariate only for the identiﬁcation of IR, while Age (p < 0.000) and Yr (p <
0.040) were signiﬁcant for SB. Separate AN COVAs for the original data (not shown here)
resulted in the same signiﬁcant results for the same two conditions.

An examination of the original data showed that disaggregate measures were better
(lower Gap) in one identiﬁcation (MB), and were equal to the aggregate measures in three
other conditions. Interestingly, the only condition for which the aggregate measures were
signiﬁcantly better was in the situation for which a manager should have the most
knowledge, the dispatch nrle (DR) condition.

The R2 values for the transformed data’s AN COVAs ranged from 0.074 to 0.425.
Given the small sample size, this was not unusual. While this research contrplled for all of

the traditional variables used in behavioral research, other factors were likely present.

106

However, the values obtained are consistent with the results of previous lab experiments

(c.f. DeVader, Bateson and Lord, 1986; Podsakoff and Williams, 1986; Stone, 1986).

5.4.4 Differences in Distributions

In addition to the means comparisons of the AN COVA, the distribution of the lab
responses was analyzed through the use of the K-S test. This supplements the AN COVA
in that it tests the assumption of equivalently distributed datasets between the two
information types.

Figure 35 summarizes the K-S analysis of the transformed dataset’s two
distributions’ ﬁt with respect to each other. The distribution of responses between the two

groups were signiﬁcantly different only for DR.

 

 

 

 

 

 

 

 

Condition GAP’
DR 0.429’
(0.035)“
DD 0.095
(1.000)
IR 0.238
(0.546)
SB 0.095
(1.000)
MB 0.238+
(0.546)

 

* Indicates signiﬁcance (KS) at a = 0.10
’ Indicates signiﬁcant difference in Repeated Measures AN COVA

Figure 35: Kolmogorov-Smirnov Test Results for Condition Identiﬁcation. Maximum
difference (signiﬁcance level)

In addition to the K-S test, a Chi-square test was performed to examine the
distribution of those answers considered “correct” and “incorrect.” Since Gap varies from
zero to six with only one technically “correct” answer (Gap = 0), the deﬁnition of “correct”
was expanded to include those responses where the correct condition was given as the ﬁrst

or second choice. The distribution of correct (0 S Gap S 1) and incorrect (Gap 2 2)

107

answers for each condition was then tested against what would be expected due to

randomness. Figure 36 summarizes the coefﬁcient values calculated

DR IX)
.3* 0.

 

10.

Y
* Indicates signiﬁcance at a = 0.10
Figure 36: Chi Square Test Coefﬁcients for Correct/Incorrect Distributions

Consistent with the AN COVA results, the two conditions for which there were
signiﬁcantly different performance means of Gap (DR and MB) also showed signiﬁcantly
different distributions of correct and incorrect answers. Thus, the differences in means
were more likely due to actual correct and incorrect answers by the two groups and not to
one group being mildly wrong (eg. 2 S Gap S 4) and the other group being very wrong
(eg. 5S Gap S 6).

5.4.5 Post Hoe Analysis

The participants were also asked to record their starting and ending times (to the
nearest rrrinute) for the identiﬁcation task as well as their conﬁdence in their ﬁrst choice.
Time was the difference of the self-reported start and stop times, and conﬁdence, also self-
reported, was on a scale of 0 to 100, with 0 being completely unconﬁdent and 100 being
completely conﬁdent in their choice. The means and standard deviations of time and
conﬁdence are reported in ﬁgure 37.

108

 

Experimental] CONFIDENCE I
Condition Disa A

(16:62) (16:98) (2:91) (1:67)
DD 8493 2365 ' °
1 . . 1.51) (1.74)
IR WL ‘%8'27L .96 3.69
(24.79 17.20) (1.37) (2.13
SB 8 479.80 3.85

MB (13.59) (15.03) (2.95) (1.50) ,

(13:98) (18:74) (11.62) (2171)
Figure 37: Conﬁdence and Time Results- Mean (Standard Deviation) Values

   
   

 

 

 

 

 

 

 

 

 

 

 

 

 

A Levene test for homogeneity of variance was performed for both Conﬁdence and
Time, with results indicating that the variances among the groups were not signiﬁcantly
diﬂ'erent (p < 0.748 and p <0.122, respectively). As a result, ANCOVAs were performed
on the original untransformed data.
5.4.5.1 Time

A repeated measures AN COVA was performed for Time with the results presented
in ﬁgure 38. Signiﬁcant interactions were found for Time x Type (p < .002) and Time x
School (p < .002), and a main effect for the covariate GPA (p < .014). The participants
with the higher GPA actually performed the task more slowly. The main effect for School
(p < .016) must be interpreted in light of the interaction with Time. Consequently, a series
of ANCOVAs on each shop floor condition were performed and the results are presenwd in
panels A through E of table 39.

109

Between Subjects

 

 

 

 

Source SS DF MS F P

Type 15.649 1 15.649 2.045 0.162
School 49.383 1 49.683 6. 453 0.016

Type x School 19.845 1 19. 845 2.593 0. 117

Age 3.172 1 3.172 0.414 0. 524

GPA 51. 681 1 51.681 6. 756 0. 014*

Yr 15. 905 1 15.905 2.078 0. 159

Exp 14.810 1 14. 810 1 .935 0. 173

Error 260. 20 34 7. 653

Within Subjects

Source 88 DF MS F P
Time 11.653 4 2.913 0.988 0.410
Time x Type 51.254 4 12.814 4.345 0.002*
Timex School ' 52.131 4 13.033 4.419 0002*
Time x Type x School 14.508 4 3.627 1.230 0.301
Time x Age 12.815 4 3.204 1.086 0.366
Time x GPA 7.476 4 1.869 0.634 0.639
Time x Yr 3.838 4 0.960 0.325 0.861
Time x E_xp 20.934 4 5.233 1.775 0.138
Error 401.09 136 2.949

Figure 38: Repeated Measures ANCOVA for Time

* Indicates signiﬁcance at a = 0.10

110

Panel A, Time AN COVA Results for Dispatch Rule (DR)

 

 

 

 

 

 

 

 

Source 8 S DF MS F P
Type 76.988 1 76.988 12.570 0001*
School 7.273 1 7.273 1.187 0.282
Type x School 8.986 1 8.986 1.467 0.232
Age 4.01 1 1 4.01 1 0.655 0.423
GPA 8.551 1 8.551 1.396 0.244
Yr 2.467 1 2.467 0.403 0.529
.21 0.406 1 0.406 0.066 0.798
Error 263.37 43 6.125
R2 0.288
Panel B, Time AN COVA Results for Due Date (DD)
Source S S DF MS F P
Type 2.025 1 2.025 0.866 0.357
School 0.452 1 0.452 0.193 0.662
Type x School 0.014 1 0.014 0.006 0.938
Age 6.717 1 6.717 2.873 0.098*
GPA 9.636 1 9.636 4.121 0.049“
Yr 0.257 1 0.257 0.1 10 0.742
Exp 0.412 1 0.412 0.176 0.677
Error 95.86 41 2.338
R2 0.257
Panel C, Time AN COVA Results for Irregular Job Release (1R)
Source SS DF MS F P
Type 4.834 1 4.834 1.805 0.186
School 0.926 1 0.926 0.346 0.560
Type x School 2.980 1 2.980 1.113 0.298
Age 2.196 1 2.196 0.820 0.370
GPA 3.587 1 3.5 87 1.339 0.254
Yr 3.375 1 3.375 1.260 0.268
_§x_p 16.485 1 16.485 6.154 0.370
Error 1 12.51 42 2.679
R2 0.31 1
Panel D, Time AN COVA Results for Stationary Bottleneck
_Source 8 8 DP MS F P
Type 8.762 1 8.762 1.732 0.195
School 20.322 1 20.322 4.018 0.051“
Type x School 14.584 1 14.851 2.884 0097*
Age 6.414 1 6.414 1.268 0.266
GPA 25.883 1 25.883 5.118 0.029*
Yr 6.452 1 6.452 1.276 0.265
Exp 2.128 1 2.128 0.421 0.520
Error 217.47 43 5.058
R2 0.220

Figure 39: Time AN COVAs for Shop Floor Conditions

* Indicates signiﬁcant at or = 0.10

Panel B, Time AN COVA Results for Moving Bottleneck

 

 

Source SS DF MS F P
Type 50.077 1 50.077 0.798 A 0.377
School 21.463 1 21.463 0.342 0.562
Type x School 26.455 1 26.455 0.421 0.520
Age 0.029 1 0.029 0.000 0.983
GPA 87.893 1 87.893 1.400 0.243
Yr 34.018 1 34.018 0.542 0.466
_gp 274.15 1 274.15 4.366 0042*

Error 2762.86 44 62.792
R2 0.253

Figure 39 (continued)

For the dispatch rule condition (DR), Information Type was signiﬁcant (p < .001),
with the participants who used the aggregate measures performing the identiﬁcation faster,
on average, than the participants with the disaggregate measures (4.35 versus 6.92
minutes). Information Type and School were not signiﬁcant for the identiﬁcation of the
due date assignment method (DD), irregular job release (IR), and moving bottleneck (MB).
Age and GPA (p < 0.098, 0.049, respectively) were signiﬁcant for DD, while School (p <
.051) and Information Type x School (p < .097) were both signiﬁcant for SB. The other
signiﬁcant results obtained in the individual AN COVAs are not discussed since there is no
overall signiﬁcance obtained in the repeated measures ANCOVA. A larger sample size may
result in more signiﬁcant factors.

It appears that the signiﬁcant Time effect was driven solely by the performance on
DR. There were no other signiﬁcant time performance differences between groups. This
result provides support for the use of disaggregate measures. The participants had
signiﬁcantly more information in the disaggregate condition, yet only in one instance did
they take signiﬁcantly more time (and for that condition that they should have the most
prior knowledge, dispatch rule, if this was an actual shop ﬂoor). Only for SB was school
signiﬁcant (p < 0.097).

112

5.4.5.2 Conﬁdence

The participants’ subjective conﬁdence was also analyzed through a repeated
measures ANCOVA (ﬁgure 40). A signiﬁcant main effect was found (p < 0.099), as were
several signiﬁcant interactions- School (p < .094), GPA (p < .071), and Manufacturing
Experience (p < .002). The individual conditions’ ANCOVAs are shown in Panels A

 

 

 

 

 

through E of ﬁgure 41.
Between Subjects
Source SS DF MS F
Type 445.375 1 445.375 0.602 0.443
School 12.297 1 12.297 0.017 0.898
Type x School 6.654 1 6.654 0.009 0.925
Age 1215.52 1 1215.52 1.644 0.208
GPA 1.418 1 1.418 0.002 0.965
Yr 466.61 1 466.61 0.631 0.432

_gp A 117.06 1 117.06 0.158 0.693
Error 25138.1 34 739.36
Within Subjects
Source SS DF MS F P
Conf 1913.48 4 478.37 1.991 0.099*
Conf x Type 1034.37 4 258.59 1.077 0.371
Conf 1: School 1948.44 4 487.11 2.028 0.094*
Conf x Type x School 674.85 4 168.71 0.702 0.592
Conf x Age 1408.63 4 352.16 1.466 0.216
Conf x GPA 2128.07 4 532.02 2.215 0.071“
Conf x Yr 358.46 4 89.62 0.373 0.828
Conf x Exp 4316.31 4 1079.08 4.492 0002*
Error 32668.13 136 240.21

Figure 40: Repeated Measures ANCOVA Results for Conﬁdence

113

Panel A, Conﬁdence AN COVA Results for Dispatch Rule (DR)

 

 

 

 

 

 

 

 

Source SS DF MS F P
Type 885.32 1 885.32 3.053 0088*
School 1.78 1 1.78 0.006 0.938
Type x School 157.36 1 157.36 0.543 0.465
Age 556.02 1 556.02 1.917 0.173
GPA 7.34 1 7.34 0.025 0.874
Yr 281.41 1 281.41 0.870 0.330
Exp 282.20 1 282.20 0.973 0.329
Error 12470.58 43 290.01

R2 0.139

Panel B, Conﬁdence AN COVA Results for Due Date (DD)

_Source SS DF MS F P
Type 867.01 1 867.01 1.988 0.166
School 14.62 1 14.62 0.034 0.856
Type x School 38.28 1 38.28 0.088 0.768
Age 205.3 1 1 205.3 1 0.47 1 0.496
GPA 136.88 1 136.88 0.314 0.578
Yr 46.61 1 46.61 0.107 0.745
Exp 67.22 1 67.22 0.154 0.697
Error 18312.92 42 436.02
R2 0.120
Panel C, Confidence AN COVA Results for Irregular Job Release (IR)
Source SS DF MS F P
Type 13.65 1 13.65 0.030 0.863
School 323.89 1 323.89 0.712 0.404
Type x School 167.48 1 167.78 0.368 0.547
Age 0.42 1 0.42 0.001 0.976
GPA 852.01 1 852.01 1.872 0.179
Yr 282.00 1 282.00 0.619 0.436
Exp 990.33 1 990.33 2.175 0.148
Error 19119.37 42 455.22
1?.2 0.134
Panel D, Conﬁdence AN COVA Results for Stationary Bottleneck

_Source SS DF MS F P
Type 0.673 1 0.67 3 0.004 0.951
School 44.64 1 44.64 0.249 0.620
Type x School 600.88 1 600.88 3.354 0074*
Age 831.93 1 831.93 4.643 0.037"
GPA 398.20 1 398.20 2.222 0.143
Yr 21.31 1 21.31 0.119 0.732
Exp 35.85 1 35.85 0.200 0.657
Error 7704.48 43 179.17
R2 0.241

Figure 41: Conﬁdence AN COVAs for ShOp Floor Conditions

* Indicates signiﬁcance at 01 = 0.10

114

Panel B, Conﬁdence AN COVA Results for Moving Bottleneck

 

 

Source ss DF MS F E
Type 502.07 1 502.07 2.165 0.148
School 569.84 1 569.84 2.457 0.124
Type x School 682.40 1 682.40 2.942 0093*
Age 2512.35 1 2512.35 10.832 0002*
GPA 27.04 1 27.04 0.117 0.734
Yr 78.17 1 78.17 0.337 0.565
Exp 3.46 1 3.46 0.015 0.903
Error 10205.06 44 231.93

R2 0.267

Figure 41 (continued)

Conﬁdence had a signiﬁcant main effect in the identiﬁcation of the DR condition,
with those using the aggregate measures being more conﬁdent. Information Type and
School interacted signiﬁcantly for both bottleneck conditions, SB and MB. In addition,

Age was a signiﬁcant covariate for the two bottleneck conditions as well.

5.4.6 Speciﬁc Measures’ Usefulness per Condition

During the laboratory sessions, the participants were asked to rank the usefulness
of the various information sets (i.e., measures) that were actually used to identify the shop
ﬂoor condition. These responses consisted of rankings on a scale of one and up,
depending on the number of measrnes used. As a result, there may be only one or two
measures indicawd (with rankings one and/or two) or there may be as many as seven or
eight measures indicated, each with a ranking. In all cases, a “1” ranking indicated the
most useful measure as judged by the respondents.

For the analysis of the responses the following question was asked: For a given
information type and condition (e.g., aggregate and DD), how would you rank the various
measures as to their usefrdness in identiﬁr‘ng the condition? It was not possible to compare
the reporwd usefulness of measures between the aggregate and disaggregate treatment
groups since the measures differed across the two groups. Each group received different
types of information, hence, the provided rankings corresponded to entirely different

115

information. So, comparisons of perceived usefulness were made across the conditions
within each information type group.

It was not possible to analyze this information statistically due to the widely varying
and sometimes limited sample sizes. For a given condition, some participants did not even
use a particular measure and, therefore, gave it no ranking at all, while other participants
did. To assess the responses, the following information was evaluated: (1) the mean and
standard deviation of each measure’s ranking for each condition; and (2) the number of
participants that actually ranked the measure for that condition. The latter value was .
important to provide some perspective on the perceived usefulness of each measure. For
example, a measure may have been ranked very highly, but only by the three or four
participants who used it. All other participants (as many as 20) may not have used it at all.
In addition, the usefulness rankings were separated into groups- those who answered
correctly (0 S Gap S 1, consistent with the Chi square test criteria) and those who
answered incorrectly (Gap 2 2).

Although no speciﬁc hypotheses were developed for the analysis of this section, ,
there were several expectations. Since the DR condition conSiswd of introducing the SPT
dispatch rule versus the FCFS base case, the expectation was that those who correctly
identiﬁed the condition would rely on the time in system measrne, since the most common
manifestation of using SPT is a reduction in throughput time (Blackstone et. al., 1982).
On the other hand, for those who incorrectly identiﬁed DR, the expectation was a higher
usage and ranking of measures other than time in system Correct identiﬁcation of the DD
condition should be accompanied by a relatively high ranking of the due date allowance
measures, while correct identiﬁcation of the bortleneck conditions, SB and MB, should
show a reliance on the queue length measures, either means or time series. As with DR,
incorrect identiﬁcation of DD and the bottlenecks should be accompanied by a relatively
high ranking of measures other than due date allowance and queue lengths, respectively.

There was no a priori expectation as to the most useful measure in identifying IR.

116

Because of their length and complexity, tables with the means and standard
deviations of the participants’ usefulness rankings are reserved for Appendix A. Figure
42, however, summarizes the analysis and indicates the number of respondents who chose
certain measures as the most useful in identifying the shop ﬂoor condition. Note that data
for both information types is included, as is that from respondents who answered both
correctly and incorrectly. Also indicated in bold type is the measme that is consistent with

expectations.

117

 

 

 

 

 

 

 

 

 

 

DISAGGREGATE AGGREGATE

Cond Correct Incorrect . Correct 7

DR Ts ys "‘ 8 Tsys 2 Ts ys“ 8 Tsys" 2
Job Late 1 Job Late 0 Job Late 7 Job Late 0
Proj Job Late 2 Proj Job Late 0 Proj Job Late 1 Proj Job Late 0
Job Tardy 0 Job Tardy 0 Job Tardy 0 Job Tardy 0
DD Allowance 0 DD Allowance 1 DD Allowance 1 DD Allowance 0
Queue Length 4 Queue Length" 7 Queue Length 7 Queue Length 0
Job Activity 1 Job Activity 0

DD Tsys 0 Tsys‘ 1 Tsys 1 Tsys 0
Job Late 0 Job Late 1 Job Late 1 Job Late 2
Proj Job Late 0 Proj Job Late 1 Proj Job Late 0 Proj Job Late 0
Job Tardy 0 Job Tardy 0 Job Tardy 1 Job Tardy 0
DD Allow" 1 9 DD Allowance 1 DD Allow"I 1 7 DD Allowance" 3
Queue Length 0 Queue Length 0 Queue Length 0 Queue Length 0
Job Activity 0 Job Activity 0 ,

IR Tsys" l Tsys 7 Tsys 2 Tsys 5
Job Late 2 Job Late“ 3 Job Late 1 Job Late 0
Proj Job Late 2 Proj lob Late 3 Proj Job Late 2 Proj Job Late 1
Job Tardy 0 Job Tardy 1 Job Tardy 0 Job Tardy 0
DD Allowance 0 DD Allowance 1 DD Allowance 1 DD Allowance 0
QueueLength l QueueLength 1 Queue Length" 3 Queue Length" 11
J ob Activity 0 Job Activity 2 -

SB Tsys 3 Tsys“ 1 Tsys 0 Tsys 0
Job Late 1 Job Late 0 Job Late 0 Job Late 0
Proj Job Late 1 Proj Job Late 1 Proj lob Late 0 Proj Job Late 0
Job Tardy 0 Job Tardy 0 Job Tardy 2 Job Tardy 0
DD Allowance 0 DD Allowance 0 DD Allowance 1 DD Allowance 0
Que. Length‘ 1 9 Queue Length 0 Que. Length" 1 8 Queue Length" 4
1 ob Activity 0 Job Activity 0 ,

MB Tsys 0 Tsys 1 Tsys 3 Tsys" 3
Job Late 1 Job Late" 1 Job Late 1 Job Late 1
Proj Job Late 1 Proj Job Late 0 Proj Job Late 1 Proj Job Late 5
Job Tardy 0 Job Tardy 0 Job Tardy 0 Job Tardy 0
DD Allowance 1 DD Allowance 0 DD Allowance 1 DD Allowance 0
Que. Length‘ 1 6 Queue Length 3 Que. Length‘ 5 Queue Length 6
Job Activity 1 Job Activig‘ 1

Total Participants n=26 n=27

 

* Most highly ranked measure for usefulness (lowest mean measure among all used). Two asterisks
indicate a tie in the mean ranking.

Thetotalsofcorrectandinconectresponses foreachcondition may notbeequal due to some
participants not performing some conditions’ identification and/or some not providing a ranking of the

most useful measure.

Bold indicates conformance with expectations

Figure 42: Summary of Most Highly Ranked Measure for Nature of Responses

118

 

Participants in both information type groups who correctly identiﬁed the conditions
consistently ranked as highest the measure they were expected to chose. Those who
answered incorrectly and who used disaggregate information, however, ranked as highest
those measures not normally associated with correct answers. Thus, those who answered
incorrectly seemed to rely on the wrong measures, possibly leading to their incorrect
results, which is consistent with expectations.

Those respondents who answered incorrectly and used aggregate information
ranked as most useful the measures normally associated with correct answers- those that
were expected to be associated with correct answers. Thus, those using aggregate
measures and answering both correctly and incorrectly all ranked as most useful the sanre

measures, with the exception of the MB condition.

5.4.7 Clarity of Information Presentation

The participants were also asked to respond to eight follow-up questions at the end
of the experimental sessions. These questions attempted to assess how well the
information was presented and how well it described the shop ﬂoor environment and

conditions. The respondents were asked to indicate their level of agreement with the

following questions:

1. As a result of the training session, I knew how to use the information presented.

2. I could have made sense of the information presented without the training session.

3 . The information presented with the cases was insufﬁcient for me to answer the
questions.

4. The information presented with the cases was excessive and was difﬁcult to sift
through.

5 . The information presented with the cases was concise in natrne and adequate to

allow me to answer the questions.

6. The information presented with the cases clearly and completely described the
overall structure of the shop, its flows and layout.

119

7 . The information presented with the cases allowed me to clearly understand and
identify the changes in conditions aﬁecting the shop.

8 . The presentation of data in graphical/tabular (disaggregate group/aggregate group) -
format made it difﬁcult to use.

Question 8 was modiﬁed according to the treatment group to which it was
addressed. The disaggregate information group was presenwd with graphical information
and was aslmd their opinion of how difﬁcult it was to use. Similar for the aggregate group
regarding their tabular data. ' '

Each question was to be answered with a value from one to seven, ranging from
strongly agree (1) to strongly disagree (7). An Analysis of Variance (ANOVA) was
' performed to determine if the two groups answered the questions in a signiﬁcantly different
manner

Although no hypotheses were put forth for this portion of the research, there were
some expectations in terms of the results, as summarized in ﬁgure 43. The “>” refers to
the relative mean sizes with a “larger” result indicating a higher expected level of
disagreement. For example, for question #3, dealing with the sufﬁciency of the
information presented, it was expected that the mean for the disaggregate group (D) would
be signiﬁcantly greater than that for the aggregate group (A). The aggregate group was
expected to agree more strongly that the information was insufﬁcient, as it consisted of
only point means and standard deviations. For question #4, a higher level of agreement
from the disaggregate group that the information was excessive was expected, since that
information consisted of multiple pages of graphs. Similarly, question #5 asked about the
conciseness of the information, which was expected to be more acknowledged by the
aggregate group. Finally, although question #8 was asked slightly differently to each
group, it was expected that the aggregate group would rate their tabular information as
more difﬁcult to use (more strongly agree) than the disaggregate group. Figure 43 shows
the expected direction of the relative value of the means.

120

 

D = disaggregate group
A = aggregate amp

Flgure 43: Expecwd Results- Clarity of Information Presenwd

The results of the ANOVA are summarind in ﬁgure 44.

121

was

was
to sift
was
to answer

 

made it

Figure 44: ANOVA Results for Follow Up Questions. Means, P value and (signiﬁcance
level, * indicates p < 0.10)

A lower mean value shown corresponds to a higher level of agreement with the

statement.

The ANOVA results revealed that three questions (3, 5 and 7) had signiﬁcantly
different responses between the two information groups. Question 3’s results show that the
disaggregate group had a signiﬁcantly higher level of disagreement with the question,
indicating that they agreed more strongly that the information was sufﬁcient to make the
identiﬁcation, consistent with expectations. Question 5’s results show that the disaggregate
group agreed more strongly that the information presented was concise and adequate to
answer the questions, which was contrary to expectations. The results for question 7 show
that the disaggregate group agrwd more strongly that the information presented allowed
them to understand the changes in conditions on the shop ﬂoor, which was key to making

the identiﬁcations. The results for questions #3 and #5 were consistent with expectations.

122

5.4.8 Summary of Results

While several different results were reported in the preceding sections, the more
important ones are listed below. These ﬁndings either directly address the research

questions or provide a means of addressing them, as indicated in the italicized discussion.

. The experts surveyed expressed an interest in non-traditional disaggregate
measures while not embracing traditional aggregate measures as identiﬁed in the
literature. '

This allowed the research to proceed to the development of the
laboratory instrument.

0 The deve10pment of a semantically based shop ﬂoor simulation model was
found to be quite feasible.

This provided the basis for the development of the simulation model and
the relational database.

. The values of conventionally derived measures and those from a disaggregate
dataset were very comparable.

This provided a “proof ’ of concept in that the non-conventionally
derived values were essentially equal to those derived in the
conventional manner.

0 Both aggregate and disaggregate measures provided an advantage in shop ﬂoor
condition identification, depending on the condition. In some instances,
neither provided an advantage.

This ﬁnding directly addressed Hypothesis #1 in that it allowed
comparison of the utility of the two types of measures.

0 For participants using disaggregate information, the measures judged most
useful differed greatly between those who made correct and incorrect

identiﬁcations; for users of aggregate measures, there was no such difference.

123

This finding pointed out an interesting result in terms of how the two
information type groups interpreted the measures.
. In general, users of disaggregate information judged the information more
sufﬁcient and adequate to make the identiﬁcations.
This goes to the heart of Hypothesis #1, in that it provides justiﬁcation
for the theoretical position that disaggregate information and measures

are superior.

5.5 Discussion of Results

The results of the limited experts’ srn'vey of performance measures’ utility were
already adequately discussed in section 5.2, along with the presentation of the results. For
this reason, the following sections will begin discussing the remaining key ﬁndings, as
liswd above.
5.5.1 Development of a Semantically Based Simulation Model

The development of the semantic data model of the shop floor was relatively
straightforward. Resources, Events and Agents on the affected portion of the shop ﬂoor
were identiﬁed, however, they were not modeled as an REA model, per se (McCarthy,
1982). Since the ultimate goal of this exercise was to provide a template for the generation
of shop ﬂoor data, only th08e entities that were relevant to this were included. Thus,
although most events in the model have an accompanying agent and resource, there is not a
clearly deﬁned “inside” and “outside” agent nor is there a strictly deﬁned incremenwd and
decremented resource for each event, as McCarthy (1982) speciﬁed. However, the model
developed does capture the entities and relationships important to day to day shop operation
and did provide an excellent basis for developing the relational database.

Within the data model, several generalization hierarchies (Smith and Smith, 1977)
were established, which helped to further deﬁne the model. This turned out to be very
important since there were several entities with multiple types that, without any

124

generalization, could not be adequately modeled These entities were JOBS, MULTI-JOB
ORDERS and EMPLOYEES.

The development of the data model readily allowed the design of the relational
database. The various types of the generalizations constituted separate tables, as
customary. Concatenated keys were required for some entities, such as OPERATTON and
JOB-WC (Job#-WC#). In order to allow repeat visits to each work center, these entit es
required a triple concatenated key within the data model, adding Sequenceit (Seq#).
However, once the data were transferred to the database and the actual compilation and
querying of data began, it was discovered that one more key was required-— that of the
simulation model run number. Since the database stored data from all separate runs of the
model (i.e., the different conditions inserted to the model), a distinction had to be made for
the different datasets. For this reason, an additional entity, RUN, was established in the
database and its primary key, Run#, became part of the key of the three main entities of the
model- JOB, OPERATTON and JOB-WC. An additional entity (DAYS) and relationship
(DAYS-WC) were also required to allow for the daily aggregating of certain shop floor
data. While DAYS-WC could have been considered part of the conceptual model, doing so
would imply some prespeciﬁcation (i.e., daily) of what measures and information were
desired. However, since the data were not actually preaggregated in the database, but only
“ﬂagged” with certain attributes, speciﬁcally Day#, the events-based model (McCarthy,
1979) was not actually violated Moreover, if an overriding managerial concern is with
daily summaries and reports, as may well be, then including DAYS and DAYS-WC may be
appropriate. RUN, on the other hand, is strictly an artifact of the experimental design and
does not belong in the data model. Thus this research showed some very interesting
implementation issues when such a data model and database were actually used to store and
present historical operational information. Such additions were not required in the
conceptual model and did not become necessary until actual implementation, of which this

research is one of the ﬁrst in this ﬁeld

125

5.5.2 Derivation of Measurements and Information

Proposition #1 locked at commring the difference in the measures that were derived
conventionally, that is with the simulation software’s built in functions, and those that were
materialized from the disaggregawd dataset. To do this, the two different values of Time in
System (TSYS), Mean Lateness (LATE) and Mean Tardiness (TARDY) were compared
Since only one model run for each condition was made, it was not possible to statistically
test the differences. However, this one run also dictated that the measures would be
derived from the same dataset, thus it was expected that the results would match very
closely.

The disaggregawd measures were derived from the entire dataset as generated by
the relational database. To obtain mean TSYS at time = 1400 horns, for example, the
records between 700 hours and 1400 hours were averaged to determine the mean time an
exiting job had been in the system. The aggregate measures, on the other hand, consisted
of a rolling average of jobs exiting between 700 and 1400 hours. This value was calculated
by the simulation software and posted at 1400 hours. Thus, the two values should have
matched very closely since they were calculated from the same jobs. In fact, the
discremncy between the two methods is very small, less than 0.4% in most instances.
Thus, even without formal statistical testing, it is very reasonable to conclude that deriving
traditional shop ﬂoor measures in an unconventional manner (from disaggregate data)
yields nearly identical results to more conventional derivation.

5.5.3 Participants’ Identiﬁcation of Shop Floor Conditions (HM)

Information Type interacted signiﬁcantly with two performance measures, Gap and
Time, indicating that the type of information did have a signiﬁcant effect on the
participants’ accuracy and efficiency of condition identiﬁcation. Subsequent analysis for
each separate condition revealed that for the dispatch rule (DR), the participants who had
the traditional aggregate measures performed more accru‘ately and more quickly than did '

those with the disaggregate measures, that is, their mean Gap and Time values were lower.

126

However, in identifying the moving bottleneck (MB), those with the disaggregate measures
performed signiﬁcantly more accurately (lower Gap).

These results provide partial support for Hypothesis #1 in that access to the
disaggregate measures improved the identiﬁcation of shop ﬂoor problems and conditions.
In the case of a moving bottleneck the disaggregate-based measures provided an advantage
in identiﬁcation. It should be noted that this is the one condition that is not static, or set, at
the beginning of the model’s run. With a moving bottleneck, the available capacity is
varied from work center to work center throughout the run of the model. Thus, the queue
lengths vary considerably over time. The other conditions- dispatch rule choice (DR),
irregular order release (IR), due date assignment method (DD) and stationary bottleneck
(SB)— consist of implementing a govemin g condition at the outset of the simulation run and
not varying it. Thus, aggregate measures may be sufﬁcient to identify relatively static
conditions, or at least, disaggregate measrues do not provide an advantage. However,
when the shop floor is experiencing a condition such as the moving bottleneck, in which
conditions are quite dynamic and changing with some frequency over time, disaggregate
measures could provide an edge in problem identiﬁcation.

The one exception is the identiﬁcation of the dispatch rule (DR). In this
experiment, the participants were asked to identify when the dispatch rule changed from
First Come First Served (FCFS), which was the base case, to Shortest Process Time
(SPT), which was introduced into the model. The identiﬁcation of SPT with the aggregate
measures raises some interesting issues. SPT is typically known to both decrease the mean
and increase the variance of time in system (TSYS) (Blackstone et. al., 1982). Given this
fact, it may be more identiﬁable than other conditions since users may tend to focus in on
the mean and standard deviation of TSYS, both of which were presented to the aggregate
information group with relatively unambiguous point measures. This was substantiated by
the fact that the correct identiﬁes overwhelmingly used and ranked TSYS as the most useful
measure. On the other hand, the disaggregate group was presented with, among others, a

127

graph of the Distribution of Time in System. Comparing two distributional graphs to
ascertain the differences in their means and standard deviations may be somewhat more
cumbersome than simply comparing point values. Thus it is possible that the disaggregate
group was not as well equipped to deal with a condition such as the dispatch rule changing
to SPT.

The results of DR’s and MB’s Gap measures, as discussed above, are consistent
with the results of a Chi square analysis done of the distribution of correct and incorrect
answers. The Chi square test indicated that the two groups, for these two conditions, had
signiﬁcantly differently distributed right and wrong answers. Thus the signiﬁcantly
diﬁ'erent means are likely due simply to more correct answers in one group versus the
other. This provides further substantiation that using the aggregate measures for DR and
the disaggregate measures for MB led participants to more often comectly identify the
underlying condition.

In addition to allowing more accurate identiﬁcation of DR, participants using the
aggregate measures identiﬁed the condition signiﬁcantly more quickly than those using the
disaggregate measures. Perhaps, as discussed previously, it is due to the substantially
lower volume and simpler nature of the information presented

This ﬁnding lends additional weight to the uniqueness of the SPT dispatch rule as a
shop ﬂoor condition. Not only is it one of only two conditions for which the type of
information makes a signiﬁcant difference in identiﬁcation (and the only “static” one), but it
is also the only condition for which the type of information makes a signiﬁcant difference
in the time to identify. SPT, therefore, appears to have some unique properties that make it
worthy of further research.

128

5 .5.3.1 Covariates
In addition to the differences discussed above, there were several cases in which

one or more of the covariates were signiﬁcant (or = 0.10). Figure 45 summarizes those

covariates that had a signiﬁcant effect.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

”Ripeated Measures
Measure ﬁtween Within Subjects
Subjects
Gap Gap x A e
Tate School more x SEW"
GPA
mnm Conf x School
Conf x GPA
Conf x MfrExp
FSemrate AN COVAs
Performance SrEﬁﬁEant
Condition Measure Covariate
_DR
HS "lime GPA
Time Ag
IR Gap School
gap Ag
aP MﬁExP
SB "lime GPA
gum School
- IE" ....... _Typsz§eh.991.-.
Con}l Type x School
. Conf A
"—85 .Garf ....... £ch ........
fuse ........ Mir Ea) .......
C0“I Type x School
Conf

 

 

 

 

 

Figrue 45 : Signiﬁcant (or = 0.10) Covariates

Under the repeated measures analysis for Gap, a signiﬁcant interaction between
Gap and Age is noteworthy. Recall that Gap x Type was signiﬁcant, leading to further
analysis to determine which condition led to such difference. Referring to the analysis of

the demographics in ﬁgure 30, one can see that the ages of the two information type groups

129

were signiﬁcantly different. This raises an additional issue, speciﬁcally the effect of a
person’s age on his or her performance. Exploration of this idea is reserved for futtue
research.

Similarly, the signiﬁcant interaction between Time and School indicates that the
school the participants attended (MSU versus LSSU) may have inﬂuenced the spwd at
which they completed the assigned tasks.

In the separate AN COVAs, note that for MB, GPA is a signiﬁcant covariate for
Gap. Apparently, this is the only condition for which a signiﬁcant covariate may have
affected the performance measures sufficiently to cause a signiﬁcant difference. In all other
cases, signiﬁcant covariates were apparently insufﬁcient to cause differences in
performance. Given the nature of the AN COVA, it is not possible to observe if the
covariates had a positive 0r negative effect on the performance measure. Such topics are
outside the scope of the original research questions and thus are left to future research.
5.5.4 Participants’ Judgment of Measures’ Usefulness

As indicated in the presentation section and Appendix A, there is a great deal of data
associated with this portion of the research. The purpose of including these questions in
the instrument was to record the users’ judgement as to which measures were more useful
in identifying the shop ﬂoor conditions. Of interest was if the users’ feelings matched the a
priori expectations in terms of the most commonly used measures for those who comectly
and incorrectly identiﬁed the conditions.

A number of consistencies with expectations were discovered All correct
respondents in the disaggregate and the aggregate group chose measures consistent with
expectations in four out of ﬁve conditions, the exception always being IR, for which there
were no expectations.

There was another very interesting ﬁnding as shown in ﬁgure 42, namely that in the
aggregate group, in four out of ﬁve conditions, those respondents who answered comectly

and those who answered incorrectly ranked the same measures as most useful. However,

130

in the disaggregate group, those respondents who answered correctly ranked as the most
useful very diﬂ‘erent measures than did the respondents who answered incorrectly. Thus,
regardless of the correctness or accuracy of identiﬁcation (Gap score), all users of
aggregate information tended to rely on the same measures. This was not the case,
however, with the disaggregate information group where the measures judged most useful
were different, depending on the accuracy of identiﬁcation.

What this analysis has revealed is that the process of interpreting the performance
measures may differ for the two different information types. As shown in ﬁgure 46, the
experimental design attemped to control for various inputs to the participant’s interpretation

process, however it did not explicitly monitor or control the process itself.

Sl-‘U’FLCXH
CORDIHONS \

 

INFORMATION TYPE \

P90053301: IDENTIFICATION
/ 'NlERPRETAT'ON "" PERFORMANCE
TRAINING

 

Figure 46: Mapping of Experimental Design

With disaggregate measures, it appears that the users can more correctly identify the
best measures or information to use. With the aggregate measures, it may not be as easy to
do so, perhaps due to its high degree of compression and aggregation. Aggregate
measures do not appear to speak as clearly to information users as do disaggregate
measures. This result indicates a potential systemic problem when using aggregate
measures for shop floor perfomnance evaluation and is a rich area for funne research, as

will be discussed in chapter 6.

131

 

5 .5 .5 Clarity of Information Presented

There were three questions in this portion of the research that had signiﬁcantly
different mean responses between the two information type groups. What is important
about this ﬁnding is that all three questions dealt with the usefulness of the information
presened Question #3 (“Information was insufﬁcient to answer the questions”) addressed
the sufﬁciency of the information presented It was found that the disaggregate group felt
more strongly that the information was sufficient, which can be considered as a key
element of usefulness.

The analysis for question #5 (“Information was concise and adequate to answer the
questions”) revealed that the disaggregate group agreed more strongly that the information
was concise and adequate to answer the questions. This was contrary to expectations in
that it was anticipated that the aggregate group would rate their information as more concise
since it consisted of merely point estimates, such as means and standard deviations.
However, this result may point to a potential problem with question #5, rather than simply
an interesting ﬁnding. Including both “concise” and “adequate” in the question may have
led to confusion as these two words may be construed as being mutually exclusive.
“Adequate” normally is thought of as “enough” whereas “concise” normally refers to
“brevity.” It is possible that participants were focusing more on the word “adequate” rather
than “concise.” Such an interpretation would be consistent with the expectation that
disaggregaed information is more adequate or more capable of allowing correct
identiﬁcation. Had the question included only the word “concise,” users of aggregate
measures may have agreed signiﬁcantly more strongly with it.

The analysis for question #7 ("Information made the changs in conditions
understandable”) revealed perhaps the most encouraging result of all, that the disaggregate
group agreed more strongly that their infomnation allowed them to understand the changes
made in the shop ﬂoor environment. Thus, on the dimension that perhaps most directly '
affected the ability of the participants to idenu'fy the shop floor conditions— their

132

understanding of them- the disaggre gate information was seen as signiﬁcantly superior.
This result was expected and supports the underlying theoretical assertion of this research.

Lastly, the results for question #8 (Presentation of data in graphical/tabular
[disaggregate group/aggregate group] format made it difﬁcult to use) bear further
examination and discussion. The means for the two groups are not signﬁcantly different
and, by inspection, are both approximately equal to 5.5. Since 4 is the midpoint of the
scale, this indicates a similar level of slight disagreement that the presentation of data made
it difﬁcult to use. The disaggregate information group had exclusively graphical data,
while the aggregate information group had exclusively tabular data. Despite the substantial
differences in style, both groups had roughly the same feelings for the way in which the
data were presented Strong advocates of either style would not ﬁnd much solace in these
results. It appears that users of information make do with what they are given and do not
complain very much.

5.6 Possible Confounding Factors

Although an attempt was made to eliminate or control for posssible confounding
factors in this research, some were inevitably present that warrant some attention and
discussion.
5.6.1 Sample Size

There were only 53 participants in the lab experiment portion of this research. This
was due to the time constraints of the research and to the limited pool of qualiﬁed
volunteers. While the repeated measures. AN COVA lent more power to this sample size
(approximately 120 records for each analysis), the overall sample size is still somewhat
limited, particularly for the individual conditions’ AN COVAs. While in this case, it was
not feasible to increase the sample size, future research in this ﬁeld should strive to do so,
to more deﬁnitively establish whether certain effects are signiﬁcant, as argued in Cascio
and Zedeck (1983).

133

5.6.2 Learning Effect

All participants were exposed to the same training scenario prior to undertaking the
identiﬁcation tasks, although the scenarios were different for each information type group,
to account for the differences in the data presented

To equalize the learning effect among the participants, all were given ample
opportunity to study example tasks and to work on one as a group and them individually,
with input from the researcher. Thus all participants had the same preparation prior to the
identiﬁcation task although, granted, some participants were more capable and/or more
experienced than others. However, every attempt was made to bring all participants to the
same level of understanding prior to the task.

An alternate approach, and one that might be considered by future researchers,
would be to incorporate this exercise into a regular OM class during which the participants
would be trailed in the shop ﬂoor environment and problems. Thus their learning would
take place more slowly and consistently over a longer period of time. More discussion
would be allowed and participants would have the opportunity to receive more feedback on
similar tasks prior to the actual experiment. Such an experimental design may well reduce
the effects of unequal leamring and produce participants who are more equally and
adequately trained for the task.

5.6.3 Use of Student Participants

The participants in this research were all students, predominantly in graduate ’
business programs, whose experience in manufacturing varied It has been argued by
Locke (1986) that such subjects are appropriate for this type of research and that the results
can be generalized to the workplace, provided the task being performed is similar to what
might be done in a real world setting. What is important to note is that laboratory
experiments such as this are to test theory and to establish not necessarily what will happen
outside of the lab, but rather what may happen (Berkowitz and Donnerstein, 1982; Mook,
1983).

134

 

Although the laboratory setting in the research was somewhat artiﬁcial, namely, it
may not have resembled actual working conditions, it has been argued that such artiﬁciality
may not be a serious flaw of laboratory experimentation (Fromkin and Streufert, 1976).
Most differences between the laboratory and the real world, including the predominant use
of student subjects, have not been found to be a serious indictment of lab work (Locke,
1986).

However, even though the literature generally supports the use of laboratory
research as a proxy for actual business settings and maintains its generalizability, the use of
relatively inexperienced students (with an average manufacturing experience of 0.80 years)
is still a possible confounding factor in this research and one that bears further examination
in the future. Given the nature of laboratory experimental design, this potential problem
may never be completely resolved
5 . 6.4 Fatigue Effects

One concern in a laboratory setting is that the sessions may become too long and
arduous, with the result being that participants’ performance suffers. While this factor was
not measrned or controlled for in this research, it was felt that the nature and length of the
sessions precluded this being a major factor. All sessions were limited to approximately
2.5 horns and participants were given the opportunity to take breaks during the training and
task sessions. Food and refreshments were also provided in an attempt to make the
sessions more comfortable. Moreover, the completely voluntary nature of the experiment
allowed participants to essentially come and go as they pleased, should the session become
uncomfortable.

Admittedly, this was not a factor that was actively monitored dining the experiment.
However, given the overall length of the sessions and the apparent willingness of the

participants to complete each task, fatigue effects did not appear to be a major factor.

135

5 .6.5 Motivation

Since this research was voluntary, some consideration was given to motivation so
that all participants would try their best at the tasks. As outlined in chapter 4, several things
were done to help assure a motivated group of participants. For recruits from one class,
the professor oﬁ‘ered to give some additional class participation points for those
volunteering for the experiment. In addition, all recruits were told of four prizes that could
be won by participating (a new bicycle and cash prizes). Although all participants had a
chance to win the prizes, they were informed that those who performed better had a higher
probability of winning.

So while the participants were not as morivated as they might have been in an actual
work setting, whenever possible they were provided with incentives to participate and to

perform to the best of their ability.

5.7 Results Discussion Summary

The results of the work blending a semantic data model with a discrete events
simulation model were encouraging. It was found that by ﬁrst developing a complete data
model, it was possible to identify clearly which entities and relationships were important to
data collection and their corresponding critical attributes. With this done, it was much
easier to design the simulation model so that the proper data could be captured. This
sequential approach is recommended for any future work in this area. Incorporating a data
model and its corresponding data tables into a simulation model did require that more
memory be allocawd within the PC dtning the model runs, but such allocation requirements
are well within the capability of PCs on the market today.

Once the data and simulation models were built and run and data were generated,
materializing the required information and measures was done readily. While materializing
the measures for use, it was found that several additional entities and relationships were

needed that were not in the original data model. While not needed from a conceptual

136

standpoint, they were necessary to allow the proper organization of data in the implemented
database. Thus despite attempts to prespecify the conceptual data model as much as
possible, there were some reﬁnements in the actual implementation stage.

In terms of the key research question- does the type of information presented make
a difference in identifying shop ﬂoor conditions?- the answer was found to be
“sometimes.” For one condition, the dispatch rule, aggregate measrn'es actually improved
the accuracy of identiﬁcation, while for another condition, the moving bottleneck, the
disaggregate measures were more useful. With this result, it was concluded that perhaps
disaggregate data is more helpful for those conditions that are more dynamic and less
stable. For those conditions that employ a more static operational rule, with the exception
of the dispatch rule in this case (SPT), information type does not appear to make a
diﬁ'erence in identiﬁcation.

The results dealing with the users’ perceptions of the measures’ usefulness
conformed to a priori expectations quite consistently, for the most part. That is, the
participants’ feelings as to which measures were most useful for which conditions were
close to what was anticipated. This is heartening in that it implies that information users,
some of whom were quite inexperienced, gravitated towards the same measures for the
identiﬁcation of conditions as did more experienced users. In addition to these ﬁndings, it
was found that users of aggregate information tended to use the same measures, regardless
of whether they correctly or incorrectly identiﬁed the conditions.

Finally, in evaluating what participants thought of the actual information presented
(i.e., the measures), there were some signiﬁcant differences between the two groups, all of
which pointed towards the disaggregate measures being more useful in identiﬁcation. 0n
dimensions such as sufﬁciency, adequacy and aiding in understanding shop ﬂoor
conditions, participants ranked the disaggregate-based measures more highly. Despite this,

however, there was no difference between the two groups in their opinion as to the ease of

137

use of the information. Both groups agreed, although slightly, that the information was
relatively easy to use.
Figure 47 summarizes the overall results according to research question and

hypothesis/proposition and shows expected and actual results.

 

 

138

I-I thesis/Protosition Extected Results Actual Results ;

R 1
(21:/ability of events based model within simulation environment?

     
 

 

 
   
    

 

 
   
       
  

 

 

Means of aggregate and Support for null. Essentially equal
disaggregate meas (error < 1%) for all
.. ual. . conditions.

PRP-2

Experts’ measures can Support for proposition. Measures similar to
be derived from events experts’ preferences were
based model. derivable from database.

   

 

' RQ#2
Can a database ofdisaggregate data generate measures that are more

 
   
 

 

 
         
       
   
   
     

useful in iden -' ' : o . ' . -'or al conditions on shop floor?
. 1
Use of different types of Rejection of null; Superior performance
measures results in different Disaggregate data to for disaggregate measures
performance. yield improved with most dynamic
performance. condition (MB): reject null.
Aggregate measures
superior for DR (SPT):
reject null.
Other conditions no
difference.

 

 

Figure 47: Expected and Actual Results Summary

139

6.0 Conclusions and Future Directions

6.1 Chapter Overview

This chapter summarizes the ﬁndings of this research and suggests some areas for
future research. First, the research problem is restated as is the general research
methodology. Major ﬁndings are then liswd and discussed. Next, new research questions
are presented to address the unresolved paradoxes identiﬁed in this h mo. The chapter
closes with a summary of the ﬁndings of this research.

As ﬁrst presented in Chapter 1, this research addressed the problem of the
inadequacy of conventional mean and aggregate reporting measures in fully describing
shop ﬂoor performance. The level of detail required to properly document and assess shop
ﬂoor operations is too substantial to be adequately described by such measures. This
research proposed a method by which more detailed and disaggregawd data on shop ﬂoor
operations could be utilized in performance evaluation. A combination of shop ﬂoor
simulation and database modeling was used to generate various performance measures that
were then evaluated by users in a controlled laboratory setting. The success of the users in
describing the shop ﬂoor’s operation was evaluated based on the type of measures they
used.

The following list summarizes the more important overall areas and ﬁndings of this
research.

0 The use of semantic data modeling in conjunction with simulation modeling was
found to be useful.

0 The type of information presented to users (aggregate versus disaggregate) proved

to be a signiﬁcant factor in identifying shop ﬂoor problems in some instances.

0 Disaggregate data was found to be more understandable and more useful than

aggregate measures on several dimensions.

140

o The process by which users interprewd the performance measures appeared to be
inﬂuenced by the type of measures used.

6.2 Semantic Data Modeling with Simulation Models

The portion of this research dealing with the development and use of a semantic
data model in conjunction with a simulation model yielded results that conformed to
expectations. The research demonstrated that such a data modeling approach could be used
prior to developing a simulation model and that such an approach may be of use even with
a uaditional simulation model. The exercise of identifying all relevant entities (Resomces,
Events, Agents- McCarthy, 1982) helped to better deﬁne the ﬁnal simulation model. One
issue fairly speciﬁc to data modeling, generalizations of entity types (Smith and Smith,
1977), was handled quite readily in the model once separate and generalized entities were

rmma

established. This was a further indication of the two approaches’ ability to coexist.

The simulation model in this research was patterned after a support tool room for a
major automobile manufacturer. As a result, it not only consisted of various work centers
and different types of jobs and operational rules, but it also generated a great deal of
disaggregated operational data. As different conditions were introduced into the model,
different datasets were generated. The use of a relational database as a repository for all
this historical operational data was effective as a means of allowing the materialization of
different performance measures for managerial review. This substantiates the theoretical
work of Roberts (1991), who suggesmd that the relational database could be used for this
purpose.

While it was possible to materialize most measures completely within the database,
it was found that it was sometimes much easier with a spreadsheet, due to its unique data
manipulation abilities. This was particularly the case when the measures to be creawd
relied on information from only one entity or relation within the database. However, when

the measures required data from more than one entity, it was necessary to perform the

141

materialimtion within the database. Thus the usefulness and necessity of the database as a
vehicle for materializing information seems to increase with more complex inter-entity
queries. As management’s need for information becomes more demanding, thest dynamic

database may well evolve into the primary source of information and reports.

6. 3 Condition Identiﬁcation

When considering the relative usefulness of aggregate and disaggregate
information, it was found that in four out of ﬁve cases, disaggregate information was either
as effective or more eﬁ'ective in identifying shop ﬂoor problems. In only one case did
aggregate measures provide an advantage in identiﬁcation. And, despite the greater amount
of data, disaggregate information, in four out of ﬁve cases, did not cause the participants to
take signiﬁcantly more time for problem identiﬁcation. Additionally, participants who
received the disaggregate data believed that they had suﬂ'rcient information to make the
identiﬁcation, while the participants with the aggregate data generally wanted more
information.

The only case in which the aggregate group performed better (both more accmately
and more quickly) was in identifying a change from the FCFS dispatch rule in the base case
to SPT. SPT has already been identiﬁed as unique among the dispatch rules in that it
normally results in the lowest mean but highest variance of throughput time. It is a strong
performer on other measures as well (Blackstone, er. al., 1982). As a result, SPT may be
more identiﬁable with only mean and aggregate measures, particularly time in system. For
this reason, one should be leery of dismissing disaggregate information’s usefulness for all
dispatch rule identiﬁcation and reserve judgement for future research.

The disaggregate information was most helpful in the identiﬁcation of the moving
bottleneck (MB) which was also the most dynamic and least stationary of all the conditions.
The moving boulemck is indicative of continuously changing resource constraints that

resulted in continuously changing queue lengths. Thus it may be that disaggregate

142

 

information has its greatest use when shop ﬂoor resorn'ces are in ﬂux and when the rules of
operation are changing dming the course of operation- a very likely scenario in the real

world.

6.4 Usefulness and Clarity of Measures

Expectations regarding the usefulness and clarity of the various meastnes were
borne out with some consistency. For every condition and for both information types,
participants judged as most useful the expected measure at least 75% of the time.
Participants who used aggregate information perceived the same measures as the most
useful regardless of whether or not they correctly identiﬁed the conditions. This was not
the case, however, with the disaggregate group. Participants who incorrectly identiﬁed the
conditions preferred different (incorrect) measures whereas participants who correctly
identiﬁed the condition preferred the correct measrnes. Participants in both groups who
correctly identiﬁed the conditions ranked the expecred measures as most useful.

There appears to be a difference in the way the participants of the two different
groups interpreted the measures. Aggregate measures appeared to confuse the users so that
they ten®d to use the same measures, whether or not they answered correctly. While it
was expected that those using aggregate measures who answered incorrectly would do so
because they used the wrong measures, it was found that they used the same measures as
did those who answered correctly, and those were the measures that were expecwd to lead
to the correct identiﬁcation. This apparent misinterpretation leads to some interesting future
research directions, as discussed in the following sections.

It was also discovered that the participants judged the disaggregate information as
superior on three dimensions- sufﬁciency, adequacy and understandability. The most
important of these three dimensions, in light of the research questions, is that of
understandability; that is, the participants believed that the disaggregate information was
signiﬁcantly more effective in making the shop ﬂoor conditions more understandable. This

143

is an extremely important result of this research in that it addresses the key issue of
condition identiﬁcation through understanding. One goal in presenting the two different
types of information was to test which type was seen as more useful for identifying
conditions and problems. Thus, while the actual results of the identiﬁcation exercise were
mixed, there was no such ambiguity on the part of the participants in labeling the most

useful information.

6.5 Future Research

While this research addressed many issues and resulwd in some interesting
ﬁndings, it also raised some compelling questions that could not be addressed by the
existing research design. Therefore, several extensions to this research, as discussed

below, are being proposed that will address these new issues and questions.

6.5.1 Managers’ Perception of Measures’ Usefulness

The ﬁrst phase of this research consisted of interviewing and surveying two shop
ﬂoor experts to receive their feedback on the relative usefulness of various performance
measures. Although there were no speciﬁc research questions or hypotheses for this
phase, in keeping with the basic theoretical assertion of this research (section 3.2.2), it was
expecwd that disaggregate information would be preferred by the experts. The results of
the survey, however, were very mixed, with the experts divided on the usefulness of a
number of measrn'es, both aggregate and disaggregate. There are several possible
explanations for this, among them being that it was due to the limiwd sample size, that it
was a function of the orientation or background of the experts themselves or was a function
of the nature of the organizations in which they work. The precise factors responsible for
the discrepancy in the responses were not identiﬁed in this research. To better address
these uncertainties, in addition to incorporating a larger sample size, the following research

questions are proposed for future work.

144

 

o Is the preference for disaggregate or aggregate measures a function of the users’
managerial level or their functional area within the enterprise?

0 Is the preference for disaggregate or aggregate measures a function of the type
of company in which they are used, or the stage of deve10pment of the
company?

0 Is the preference for disaggregate or aggregate measures a function of the uses
to which the information is put- descriptive, explanatory or predictive?

These questions can help further deﬁne users’ preferences for certain measure

types. The ﬁrst question addresses speciﬁc work-related characteristics of users and

whether they inﬂuence their preference for disaggregate or aggregate measures. With the
exception of years of manufacturing related experience, these were not evaluated in this
research. The second question explores whether the differences are due to characteristics
of the company at which the information is being used. This could include the type
manufacturing company (job shop, repetitive assembly, continuous flow) or if it is a
service company. It could also address the effect of the company’s development or
maturity on the managers’ information preferences. The third question explores the issue
of depth. How much is being asked of the information? Managers who require only
relatively simple descriptive statistics regarding their shop ﬂoor operations may have
diﬁ'erent preferences or alternate information sources than those who rely on performance
measures to explain unusual behavior or to provide a model for predicting when such
behavior is likely to recur.

To adequately address the proposed questions and to provide sufficient statistical
power to any future analysis, it is recommended that the survey be distributed to a much
wider sample. Expanding the survey would likely involve reducing the complexity of the
instrument to avoid the necessity for interviewing. This approach may lead to the loss of

some insights that were obtained with a smaller group of respondents, but could allow the

145

researcher to more deﬁnitively conclude which measures are deemed more useful, by
whom and for what reason.

A previous sm'vey/interview conducwd by McKinnon and Bruns (1992) involved
contacting numerous manufacturing managers to determine what information was deemed
useful. The insights generated by this research were considerable in that it was one of the
ﬁrst instances in which a large sample of manufacturing executives were interviewed
regarding the usefulness of speciﬁc information. A similar effort is proposed here, but one
that would focus on the relative usefulness of two very distinct types of information,
aggregate and disaggregate measures. Thus it would expand upon McKinnon and Bruns’
work on information’s usefulness, but would deal with the very speciﬁc information types
discussed in this research and provide additional support (or lack of support) for the use of
aggregate or disaggregate measures.

6.5.2 Non Stationary Conditions

The condition that was identiﬁed more readily with disaggregate information, the
moving bottleneck (MB), has already been characterized as being the most dynamic or non
stationary of all the conditions introduced to the shop floor. Conditions other than the MB
were more stationary and were linked to speciﬁc planning procedures (dispatch rule, due
date assignment rule) where rules and resources were ﬁxed at the outset of operations and
not allowed to vary. The MB, on the other hand, as a non stationary condition, represents
a problem for which the symptom is changing over time. Thus, while the simulation model
in this research created an artiﬁcial MB by sequentially varying machine resources, the
result was a set of disaggregate measures similar to those that might be observed when
other non stationary conditions are present. The question that is raised by this ﬁnding is
whether disaggregate measures consistently provide an advantage in identifying other non
stationary conditions.

Problems that lead to non stationary behavior might include those that do not deal

with speciﬁc resource or planning rules, such as varying product mix on the shop ﬂoor,

146

 

 

changing scheduling or dispatching rules, certain order release rules or varying personnel
and equipment resources. The potential presence of these non stationary conditions leads
to the following questions:

a What other shop floor conditions may contribute to or create non stationary

behavior on the shop floor?

0 To what extent are non stationary conditions more readily identiﬁed with

disaggregate measures?

The ﬁrst question addresses the issue of what leads to non stationary behavior on
the shop floor. Addressing this question can provide researchers with a framework for
identifying those conditions for which disaggregate measures may provide an advantage in
identiﬁcation. The second question addresses the issue directly raised- do disaggregate
measures provide an advantage in identifying other non stationary conditions? Exploring
this issue may reveal if the results of this research are generalizable to other conditions.

In addition to MB, other non stationary conditions may include dynamic order
release rules that use shop floor congestion and loading information in their release
mechanism. Release rules similar to this have been explored in the past (Ragatz and
Mabert, 1988; Melnyk and Ragatz, 1989). Another complex environment may involve the
' dynamic use of diﬁ'erent dispatch rules, depending on the conditions on the shop ﬂoor.
For example, the shop may start up using FCFS, but change to SPT or EDD as conditions
change. These situations clearly represent environments in which the operational rules are
changing during the come of operation.

The dynamic environment tested in this research, the moving bottleneck, involved
constraining just the machine resource of the floor. However, employees may similarly be
the conth that leads to a moving bottleneck. Another possibility is the dynamic use of
the tooling resource on the shop ﬂoor, extending the work of Melnyk, Ghosh and Ragatz
(1989). Or, dynamic and complex employee u'ansfer rules, simpler versions of which
were evaluated by Trelevan (1987), could be implemented.

147

 

6.5.3 Dispatch Rules

Identifying the dispatch rule SPT was done more accurately and more quickly with
aggregate information, contrary to expectations. However, the question remains, is this
result unique to SPT, or would the same result hold true with other dispatch rules? SPT is
one of the most local of all dispatch rules and uses only one dimension of job information-
the imminent process time. Other dispatch rules use different dimensions of job
information, such as due date (Earliest Due Date- EDD) or due date and process time
(Minimum Slack- MinSlk). At this point it is not clear if other dispatch rules are equally
more identiﬁable with aggregate measures or if such identiﬁcation is a function of the type
of dispatch rule chosen. To address these issues, the following future research questions
at imposed:

0 Does one type of measure (aggregate or disaggregate) provide a signiﬁcant
advantage in identifying dispatch rules from shop ﬂoor performance
information?

0 Do aggregate meastues consistently result in superior SPT identiﬁcation in all
shop ﬂoor environments?

Repeating this research with different dispatch rules as conditions to be identiﬁed
would help to determine if information type’s effect on identiﬁcation accmacy is dependent
on the individual dispatch rule or, perhaps, on the type of information used by the dispatch
rule. Speciﬁcally, the identiﬁcation of dispatch nrles using one dimension of job
information (process time) could be compared with that of rules using a diﬂ‘erent dimension
(due date). More complex rules, such as MinSlk, that use two dimensions of job
information could also be evaluated. There is some support in this research for the
conclusion that more complex and dynamic conditions are more readily identiﬁed with
disaggregated based measures. Extending the research in this manner to evaluate more

complex dispatch rules could further substantiate this conclusion.

148

 

Actual shop ﬂoor operations are likely to be quite complex. In fact, practitioners
have commented in the past that OM research does not fully reﬂect the complexity of real
world shops (Melnyk, Vickery & Carter, 1986; Schafer, 1997). Thus, as simulated shop
ﬂoa's increase in complexity, they may begin to better approximate their counterparts in the
manufacturing world. As a result, the role of disaggregate information in simulation based
research may well increase.

6.5.4 Users’ Process of Interpreting Information

In the post hoc analysis it was found, somewhat unexpectedly, that the users of
aggregate information tended to use the same measures, regardless of whether or not they
correctly identiﬁed the appropriate condition, whereas those using disaggregate information
used different measures depending on the correctness of their identiﬁcation. Thus, it
appears that the cognitive processes employed by the users to interpret the measures and
information are different depending on the type of information presented. This is shown

graphically in ﬁgure 48.
USER

INFO TYPE AGGREGATE DISAGGREGATE

_e_____’./L _______ ’_/__\:‘__
g g Correct Incorrect Correct Incorrect
2
C
8
‘2 . Correct lnconect Co Incorrect
8 a

Figure 48: Interpretation Process Steps

149

As shown above, participants using aggregate measures tended to make only one
choice for measures, which was the one that was expecwd of them for making the correct
identiﬁcation. However, once that choice was made, these participants made both correct
and incorrect identiﬁcations. Users of disaggregate information, on the other hand, tended
to make distinct choices as to which measures they would use, and from then went on to
make the expecwd identiﬁcation. Thus, the choice of measures was critical to their success
in identifying the correct condition. These results indicate that the decision maldng
process, both in choosing measures and in then identifying the condition, is different for
the two groups. Since the laboratory experimental design only measured the actual
outcomes (measures chosen and conditions identiﬁed), there is no indication of the actual
decision making process. Thus, it is proposed that this issue be explored further to gain a
better understanding of the process. To do this, the following research questions are
proposed.

0 How does the type of measures (disaggregate versus aggregate) inﬂuence the
users’ process of choosing and interpreting the measrues to identify shop ﬂoor
conditions?

0 How does the type of condition introduwd to the shop ﬂoor inﬂuence the users’
process of choosing and interpreting measures to identify the condition?

These two questions begin to address the issue of the information users’ processes
of interpreting the measures while they attempt to identify the conditions. The ﬁrst
question examines the effect, if any, that the information type has on this process. It asks
if the nature of the measures inﬂuences the way users think about them and use them. The
second question is similar, but explores the inﬂuence the type of condition has on process.
This question asks if the condition type (DR, DD, MB, etc.) inﬂuences the way users
formulate their responses. Since this research utilized only a limited number of conditions,
it is not known whether the apparent interpretation discrepancy is due to information type

or condition type.

150

Both questions can be evaluated in a similar setting to that utilized in this research.
It is suggested that similar lab experiments be conducted in the future, but with the
following additions and reﬁnements:

. Have the participants provide concrurent verbal protocols so that the underlying
reasoning processes could be determined. The focus could be on how and why
the choices were made and what the participants found useful and why.

. Include more process related questions into the questionnaire for all

participants.

Summarizing the previous sections is ﬁgure 49, showing a listing of the possible

future research topics related to this work.

 

Managers’ perception of measures’ usefulness
Expand initial survey to include widespread audience
Focus on differences among respondents and their companies

 

Identiﬁcation of other dynamic conditions
Complex order review and release rules
Dynamically changing dispatch rules
Employee or tool resource constraints
Employee transfer rules

Identiﬁcation of other dispatch rules
Other single dimension rules (EDD)
Multiple dimension rules (MinSlk)

 

 

Analysis Users’ Interpretation Process
Expand lab experiment format to include subsequent interviews
Include use of Verbal Protocol Analysis in methodology
Incorporate more process related questions into questionnaire

 

 

 

Figure 49: Summary of Possible Future Research Topics

6.6 Summary

This research makes a signiﬁcant contribution to the understanding of the uses of

aggregate and disaggregate information within a manufacturing environment. The

151

combination of several different methodologies— survey, interviews, semantic data
modeling, simulation modeling and laboratory experimentation- provides a unique
approach to the study of the adequacy of information for managerial mision making.
Through this work the relative beneﬁts and shortcomings of traditional aggregate measures
as well as those derived from a more disaggregate dataset were explored. The results of the
research, while supporting the claims of usefulness of different information types, begin to
answer some of the research questions. In addition, some very compelling future research
issues were also identiﬁed: further examination of the design and use of relational databases
for managing manufacturing performance information; the exploration of additional shop
ﬂoor conditions as to their speciﬁc behavior; and an examination of the actual interpretation
process information users go through when identifying shop ﬂoor conditions. The results
obtained in this research are seen as a very encouraging beginning and it is hoped that they
will aid shop ﬂoor managers and encourage others to ftuther develop this line of research.

152

REFERENCES

Adam, N.R., Bertrand, J.W.M., Moorhead, DC. and Surkis, J. (1993) “Due Date
Assignment Prowdures with Dynamically Updawd Coefficients for Multi-Level
Assembly Job Shops.” European Journal of Operational Research, 68(2), 212-227.

Anderson, J.C., Chervany, N.L. and Narasimhan, R. (1979) “Is Implementation Research
Relevant for the OR/MS Practitioner?” Interfaces, 9(3), 52-56.

Ashby, JR. and Uzsoy, R. (1995) “Scheduling and Order Release in a Single Stage
Production System.” Journal of Manufacturing Systems, 14(4), 290-306.

Bagchi, U., Julien, F.M., Magazine, MJ. (1994) “Note: Due-Date Assignment to Multi-
Job Customer Orders.” Management Science, 40(10), 1389-1392.

Basta, N. (1995) “Making History— Millisecond by Millisecond by Millisecond.”
Chemical Engineering, 102(12), 119-121.

Batini, C., Ceri, S. & Navathe, S.B. Conceptual Database Design- An Entity-Relationship
Approach, Redwood City, CA: Edward Cummings Co., 1992.

Berkowitz, L. & Donnerstein, E. (1982) “External Validity is More Than Skin Deep.
Some Answers to Criticisms of Laboratory Experiments.” American Psychologist,
37(3), 245-257.

Bhimani, A. and Bromwich, M. (1991) “Accounting for Just-In-Time Manufacturing
Systems.” CMA Magazine, 65(1), 31-34.

Blackstone, J.H., Phillips, D.T. and Hogg, GL. (1982) “A State-of-the-Art Survey of
Dispatching Rules for Manufacturing Job Shop Operations.” International Journal of
Production Research, 20(1), 27-45.

Brodie, M.L. “On the Development of Data Models” in On Conceptual Modeling:
Perspectives from Artiﬁcial Intelligence, Databases and Programming Languages, M.L.
Brodie, J.Mylopoulos and J.W. Schmidt, eds., New York: Springer-Verlag, 1984.

Cascio, W.F. and Zedeck, S. ( 1983) “Open a New Window in Rational Research
Planniérég: Adjust Alpha to Maximize Statistical Power.” Personnel Psychology, 36(3),
517-5 .

Centeno, M.A. and Standridge, CR. (1993) “Databases: Designing and Developing
Integrated Simulation Modeling Environments.” Proceedings of the 1993 Winter
Simulation Conference, 526-534.

Chen, PP. (1976) “The Entity-Relationship Model- Towards a Uniﬁed View of Data.”
ACM Transactions on Database Systems, 1(1), 9-36.

Cheng, T.C.E. and Gupta, MC. (1989) “Stu'vey of Scheduling Research Involving Due
Date Determination Decisions.” European Journal of Operational Research, 38(2),
156-166.

153

Cheng, T.C.E. (1989) “A Heuristic for Common Due-Date Assignment and Job
Scheduling on Parallel Machines.” Journal of the Operational Research Society,
40(12), 1129-1135.

Cheng, T.C.E. (1988) “An Alternative Proof of Optimality for the Common Due-Date
Assignment Problem.” European Journal of Operational Research, 37, 250-253.

Christy, DP. and Kanet, JJ. (1988) “Open Order Rescheduling in Job Shops with
Demand Uncertainty: A Simulation Study.” Decision Sciences, 19(4), 801-818.

Codd, ER (1970) “A Relational Model of Data for Large Shared Data Banks.”
Communications of the ACM, 13(6), 377-387.

Colantoni, C.S., Manes, R.P. and Whinston, A. (1971) “A Uniﬁed Approach to the
Theory of Accounting and Information Systems.” The Accounting Review, 46(1), 90-
102.

Denna, E1... Cherrington, 1.0., Andros, DP and Hollander, A.S. Event-Driven
Business Solutions- T oday’s Revolution in Business and Information Technology,
Burr Ridge, IL: Irwin Professional Publishing, 1993.

DeVader, C.L., Bateson, AG. and Lord, R.G. “Attribution Theory: A Meta—Analysis of
Attributional Hypotheses” in Generalizing from Laboratory to Field Settings, E.A.
Locke, ed., Lexington, MA: Lexington Books, 1986.

Dunn CL. and McCarthy, W.E. (1997) “REA Accounting Systems: Historical
Antecedents and Futme Directions.” Working paper, Michigan State University.

Elvers, D.A. and Treleven, MD. (1982) “Job-Shop vs. Hybrid Flow-Shop Routing in a
Dual Resource Consu'ained System.” Decision Sciences, 16, 213-222.

Excel User’s Guide, Redmond, WA: Microsoft Corporation, 1995.
Extendm Users Manual , San Jose, CA: Imagine That, Inc., 1995.

Fromkin, H.L. and Streufert, S. “Laboratory Experimentation” in Handbook of Industrial
and Organizational Psychology, M.D. Dunnette, ed., Chicago: Rand McN ally College
Publishing Co., 197 6.

Geerts, G. and McCarthy, WE. (1995) “The Economic and Strategic Su'ucture of REA
Accounting Systems” Working paper, Michigan State University.

Gere, W.S. (1966) “Heuristics in Job Shop Scheduling.” Management Science, 13(2),
167-190.

Ghosh, S., Melnyk, SA. and Ragatz, GL. (1992) “Tooling Constraints and Shop Floor
Scheduling: Evaluating the Impact of Sequence Dependency.” International Journal of
Production Research, 30(6), 1237-1253.

Gibbons, J.D. Nonparametric Methods for Quantitative Analysis, New York: Holt,
Rinehart and Winston, 1976.

Goetz, BE. (1939) “What’s Wrong with Accounting.” Advanced Management, 4(5),
15 l - 157.

 

154

Goldratt, EM. and Cox, J. The Goal, Croton-on-Hudson: North River Press, 1992.

Goldratt, E.M. What is this Thing Called THEORY OF CONSTRAINTS and How Should
it be Implemented? Croton-on-Hudson: North River Press, 1990.

Goodwin, 1.8. and Goodwin, J .C. (1982) “Operating Policies for Scheduling Assembled
Products.” Decision Sciences, 13(4), 585-603.

Grabski, S.V. and Marsh, RJ. (1994) “Integrating Accounting and Manufacturing
Information Systems: An ABC and REA-Based Approach.” Journal of Information
Systems, 8(2), 61-80.

Groff, GK. and Clark, T.B. (1981) “Commentary on ‘Production/Operations
Management: Agenda for the 808’.” Decision Sciences, 12(3), 578-581.

Gupta, U.G. (1994) “An Empirical Investigation of the Conuibution of Information
Systems to Productivity.” Information Management, (March/April), 15-18.

Harty, JD. (1969) “Controlling Production Capacity.” Proceedings of the 12th Annual
Conference of the American Production and Inventory Control Society, 60-64.

Hax, A.C. (1981) “Commentary on ‘Production/Operations Management: Agenda for the
808’.” Decision Sciences, 12(3), 574-577.

Helix Express User’ s Guide, Northbrook, IL: Helix Technologies, Inc., 1994.

Hill, D.T., Koelling, CF. and Kurstedt, HA. (1993) “Developing a Set of Indicators for
Megsugng Igrfggrmation-Orienwd Performance.” Computers in Industrial Engineering,
24( )p 7 " o

Hitz, M., Werthner, H. and Oren, T.I. (1993) “Employing Databases for Large Scale
Reuse of Simulation Models.” Proceedings of the 1993 Winter Simulation Conference,
544-5 5 1 .

Hollander, A.S., Denna, EL. and Cherrington, 10. Accounting, Iry‘ormation
Technology, and Business Solutions, Chicago: Irwin, 1995.

Huq, F. and Juq, Z. (1995) “The Sensitivity of Rule Combinations for Scheduling in a
Hybrid J 3b Shop.” International Journal of Operations and Production Management,
15(3), 5 -75.

Johnson, HT. and Kaplan, R.S. Relevance Last: The Rise and Fall of Managerial
Accounting, Boston: Harvard Business School Press, 1987.

Kaplan, RS. and Norton, DP. (1992) “The Balanced Scorecard— Measures That Drive
Performance.” Harvard Business Review, 70(Jan-Feb), 71-79.

Keppel, G. Design and Analysis: A Researcher’s Handbook, Englewood Cliffs, NJ:
Prentice Hall, 1991.

Knight, R. (1994) “New Architectures Aligning Manufacturing and RDBMSs.” Software
Magazine, 14(3), 65-68+.

155

 

Koh, K—H., de Souza, R. and Ho, N-C. (1995) “Direct Database Simulation of a Job-
Shop.” International Journal of Production Economics, 39(3), 281-287.

Law, A.M. and Kelton, W.D. Simulation Modeling & Analysis, New York: McGraw-Hill
Inc., 1991.

Law, A.M., McComas, M.G. and Vincent. S.G. (1994) “The Crucial Role of Input
Modeling in Successful Simulation Studies.” Industrial Engineering, 26(7), 55-59.

Law, A.M. and Vincent, S.G. Uniﬁt II User’s Guide, Tucson, AZ: Averill M. Law and
Associates, 1993.

9

Lee, Y.C. & Fu, KS. (1985) “A Relational Approach to the Integraed Database
Management for Computer-Aided Manufacturing.” in AST M Special Technical
Publication 862, L.B. Gardner, Ed, American Society for Testing and Materials,
Philadelphia, 1985, 150-162.

Levene, H. “Robust Tests for Equality of Variance” in Contributions to Probability and
Statistics, I. Olkin, ed, Palo Alto, CA: Stanford University Press, 1960.

Lingayat, S., Mittenhal, J. and O’Keefe, RM. (1995) “Order Release in Automated
Manufacturing Systems.” Decision Sciences, 26(2), 175-205.

Locke, E.A. “Generalizing from Laboratory to Field: Ecological Validity or Abstraction of
Essential Elements?” in Generalizing from Laboratory to Field Settings, E.A. Locke,
ed, Lexington, MA: Lexington Books, 1986.

Magnan, G. An Analysis of the Relationship Between Selected Manufacturing Strategies,
Production Competence and Competitive Priority Compentence. Unpublished doctoral
dissertation, East Lansing, MI: Michigan State University, 1994.

Malhotra, M.K., Jensen, J.B. and Philpoom, PR. (1994) “Management of Vital
Customer Priorities in Job Shop Manufacttuing Environments.” Decision Sciences,
25(5/6), 71 1-734.

Marsh, R.J. (1996) “A Database Approach to Shop Floor Information Management.”
Proceedings of the 1996 Midwest Decision Sciences Institute, 103-107.

Marsh, R.J. and Melnyk, SA. (1994) “Identifying Sources of Job Sho Variance: Using
Empirical Data Mthin a Simulation Study.” Proceedings of the I Annual Meeting
of the National Decision Sciences Institute, 1696-1698.

Marsh, R.J. and Melnyk, SA. (1995) “Assessing the Impact of Changes in Strategic
Focus Through Computer Simulation: Describing the Baseline Study.” Proceedings of
the 1995 Annual Meeting of the National Decision Sciences Institute, 1418- 1420.

Marsh, R.J. and Vickery, S.K. (1995) “Relational Databases in Manufacturing: An MRP
Implementation and Framework for Cross Functional Integration.” Proceedings of the
1995 Annual Meeting of the National Decision Sciences Institute, 1277-1279.

Maybury, M.T. (1995) “Generating Summaries From Event Data.” Information Processing
& Management, 31(5), 735-751.

156

McCarthy, WE. (1979) “An Entity-Relationship View of Accounting Models.” The
Accounting Review, 54(4), 667-685.

McCarthy, W.E. (1982) “The REA accounting model: A generalized framework for
accounting systems in a shared data environment.” The Accounting Review,
57(3): 554-578.

McKinnon S.M. and Bruns, WJ. The Information Mosaic: How Managers Get the
Information They Really Need. Boston: Harvard Business School Press, 1992.

Melnyk, S.A., Carter, P.L., Dilts, D.M. and Lyth, D.M. Shop Floor Control,
Homewood, IL: Dow Jones Irwin, 1985.

Melnyk, S.A., Ghosh, S. and Ragatz, G.L. (1989) “Tooling Consu'aints and Shop Floor
Scheduling: A Simulation Study.” Journal of Operations Management, 8(2), 69-89.

Melnyk, S.A., Pagell, M., Jorae, G. and Sharpe, A.S. (1995) “Applying survival analysis
to operations management: Analyzing the differences in donor classes in the blood
donation process.” Journal of Operations Management, 13(4), 339-356.

Melnyk, SA. and Ragatz, G.L. (1989) “Order Review/Release: Research Issues and
Perspectives.” International Journal of Production Research, 27(7), 1081-1096.

Melnyk, S.A., Vickery, SK. and Carter, PL. (1986) “Scheduling, Sequencing and
Dispatching: Alternative Perspectives.” Production and Inventory Management, 27(2),
58-68.

Mook, DC. (1983) “In Defense of External Validity.” American Psychologist, 38(4), 379-
387.

Nandakumar, G. (1990) “Bill of Material Processing with a SQL Database.” Computers &
Industrial Engineering, 18(4), 471-483.

N avathe, SB. (1992) “Evolution of Data Modeling for Databases.” Communications of the
ACM, 35(9), 112-123.

N eter, J. and Wasserman, W. Applied Linear Statistical Models, Homewood, IL: Richard
D. Irwin, Inc., 1974.

Panwalker, 8.8. and Iskander, W. (1977) “A Survey of Scheduling Rules.” Operations
Research, 25(1), 45-61.

Plenert, G. (1993) “Optimizing Theory of Constraints When Multiple Consu'ained
Resources Exist.” European Journal of Operational Research, 70(1), 126-133.

Podsakoff, RM. and Williams, L]. “The Relationship between Job Performance and Job
Satisfaction” in Generalizing from Laboratory to Field Settings, E.A. Locke, ed,
Lexington, MA: Iexington Books, 1986.

Ragatz, G.L. and Mabert, VA. (1988) “An Evaluation of Order Release Mechanisms in a
Job-Shop Environment.” Decision Sciences 19, 167-189.

157

Ragatz, G.L. and Mabert, V.A. (1984) “A Framework for the Study of Due Date
Management in Job Shops.” International Journal of Production Research, 22(4), 685-
695.

Roberts, R.S. (1991) “Simulation Languages and Database Theory: Some Considerations
from the Entity-Relationship Model.” Proceedings of the 1991 Winter Simulation

Conference, 1228-1235.

Roy, R. and Meikle, SE. (1995) “The Role of Discrete Event Simulation Techniques in
ﬁnite Capacity Scheduling.” Journal of the Operational Research Society, 46(11),
13 10- 1321 .

Schafer, R. (1997) “A letter on the Ourent State of OM Research.” Newsletter of the
Midwest Decision Sciences Institute, 9(1), 3-4.

Shooshtarian, F.D., Chang, D., Dong, J. & Parsaei, HR. (1993) “Design and
Implementation of a Relational Data Base for Automaed Process Planning.” Computers
and Industrial Engineering, 25(1-4), 309-312.

Skinner, W. (1969) “Manufacturing- Missing Link in Corporate Strategy.” Harvard
Business Review, May-June.

Smith, J.M. and Smith, D.C.P. (1977) “Database Abstractions: Aggregation and
Generalization.” ACM Transactions on Database Systems, 2(2), 105-133.

Son, Y.K. (1990) “A Performance Measurement Method Which Remedies the
‘Productivity Paradox’.” Production and Inventory Management Journal, 31(2), 38-42.

Stevenson, W.J. Production/Operations Management, Sixth edition, Chicago: Richard D.
Irwin, 1999.

Stone, E.F. “Job Scope-Job Satisfaction and Job Scope-Job Performance Relationships” in
Generalizing from laboratory to Field Settings, E.A. Locke, ed, Lexington, MA:
Lexington Books, 1986.

SYSI‘ AT: Statistics, Version 5 .2 Edition. Evanston, IL: SYSTAT, Inc., 1992.

Trelevan, M. (1987) “The Timing of Labor Transfers in Dual Resornce-Constrained
Systems: ‘Push’ vs. ‘Pull’ Rules.” Decision Sciences, 18(1), 73-88.

Trelevan, M. (1989) “A Review of the Dual Resorn'ce Constrained Systems Research.” IIE
Transactions, 21(3), 279-287.

Tukey, J .W. (1951) “Quick and Dirty Methods in Statistics, Part 11: Simple Analyses for
Standard Designs.” Proceedings of the 5th Annual Convention of the American Society
for Quality Control, 189-197.

Wang, M. & Walker, H. (1989) “Creation of an Intelligent Process Planning System
within the Relational DBMS Software Environment.” Computers in Industry, 13(3),
215-228.

Weeks, J. and Fryer, J. (1977) “A Methodology for Assigning Minimum Cost Due Dates.”
Management Science, 22(8).

158

Wisner, JD. (1992) “A Review of the Order Release Policy Research.” International
Journal of Operations and Production Management, 15(6). 25-40.

Yamaguchi, K. Event History Analysis, Newbury, CA: Sage Publications, 1991.
Yancey, DP. (1987) “Database Management Systems Can Provide Way to Manage

Information Generated in a Computer Simulation Program.” Industrial Engineer,
19(5), 50-53.

159

APPENDIX A

Simulation Model Description
Shop Floor Conditions Description
Disaggregate Measmes Materialization

Perceived Measure Usefulness Presentation/Discussion

160

SIMULATION MODEL DESCRIPTION (section 4.3.2.1)

A.1 Job Types

There are four primary types of jobs in the shop, each of which has its own
characteristics in terms of size, interamival times, routing and due date assignment. The job
types are Projects, Walk-ins, Quick Response and Cribs.

W3 These jobs are part of large multi-job orders (10-180 jobs), which are
inmoduced to the shop in relatively large batches of 1060 jobs at a time. All jobs within a
project are typically assigned one aggregate the date. While the shop historically has
handled a relatively low proportion of project jobs in its total workload, this has been
changing in recent years with the proportion of project jobs increasing. This proportion is
afactorinthisresearch, aswillbediscussedlater.

W: These jobs enter the shop more continuously than projects, with
their interarrival times varying by tle day of the week and the particular shift. They consist
of smaller work, usually only one job. The customer service clerk makes a determination
upon each job’s entry as to its status as a regular walk in or a quick response, which is
discussed in the next paragraph. The jobs designated simply “walk in” are routed and
scheduled normally.

W: Quick response jobs are walk in jobs that have been
designaed by the customer service clerk as emergency. They are expedied through the
shop and have an overriding priority in a work center’s queue.

91.111.10.123: These jobs are inventory replenishment items for other divisions of the
company. They also enter the shop fairly continuously but only during day shift on
weekdays and represent a relatively steady load on the shop. They are of a recuming nature
and thus are familiar to the operators and schedulers and do not represent near the
scheduling challenge as do the other types.

161

A.2 Job Routing

The routing for each job will vary by its job type and is determined by empirically
derived tables. Approximately 500 historical job records were examined to determine the
routing patterns of the various job types. It was found that the work center to which a job
was likely to proceed was a function of its present work center. Thus, the shop
experienced sequence dependent routing. This is capttmed within the model with
probability look up tables. Figure 50 is a portion of such a routing table.

 

Work Center

 

 

 

 

Figure 50: Work Center Routing Look up Table

Routing of jobs in the model is accomplished in the following manner: The model
generates a random number on the interval (0, l), which is used to determine to which
work center the job will be sent. For example, if the job is at work center #5 and the
random number generated is 0.457, the next stop for the job will be at work center #3,
since 0.517 is the ﬁrst value in row 5 greater than or equal to 0.457, and is in column 3, as
shown by the shaded cells in ﬁgure 56. Note that if a job is in work center #4, it has no
probability at all of being roued to work center #2, since there is no difference in the
cumulative values in columns 1 and 2 in row 4.

A3 Process Times

The same 500 historical jobs that were reviewed to determine job routing were also
analyzed to determine jobs’ process times at the various work centers. It was found that
process times were independent of job type, but were dependent on the work center. For
example, all job types have the same disuibution of process times at, say, the lathe.

However, the disuibution of process times at the boring mill will be different than at the

162

lathe. This is captured within the model with an additional probability look up table. An
example of such a table is shown in ﬁgure 51.

 

 

 

 

 

Figure 51: Process Time Look up Table

As with the routing table, the model generates a random value on (0, 1) and looks
up a process time, depending on the work center. For example, if a job is scheduled to go
to work center #4, and tie generated random value is 0.321, the process time chosen will
be 2.00 hours, since 0.338 is the ﬁrst value in row 4 that is greater than or equal to 0.321,
and is in the column labeled “2.00,” as shown in the shaded cells in ﬁgure 51. Note that a
job in work center #4 will never have a process time of 1.50 hours, since the value in that
cell (0.251) is unchanged from that in the cell to its immediate left.

163

SHOP FLOOR CONDITIONS DESCRIPTION (section 4.3.2.3)

A.4 Bottleneck Resomce

Both a stationary and moving bottleneck were conditions inserted into the model.
In both cases, the actual constraint was simulated by a very limited number of machines in
a particular work center. To establish a moving bottleneck, machine availability was
deliberately scheduled such that various work centers had very limited capacity (1-2
machines versus 4-6) at varying sequential times. This caused the bottleneck to move
throughput the shop.
A.5 Irregular order or job releases

Dming a previous empirical study of the subject shop (Marsh and Melnyk, 1994;
1995), it was determined that Project Jobs had order release rules that could be altered.
These release mechanisms ranged from continuous individual releases to very disjointed
and large batch (60+ jobs) releases. Therefore, in this research, the effect of such varying
project job releases was studied. The actual practice of releasing large batches of jobs was
modeled, as was a more continuous job release.
A.6 Due Date Assignment

The shop being modeled experienced a similar problem as has been repored in the
literature (section 2.6.3)— that of one common due date versus individual due dates for jobs
within one order. Project Jobs had traditionally been given one common due date, despite
the fact that the actual dates the various jobs are needed can be quite different. In some
cases, a project may have an overall due date six months in the future, but certain
components are needed within a few weeks. Similarly, some components are not required
for several months. However with a common due date method, all components are
assigned the same due date— six months out. For this reason, and in conjunction with
previous reported research (Cheng, 1988; 1989), it was determined that evaluating the

differences in performance attributed to common versus individual job due date assignment

was appropriate.

164

A.7 Different Job Types

Another experimental condition introduced into the model is the mix of job types,
that is, how many of each job type there will be. It was found that for the actual shop
studied, the percentage of job mix can be a crucial strategic, as well as operational factor.
For example, taking on more large project work in lieu of emergency work shifts the shop
away from being a service and cost center towards being its own proﬁt center. But, due to
the jobs’ different natme, moving to such a job mix has operational ramiﬁcations for
management as well. The suategic perspective of varying job mix is described in more
detail in Marsh and Melnyk (1995). In this research, the mix is varied by adjusting the
interarrival times for project and emergency (quick response) jobs.
A.8 Dispatch Rules

The ﬁnal inde condition is the choice of dispatch rule. Several different rules
were used, each one constituting a different model run and resulting measrnement or data
set. The dispatch rules studied are First Come First Served (FCFS), Shortest Process Time
(SPT) and Earliest Due Date (EDD). For this research, only FCFS and SPT were given to
the laboratory participants.

165

DERIVATION/MATERIALIZATION OF DISAGGREGATE MEASURES (section 4.4.3)
A.9 Derivation of Time_out

Helix Express® is what is known as a non-SQL relational database, meaning it
does not use the standard Structured Query Language (SQL) queries common among
relational databases (Date, 1981). The queries used to derive Time_out and other values
are not familiar SQL commands but rather are created with objects within the Helix
database. Previous research has shown this to be effective in deriving other operational
data from a disaggregate dataset (Marsh, 1996). Figure 52 shows the query used to derive
Time_out within the JOBS relation. Note that it is obtaining its information from the

OPERATIONS relation.

    

Calculation for Time_out in OPERATION

  
    

for in

     
 

job-run job-mn I: Operation

     
 

Query for Time_out from JOBS into OPERATION
Figure 52: Job Completion Time (Time_out) Query/Calculation in Database

Time_out is being calculated in the OPERATTON relation as shown, and is being
queried by and extracted to the JOBS relation by the “Sub Max” procedure. This procedure
searches the OPERATION relation for the maximum Time_out of each Job and returns it.

Note that this procedure queries by Job-Run. This is because the actual database contains

166

information from all seven runs of the model and the runs must be distinguished from each
other in order to reuieve the correct value for the appropriate condition. A double,
concatenated key is used for this since Jobi‘t by itself is no longer unique in the
OPERATTON table.
A. 10 Derivation of Operational Measures

Within the Excel spreadsheet, the following measurement information was derived:
Time in System (T SYS), Lateness- all jobs and project jobs (LATE), Tardiness (TARDY)
and Due Date Allowance (DD Allow). To accomplish this and to present the data
graphically, additional reﬁnements were necessary. For instance, TSYS is the difference
between a Job’s entry and completion times. However, for those Jobs which were not yet
completed as of the end of tie simulation model’s run, there would be no completion time
(Time_out is a null value) and TSYS will be undeﬁned The same concept applies to LATE
and TARDY— a completion time is needed to calculate each. To accomplish this, some
relatively simple formulas were used within Excel, the ﬂowchart logic of which are shown
in ﬁgures 53 and 54.

 

 

 

 

Time_out - Time_in

I

LATE- LATE . Undefined
Time_out - JobDD

TSYS .. J I TSYS - Undefined I

 

 

 

 

 

 

 

 

Figure 53: TSYS and LATE Calculations within Spreadsheet

167

 

 

 

 

 

 

TARDY - TARDY - Undefined
Time_out - JobDD

 

 

 

 

 

Figme 54: TARDY Calculation within Spreadsheet

The calculations shown in ﬁgures 53 and 54 are done quite simply within Excel.
Note that TARDY has an additional conditional step, to determine if the the Job was indeed
tardy, or behind schedule. If not, TARDY is undeﬁned

DD Allow is calculated with a simple non-conditional calculation as follows:

DD Allow = JobDD - Time_in

While the simulation model does calculate a due date allowance when setn'ng each
Job’s due date, only the date itself is captured and transferred to the database. Thus, the
allowance has to be recalculated based on the entry time and the assigned due date.

In the spreadsheet, then, the following values make up the JOB worksheet:

Job I Job I Total I TimeIT
Job# 1])

PT out rmeI TSYS I LATE I TARDY IDD Allow

IT | l l l l l

Excel’s Histogram tool was then used to create the required distributional graphs.
TSYS was calculated for all jobs as was LATE, TARDY and DD Allow. LATE was
further compiled just for the Project jobs by sorting the worksheet based on Job Type
(project job type = 1) and creating a histogram just for that type.

It is important to note that the calcuations for TSYS, LATE, TARDY and DD Allow
could have readily been done within the relational database, with the completed values

168

transferred to the spreadsheet. The decision was made to use Excel to derive the values
simply for computational convenience. Since all the data needed for these values was
completely within the JOBS relation and no “look ups” to other relations were necessary,
they could be derived outside of the database. For demonstration purposes, however,
ﬁgure 55 shows the calculation for TARDY as it would have been done within Helix.

 
 

>

  
  

Time_out Job_DD

   

Blank indicates undeﬁned

  

Time_out

    

TARDY or undeﬁned value returned

 

Figure 55: TARDY Calculation in Database

There were seven runs made of the model resulting in seven different worksheets in
Excel for which the above calculations and graphs were made.
A.11 JOB-WC Relation Measures Materialization

Two additional measures/information sets were derived for presentation to the lab
participants: Time Series of Queue Lengths and Jobs Processed per day. The data for these
were taken from the JOB-WC relationship and the OPERATION entity, respectively.

169

Time Series of Queue Len gths presents the queue length data in average number of
Jobs in the queue per day. That is, it plots daily average quete lengths versus the days of
the model’s run. Since the database is quite disaggregated and records queue lengths for
each job, many of which may enter the queue (JOB-WC relation) in a given day, some
initial aggregationwas needed to derive daily average queue lengths. For example, ﬁgure
56 shows a portion of the data ﬁle from the JOB-WC relation, including queue length.

J ob# WC# Time_in_Q Q

       

Figure 56: Portion of JOBS-WC Relationship Table

It can be observed above that the number of records of queue length even within the
short period of time shown (0.25 hours) can be quite large. Aggregating these records per
day rendered them much more interpretable without sacriﬁcing any precision. To achieve
this aggregation, an additional entity and relationship were set up within the database:
DAYS and DAYS-WC. Each day within DAYS was the aggregation of 24 horns of run
time. DAYS-WC is a relationship necessary to hack the daily queue lengths per work
center (WC). Figure 57 shows the query used to calculate average daily queue length per

work center.

170

 

Figure 57: Daily Average Queue Length Calculation in DAYS-WC

Helix has a built in Average procedure that allows us to easily obtain the value
required. Note that the query is by Day and WC into JOB-WC. Each record in JOB-WC
is within a given day, thus this search criteria. And, as before, the records are further
queried by the Run#, which indicates the storage of multiple runs within the database. The
above procedrne returns the daily average queue length by work center to DAYS-WC from
where the records are transferred to the spreadsheet.

The other measure derived from the JOB-WC and OPERATION tables, Jobs
Processed per day, is a distributional look at work center activity. It is a histogram of the
number of jobs processed each day by the entire shop (all work centers). It tracks
individual operations completed per day, but not necessarily completed jobs. That is, many
jobs will have operations completed in a given day, but far fewer jobs will actually be
completed in their entirety. For this measure, of interest is the number of operations
completed daily. To determine this, DAYS and DAYS-WC can be used in the database.
Figure 5 8 shows the calculation for this value.

171

 

Figure 58: Number of Jobs Processed per Day Calculation in DAYS-WC

For this calculation, another of Helix’s built in procedures was used: the Counting
function. This procedure merely counts the number of records per day per work center in
OPERATION.

The result of the above queries and calculations is the following text ﬁle in the JOB-
WC relationship:

Day# | WC# I # ofjobs I Q Lengﬂ
I | I
Once this ﬁle is in Excel, the number of jobs per day is summed across all work
centers. These values are then used to create a histogram of the distribution of work center

activity. The queue lengths are put into a single time series graph showing the time varying

lengths of all four work centers’ queues.

172

EVALUATION OF LAB PARTICIPANTS’ USEFULNESS RANKINGS (section 5.4.6)

A.12 Disaggregate Group

Figure 59 indicates the mean, standard deviation and sample size for each condition
and measrue for participants who correctly identified the condition (0 S Gap S 1) while
using the disaggregate based measures. Shown in the ﬁnal row is the total number of
respondents who correctly identiﬁed each condition.

 

 

 

 

 

 

 

 

 

 

 

 

 

easure Condition D ' D" I" SB I“
. ‘ ' . . .01
Distribution of Time in System (1.47) (1.18) (0.94) (1.74) (1 10)
I6 7 3 22 18
1M 5.46 5.36 3.13 3 IS
Distribution of Job Lateness (1.25) (0.49) (1.36) (1.10) (1 17)
9 15 5 I7 13
5.67 5.76 5.56 3 76 3 33
Distribution of Project Job Lateness (1.70) (0.73) (1.35) (1 l 1) (1 24)
9 17 6 17 14
3.63 31% 3.73 4 44 .
Distribution of Job Tardiness (1.49) (1.49) (1.48) (1 06) (0.91)
8 9 4 I6 14
. 1.00 4.00 530 4 60
Distribution of Due Date Allowance (2.17) (0.00) (1.22) (2.10) (2 15)
4 19 4 10 10
5.64 5.67 4.00 I 3'8— 1 36
Time Series of Queue Length (1.44) (1.70) (2.00) (0 81) (0 71)
14 6 5 24 20
3.72 6.00 637— 3 73 3.76
Distribution of Job Process Activity (1.31) (1.00) (0.47) (1 79) (2.19)
9 6 3 I6 10
TOTAL RESPONDENTS l 6 l 9 7 2 4 2 0

 

 

 

 

 

 

 

 

Figure 59: Usefulness Rankings for Correct Identification Using Disaggregate Data-
Mean, (Standard Deviation) and Number Responding

Figure 60 represents similar information as shown in ﬁgure 59 above, but
summarizes the usefulness rankings by those participants using disaggregate data and who

incorrectly identiﬁed the condition (wrong answers- Gap 2 2).

173

 

 

 

 

LIV-lemme CbnﬁdmlDRlDDlIﬁlSBlggﬁz!

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Distribution ofTimein System (1:26) (0:00) (2.23) (1:00) (1:50)
9 1 11 2 4
4.37— 5.155 5.33 5.36 5.73 '
Distribution ofJob Lateness (1.05) (0.82) (1.72) (0.50) (1.48)
7 3 11 2 4
456—767 3.09 130—. TOT ‘
Distribution of Project Job Lateness (1.38) (1.25) (1.83) (2.50) (1.00)
6 3 11 2 4
3.38 3.37 "—333— 1.00 4.00
Distribution of Job Tardiness (1.43) (1.25) (2.06) (1.00) (1.26)
8 3 9 2 5
4.40 1.80 3.73 W736—
Disuibution of Due Date Allowance (2.42) (0.75) (1.66) (0.00) (1.50)
5 5 11 2 4
1.36— 7.135 3.33 3.36 5.33
Time Series of Queue Length (0.81) (0.00) (1.82) (1.50) (2.34)
10 1 11 2 6
443—130". "47. 0 6.00 2'75".
Distribution of Job Process Activity (2.50) (0.50) (2.44) (0.00) (1.92)
7 2 10 1 4
TOTAL RESPONDENTS 10 5 15 2 6

 

Figure 60: Usefulness Rankings for Incorrect Identiﬁcation Using Disaggregate Data-
Mean (Standard Deviation) and Number Responding

To clarify the above tables, consider some comparisons. For instance, for those

who correCtly identified the DR condition (ﬁgure 59), the most useful measure (highest

ranked) was Distribution of Time in System, which was also the most widely used (16 out

of 16 respondents). On the other hand, ﬁgure 60 shows that of those who incorrectly

identiﬁed the DR condition, the most highly ranked measme was Time Series of Queue

Length, which was used by all respondents. This indicates that the perceived most useful

measure for the same condition varies depending on how successful the participants were

in identifying the condition, as expected. Similarly, those who correctly identifcd DD

ranked Distribution of Due Date Allowance as the most useful, while those who made an

incorrect identiﬁcation felt that Distribution of Time in System was the most useful.

174

However, with only one incorrect respondent ranking distribution of time in system, few
conclusions can be drawn. For those who correctly identiﬁed the two bottleneck
conditions, SB and MB, there was a strong reliance on the queue length measure, with it
being the most highly ranked for correctly identifying both conditions. Those respondents
who incorrectly identiﬁed the bottleneck conditions used the queue measures, but did not
rank them as the most useful. This was particularly the case for SB, where Trme Series of
Queue Length was ranked as the second to the least useful. The measures ranked as the
most useful by those respondents for SB and MB, Time in System and Job Lateness/Job
Process Activity, respectively, are not those that would be expected to be useful in this
identiﬁcation. Note that those measures ranked most highly for the DR, DD, SB and MB
conditions also had either the lowest or next to the lowest standard deviation of responses,

indicating relative uniformity of feeling of usefulness, and at or near unanimcus usage.

A. 13 Aggregate Group
The following tables summarize the usefulness rankings of the participants using

the aggregate based measures. The ﬁrst table below (ﬁgure 61) is for those correctly
identifying the condition (0 S Gap 5 l)

175

 

 

TimeinSystem (0.79) (1.15) (0.97) (0.77) (1.03)

 

Job Lateness (1.20) (0.70) (1.45) (0.85) (0.99)

 

JobTardiness (1.00) (1.11) (1.83) (1.09) (1.18)

 

DueDateAllowance (1.22) (0.95) (1.39) (1.58) (1.68)

 

 

 

IQueue Lengths (1.12) (1.00)

 

 

Machine Utilization (1.11) (0.47) (1

 

 

 

 

 

 

 

 

TOTAL RESPONDENTS 2 4 2 l

 

Figure 61: Usefulness Rankings for Correct Identiﬁcation Using Aggregate Data- Mean
(Standard Deviation) and Number Responding

Figure 62 summarizes the results from the group using aggregate data but
incorrectly identifying the condition (Gap 2 2).

176

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Time in System (0.00) (0.80) (1.37) (0.47) (0.74)
1 5 10 3 14
3166— 75387—5738— 730 6.08
Job Lateness (0.00) (1.10) (0.86) (0.50) (10.5)
1 5 12 2 12
4.06— 3185 5.88 1.33 5.36
Job Tardiness (0.00) (1.00) (1.11) (1.70) (1.36)
1 2 16 3 15
6.00 1.60 4.38— 130". 733‘.
DueDateAllowance (0.00) (0.80) (1.11) (0.50) (1.37)
1 5 8 2 6
5187— 3.36 5.67 1.6 5.18
toueue Lengths (0.00) (0.87) (1.65) (0.00 (1.27)
1 2 15 3 11
W. "5'30". ‘5'1'7. 4.00 T71“.
Machine Utilization (0.00) (0.50) (1.21) (2.00) (1.67)
1 2 6 2 7
TOTAL RESPONDENTS 1 5 1 6 3 1 5

 

 

Figure 62: Usefulness Rankings for Incorrect Identiﬁcation Using Aggregate Data— Mean
(Standard Deviation) and Number Responding

In evaluating ﬁgures 61 and 62, similar results can be observed as thosepresenwd
previously for the disaggre gate information group. For those who correctly identiﬁed DR,
the most useful measme, as expected, was Time in System and the measure was used by
almost all respondents. Moreover, Time in System had the lowest standard deviation,
indicating broad agreement on the measure. However, for those who incorrectly identiﬁed
DR, Time in System was also ranked as the most useful, somewhat contrary to
expectations. All of those who both correctly and incorrectly identiﬁed DD ranked due date
allowance as the most useful, which was contrary both to expectations and to what was
found in the disaggregate group. As expected, all of those who correctly identiﬁed the
bottleneck conditions, SB and MB, ranked the queue lengths as most useful measure.
Among those who incorrectly identiﬁed the bottleneck conditions, time series of queue

length was deemed as most useful for SB while time in system was ranked highest for MB.

177

The expected most useful moving bottleneck identiﬁer, queue lengths, was ranked as next

most useful, but was not as widely used.

178

APPENDIX B

Survey Form to Shop Floor Experts

Summary of Experts’ Responses

179

Experts' Survey Appendix B

PERFORMANCE MEASUREMENT EVALUATION

The following pages are possible performance measmes that can be obtained from a
simulated job shop. The intent of this exercise is to gauge your assessment of the
measures’ usefulness from a shop ﬂoor management standpoint and on several
dimensions. To that end, ﬁve tentative dimensions (A-E) of evaluation are indicated in the
columns. ' .- m. w. t tt .-1' m it -c '1 . 011' Van . 1-, m, ~.- 3' H,

mm on each dimension. The dimensions are provided below:

A. Winged: How well can this measm help shop
management align their goals with those of the overall corporation? Granted,

this will depend somewhat on the varying goals of different companies, so
please answer this question in terms of your own experience.

B. W- How well does this measure provide a
good instantaneous snapshot of what is happening on the shop ﬂoor? Can it
be used by management to gauge what is happening dynamically from
“snapshot to snapshot.”

C. W- Does this measure enhance the day-to-day or week-to-
week monitoring as you deem necessary for shop floor management?

D. Emblemldemiﬁcatign- Does this measure help to identify problems on the
shop ﬂoor? Can it enable management to see things that may not be possible
with other measures? Please use a broad deﬁnition of “problem” and provide
some explanation of the speciﬁc problems you envision.

E. W- Does this measure help to identify the direct or indirect
cause of problems? This is distinct from (D) above in that it goes beyond
merely identiﬁcation and goes to the heart of the problem. Why does the
problem exist? Again, please use a broad deﬁnition of “problem” and if
possible provide some explanation of the speciﬁc problems you envision.

180

Experts’ Survey Appendix B

Usefulness Scale: 1=very useful; 5=not useful at all
Aggregate Operational A B C D E

 

Mean time in system
Mean/maximum lateness
Mean/maximum tardiness

 

 

 

Mean/maximum queue length

 

 

 

 

 

 

 

 

 

 

 

 

Additional comments:

181

Experts' Survey Appendix B

Usefulness Scale: 1=very useful; 5=not useful at all

 

Financial

, Throughput"
lnventory"
Operational expense"

Shop net proﬁt (loss)
monthly or for period

 

 

 

 

 

Mean cost per job

 

 

 

 

 

 

Mean revenue per job

 

 

 

 

 

* As deﬁned by Goldratt (The Haystack Syndrome, 1990)
Throughput: The rate at which the system (shop) generates money through sales.
Inventory: All the money that the system has inveswd in purchasing things
which it intends to sell.
Operational Expenses:
All the money the system spends in order to turn inventory into
throughput.

Additional comments:

182

Experts' Survey

Usefulness Scale: 1=very useful; 5=not useful at all

Detailed Operational

A

B

Appendix B

 

ENE .11.:

 

Job Type

 

Sequence of routing

 

Employees involved

 

RawMaterialCost

 

Sales Price versus labor
and/or material cost

 

 

 

 

 

 

 

W

 

Distribution of activity
by shift. job type

 

Distribution of process
times, IATs

 

Single bottleneck
identiﬁcation

 

Double or sequential
bottleneck identiﬁcation

 

Distribution of queue
lenggs, by job UPC. Shift

 

Vendor delivery
performance

 

Scrap/rework reports

 

WIP Inventory reports

 

 

 

 

 

 

 

 

 

 

183.

 

Experts’ Survey Appendix B

11111"! a]
mldMseful

Usefulness Scale: ' 1=very useful; 5=not useful at all
Additional Measures A B C D E

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Please indicate reasoning (why) for any added above:

184

Experts’ Survey Appendix B

RESPONSE SUMMARY

Usefulness Scale: 1=very useful; 5=not useful at all

 

 

 

 

Aggregate Operational A B C D B
canmmsysm WA.--1--‘b--Z--J.--Z--qp--£-Dib-Cﬁ-Cd
Mean/maxlateness WAg__1__-,,_-l--.t--1------l--t>---l---
eanlmaxtardiness WA“"2"”_Z",___§--.t--3--..--3--.
can/ma.xqueuelength WAL__z__”__§__‘L__§__,,_-§--Aug-"
W3 2 2 2 2 5

 

 

 

 

 

 

 

 

 

 

 

 

185

Experts' Survey Appendix B

Usefulness Scale: 1=very useful; 5=not useful at all

 

 

 

 

 

 

 

Financial A B C D E

"Throughput‘ WA 1 l 5‘ l 3
we'"5"""5"""5' """ 5 """ 5'"

ventory“ WA 1 1 5 5 4
W8"? """ 2"""5' """ 5' """ 3'"

tionalexpense’r WA 1 1 3 3 T
W97"5"”"3"""5' """ 5"”"3'"

hop net proﬁt (loss) WA 1 j 3 1 I
monthlyorforperiod We"'f"""3"""5' """ 5"”"3'"

Meancostperjob WA 1 7 3 4 4
W8”'5"”“3"""5' """ 5' ““““ 3'“

eanrevenueperjob WA 1 l ’ l 1 l
WB'"3"”"3"""5' """ 5"”"3'"

 

 

 

 

 

 

 

 

 

* As deﬁned by Goldratt (The Haystack Syndrome, 1990)
Throughput: The rate at which the system (shop) generates money through sales.
Inventory: All the money that the system has inveswd in pmehasing things
. which it intends to sell.
Operational Expenses:
All the money the system spends in order to turn inventory into
throughput.

186 t

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Experts' Survey Appendix B
Usefulness Scale: 1=very useful; 5=not useful at all
Detailed Operational A B C D E
i8 If)“ “mm“ ___3 ______ 4 4 2 2
B --------------- db ------
5 5 5 5 5
Sequence of routing WA 5 4 5 4 3
”mm“ 5 5 5 5 5
Employees involved ReepondentA 4 4 3 2 2
"""°"°""° 5 5 5 5 5
—RawMaterialCost ReepondentA 2 2 3 2 2
----- qc-n-unin—uu—c-—--—-dr--—--q
"°"’°"""'° 5 5 5 5 5
errsuslaborwrt 1 2 2 2 l
and/ormaterialcost we --.5.-a--.5.--,.--2--_--2--..--.5_-.
strr utron o actrvrty WA 4 5 3 5 3
byshiftjobtypc We "'5"""3' """ 3""‘3 """ '5’"
Distribution of process WA 4 5 3 5 3
times,IATs We "'5' """ 3' """ 5 """ 3 """ 3 ""
Singlebottleneck WA 3 1 5 l 1
identiﬁcation We "'5' """ 1"""1'""'I"""7t'"
Double or sequential WA 3 I 5 1 l
bottleneckidcntiﬁcationnapome "'5' """ 1 """ 1' """ I"""'4"‘
Distributim of queue WA 5 5 5 5 5
lengthsbyjobtype/shiftnaupotme "'4 """" 1 """" 1' """ 1“"“3‘”
endBrdelivery WA 1 5 5 1 5
performance We "'5' """" 5' """ 5 "'"5"""Z"‘
crap/leworkreports WA 1 5 5 l 5
we 45L334
Wlnventoryreports WA 1 1 5 l l
W8 "'4 """ 3' "’"3"'"3"“"2"‘

 

 

 

 

 

 

 

 

 

 

187

 

 

 

 

 

 

Experts' Survey Appendix B
I l l l I. . 1 ﬂ 1
mm
Usefulness Scale: 1=very useful; 5=not useful at all
Additional Measures A B C D E
Respondent A .
.é§999§.ﬁ98 .......... .---1.--.---.1 ........ 2 ....... 1 ..... 1. -.
.I'ensnMarSe ............. 1..--.---§ ........ 2. ...... 1 ..... 1. -.
Benchmarking 2 1 2 1 1
Respondent B
leme in queue vs. standard
.glerlncéqusec ........ .---1._-.--_.l--_..-.._1 ...... 1 4
tandird hours past due'in """""
Rustic nastiest? .......... 1. - - .. - - -1 ........ 1 ....... 1 4
r ctua'lsetuptrmevs. -""".
“8.11.9134- .......... .---2..--a---2 ........ 1 ...... 1 4
rActua] main, tenance hrs 5y -------
workcenter vs. planned 2 2 2 2 4

 

 

 

 

 

 

 

 

188

 

APPENDIX C

Recruiting Message
Informed Consent Form

 

189

RECRUITMENT PROCEDUFE AND SCRIPT

SUBJECT RECRUITMENT PROCEDURE AND SCRIPT

Volunteers were recruited from various undergraduate and graduate classes in
MSU’s Eli Broad College of Business. Preference was given to junior, senior or graduate
MLM majors or those with relevant experience, although other subjects were considered.

The script read to the prospective subjects is (approximately) as follows:

“I am looking for volunwer participants in a research project involving the
management and evaluation of shop ﬂoor and manufacturing information. In this research,
you will be asked to review various information outputs as generated by a computer
simulation model of an operating shop ﬂoor and to answer several questions.

“The exercise will consist of a single session, divided up into two main parts. The
ﬁrst section is a 45-60 minute introductory and training session in which I will familiarize
you with the manufacturing environment being studied, describe the model being used and
the different types of information you may be asked to evaluate. The second session will
last between one and one and a half hours and will consist of you being presented with
various scenarios cf data and information and being asked to evaluate certain questions and
conditions. Depending on the time of day of the session, appropriate refreshments will be
provided.

“The level of difﬁculty of the task will vary depending on the individual and on
what particular information sets you are provided. Ultimately, the tasks are designed to be
relatively self explanatory and should not cause any undue burden. The tasks being
performed relate directly to manufacturing information management and may provide a
good opportunity to evaluate data and to investigate shOp ﬂoor problems, much as you may
be asked to do in your future careers. Thus, it has the potential of contributing to your
manufacturing and general business knowledge base.

“All participants will be eligible to win one of four prizes- a new mountain/all
terrain bike, $200, $100 or $50- those being ﬁrst, second, third and fourth prizes of a
drawing. The chances to win in the drawing increase with your performance in the
exercise.

“Of come, this is a completely voluntary project. Your participation may be
terminated by you at any time. Those of you who are interested are free to sign up on the
sheet I will leave with your instructor and I will contact you with flnther details. I expect to
schedule experimental sessions within the next couple of weeks. If you have a preference
as to a time of day, please indicate so on the form. If there is a day of week preference,
indicate that as well. I’ll do my best to accommodate you.”

190

 

SHOP FLOOR INFORMATION MANAGEMENT AND EVALUATION
EXPERIMENT

INFORMED CONSENT FORM

This research is exploring the usefulness of various types of information and
performance measures for the shop ﬂoor. Subjects involved in this research will be
asked to review different information sets and then to attempt to identify, ﬁ'om a
provided list, certain conditions that may be present on the shop ﬂoor and that are
leading to the observed paformance.

All information and measures are generawd from a computer simulation model of a
relatively standard shOp ﬂoor. Subjects will be reviewing the information only, not
the model, although the design of the model will be explained prior to the experiment.

It is estimated that the following time will be required for the experiment:

 

Training/information session 45-50 minutes
Experimental, evaluation session 1.5-2 hours

In signing this form, you are freely consenting to participate in this experiment.
However, participation is and will continue to be completely voluntary. You may
choose not to participate in certain parts ofthe experiment and may terminate your
participation at any time.

All results will be treated with the strictest conﬁdence and all the subjects will remain
anonymous in any report of research ﬁndings. You may request and receive
information on your participation in the experiment, subject to the aforementioned
conﬁdentiality and anonymity restrictions.

Compensation for this experiment will be as follows:

Points will be awarded according to performance on the task, with each participant
able to earn between 5 and 11 points. Approximately 60 subjects are expected to
participate. Each point will count as one ticket for three drawings. Prizes will be a
follows:

1st prize- New Trek Mountain Bike
2nd prize- $200

3rd prize- $100

4th prize- $50

The drawing will be done after all sessions are complete. Subjects need not be
present to win a prize, but are entitled to be. Please advise the researcher as to how
you can be notiﬁed as to the drawing’s time and location.

The researchers involved in this project are:

Robert J. Marsh Professor Steven A. Melnyk

Principal Researcher Research Commime Chair

Department of Marketing and Supply Department of Marketing and Supply
Chain Management Chain Management

N451 BCC N327 BCC

353-6381 353-6381

marshrol@pilot.msu.edu 16513sam@msu.edu

191

If there are any questions or concerns regarding this experiment they should be
direcwd to the principal researcher. If you wish, you may contact the research
committee chair, as shown above.

SUBJECT TO THE CONDITIONS DESCRIBED ABOVE, I HEREBY CONSENT TO
PARTICIPATE IN THIS EXPERIMENT. I AM AT LEAST 18 YEARS TO AGE.

Date:

 

 

 

(please print name)

192

 

APPENDIX D
Sample Laboratory Experiment Instruments

Disaggregate Measures (D-36) Instrument (with follow up
questionnaire)

Aggregate Measures (A-36) Instrument (with follow up
questionnaire)

Debrieﬁng Letter

 

193

INSTRUMENT D-36

SHOP FLOOR INFORMATION EVALUATION EXERCISE
CASE 36

This package contains information regarding a simulated shop ﬂoor and is an
exercise designed to assess your ability to identify certain operational conditions based on
the shop ﬂoor information presented. In reviewing this information, you may refer to the
notes provided in the training session. This package consists of two major parts:

I . Performance measurement information
This section consists of nine information sets, each presenting operational
performance of the shop floor. Various measmes and/or graphics may be
presented.

II. Assessment Questions

These questions allow you to assess the usefulness of the information
presented in I. when used to evaluate shop ﬂoor performance.

194

 

INSTRUMENT 0-36 .

PLEASE ENTER YOUR STARTING TIME HERE:

 

1. PERFORMANCE INFORMATION
INFORMATION SETS
The information provided in this section consists of nine separate graphs or

tables, as presented below. Each set is numbered and described.

Set#l Distribution of Time in ﬁstem— all Job Types

 

Time In System, All Jobs. Case 36. .

 

 

 

L—Froquencvl

 

 

42

cc
('3 G)

48
54
60
More

Days

 

 

 

 

Set #2a Distribution of lateness for all job types:

 

Lateness, All Jobs. Case 36. .

 

 

 

 

 

500
>400"
2300
g . [—Frequencﬂ
5200-»
"- .00 d. A
O- 4-- ‘
O Q to V N O 0 CO V N O N V d) O
or c or on r~ to v a) or v- v- N o 3 Days
'— '.— I I 1 I I o 2

Early l Late

 

 

 

195

INSTRUMENT D-66

Set #2b Distribution of Lateness for Project Jobs only:

 

Lateness, Projects. Case 36. .

 

 

 

  
 

200
E 150 y
3' 100 -* L—-Frequenal
3
II- 50 '

 

 

00 00V
NOOQNOVGN
v-v- 0'
1|

.Eaily l Late

 

 

 

 

Set#3 Distribution of Tardiness, all Job Types:

 

Tardiness, All Jobs. Case 36. .

 

 

 

Freq uenm

50 -~ L

 

 

 

 

 

 

 

196

INSTRUMENT D436

Set #4 Distribution of Due Date Allowance for Project Jobs:

 

Due Date Allowance, Projects. Case 36. .

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

500
400 ..
>0
0
5300 ‘-
3 [1Frequenm
5200* .
u.
0
oocvmooovwoomvmcm
PNMVVDCDNQmmezt'E'O-{Days
Set#5 Queue Lengths by Work Center— Time Series
Que a La the Case 36 wc‘
u no ’ . . w—m
l—wes
50 ‘_—W05
45 --
4o~- WC6

 

Number at Jobs

30 " '
J I .

 

 

 

 

 

Set#6&7 Machine and Labor Utilization

lWorkCenter ‘ MachineUtilization % IaborUtilization l
97% 90%

 

 

 

 

 

 

l

4 77% 74%
5 80% 86%
6 96% 96%

 

 

197

INSTRUMENT 0-36

Set#8 Distribution of Job Processing Activity

 

.Jobs Processed/day, All Jobs. Case 36. .

 

Frequency
at
O

(a)
O
r

 

 

1)-
l-
11-

 

 

 

21 .-
25 ..
29 -
33 a
37 ~L
41

 

Frequencﬂ

 

198

 

INSTRUMENT D-36 .

II.

ASSESSMENT QUESTIONS

Given the information in I. above, please answer the following questions to the best of
your ability:

1)

2)

3)

What conditions could be present on the shop ﬂoor to cause these results (the
information sets in 1.)? From the following list, rank order your seven choices
from 1 to 7, with 1 being the most likely condition, 7 being least likely.

_ Stationary bottleneck _ Moving bottleneck
_ Irregular job release _ Varying job type mix _
_ Dispatch nrle variation _ Aggregate due date assignment

__ No differences in conditions from the base case could be observed

How conﬁdent are you that your choice for the most likely existing condition
(ranked 1 above) is actually present on the shop floor? Indicate your degree of
conﬁdence on a scale of 0 - 100% below, with 0% being not conﬁdent at all and
100% being extremely conﬁdent:

%

 

Which information sets did you use in coming to your determination in questions 1
and 2 above? Rank order the information sets based on their importance to your
decision making, with 1 indicating the most important. Please rank only that
information actually used. The information set numbers correspond to those
on the previous page.

__ Set# 1 Distribution of Time in System

__ Set# 23 Distribution of Job Lateness

_ Set #2b Distribution of Project Job Lateness
j__ Set# 3 Distribution of Tardiness

_ Set# 4 Distribution of Due Date Allowance
_ Set# 5 Queue Lengths

_ Set# 6&7 Machine and Labor Utilization

__ Set# 8 Job Process Activity

_ Set# 9 Summary Statistics by Job Type

PLEASE ENTER YOUR ENDING TIME HERE:

 

199

INSTRUMENT 0-38

II. ASSESSMENT QUESTIONS (continued)

How did you arrive at the answers on the previous page? These answers will not
qﬁrect the number of tickets earned by your performance. If you need more space,
please continue on the back of this page.

A. Of the following information sets, which were most useful to you in
identifying the most likely shop ﬂoor condition and your conﬁdence in your

decision? Why?
Time in system Due Date Allowance Job Process Activity
Job Lateness Queue Lengths Job TypeSummary Statistics
Job Tardiness Machine, Labor Utilization

B . How did you determine your answer to question #1 on page 4 when
identifying the shop ﬂoor condition that you believe to exist? If you guessed,
please indicate so.

C. What other information would have made your identiﬁcation of the shop ﬂoor
condition easier? Please indicate what you feel the usefulness of such
information would have been.

Somewhat Useful Very Useful
1 2 3 4 5

 

 

 

200

INSTRUMENT D436

I . PARTICIPANT INFORMATION
(You must provide Name and Phone# and/or Email to be eligible for prizes)

 

 

 

 

 

 

 

1. Name: Phone
3 . Age: Sex: Email:

Yearinschool Fr__ So_ Jr___ Sr_ Grad_
4. MLM Course Grades Received (where applicable):
MGT 303__ MGT304__ MGT401__ ML446___
MGT402__ ML345___ ML442___ ML470__

Other

5 . Overall GPA: __
6. SAT Scores- Overall: __ Verbal: _ Math: _

ACI‘ Scores- Composite: _

(English _ Math _ Reading _ Science _)

[Please provide test scores to the best of your recollection]

7 . Manufacturing or J ob Shop Relawd Work Experience (please brieﬂy describe
and indicate number of years at each position- use back of sheet if necessary):

Position Description Years

 

 

 

 

 

 

Please use back to describe additional experience, if necessary.

201

INSTRUMENT D-36

FOLLOW UP QUESTIONS

1 . As a result of the training session, I knew how to use the information presented.

1 2 3 4 5 6 7
Strongly Strongly
Agree Disagree

2 . I could have made sense of the information presented without the training session.

1 2 3 4 5 6 7
Strongly Strongly
Agree Drsagree

3 . The information presented with the cases was insuﬂﬁcient for me to answer the

questions.
1 2 3 4 5 6 7
Strongly Strongly
Agree Disagree

4 . The information presented with the cases was excessive and was difﬁth to sift

through.
1 2 3 4 5 6 7
Strongly Strongly
Agree Disagree

5 . The information presented with the cases was concise in natme and adequate to allow
me to answer the questions.

1 2 3 4 5 6 7
Strongly Strongly
Agree Disagree

PLEASE CONTINUE ON FOLLOWING PAGE

202

INSTRUMENT 066

6 . The information presentedwith the cases clearly and completely described the overall
structure of the shop, its ﬂows and layout.

1 2 3 4 5 6 7
Strongly Strongly
Ages Disagee

7 . The information presented with the cases allowed me to clearly understand and
identify the changes in conditions affecting the shop.

1 2 3 4 5 6 7
Strongly Strongly
Ages Drsagee

8 . The presentation of data in graphical format made it difﬁcult to use.

1 2 3 4 5 6 7
Strongly 50’0"le
Ages Disagee

203

 

INSTRUMENT A66

SHOP FLOOR INFORMATION EVALUATION EXERCISE
CASE 36

This package contains information regarding a simulated shop floor and is an
exercise designed to assess your ability to identify certain operational conditions based on
the shop ﬂoor information presented. In rsviewin g this information, you may refer to the
notes provided in the training session. This package consists of two major parts:

I . Performance measurement information
This section consists of 6 information sets, each presenting operational
performance of the shop floor. Various measmes and/or g'aphics may be
presented.

11. Assessment Questions
These questions allow you to assess the usefulness of the information
presented in I. when used to evaluate shop floor performance.

 

INSTRUMENT A66

PLEASE ENTER YOUR STARTING TIME HERE:

I . PERFORMANCE INFORMATION

INFORMATION SETS

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Case 36
Set# Measure Mean Std Dev

1 Time in System 11.0 days 10.0
2 Job Lateness -15.4 days 22.8
3 Job Tardiness 9.7 days 9.7
4 Due Date Allowance 26.7 days 21.2
5 Queue Lengths

WC#l 11.5 jobs 5.3

WC#4 1.7 jobs 1.7

WC#S 17.0 jobs 7.8

WC#6 25.4 jobs 10.9
6 Machine Utilization 88%

 

 

 

 

 

205

INSTRUMENT A66

II.

ASSESSMENT QUESTIONS

Given the information in 1. above, please answer the following questions to the best of
your ability:

1)

2)

3)

What conditions could be present on the shop floor to cause these results (the
information sets in 1.)? From the following list, rank order your seven choices
from 1 to 7, with 1 being the most likely condition, 7 being least likely.

_ Stationary bottleneck _ Moving bottleneck
__ Irregular job release __ Varying job type nrix
_ Dispatch rule variation _ Aggregate due date assignment

__ No differences in conditions ﬁom the base case could be observed

How conﬁdent are you that your choice for the most likely existing condition
(ranked 1 above) is actually present on the shop floor? Indicate your deges of
conﬁdence on a scale of 0 - 100% below, with 0% being not conﬁdent at all and
100% being extremely conﬁdent:

%

 

Which information sets did you use in coming to yom' determination in questions 1
and 2 above? Rank order the information sets based on their importance to you
decision making, with 1 indicating the most important. Please rank only that
information actually used. The information set numbers correspond to those
on the previous page.

__ Sedl 1 Time in system
_ Set# 2 Job Lateness
_ Set# 3 Job Tardiness
__ Set# 4 Due Date Allowance
_ Set# 5 Queue Lengths
-_ Set# 6 Machine Utilization
_ Set# 7 Labor Utilization

PLEASE ENTER YOUR ENDING TIME HERE:

 

INSTRUMENT A66

11. ASSESSMENT QUESTIONS (continued)

How did you arrive at the answers on the previous page? These answers will not
aﬁ'ect the number of tickets earned by your performance. If you need more space,
please continue on the back of this page.

A. Of the following information sets, which were most useful to you in
identifying the most likely shop ﬂoor condition and your conﬁdence in your
decision? Why?

Time in system Due Date Allowance Machine Utilization
Job Lateness Queue Lengths Labor Utilization
Job Tardiness

B . How did you determine your answer to question #1 on page 3 when
identifying the shop ﬂoor condition that you believe to exist? If you guessed,
please indicate so.

C. What other information would have made your identiﬁcation of the shop ﬂoor
condition easier? Please indicate what you feel the usefulness of such
information would have been.

Somewhat Useful Very Useful
1 2 3 4 5

 

 

 

INSTRUMENTA66

I . PARTICIPANT INFORMATION
(You must provide Name and Phone# and/or Email to be eligible for prizes)

 

 

 

 

 

 

 

 

1. Name: Phone
3 . Age: Sex: Email:

Yearinschool Fr_ So_ Jr_ Sr_ Grad—
4 . MLM Course Grades Received (where applicable):
MGT303_ MGT304____ MGT401_ ~ML446_
MGT402____ ML345_ m442__ ML470__

Other

5 . Overall GPA: __
6. SAT Scores- Overall: __ Verbal: _ Math: __

ACT Scores- Composite:

(English _ Math _ Reading __ Science _)

[Please provide test scores to the best of your recollection]

7 . Manufacturing or Job Shop Relawd Work Experience (please brieﬂy describe
and indicate number of years at each position- use back of sheet if necessary):

Position Description Years

 

 

 

 

 

 

Please use back to describe additional experience, if necessary.

208

INSTRUMENT A66
FOLLOW UP QUESTIONS

1 . AS a result of the training session, I knew how to use the information presented.

1 2 3 4 5 6 7
Strongly Strongly
Ages Drsagee

2 . I could have made sense of the information presented without the training session.

1 2 3 4 5 6 7
Strongly Strongly
Ages Drsagee

3 . The information presenmd with the cases was insufﬁcient for me to answer the

questions.
1 2 3 4 5 6 7
Strongly I 39’0“le
Ages Drsagee

4 . The information presented with the cases was excessive and was difﬁcult to sift

through.
1 2 3 4 5 6 7
Strongly Strongly
Ages Drsagee

5 . The information presented with the cases was concise in nature and adequate to allow
me to answer the questions.

1 2 3 4 5 6 7
Strongly Strongly
Ages Disagee

PLEASE CONTINUE ON FOLLOWING PAGE

209

INSTRUMENT A66

6 . The information presented with the cases clearly and completely captured described
the overall structure of the shop, its ﬂows and layout.

1 2 3 4 5 6 7
Strongly Strongly
Ages Disagee

7 . The information presenwd with the cases allowed me to clearly understand and
identify the changes in conditions aﬂ'ecting the shop.

1 2 3 4 5 6 7
Strongly . Strongly
Ages Drsagee

8 . The presentation of data in tabular format made it difﬁth to use.

1 2 3 4 5 6 7
Strongly Strongly

210