AND T. VALIDATION OF A SECRETARIAL EVELOPMEN D I THE DESIGN FN BASKET T0 EVALUATE- . IN DECISION MAKING .N THE sm 6 ORITIES WORK PRI 0F A... ea. r. .. xr may, “run-Ht 53A? ’ irrrrf. . .6772? J . (i 2, r?) I... J F {5.1} . '37:; $5? , 3.. {11...} 11 . fl {in} 1.1.1.5.: {1 f; . . . . 44.42:)? >r J .1. inswigzwww ; . ...rr£rl).vvrt . in: l a: ._. r; 5f; . 1 5t rhfgs iris 7.3. bar: av I rt). 1. r. 3!. 5. 2;: J, , . IHIHllIlLlIlllHllliIHIHHIIUIIllfflllllHlllllllHllllll 1293 10391 5256 LIBRARY Michigan State University This is to certify that the thesis entitled THE DESIGN, DEVELOPMENT, AND VALIDATION OF A SECRETARIAL IN-BASKET TO EVALUATE DECISION MAKING IN THE SETTING OF WORK PRIORITIES presented by Betty Louise Schroeder has been accepted towards fulfillment of the requirements for _2h~_D.__degree in Jducation. (3 v . \1 / 1" . 6K” 1.x, 4/6 c ()9 Date August 11, 1972 0-7639 3111011116 or g HMS G SM' 4- ‘9', 800K BWDERY INC. LIBRARY Bl N 0! RS mum-t. awn; 1W .. ‘h A.» 4:'~- .- 'w eh: n": -' .; yfi'MtL-wu. ”a. _‘ W 'm we Tr av.“ mart a! r. ‘ 18 hump. U to ash. flu nu. ‘fi 4” 1.3. - ABSTRACT THE DESIGN, DEVELOPMENT, AND VALIDATION OF A SECRETARIAL IN-BASKET T0 EVALUATE DECISION MAKING IN THE SETTING OF WORK PRIORITIES By Betty Louise Schroeder The Problem The need exists for criterion instruments that represent situations in1flflch the secretary is required to use the integrated tasks involved hithe secretarial position to make decisions relevant to problems related to the position and to determine appropriate solutions to these pmflflems. In evaluating the secretarial student's achievement in making specific types of decisions in specific situations, an evaluative instrument is needed that involves the student in a job situation and raudles her to make appropriate decisions. This study was designed to accomplish these four objectives: 1. To design an evaluative instrument using the in—basket format based on representative problems requiring the decision-making ability of the secretary in setting work priorities. 2. To develop criterion answers for THE SECRETARIAL IN-BASKET based upon answers given by a panel of experts. 3. To determine whether the instrument does, in fact, distinguish between the performance of experienced secretaries and the performance of potential secretaries in terms of three types of decision making contained in the in-basket: Betty Louise Schroeder Quality of action responses to in-basket items Quantity (number)of in-basket items attempted The priority order of the in-basket items as indicated by the subjects To develop a suggested measurement schema based on the results of field testing THE SECRETARIAL IN-BASKET with . groups of experienced secretaries and groups of potential :1 secretaries. 1" Methods and Procedures for the Study One of the questions arising in this research study asked if in- basket simulation can be used to evaluate the ability of an individual to handle specific decision-making required in a particular position, that of secretary. The design and development of THE SECRETARIAL lN-BASKET into an instrument that could, in fact, evaluate an individ- iual's ability to handle these kinds of decisions was dependent upon the following operational phases: (1) Preliminary planning; (2) review '0: related research and literature pertaining to design and develop- ‘iemtal theories, historical development of in-basket simulation, iinalyses of the secretarial role, and research in curriculum develop- "§jint; (3) the critical incident study; (4) the development of THE ' .3; ~EflKm£TARIAL IN-BASKET; (5) field testing THE SECRETARIAL IN-BASKET; r-uina (6) analysis of the data. " .:l¢ Findings of the Study Three general areas of concentration are represented in the '-ggs that result from this in-basket study: 71. Findings that relate to the actual design and development of THE SECRETARIAL IN- BASKET. Findings that relate to the validation of the in-basket simulation. Findings that relate to the development of a measurement '.schema for the in-basket simulation. Betty Louise Schroeder Conclusions The three basic conclusions drawn from the analyses of the data in this experimental study emphasize the theory of decision making as it must be combined with the career ladder concept, the importance of quality of performance, and the need for levels of secretarial education. Implications for the Study Implications resulting from this in-basket research are evident tithe areas of curriculum development (both secondary and post— seanflary), research, teacher education (pre—service and in-service), andtraining within industry. Instructional programs now being dewfloped on the secondary and post—secondary levels will require alearning environment which teaches students how to make decisions andthen provides opportunities to practice making decisions. Teachers mmt be acquainted with the career ladder concept and its relationship tothe vocational curriculum and with appropriate instructional tech— quues and materials to enhance the instruction. There is also a need fin a coordinated effort between education and business to provide a cmmflete secretarial education for the secretary. 'HHEDESIGN, DEVELOPMENT, AND VALIDATION OF A SECRETARIAL IN-BASKET TO EVALUATE DECISION MAKING IN THE SETTING OF WORK PRIORITIES by Betty Louise Schroeder A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Secondary Education and Curriculum 1972 (§)Copyright by BETTY LOUISE SCHROEDER 1972 DEDICATION To my father and the memory of my mother who have always worked hard to give their kids the opportunity to pursue their own goals -— both in education and in chosen occupations. ii ACKNOWLEDGEMENTS A study like this one needs people--those who assist with the A developmental process, those who participate in the experimental phase, those who critique the progress of the study, and those who become # associated with the success of the venture. {. A very special tribute is hereby given to ... .. the members of my doctoral committee-~Dr. Elaine Uthe, Dr. Maryellen McSweeney, Dr. Mary Virginia Moore, and Dr. Robert Poland--for their interest and assistance in > the study. Dr. Maryellen McSweeney for her willingness and readiness } at all times to share her expertise in research and in quantitative analysis. .. Dr. Elaine Uthe, my chairman, for her enthusiasm for the study and her friendly encouragement. Dr. Peter Haines who gave me the opportunity to pursue two goals, the doctoral program and my own professional development in business education. James Maas for his consultation and assistance during the statistical analysis of the data. the National Secretaries Association (International) and the Institute for Certifying Secretaries for their permission to conduct the study. the many Certified Professional Secretaries and other members of the National Secretaries Association (Inter— national), and especially those members of the Lansing Teubor Chapter, who gave much time and energy to helping with the study. the Michigan Bell Telephone Company who, through their Personnel Research Division, provided an opportunity to administer the in-basket to company secretaries. . James Huck, Betty Dain, Carol Ray, and Ruth Gustafson of Michigan Bell Telephone Company for their assistance in the experimental phase of the study. . the divisional secretaries of Michigan Bell Telephone Company who participated in the study. . the teachers who administered the in-baskets to their students: Marilyn Bowker, John Forte, Barbara Lanthorn, L. Ann Hansen, Loretta DeMino, June Brasier, Susan Maxam, Sandra Trombley, Virginia DeWitt, Miriam Elwell, Kay Sheedy, Jerry Ann Beaufore, Mary Frost, Donalda Warner, and Mary Wadsten. . the students (office block and secretarial classes) who participated in the study. the National Science Foundation who support, in part, the research facilities at Michigan State University, thus providing doctoral candidates with computer assistance. my three special friends—~Rosemary Gunn, Lorraine Furtado, and Lynn Tillock--who have always provided me with the understanding and inspiration needed throughout the study. iv .... o... '1- In. o |-.l . . “ml TABLE OF CONTENTS Page grammar-:3...”.................... ix ‘mxfloariwsrmrrons..................... xiv 951‘: H THE PROBLEM OF THE STUDY . . . . . . . . . . . . . . . . A. Introduction . . . . . . . . . . . . . . . . . . . . y B. The Problem . . . . . . . . . . . . . . . . . . C. Methods and Procedures for the Study . . . . . . . . " Assumptions . . . . . . . . . . . . . . . . . . . Limitations . . . . . . . . . . . . . . . . . Definition of Terms . . . . . . . . . . . . . . . D. Rationale of the Problem . . . . . . . . . . . . . . ~. Evaluation of Student Progress . . . . . . . . . ‘.* . The Application of Simulation in the ' “’ Evaluative Process . . . . . . In-Basket Simulation as an Evaluative Tool . NONOO‘huI-I I-IH NH 00 Search THEORY FOR IN- BASKET SIMULATION . . . . . . . . . 24 : A. Introduction to the Review . . . . . . . . . . 24 ;3, Design Theory for In- Basket Simulation . . . . . . . 25 Basic Approaches to Test Development . . . . . . 26 Developmental Theories for In- Basket Simulation . 31 Measurement of Individual Performance in In-Basket Simulation . . . . . . . . . 45 Factors in Measurement of Individual Differences 46 I I o n a a e I e I n o I e e I a e a a a a 53 Chapter Page III. PHSTORICAL DEVELOPMENT OF IN-BASKET SIMULATION . . . . . 56 A. The In— Basket in Research . . . . . . . . . . . . . 56 The Air Force Study . . . . . . . . . . S7 The Bureau of Business In- Basket Study . . . . . . 59 The School Administration In- Basket Study . . . . 60 The Organizational Climates Study . . . . . . . . 63 The Shulman Study . . . . . . . . . 66 The Vocational Office Block Project . . . . . . . 67 Rimoldi' 5 Test of Diagnostic Skills . . . . . . . 69 The Diagnostic Management Problem . . . . 71 McGuire' 5 Process Approach to Medical Evaluation . 72 The Medical Inquiry Project . . . . . . . . . 75 Other Medical Evaluation In— Baskets . . . . . . . 77 B. The In— Basket in Management Training and Selection . . . . . . . . . . . . . . . . . . . 78 The Bell System . . . . . . . . . . . . . . 78 Sears, Roebuck and Company . . . . . . . . . . . . 79 General Electric . . . . . . . . . . . . . . . . 81 The Boeing Company . . . . . . . . . . . . 85 Related In- Basket Development . . . . . . . 86 C. The In— Basket in Secretarial Training and Development . . . . . . . . . . . . . . . 87 The Popham In— Baskets . . . . . . . . . . . . . 88 The Port of New York Authority . . . . . . . 89 The Montgomery Ward Secretarial Seminar . . . . . 89 The National Newark and Essex Bank . . . . . . . . 90 The Secretarial Practicum . . . . . . . . . . . . 90 The CPS In—Basket . . . . . . . . . . . . . . . . 91 D. Summary . . . . . . . . . . . . . . . . . . . . . . 92 IV. RESEARCH IN THE SECRETARIAL ROLE AND CURRICULUM DEVELOPMENT WITH IMPLICATIONS FOR SECRETARIAL EDUCATION . . . . . . . . . . . . . . . . 94 A. The Task- Trait Analyses . . . . . . . . . . . . . . 95 The Charters- -Whit1ey Study . . . . . . . . 96 The Nichols Study . . . . . . . . . . . . . . 97 The Place Study . . . . . . . . . . . . 98 The Young Study 99 The Noyes Study 100 The Paddock Study 101 The Wagoner Study . 102 The Perkins-Byrd Study . i8: Other Task-Trait Analyses vi < The Situational Analyses . . . . . . . . . . . The Casebier Study . . . . . . . . . . . . . The Kosy Study . . . . . . . . . . . . . . The Cook Study . . . . . . . . . . . . . The Lanham Study. . . . . . . Research in Curriculum Development . . . The Senior Intensified Program . . . . . . . The Vocational Office Block Program The NOBELS Study. . . . . . .'. . . . Summary . . . . . . . . . . . . . O P THE DESIGN AND DEVELOPMENT OF THE SECRETARIAL IN-BASKET . . . A. Preliminary Planning . . . . B. Review of Related Research and Literature C. The Critical Incident Study Preparing the Letter- Decision Card Format Selecting the Samples of Secretaries . Cataloging and Classifying Critical Incidents Conducting a Preliminary Review of Selected Incidents . . . . . . . . . Selecting the Problem Categories for THE SECRETARIAL IN- BASKET . . . D. The Development of THE SECRETARIAL IN— BASKET . Planning, Writing, and Sequencing the In- Basket Reviewing the In-Basket Items as a Reality Measure . . Developing the Procedures for Administration of THE IN- BASKET . . Administering THE SECRETARIAL IN- BASKET to a Panel of Experts Developing the Scoring Procedure to be Used in Evaluation . . . . . . . FIELD TESTING THE SECRETARIAL IN- BASKET AND ANALYSIS OF DATA. . . A. Tryout 1 . . . . . . . . . . . . B. Analysis of Tryout I Data . Analysis of Total Scores on THE SECRETARIAL IN- BASKET . . . . . . Analysis of Total Scores on the In-Basket Envelope . . . . . . Analysis of Total Scores on Envelope A (Incoming Mail) . . . . . . . . . . . . . vii Page 108 108 109 112 115 117 117 119 122 123 125 126 128 129 129 131 132 134 135 145 145 147 155 156 158 160 161 163 164 177 182 Chapter Analysis of Total Scores on Envelope C (Additional In- Basket Items) . The Importance of the In Basket Items Item Analysis Based on Mean Scores on Quality Responses . C. Revision . D. Tryout 2 . . E. Analysis of Tryout 2 Data . . . . . . . . . Analysis of Total Scores on THE SECRETARIAL IN-BASKET . .-. . . . . . . . Analysis of Total Scores on the In-Basket Envelope . . . . . . . . . . Analysis of Total Scores on EnveIOpe A (Incoming Mail) . . . . . . . Analysis of Total Scores on EnveIOpe C (Additional In- Basket Items) The Importance of the In— Basket Items Item Analysis Based on Mean Scores on Quality Responses . A Measurement Schema for THE SECRETARIAL IN- BASKET . . . . . . . . . . VLL SUMMARY, CONCLUSIONS, AND IMPLICATIONS . A. Summary . The Problem - Methods and Procedures for the Study . Theoretical Principles for the Development of In- Basket Simulation The Design and DeveIOpment of THE SECRETARIAL IN- BASKET . . . . . . . . . . . Findings of the Study B. Conclusions . C. Implications for the Study . Curriculum Development . Research . . . Teacher Education . Training within Industry . BIBLIOGRAPHY . APPENDICES . Training, and Evaluation . B. Tables 1- 4: Classification of Secretarial Problems . viii A. Developers of In- Basket Simulation for Selection, Page 185 191 196 199 202 204 205 213 218 221 225 248 252 264 264 264 265 267 268 272 277 282 282 284 284 285 287 299 299 300 LIST OF TABLES 2. Classification of Problem Factors by Problem Sub-categories . . . . . . . . . . . . . . . . . . ‘3. In-Basket Items Developed for THE SECRETARIAL IN-BASKET 4. Tryout l: Secretaries and Students Participating in ~. a . the Tryout . . . . . . . . . . . . . . . . . . . . 5. Tryout 1: Multivariate Analysis of Variance of Total =-. Response Scores, Total Problems Attempted, and Mean -.j 56. Tryout 1: Group Means for Total Response Score . . . ":<;Y 7. Tryout l: Scheffg'Post Hoc Comparisons for Total 2' ‘Response Score . . . . . . . . . at! ’8. Tryout l: Scheffé Post Hoc Comparisons for Total “‘25. .Response Score . . . . . . . . Tryout 1: Group Means for Total Problems Attempted ~ iv. Tryout 1: Scheffe Post Hoc Comparisons for Total ' Problems Attempted. . . . . . . . . . . . ' Tryout.l: Group Means for Mean Response Score . . . . . . l: Scheffé'Post Hoc Comparisons for Mean ”spouse score I C O O I I I I D O I O I O I O I O I Tryout l: Multivariate Analysis of Variance of Total gghumber of Problems Attempted and Percentages of ('31nversions in the In-Basket Envelope . . . . . . . . . . . Group Means for Total Number of Problems »gqpted in the In-Basket Envelope . . . . . . . . . . ix Classification of Secretarial Problems . . . . . . . . . . Response Scores . . . . . . . Page 136 142 148 161 165 166 168 172 173 174 175 176 178 179 180 11511 '. :4; Table 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. Tryout 1: Group Means for Percentages of Inversions in the In- Basket Envelope Tryout l: Multivariate Analysis of Variance of Total Number of Problems Attempted and Percentages of Inversions in Envelope A . . . . . . . Tryout 1: Group Means for Total Number of Problems Attempted and Percentages of Inversions in Envelope A Tryout l: Multivariate Analysis of Variance of Total Number of Problems Attempted and Percentages of Inversions in Enve10pe C . . Tryout 1: Group Means for Total Number of Problems Attempted in Enve10pe C . . . . . . . . Tryout 1: Scheffg'Post Hoc Comparisons for Total Problems Attempted in Envelope C . . . . Tryout 1: Group Means for Percentages of Inversions in Enve10pe C Tryout 1: Scheffé’Post Hoc Comparisons for Percentages of Inversions in Envelope C Tryout 1: The Importance of the In-Basket Items in the Opinion of the Secretaries . Tryout 1: The Importance of the In-Basket Items in the Opinion of the Office Block Students . Tryout 1: The Importance of the In-Basket Items in the Opinion of the Students in Single-Period Classes . Tryout 1: Mean Scores on Action Responses to In-Basket Problems . Tryout 1: Items in THE SECRETARIAL IN— BASKET Requiring Revision Prior to Tryout 2 . Tryout 2: Secretaries and Students Participating in the Tryout . . . . . Tryout 2: Multivariate Analysis of Variance of Total Response Scores, Total Problems Attempted, and Mean Response Scores . . . . . . . Tryout 2: Group Means for Total Response Score Page 181 183 184 186 187 188 189 190 192 193 194 197 200 203 207 208 l‘l '51 in. Table 32. 33. 34. 35. 36. 37. 38. 39. 40. 41. 42. 43. 44. 45. 46. Tryout 2: Scheffg Post Hoc Comparisons for Total Response Score . . . . . . . . . . . Tryout 2: Group Means for Total Problems Attempted Tryout 2: Group Means for Mean Response Score . Tryout 2: Scheffé Post Hoc Comparisons for Mean Response Score . . . . . . . . . . . Tryout 2: Multivariate Analysis of Variance of Total Number of Problems Attempted and Percentages of Inversions in the In-Basket Envelope . Tryout 2: Group Means for Total Number of Problems Attempted in the In-Basket Envelope Tryout 2: Group Means for Percentages of Inversions in the In-Basket Envelope Tryout 2: Scheffé'Post Hoc Comparisons for Percentages of Inversions in the In-Basket Envelope Tryout 2: Multivariate Analysis of Variance of Total Number of Problems Attempted and Percentages of Inversions in Enve10pe A . . . . . . . Tryout 2: Group Means for Total Number of Problems Attempted and Percentages of Inversions in Envelope A . Tryout 2: Multivariate Analysis of Variance of Total Number of Problems Attempted and Percentages of Inversions in Envelope C . . . . . . Tryout 2: Group Means for Total Number of Problems Attempted and Percentages of Inversions in Envelope C . Tryout 2: The Importance of the In-Basket Items in the Opinion of the Certified Professional Secretaries (CPS) Tryout 2: The Importance of the In-Basket Items in the Opinion of the NSA Secretaries . Tryout 2: The Importance of the In-Basket Items in the Opinion of the Michigan Bell Telephone Company Secretaries (MBT) . . . . . . xi Page 209 210 211 212 215 216 217 218 219 220 222 223 227 228 229 ...—— A --..__.- Table 47. 48. 49. SO. 51. 52. 53. 54. 55. 56. S7. 58. $9. Tryout 2: The Importance of the In-Basket Items in the Opinion of Students in the Office Block Program . . . . . . . . . . . . . Tryout 2: The Importance of the In—Basket Items in the Opinion of Students in Single-Period Classes A.One-Sample T Test of the Differences in Opinions on the Importance of the In-Basket Items as Shown by Analyzing Responses of CPS's and the Panel of Experts A One-Sample T Test of the Differences in Opinions on the Importance of the In-Basket Items as Shown by Analyzing Responses of NSA Secretaries and the Panel of Experts . A One-Sample T Test of the Differences in Opinions on the Importance of the In-Basket Items as Shown by Analyzing Responses of Michigan Bell Secretaries and the Panel of Experts . . . . . . . . . A One-Sample T Test of the Differences in Opinions on the Importance of the In-Basket Items as Shown by Analyzing Responses of Students in Block Programs and the Panel of Experts . A One-Sample T Test of the Differences in Opinions on the Importance of the In-Basket Items as Shown by Analyzing Responses of Students in Single-Period Classes and the Panel of ExPerts . The Magnitude of the Differences Among the Groups on the Importance of the In-Basket Items . Summary Table for a Two-Way ANOVA of the Magnitude of the Differences Among the Groups on the Importance of the In—Basket Items When Compared with the Experts . . . . . . . . . . . . . Tryout 2: Mean Scores on Quality Responses to In—Basket Problems Percentile Rank of Total Quality Responses by Secretaries and Students . . . . . . Percentile Rank of Total Number of Problems Attempted by Secretaries and Students . . . . . . Frequencies of Secretaries and Students on Percentages of Inversions within the In—Basket Envelope xii Page 230 231 236 237 239 241 243 245 246 250 254 256 259 Table 60. 6L Frequencies of Secretaries and Students on Percentages of Inversions within Envelope A Frequencies of Secretaries and Students on Percentages of Inversions within Envelope C xiii Page 260 261 LIST OF ILLUSTRATIONS Illustration 1. Basic Approaches to Test Development . 2. The Decision Card for Recording a Critical Incident . . 3. Analysis of Decision Card for Problem Identification and Determination of Response Patterns . xiv Page 32 130 133 CHAPTER I THE PROBLEM OF THE STUDY A. Introduction The secretary in the business office is a file clerk .. a general office worker .. a receptionist .. a stenographer .. an administrative assistant .. all in one "package.” At one time her job might have involved, for the most part, the taking and transcribing of dictation; and for this her achievement of typewriting and shorthand skills and her general understanding of office procedure may have been enough to prepare her for the tasks she handled in her position. However, today's business world in which automation is placing much more stress on the secretary requires her to be responsible for more creative work, to understand detailed instructions, and to handle more problem solving and decision making.1 The Dictionary of Occupational Titles states in part that the secretary is a "Girl Friday ... who relieves officials of clerical work and minor administrative and business detail.”2 In our fast-growing economy the office has become known as the center of business activity and is known by many names and titles including 1U. S. Department of Labor, Adjustments to the Introduction of Office Automation, Bulletin 1276 (Washington, D. C.: Government inting 0 ice . 2 U. S. Department of Labor, Dictiona of Occu ational Titles, volume II (Washington, D. C.: Government Printing Office}, p. 263. ; kt, ~o.‘ 21.. ~ ‘ I in l r '\ \n CHAPTER I THE PROBLEM OF THE STUDY A. Introduction The secretary in the business office is a file clerk .. a general office worker .. a receptionist .. a stenographer .. an administrative assistant .. all in one "package." At one time her job might have involved, for the most part, the taking and transcribing of dictation; and for this her achievement of typewriting and shorthand skills and her general understanding of office procedure may have been enough to prepare her for the tasks she handled in her position. However, today's business world in which automation is placing much more stress on the secretary requires her to be reSponsible for more creative work, to understand detailed instructions, and to handle more problem solving and decision making.1 The Dictionary of Occupational Titles states in part that the secretary is a "Girl Friday who relieves officials of clerical work and minor administrative and business detail."2 In our fast—growing economy the office has become known as the center of business activity and is known by many names and titles including M 1U. S. Department of Labor, Adjustments to the Introduction of Qfifice Automation, Bulletin 1276 (Washington, D. C.: Government Printing Office). 2U. S. Department of Labor, Dictionary of Occupational Titles, Volume II (Washington, D. C.: Government Printing Office), p. 263, out a n“ n. .... . owl‘ '~\ ‘un \n“ N "communication center," "information center," "decision center," and ”records center."3 The secretary who holds a position in the modern office must be flexible enough to perform within this type of office environment . Lehnert views her role as a secretary as one involving ever- changing responsibilities and adaptation to new dimensions of her employer's role: the secretary of today who 9_n_l_y keeps records, files, takes notes and types won't be around--or at least not employed as a secretary-—in the year 2000. She further implies that what will be essential is a secretary who can relate to the goals of management and “who thinks, sees relationships, and has enough practical knowledge to follow his thinking."5 While technological and scientific innovations have caused changes in the office, the basic instructional program intended to prepare individuals. for office occupations has remained virtually unchanged.6 In secretarial education much of the emphasis in the preparation of secretaries has been with the acquisition of typewriting, shorthand, and machines skills and knowledges, sometimes to the exclusion of —-___ 3Norman E. Kallaus, "Hardware: A New Man—Machine Interface," flaxonomy of Office Activities for Business and Office Education, Interim Report, Project No. 7-1223, Grant No. OEG-l—7-O71223-5134, Research 12 (Columbus, Ohio: The Center for Vocational and Technical Education, July, 1968), p. 85. 4Marie Lehnert, CPS, "P. S. for Private Secretaries," Volume 14, Number 7 (Waterford, Connecticut: Bureau of Business Practice, Inc., April 15, 1971), p. 1. 5Ibid. , p. 2. 6Harry Huffman et al, A Taxonony of Office Activities for Business flipffice Education, Interim Report, Project No. 7-1223, Grant No. fl“ OEG-l-7—071223-5134, Research 12 (Columbus, Ohio: The Center for Voca- tional and Technical Education, July, 1968), P- 3' II I «.1 A 1 been tn. u“ “5‘ .- N _\ .." \‘ '1. [.4 f“! ‘N abilities in decision making that are equally important to the secre- tary's success in handling the demands of her position. The evaluative instruments used in secretarial education have reflected the skill- knowledge concentrations in the preparation of the secretary, with a need resulting for Criterion instruments which provide the opportunity for terminal evaluation of the prospective secretary's ability to cope with Specific occupational situations. For the secretarial student with a career goal in mind, the assessment of her capability to cope with all aspects of the secretarial position for which she is being trained (skill—knowledge acquisition as well as the interactive and decision-making aspects) is vital for reasons of employability. Therefore, the means whereby this assessment takes place must be as much as possible like the demands of the job as secretarial education can make possible. B. The Problem The need exists for criterion instruments that represent situa- tions in which the secretary is required to use the integrated tasks involved in the secretarial position to make decisions relevant to problems related to the position and to find appropriate solutions to these problems. The prospective secretary's ability to make appropriate decisions in setting work priorities is one area in which numerous learnings are integrated, learnings which involve skills previously acquired, busi- ness information knowledges, organizational information, affective qualities of the individual, to name only a few. In evaluating the secretarial student's achievement in making Specific types of decisions hispecific situations, an evaluative instrument is needed that involves the student in a job situation and requires her to make appropriate decisions. This research study is designed to accomplish the four fundamental objectives presented here: 1. To design an evaluative instrument using the in-basket fermat based on representative problems requiring the decision-making ability of the secretary in setting work priorities as determined by a critical incident analysis of problems submitted by randomly selected samples of experienced secretaries (Certified Professional Secretaries and secretaries who are members of the National Secretaries Association (International). 2. To develop criterion answers for THE SECRETARIAL IN-BASKET based upon answers given by a panel of experts, such answers validated during the first field testing tryout. 3. 'To determine whether the instrument does, in fact, dis- tinguish between the performance of experienced secretaries (CPS, NSA, and Michigan Bell Telephone Company secretaries) and the performance of potential secretaries (students enrolled in office block programs or secretarial practice/ advanced shorthand classes in secondary schools in Michigan) in terms of three types of decision making contained in the in-basket: a. Quality of action responses to in-basket items b. Quantity (number) of in-basket items attempted c. The priority order of the in-basket items as indicated by the subjects 4. To develop a suggested measurement schema based on the results of field testing THE SECRETARIAL IN-BASKET with groups of experienced secretaries (CPS, NSA, MBT secre- taries) and groups of potential secretaries (students in office block programs or secretarial practice/advanced shorthand classes). C. Methods and Procedures for the Study This study will encompass six operational phases in the design, development, and validation of an in-basket simulation that will ..- measure the ability of the secretary to handle specific decision making required in the setting of work priorities required in her position. These operational phases and a description of the procedures to be utilized within each phase follow: 1. Preliminar Plannin : Experts in the field of in-basket research will be contacted for information on the develop- ment of in-basket simulation. Preliminary investigation of the possibility of selecting samples of secretaries from the membership of the National Secretaries Association (International) and from the Institute for Certifying Secretaries will be made. Preliminary planning for the inclusion of high school students in the office block programs and in secretarial practice classes will be conducted and investigations made into the possibility of utilizing several high school programs or classes. Review of Related Research and Literature: The review will focus on these four specific areas of study to determine relationships to the present study: a. Design and developmental theories b. The historical development of in-basket simulation c. Analyses of the secretarial role d. Research in curriculum development for secretarial education The Critical Incident Study: A study will be made of the kinds of problems and decisions made by experienced secre— taries in their jobs. Flanagan's critical incident tech- nique will be modified for use in this study. Samples of secretaries will be selected from the membership rolls of the NSA and the Institute for Certifying Secretaries. A classification will be made of those problems and decisions to determine the most frequently occurring problems. These will be included in the in-basket simulation to be developed. The Development of the In-Basket Simulation: Based upon Eh; results of the critical incident study, an in-basket simulation will be developed with at least 20 of the most frequently occurring problem factors from the critical incident study. The in-basket simulation will be reviewed by experienced secretaries and business educators to check its content and face validity. A panel of experts (experi- enced secretaries and business educators) will be used to determine the response patterns for the scoring procedures for the in—basket. The in-basket will first be administered in a preliminary tryout with secretaries. =- "" '- 'r x—V _.____-. _, ”m r — --— _- ' ' ‘ -. _ "......— ‘ 3" -. -'-e.‘L:_-er_-_gv-.-_gu_v “47.-... s are m TEILIIB’J 1 7:25:- over .,. ._ " 51'5”? ' «"3153 ‘I H I ..‘_’ .-h I) ..a . nwamnmathe ability of the secretary to handle specific decision making renamed in the setting of work priorities required in her position. These operational phases and a description of the procedures to be Injlized within each phase fellow: 1. Preliminary Planning: EXperts in the field of in-basket research will be contacted for information on the deveIOp- ment of in—basket simulation. Preliminary investigation of the possibility of selecting samples of secretaries from the membership of the National Secretaries Association (International) and from the Institute for Certifying Secretaries will be made. Preliminary planning for the inclusion of high school students in the office block programs and in secretarial practice classes will be conducted and investigations made into the possibility of utilizing several high school programs or classes. Review of Related Research and Literature: The review will focus on these four specific areas of study to determine relationships to the present study: a. Design and developmental theories b The historical deve10pment of in-basket simulation c. Analyses of the secretarial role d Research in curriculum development for secretarial education The Critical Incident Study; A study will be made of the kinds of problems and decisions made by experienced secre— taries in their jobs. Flanagan's critical incident tech- nique will be modified for use in this study. Samples of secretaries will be selected from the membership rolls of the NSA and the Institute for Certifying Secretaries. A classification will be made of those problems and decisions to determine the most frequently occurring problems. These will be included in the in—basket simulation to be developed. The Development of the In-Basket Simulation: Based upon the results of the critical incident study, an in—basket simulation will be deve10ped with at least 20 of the most frequently occurring problem factors from the critical incident study. The in—basket simulation will be reviewed by experienced secretaries and business educators to check its content and face validity. A panel of experts (experi- enced secretaries and business educators) will be used to determine the response patterns for the scoring procedures for the in-basket. The in-basket will first be administered in a preliminary tryout with secretaries. sheet. Within the time limit set for the in—basket it would be impossible to have the participant actually pro- duce typewritten problems in addition to making appropriate work—priority decisions. 3. The in-basket itself will be based upon a time limitation of 45 minutes. Therefore, the number of in-basket items will be limited to 40 items, two in-basket items for each i of 20 problem categories. 4. The samples of secretaries contacted during the critical incident study will be drawn from the membership rolls of the Michigan Division of the National Secretaries Association (International) and the Institute for Certifying Secretaries. The samples of secretaries selected for field testing will include: (a) Certified Professional Secretaries residing in the State of Michigan, (b) secretaries who are members of the Michigan Division of the National Secretaries Associ- ation (International) and residing in the State of Michigan, and (c) secretaries to divisional managers of the Michigan 1 Bell Telephone Company, Detroit, Michigan. L 5. High school students participating in the field testing will be from selected high schools in the State of Michigan, selected on the basis of whether there are single-period secretarial practice and/or advanced shorthand classes or vocational office block programs. 6. The number of tryouts for the in-basket will be limited to two, the first tryout followed by a revision stage. Definition of Terms The following definitions will be used throughout the study to I explain the given terms: 1 Certified Professional Secretary: an experienced secretary who 1 has successfully completed the standard 12-hour examination adminis- tered by the Institute for Certifying Secretaries covering a wide variety of topics including human relations and personal adjustment, business law, secretarial accounting, economics, business organization, and management.7 7Ruth I. Anderson et al, The Administrative Secretary; Resource (New York: McGraw-Hill Book Company, 1970), p. 19. 0 ;.~ :._‘ o s_ “fr sheet. Within the time limit set for the in-basket it would be impossible to have the participant actually pro- duce typewritten problems in addition to making appropriate work-priority decisions. 3. The in-basket itself will be based upon a time limitation of 45 minutes. Therefore, the number of in—basket items will be limited to 40 items, two in-basket items for each of 20 problem categories. 4. The samples of secretaries contacted during the critical incident study will be drawn from the membership rolls of the Michigan Division of the National Secretaries Association (International) and the Institute for Certifying Secretaries. The samples of secretaries selected for field testing will include: (a) Certified Professional Secretaries residing in the State of Michigan, (b) secretaries who are members of the Michigan Division of the National Secretaries Associ- ation (International) and residing in the State of Michigan, and (c) secretaries to divisional managers of the Michigan Bell Telephone Company, Detroit, Michigan. 5. High school students participating in the field testing will be from selected high schools in the State of Michigan, selected on the basis of whether there are single-period secretarial practice and/or advanced shorthand classes or vocational office block programs. 6. The number of tryouts for the in-basket will be limited to two, the first tryout followed by a revision stage. Definition of Terms The following definitions will be used throughout the study to explain the given terms: Certified Professional Secretary: an experienced secretary who has successfully completed the standard lZ-hour examination adminis- tered by the Institute for Certifying Secretaries covering a wide variety of topics including human relations and personal adjustment, business law, secretarial accounting, economics, business organization, and management.7 7Ruth I. Anderson et al, The Administrative Secretary: Resource (New York: McGraw-Hill Book Company, 1970), p. 19. u 0-. I hl' 4 I 9 tom. ... uh. ‘ . 12‘ "-8" I '{'l V7 ..— vv—v i—vl— Critical Incidents: those incidents from actual cases of on-the- job behavior that determine either effective or ineffective on-the-job behavior; the difference between success and failure in carrying out the important components of the job.8 The critical incident procedure developed by Flanagan consists of a set of procedures for collecting direct observations of human behavior in such a way as to facilitate their potential usefulness in solving practical problems.9 Decision: a choice between two or more plausible alternatives. Decision Makin individual): a judgment to promote the welfare of one person, considered by himself. Decision Making (institutional): a judgment leading to the greatest possible attainment of institutional goals. Decision Making in Setting Work Priorities: judgments made by individuals in establishing the arrangement of individual tasks in the secretarial position leading to the attainment of institutional goals. In-Basket Simulation: an experience involving the participant in a hypothetical work situation in which he must make decisions on a series of business papers deposited as incoming mail in his "in—basket" as well as a specified number of interruptions in the work day. Michigan Bell Telephggg Secretaries: secretaries to divisional managers of Michigan Bell Telephone Company, Detroit, Michigan, who are a part of a secretarial training program for divisional secretaries. 8Edward J. Furst, Constructin Evaluation Instruments (New York: Longmans, Green and Company, 1958 , p. 22. 9John C. Flanagan, "The Critical Incident Technique," Psycho- logical Bulletin, Volume 51, Number 4 (July, 1954), p. 327. .m- “I. «.1- . .II.‘ In “I. . "‘4‘. . \"u "A. N.“ NSA Secretaries: secretaries who are members of the Michigan Division of the National Secretaries Association (International). Secretary: an occupation concerned with carrying out administra- tive and general office duties, taking and transcribing dictation, maintaining files, preparing reports, opening and routing mail, making appointments, originating memorandums and letters for her superior's signature, and related duties and responsibilities; an occupation which may be entry-level but may also lead to positions as social secretary, legal secretary, medical secretary, and executive secretary.10 Students in Office Block Program: high school students who are seniors and enrolled in a two-hour office block program. Students in Sinnge-Period Classes: high school students who are seniors and enrolled in a onevhour secretarial practice or advanced shorthand class. Terminal Evaluation: measurement of an individual's ability to integrate learnings with application in a simulated but realistic on- the-job situation in order to make apprOpriate decisions relating to the position. Mk Priorities: the ranking of those tasks or responsibilities involved in the secretarial position in the order of their importance and/or in the order in which those tasks must be performed or respon- sibilities assumed by the secretarY- U. 8. Office of Education, Vocational Education and Occupations {gaShingtom D. C.: U. S. Department of Health, Education and Welfare, 69), pp. 78-79. vs. .... .:,, ‘4". ...», Lu: i \ 10 D. Rationale of the Problem From, in one of his recent philosophical essays, emphasizes that most intelligence tests measure not so much the capacity for reason and understanding as the capacity for quick mental adaptation to a given situation. His concern is that this type of assessment, requiring the application of categories of comparison and quantitative measurement, has become a common evaluative procedure rather than thOI‘OUgh analysis of each given phenomenon and its respective quality: We find today a tremendous enthusiasm for knowledge and education but at the same time a skeptical or contemptuous attitude toward the allegedly impractical and useless thinking which is concerned 'only' with the truth and which has no exchange value on the market.11 Ewing appears to be in agreement with From in regard to intelli- gence and proficiency tests and their potentiality as evaluative tools: The questions normally included in an intelligence test usually involve vocabulary, sentence completion, analogies, reasoning problems, and others that emphasize academic know-how more than reality judgments. It is necessary to look more closely at the performance record and count more heavily on intuitive judgments when evaluating one's knowledge at this level.12 Goslin states that a "person's abilities may be assessed by vari— ous means, the most obvious of which is his performance in the position for which he is a candidate."13 Because of the increased demand for office workers possessing quality levels of personal judgment and M l . Erich Fromm, ”Personality and the Market Place," Man, Work, and Society, ed. Sigmund Nosow and William H. Form (New York: Basic Books, Inc., 1962), p. 451. 12 . Dav1d W. Ewing, The Managerial Mind (New York: The Free Press of Glencoe, 1964), 'p. 133. F David A. Goslin, Teachers and Testing (New York: Russell Sage oundation, 1967), p. 3. ... .__,__. . _ vfit a... “'04:” 'n-e .01» l-h I b avug. w.“ r . V'JI '0“ . .-‘;\ a.“ e “O“ c.’_. . . "In! a... N 1 i...l 11 responsibility, those individuals being trained and educated as secre— taries should have the opportunity to eXperience situations prior to amfloyment like those they will find in that initial secretarial posi- tflnn Vocationally reimbursed programs funded as a result of the pas- sagecmfthe Vocational Education Act of 1963 and the Vocational Amend— mmusiof 1968 have enabled an increasing number of students to receive vocational education in preparation for future employment. Crawford huficates that urgency is required in directing some attention to the nature and demands of the offices in which future typists and secre- taries may go.14 Stierwalt indicates that mutual agreement seems to mdst among both the businessmen and the secretaries: . that superior secretaries not only need to be adept in the basic skills (typewriting, shorthand, business mathematics, filing, etc.), but they also need to be able to work things out for themselves. A secretary is given very few instructions, and then she works under very little supervision. Therefore, it would seem that our curriculum in business education should include some practice in decision making for the prospective secretary.15 Lanham et a1 noted the need for students preparing to enter office occupations to be acquainted with the demands of the position: Students have too often been last to be eXposed to the Specific expectations of employers for office performance. As an instructional innovation, we prOpose that students, those contemplating office employment or those committed to prepare for office work, have access to this set of office expectations. 4 T. James Crawford, "Typewriting and the Vocational Education Act °f1963." Selected Readings in Business and Office Occupations, National __ .Efiflmess Education Yearbook, No. 5 (T9677, pp. 95-96. 5 D _ . Irma Lee Stierwalt, "Help Prospective Secretaries DeveIOp eelsion-Making Ability," The Balance Sheet (May, 1969), p. 401. New 0f:Frank W. Lanham et a1, Deve§pment of Performance Goals forfia We and Busmess Education Learnings Sistem, Final Project (wary’ PrOJect No. 8-0414, Grant No. OEG-O-0—080414—3733 (083) 5 1ngton. D. C.: u. 5. Office of Education, 1970), p. 54. :v;. ‘5 12 Oliverio suggests that realistic office tasks give students experience in independent decision making.17 Lanham, in summing up the qualities needed in the future office worker, stated that: The clerical workers of tomorrow must be able to solve problems, to think, to compute, to arrive at wise decisions, and to com- municate thought . Evaluation of Student Progress Evaluation of learning can be achieved in a variety of ways using published materials that are available in business education. The following brief review of several current practices will indicate those aspects of the evaluative process for which each testing device may be best suited. Skill or Performance Tests. Timed writings in typewriting are an example of a skill or performance test used to judge the speed and/or accuracy of an individual's ability to typewrite. This particular type of test measures the ability of the typist to manipulate the mechanical parts of the typewriter, resulting in a words-per-minute rating. The dictation test in shorthand, another form of isolated performance test, measures the student's ability to take dictation at various speeds and is useful in the initial stages of shorthand skill development while forcing the student to strive for faster dictation speeds. However, the measurement or rating obtained in isolated skill tests like these F 17Mary Ellen Oliverio, "Teaching Clerical Practice Students How to Work," Selected Readings in Business and Office Occupations, Regional Business Education Yearbook, No. 5 (196fi, pp. 77—78. 18Frank W. Lanham, "Electronics and the General Clerical Program,‘l Mness Education Forum, Volume 16 (February, 1962), p. 4. “2' ...;, LA T u 13 represents achievement at only certain levels of the learning process and gives little or no indication of the student's capability of transferring these skills into performance in an actual work setting. Business Information and Related Knowledge Tests. Basic knowledge tests measure one thing in particular: the ability of the student to memorize the general business information needed in preparation for problem solving. Business terminology, organizational data, and infor— mation needed in order to understand specific aspects of the business world are included in business information tests. Such tests may be used as pretests to provide a check on information already attained by the student or perhaps as self—check activities whereby an individual student can check his competency in a given area of interest. Initial cagnitive learning in skills and secretarial procedures may involve learning such fundamental knowledges as names of machine parts, proce- dures in determining vertical and horizontal measurements, and amount and/or type of spacing required for specific kinds of problems. Tests of this nature are strictly informational, and little or no immediate attempt is made in the test to apply these knowledges to an actual business situation.19 Self—Evaluative Instruments. Questionnaires, surveys, and check— lists designed for self-evaluation by the students are helpful in determining personal qualities possessed by the students as well as their attitudes toward their occupational goals. However, the legiti— macy of such measurements in the final analysis of the student's capability of handling the work load on the job is questionable. -—___ 1gMathilde Hardaway, Testing and Evaluation in Business Education, Third Edition (Cincinnati: South-Western Publishing Company, 1966), P 301. ~ 2;..-:===~—~:§>-w-.:-£~— ..o “a. . 9". too. . 14 Production Tests. A series of problems pertaining to a Specific unit of study and completed by the student under time pressure is known as a production test. These problems may also be integrated so that there is a variety in the types of problems involved in one such test, such as tabulation problems, business letters, memoranda, statistical problems, thus giving the students the opportunity to solve several kinds of problems in one testing situation. However, these production tests usually do not require decision making in the setting of work priorities. The problems may be completed in the same order in which they are presented on the test; no problem is any more important than any other problem in setting priorities. Another means for handling the problems is to "batch process" the tasks into groups of similar tasks to avoid several "setting-up" time losses. The students will be judged on how much production is completed in a given amount of time. An added emphasis on production tests for prospective office workers has been noticeable in recent years, perhaps as a result of typewriting research which indicates increased transfer of learning when instruction stresses the production typewriting method rather than the straight-copy typewriting method.20 Hardaway emphasizes that the production test may have the disadvantage of including only a restricted sampling of applications, thus making the test "unreliable from the standpoint of subject coverage and relative time devoted to . . . . . 21 nontyplng vs. typing act1v1t1es." h. 20T. James Crawford, Production Typewriting, Monograph 97 (Cincinnati: South—Western Publishing Company, 1960), p. 20. 2 1Hardaway, op. cit., p. 277. 15 Practice Sets. The practice set focuses on a series of problems related to a hypothetical work situation and incorporated into a par- ticular unit of study. Problems included in the set are solved in the order in which they are presented, and there is seldom any time pres— sure as in the production test. The practice set may require several hours, days, or weeks to complete; and, therefore, terminal evaluation of the student's progress may depend upon what he has accomplished throughout the entire practice set. Witherow developed a practice set designed for secretarial students, The Secretary on the Job, which offers a variety of office tasks (taking dictation, transcribing, business letter writing, arranging reservations, handling the mail) in a 28-unit program.22 Although production gives the student practice in solving problems, there is little or no need for the student to make any decisions in regard to work priorities. Situation Tests. Fundamentally, the situation test evaluates typical performance on the job and is a standard work sample test.23 The test is usually designed in such a way that the test items are situations based upon those typical job activities the student would face if he were involved in a particular job situation. The situation test permits evaluation of three aspects of typical performance:24 22Mary Witherow, The Secretary on the Job (New York: McGraw— Hill Book Company, 1967}. 23Robert L. Weislogal, "Development of Situational Tests for Military Personnel," Personnel Psychology, Volume 7 (1954), p. 493. ”mm. , p. 494. ._ ...»)... _—i $11.2" _ . 16 Possession of the necessary skills. 1. 2. Rec0gnition of the need to apply these skills at the appropriate time. 3. Motivation or willingness to apply the skills. Situation tests have been used to a limited extent in office education, but with the need to measure more complex qualities of office employees this testing format may become more meaningful in diagnosing and evalu— ating the performance of an individual in an occupational setting. Evaluation is a fundamental part of the learning process because it represents the means by which progress in learning is determined. The evaluative process, however, will be helpful only "if goals are realistic, if the evaluation tools used are appropriate, and if inter- . . . 25 pretation of the ev1dence 15 sound." The Application of Simulation in the Evaluative Process One of the emerging strategies in education appropriate for applied research and development is simulation. Evaluation of learner behavior may be attained through (a) an observation of actual on-the- job performance or (b) an observation of simulated on-the-job perfor— mance. When the conditions of actual performance cannot be present, the technique of simulation may be used to create a representative situation. Researchers have found simulation a useful technique in Studying behavior in a diversification of activities, such as: Reactions to bureaucratic authority (Evan and Zelditch, 1961) l. 2. Performance of school administrators (Hemphill, Griffiths and Frederiksen, 1962). 5 . . . . National Education Assoc1ation, Toward Better Evaluation of Learning (Washington, D. C.: Council on Instruction, National Education Association, 1962), p. 1. Education (Bloomington: 17 3. Inter-nation relations (Guetzkow, 1959). Business decision-making (Cohen, Cyert, Dill, Kuehn, Miller, 4. van Werner and Winters, 1960). Operation of an air defense direction center (Chapman, 5. Kennedy, Newell and Biel, 1962). When the "real-life” model is not available for representation, snmflation provides the tool whereby the learner's potential for ' hmxfling applied problems made up of many interacting variables may be nwasured. .As a strategy in the development of particular applications oflnmic research, simulation permits the researcher to take into acannn:intangibles of human motivation, decision making, and physical mnuoundings and stimuli. Here are but a few of the definitions qnflied to the word "simulation”: 1. Simulation is a means of pooling many skills and information in an orderly way. 2. Simulation is a representation of both the ”real-life" 27 qualities from the general setting and from the content. Simulation is actual performance of executive skills in a 3. hypothetical situation. 4. Simulation is a teaching process which uses the basic functions, eguipment, and interactions which occur in a real office. 9 5. Simulation is operationally defined as a dramatic activity, condition,or process that involves manipulative transaction 26 . Nicholas A. Fattu and Stanley Elam, Simulation Models for Phi Delta Kappa, 1965), p. 18. Donald W. Fiske, "Why Do We Use Situational Performance Tests?" fkg§onnel Psychology, Volume 7 (1954), p. 465. 28 . . . .Paul S. Greenlaw, ”The In—Basket as a Training Instrument," Mflflfifllng Keys to Profits in the 1960's (Chicago: American Marketing Association, 1960), p. 452. 29 . . . Garth A. Hanson and H. Robert Stocker, "Mobile Simulation in Offite Education," Business Education Forum (October, 1968), p- 18. m. , ,. ...; 59111 snared relate ‘ 11 :Taiuat. Zinnia Emilio: 35-) inst Fir-and ‘- of 5‘ 115 intm hm "he ‘7” anti 3%.; ‘n. 111 a ‘iu \; 0f 111 :5 113111: ——’.'- \_'_- *- 18 or the interaction of abstracted elements of real—life with a motive to induce a phenomenal experience or state that replicates real-life. 0 Because of innovations in the development of instructional systems in secretarial education, such as media development and utilization, block-time programming, and individualization of instruction, evalu- ative techniques utilizing simulated experiences are gaining prominence in secretarial education. Evaluation most often takes place in a class— room setting; therefore, students may be expected to perform tasks in simulated situations. Few evaluative instruments are available which correlate the utilization of instructional simulation with instruments to evaluate the student's attainment of the instructional objective. Instructional laboratory programs have been developed which present instructional sequences geared toward the career goal of the student. Every instructional sequence, whether simulated, individualized, or programmed through some other means, requires terminal evaluation by means of specific performance tests or situation tests appropriate to the instructional objectives.31 Instructional modules designed for student attainment of basic competencies needed for entry-level occupational requirements are based on a "pretest-instruction-posttest" approach wherein the state- ment of the instructional objectives for the module sets the pace for the learning sequence. The pretest evaluates the student's achievement 30Omotosho Ogunniyi, "The Methodology of Educational Simulation and Design of a Simulated Instructional Model for Occupational Educa- tion." Unpublished Ph. D. dissertation, Michigan State University, 1969, p. 12. 31Ibid., p. 166. nor to 1 rated at I :guisitic 3:39:ng 23::in sectezarie 19 prior to the module of instruction, and a criterion test is adminis- tered at the completion of the instructional sequence to measure acquisition of learnings during the module. In the simulated office laboratory, students may spend full days completing the tasks that are deemed typical in the business office for a particular level of work.32 Teachers involved in the preparation of secretaries realize that the standards for evaluating prospective steno— graphers and secretaries at or near the end of their training programs should be as much like the job situations in which they will be working The simulated office expe— as is possible to achieve in a classroom. rience may involve the simulation of tasks, positions, personnel, pro— The result is a cedures, environment, or combinations of these. simulated "input—production-output" process keyed into the educational setting representing an interactive decision-making process among The decisions made by the participants become several participants. interrelated in order to accomplish the goal of the activity in the most rapid and accurate manner. The simulated office experience is perhaps as close to the actual office setting as an instruction or activity in an office education laboratory can be. An evaluative instrument is merely a tool in the evaluative process; it is an aid to evaluation and possibly in observing students The more efficiently in order to make more reliable comparisons. ..____________________ 32 . . . Mary Ellen Oliverio, "PrOJections for the Future," BuSiness Education: An Evaluative InventorL National Business Education Yearbook, No. 6 (1968), p. 126. 33 Alfred Patrick, ”Strengths of the Past and Present," Business An Evaluative Inventory, National Business Educaron‘ Education: 11 . Yearbook, No. 6 (1968), P v. —-._~ ~_— -_-_ '-~-* 2-'_--_-...:_-_.._._ .-. _ ._— . - . ' " L'I-v."‘-g -_,.- lztional Ed tame mu 3: highly w l :: far whic casket E .——— .55 a s 19“; for 5 anion is .. :espnn ————...A - ' 20 National Education Association emphasizes that tools of evaluation have become increasingly sensitive but that "there remain many intangible but highly important educational outcomes that are not 'measureable' or for which available tools are still imprecise."34 In—Basket Simulation as an Evaluative Tool As a special form of simulation, the in-basket presents a tech- nique for simulating a wide variety of real-life problems requiring decision making. Emphasis on a particular office position and level of responsibility enable the in-basket to be a test of analytical ability, knowledge of business principles, and decision-making skill for the employee who is solely responsible for the performance of spe- cific job tasks as contrasted with interactive behaviors essential in other components of the job. The in—basket may usually be utilized within relatively short periods of time and has, as its main purposes, the following:35 1. To illustrate specific points covered in a presentation or lecture. 2. To enable the learner to test ideas and principles presented. 3 To help the learner gain insight through self-discovery into his way of handling administrative matters. 4. To arouse the learner's interest in a subject to be presented. 5. To reform the learner's ideas about the management function. 6 To be a test of what has been learned. The merit of in-basket simulation is still being tested in indus- try as well as education in order that its usefulness as a tool for 34National Education Association, op. cit., p. 4. 35Allen A. 2011, III, The In-Basket Kit: Materials for the Creation and Use of In-Basket Materials (Reading: Addison-Wesley shing Company, 1971 , pp. 3-4. research, training, and sele Hansen, Jensen and Beat :séistic situational test i administrator's paperworl :12 of an administrator but ii; the incumbent of the ne :sxai: kinds of eXperimenta temipulated by varying t1 .1: test begins. The reseaI M: :3 many ways, includir ..mnany, changing persor ....ngmg the purposes of 1... L19 rollouing as possit .. Them-basket alloy cUbJECtS react but problems are handle .. T‘ ‘ he ln-basket prox'i Le selection of it an one facet T u fin-basket has I cSIght, not mere 1 21 research, training, and selection may more closely be delineated. Frederiksen, Jensen and Beaton36 used an in-basket test as an elaborate realistic situational test in an attempt to simulate certain aspects of an administrator's paperwork. The subject was not asked to play the role of an administrator but instead to behave as though he were actu- ally the incumbent of the new job. In—basket tests are well adapted to certain kinds of experimental applications, specifically those that can be manipulated by varying the background information presented before the test begins. The researcher can vary the organization experimen— tally in many ways, including changing the organizational structure of the company, changing personalities of the subject's superiors and peers, or changing the purposes of nature of the organization. Researchers view the following as possible merits of in-basket simulation: 1. The in-basket allows for potential problems to which some subjects react but others do not; the order in which the problems are handled is not specified by the developer.37 2. The in-basket provides for use of situational variety in the selection of items and in developing items with more than one facet. 3. The in—basket has the potential to measure recall and insight, not mere recognition of problems. 4. Because the in-basket provides the vehicle for individu— ality and originality, it requires analytical and critical thinking, logical reasoning, and problem solving by the subject. 36Norman Frederiksen, Ollie Jensen, and Albert E. Beaton, Or anizational Climates and Administrative Performance (Princeton: Educational Testing Service, 1968), p. 9. 37Lee S. Shulman, Michael J. Loupe, and Richard M. Piper, Studies of the Inguipy Process (East Lansing: Michigan State University, 1968), PO 31. 3. The in-basket test correctly 3 situat problems. 6. The in-basket meas make decisions and decisions. '. If the problems po the subject, the i in a situation res . Either during the exercise, feedback of action taken bv 9. The in-bashet is c m a situation wit . The in-bashet pert: mental treatments all subjects in id complex and realis Selec‘i “ 0“ Of an appro; ;:'.':L'e~ . ' ' \sdecemlnatlon of t clippers . . or recording ta 22 5. The in-basket tests the individual's ability to judge correctly a situation involving a selected range of problems. 6. The in-basket measures the individual's willingness to make decisions and the resultant ability to make actual decisions. 7. If the problems posed in the in-basket are realistic to the subject, the in-basket involves and interests learners, in a situation resembling reality. - 8. Either during the in-basket or immediately following the exercise, feedback is provided explaining the consequences of action taken by the subject. 9. The in-basket is capable of eliciting typical performance in a situation without the need to deceive the subjects. 10. The in—basket permits the assignment of subjects to experi- mental treatments according to plan and the placement of all subjects in identical problem situations that are both complex and realistic. Selection of an appropriate design of an in-basket simulation inwflves determination of the degree to which interaction need be only wiflipapers or recording tape, determination of the need to individu— alimathe in—basket for a particular purpose, and the compression of thmein putting the student into a reality situation. Basic in-basket ckwigiis derived from one of these three organizational alternatives: 1. The Solitaire Format: The subject is confronted with an imaginary environment by committing himself, individually and in writing, to specific courses of action without the interference or interaction of others. 38 . Allen A. 2011, III, Qynamic Management Education, Second Edfifion (Reading: Addison-Wesley Publishing Company, 1969), p. 133. 39min. 0 Frederiksen et a1, loc. cit. l Frederiksen et a1, 0p. cit., p. 11. Z. The Snall-Grou Fo: usually two to fiw taken dePen‘ienlt on format is used pri! relations skills- (4- The Gro 01‘ Team in team-like compe‘ each team may repr‘ are a venture of t? lope: describes the in :zsimlate the tasks an of biogagiven time period. the Educational Testing 'l popularity as a tech ESih-bashet h as been used 513m to simulate the "rea ..uattempt to educate an old to\" ' ‘ ‘ ' , 1e» an individua cf: ‘ ' thetechmque in cario 23 2. The Small-Group Format: Interaction with other persons, usually two to five in number, is required with actions taken dependent on the actions taken by the others. This format is used primarily for the assessment of human relations skills. 3. The Group or Team Format: Each group of players is involved in team-like competition. If the team approach is used, each team may represent one company; and the actions taken are a venture of the team, not the individual player. Lepez describes the in-basket as a "situational technique" used to simulate the tasks an office employee would be confronted with during a given time period.42 Since the in-basket was first developed by the Educational Testing Service (ETS) in 1957, it has gained consid- erable papularity as a technique for research, training, and selection. The in-basket has been used by business, industry, and education in an effbrt to simulate the "real world" in hypothetical situations, perhaps in an attempt to educate an individual into the reality of the business world, to view an individual as a future employee, or to experiment with the technique in various research efforts. 2Felix M. Lapez, Jr. , Evaluating Executive Decision Making: The In- Basket Technique, AMA Research Study 75 #(American Management Association, Inc. , 1963), p.17. __12L2=H== otsro THEOE 3,. Int: The foci of the review ling I. Design and Develonz research and liter; in-basket simulatia from areas of educ. 0gp, philosophy, 3: . The Historical Det'u use of in-basket 5. mg, and secretari; training, and/or t: CHAPTER II DESIGN THEORY FOR IN-BASKET SIMULATION A. Introduction to the Review The foci of the review of related research and literature are related to four specific areas of study in order to adequately evaluate the information and data obtained with their degree of relationship to this study: 1. Desigp and Develgpmental Theories: Selected reviews of research and literature pertinent to the development of in-basket simulation; design and developmental theory from areas of education, tests and measurements, psychol- ogy, philosophy, and other areas of concentration. The Historical Development of In—Basket Simulation: The use of in-basket simulation in research, management train- ing, and secretarial education as an instructional, training, and/or testing procedure. Analyses of the Secretarial Role: Research pertaining to the role of the secretary and delineation of personal traits, duties, reSponsibilities, behaviors, and/or tasks perfbrmed. Research in Curriculum Development: Application of instructional mOdels, implications for instructional materials and evaluative instruments in the development of secretarial training programs. West's review of three general phenomena appearing in research and literature pertaining to the entire field of business education bears direct resemblance to the foregoing review phases:1 1Leonard J. West, "Research on Teaching Business and Commercial Subjects," Research Report 71-1, Research and Evaluation Unit, Division of Teacher Education, The City University of New York (January, 1971), p. 3. 24 1. the necessary se out: and to the ' 3. Concern with the 5.. "-ae development 0 fora. Because of the panic Iizmture, three chapters :ti: dapter highlights 2“: to in-basket simulati ieelopnent of in-basket 5 Jon the in-basket has Inaiyses of the secreta Erin-sent in secretarial B. Desi n T The soundness of methc iselepnont of evaluative 1' fiber, special consideral Eligsinulations must be i EMS innovative and meat march related to the de: emu by leading theori “El, and other concenti es Little has been writtt 1., . 2 ‘ulition. However, a n1 olum\iyi, Op. cit., 1 25 l. The necessary sensitivity to occupational trends and fore— casts and to the impact of technology on office occupations. 2. Concern with the development of character and personality traits thonght to be associated with obtaining and retain— ing a job. 3. The development of instructional materials in programmed form. Because of the particularistic nature of the related research and literature, three chapters will be devoted to the presentation. Part 1 in Hus chapter highlights the design and deve10pmenta1 theories perti- rwnt'u)in-basket simulation. Chapter III highlights the historical dewflopment of in-basket simulation and presents the fundamental ways :htwhiCh the in-basket has been used in research and training. Research on muflyses of the secretarial role and implications for curriculum dewflopment in secretarial education is the focus of Chapter IV. B. Desiginheory for In-Basket Simulation The soundness of methods and proposed techniques utilized in the (Evehnmwnt of evaluative instruments has not always been tested. hhwewng special consideration for these and other theories for devel- Ophugsimulations must be in the forefront when planning the deveIOp- mmn:of innovative and meaningful tests for eventual classroom use. lkmearch related to the design of simulations reveals that propositions expounded by leading theorists in areas of education, psychology, phi- RBQNDU and other concentrations create a unique orientation for this study. Little has been written on the methodOIOgy of designing in-basket shmuation.2 However, a number of proponents of educational Simulation h 2 . . . Ogunnlyi, op. Cit., p. 80. Enduised theories which simlation Those aspects of in—b adieu, psychological, 0 Ellis chapter because it Lmdegrees of relationshi flirts of theoretical b ifpesented. Therefore, t these theories are pre " . roaches to Test In developing testing “dual subject's perfor Mule approaches to tes ’ 3‘: leveleped as a result 1- The Classical Psyc l eliciting tests of skill~knowledge co ducing tests of ty l 3- The Rational H at ‘ Rpresenting real- he Classical Psychome fires of the classical 1: )3 . “\m performance of b - “lemma ‘ 1 . 81 PS chc lamina. compar’tyj ‘isei 26 have devised theories which should be examined for their applicability to the development and design of in—basket simulation. These basic theories may be grouped within the following categories: 1. Basic approaches to test development 2. Developmental theories for in-basket simulation 3. Measurement of individual performance in in-basket simulation Those aspects of in-basket design which reflect specific philo- sophical, psychological, or sociological premises are merely highlighted in this chapter because it is felt that, though a number of theories have degrees of relationship to the study, the isolation of only those elements of theoretical bases definitely pertaining to the study should be presented. TherefOre, the incorporation of the fundamental bases for these theories are presented as documented in the literature. Basic Approaches to Test DeveIOpmep£_ In deve10ping testing devices to be used in evaluation of an individual subject’s performance, one must consider the components of the basic approaches to test development. Evaluative instruments have been developed as a result of these two basic approaches: 1. The Classical Psxchometrical Approach: Utilized in eliciting tests of maximum performance in specific skill-knowledge concentrations. 2. The Rational Hypothesis Approach: Utilized in pro- ducing tests of typical performance in situations representing real-life phenomena. The Classical Psychometrical Approach. One of the characteristic features of the classical psychometrical approach is its measurement of the maximum performance of the subject in a specific skill or knowledge.3 3Anne Anastasi, Psychological Testing, Second Edition (New York: The Macmillan Company, 1961), p. 34. mint of a specific 5 nice represent a hypoth net jldgnent of the tes Mingle external criteri an the subject's achi helopnent of evaluative titles designed to dete Lair": of information or lest itens for such or trees or expert judgment isemltiple-choice, fi incest classroom tests in my be few standardi: EstoSpecific content are Sale textbook publishe texts. These can be h Included have not beer critically by other e) Orin the subject fie] =51 upbasizes that the cc L‘lhol‘king closely with 4 3iii)‘ result in excellent ~‘ee development of extel £l i‘t‘lc' Flanagan, "Some Mu Tests," Personnel . N 3 a, We“ L. Ebel, Measr ' Prentice-Hall, Inc 27 A testing device utilizing this approach will encourage the subject to earn the best score he possibly can. In evaluating the subject's attainment of a specific skill or knowledge, a criterion measure is used to represent a hypothetical true score, this measure derived from expert judgment of the test administrator or from a referrent source. No single external criterion is used as a guide to appropriate measure- ment of the subject's achievement. The approach, when used in the development of evaluative instruments, produces tests of aptitudes and abilities designed to determine whether the individual knows a particu- lar bit of information or whether he can apply a specific skill.4 Test items for such measurement are usually derived from documented sources or expert judgment and culminate in a variety of formats (true- false, multiple-choice, fill—in-the-blanks, matching). Ebel indicates that most classroom tests must be prepared by the instructor since there may be few standardized tests of achievement available apprOpri- ate to specific content areas:5 Some textbook publishers furnish tests to accompany their texts. These can be helpful, but too often the items included have not been carefully prepared or reviewed critically by other experts in educational measurement or in the subject field itself. Ebel emphasizes that the combination of (a) experts in test construc- tion working closely with (b) expert teachers of the subjects involved usually result in excellent evaluative instruments. Substantial costs in the deveIOpment of external tests and the problems of matching the M 4J. C. Flanagan, "Some Considerations in the Development of Situation Tests," Personnel PsychOIOgy (1954), p. 461. 5Robert L. Ebel, Measuring Educational Achievement (Englewood Cliffs: Prentice-Hall, Inc., 1965), pp. 8-9. mentofthe test to th firetotbe necessity for dtlspecific skills and L'lerrelatiomhip of cont ppc'suetrical approach h helping skill tests an res. In testing format ilemrltiple-choice) o gmlasolving), the c1 finaderelopnental desi :ihorledges to be meas lhe Rational H\ othes trite based upon the rati iic behaviors relevant t File: situations are dew 5mm in a social situa‘ ”liter of traits, not ju: L'ifipiven phenomenon or E31103 such as those in SaksWUJlllpare the relat ztin task and direction. W1 Criteriong This 6\—‘ [b.1113 P- 9. -. {Imam "Some Cons: ' Pl 463. 3. tiske lo ‘ C . : Clt . hill) P- 467, 28 content of the test to the material emphasized in the classroom corre- late to the necessity for teacher-made tests.6 Test items prepared with specific skills and knowledges incorporated may have little or no interrelationship of content within the testing format. The classical psychometrical approach has been used extensively in designing and developing skill tests and specific informational tests in many subject areas. In testing formats where there is a right or wrong answer (true- false, multiple-choice) or where there is a style format to follow (problem solving), the classical psychometrical approach appears to offer a developmental design that will identify only specific skills and knowledges to be measured. The Rational Hypothesis Approach. The principal aim of a testing device based upon the rational hypothesis approach is to identify spe- cific behaviors relevant to the variables being measured before the problem situations are developed.7 This approach evaluates total per— fbrmance in a social situation and attributes the variance obtained to a number of traits, not just one pure trait.8 The assumption is made that a given phenomenon or situation includes a complex variety of behaviors such as those involved in a real-life situation. The test seeks to compare the relative effectiveness of the subject's involve- ment in task and directionality of effort with an accepted contemporary . . 9 . . . external cr1ter10n. Th1s external cr1ter1on, gauged as the common 61bid., p. 9. 7Flanagan, "Some Considerations in the Development of Situation Tests," p. 463. 8Fiske, loc. cit. 9 . Ibld-, p. 467. cult of the nethod used Scientific nethods o instructional material solve (a) observational imminent technique emotion is the most b 3.51, shether he himsel :iitidcal perform in thi “:3 type of observation floured by the persona iegenerosity error are ital cases of on-the—jo Citlcal incident techniq incite on-the-job behavio Ehrence between success 3ij Flanagan's proce flethniques for collecti Siauay as to facilitat r‘eliul problems. Becau ESEnotions recorded by t \‘N 10 its Calfrey C. Calhoun \.~~'- earth to Business Ed coho. 9 (1971), pp. 3c Md... pp. 304-305. 1 FlaMean. "the Crit 29 standard for the measurement of achievement, is often derived as a result of the method used in developing the test content. Scientific methods of data gathering for use in the development <fl?instructional materials and correlated evaluative instruments may :hwolve (a) observational techniques, (b) questioning techniques, and “3 measurement techniques.10 Heimerl and Halldorson emphasize that (nwervation is the most basic measuring device available to the experi- imnner, whether he himself is the observer-recorder or has another hufividual perform in this capacity. One of the problems existing with 1jus type of observational analysis is possible distortion of the data hfifluenced by the personal values of the observer. The halo effect and 1fim generosity error are examples of such distortion.11 Incidents from acUufl.cases of on-the-job behavior may be obtained using Flanagan's (ndtical incident technique12 that determine either effective or inef- finmive on-the-job behavior. Such critical incidents identify the dUHkrence'between success and failure in completing given aspects of fluejob. Flanagan's procedure for critical incidents consists of a set oftechniques for collecting direct observations of human behavior in sudiaiway as to facilitate their potential usefulness in solving pnumical problems. Because the critical incident technique utilizes (flwervations recorded by the job incumbent, it has the advantage of __ 10 Calfrey C. Calhoun and Mildred Hillestad (ed.), Contributions Oflkmearch to Business Education, National Business Education Year- kqglg. No. 9 (1971), pp. 304-310. 11 . Ib1d., pp. 304-305. 2 Flanagan, "The Critical Incident Technique." rersonai interaction and significance. Other types ineioped within the categ i. is the observatic ', 15 the observer: ' 3.1 Isthe vi en of t} Interviews, questiont attaining information from is (structured, nonstruc 755331 and nonverbal clari tie interview technique Kl 14 441's. Questionnaires @155 ::e is large enoug irfnds to observe or int tzzes referred to as a lil Elthtined Forces and ti 1.: :0 select individuals ltasurenent technique :Itmlg‘ The Thurstone n z‘ele « ,and the Guttman Sce \za‘fmeasurenent lb 05 . gc 13c “hm“ and Hillest blh N'Boppenheim Q] Basic Books, 30 impersonal interaction and allows the respondent to select incidents of significance. Other types of observational techniques involve patterns developed within the categories of these basic questions:13 1. Is the observation known or unknown to the subjects? 2. Is the observer a participant or a nonparticipant? 3. Is the view of the situation direct or indirect? Interviews, questionnaires, or self-inventories are three ways of obtaining information from questioning. The basic design of the inter- view (structured, nonstructured, single, multiple) will reflect both verbal and nonverbal clarification of a situation. Oppenheim compares the interview technique with mailed and group administered question- naires.14 Questionnaires are particularly advantageous whenever the sample size is large enough to make it uneconomical for reasons of time or funds to observe or interview every subject. A self-inventory, some- times referred to as a life history inventory, has been used frequently by the Armed Forces and the National Aeronautics and Space Administra- tion to select individuals for flight training and scientists.15 Measurement techniques utilized in the affective domain are not as numerous. The Thurstone method of equal-appearing intervals, the Likert scale, and the Guttman Scale-Analysis are three techniques for attitu- . 16 . . . . dinal measurement. Osgood's semantic differential scale 18 used to 13Calhoun and Hillestad, loc. cit., p. 305. 14A. N. Oppenheim, Questionnaire Design and Attitude Measurement (New York: Basic Books, 1966), pp. 30-35. 15C. W. Taylor and R. L. Ellison, "Predicting Creative Performance from Multiple Measures," Widening Horizons in Creativity (New York: John Wiley 8 Sons, 1964). 16 Calhoun and Hillestad, 10c. cit., p. 308. eternine a semantic distal attitudinal referents withi rises the importance of ti Elenagan indicates the 2ng demonstrate what } iii in in future situation tests of t)pical performanl tabulations representing and. ltest based upon 1 en three criteria-~its c; izznfthe subject in the in together to arrive at isumarization of the :. .ne theoretical structi .:n the following page- l . s ..l: iuuuental Theories for . . u ill-basket simulatit 313i“; 3 ‘ ~ sub e ' ‘ J ct s perrorr linden: ‘ Upon interperson' fi him . SSsetting. Howeve r. “lithe ' SUbJECt responds 1 17 .,-. C. Us “plums 800d et a1 Th Press ’ \< 13' , 1957) !_____4 31 determine a semantic distance between groups and individuals and between attitudinal referents within groups and individuals.17 Stephenson dis- cusses the importance of the Q-technique in ordering items or concepts.18 Flanagan indicates that the subject's response in situational test— ing will demonstrate what he actually does (typical performance) or will do in future situations, not what he thinks might be done.19 Such tests of typical performance may be work-sample tests, situation tests, or simulations representing phenomena found in the actual business world. A test based upon the rational hypothesis approach is dependent upon three criteria--its capacity to represent a situation, the involve- ment of the subject in the situation, and the directionality of effort-- taken together to arrive at a relative effectiveness measure. A summarization of the descriptive elements of these two approaches and the theoretical structure peculiar to each is shown in Illustration 1 on the fbllowing page. Developmental Theories for In-Basket Simulation An in-basket simulation is basically soci010gical in nature as it relates a subject's performance in a given situational role, a situation dependent upon interpersonal relationships and behaviors in a particular business setting. However, other disciplines may also be influential when the subject responds to certain stimuli presented in realistic 17C. Osgood et al, The Measurement of Meaning_(Urbana: University of Illinois Press, 1957). 18W. Stephenson, The Study of Behavior: _Q:Technique and Its hbthodology (Chicago: University of Chicago Press, 1953). 19Flanagan, "Some Considerations in the Development of Situation Tests," p. 462. a cu mocauuno . o; . . . luau“. “dquou a :u oucufihomuom “duo“ nevusau>z .m nuov .m acnudcaumu .MCOAuazuan 95.53..“ c.“ CV and: no noon 3.23.306 Mocha—flu on» Harv: nko>oo new .Q uni—UNQCORU Inufl «EOduUDUAE EOE—"N a C.“ EROkHoA AnadOfiQku «NOV .Q .EGMflCdHlu .XUCOHDQEOU ESEfiKNE H“ Show" fine? In" qflfludflfldlu .UORSISOE 3:400. QDHQI—Cnflsr 9:“ On. HCU>9H0¢H A . c . M. 40.? ha :umflcafluu HVQHJWNWE ”fin OH. “0 ~00H3°n~v~ Ik0d>lfiln~ UnNfiUOrn-u urn n-OHWIU Hindu-:01... CH Can—3 nvrnnv lfifluqxn ”unit—‘99:“ nHO cudpuoflun “gramme.“ uh.“ CORR: BUM-V“ unbounduvuhhn - W 1.9!!5 lflCI—FHOLHOA ulnvwnnku eva- aqu-vur IOU-Juuophn- .u. Unvldvfi In! fiUflHfiu...‘ 1:: InduUIWHu—hwiu “.0 _ _ _ _ . sun—.53 I: a! EOhkOQ 0n. HUOfinuln 0S9 oomfikSOUfim .N .. IROQ 0U XHHOUQUU n.900fi05m 0S“ MCHHOEONQ u.“ MEund‘ .N — . . ..wvoQ amok Op monomonmg< oflmmm “H cofiumaumDHHH hem .a ..aaaaazv .mmsouw compouflho vmpooHom ow pmmu mg» mcflaop mom .m .Hon mam mcousozv .mfinmfiam> voflmflommm m EUE Amm A9 nomoHo>ow pcoESgpmcfl opp mopmwfifim> .n “ nufiz mopmfioaeou mcflon mo pudendum oHow ecu muooz .n have .d .oxmuav _ . . .oocmEHomaom .mwoomnzm mo mmoao>wuooMMo o>fiumfio~ _ mo .m ammpmmco . ca moocoaommfiw Hmswfl>flvcfi mwamfi moummflofluq< .o _ :« muooflnsm mcoew xufleuomfic: ow o>fiucoupm mH c hum .m acmsovcfiq ”wow .9 .mxmflmv .COHHouflHo " . Hmcaouxo xamuomEopcoo mongooom cm and: unommo mo _ nmov .m oxmwmu xpflflmzofluoohflw mam xmmu cw pcoEo>Ho>cfi m.uuom _ :.ohoom asap: Hmufluoguomx: m was oHoom vocfimuno . :QSm mo mmoco>fipoommm o>fiumaoa ohmmaoo cu mxoom .m _ one coozpon oocomcommoanoo may oufleflxme on mxoom m Anew .m .mxmfimv .unoEo>oficom mo _ nmov .m .oxmwmv ucoEoHSmmoe How cymvcmum :oEEoo exp mm mphommo _ .coflnopflho Hmcnmpxo mfimcflm o: wcflon macaw mo xpfifimcowpuonfiv .pcoEo>Ho>cfi .xwflommmov _ :aohoom 039p: Hmoflumnuomxn map mumommamon goes: Hofl>mgop scanoufino Hmcmouxo moumuomaoocH .v _ cofihouflhu o>wuoohno no umNflvamcmpm m momfiflflu: v _ now .d .Hmnm _ cam maoacozv .cofiuwSuHm co>fiw a mo mmcfivcmum _ lumen: venom waflostHm um mEfi< meow .m .oxmfimv _ .uflmau chum oco push you .mufimhu mo Hones: _ . m on wocfimuno oocmflnm> may mousnfiauum ”soap _ Aflov .m .cmmmcmamv .omwoflzocx no HHfixm -msufim Hmfloom m :M mocmeuomaom Hmpou moumsam>m .m _ oflmfloomm m mo «coacflmupm m.uuonn:m esp mouasfim>m .m mmov .m .zmmmcmamv .mQOHumzpflm enduzm :« ow _ Hafiz Ho meow xHHMSpom poohnsm map was: mko>oo _ hem .m .nompzoau imam “coaumSufim co>ww m ea snowman AHHmofimxp _ ”Nov .m acmwmcmamv .xucopomaoo Esawxme um snow vflsoz o: mm anomaom on poomn3m can mommasoocm .N _ -Hom ou xpflommmo m.poomnsm may weapoEoum um mefi< .N _ mace .m .eemaeeuav _ .moHSmmoe memos moflpmflam> can 0“ pcm>oHoH . mace .m .cmmmcmamv .poysmmoa on on momvomzoax ”HOME/@303 Ufim—HHUOQW “HO GOHHNUHmnfiHfiOUH Cw COQS _ V5 mfififixm UHMHUmQW W0 COwHNOflMfivfimfifl and C093 momma oUCMEHOMHom Hmuwmxu mo mummy mooswogm .H _ momma mowpwaflam mam movduwpmm mo mumow moUDVOHm .H I. . examinees (psychology) ‘ml" business world (phi psychological, and sociolo :«baslet simlation will human lhich lliE SEC .lPhiloso hical Basi iibsophically based the ment as closely as p nld. cagne’ believes th .. the practice of e decisions that put ' lust effectively car the real situation. N helli alphasizes the Cl“ uroresentation of some filial perfornance of busi illution, it represents 0 dual classroom and a rea tiltines believe that in .. a simulated office unetencies to use k In a work situation find in employment. 2 \__ 20 a} Robert Glaser, Tr ' ”Hons. Inc., 1962 l i, Robert P01 Triathin and and 33 circumstances (psychology) or when the simulation must represent the "real" business world (philosophy). A brief review of phi1050phical, psychological, and sociological premises underlying the development of in-basket simulation will provide a basic understanding for the founda— tion upon which THE SECRETARIAL IN—BASKET was later developed. A PhilosoPhical Basis for In-Basket Simulation. Proponents of philosophically based theory believe that in-basket simulation must represent as closely as possible a situational excerpt from the real world. Gagne believes that .. the practice of essential motor skills, of procedures, of decisions that put 'knowledge' into practical action can be most effectively carried out in a situation which represents the real situation. Ghiselli emphasizes the quality of the theoretical model as a stand-in or representation of some phenomenon. Because simulation requires actual performance of business skills incorporated into a hypothetical situation, it represents one means whereby "the gap between the conven— tional classroom and a real business office can be bridged." Poland and “aims believe that in instructional applications .. a simulated office environment should enhance students' competencies to use knowledges and skills through application in a work situation closely resembling that which they will find in employment.” 2 . ORobert Glaser, Training Research and Education (New York: John Wiley 8 Sons, Inc., 1962), p. 241. 2 . 1Edwin E. Ghiselli, Theory of Psychological Measurement (New York: McGraw-Hill Book Company, 1964), p. 4. 22 . Hanson and Stocker, loc. c1t. 23 . - Robert Poland and Peter Haines, A Study of a Block Time Schedule for Teaching Vocational Office Practice, Final Report, Project No. 201, Grant No. OEG-3-7—O70211—2679 (East Lansing: Michigan State University, 1969). p. 12. Simlation may be an timeoqlete task ins dilation, the degree 0f tryofthe presence of igrees of abstraction co simlation (the nuiier 0f fidelity of reproduction, The process, sit is based. A replication of _...., ... >- 8 5’. ."9‘. p—4 3 O H 91 u..— .>> no» ’6“; we 8.0 n E” ”n ... age Hr? hssuggests that two of tunes on which the model Flues or situation) have title-em, usually coup bid is Ianipulated in st “W1 Illl‘ld represented 0min theorizes th an for instruction se Educational simulati nth the outside wor emung experiences 6 OMiYi, OE. Cit- 34 Simulation may be an effective vehicle whereby an individual prac— tices a complete task instead of disconnected parts. Two elements of simulation, the degree of abstraction and the time element, show ten- dency of the presence of philosophical premises. Moss identifies five degrees of abstraction contained within each of four criteria of simulation (the number of elements reproduced, centrality of elements, fidelity of reproduction, replicability) :2 l. The process, situation, or activity on which the model is based. 2 A replication of the initial process or situation. 3. A control laboratory type of model capable of being replicated. 4. A synthetic abstraction of essential elements. 5. A closed analytical model. Moss suggests that two of these five degrees of abstraction (the process on which the model is based, a replication of the initial process or situation) have as their focal point the real world. The time element, usually compressed in in-basket simulation, is a factor which is manipulated in such a way that certain central features of the real world represented are simulated in less than real-world time.25 Ogunniyi theorizes that the main philosoPhical rationale of simu- lation for instruction seems to be based upon the premise that Educational simulation simultaneously links the student with the outside world through the realistic participatory learning experiences in the classroom. 24 Fattu and Elam, loc. cit., p. 19. 25 . . . Lee F. Anderson et al, A Comparison of Simulation, Case Studies, and Problem Papers in Teaching Decision Making (Evanston: Northwest—e—fn University, 1964), p. 35. 26 . . . Ogunn1y1’ Op. c1t., p. 23. , ms indicates that It moot expect the cultu on, and no single educa my a range of education i'n'erent cultures-—indiv his theory is the basis ':lusters" of problems C0 an.” Flexibility and suits for the progressiv megration theory harmon: tuner designs for learn tenable relations of r4 ferries emphasize that a hint only if it will 01 fisuhject particularly 1 We it with another. Such theories of cul i’u integrationists, an hitl to eXperience beha 27 . ‘George F. Kneller, 0T1: John Wiley G 28 GurgeD S ' u . pindler 5% (New York: Ho 29 in p. 106. 35 Boas and Brameld both have depicted theories of culture that border on philosophical-anthr0pological approaches. A look at the cultural environment and the theory of cultural relativism formulated by Franz Boas27 indicates that it is the individual who must seek change and who must not eXpect the culture to change. Man is the product of his cul- ture, and no single education suitable to man per 26: is available. Only a range of educational systems would be appropriate to men of different cultures--individual differences, to put it another way. This theory is the basis for progressive education which centers on the "clusters" of problems confronting young peeple in periods of transi- . 28 . . . . . . . tion. F1ex1bility and Willingness to eXperiment are the baSIC neces- sities for the progressive educational system. Brameld's cultural integration theory harmonizes knowledge, values, practices, and beliefs into new designs for learning. New curricular designs depend upon the observable relations of real people living in real cultures. These theories emphasize that a particular school subject is of worth to the student only if it will enhance that student's life. Time itself makes one subject particularly relevant or provides the circumstances to replace it with another. Such theories of culture expressed by cultural relativists, cul- tural integrationists, and progressivists affect the student's Oppor- tunity to experience behavior required in specific occupations. If ____ 27 George F. Kneller, Educational Anthmiology: An Introduction (New York: John Wiley 8. Sons, Inc., 1965), p. 31. '— 28 George D. Spindler, Education and Culture-~Anthrgpological Aggroaches (New York: Holt, Rinehart and Winston, 1965), p. 105. 29 Ibid., p. 106. elated vomioual curri all/or mlaent actual fietmsition into the mile: Hay. APs cholo ital 3351 in reinforce-ant theo fiole theories present ‘ isfaced lith a stilulus morning to regulatory inmates a stimlus-re yriate and relative fee ffipmses. Silulations eredback, espe ‘Idiately following the ’ifyfive criteria relate Mack, explicitness, i dearly psychological in Motion of performance 721g Vlrgil M. Howes, $111“ 0°um» 1 36 related vocational curricula involve learning activities which resemble and/or complement actual on-the-job eXperience, the student will make the transition into the business community as an employee in a much smoother way. A Psychological Basis for In-Basket Simulation. Skinner's imme- diate reinforcement theory30 is perhaps one of the most easily identi- fiable theories present in a simulated environment. When the subject is faced with a stimulus from the real-world situation, he will respond according to regulatory stimuli he has previously experienced. Twelker advocates a stimulus-response-feedback theory31 which encourages appro- priate and relative feedback to provide the subject with evaluation of responses. Simulations are largely student feedback rather than instructor feedback, especially when debriefing sessions are held immediately following the simulated experience. Anderson gt_§l_iden- tify five criteria related to the stimulus-response theories: interest, feedback, explicitness, facts, and principles.32 Two of these are clearly psychological in nature: feedback (appr0priate and meaningful evaluation of perfOrmance) and eXplicitness (the capability of the sub- ject in identifying problematic elements in an analytical or technical sense). Ogunniyi concludes that the stimulus-response theory is basic to all simulation designs with these five criteria illustrative of that design:33 30Virgil M. Howes, Individualization of Instruction (New York: The Macmillan Company, 1970), p. 83. 31Ogunniyi, op. cit., p. 63. 32 Lee F. Anderson et al, pp. cit., p. 12. 33Ogunniyi, loc. cit., p. 66. 1. Astiuulus situa‘ 2. Aresponse situa‘ 3. ltonsequence si the stimulus and 1. .lfeedback seque 5. Acontrol and ev Thomdike's identica resent in the original 1 :aleaming if transfer .. for transfer of l to another, the orga not necessarily ider present in the origi agar? and Bower regard :ezencial because of the than older situation I :zitch “ , ple of analogy orte i: this ' type of experienc .n rejects were clearl: 5‘:€ , . bmull) then the sul :Lthe '7' I omng the value < 3:30? .. I: the theory that Effect '. . 15 more immediate \\ 34 . H. ' ‘iidem PJ‘ hlausmeier re55,1963) ’ h J 37 l. A stimulus situation 2. A response situation 3. A consequence situation representing the interaction of the stimulus and response 4. A feedback sequence 5. A control and evaluation sequence Thorndike's identical-elements theory signifies that those elements present in the original learning situation must also be present in the new learning if transfer is to occur: .. for transfer of learning to take place from one situation to another, the organism must recOgnize something similar but not necessarily identical in the new situation that was also present in the original learning situation. Hilgard and Bower regard the subject's reaction to a new situation beneficial because of the identity of this new situation, in part, with an older situation previously eXperienced by the subject. The principle of analogy often described as assimilation is also applicable in this type of experience. Kohler has suggested that, if the rele- vant objects were clearly presented, problem solving may be accomplished 37 insightfully when the subject sees the relationships between the items. In theorizing the value of transfer of training in simulation, Cronbach supports the theory that in applicational transfer of training the effect is more immediate than gains in aptitude as a result of the 3 4H. J. Klausmeier, Learning and Human Abilities (New York: Academic Press, 1963), p. 361. 351mm, p. 353. 3 6E. R. Hilgard and G. H. Bower, Theories of Learning (New York: Appleton-Century-Crofts, 1966), pp. 44-45. 37 - Sarnoff A. Mednick, Learning (Englewood CllffSI Prentice-Hall, Inc., 1964), pp. 11-12. :8 . . enerience. Posrtive t 3: factors are present i 1. Relevance of rare 2. Situational eqUi equivalence in t to the learner). Fattu and Elan propose a Scarring strategy." Thi tstarce of the concept t attributes of this instar .5 identical to the first aiiitional understanding impriate application, ilausneier summaril) lam: . l to Belle any respons Ltic ‘ ' .able to drscrininat :l: s A P - duster 1n the learr :eliet ‘ ‘ es that eacn variet ”“ditionl and this ' iterI‘OIe ) One ' . “Child exp :8 Cronbach, ' Lee J. {Ear-x hh 0” Instru 38 . 38 . . . . experience. Posrtive transfer of training may also be enhanced when two factors are present in a simulation: 1. Relevance of task to real office demands. 2. Situational equivalence (extent to which the simulation equivalence in the learning situation is or appears real to the learner). Fattu and Elam pr0pose a focus strategy of concept attainment they call ”scanning strategy." This strategy presents the subject with a positive instance of the concept being studied. He then examines all of the attributes of this instance as well as a second instance to see if it is identical to the first. With additional instances, the subject gains additional understanding of the concept. This theory, if put into an apprOpriate application, may enhance a theory of transfer of training. Klausmeier summarily reports that the idea that an organism can learn to make any response of which it is capable to any stimuli which it is able to discriminate is central to eXplanations of acquisition and transfer in the learning theories of Skinner and Hull. Gagne’ believes that each variety of learning begins with a different state 42 (or condition) and this leads to a different capability for performance. Therefore, one would expect two different subjects to perform differently __ 3 8Lee J. Cronbach, "Evaluation for Course Improvement," Current _R_e§earch on Instruction, ed. Richard C. Anderson et al (Englewood 371. Cliffs: Prentice-Hall, Inc., 1969), P- 39 Poland and Haines, loc. cit., p. 13. 40 Fattu and Elam, loc. cit., pp. l4-15. 41 Klausmeier, loc. cit. 42 . Robert M. Gagne’, The Conditions of Learning (New York: Holt, Rinehart and Winston, Inc., 1965), p. 59. in given situation. 31' ansition in this lay: lnorledge one has ac tie it together is k hintional theory, as i iris for applying uhatev mares in terns of an a Ogunniyi highlights lain: based on psycholc l. htheory of lear 2. Silulation repre 3. Adequacy of repr Mares the theory of l Ethos have been develop fisponse theories and cos archological importance auction. Both the medi Epnsentation are of vi Te adequacy of represen liupresentation guaran lSociolo ical Basi Ethics based upon gm“ ”Age of social groups filers. Some group int 539. sociological theor \w 4 _ 3Jerome S. Bruner. hrersity Press, 1960) r 4 . 40gunniyi, loc. Cl 39 in a given situation. Bruner acknowledges the importance of knowledge acquisition in this way: Knowledge one has acquired without sufficient structure to tie it together is knowledge that is likely to be forgotten. Motivational theory, as it may be applied to simulation, provides the basis for applying whatever the subject learns, knows, practices, and executes in terms of an actual task situation. Ogunniyi highlights three points he considers the pivot of simu- lations based on psychological rationale:44 l. A theory of learning for the design 2. Simulation representation 3. Adequacy of representation As far as the theory of learning may be depicted, he states that simu- lations have been deve10ped based on the principles of both the stimulus- response theories and cognitive theories. However, two major issues of psychological importance relate to the second point, simulation repre- sentation. Both the medium of representation and the objective of representation are of vital importance in the design of a simulation. The adequacy of representation refers to the degree to which the medium of representation guarantees the occurrence of the exPected behavior. A Sociological Basis for In-Basket Simulation. Soci010gical theories based upon group interaction have specific concern for the range of social groups and the effect of such interaction on group members. Some group interactions arise out of a work situation. There- fore, sociological theory involves a study of those social roles which 43Jerome S. Bruner, The Process of Education (Cambridge: Harvard University Press, 1960), p. 31. 44Ogunniyi, loc. cit., pp. 57-58. rise {ml the classificat 3111er to be mason-ed gnfomnce. lluIan relat significant deterninant o 'lanlll years ago.45 As lcfineuith some degree 0 laser and Fern state that rich affect individuals Theory holds that, g playing various cult Iain personality att clilates found in at behavioral outcomes tonnrtity.46 Fro: the group inter ‘itory of social interac1 titiduals differ in thf inane person will show ‘L‘cle series of events. iii!“7 postulates that " linens that people W Mtional feeling toward tvluation, is some per \——....__. ‘ ‘ 45Theodore Caplow. army, 1964). p- 4 6Sigmund Nosow and Work: Basic Books. 4 .. 7Helen I. Snyder. “it liley & Sons, Inc. ’ 40 arise from the classification of peOple by the work that they do. Work is likely to be measured in terms of the performer rather than of the performance. Human relations in the working situation have become a significant determinant of output ever since the Hawthorne study more than 20 years ago.45 As a result, industrial sociologists are able to define with some degree of accuracy behavior in the work situation. Nosow and Form state that social and cultural climates create conditions which affect individuals in patterned ways: Theory holds that, given a certain social milieu, persons playing various culturally defined roles will take on cer- tain personality attributes . . . particular types of social climates found in occupational life contribute to certain behavioral outcomes both on the job and in the wider community . 46 From the. group interaction perSpective, it is vital to examine the theory of social interaction in interpersonal relationships. Because individuals differ in the way they perceive events and rationalize them, any one person will show a consistent pattern of interpretation of a whole series of events. Heider in his theory of interperson percep- tion47 postulates that "peOple tend to personify effects of events.” This means that people will judge the observed act according to their emotional feeling toward the origin of the act. The origin, in this eXplanation, is some person or personlike force. g 45 Theodore Caplow, The Sociology of Work (New York: McGraw-Hill Book Company, 1964), p. 6. 46 . Sigmund Nosow and William H. Form, Man, Work, and Society (New York: Basic Books, Inc., 1962), p. 441. 47 .Helen I. Snyder, Contemporary Educational Psychology (New York; J°hn Wiley e Sons, Inc., 1968), p. 206. i This theory stels fr the response of an organi ablevhole rather than a desituation. The obsen ryto achieve a balance the person and approves 1 who be balanced if ] Jrn'ce versa.48 In this niche unit by the perc: bison attitudes and fen Three design coupon: hhaslet simulation: 1. The situation e 2. The roles which 3. The environment within the orga i’uuse there must be a 3 national approach may the simulation design heposition. The situa Emery-day activity 0 the in-hasket simlllat Enlanned action may be hole in the orgc'zmizati hearth have been atte Sinflation, with the maj \m “iii-a p. 207. 41 This theory stems from gestalt psychology, which emphasizes that the reSponse of an organism to a situation is a complete and unanalyz- able whole rather than a sum of the responses to specific elements in the situation. The observer of the act, in Heider's theory, will always try to achieve a balanced perception; that is, if the perceiver likes the person and approves of his act, his perception is "balanced." It may also be balanced if he dislikes the person and disapproves the act, or vice versa.48 In this theory, the person and his act are viewed as a whole unit by the perceiver and judged by the perceiver in terms of his own attitudes and feelings about the unit. Three design components complement the sociological nature of in-basket simulation: 1. The situation exemplified in the simulation 2. The roles which are simulated within the situation 3. The environment consisting of other people and groups within the organization Because there must be a model that epitomizes the roles simulated, a functional approach may be necessary so that the role elements included in the simulation design may be based on the functional requirements of the position. The situation may represent an identical one found in the everyday activity of the job that is being simulated. Another focus of the in-basket simulation is on the individual role when the response or planned action may be influenced by other individuals or groups of people in the organization or within the social environment. Those in research have been attempting to develop role competence through role simulation, with the major purpose one of assessing performance that is 481bid., p. 207. i hedonistic of the ins hon suggests that the v nnmnlent is to use re: sililation.49 [orbs50 values reco penny strong people replex and less absolut structured. Ever strong heahle to function effe in Integative Theo moths integration of Ethical, psychological, ntical approach to educ mimiological bases h this chapter. Ogunni E'Icational simulation51 S‘illations such as in—b irse six basic principl 1. Nature of Simul simulation is Z. Fidelit of Si possess physica fidelity is the a real—life mo \—__.___ 49 . Charles G. Moore, "WW Simulation 1 Polishing Company, 196 50 . Hones, loc. c1t . , 51 - Ogunniyi, 0 . Clt 42 characteristic of the incumbent of a given occupational position. hhore suggests that the way to test the degree of realism of the mndronment is to use real-life practitioners as participants in the simulation.49 Combs50 values recognition of the importance of developing inde- pendently strong people in a society that is rapidly becoming more cxmplex and less absolute, more relative, more ambiguous, and less stmumured. Ever stronger peOple must develOp if they are going to bezflfle to function effectively in this type of society. An Integrative Theory of Educational Simulation. Ogunniyi pro- pxmes the integration of four schools of simulation rationale (philo- anmical, psychological, sociological, and mechanistic) into a theo- mfiical approach to educational simulation. Philosophical, psychological, mulsociological bases have already been cited in the previous sections of Ufis chapter. Ogunniyi's theoretical principles for the design of mhmational simulation51 have merit in designing specific types of shmflations such as in—basket simulation. The following summation of these six basic principles are preposed in the Ogunniyi study: 1. Nature of Simulation: There is general agreement that simulation is a model of reality and not reality itself. 2. Fidelity of Simulation MOdel: The simulation model may possess physical or psycholOgical fidelity. Physical fidelity is the representation of essential elements of a real-life model accurately reflecting the important 49 Charles G. Mbore, "Simulation of Organizational Decision-Making: Afiqvey," Simulation in the Study of Politics (Chicago: Markham Hflflishlng Company, 1968), p. 195- 50 Hewes, 10c. cit., p. 87. 51 Ogunniyi, OE. cit., pp. 91-98. i, i characteristics Psychological f an individual V‘ 3... Simlation and l sary. The beha in the simlate of providing cl and requires in Iachines, or pe . Ob'ectives of E in the design 0 tion of the lea provide. In de of any sinulati aspects of the .... u- Representations lodel and the 0 choice of symbo simulation desi poses of a good 9 Steps in Design evidence in the any clear‘cut S validating eduC from also proposed t allowing six design que a». ~ - ‘ 52 ...luanve criteria: ,_. . Does the desig special or gen . Does the desi . Does the desig for representat situations that WM 43 characteristics of the real situation being simulated. Psychological fidelity is a mental process which leaves an individual with a feeling of "realness.” Simulation and Game: In simulation the actual presence of individuals other than the participant is not neces— sary. The behavior of the components is taken as given in the simulated situation. A game, however, is a means of providing clinical experience about a model's behavior and requires interaction between people, people and machines, or people and the environment. gbjectives of Educational Simulation: An important stage in the design of educational simulation is the specifica- tion of the learning objectives that the simulation will provide. In determining the parameters of the objectives of any simulation, attention needs to be given to specific aspects of the real—life model. Representations in Simulation Designs: The nature of the model and the objectives of the simulation dictate the choice of symbolic representation of the model. The simulation design may integrate several media for pur- poses of a good design. Steps in Designing Educational Simulation: There is no evidence in the literature to reflect the existence of any clear—cut systematic guidelines for designing and validating educational simulations. Ogunniyi also proposed that all educational simulations answer the following six design questions and form the basis for simulation design . . . 52 evaluative criteria: 1. Does the design represent a real—life model? What kind—- sPecial or general? 2- Does the design serve specified functions? Which? . 3. Does the design employ suitable synthetic or anlmated media for representation and integration of selected operational situations that characterize the model? Which? ‘ 4- Does the design provide the sequential context of real—life active or interactive processes within the system model? In what way? 5- Does the design provide appropriate feedbacks to the learner in terms of stated learning experiences simulated? How? 6- DOes the design provide for evaluation and control of learn— ing that it offers in terms of the real-life model? M 52 Ibid., pp. 100-101. theoretical concl rd] relates simlat ion lhe evalmtion of a1 simlated evaluatio the hovledge that not be realistical standards. This is khanistic theory for; to draw upects o fieries together in a c “in: With computers 35in with control, the agitation so that each ésip: 1- Conpression of ‘ is being evalua' limitation of t of individuals prompt decision Individualizatil by bringing him situations . 3x; 5.: - 54 “55 and Shulman55 us tices in their simulat life(lees not permit the may as to permit : \_———— .3 33111., p. 113. 54 :- Bert Y. Kersh, Cl Minn, Project No. 8 Mb: Teaching Res aation, June, 1963). “Shulman, Loupe, 44 A theoretical conclusion of primary significance to the present study relates simulation to the evaluative process: The evaluation of any simulated instruction needs to be a simulated evaluation. That is to say, the evaluation of the knowledge that has come out of the simulated learning must be realistically evaluated in terms of real-life 53 standards. This is often overlooked in simulated instruction. Mechanistic theory is often seen by researchers as the "tie" nec- essary to draw aspects of phiIOSOphical, psychological, and sociological theories together in a common bond and explain the elements of control theory. With computers or less sophisticated mechanical devices to assist with control, the researcher has the Opportunity to stage the simulation so that each of the following elements are present in the design: 1. Compression of time when only a single training objective is being evaluated. 2. Limitation of time in which a given individual or group of individuals is permitted to perform, thus encouraging prompt decision making. 3. Individualization of the experience for the participant by bringing him in contact with people, processes, or situations. 54 SS . . . . . Kersh and Shulman used staging procedures 1nvolv1ng mechanistic devices in their simulation research. Observation of behavior in real life does not permit the researcher to control conditions or events in such a way as to permit him to attribute variation in performance to 53Ibid., p. 113. S4Bert Y. Kersh, Classroom Simulation: A New Dimension in Teacher Education, Project No. 886, U. S. Office of Education, NDEA Title VII lMonmouth: Teaching Research Division, Oregon State System of Higher Education, June, 1963). 55Shulman, Loupe, and Piper, op. cit. irrational conditions. histamine those facto hum: of Individua Evaluation is used . 57 nuance: 1. Identifying the his instructio Judging the me selection and g 3. Acquainting the deficiencies. ; as. . int indicates the a “toss dolinates the me leaned. He endorses th testament of achieveme inlaan describes evalu i‘éitidual student score inlitre judged to be sat Estahen place.59 Q1185 mess: \—-.______ filmderiksen, Jen S? - Cronbach, IOC- C _ ”trauma. Linde um and Compam’» 1 situational conditions. Utilization of the mechanistic theory gives him the ability to assign subjects to treatments according to a pre- arranged plan. Consequently, he can correlate behavior with situations to determine those factors attributable to situational changes.56 Measurement of Individual Performance in In-Basket Simulation Evaluation is used in order to make decisions about an individual's 57 performance: 1. Identifying the needs of the individual in order to plan his instructional program. 2. Judging the merit of the individual for purposes of selection and grouping. 3. Acquainting the individual with his own progress and deficiencies. Horstl58 indicates the achievement (proficiency) domain of the evaluative process dominates the measurement of what the individual has actually learned. He endorses the job sample test as a vehicle in the accurate measurement of achievement and proficiency in given vocational traits. Lindeman describes evaluation as more inclusive than measurement. When individual student scores are compared with other scores in the group and are judged to be satisfactory or unsatisfactory, then an evaluation has taken place.59 Questions such as these arise during the evaluative process: 5 . 6Frederiksen, Jensen, and Beaton, loc. c1t. S7 Cronbach, loc. cit., p. 362. Paul Horst, Psychological Measurement and Prediction (Belmont: Wadsworth Publishing Company, Inc., 1966), p. 29. 9 . . Richard H. Lindeman, Educational Measurement (GlenV1ew: Scott, Foresman and Company, 1967), p. 12. 1. lion does actual 2. How is a "score 3. hhat meaning is hatesting situation, ulbeing tested; and ot dosen sample of an indi theory vhich accounts f< tens of variations in r cepts hhich include deci ation, and processing. frequently used method 4 ability to perform some tout something). Howe‘ ioral changes should al: ationplan:61 1. Oranges in att: 3. Changes in awa: 3. Changes in not: Fattu rs in Measurement l Standardization, tl factors involved in the lftheir importance in ‘ \\ so . ,J0hn K. Hemphill l u “Mame Performer: iaSIWJlated E1 ).p.6 2 . 011, W F 46 1. How does actual measurement occur? 2. How is a "score” obtained? 3. What meaning is attached to a "score"? In a testing situation, the single independent variable is the individ- ual being tested; and observations are made upon a small but carefully chosen sample of an individual's behavior. Hemphill EE.2$FO purport a theory which accounts for differences in administrative behavior in terms of variations in decision-making behavior utilizing a set of con- cepts which include decision making, organization, perception, communi- cation, and processing. 2011 states that in training programs the most frequently used method of evaluation measures changes in skills (the ability to perform some act) and changes in Specific knowledge (ideas about something). However, he feels that three other types of behav- ioral changes should also be measured as a part of the terminal evalu- ation plan:61 1. Changes in attitudes 2. Changes in awareness of self 3. Changes in motivation to perform Factors in Measurement of Individual Differences Standardization, test reliability, and test validity are three factors involved in the measurement of individual differences. Because of their importance in the deve10pment of all evaluative instruments, a 60John K. Hemphill, Daniel E. Griffiths, and Norman Frederiksen, Administrative Performance and Personality: A Study of the Principal in a Simulated Elementary School (New York: Teachers College, Columbia University, 1962), p. 6. 612011, Qynamic Management Education, p. 1. inf explanation is vi Sign and developent Standardization. moms in research p criterion. Every detai ilsdjects tested if nits of procedure have ... Each subject it given to anoth Exact time 1i Instmctions Im fashion. . Preliminary ini sented in exact . Methods for bat administration . Each subject's with the score: you; 0-- .u. as Standardization imp tin. test administratit Estbe in accord with t Evillasis of appropriat "Jifunity in test admi identical materials, ti Flietemined "pass" or l'ecolpared only with islandard scoring pro :em for Hemphill 333i filing.63 Both the con 6 2iiliastasi, o . 63Hemphill et 81: 47 brflfi3explanation is vital in delineating how each is involved in the design and development of in—basket simulations. Standardization. In the development of evaluative instruments, exactness in researCh procedures will result in the establishment of a cuiterion. Every detail of the testing situation must be identical for zfll subjects tested if any form of standardization is to occur. Several :mfles of procedure have been adopted by researchers:62 1. Each subject will be given identical materials to those given to another subject. Exact time limits must be established. 3. Instructions must be presented to subjects in identical fashion. 4. Preliminary information and demonstrations must be pre— sented in exactly the same way to all subjects. 5. Methods for handling subjects' questions during the administration of the test must be identical. 6. Each subject's score is evaluated only by comparing it with the scores obtained by other subjects. ' N Standardization implies uniformity of procedure in test construc— tion, test administration, and scoring procedures. Test construction Immt be in accord with detailed specifications and items selected on the basis of appropriateness of difficulty and discriminating power. lhflformity in test administration requires that all subjects confront identical materials, time elements, and evaluative techniques. With no {undetermined "pass" or ”fail" score, the individual subject's responses amecompared only with others taking an identical test. The development (fifstandard scoring procedures for in-basket simulation was a prime con- cenrfor Hemphill eg_al_in their research of administrative decision— making.63 Both the content of the perfbrmance (what is done) and the 62 Anastasi, o . cit., pp. 23—24. 63 Hemphill et a1, 0p, cit., pp. 86-87. 3301‘ the perforlanc wring categories for W to the evaluat if guidelines for const tie eraluation was obse l. Problems shoul the same criti list he presen 5" Items within a order of their .9" The use of sec for checking s for judgmental ‘- Scoring is per on a continuum performance. Standardization of an e‘ i’fine behavioral changi Mp as the standard 0: atented market—place 5' Test Reliability. Minn in judging the “Presents the consiste minded to measure.67 inill approximately th \___ 64 Ed... p. 87. 65 . leislogal, loc. 48 style of the performance (how it is done) are incorporated into the sunflng categories for the Hemphill in-basket, indicating an analytical annpach to the evaluation of responses.64 WeiSIOgal developed a set (HFguidelines for constructing and scoring situational tests in which the evaluation was Observer—based:65 1. Problems should be structured so that each subject faces the same critical situation. The same need for action must be present for all participants. 2. Items within a problem should be arranged in the probable order of their occurrence. 3. The use of scoring techniques based upon objective criteria for checking Specific behaviors should minimize the need fOr judgmental decisions on the part of the observers. 4. Scoring is perhaps easiest when behaviors can be identified on a continuum of effective performance vs. ineffective performance. Standardization of an evaluative process enables the researcher to defhmabehavioral change in terms of a norm—reference (the norm of a gnoup as the standard of evaluation) or a criterion-reference (the . 6 accepted market—place standard of evaluation). lest Reliability. As one of three criteria found useful by ldndeman in judging the quality of a measurement device, reliability Itpresents the consistency with which a test measures what it is :hmended to measure.67 Will the same person under similar conditions tfimain approximately the same score when taking the test at a different *— 6 4Ibid., p. 87. 6S . WeisIOgal, loc. cit., pp. 496-497. 66 . . .. E. Lowell Kelly, "The Place of Situatlon Tests in Evaluating (llnical Psych010gists," Personnel Psychology, 1954, No. 7), p. 485. 67 . . Lindeman, op. c1t., p. 43. tilt in the situati typical“ situations f0 gathered! Itu conten aims are factors us mt from one tile ment to uhich individ hehance errors of me mile to true differen Moan identifies reli hen one is assessing ialidity."69 Hulphreys indicates lnitrite the most relial treasured. A high deg filming the focus of 1 reliability, then, is a any.” Hertphill et ticket score would it minim set of items iiility estimates in th V 68 Anastasi, 0p. ci 69 *l lees. Shulman, inauonal Research 70 . Lloyd G. Humphre mt Test Theory, \, it: “look of in. 1967), p. 37. ll lentil, °L°i 49 time? Are the situations selected an appropriate representation of the "typical" situations for which the individual's performance is to be predicted? Item content, difficulty of items, and discriminatory power of items are factors used in determining the consistency of test mea- surement from one time to another. Test reliability refers to the extent to which individual differences in test scores are attributable to chance errors of measurement and the extent to which they are attrib- utable to true differences in the characteristics under consideration.68 Shulman identifies reliabilig and internal validity as synonymous terms: "When one is assessing a test's reliability, he is measuring its internal validity."69 Humphreys indicates that one of the steps in test construction is to write the most reliable items one possibly can for the function to be measured. A high degree of test reliability can be achieved by narrowing the focus of the test and attaining high homogeneity. Test reliability, then, is a function of both item reliability and homo— geneity.70 Hemphill 3331 concluded that the meaning of a category in-basket score would not change significantly with changes in the particular set of items from which it was obtained. However, reli- ability estimates in their study reflected three sources of unreliability:71 68 Anastasi, op. cit., p. 106. 69Lee S. Shulman, "Reconstruction of Educational Research," Review _0_'I:Educational Research, Volume 40, Number 3 (June, 1970), p. 379. 70Lloyd G. Humphreys, "The Normal Curve and the Attenuation Paradox in Test Theory," Principles of Educational and EychOIOgical Mgasurement: A Book of Selected Reading (Chicago: Rand McNally? Company, 1967), p. 37. 7 1Hemphill, op. cit., p. 125. l. m reliabilit 2. The imreliabil subjects' beha was noticeable y. . Any general di sets of items Flanagan characterizes ingappmach to evalua litany chance factor tional testing indica required to obtain rel" ferobtainiug the indep reliability. These met atuivalent forms, split lite coefficients.73 '[ all tiles because of o l. Alonger test . Atest compos reliable than . Atest compos more reliable nating items. . Atest whose ' reliable than or quite easy \‘N n.) w 4:- 72 . Fluagan, "Some his," p, 463. ii Eh“. 0p. cit., n . iii. in. 336. l. The reliability of the scorer would be attenuated by any lack of agreement among the eight scorers as to how the scoring categories applied to the responses of the principals. 2. The unreliability introduced by inconsistencies in the subjects' behaviors from item to item in the in—basket was noticeable. 3. Any general difference between the odd— and even-numbered sets of items would reduce the reliability of the cate- gory scores. Idanagan characterizes one of the shortcomings of the situational test- ing approach to evaluation as time required to obtain a reliable score. The many chance factors that enter into behaviors represented in situ- ational testing indicate that a relatively long period of time may be required to obtain reliability.72 At least five methods have been used forefinaining the independent measures necessary for estimating test reliability. These methods yield reader reliability, test-retest, mndvalent forms, split halves, Kuder-Richardson, or analysis of vari- ancecoefficients.73 Typically, improved test reliability is obtained many times because of other test and group characteristics:74 1. A longer test may be more reliable than a shorter test. 2. A test composed of more homogeneous items may be more reliable than a more heterogeneous test. 3. A test composed of more discriminating items may be more reliable than a test composed of less discrimi- nating items. 4. A test whose items are of middle difficulty may be more reliable than a test composed mainly of quite difficult or quite easy items. 2 . . . . Flanagan, "Some Con51derations in the Development of Situation Tests," p. 463. 73Ebel, op. cit., p. 312. 74. Ib1d., p. 336. 5_ A group having reliable than 6. Aspeeded test all subjects c le. lest reliability, iiic'nasubject will re sailor mm. In che ithat the same indivi untimely the same border for a test to torment from one time Test Validity. Eh fiat content are the tea In is designed to mea dill or knowledge (or existence of valid conc fielder for understand efforts to change behav Iieternining the qual l. The importance the test score 2. The meaningful 3. The convenienc Four types of Van fin-basket simulation 2mm validity. and F \_———— ”Robert L- Ebel’ inactional and PS °h°1 .dicago: Rand MliiNally 51 5. A group having a wide range of ability may be more reliable than a group of more homogeneous ability. 6. A speeded test may be more reliable than one in which all subjects can complete the test in the time avail— able. Test reliability, then, is dependent upon the consistency with which a subject will reSpond to similar environmental situations in a similar manner. In checking for reliability, the researcher's concern is that the same individual under similar conditions would obtain approximately the same score when taking the test at a different time. In order for a test to be reliable, there must be consistency of mea— surement from one time to another. Test Validity. What the test measures and how well it measures that content are the two aims of test validity. An evaluative instru— mmn is designed to measure how well the subject has mastered a specific skill or knowledge (or accumulations of skills and knowledges). The existence of valid concepts within the testing structure is essential htorder for understanding of the test results, analysis of data, or efforts to change behavior to take place. Ebel regards three elements - . . . 75 Uldetermining the quality of a measurement procedure: 1. The importance of the inferences that can be made from the test scores. 2. The meaningfulness of the test scores. 3. The convenience of the test in use. Four types of validity have operational value in the deve10pment Ofin-basket simulation: content validity, construct validity, con— current validity, and predictive validity. One other type of validity, -__~____________ 75Robert L. Ebel, ”Must All Tests Be Valid?" Princi les of Educational and Psychological Measurement: A Book of selesfffii_gfiééiflg§ (Chicago: Rand McNally a Company, 1967), p. 230. Paudidity, is really 'i that the test superfi he" to bear a comm-s npnpriate items and t( ates that there is obvi eriaution of the four iii: inortance in the .... Content Validit reSponses to tl than upon the i expert review a specialist wit} 1; Construct Valia m to which the to trait.79 Io: Concurrent \-‘ali m relation betwea measure obtaina tion by method a. . Predictive Vali predicting some the dimensions future job peri \___._. ‘ yshnastasi, M ‘ ”gin, p. 136. Tsiindeman, M 79Anastasi, _12_C_~_£ s . 0Lindeman, 02. C11 } SIAnastasi, $9542 33 933., p. 138. 83Lopez. lee-.932 52 face validity, is really a part of content validity because it refers to what the test superficially measures.76 The items appear "on their face" to bear a common-sense relationship to the total universe of appr0priate items and to the objective of the test. Face validity indi- cates that there is obvious foundation in real-life situations. A brief explanation of the four basic types of validity will assist in relating their importance in the deve10pment of in-basket simulation: 1. Content Validity: The relevance of the individual's test responses to the behavior area under consideration rather than upon the apparent relevance of item content;77 the expert review of the instrument by a subject-matter specialist with the expertise of the test-designer. 2. Construct Validity: The extent to which a test tells about the achievement of the individual; the extent to which the test measures a theoretical construct or trait.79 3. Concurrent Validity: Relationship between test perfor- mance and an accepted contemporary criterion;80 the relation between test scores and indices of criterion measure obtained at approximately the same time (valida- tion by method of contrasted groups).8 4. Predictive Validity: The effectiveness of the test in predicting some fUture outcome;82 the significance of the dimensions of in-basket performance compared with fUture job performance. 76Anastasi, 0p. cit., p. 138. 77Ibid., p. 136. 78Lindeman, op. cit., p. 137. 79Anastasi, loc. cit., p. 145. 80Lindeman, 0p. cit., p. 37. 81Anastasi, loc. cit., p. 141. 8 21bid., p. 138. 3Lapez, loc. cit. lhe validity of si was between perfo lftetan iii-basket test iish relationships betu htea‘ situation without ingreal on-the-job beh fwd in sole past stud Lise tIo deficiencies: l. The pertinent 2. A discrepancy elicited first actual job. harder for an in-bask list not only appear to thejoh performance but 5mm these traits an lien iii-basket simulat If testing in specific iisures requires the u fatal groups in order Momma, both on t Basic theories rel Si-llltttion require the g“... _ 84h. Ray Cross, "R flasket Performance 31 “hational Research, V 8 . 5Fiske, 10c. (:11:- 53 The validity of situational tests is proportional to the corre- Spondence between performance on the test and performance on the job. After an in—basket test has been validated, one may proceed to estab— lish relationships between other variables and performance in a simu— lated situation without the necessity for costly procedures for measur— ing real on-the—job behavior.84 Fiske believes that the low validities found in some past studies of situational testing may be the result of these two deficiencies:85 1. The pertinent trait may have been inaccurately identified. 2. A discrepancy existed between the relative effectiveness elicited first in the situational tests and later in the actual job. htorder for an in-basket simulation to be a valid representation, it must not only appear to measure the traits essential for adequate on- the~job performance but it must also prove that a relationship exists between these traits and on—the—job performance. The predictive ability of an in—basket simulation is one of the purposes in utilizing this form of testing in specific training programs. The establishment of validity nwasures requires the utilization of both criterion groups and experi— mental groups in order to establish apprOpriate relationships between perflumances, both on the test and on the job. C. Summary Basic theories relevant to the design and development of in—basket simulation require the integration of sound and innovative concepts from M 4W. Ray Cr055, ”Relationships Between Elementary School Principals! thasket Performance and Their On-the-Job Behavior," The Journal of Emmational Research, Volume 63, Number 1 (September, l§6§j:—pT—§ET_ 85 . Fiske, 10c. cit., p. 469. is real business world £101 in the classroom e this dapter represent c: sociological premise Moment, developmen tiindividual perfoman ‘11 principles which pr lihll Iii-BASKET in thi scented in the procec‘ 5M input for the de 1. r9 5,.- .5. m 9 Basic Approach thesis approac development 01 tests of typi: Philosophical basket simulat a situational to bridge the conventional ( Psychological stimulus—respc elements theor in-basket simt occurs and not . Sociological '1 development 01 a given occupe social role a: study with spa: required in t? Integrated The integrated 8P1 essential in C psychological a mechanistic t? W measurement 01 skills and knc 54 the real business world with meaningful and relevant methods of instruc- tion in the classroom environment. The design theories presented in this chapter represent given aspects of philosophical, psychological, or sociological premises in their relationships to approaches to test development, development of actual in-basket simulation, and measurement of individual performance in such in-basket simulations. Those theoreti- cal principles which provide a basis for the deve10pment of THE SECRE- TARIAL IN-BASKET in this study and which lead to the empirical methodology presented in the procedures for the study are summarized here to provide direct input for the design of the in-basket simulation: 1. Basic Approach to Test Deve10pment: The rational hypo- thesis approach provides the basic approach to the deve10pment of in-basket simulation because it produces tests of typical performance in real-life situations. 2. Philosophical Theory for In-Basket Development: In- basket simulation must represent as closely as possible a situational excerpt from the real world in its attempt to bridge the gap between the business world and the conventional classroom. 3. Psychological Theory for In- Basket Development: The stimulus- -re5ponse- -feedback theory and the identical elements theory are prominent in the deve10pment of in-basket simulation in order that transfer of training occurs and motivation for learning enhanced. 4. Sociological Theory for In-Basket Development: The development of an in-basket simulation authenticating a given occupational role requires a study of the social role arising from the job classification under study with specific relevance to the human relations required in the work to be performed on the job. 5. Integrated Theory for In- Basket Develppment: An integratedapproach to in-basket simulation appears essential in order to tie the foregoing philosophical, psychological, and sociological elements to the mechanistic theory essential for control in design. 6. Measurement of Individual Perfbrmance: Accurate measurement Owahat the indiVidual has learned (Specific skills and knowledges, behavioral changes in attitude, awareness of . evaluating ant performance i1 ‘- Standardizatil the testing 5 tested if any 9: Maintenance 0 the consisten measures what to another. :9 Establishment simulation id is proportion performance 0: These theoretical Std in the deve10pmen‘ Eittle published on th‘ fire, this smnmation p :fintiples that are ap; tion and should provid Epetts of in—basket st awareness of self, and motivation) is essential in evaluating and making decisions about that individual's performance in the test. 7. Standardization of Testing Procedures: Every detail of the testing situation must be identical for all subjects tested if any form of standardization is to occur. 8. Maintenance of Test Reliability: Test reliability is the consistency with which the in—basket simulation measures what it is intended to measure from one time to another. 9. Establishment of Test Validity: The validity of in-basket simulation (Content, construct, concurrent, and predictive) is proportional to the degree of correspondence between performance on the in-basket and performance on the job. These theoretical principles form the basis for the methodology used in the development of THE SECRETARIAL IN—BASKET. There has been little published on the actual design of in-basket simulation; there- fore, this summation provides a rather comprehensive analysis of those principles that are applicable to the development of in-basket simula— tion and should provide an informational base for the operational aspects of in—basket simulation for others. HISTORICAL In-basket simulati 13g, and secretarial e< testing procedure. A 1 Easter simulation in ti seat the in-basket t4 acceptance of the in-b; tin, medical educatior teen dependent in a 133 mil . ‘ ..tetelopers 1n devil A human Frederiksei noted with being the i: In ' lhlng evaluation ia-ha tskets and researcl .iirF ' Orte1n~baskets wt iii'fi‘dtb r lie behavior (Fre Etifs' nths and Frederik' .ifs en d Beaton, 1962 CHAPTER III HISTORICAL DEVELOPMENT OF IN—BASKET SIMULATION In-basket simulation has been used in research, management train- ing, and secretarial education as an instructional, training, and/or testing procedure. A brief look at the historical development of in- basket simulation in these areas indicates the growing interest in the use of the in-basket technique, especially since the 1950's. The acceptance of the in-basket technique in such fields as teacher educa- tion, medical education, and management training and development has been dependent in a large part on the interest shown by researchers and deve10pers in devising procedural designs for in-basket simulations. A. The In-Basket in Research Norman Frederiksen of the Educational Testing Service (ETS) is credited with being the originator of the in-basket test and its use in training evaluation. ETS has been involved in the deve10pment of in-baskets and research pertaining to the use of in-baskets since the Air Force in-baskets were deve10ped in 1957. Subsequent research in executive behavior (Frederiksen, 1962); school administration (Hemphill, Griffiths and Frederiksen, 1962); and organizational climates (Frederiksen, Jensen and Beaton, 1968) have resulted in major ETS studies involving the in-basket as the research instrument for comparing simulated behavior with on-the-job behavior. 56 FM.- .1..- Alt.- ArIIU S7 In-basket research conducted at Michigan State University has focused in the area of teacher education where the in-basket test was used in the study of teacher behavior (Shulman, Loupe and Piper, 1968) and in the study of student behavior (Poland and Haines, 1970). In medical education the in-basket technique has been used in the study «of patient management (Rimoldi, 1963; Helfer and Slater; McGuire, 1967; Elstein, 1971). Brief reviews of these research investigations will emphasize the growing importance of the in-basket as a research tool. act-- The Air Force Study rm. .. The Officer Education Research Laboratory of the Air Force Personnel and Training Research Center contracted with ETS to study the desired outcomes of training in the Command and Staff School of the Air University in 1957.1 A careful study of the curriculum of the Command and Staff School and of statements by instructors about changes in performance they hOped to produce in their students yielded a classification of six behaviors that were primarily individual (exhibited by a person working alone) and six behaviors primarily interactive (involved interrelation- ships with other pe0ple).2 The prototype instrument had to be a test that would tend to elicit the relevant kinds of behavior in a situation that resembled as closely as possible the real job of an Air Force officer. Analysis of the tasks performed by an officer revealed that the majority of the desk work centered around the contents of one of 1Norman Frederiksen, D. R. Saunders, and Barbara Wand, "The In-Basket Test," Psychological Monographs: General and Applied, Volume 71, Number 9 ”9575, p. 1. 2Frederiksen, Jensen, and Beaton, op. cit., p. 30. the two baskets ("inn 5 riir Force officer- snnal test that requj it-basket. Several criticism! .111 Force study.3 Air :ttaiwe actions on the irfonation was availal :ated in the evaluatior 2:“ when situational resulted in the develop inanew position becat prior to the start of 1 “PM My applicable The in~bashet test iilForce Base requirec finding Officer of e ‘iitriel; Director of i :19 he was given an ir staff - studtes, and othe Frederiksen et a \ 58 the two baskets ("in" and "out" baskets) usually found on the desk of an Air Force officer. The format of the instrument involved a situ- ational test that required the subject to deal with items in the in-basket. Several criticisms resulted from the first in-basket used in the Air Force study.3 Air Force officers stated that it was not reasonable to take actions on the items in the in-basket test because too little information was available to the subject. The "correct" action desig- nated in the evaluation scheme for the in-basket depended upon a number of unknown situational factors. A compromise in the test structure resulted in the development of an in-basket that placed the subject in a new position because of an emergency and allowed a period of time prior to the start of the in-basket exercise during which the subject would study applicable background materials. The in-basket tests that were prepared and tried out at Maxwell Air Force Base required the subject to play four roles in succession: Commanding Officer of a hypothetical 7lst Composite Wing; Director of Materiel; Director of Personnel; and Director of Operations. In each role he was given an in-basket containing incoming letters, memoranda, staff studies, and other similar material. Frederiksen p_t_a_1_ report that the Air Force in-baskets were dis- appointing from the standpoint of their psychometric properties. In subsequent studies different methods of scoring were employed that yield qu1te sat1sfactory reliab111t1es for many scor1ng categor1es. 31bid., p. 31. “ 4Ibid., p. 32. -—— he Bureau of Business The Bureau of BUS rude, records of tele fut-basket of a newly-h hu‘siou of the Bureau lation of some major a requested to actually he letters, memoranda Pleusinulation of an Behavior which occurs The purpose of th seething about the ma Siulated lOb-S These :c ' ‘ . Intermediate criter :Bie ction tests. The i‘iith would refleCt Pe he developed for 590 iiznistration of the Cilidradua tes) gradUa 73in . ess ehecutives a 1 59 The Bureau of Business In-Basket Study The Bureau of Business In-Basket Test consists of letters, memo- randa, records of telephone calls, and other business papers from the in-basket of a newly-hired Executive Officer of the Northeastern Division of the Bureau of Business. The in-basket represents a simu- lation of some major aspects of an executive's job, and the subject is requested to actually perform the tasks contained in the in—basket simulation as he would if he were actually employed in the position. The letters, memoranda, and notes written by the subject in this com- plex simulation of an executive's position constitute a record of behavior which occurs spontaneously in a rather unstructured situation. The purpose of the Bureau of Business In-Basket Study was to learn something about the major dimensions of behavior which occur in the simulated job.S These behavioral dimensions are of interest as well as intermediate criteria for studying a variety of potentially useful selection tests. The in-basket test was considered a performance test which would reflect personality of the individual subjects, and methods ‘were~developed fbr scoring 70 aspects of behavioral dimensions. The zuhministration of the test to 335 pe0p1e representing subgroups of undergraduates, graduate students of business, government administrators, business executives, and army officers resulted in a study of the major differences between these groups as revealed by mean scores on in-basket categories. A factor analysis of 40 of the most reliable in-basket . . 6 category scores revealed eight primary factors: sNorman Frederiksen, ”Factors in In-Basket Performance," Psychological Monographs: General and Applied, Volume 76, Number 22, ole No. 41 1962 , p. 24. 61bid., p. 25 s ‘ Lr-I C g D—‘- ;‘ - - (J I u .365 SB Eehar'io: The 35 Emir iieu-eutz esearc} Escause 60 Acting in compliance with suggestions Preparing for action by becoming informed Concern with public relations Procrastinating Concern with superiors Informality Directing subordinates Discussing WNO‘U'i-bMNO-d Three second-order factors were defined in terms of the administrative behavior represented by each:7 1. Preparing fbr action 2. Amount of work 3 Seeking guidance The Hemphill study of school administration utilized the Bureau of Business In-Basket Test as one of four in-baskets administered to elementary school principals. One major difficulty facing personality research has been that of making suitable real—life observations because not every real-life situation will provide opportunities for the expression of any or all personality tendencies. Therefore, the behavioral dimensions obtained from these in-basket studies relate subject perfOrmance on a series of situational in-baskets. The School Administration In-Basket Study During the ten-year period from 1948 to 1958 the focus of theo- retical discussions began to shift from emphasis on traits of adminis- trators to emphasis on the behavior of administrators in school situations.8 An interest also developed in an actual analysis of the adumtnistrative situation, and the Hemphill et al study had as its main 71bid. 8Hemphill, Griffiths, and Frederiksen, Op. cit., p. 3. ll 61 focus the investigation of major dimensions of administrative perfor— mance. Three major objectives of the study were identified:9 1. The Development of Concepts: To determine dimensions of performance in the elementary school principals and thus to develop a better understanding of the nature of the job of the school administrator. The Determination of Measurable Dimensions of Performance: To provide information helpful in the solution of the problem of selecting school administrators. The Development of Materials: To provide materials and instruments for the study and teaching of school adminis— tration. The purpose of the Hemphill study was to try to identify some major dimensions of administrative performance and to investigate the relation— ships of these major dimensions to a variety of other measurable charac— teristics of people.10 Subjects were 232 elementary school principals from a wide variety of elementary schools in the United States. The background materials in the school administration in-baskets were much more elaborate than for the Air Force study. A day and a half of the five—day testing period was spent by subjects in learning about a simulated elementary school and the community in which it was located. Training materials included filmstrips, motion picture films, and tape recordings of several school events. At the end of the orien— tation period, the subjects had as much information as could reasonably be expected of a new principal in an actual situation. Four in—baskets were presented, one of which was the Bureau of Business In—Basket. The other three represented the paperwork of the principal on three specific days Of the school year. ..._______________ 91bid., p. 7. 10mm. 'e"*“" 7 7» v—__ ~_--, _ . ;,,,.———J4 . n \- ... 0": 62 Scoring methods developed for scoring the school administration in-baskets and the Bureau of Business In—Basket were simpler and more direct than those employed in the Air Force study.11 In general, the scores proved to be of satisfactory reliability. The Air Force study and the school administration study differed in another important way. The Air Force in—baskets Were prepared specifically to elicit certain pre-selected kinds of behavior judged relevant to the purposes of a training program. The school administration study, on the other hand, began with the question of how to identify some important aspects of behavior in administration. This approach led to a quite different approach in the development of testing materials. Eight primary factors resulting from a factor analysis accounted for most of the common variance in the scores obtained in the school . . . 12 administration study: Exchanging information Discussing before acting Complying with suggestions made by others Analyzing the situation Maintaining organizational relationships Organizing work Responding to outsiders Directing the work of others A second—order factor analysis of the intercorrelations of the . . . 13 eight primary factors yielded two factors: 1. Amount of work done in handling items 2. Preparation for decision vs. taking final action _~______________ 1Frederiksen, Jensen, and Beaton, loc. cit., p. 34. 12 lbid., p. 36. ”ma, p. 37. ()3 The Hemphill study also involved administering to the 232 elemen- tary school principals a large battery of tests, inventories, and questionnaires in order to investigate the relationships of cognitive abilities, personality, and biographical information to the factors in school administration. A number of relationships were feund that appeared to show some consistency in performance across a domain that includes ability tests, personality inventories, and ratings as well as performance in the simulated school. The Organizational Climates Study Frederiksen, Jensen, and Beaton conducted an experimental study which involved a simulated organization with specific modifications of the organizational climates.14 These organizational climates were communicated in both subtle and overt ways, and the subject's perceptions of the climate were reinforced during the test by including suitable materials in the in-basket itself.15 The two main purposes of this study were: 1. To investigate the effects of organizational climates on the means and on the factor structure of the dependent variables. 2. To investigate the effects of organizational climates on the correlations between predictor measures and the dependent variables .. the possible role of situational variables (the organizational climates) as moderators of relationships of predictors to measures of perfor- mance in an organization. 141bid. 15Ibid., p. 10. 16Ibid., p. 21. 64 Most of the experimental data were obtained at a two-day Research Institute in Sacramento, California, in April, 1965. The subjects in the experiment were 260 male executives employed by the State of Chlifbrnia in a variety of positions ranging from forestry to prison service and from heads of departments to middle-management levels. The shmflated job was that of Chief of the Field Service Division of the Impartment of Commerce for the State of California, a job that required little technical skill or information beyond that already known to all subjects because of their experience as state employees. The exPeri— nwntal treatments involved two dichotomies of organizational climate: 1. A climate in which innovation and originality are encouraged vs. a climate that encourages following rules and standard procedures. 2. A climate in which a type of supervision was utilized wherein the supervisor was expected to monitor the details of the work of a subordinate vs. a global type of supervision in which the SUpervisor was mainly con— cerned with assigning work and evaluating the final products of the subordinate's work. The perceptions of the organizational climates were enhanced throughout the eXperiment by the documents included in each subject's in-basket. (me of the implications derived from the use of the in-basket in this research was that to some degree subjects perform differently in a situational test than they do in a real job, knowing as they do that dmflsions are not "real” and that no one's career would really be influenced by any action taken. Another observation, the researchers noted, concerned the degree of accuracy to which performance in a shmflated job is representative of one's typical behavior. __ 17Ibid., p. 336. 65 Ten factors accounted for 61 per cent of the total variance. Scores representing these ten performance factors served as dependent variables in the investigation of effects of organizational climates on performance. These performance factors are as follows:1 Productivity Acts in compliance with suggestions Interacts with superiors Thoughtful analysis of problems Plans and discusses Defers judgment and action Interacts with peers Orderly work Informality Accepts administrative responsibility C‘OQNO‘U‘kWNI—l ...: An additional variable, the average of the in-basket scorers' rating of overall quality of performance, constitutes the eleventh criterion. The criteria of performance used in studying effects of organizational climates are the ten in-basket performance factor scores and the average rating of quality of performance by the in-basket scorers. One further recommendation made by Frederiksen, Jensen, and Beaton involved the selection of a test battery for the selection of managers. These two considerations are of prime importance in selecting appropri- . 19 ate testing packages: 1. What criteria of performance are considered important? 2. In what kind of situation or organization will the candidate work? .Prederiksen, Jensen, and Beaton indicate that the variations in settings, experimental treatments, and dependent variables that might be employed iJl such social-psychological experiments are limited only by the experi- menter's imagination. The use of a complex situational test as a device \ 18Ibid., pp. 338-339. 191bid., p. 352. ()6 for performing experiments in the behavioral sciences provides advantages over field studies from the standpoint of eXperimental controls; and an implication derived from this study focuses on the feasibility of using . . . . . . . 2 the Simulated technique more extenSively as an investigative technique. 0 The Shulman Study Shulman g£_al_deve10ped the Teacher's In-Basket in an attempt to study individual inquiry behavior in a situation involving a female elementarx_school teacher-in-training as the principal subject.21 The ‘B situation involved her in the role of a new sixth grade teacher hired in the middle of the semester with three kinds of materials available for her perusal:22 1. The contents of an in-basket. 2. Written materials, records, report cards, etc., concerning both the school and pupils in the teacher's class. 3. Human resources that are in the situation (a school secre- tary, a school principal, and a reference memory, all accessible by a telephone placed on the teacher's desk). In addition to observation of the teacher's behavior in written communi- cation, she was also viewed by means of a one-way viewing mirror with rooms connected through a two-way telephone exchange. A number of quantitative measures were derived from the in-basket situation:23 1. Problem Sensitivity: The number of potentially prob- lematic elements reacted to as problems. 20Ibid., p. 360. 21Shulman, Loupe, and Piper, op. cit. 221bid., pp. 5-6. 23Ibid., p. 8. 67 2. Time: The number of minutes the subject chooses to spend in the inquiry situation. 3. Materials Attended: A measure of input, the number of pieces of material to which the subject attends in the inquiry period. 4. InfOrmation Sources: A count of the number of categories of information brought to bear by the subject on ten selected problems in the in-basket. 5. Competence: A measure of problem resolution; how well each subject comes to understand the nature of the prob- lem situation in the selected problems. Shulman concluded that it is possible to conduct systematic studies of inquiry performance through the use of such an in-basket or other simulation of a complex problem situation. The 'thinking aloud' tech- niques used in the study aided the researchers in identifying actual behavior of the subjects. However, Shulman noted that the purpose of the research was not the development of a theory of teaching behavior and that in this particular study teaching was merely a specific exem- plar of inquiry.24 The use of simulation in similar research studies may prove to be an effective tool, he noted, in conducting descriptive studies of the behavior of criterial exemplars of such complex skills as teaching and medical diagnosis.25 jlhe Vocational Office Block Project Initiated in August, 1965, the Vocational Office Block Project CWOB) was one of several research efforts of the Research and Development pI‘Ogram in Vocational-Technical Education, Michigan State University. Clinical schools in five states (Arizona, Florida, Michigan, New Jersey, \ 24Ibid., p. 189. 251bid., p. 191. ()8 and Washington) utilized a block-time approach in the implementation of the vocational office block concept. Planned outcomes of the project resulted in a curriculum development effort with two basic goals: 1. To fuse the fundamental processes and skills into simulated job tasks. 2. To present learning situations when apprOpriate to the skills. Special instructional materials were deve10ped for the VOB project, including integrated exercises, case studies, and model office simula— tion. Evaluation was deemed a high priority by VOB personnel: It was felt that evaluation should consider the process of the project, the outcomes of student achievement, and the related aSpects of local practice and adoption. One of the ten evaluative activities was an experimental situation test of student performance administered by research associates to 590 students in 1966-67. The VOB personnel concluded that the test needed substantial revision if it were to be a.truly evaluative instrument. The situation in~basket was used again in 1967-68 only as an introduc- tion to situation testing and not as an experimental vehicle. A cutback in the availability of federal funds in the midst of the :fiist l8-month contract necessitated a reduction in staff personnel and exam of work in such areas as evaluation, materials development, and . 28 . 1xmcher education. As a result, a number of recommendations were made flnrsubsequent research in these areas. One of the recommendations snemming from the VOB project emphasizes the need for continuing efforts _—_ 26Poland and Haines, loc. cit., pp. 6-7. 27Ibic1., p. 40. 28Ibid., p. 10. 69 to deve10p appropriate evaluative materials for use with a block-time approach.29 Evaluative instruments must be deve10ped which will support a research design comparing the program's effectiveness with the traditional one-hour classes taught in office education. Rimoldi's Test of Diagpostic Skills Within recent years medical educators have been experimenting with simulated means for evaluating and training medical students in certain facets of the profession. Specific inquiry regarding the process of reaching a clinical diagnosis was begun by Rimoldi who developed an instrument, the Test of Diagnostic Skills, for studying "high mental processes with special reference to the clinical diagnostic problem."30 The test, an application of a technique deve10ped at the Loyola Psycho- metric Laboratory for the study of problem solving in general, interested many medical educators because of its potential use as an instrument for evaluating and training medical students.31 The primary purpose of the test is to estimate how a medical student proceeds when diagnosing a given case. Through a simulated approach, approximating an in-basket technique, the subject is requested to reach a diagnosis, given prelimi- nary information about a specific case and the Opportunity to ask Pertinent questions and seek additional information concerning the case. —; 291bid., p. 71. . 30H. J. A. Rimoldi, "The Test of Diagnostic Skills," Journal of fiEfiEfigal Education, Volume 36, Number 1 (January, 1961), p. 73. _ 31H. J. A. Rimoldi, "Rationale and Applications of the Test of Dlalgnostic Skills," Journal of Medical Education, Volume 38, Number 5 May. 1963), p. 364. The fr pockets vhf folder. 01 subject ma) reverse sic of verbal 1 three disti :edical ed 3, m t7] 70 The format of the test itself consists of cards contained in flat pockets which partially overlap and are evenly arranged in a display finder. (hithe top edge of each of these cards a question that the subject may ask is indicated. By drawing a card and looking at the reverse side, the subject gets information that is given in the form of'verbal reports, laboratory analyses, x-rays, etc. Rimoldi found three distinct advantages for use of this simulated technique in medical education: 1. It is possible to study the process followed by a subject in diagnosing a case. 2. Aspects of the diagnostic process appear when using this technique that do not show up in true—false or other types of tests. 3. Subjects who reach the same diagnosis may do so by following different processes. Five major scoring procedures involved the following categories: 1. Number of Questions Asked: Two forms of the test (A and B) contain respectively 56 and 59 items, and it is possible to observe how many questions are asked until diagnosis of the case is reached. 2. The Utility Index of Each Card: The ratio between the num— ber of times that a given question has been asked and the number of subjects in the group can be taken as an indica- tion of the utility of the particular question. 3. The Utiligy Score: The average of the utility indices of the cards selected by the subject results in the utility score . 4. Diagnostic Accuracy: The accuracy of the final diagnosis is obtained for each case. 5. Sequential Relationships: Relationship between the sequences followed by different subjects and average intercorrelation between the sequences followed by all the subjects in a given group enable the researcher to obtain sequential relationships. 32 . . . . Rimoldi, "The Test of Diagnostic Skills," p. 74. 3 31bid., p. 75. 4 £1 71 Rimoldi suggests that the evaluation of each subject's performance is not tied to norms based on average performance but becomes possible in terms of the "intrinsic" difficulty of each problem, the content used, and the group performance. Weitmanz)5 also made use of Rimoldi's Test of Diagnostic Skills in a study of the factors which influence the training of medical students. The Diagnostic Management Problem Helfer and Slater36 were concerned with the process of reaching a clinical diagnosis, the steps taken or sequence used by a medical stu— dent as he works through a clinical problem. In order to better prepare the future physicians to solve clinical problems, equal emphasis must be placed on (1) the quality of the diagnosis or end product and (2) the means or process used to reach this diagnosis. The Diagnostic Management Problem was constructed from case material that appeared in either the in-patient or out-patient departments of the medical center. Information relating to a specific historical fact, a given physical finding, or a single laboratory study relating to that case was recorded on a separate card; and this card was added to the total deck of cards representing one Diagnostic Management Problem. The subject is given the setting in which he is working and provided with an index sheet 34 . . . . . . Rimoldi, "Rationale and Applications of the Test of Diagnostic Skills," p. 365. 3 . 5M. Weitman and F. Coisman, "Medical Student Pathways to Diagnosis," glogmal of Medical Education, Volume 40 (1965), pp. 166-179. 3 . 6Ray E. Helfer and Carl H. Slater, "Assaying the Process of Reaching a Clinical Diagnosis," University of Colorado Medical Center, PP- 79-87. 3 7Ibid. , p. 79. 72 ranch itemizes the type of information available on each of the numbered cards. He then proceeds to work through the problem by selecting cards in any order he chooses; he may select as many or as few cards as he desires and quit at any point in his evaluation of the clinical problem. On a separate answer sheet the number of each card selected and the order in which it was selected are recorded. His final diagnosis or differential is also recorded. The evaluative instrument deve10ped by Helfer and Slater provides a.method for assaying the process a medical student uses to arrive at a clinical diagnosis. Its reliability and validity have shown that it has the capability of selecting out those students who may require special assistance in the development of the skills unique to the process of clinical problem solving. The researchers made the follow— ing observation in relating the importance of such a technique in medical education: Medical educators are continually confronted with the problem of rec0gnizing a student's deficiencies in his prob- lem solving skills very late in a student's clinical training. The Diagnostic Management Problems in addition to the contri- butions made by Rimoldi, Cowles and Weed now provide an answer to this dilemma. Those students who will require special assistance in the deve10pment of their problem solving skills can now be identified at a point in their clinical educational experience when constructive intervention may be both feasible and beneficial. lEEBire's Process Approach to Medical Evaluation Since its establishment in 1959, the Office of Research in Medical Hhmation of the University of Illinois College of Medicine has completed M 3 81bid., p. 82. 73 a series of studies of learning climates at the University, the findings of which support the following view: The then current system of student examination, grading, and promotion not only failed to provide evidence regarding stu- dent achievement with respect to many of the most important goals of medical education ... it actually jeopardized their attainment by exacerbating tendencies toward fragmentation of learning, by focusing student attention on esoteric or trivial detail, and by intensifying unhealthy competition among students for grades and among departments for students' time and attention. 9 lks a result of these preliminary studies, a multi-disciplinary approach 1:0 evaluation40 has been making a significant contribution to the sys- 1:ematic modification of medical education. Patient management problems tiave been developed based upon videotapes of patient interviews, movies c>f patient examinations or autOpsies, audiotapes of heart, breath, lung, aund abdominal sounds. These situational exercises require the medical student to demonstrate: l. Accurate observation of data presented. 2. Knowledge of significance of data and possible inter- relations. 3. Recognition of basic biochemical or patho-physiologic processes that would explain data. 4. Anticipation of other findings.41 A number of simulations were developed to test the complex skills of gathering data and making judgments. McGuire indicates that "these Simulations . .. constitute branched problems in patient management or \ 39Christine H. McGuire, "An Evaluation Model for Professional EducationnMedical Education," Proceeding: of the 1967 Invitational (Ztrnference on Testin Problems (Princeton: Educational Testing Service, 1968 , pp. 38-39. 4°Ibid., p. 38. 41Ibid., p. 40. a; ...f... JHI 74 in laboratory investigation that require sequential analysis and decision."42 The final deve10pmental phase is the determination of minimum acceptable performance on the total comprehensive examination and on each sdbtest. Administration of the test itself yielded two sets of results:43 1. The performance of students at different levels of the curriculum on identical exercises in order to assess student progress toward goals. 2. Any unusual trends in numbers of students failing to meet the pre-established standards. The climate in medical education, according to McGuire, is conducive to the kind of systematic evaluation process that encourages: 1. Empirical determination of essential components of professional competence. 2. Employment of simulation techniques to supplement more conventional methods of assessment. 3. Application of pre-established standards. 4. Utilization of numerous feedback mechanisms to assure fuller exploitation of evaluation data. The patient management problems 5 described by McGuire have been extensively used as part of the comprehensive examinations of third— andzfinuth-year medical students at the University of Illinois and also aseinmjor portion of the examination for certification by various Specialty boards such as that in orthopedic surgery. Scott reported that these same types of simulated exercises were used at Michigan __ 421bid., p. 41. 4 3Ibid., p. 44. 4 4Ibid., p. 51. 45 - ‘ Center for Educational Deve10pment, Patient Manpgement Problems Umiversity of Illinois College of Medicine). 75 State University as part of the evaluation of the deve10pment of problem- solving skills for sixth-year medical students. 'Ihe Medical Inquiry Project Elstein et al conducted an investigation of the reasoning process (of expert physicians, known as the Medical Inquiry Project, at Michigan EState University. The basic objectives of these studies were: 1. To identify the intellectual strategies and tactics characteristic of eXpert clinical reasoning. 2. To generate a psychological theory to eXplain these features. 3. To relate this theory to current theories of thinking, human information processing, decision-making and problem-solving. 4. To develop instructional methods and materials which will assist medical students to acquire and refine their problem-solving skills. Actual strategies employed by physicians in the medical work-up elicited a methodology for inquiry or problem-solving which was utilized :fiar data gathering in a setting closely resembling the one in which IJIIysicians work. It was felt that not only could there be an impact (>11 medical education through the development of new instructional' Unatterials and methods for teaching clinical skills but that a comple- Intentary impact could be made on scientific psychology with actual Inedical problems as exemplars of the complex practical problems encountered regularly in the real world.48 \ 46Letter from Hugh M. Scott, M. D., Royal Victoria Hospital, Montreal 112, P. Q., March 3, 1970. 47Arthur S. Elstein et al, "Methods and Theory in the Study of Medical Inquiry," Office of Medical Education Research and Deve10pment, gollege of Human Medicine, Michigan State University (Sumner, 1971), . 10 48Ibid., pp. 1-2. 5. 1’ m- .1 I}; q n.- ..)u n .v ‘n \ :5. 76 The initial step in developing the methodology for the study was to observe the data-gathering and reasoning processes of physicians .selected from three medical fields: private practice, salaried group practice, and academic medicine. A minimum of two days on the project tuas required with the majority of time allotted to working up and dis- (:ussing three simulated medical problems. The planned research facility jincluded a specially designed room resembling a physician's office and 1two television cameras mounted near the ceiling with subsequent video— t:aping of the doctor-"patient" sequence. A "data bank" technique was Litilized to provide physical findings of the physical examination of tihe patient, but interpretations of this data depend upon each physi- c:ian's judgment. The tentative formulation of a theory of medical inquiry resulted from this initial observation. This theory is stated as follows: Physicians generate specific diagnostic hypotheses early, well before they have gathered most of the data of a particular case. As a part of a long-term prOgram of studies of the medical diag- 11<>stic process, three simulated cases were administered to 19 third- )'e:ar medical students and 29 physicians, all members of the medical Eicihool faculty or physicians in the University Health Service. The results indicate that physicians and students do in fact differ in the ‘Vary'they think about medical problems. Physicians consistently offered ‘ncrre hypotheses (diseases) than students while no differences existed ii!) the number of constraints sought (symptoms, signs, and laboratory findings). What accounted for these differences in hypotheses? \ 491bid., p. 9. Q: [‘9 77 1. Students are more cautious about inferring hypotheses from limited data. 2. Students know fewer hypotheses. 3. Students know an equal number of hypotheses but have weaker links between constraints and hypotheses. 4. The kinds of constraints sought are not the same; thereforeg hypotheses are less strongly implied for students. The observable procedures and reasoning of the physician are efforts to test deductions derived from provisional hypotheses. The hypothetico— demnXive strategy of systematic search, used primarily to confirm or reject provisional formulations, appears to contrast sharply with the progressive constraint-seeking inquiry strategy generally taught by nwdical schools and endorsed by many c0gnitive psychologists. Other Medical Evaluation In-Baskets An elaborate in-basket has been used by the Canadian College of Family Practice in their certifying examinations.52 The in-basket includes a mock—Up of a doctor's office and materials ranging from actors who are trained to simulate different patient types for examiners mulcandidate, telephone interruptions, and correspondence in the doctor's ”in—basket.” The "programmed patient"53 is a similar form of simulation devel- of unit managers provided a more accurate picture of the effec— tiveness of the test as a predictor of job performance than could be ascertained only from the results obtained on the original group.66 The results of the General Electric in-basket research indicate that the in—basket test can serve as a valuable aid in the selection and deve10pment of managers. Subsequent in-basket research by the Behavioral Research Service has indicated that the in-basket test may have potential as an aid in the selection of supervisors or managers at other levels in a manufacturing organization or in other functions. MeYer reports that several departments use abbreviated versions of in-basket tests for training potential managers in particular aspects afnmnagerial work, such as planning.67 The Ordnance Department at ‘— 6 5Ibid., p. 6. 66Ibid., p. 7. 67Letter from Herbert H. Meyer, Manager, Personnel Research, Cbneral Electric Company, December 2, 1969. 84 Pittsfield was particularly interested in the use of the in-basket not only in the selection but also in the deve10pment of first-line fore— nwn. In the eXperiment 57 foremen in the Manufacturing Section of the Ordnance Department agreed to take the Plant Manager InaBasket Test that had been used in the prior research. Appraisals of the actual on—the-job performance of the 57 foremen were obtained from their nmnagers. An analysis of these ratings indicated that foreman perfor- nmnce could be described as consisting of abilities in four general .68 areas. 1. Supervising the work of subordinates 2. Organizing personnel and facilities 3. Performing general administrative duties 4. Providing guidance on technical problems Since technical competence is specific to the type of operations performed by the group supervised (and the in—basket test was not designed to measure technical knowledge), predictive ratings of on- the-job success were made only in the first three areas. An additional evaluative measure resulted from a comparison of the overall effective- rmss rating of the individual as a foreman compared with in-basket per- formance. The study indicated that foremen ranked high in on-the—job perfinmmnce also scored high on the in—basket categories. This result reflected: 1. A systematic, organized approach to problems. 2. Spending considerable time in analyzing problems, seeking more information, etc., rather than making many final decisions on the items. 6 . 8Behavioral Research Service, The In-Basket Test as a Measure (HFForeman Performance (Crotonville, New York: Management Development a1ldEmployee Relations Services, General Electric Company, 1963), p, 2. 85 3. Involving subordinates in decisions, seeking their opinions on problems, etc. 4. A considerable amount of communicating with others in handling the in-basket items.69 In those departments where the test was administered to candidates as a part of these studies, it appeared to provide valuable information regarding (1) the present levels of administrative skill of the men and (2) the management development needs of the participants. In general, the test results tended to corroborate other sources of evidence regard- ing the abilities and work habits of the men. Follow—up conferences with the participants also showed that the actual involvement in a "work sample," which the in-basket test required, tended to make this approach more valuable as a deve10pment aid than merely discussing hypothetical case problems. Long-range plans of General Electric call for additional studies to determine the value of the in-basket as a tool fer appraising candidates for a variety of managerial positions. :[he BoeigggCgmpany_ Zoll, presently a principal in the consulting firm of Management lEducation Associates, was formerly an educational psychologist on the IBoeing staff who served as the Corporate Management Education Advisor. 12011 and his associates are responsible for the development of in- laaskets used in the management training programs of several divisions \vithin the company, including the Pilotless Aircraft Division and the ¥ 69Ibid., p. 6. 70Behavioral Research Service, The In-Basket Test as a Measure of Fdanagerial Aptitude, p. 65. 86 Vertol Division.71 The Boeing Company utilizes in-basket simulation primarily as a training tool in the management education program. Four of the in-baskets currently being used at The Boeing Company and a fifth in-basket, with detailed comments on its construction, have been released for recent publication. Related In-Basket Development Several models for in—basket simulation have been developed by business and industry and are included here as referrent sources for in-basket utilization. IBM Corporation first used the in-basket tech- nique in August, 1957, in conjunction with a training program for the managers of 200 branch sales offices varying in size from 215 to several hundred pe0ple.73 The in—basket is used primarily as a training tool in IBM's management training program. Greenlaw, now Professor of Management at Pennsylvania State University, deve10ped an in-basket designed for a specific industry, the Dayco Corporation, about eleven years ago.7 The Port of New York Authority utilized the in-basket technique in development of training programs in both management and non—management aspects of their organi- zation. John Little developed an 80-minute in-basket known as the Port Authority's Maintenance Foreman Game which was used in the Personnel 71Zoll, The Ianasket Kit: Materials for the Creation and Use of LILBasket Materials. “— 7 . . 2Letter from Allen A. 2011, Management Education AdV1sor, The Boeing Company, November 24, 1969. 73 Lopez, loc. cit., p. 49. 74 Letter from Paul S. Greenlaw, Professor of Management, The Pennsylvania State University, November 21, 1969. 87 and.Training Division.75 Procter and Gamble's "Matrix" and "Management 'Trial Exercise" combined the business-case method, the in-basket tech- 'nique, and a system of sequential decision making in a single business lexercise in order to simulate the administrative aspects of marketing soaps and detergents.76 Another management in-basket was deve10ped by ‘Mbntgomery Ward and used in the management training of store managers.77 In 1962 the American Management Association designed an in-basket exercise for use in its management prOgram for top management executives.78 The AMA had previously been involved in the use of other types of busi- ness games and simulations in its management training program, and sub- sequent use of the in-basket has gained popularity in its seminars and training sessions. Since that time the AMA has also eXpressed interest in the development of in-baskets for use in other areas of business training, including the training of secretaries. The University of Michigan Bureau of Industrial Relations has deve10ped a series of 22 standardized in-baskets for use in specialized training pragrams. Each in-basket contains at least 15 in-basket prob- lens with accompanying sets of background materials and instructor's guides.79 C. The In-Basket in Secretarial Trainingand Development The influence of in-basket simulation for the training and selec- tion of prOSpective managerial personnel has expanded to include the * 75Lopez, loc. cit., p. 51. 76Ibid. 77Ibid., p. 50. 7§l§ifls. p- 23- 79Ibid. 88 training of non-management personnel, especially those in the secre- tarial levels. Prior to 1962, the only secretarial in-basket reported in the literature was administered by the Port Authority of New York as a selection device for promoting secretarial employees to higher-level positions.8o Since that time secretarial in-baskets have been deve10ped for secretarial seminars and workshOps sponsored by professional associ- ations (P0pham) and by business and industry (Port Authority of New York, Montgomery Ward's, National Newark and Essex Bank). In the field of business education the in-basket technique has been utilized primarily as an instructional device in conjunction with three texts (Popham; Place and Hicks; Anderson g£_21). Most recently, the National Secre- taries Association (International) and the Institute for Certifying Secretaries have adopted the in-basket technique for use in the annual CPS certification examination. However, the acceptance of the in-basket technique has not been readily forthcoming, perhaps because of its limited use in the development of instructional materials and evaluative instruments for secretarial programs. The Popham In-Baskets In 1962 Estelle P0pham designed an in-basket for use in the First Executive Secretarial Seminar of the American Management Association, a three-day seminar entitled "Self-Improvement and Skills Development for Executive Secretaries."81 The annual AMA seminar series is now scheduled in various locations throughout the United States, Canada, g 801bid., p. 24. 81 Ruth I. Anderson et a1, loc. cit., p. 20. 89 and Mexico. Popham has developed several in-baskets now published in secretarial texts and has used these materials in AMA workshops, in secretarial workshops, and in workshOps sponsored by the Commerce and Industry Association of New York City.82 POpham credits a management tunkshOp at the Cornell School of Labor Relations with affording her an insight into the adaptability of the in—basket technique to secre- tarialtraining.83 Iannizzi deve10ped an in-basket for use in a work- shOp for secretaries at the City University of New York patterned after the P0pham secretarial in-basket.84 The Port of New York Authority The in-baskets devised by the Port Authority were primarily used as a selection device in secretarial training. LOpez, while manager of the Manpower Planning and Research Division, compiled an informational source on the inébasket and the basic theory on which the technique is based. DeSanto, manager of the Training Services Division, has also twen involved in management games as a part of the training program. 1h; Montgomery Ward Secretarial Seminar In-basket simulation was first introduced as a part of the Mnngomery Ward secretarial training program by Ann Miller of the Administrative Personnel staff of the Chicago office. h; 82Letter from Estelle L. Popham, Chairman, Department Of Business mamtion, Hunter College of the City UDiVeTSitY Of New York, December 18 1969. ’ 8 . 5Letter from Roscoe W. Wisner, SuperVISOT, Personnel Testing, THw Port of New York Authority, December —-, 1969. 90 The main purpose of the in-basket exercise [according to Breen] is to develop the secretaries' ability to appropriately handle the assignments, indicate what action should be taken and by whom in each case, and list in sequence the priority in which to handle the items.86 The Miller in-basket has been administered during two-day secre- tarial seminars held in both the Chicago and New York offices. The typical seminar format reserves approximately two hours for the in- basket plus a group discussion session immediately following. The National Newark and Essex Bank A secretarial in-basket has been used by the National Newark and Essex Bank as a selection device. Bruce Dunnan, Personnel Director, . . . 8 was instrumental in its development. The Secretarial Practicum In one of its most recent applications, the in-basket has been used as a project method for training prospective high-level secretaries. A series of long-range assignments have been developed by Anderson et a1 patterned closely to actual business problems facing administrative . . . . 8 . . . . secretaries in a variety of offices. 8 PrOJects 1n these 51x categories Relocating the Office Handling Administrative Finances Assisting the Security Manager Planning In-Service Seminars . Organizing the Stockholders' Meeting Finding a Secretarial Replacement O‘UT-BWNH 86Letter from C. P. Breen, Training Services Manager, Montgomery Ward, November 25, 1969. 87LOpez, op. cit., p. 83. 88Ruth I. Anderson et al, op;_cit. 91 provide appr0priate stimuli for secretarial students to think, analyze, research, and set priorities and schedules, thus requiring them to make decisions and implement them. Each project includes 20 to 30 related . . . . . . . . . 89 jobs, s1m11ar to those encountered in SpeC1f1C secretarial p051tions. The CPS In-Basket As an initial trial in 1970, the Institute for Certifying Secre— taries, a department of the National Secretaries Association (Inter- national), incorporated a secretarial in-basket into the "Communications and Decision Making" section of the Certified Professional Secretary Examination.90 The in-basket, as it has been used in both the 1970 and 1971 CPS Examinations, is a performance test in which the candidate is required to make judgments about how to handle various situations. Specific aspects of the in—basket involve editing, abstracting verbal and quantitative data, composing, transcribing verbatim dictation, and judging the acceptability of the work of subordinates. Three basic response patterns are sought for each item in the CPS In—Basket:91 1. What action will be taken? 2. How should concerned persons be informed of the action taken by the examinee? 3. What follow—up procedure should be employed? ”All in a Day's Work" and "The Secretary's In-Basket" are two in- baskets published by the NSA in The Secretary, its monthly publication. These in-baskets were based respectively upon the 1970 and 1971 CPS 891bid. 0"You Too Can Be a Certified Professional Secretary," The W (October, 1970), p. 39. —"' 1Leonard Robertson, "Everything You Always Wanted to Know About PreParing for Part V," The Secretary (March, 1971), P- 13- 92 examinations.92 Place and Hicks have deve10ped several secretarial in-baskets which have also appeared in The Secretagy and in their secretarial office practice text materials. D. Summagy, The historical development of the use of in-basket simulation in research, management training and development, and secretarial training and development indicates that there has been growing interest in the use of the in-basket technique since the 1950's. Frederiksen, though credited with being the originator of the in—basket test and a leading researcher in in-basket simulation, has emphasized that there are numerous fields of concentration in which in—baskets may be deve10ped and utilized to improve training that is now going on in those areas. Procedural design for in-basket simulation has, however, remained some- what experimental in nature; and such aspects of in-basket simulation as scoring and evaluation are still being questioned. An examination of simulation techniques resembling the in—basket technique being utilized in medical education demonstrates that Rimoldi, Helfer and Slater, McGuire, and others are utilizing the simulation approach in the training of medical personnel. In management training and deve10pment the initial use of the in-basket was in the evaluation of management personnel, but the emphasis is changing to include non- management personnel, including the secretarial ranks of employees. Such businesses as The Bell System, Sears, General Electric, The Boeing Company, and IBM Corporation have laid the groundwork for the use of 92"A11 in a Day's Work," The Secretary (October, 1970), pp. 26-29; "The Secretary's In-Basket," The Secretagy (June-July, 1971), pp. 28-31. 93 the in-basket in the training of management personnel. The use of the in-basket in secretarial training and development has been rather late in starting. However, the American Management Association, the National Secretaries Association (International), the Port of New York Authority, and Mbntgomery Ward's are among the professional and business organiza— tions that have become more interested in recent years in the promotion of in-basket and other simulation techniques in the training of secre- taries. Popham, Place and Hicks, and Anderson g£_21_have deve10ped secretarial in-baskets used in secretarial workshops and training pro- grams. The acceptance of the in-basket technique as a tool for training secretaries is just beginning to be noticeable in the development of instructional materials for secretarial programs. CHAPTER IV RESEARCH IN THE SECRETARIAL ROLE AND CURRICULUM DEVELOPMENT WITH IMPLICATIONS FOR SECRETARIAL EDUCATION The secretary, though her job classification in one business firm may be somewhat different from that in another, is described and classi- fied in the Dictionary of Occupational Titles as a "girl friday; a secretarial stenographer" who performs the following tasks on the job: Schedules appointments, gives information to callers, takes dictation, and otherwise relieves officials of clerical work and minor administrative and business detail: Reads and routes incoming mail. Locates and attaches apprOpriate file to correspondence to be answered by employer. Takes dictation in shorthand or on Stenotype machine and transcribes notes on typewriter, or transcribes from voice recordings. Composes and types routine correspondence. Files correSpondence and other records. Answers telephone and gives information to callers or routes calls to appropriate official and places outgoing calls. Schedules appointments for employer. Greets visitors, ascertains nature of business, and conducts visitors to employer or appropriate person. May not take dictation. May arrange travel schedule and reservations. May compile and type statistical reports. May supervise clerical workers. May keep personnel records. May record minutes of staff meetings. Research on the role of the secretary in business has been con- ducted periodically from 1924 to the present. Responsibilities and duties on the job, personal qualities and traits desirable for adequate yflJPerformance, tasks to be performed on the job, and the secretary's -—___ 1 . . . U. S. Department of Labor, Dictionary of Occupational Titles, 1 . oc. Cit. 94 95 adaptation to the job and fellow employees constitute some of the kinds of information and data which have been collected and analyzed. Charters and Whitley (1924) conducted what is considered the classic study of the role of the secretary, and subsequent investigations (Nichols, 1934; Place, 1945; Kosy, 1958; Noyes, 1960; Paddock, 1967; Wagoner, 1967; Cook, 1968) utilized findings from the earlier study on which to base some of the research Operations. Analyses of the secretarial role indicate two distinct phases of development: 1. The Task-Trait Analysis: Primary attention is given to the individual and personal characteristics of the secre— tary and her ability to perform those responsibilities of her job (Charters-Whitley, 1924; Nichols, 1934; Place, 1945; Young, 1954; Meyer, 1954; Wink, 1956; Mickelson, 1957; Welling, 1957; Noyes, 1960; Wagoner, 1967; Paddock, 1967; Perkins-Byrd, 1968). 2. Th; Situational Analysis: In addition to the delineation of tasks and responsibilities of the secretarial position, the importance of the work group and the work situation is considered in establishing the adequacy of the secretary's performance on the job (Casebier, 1957; Kosy, 1958; Cook, 1968; Lanham et al, 1970). Oftentimes it is difficult to separate studies into one of these two categories since some of the studies used a form of situational analysis to arrive at a list of effective task performances. However, the above categorizations are based upon the resultant outcomes of each study. A. The Task-Trait Analyses Twelve research studies conducted within the period from 1924 tlrrough 1968 delineate the tasks and/or traits required of the secre- tary in her position. Of the 12 studies, only three concentrate on the secretary in her general capacity. A review of selected studies is :included here to provide a brief resume of the important findings, 96 conclusions, and/or recommendations which have particular significance to the present study. The Charters-Whitley Study The classic study of the role of the secretary was conducted by Charters and Whitley (1924) and involved the delineation of duties performed on the job as well as the determination of personal qualities evident in successful secretaries. An initial interview with 125 secretaries resulted in a list of 871 duties, and this list was then submitted to 715 secretaries in the field testing phase of the study. A second compilation of 47 traits present in successful secretaries was obtained through employer interviews. Those duties receiving the highest number of tallies were recategorized into the following func— tional areas within 15 occupations: Mailing duties Taking dictation Transcription Typewriting Filing Duties connected with filing, indexing, etc. Telephones, etc. Editorial duties Duties involved in meeting and handling peOple 10. Financial and bookkeeping duties ll. Clerical duties 12. Miscellaneous duties and personal services for employer EDOOVONU‘IAQJNH Personal traits and qualities of the secretary as rated by the 28 mmfloyers interviewed in the study were tabulated so that "trait-actions" unfld be formulated for each of the traits. Not only was the trait h 2 . . W. W. Charters and Isadore B. Whitley, AnalySis of Secretarial Manes and Traits (Baltimore: Williams G Wilkins Company, 1924),Vfi_ P- 11. 3 . Ib1d., pp. 97—121. 97 "accuracy" itemized, but also actions which indicated that accuracy was taking place. Those traits receiving the highest ratings in the 28 interviews were: accuracy, responsibleness, dependability, intelligence, courtesy, initiative, judgment, tact, personal pleasantness, and personal appearance. Charters and Whitley attempted to provide a research base for the development of the content of commercial courses for prospective secre- tarial employees: Heads of commercial departments, by checking the content of commercial courses against the duty list, will be able to see the extent to which training is given in those duties which are most frequent and which can be most satisfactorily taught in school. In the office practice course, as well as in dic- tation, transcription, and typing, the materials most frequently handled in the business office should be generally used as the practice medium.5 The Nichol s Studx A study conducted by Frederick G. Nichols in cooperation with the American Institute of Secretaries of Boston, Massachusetts, resulted in the identification of 21 major secretarial duties based upon the com— bined judgment of secretaries and employers. The Nichols study (1934) had as one of its main purposes the identification of duties which distinguished private secretaries from other office workers. From available research and literature, a list of 35 duties of the secretary was constructed and sent to 692 private secretaries and their “ 4Ibid., p. 174. 5Ibid., p. 13. 6Frederick G. Nichols, The Personal Secretagy (Cambridge: Harvard University Press, 1934), p. 44. 98 employers who were asked to indicate the rank order of these duties by assigning to each a number according to its importance to the private secretary. Usable replies were received from 237 secretaries and 86 employers. The major secretarial duties derived from this study include the following:7 Take dictation Transcribe shorthand notes Handle callers Write original letters Organize facts Answer letters Organize office routine Note information on letters Handle incoming mail 10. Read and release letters for mailing ll. Organize files and filing systems 12. Take care of personal accounts 13. Consult reference books 14. Make appointments 15. Do banking for employer OQVO‘U‘l-hWND—I Because the secretaries and executives were asked to base their replies upon the importance of a given duty to most private secretarial posi- tions, the replies were in the form of opinion concerning most secre- tarial positions. The Place Study, With the Charters-Whitley and the Nichols studies as initial guides, Place accumulated a list of duties for secretaries and/or executive aides. Questions were developed to cover every secretarial aspect of each duty, and respondents were asked to indicate frequency of performance. The data emanating from the study, based upon usable responses from 226 secretaries and 237 employers, provided a frequency -___i 7Ibid., p. 64. 99 ranking of secretarial duties.8 The more frequently occurring duties included the following: Take dictation and transcribe General filing Make appointments Alphabetical filing Answer the telephone and route callers Read and sort mail Place and make long-distance calls Operate an adding or calculating machine Keep a record of appointments Assist with the preparation of written reports of a general, financial, government, or research nature ODWN0M§MNH H One of the significant contributions of the Place study (1945) was its attempt to delineate not only the duties that were performed by .secretaries but also the frequency with which Specific duties were performed. The Young Stud! The primary purpose of the Young study (1954) was to determine those duties of a bookkeeping or accounting nature performed by the secretary as well as the nature of other duties performed. Data were collected by personal interview of 100 secretaries who were asked to place duties performed on one of five categories of importance.9 Rankings were obtained of the 71 duties appearing in the questionnaire, and frequencies resulted showing the most important duties of the secretary. 8Irene Place, "A Study of Personal Secretaries in Sixteen Communities in the State of Michigan." Unpublished Ed. D. dissertation, New York University, 1945, p. 122. 9John R. Young, "A Study of Secretarial Duties with Special Attention to Those Duties which Emphasize Bookkeeping or Accounting Functions." Unpublished Ph. D. dissertation, State University of Iowa, 1954, p. 276. 100 The ten most important duties of secretaries, according to Young, are presented here:10 Take dictation and transcribe notes Prepare reports for employers Obtain and organize information Answer phone and make calls File for employer Typewriting from drafts Making appointments Read and classify mail . Prepare customer statements Receive callers ...; O‘DmVO‘m-§MNH The Noyes Study The position of the woman executive secretary was the focus of 'the Noyes study (1960). The primary purpose Of the study was to deter- mine the responsibilities Of the woman executive secretary, who was defined as a secretary to a member of tOp management, as differentiated from those of the woman non-executive secretary and to determine her place in office management.11 Data were obtained by personal interview and mail questionnaire from 236 secretaries (133 executive secretaries and 103 non-executive secretaries). A listing of 32 management functions performed by executive secretaries emanated from the Noyes study. The tOp ten functions, listed in order of performance frequency, are included here:12 . Keeping and filing confidential papers Composing non-routine letters 1 2 3. Superiors ask Opinions and advice 4. Secretaries make unsolicited suggestions to superiors Ibid. 1Honora M. Noyes, "The Role of the Executive Secretary as a Member of Management." Unpublished Ed. D. dissertation, University of Maryland, 1960. 121bid., p. 349. 101 Superiors adopt ideas Interviewing and relaying stories Required to type and file confidential papers Explaining superior's viewpoint to others . Making decisions on handling exceptions to ordinary office problems 10. Handling superior‘s personal banking OmVOUI A comparison of the job activities of the executive secretaries with those of the non—executive secretaries did not show any activities which belonged particularly to either group of secretaries. The Paddock Study Paddock (1967) attempted to determine a more accurate definition of the high—level secretarial position than had previously been avail- able. The study was limited tO secretarial positions for business executives or professional people who make executive decisions and to 50 selected secretarial—executive teams in the Indianapolis area. Findings resulting from the study were categorized in the following 13 areas: 1. Responsibilities of the secretary 2. Qualifications needed to fill the position 3. Patterns followed by secretaries in preparation for the position and for growth within the position 4. Patterns of development available to secretaries and to potential secretaries 5. Conditions influencing the job satisfaction of secretaries Paddock found that the assignments of the high-level secretary were in communications, office management, and performance of personal duties. The activities common to all assignment classifications for secretaries included planning and scheduling the work of the office: M 13 . . Harriet L. Paddock, "Personnel Needs for High-Level Secretarial P051tions," Delta Pi Epsilon Journal, Volume XI, Number 2 (February, 1969)) pt 17. 102 . representing the executive, usually as an intermediary in the handling of telephone calls and in receiving Office visitors, as liaiSon between the executives and Others, as coordinator of office activities, and as the medium through whom im ortant papers were transferred and information transmitted. Because of the nature of secretarial activities in the office, the executive was assisted in making decisions, conserving time, and expe- diting the work of the office. However, the executives were eXplicit in delimiting the secretary's authority to executive wishes or firm policy. The majority of the executives surveyed believed that the secretary's authority was limited to her own work or to delegated work. The Paddock study indicates varying emphases in secretarial work: (1) duties and activities of a secretarial nature and (2) the degree of re5ponsibility. Secretaries need an educational background which will prepare them for the responsibilities of top level positions. Tech- nical education alone will not be adequate for the demands imposed upon secretaries in executive offices today. EXperience, other than secretarial, such as managing an office may be of value to those in the high-level secretarial position. Paddock also concluded that further attention needs to be focused upon proper work attitudes and the development of desirable personal traits needed for continued employment. The Wagoner Study 6 . . Wagoner (1967)1 conducted a survey of executives and secretaries in order to clarify the functions and duties performed by the secretary 14Ibid., p. 19. 151bid., p. 22. 16Kathleen P. Wagoner, "The Role of the Secretary in a Changing World: An Analysis of the Duties and Functions Performed by and Under- Standings Required of the Secretary." Ph. D. thesis, University of Iowa, 1967. in today's changing world and to determine if, in effect, the secre— tary's role is changing. To ascertain the degree of change, a list of 162 duties compiled by Wagoner in a survey of executives and secretaries was compared with those composite lists of duties in previous studies (Charters-Whitley, 1924; Nichols, 1934; Place, 1945; Young, 1954; Noyes, 1960). The functions and duties used in the survey and selected from previous research served as the basis for evaluation of change. Those duties identified as new duties occurred as a result of recent develop— ments in business. Change in the role of the secretary was measured by determining change in the degree of responsibility assumed by a secre— tary when performing a given duty and change in the method of performing a function. Respondents were asked to indicate the importance of 13 functions of the secretary in the total consideration Of the secretarial role and to indicate the frequency of performance of 162 duties accord- ing to level or degree of responsibility assumed by the secretary and methods or equipment used by the secretary in performing the duties and tasks that make up her role. Data collection was accomplished through a proportionally strati— fied random sample of 250 business organizations in the State of Indiana, representing the four Standard Industrial Classifications (SIC) in which a majority of secretaries were employed according to 1960 census infor— mation. Usable responses were received from 145 pairs Of executives and secretaries from the areas of transportation, communications, and utilities; wholesale and retail trade; finance; and manufacturing.17 MW 17Ibid., p. 245. 104 Wagoner found that the role of the secretary is changing especially with regard to written communication and assistance to employer. Today's . . l secretary performs two b351c types of duties: 8 1. Those which are clerical or general in nature but are per- formed by the secretary in the capacity of assisting the executive. 2. Those which are more narrowly defined as secretarial duties for which the secretary is primarily and usually responsible and which require a knowledge and understanding possible only through close contact with the situation. Functions and duties related to Office management and accounting are of lesser importance than formerly and appear to have been delegated to others to perform because of increased specialization in these areas. New kinds of knowledge and new types of duties affect the secretary's role:19 1. More use of various techniques involving composition of business correspondence 2. Collection of background research and information for preparation of reports and presentations 3. Organization of new filing systems 4. Maintenance of confidential records 5. Increased use of copying machines and other specialized production units Both executives and secretaries agreed that three other directions . 20 f change are affecting the secretarial role: 1. The secretary is performing many more duties formerly performed by the executive. 2. The pace of the business office is increasing, and greater emphasis is being placed on Speed of Operation. 3. A number of duties formerly performed by the secretary are now assigned to specialized departments. 18113101., p. 253. 191bid., pp. 239-243. 201bid., p. 243. 105 The importance Of the Wagoner study in analysis of the secretarial role is its contribution in relating the changing role of the secretary to the pace of the business world. The Perkins-Byrd Study The model deve10ped during the first phase of the Perkins-Byrd study (1968) provided the basis for the identification of clusters of tasks performed by a comprehensive sample of Office workers employed in six Office job classifications.21 For each of the classifications sampled (supervisory workers, secretarial-stenographic workers, clerical workers, bookkeeping-accounting workers, business machine Operators, data processing workers), tasks performed were grouped or "clustered" within these 13 major categories:22 Performing typewriting tasks Operating office machines and equipment Taking dictation and transcribing Performing mailing tasks Performing filing tasks Performing telephoning and communicating tasks Performing clerical tasks Securing data Using mathematics 10. Performing financial and recordkeeping tasks 11. Performing editorial tasks 12. Meeting and working with people 13. Performing miscellaneous tasks woooxzosm-huwt-o 21Edward A. Perkins, Jr., and F. Ross Byrd, A Research “9321 for Identification of Task and Knowledge Clusters Associated with Perfor- mance of Major Types Of Office Employees' Work, Final Report Number 5, Project No. ERD-257-65, Contract No. OE-5-85-109 (Pullman: Washington State University, December, 1966), p. 4. 22Edward A. Perkins, Jr., F. Ross Byrd, and Dennis E. Roley, Clusters of Tasks Associated with Performance of Major Types of Office WOrk, Final Report, Project NO. 7-0031, Grant No. OEG-4-7-070031-l626 (Pullman: Washington State University, January, 1968), pp. 20-52. 106 In addition, composite clusters of tasks performed by Office employees in general resulted from the classification. Prior to this time, no research had been conducted (that was known to the researchers) which Specifically related to the development of a task—knowledge research prototype designed to encompass the entire spectrum of office work.23 No studies could be located which related specifically to the consideration of tasks performed by all types of Office workers in all types of businesses. The first phase (Perkins-Boys, 1966) resulted in a model for col— lecting up-to—date information on Office tasks as well as task data needed in the evaluation of present curricula and in the design of new curricula. The second phase of the study (Perkins—Byrd, 1968) involved the identification Of clusters of tasks performed by a comprehensive sample of office employees working in Offices of varying sizes in the 12 Standard Industrial Classifications (SIC). Extrapolation of those findings pertinent to the role of the secretary indicate that the majority of the secretary's time involves the following duties and tasks:25 Typewriting Operating office machines and equipment Taking dictation and transcribing Handling mail Filing Telephoning and communicating ChenAwmr-a M 2 . 3Perkins and Byrd, loc. c1t. 2 4Ibid., p. 19. 25 . . . Edward A. Perkins, Jr., and F. Ross Byrd, ”Identifying Office ctivities by Clusters of Tasks," The Emerging Content and Structure of u51ness Education, National Business Education Yearbook, No. 8 (1970), p. 107—109. 107 7. Performing clerical duties 8. Securing data 9. Using mathematics 10. Meeting and working with pe0p1e The secretary, however, is not generally involved with financial and recordkeeping tasks or editorial tasks. Of the 599 tasks, 417 tasks were significant at the .05 level or higher.26 Therefore, the hypo- thesis that there was a significant difference in the tasks performed by office employees in the various SIC categories was accepted. Another hypothesis accepted as a result of the research indicated that there was a significant difference in the tasks performed by office employees working in small or large offices.27 Perkins and Byrd indicate that the analysis of the task data suggests that the acquisition of perfor- mance skills represents only a portion of the office worker's function. Two conclusions of the study have particular importance to the present study: 1. Teaching objectives, learning eXperiences, and evaluation can profitably be organized within the framework of the cognitive, affective, and psychomotor domains.28 2. Self-paced learning packages [should] be developed and field tested and that the design of these instructional systems be based on the [general and specific statements of instructional] objectives identified. The task data collected and analyzed in the Perkins-Byrd study represent duties performed by major office occupational groups. Cur- riculum specialists may find these data useful in the development of 26Perkins, Byrd, and Roley, op. cit., p. 14. 27Ibid. 281bid., p. 167. 29Ibid., p. 168. 108 more specialized high school and post-high school Office education programs. Lanham30 Observed that the Perkins' study sought relevance for job preparation from current jobs. However, a weakness of the Perkins-Byrd task clusters is a lack of definition of task, thus making the transference of the base data to learning programs a grave problem. Other Task-Trait Analy§es Welling (1957), Mickelson (1957), Wink (1956), and Meyer (1954) conducted further analyses of the duties and responsibilities of secre- taries in specialized areas of employment. B. The Situational Analyses Four research studies conducted within the period from 1957 to 1970 emphasize the importance of the work group and the work situation in which the secretary is placed. Of these four studies, one relates to several office job classifications (Lanham, 1970). Casebier (1957) analyzed the duties and responsibilities of the general secretary, while Kosy (1958) and Cook (1968) researched the private secretary and secretary/stenographer classifications respectively. The Casebier Study Casebier (1957) used the time study technique to gather data to be used in the analysis of the secretary's daily activities. Secretaries representing 24 different firms in the Chicago area participated in ‘the study, and activities related to their positions were recorded on t:ime-study charts in order to make an accurate analysis. 30Lanham et al, pp. cit., p. 6. 109 Findings of the Casebier study pertinent to the present study include:31 1. The typical day for a secretary lasts eight hours. One hour is required for lunch and one hour is lost in taking breaks and in doing nothing. Six hours are devoted to completing work assignments. 2. During the time that the secretaries were on the job, they were engaged in 47 different activities in the fifty days timed. Ten of these activities required three per cent or more total time. 3. The secretary spends 72 per cent of her time in the per- formance of the ten major activities and their applications. She spends one-third of her time at the typewriter. 4. The time study identifies two new major activities Of secretaries. These are: conferring with the supervisor and preparing for the day's work and for closing. 5. There is no uniform pattern of work procedure according to time; the sequence Of activities of the secretary's day is determined by the nature of the particular day's work. 6. The main factors that cause problems and anxieties for secretaries are: (l) the supervisor, (2) working condi- tions, and (3) interruptions. The Kosy Study Kosy (1959) derived a compilation of critical requirements for effective on-the-job performance of the private secretary. A critical incident analysis, including self-analysis and self-reporting by secre- taries as well as employer observations, was used in determining effective and ineffective behavior of these secretaries. Interviews of 50 executive-private secretary pairs from 50 manufacturing firms in Seattle, Washington, produced 352 usable incidents which were analyzed 1Virginia Casebier, "A Time Study of Activities and Responsibilities Of Secretaries with Implications for the Training of Prospective Secre- taries," Dissertation Abstracts (Ann Arbor: University Microfilms, 1957), pp‘TT'T”. 28 5-2 76. 110 . . . . . . . 3 . . into the duties and respon51bilit1es of secretaries. These critical incidents produced over 400 individual or grouped behaviors. After the incidents were analyzed, four broad areas were developed so that :he incidents could be classified:34 1. Relationships with individuals and firms outside of the business establishment 2. Relationships with employees 3. Relationships with the executive 4. The secretary as an individual The critical requirements for the successful private secretary, ccording to Kosy, are presented here in total as they are listed in 1e study. Because no particular order of importance was indicated, )ne could be excluded.35 1. Composes and types various kinds Of business letters and presents them to the executive for his signature. 2. Applies her knowledge of the rudiments of grammar, punctu— ation, and the vocabulary of business to the composition, transcription, and editing of the executive's written communications without changing the desired meaning. 3. Proofreads, checks, verifies, and has all errors corrected. 4. Spells accurately. 5. Displays a knowledge of and skill in arithmetic in the performance of her duties which involve mathematics. 6. Takes dictation, transcribes, and types in a manner that brings credit to her and her employer. M— 32 . . . . Eugene J. Kosy, "The Critical Requirements for Private Secre— ies Based upon an Analysis of Critical Incidents.” Doctoral sertation, University of Wisconsin, 1958, p. 155. 3 3Ibid. 34 Ibid., pp. 155—156. 35. Ibid., pp. 146-148. 10. 11. 12. l3. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 111 Records telephone messages completely, accurately, and systematically. Projects an interest in the caller through a pleasing telephone personality. Receives telephone calls, establishes the purpose of the call, finds a solution to the problem presented, and when appropriate notifies the caller of the solution. Obtains maximum productivity from the employees under her supervision. Assists other employees during over-load periods. Keeps her personal affairs and private life out of the business Office. Maintains respect for authority and conforms to the same office policies as other workers. Establishes and maintains good working relationships with employees throughout the firm by expressing an interest in them and reSpecting their position. Greets callers pleasantly, determines the purpose of their call, and assists them when she is able to do so. Makes decisions as to who will be permitted to see the executive. Answers routine questions concerning the business without disturbing the executive. Associates names and voices with individuals. Establishes a system and codes, files, and locates infor— mation in the files. Keeps her employer informed of all items requiring his attention. Insures that the employer's records, reports, and bids are complete, accurate, and submitted on time. Assumes responsibility for routine business activity as an "agent" of the executive. Maintains a work schedule without directions from the executive. Knows where the executive is at all times and is able to locate him if necessary. 112 25. Identifies and retains confidential information. 26. Adopts a routine which offers maximum support to the executive in the fulfillment of his duties. 27. Endeavors to promote the welfare of the firm and the executive in the eyes of the public and his superiors. 28. Volunteers suggestions which increase the efficiency and productivity of the firm. 29. Reports to work daily and arrives on time. Many of the foregoing critical requirements reflect decision-making capacities required in the on-the-job performance of the private secre- tary. One of the noticeable limitations of the study, as it relates to the present study, is the inability to delineate the order Of importance of the critical requirements as they were derived from the Kosy study. The Cook Study Attention to the work group and the work situation was the target of the Cook study (1965-1968). The four major Objectives of the study, sponsored by Delta Pi Epsilon, were:36 1. To identify successful secretaries. 2. To analyze which variables contributed to or were associated with secretarial success. 3. To identify factors relevant for the education and training of secretaries. 4. To develOp possible variables and research designs that might be utilized in subsequent studies in this subject area. A single public utility was selected as the only practical source avail- able for data collection because of the total number of secretaries needed to acquire pertinent data. Primary data were secured from the 36Fred S. Cook and Edward Gary Shapiro, "Factors Affecting Secre- tarial Success," Business Education Forum, Volume 24 (February, 1970), p. 33. 113 employees of 67 work groups, a total of 326 employees. In the Cook study, a "work group" is defined as three individuals: a secretary/ . 37 stenographer, a superVisor, and a peer. The Cook study was a pilot study based upon an interactionistic point of view, resulting in a general overview of the successful adapta- tion of the secretary/stenographer to her role in business. The study, divided into ten major areas, resulted in a total of 70 findings dis- . . . . 8 cussed in detail in the complete report. These ten major areas are:3 Beliefs about secretaries Social characteristics Education of secretaries Occupational history Secretarial skills (objective and subjective ratings) Job characteristics of secretary Personality characteristics Job satisfaction . Characteristics of the work group . Supervision O‘DOOVGU'IAUJNH H Cook found that there is little indication that high skill levels, when measured objectively, are significantly related to secretarial success. Successful secretaries are considered more a part of the total work group than less successful secretaries. A particularly interesting finding relates to the degree of decision-making required of secretaries on the job:39 Successful secretaries were more likely to make minor decisions on the job than were less successful secretaries. In terms Of major decisions made on the job, however, the hypothesis of a 371616. 381616. 39Fred S. Cook and Edward Gary Shapiro, Factors Associated with Successful Adaptation to the Secretarial/Stenographic Role, Project *No. 6: 1281, Grant No. OEG- 3- 6- 062181- 2079 (Detroit: Wayne State University, 1968), p.126. 114 significant difference existing between the successful groups and less successful groups of secretaries could not be supported. The researcher's intent in conducting such an analysis of the secretarial role was to focus attention upon the work setting and the various situ- ational variables which contribute to secretarial/stenographic success or failure. He also felt that the analysis could serve as a basis for revision and updating Of current secretarial and stenographic curricula . 4 . in the secondary and post-secondary programs. 0 Three concluSions of . . . . 4 the study bear particular Significance to the present study: 1. Tests of secretarial skills and general intelligence, given as prerequisites to hiring, are not significantly related to secretarial success. 2. Successful secretaries were those whose contribution was rated as vital to the organization, who performed more job duties or made minor, as compared to major, decisions. 3. The work situation, rather than the emphasis upon development Of one's personality traits, is a major determiner of the degree of success. It does not always appear that emphasis upon personality deve10pment is the most effective method of preparing young peOple to per- form adequately in job situations. Generalized attitudes and traits, such as energetic, decisive, flexible, initi- ating, confident, organized, and accurate are the traits of the highly successful secretary. Cook's identification of the role of the secretary was enhanced by the perspective obtained from the work-group situational approach that was used in the data gathering. Perhaps the most profound limitation of the entire study was the fact that only one type of business was represented in the study. 40Cook and Shapiro, "Factors Affecting Secretarial Success," p. 33. 41Cook and Shapiro, Factors Associated with Successfhl Adgptation to thgySecretarial/StenOgrgphic Role, p. 198. 115 The Lanham Study One of the sub-studies conducted as a part Of the research associ- ated with the deve10pment of the New Office and Business Education Learnings System (NOBELS) was a study Of the characteristics, attitudes, and traits of workers in terms Of the social context of office work. The research was not limited to one particular job classification within the Office cluster of occupations. Instead, each Office task was ana- lyzed in relation to its performance in the office situation. Because the job classifications from which the incidents were collected are closely related within the Office cluster, Lanham and his colleagues chose to eXpand the study horizontally rather than in depth.42 Four basic steps were involved in this phase of the NOBELS study:43 1. A decision had to be made as to what social roles the various office employees are expected to perform on the job. 2. These roles were then observed in life situations in order to determine the essential skills, attitudes, and rules which are necessary for successful performance. 3. Wherever possible, the skills, attitudes, and rules were quantified (enumerations, scores, physical measurements, and to a limited extent, rankings). 4. These skills, attitudes, and rules became the foundation for the deve10pment of social-role performance goals of the educational program and its evaluation design. To accomplish these four goals, Flanagan's critical-incident tech- nique was adapted and modified for the interview process in which Office supervisors were asked to describe four observed events. The collection of 780 critical incidents during the pilot phase of the sub-study yielded 42Lanham et a1, Op. cit., p. 34. 431614., p. 17. 116 a classification based on the worker in relation to the occurrence of the incident in the system. In order to conduct a systematic inventory of Office tasks as performed in current and emerging Office occupations, Lanham g£_§l conducted a personal interview Of 1,253 Office workers and their respective supervisors yielding 1,232 usable cases.44 The total number of basic tasks performed was 4,548; and these tasks were later identified, classified, and analyzed. The basic task data produced 374 performance goals. Calhoun was instrumental in the categorization of those performance goals deve10ped from the 829 incidents in the NOBELS study that involve social interaction situations.4S Wilms analyzed 326 critical incidents collected as a part of this study and grouped them into fOur major categories:46 1. Knowledges necessary to job performance 2. Skills necessary to job performance 3. Personal characteristics necessary to job performance 4. Necessary personal relationships with other individuals The critical behaviors within each incident were grouped according to basic similarities, and critical requirements were written for each group of similar behaviors. This NOBELS sub-study concluded with a specification of three fundamentals of office task performance:47 44Ibid., pp. 22—23. 4SCalfrey C. Calhoun et al, Development of Performance Goals for a New Office and Business Education Learning§_5y§tep, Final Report, Project 91E, Grant No. OEG-8-084l4 (Washington, D. C.: U. S. Department of Health, Education and Welfare, 1970). 46Lynda Britt Wilms, "Critical Requirements for Beginning Office lVorkers Based upon an Analysis of Critical Incidents." Unpublished Rd. A. thesis, The University Of Georgia, 1969. 47 Lanham et al, Op. cit., p. 291. 117 1. Accuracy as a desired generalized behavior of office workers is a function of checking for potential error and of correcting error. 2. Inadequate communication skills, both oral and written, are a frequent source of unsuccessful office performance. 3. Organizing and classifying of both numerical and verbal data (all symbolic data) are even more necessary behaviors in our emerging electronic technology than ever before. AS a result of the Lanham study, design criteria were developed for the analysis of office occupations and the establishment of critical require- ments fOr Specific job classifications for later use in the development of perfOrmance goals for instructional programs. C. Research in Curriculum Development Analyses of the role of the secretary (and other Office workers) have led to research in the development of curricula for the training of peOple for Office occupations. NO attempt is being made here to review all research that has been done nationwide in curriculum develop- ment for office occupations. Instead, a brief look at three research efforts within the State of Michigan, all of which have received nation- wide recognition, will support the need for the development of the secretarial curriculum (units of instruction, evaluative instruments, and supportive materials) based upon actual requirements Of the secre- tarial position. The Senior Intensified Program Two major studies contributed findings used in developing Wayne State university's Senior Intensified Programs (SIP) in office and retail occupations. The first Study delineated the absence of accurate infOrmation on entry level jobs and the need for a comprehensive study 118 of entry jobs in office and retail occupations.48 The second study was deve10ped to fill this informational gap in the Detroit area.49 The findings resulting from the second study (OREOS) led to two conclusions which influenced the pattern for the development of the clerk-typist and 50 clerk-stenographer SIP programs, two of the four programs deve10ped: 1. Few business "skills" are demanded as prerequisite for employment in office and retail jobs. 2. Typewriting is the one Single business skill most often required in an entry office and retail job. The SIP programs developed to meet the entry-level requirements for the clerk-typist and clerk-stenographer positions have been used in twelve Detroit high schools during a two-year demonstration project in order to test their effectiveness in preparing youngsters fOr entry- level positions in Office occupations.51 The early identification Of minimal skills essential for these entry jobs (typewriting, shorthand, transcription, and related job tasks) led to the preparation of rele- vant materials to teach these skills. Once the minimal time necessary to teach these skills was delineated, the program was ready for the actual instructional phase at the 12th grade level. Three of the 48Fred S. Cook, Detroit Study for the Effectiveness of High School Education for Entrance into the World of Work, Report to the Michigan Department of Education, Division of Vocational Education (June, 1963). 49Fred S. Cook and Frank W. Lanham, Opportunities and Requirements for Initial Employment ofySchool Leavers with Epphasis on Office and’ Retail Jobs, Project NO. 2378 (Detroit: Wayne State University, June, 1966). 50Fred S. Cook, Francis J. Brown, and Frank W. Lanham, Senior Year Intensified Demonstration Curriculum Proggpm, Final Report, Project NO. 6-1968, Grant NO. OEG-O61968-1993 (Detroit: Wayne State University, April, 1970), p. 4. S 1Ibid., p. 75. 119 recommendations stemming from the study have particular significance to the methodology of the present study:52 1. Emphasis should be given to the use Of DOT (Dictionapy of chgpational Titles) descriptions in describing specific job Opportunities, and not to DOT classifica- tions, nor to SIC (Standard Industrial Classifications) codes. 2. Task analyses should be made for specific entry occupa— tions in order to deveIOp a series of performance goals based upon common tasks. These tasks should be analyzed and specific training materials should be deve10ped to facilitate the learner's acquiring those skills needed for a cluster of entry occupations. 3. Since relatively few skills are demanded, it is recom- mended that depth interviews be conducted on a systematic basis with immediate supervisors to determine the extent to which those skills demanded are utilized on the job. Perhaps the most important outcome of the SIP programs (aside from the instructional materials which were deve10ped) is the emphasis placed on acquiring data on the requirements of the Office employee in the actual job classification and translating this information into instruc- tion to be used in the actual training program. The Vocational Office Block Program The impact Of the nationally recognized Vocational Office Block Programss deve10ped in the Research and Deve10pment Program in Vocational- Technical Education of Michigan State University increased as the need for continued deve10pment of career-oriented programs (high school and post—secondary) became the focus for providing individuals with better preparation for office occupations.l The block-time program in vocational 521614., p. 78. 53Poland and Haines, Op. cit. 120 office education is a system of developing Office competencies under simulated office conditions. In order to be employable, students must have an awareness of how they will be expected to perform in the world of work. The instructional materials development phase of the VOB project began in the Spring of 1966.54 A skeleton curriculum guide and sample instructional materials were developed for use during the national work- shop in the summer of 1966; and Research and Development staff, research associates, and contract consultants were involved in the further refine- ment and deve10pment of these materials for use in block programs during the 1966-1967 school year. Five states were involved in the consortium (Arizona, Florida, Michigan, New Jersey, and Washington); and 39 schools within these five states were designated pilot schools for the project. Special instructional materials deve10ped for use in the program included:55 1. Curriculum Guide: Integrated exercises, classroom battery eXperiences, realistic laboratory experiences, individual and classroom problem-solving experiences, weekly evaluation guide. 2. Integrated Exercises: Practice in those office tasks which compose a stenographic or clerical job. 3. Model Office: Functions of various work stations of a total Office system. 4. Case Studies: Case problems (108) for transfer of learning to the Office situation. 5. Handbook for Organizipg_and Operating an Office Block- Time Pro ram: Principles, practices, and procedures for the administration of a block-time program. 54Ibid., p. 22. 55Ibid., pp. 23-29. 121 Additional materials were developed by research associates for use their schools. Many of the materials utilized in the project, includ- ; the curriculum guide, the integrated exercises, the model Office, ithe case studies, were developed by research associates and later :seminated among other project personnel. The following conclusions, based upon the results of the three- r study, are pertinent to the present study:56 1. Block-time allows teachers more adequately to meet individual differences by varying amounts of practice, assigning advanced projects, providing remedial learning when appropriate, counseling and guiding students, and knowing and understanding each student. 2. Clinical schools can be a valuable training site for the experienced Office teacher. (N The project demonstrated the value of the clinical site as a way of inducing creative materials from the research associates (the model Office, integrated exercises, methods of instruction utilized). 4. Teachers related that students grasped problem—solving situations more quickly than in traditional organiza- tional patterns. 5. Teachers believed that students made decisions on how and in what order they would do the work in Simulation exercises. 6- The Vocational Office Block resulted in a questioning attitude on the effectiveness of present—day Office education on the part of the profe551on. Some of the recommendations resulting from these research efforts Size the need for efforts to develop instructional and evaluative 57 1315 for use with such a block-time approach: M 56 . Ib1d., pp. 68—69. '7 >Ibid., p. 71. 122 1. Evaluative instruments must be deve10ped which will support a research design comparing the program's effec- tiveness with the traditional one-hour classes taught in office education. 2. Comprehensive research into the theory of simulation as an instructional vehicle in office education is needed. 3. Intensive effort Should be given to producing an employ- ability measurement program for use in Office education. Evaluation was deemed a high priority by VOB project personnel. One of the kinds of evaluative means considered for use in the project was a situational test Of Office behavior. One recommendation from the study indicates how such a test could be used to determine program . 5 effectiveness: If a test of behavior in an Office situation could be pre- pared, it would be a breakthrough in the type of test normally given in "traditional classrooms." Further, it would provide qualitative information as to whether or not the vocational office block concept is doing as effective or more effective job of training Office workers than in the "traditional" single-hour classes. One final note emphasized the importance Of further research in the deve10pment of an employability measurement program.59 The NOBELS Study Perhaps the most detailed study in recent years in the translation of occupational data and information into instructional components for office occupations programs is the New Office and Business Education Learnings System (NOBELS) Study.60 The overall purpose of the study was "to assess and modify learning programs in which purposes are 581bid., p. 50. 591616. 6oLanham et al, pp. cit. 123 preparation for office jobs."()1 The first two Operational phases of the NOBELS study are now complete: (1) the development of design criteria and (2) performance goals for learning programs for learners being trained for Office employment. A situational analysis was con- ducted by Lanham et al62 using a modification of Flanagan's critical- incident technique to collect actual samples of on-the-job behavior evidenced in specific Office activities. Huffman and Brady63 developed a framework for the analysis of these critical incidents based upon a preliminary analysis of office tasks. Huffman also isolated traits and concepts from emerging office occupations and converted these to per- formance goals. Forthcoming phases of NOBELS curriculum deve10pment include these five sub-studies:64 l. The selection from the inventory of educational perfor- mance goals those that are relevant to the preparation of students. 2. The preparation of learning experiences for target students. The application Of the learnings in a classroom. The systematic assessment of the learning results. The modification Of the specifications or their application based on the assessment. (fl-#04 D. Summagy Analysis Of the role of the secretary is necessary in order to make adequate adjustments in the secretarial curriculum, whether it be at the high school or post-high school level. Two distinct phases of 61Ibid., p. 1. 621618. 63Huffman et al, Op; cit. 64Lanham et al, loc. cit., p, 2, 124 development are evident in research that has been done on the role of the secretary: (l) Task-trait analyses wherein primary attention is given to the individual and personal characteristics Of the secretary and her ability to perform those responsibilities of her job, and (2) situational analyses wherein the importance of the work group and the work situation is considered in establishing the adequacy of the secretary's performance on the job. Although several of the studies reviewed in this chapter alluded to a basic recommendation that the results Of the research be utilized in revising and updating curricula, none of the studies have actually been used as the bases for further curriculum development. The Lanham sub-study of the NOBELS project, however, is the basis fOr the development of performance goals for office tasks and appears to be a preliminary measure for the use of these performance goals in further developmental phases. Two other curriculum development projects in the State of Michigan, Wayne State University's Senior Intensified Program and Michigan State University's Vocational Office Block Program, have gained nationwide recognition as block programs designed for the student with a career goal in one of the Office occupations. CHAPTER V THE DESIGN AND DEVELOPMENT OF THE SECRETARIAL IN-BASKET One of the fundamental questions that arises in in—basket research asks if in-basket simulation can be used to evaluate the ability Of an individual to handle specific decision—making required in a particular occupation. One of the main purposes for deveIOping THE SECRETARIAL IN-BASKET is to determine whether or not secretarial decision—making in the setting Of work priorities can be evaluated by means of such an evaluative instrument. The design and development of THE SECRETARIAL IN-BASKET was dependent upon six basic phases of Operation which characterize the methods and procedures fOr this research study: 1. Preliminary Planning: Contacting experts in the field of in-basket research; conducting preliminary investigation of secretarial samples and student samples for purposes of the study. Review of Related Research and Literature: The review of design and developmental theories; the historical deve10pment of in-basket simulation; analyses of the secretarial role; and research in curriculum develop- ment for secretarial education. The Critical Incident Study: The derivation of a classification of problems requiring decision-making in the setting Of work priorities in the secretarial position. The Development of THE SECRETARIAL IN-BASKET: Planning, writing, and refining an in-basket simulation based upon the problems and decisions obtained in the critical incident study. 125 126 5. Field Testing THE SECRETARIAL IN-BASKET: Conducting two tryouts with'selected secretaries and'students, with a revision stage following the first tryout. 6. An Analysis of the Experimental Results: Analyzing the results of the tryouts and presenting the data with an analysis of the findings, conclusions, and implications stemming from the study. This chapter summarizes the procedures utilized in the first four phases, and the following chapter summarizes the field testing and the analysis of the data. The methodology for the study was dependent upon both a theoretical approach and an Operational approach to the design and development of THE SECRETARIAL IN-BASKET. The combination of these two approaches was essential in, first of all, developing a theoretical base of understanding resulting from a thorough review of related research and literature and, secondly, develOping an Operational under- standing as contributed by experienced secretaries during the critical incident study and subsequent reviews. Each phase of the study is summarized here with specific data pertinent to the study. A. Preliminarnylannipg_ Preliminary preparation for the in-basket research involved two basic operational stages: the establishment of initial contact with developers of in-basket simulations and the establishment of initial contact with the National Secretaries Association (International) and the Institute for Certifying Secretaries. A number of experts in the areas of personnel management, management training and deve10pment, testing and evaluation, manpower planning and research, and teacher education were contacted because of their work in the development of in-basket simulations for various selection, training, and evaluative 127 purposes (see Appendix A). ReSponses from these individuals included helpful information pertaining to their own research efforts as well as further "leads" to other in-basket developers. Several of the in—basket developers provided additional research materials for perusal and pos- sible adaptation to the study and enhanced research review conducted in subsequent developmental phases. Personal contact with the National Secretaries Association (Inter- national) and with the Institute for Certifying Secretaries was essential in discovering if samples of secretaries to be used in the study could be selected from those secretaries who were members of NSA and/or who were Certified Professional Secretaries (CPS). The Michigan Division of the National Secretaries Association (International) is composed of 19 NSA chapters, and random samples were selected from this population. The Institute for Certifying Secretaries has one general file listing all CPS's in the State of Michigan, and CPS samples were randomly selected from this pOpulation. As a result of the initial contact with the President and the Executive Board of the National Secretaries Association (International) and the Dean Of the Institute for Certifying Secretaries, arrangements were made whereby samples Of secretaries who were NSA members in the State of Michigan were contacted by mail through the Michigan Division and samples of Certified Professional Secretaries who reside in the State of Michigan were contacted by mail through the Institute for Certifying Secretaries. The three purposes for contacting NSA and the Institute were: 1. To request permission to contact secretaries who live in Michigan and belong to NSA and/or secretaries who live in Michigan and are CPS'S. 128 2. TO inquire about the in-basket utilized in the Certified Professional Secretary (CPS) Examination. 3. To inquire about the in-baskets published in The Secretary, the monthly publication Of NSA. Tentative arrangements were also made for the possible participation of secretaries from these two groups in the final in-basket tryout phases. B. Review of Related Research and Literature The foci of the review of related research and literature related to four specific areas of Study in order to adequately evaluate the information and data Obtained with its degree of relationship to this study: 1. Desigp and Developmental Theories: Selected reviews of research and literature pertinent to the deve10pment of in-basket simulation; design and deve10pmental theory from areas of education, tests and measurements, psychology, philosophy, and other areas of concentration. 2. The Historical Development of In-Basket Simulation: The use of in-basket simulation in research, management train- ing, and secretarial education as an instructional, training, and/or testing procedure. 3. Analyses of the Secretarial Role: Research pertaining to the role of the secretary and delineation of personal traits, duties, reSponsibilitieS, behaviors, and/or tasks performed. 4. Research in Curriculum Development: Application of instructional models, implications for instructional materials and evaluative instruments in the deve10pment of secretarial training programs. Such a review of research and literature provided an understanding of 211 elements involved in the deve10pment of in-basket simulation and in the validation procedures involved in the eXperimental phases. The review also provided the vehicle for relating the historical use of in-basket simulation in fields other than secretarial training in an 129 attempt to evaluate its adequacy in future applications in the design and development of instructional and testing "packages" for secretarial education. C. The Critical Incident Study Flanagan's critical incident technique1 was modified for use in the preliminary search for the kinds of typical but significant decisions the secretary faces in setting work priorities during the routine busi- ness day. Several Operations were necessary in conducting this prelimi- nary study in a research-wise manner and may be summarized in this way: 1. Preparing the letter-decision card format. 2. Selecting the samples of secretaries to be used in the study. 3. Cataloging and classifying the critical incident responses. 4. Conducting a preliminary review of selected incidents. 5 Selecting the problem categories for THE SECRETARIAL IN-BASKET. The critical incident study was conducted during the period from February through May, 1970, in the hOpe that a sufficient number of incidents would be collected through this procedure to incorporate up to 20 problem categories in the final version of the in-basket. Preparing the Letter-Decision Card Format The packet of materials prepared especially for the collection of critical incidents from secretaries included: 1. A letter addressed to the secretary explaining the research study and the need for critical incidents on decision making. 2. Decision cards (20) for the secretary to complete and return. 3. A complete set of directions for completing the decision cards and for their return. 1Flanagan, "The Critical Incident Technique," pp. 327-358. 130 4. A letter addressed to the secretary's employer explaining the purpose of the study and requesting the secretary's help in securing data. 5. A return envelope for the return of the decision cards. The decision card was the central focus for the collection of critical incident data. The sample decision card shown here indicates that the kinds of information requested included the day of the week on which the incident occurred, the time of the day, a brief statement of the problem, the secretary's decision (if any) for the problem, and the frequency with which the secretary would anticipate the critical incident recurring. Day of week: cede No. Time of Day: Brief Statement of Problem: Your Decision: Do you consider this problem one that is likely to occur: Daily Monthly weekly Occasionally Illustration 2: The Decision Card for Recording 3 Critical Incident The purpose of the letter addressed to the employer was basically to increase the chance of a returned response since both the secretary and her employer would receive information pertaining to the study. In several cases the secretary was permitted to spend working hours on completing the decision cards as indicated in letters received. 131 Selectlpglhe Samples of Secretaries From the membership roll of the Michigan Division of the National Secretaries Association (International), a sample Of 50 secretaries were selected by means Of a stratified random sampling procedure initi- ated by the president Of the Michigan Division. The secretary to the Dean of the Institute for Certifying Secretaries initiated a mailing from the Kansas City Office to a randomly chosen sample of 50 secre- taries who are Certified Professional Secretaries. The number of usable responses from the first mailing was 11, seven groups of decision cards received from NSA secretaries and four groups of decision cards received from CPS'S. A follow-up mailing was initiated to an additional sample of 50 secretaries who were members of the Michigan Division of NSA and an additional sample of 50 CPS'S. In addition, there was one follow-up mailing to each secretary not responding to the initial contact within one month of the first mailing. The total number of responses was 68, and 20 of these responses were usable. Among the reasons given for non-participation in the critical incident study were the following: I am involved in tOO many other commitments at the present time to do justice to your project. I am no longer employed as a secretary. ... Since I have been functioning in a supervisory capacity for the past several years, I feel that it would be inappro- priate for me to participate. .My present job responsibilities are such that I am sure they would be of no value in your survey. The total number of critical incidents collected through this pro- cedure was 181, and the total number of Secretaries participating in the critical incident study was 20 (ll CPS'S and 9 NSA secretaries). 132 Cataloging and Classifying Critical Incidents Each critical incident was examined and classified according to the specific problem and decision factors inherent in the incident. The secretary's statement Of the problem was analyzed for pgpblem factors which constituted components of the problem. In some incidents there was only one problem factor; in others, there were several. The secre- tary's decision in handling the given problem resulted in one or more responses which are termed decision factors for purposes of this study. The specific problem factors Obtained through the analysis of each critical incident led to the formulation of the system for problem identification, whereas the Specific factors contained in the secretary's decision indicated response patterns leading to the formulation of a problem-solving technique or answer. The illustration on the following page (Illustration 3) shows the progression from decision card to sub- sequent analysis of problem and decision factors with ultimate problem identification (input, processing, and output orientations) and deter- mination of response patterns. As indicated in the tabulation of all problem factors identified through this analytical process (Table 1), the input-processing-output pattern resulted in a classification of the problem factors in three divisions: incoming communication, processing, and outgoinglcommunication. The 181 critical incidents classified according to this scheme yielded 415 problem factors, and those problem factors receiving the largest number of tallies were utilized later in the development of the actual in-basket items. The classification of decision factors resulting from analysis of each secretary's decision(s) concerning a specific incident was based 133 Day of Week: W— Code No. SnIE-l Time of Day: 3 Earn. Brief Statement of Problem: Eagle . '4 .4 I ‘4’ “(IA 0 :61 '1I1'a4a 04‘ ‘. 0|.t: 71' \' ‘ ' .II I - . Ol.- . . o l _ . c ‘ Your Dectsron: . v q a q 0 l ,.¢1 ._ l ; I: Do you consider this problem one that is likely to occur: Dai ly Month ly g WPekly Occasionally PROBLEM FACTORS DECISION FACTORSa r--1. Preparing Business Papers-- 1. To type labels. I Typewriting 2. To stuff enclosures in : envelopes. "' I | F2. Outgoing Written Communica- 3. To send mail. ' |F tion--Mail | | l l.____.... _______ a, ' INCOMING COMMUNICATION PROCESSING OUTGOING COMMUNICATION : . . . . l 1. Preparing Bu51ness 2. Outhing Written ' Papers—-Typewriting Communication-- | Mail I __-—-_—-—-_-J I DEVELOPMENT OF RESPONSE PATTERNS;] Illustration 3: Analysis of Decision Card for Problem Identification and Determination of Response Patterns aThe verbs used by secretaries in submitting critical incidents were modified in order to have continuity with the Huffman et a1 taxonomy of office activities. 134 upon the Huffman gp_dl_taxonomical framework for analyzing Office activities2 into those actions required in the Operating, interacting, and managing domains. Action verbs contained in the taxonomy were used in analyzing each response factor. The total number of response factors recorded was 685. The reason fOr this large number of responses, com- pared with the total number of critical incidents collected (181) and the total number of problems within those critical incidents (415), is that for each problem factor identified there may have been no immedi- ate response by the secretary, one response only, or a series of responses that were readily recordable. The main purposes fOr such an extensive analysis of decision factors within each incident were two- fold: to find out exactly what kinds of decisions secretaries make to solve given problem situations and to see if there appeared to be any basic pattern to the responses fOr given types of problems. This cate- gorization of re5ponses was later used to formulate a five-point scale of quality responses for each of the items developed fOr the in-basket. Conducting a Preliminapy Review of Selected Incidents Two types of review procedures were established fOr a complete review of the critical incidents and the analysis of these incidents into problem and decision factors: 1. [be random selection Of 15 critical incidents for review: The incidents were duplicated for transmittal to four secretaries for review (two CPS's and two NSA secretaries) to determine if the reviewers' responses to the problem situations would be somewhat identical to those given by the secretary submitting the critical incident. 2Huffman et al, Op. cit. 135 2. A review Of the final classification of problem factors: With the assistance of one CPS, one NSA secretary, and7 one business educator, the classification system utilized fOr determining the problem and decision factors was reviewed. This review served as an intermediate step in the analytical process and as an additional source of information relating to the adequacy of the system by which the critical incidents were analyzed. The judgmental answers of the two CPS's reviewing the 15 selected incidents agreed with those Of the secretaries who submitted the incidents in 11 of the 15 incidents. The judgmental answers Of the two NSA secretaries reviewing the 15 selected incidents agreed with those of the secretaries who submitted the incidents respectively in 14 of the 15 incidents (the first reviewer) and 10 of the 15 incidents (the second reviewer). In the review of the final classification of problem factors, the three reviewers agreed that the classification system had been satisfactorily utilized in the study and that the 20 problem cate- gories with the largest number of tallies should be utilized in the development of the in-basket items. Selecting_the Problem Capdgpries for THE SECRETARIAL IN-BASKET The classification of the secretarial problems into Specific problem categories yielded a total Of 415 problems (Table 1). The three main problem groupings were: incoming communication, processing responsibilities, and outgoing communication. Incoming Communication. Those problems classified as incoming communication constituted 20.5 per cent of the total number Of problems, or 85 problems. Those problems classified as verbal incoming communication 136 m.om mm onH u- onHmmm=m v N CH mwcHuooz use neoconomcou wchemuu< .N m N NH mucosucHomm< mmocham mestpocom .H m.m N.oH NN mhzm2mozom pee weHqum .m HH mereUHHmso pee meesaou .N o wchomsou .H m.oN m.mm mm mammonmm< .m N. v.H m mcoHumommsm no mcoHumpcoeaooom «nanommo .N m.H N.m m mEoHnOHd o>HumemHeHEp< mcwanmc< .H m.m m.o eH mzmqmomd m>Hhmmdsm .m Hposereouv mmHeHsHmHmzoammm oszmmuoma .HH Hench mo cfigqu msoHnOHd , ommueoouOd ommucooe6d mo Homesz mowuowoumu aoHnoad Heosereouv H mHmae 139 N.NN mHH oneHHoo Haeompoa .e o.H m.m e mowemmoz .m N.H H.e N 66H66H69-eeam6Hpe .N e.m o.mH mH HHez .H e.m m.mm mm omomoumm--one--oneHHoo Hecomuod ”amoaoumm .onH .onamma=m .m m-e .<-e m.e mN . weHeoeeoHoe ”Hamxm> .onH .onHm mmOHU .oneHpapHmHeHap< weHNHHeea "mstmoaa m>HHmma=m .NH m-oH .<-6H e.N oH HoNoHdam eeHz uH .onNmma=m .mH m4; €13 «N S magnum: new neoconomcou weflmcenusq "mezm2muzmma=m .mH m-NH .<-NH N.N HH mereoHHdsa pee meHHdou "mamaHoOOH omemmoa ozone m o» NHmoH Op EH: Hmoacon pew O>Hueueomoumon «one on» oeocmoHou ob choaxmm .Hz How omemmoz ozondoHoh "m-H EouH .oucmumenmcoH pea HmOOH neon .mHHmo ecosmoHou .coogoeaH axon um xmomm on NHHHHAN=H meHomuso mo peoaoode one o>HO>cH :Oan mxmee m.pcoaxmm .Hz mchuooeoo coHueuflemmeo m.:oaoz HmOOH a mo peopHmoum Homoeou OH fimdflcozmoHohu "83.426 Nunez 3 2258 Had sod 656: 2H an: imam.) .onSOHzaeHou 658.50 "H fiancee maouH uoxmmmncH mo coHumHaomon moHHomoumu aoHnoud «Hm¥mmo mZmHH memHom1=N mo Soap NmHuoc ou HooHHp OH Numuouoom one oufisvon AOHsz mxmmh HmHHmo mecca mcHHpcm: cH mouspooonm mo moHumuouoom HHm Nmeoa OH "nowheuouoom Aond nomad mewuoonfiov HH< on mNNHm pnezox acne aspeepoauz u<-m aoeH monemma=m "m Nmmmmueu .:H moaoo on can: pcoexem .H: on ommmmoa on» O>Hm ON "pcoaxmm Hand How omemmoz ozonmoHoh "muv EOHH .HHmo some on O>HumHou :oHpmahomcH ponwswon on» O>Hm on can mHHmo OnonmoHou mcfisoocH .oonmo on» Hozmca Op Nnmuonuom may oanvou goes: munch ueH moaoo on enemas op Op mama» OHmHuomm 050m :0 :OHueauowcH Nnmuouoom onu O>Hm ON nwcficocmoHoev coexmm Home Scum HHeu ozondoHoe mafiaoocH "<1v aouH H .onNeo oumHnmoudmm pom veoaxem .Hz Na madman HouuoH mmaum can vHom ON HeouchHON .m :H>Hmz Op nouuog mmochzm "mum souH .Nuo>HHop Hem HHma .HHea coo:96pmm on» :H use mnopHonxu6um anemone Op Numuonuom new ouHacou nOan mxmmh one on mcHHHma xHan a mchuom :« .mowumu -ouoom Hague one we one .mcquod NHHOH nHHezV umHmmm ON ”mnopHonxOoum ou mcHHHmz stm HHm maouH Hosea pew needed mmocflmsn HmeoHqupm oHpee: .noNOHmao one gufiz :owumuHcaaaoo Henho> ON upcoexmm Heed scum maouH HanoHqup< “muN aouH wcH>Ho>eH mxmmu on Numuonoom one mo noncommom .xoo: mom mucmaucwommw mo umHH a axes OH aHoNOHgam :pwzg HH8388 Nunez on ucosxmm Head soum 036: H .onNHanm OH eon: HouuoH NHo>HHoo HmHoomm a oHpcm: ON "HHmz Nuo>HHoo HmHoomm "muo EouH .coHuanupmwp HHea oHpea: Op Numponoom on» oqucou sown: mxmeh .po>Huue nmsH 6>ee peep HHee mo mouuHa «Hope; HHHezc pee .6panHHome .Hpom as "Hsz ueHeuoz u<-e aopH omomoumm .chNma onomon xmpfium ”sommeHu sameneoaoz .AHo>Numpcou unoEucHommm OHspoaom Op Hpcoexmm .Hz Op HouuoH O>Hm oh "unusucflomm< mcfipmoscom ..ocH .moumHOOmm< cosmomom Eoum Houpoq .moumHoommm .nmomom pH>wQ oHSpozum OH New umozwom m.ouoo: .o :co% mo oco How ucospcflommm mmocstn m ecosxmm Home new: ucosucfiomm< .Ho>mumco op osmem Ho>HHop OH ”:0mmeHo Nunez Op pcosxem Hand Eoem ezpcmnosoz .ESpemH loses HmecopHmcoo Ho>wHop ON ”mmmfim phase: Op pcosxmm Heed Seam fizpcmsosoz mEOHH poxmemncH mo :oflumfluomoo .NOmHSOOw How mumHHOmcme xooco on no .ommu :ofluwHOHp m HmIHH EOHH othomcmHu Op .mouo: pcmgphocm onHHomsmHu ou NHmHOHOOm on» OHHDVOH :Och mmeH HmerHHomeeeNv mmmaHHop OH Newposoom on» OHHscoH :Oflcz mxmmh ANHO>HHOQ Hw:OmHOdV H oOHmmo .NovOH mvaom ozone Op awn» mmocfimaa m.e:OaNom .Hz :6 :OHuoauom:H o>Hm Ob .mmochsp a.uoNOHmao no: meruomoH HoHHoo no nouHmw> o O» poo o>Hm Ow pouuwauod "mumH aouH mH ogm :OHHosuomcH me chx one ucsoao on» Hoeucoo Op Nuououoom one onesvoh :OHcs mxmob fiond :oHposHomeH mcHHHOHuaoov .HoNHm mo moHdoo Nhommooo: one ouomonm Oh "HoNHm Op vogoouu< vcoaxom Hand scum aspeouoaoz ueoauuodoa meHuesooo< scum HHou oeonmoHoe u<12 souH monNmmd=m "NH Nuowouou .ahwm Ou pommouppo :OHHoOHHamo mo . . umommoooum maouuoH How NOHHOQ Neomaoo powwow soHHom mcHospoudou no .meHuooHHgsv .mcwxmooouonm OH "sewuooNHdm< mo Houuoa mo umHooom "muNH aouH msofiuo> Np mueoasoop mmocfimsn.mo monoo onomoum Op Nuououoom on» ouflzcoh gown: mxmok HmeHupuHHasa pep meHHaooc mxmaHuonuchHEpo onfioomm onNHoco Op Numpoaoom one oHHDon :OHgs oxmoe .mHo: HoHHmuoeoom one moHHmmsm oOHmmo :o moHDMHm Homers nmEoHnoud o>HuoHpmflcHEp< mcHNNHocHH§NmHzHE< oszHimgm "NH Demons .pouonEOo on Op mxmou Hoao>om mzuopouoom osuv :OmxeoHu Nocoz new: ob upcosxom Hand Np :OHumsnuoueH ”mnoH EouH .poEHOMHom on Op poo: Lafinz oxmou on pnowoh :H HoNOHmEo spa: COHHoOHCDEEOO uooHHo .oHQHmmom mo News mo oHeco: OH pooooum pee xmop :o mEouH “How OH mHoNonEm spflzv “comxsoHu Nunez OH pcosxom Hood anm ouoz " .onHmmd3m "mH Naomouou meouH uoxmom-:H mo :OHumfluomoQ moHHOMopou EoHLOHm .uonusw ozu scum vocwmupo on xms mawwuouoa AHMwwaam Ho maoufl oflmwuomm :o h0\v:o mem on» .mcwxmo: anon uoonm oe "noufimw> oofimmo ”muom aouH .ahflm a“ muosuo can xumuonuom ogu .ocwuaon onwmmo nu“: mfiomuo: on“ :oozuop coflumowcsaaoo o>~o>cw scan: mxmme unawawadm on novno cw Anouonoom m.v:osxmm .nz x9 umofl came on» coca op ”comxuafiu may“; no muocpo :pflzu xocmz ou axonm Ana: scum asncmnoaoz "<-oN aopH 4 .onamuu you muawouon ops nfinmonmmw ouwuoH ch "vo:H:uom ucosuuomoa .mQOwuuwmcmuu mo ovaouou oumASooa causaouu< on vcosxmm anon Scum asvcouoao: "muma aouH moox ou novuo aw newpmahomcw Howucocflm vacuon ow xumuouuom on» ouwsvon scan; mxmmb .xumuouuom noguocm x9 vomonouam moMHQASm ouflmmo ogu wuooou 0% among Howocmcfim mo mvnooom wcflcwmpzwozv A "vomazouam moflfimmam oofiomo you uaflouom "<-mH aopH mnmoumm oqugompzou oz< oszHuomoh on“: Oh ”mfihh “sag: oh vcoaxmm non mommmmoa omxuoaou\msmhwoaou vcom wad Hang on x099 House“: scam aavcmuoaoz "mswa aouH omomaoo cu xuououoom on» oufiawon sows: mxmoh .xavwnm msow>oum on» vo>fioooh auumofiou nomxuofiohuamnuomohv 0» xamon onw3 ch "amumoaoe mo umwouom u<-mH aouH amnaoumm .onh6fi mo. 6:5 56 .o mmoao ou pofifiom Hm>Houcfi oucopfimaoo och ”mcfipcflm . cm . coospon 07:6 00 + w vmwm m + xfifimn—oam mum ‘7 THIN. m I :oos o +.IU mm.oom Nn.N mHI¢V\ + noo.om I om.moHU u a Afinmaoam ma «7 mh n. mam—<9 169 .mouoom omcommoh dance on» co seaweed xooH oofim 0 one ca mucopSHm ocu pew moflumuouoom oH mo. on» um oocouommflv ucoofimfic am a ma owes» .onomowoch .o mmowo on woafimm Hm>houcfi oocopfimcoo one "wcfiecfim . co . coo3uon mamv O? + v mwma m + XHDMDOHQ mw } mm a . . - - . I . 8652 ml + .ch mm 63% 2 N 3 $3 + AS 2. 8 at 3636.8 2 qmww u oH mo. onu um oocohommflp ucoowmficmfim m we ouozu .oaomouoce .o moose ou poflfimm Ho>noucfi oesopflmcoo one ”mcfivcflm . cm . coozuon News we + m wmnn A: ... ~353me mun } .DN .M . . I I . I . coozuon r I as am 8% S. N 2 EX + as. 06 om mo: began 3 \r ”WI Hm u men an comfinomEou .monoom omcommou Hmpou on» so seaweed xoofln oofimmo one ca mucovSum ogu paw m.mau on» coozuon Ho>oH mo. on» no oocoaommfip ucooflmficwfim a ma ouosu .ouomonoch .o moose ou voafimm Hm>houcfi oocopfimcoo och ”wcfivcfim mposcfiucouv qmwm u mdu "N comfiuomEou Aconcaocouv a msmo~ mo. opp um oocouommflp ucmoflmficmflm m m“ ouonu .ouomouose .o mmoao on noafimm Hm>noucfi oucopwmcoo one "wcflpcfim . . coozuon moon I 28 meme mm I xflnononm ma } I C003 0 a.m.m Immc $.68 NNN 2-35 I 5..vo - N53 inflowmnfl > .6 mm Hm u em no cemflummamu .onoom omeommon Howey on» :o mommmdo voflhomIoncwm ocu :H mucopzum ocu pee mofipmuouoom oH mo. on» no oocouommflp ucmowmflcmflm on ma onogu .ouomohose .o vommoao Hm>noucfl oocopfimcoo one "wcwvcwm . . :oozuon Nmem em + new mmz m I Sponge ma } S. N . . I - . - . c6653 a.m. I mi mm 89% N N S is I 3,. oo 8 ob 3632a 3 AU. hm u Ho>:fi mmoooum one oom op oHnm on NH“: novmoa on» page om moonwaomeou mo macaw pmuwm own» new xaco woueoooam ma mpfleflfi oocopfimcoo one m:H:HEHouop pea mcomwnoaeoo on» wcwow you now: vocuos ocu one mcomfinmmsoo oog umoa.wmmocom wcwusmeoo new masenom ogbm .ouoom omcommon Nope» on» no avocfineoov mownmuouoom oH mo. onu um oocoaommfiu ucmofimfimwwm a ma oposu .onomohone .o mmowo ou vofiflwm Ho>uoucfl oocopflmcoo one "mcavzam meosefiueouc sumo I mwm I ozo: .mvo.v mucousum noofin oofimmo on» pee mommoflo poauomIoncam cw mueov Iaum one song Amo.v nonman woven mowuououoom one .ousuocmfim m oufiscoe pony cofiufiuom pew espconoEos < wm momcommon one we :ofluonflsoxo c< .xuwnwnm peonunonm oufizcou op xonu omsooon uoxmonIc« one :fi umoH oeoz mono: pnonunonm one .moweouoeoom one son» nonmfin 5H. ouo: munopsum xOoHn one .osom one xfioumEMxoammo pomeuo>o nounopsum pofinoanHmcwm use .munopzum nooHn .moflumuoeoomv mmdonm oonnp HH< 200 .ponweomnmnu on pfisonm omonu zon ovfloop pose :xuouonoom: one .ponfieomnouu on on mono: :oflumu0flo mIHH EouH .oefiu one we ouomfioaueom unsoo onoom mo msoum o no Hmsvfl>fivnfl one pone :Hmuuoo on cu m unease on Henna popuoooHIoe oamu Hoeunoo owesm on» one pomw>on onoz mcofiuoonfip onu .oaomoeone .poufimonQEo on on no oo: nomad mmocfimsn Neo>o one :ofiumsunoucfl pocnoam mmo>o wcflpaooon mo ponuoe one .ufl oHpnon on son opwoop ooze :xeouonoom: one .Hfioa mcfichoe on» wnfi>no>cfi cowumannoucfi ponnoam < om no :oflmwooo EopH mo cofiumwnomoo N RDOSmP 0H monm onmH>mm oszHaomm Hm¥mmh mm mnmo noon ou oHno wnwon Eonm onoom mo macaw o no nomaom one gamma“ paces umnp ..ouo .omfion pnsoam Ixomn mo HooHo AHoOSHOmnm ma ow pone om povhooou Ion code we; we no mcofiumsnaopnfi and new: one» Heepnoo ofipso one .poNNmonmEo we Momma mmonemsn Npo>o can nofipmsnaounfi N~o>o wnfipaoooe mo ponuoE one umnu om womfi>oa on ow powoon mnofiuooeflp one IIIIIIIIIIIIIIIIIIIII . .Honpnzm we omfi>oe on won one Eoufl one o>ooH ou opus mm: cofimfioop one .oaom Ioeonh .pw ow pow ou oEwu o>on no: pap mpooflnsm oEOm pan .omoHo>co penny on» :H pouooofi ma mINH EopH .o>fiuonfiafieomfip oHoE noon o>on eases memos one mmmnaom .mu omoHo>nm :fl soon on» mo :ofiuoooHv Eoufl one on :ouuom won mpoonnSo oHoE mH .mmsoew noodn cam xeouonoom one noon ca Hamem mo: 2 one IIIIIIIIIIIIIIIIIIIIIIIIIITII Ede ca noofimfi> .pcoEocnomde man now o an noecafim < ooammo no no“: nonumsneou IIIIIIIIII mION EwuH illilllllllivl‘llil‘l‘llll .poeo>fifiop pcm pooh on on mpoo: pone Houpofi Hofipcomflmnou IIIIIIIII NINH seeH .mmzoam 03H Honuo one can» nonwfin money can mofleouoaoow one oonfim nonunsm we omfi>on cu eon paw Soon on» o>moH on oven was nofimfioop one .mEopfi Honpo can“ xpfieoeam noon soon on» o>flw op uoomnSm m uwanom pasoo noflnz < omoHo>cm an popmoofi no: mafioe manages one an pouooofiv soon one meeeeeeeeov om no nowmflooo . EouH mo :ofiumfiaumoo fleeeeeeeooc wN mam w >-u1> w >-w > w:> l air-u1>-w > I l >-uv> w > D—II—‘r—JI—‘P—l NHHCIDOLQQ p—J N I Put—INNNI—INbr—Ir—Itan—INI—II—INHMNNHHNHHHNHHHHNMNNNHNH Experts' Opinion CPS’s Opinion l—‘D—IDJNNI—‘N-bh-II—IMMi—‘NHHNHNNNHHNHHI—‘NHI—JHHpmNNHHNH + + I I—‘OOCCCOF—‘OOOOOOOOOOOOO Ic>o I 5.9 N NOOOHQOO HOOOOOOOO d2 lo 0 H13 0 o o<3¢3<3 c>h-o C’C o 010 H o o o o c>c>c>o C’O c>c>o e-o o c>h-o C>c S Hypothesis Testing Ho:z’9=~/“a. Hid-HA Assumptions: a. Random sample of differences b. Normality of the differences Test Statistic: 2 2 t— E—Ai sd2=id—Tj%fitgjqil— .A 5d l?— Decision Rulezcx3= .05 Reject H if t é—2.02 or if t il+2.02. Comppte the Value of the Statistic: 15:12 = 532 _ (£102 n(n—1) 42_408)—(—22 :‘d ‘ “To—(“393 h! ' 2 — — sd — .2025 sd — .45 t = d :g‘% = -.05 — 0 = -.7022 ————-———- .45 A ......— sd J40 JD Decision: Fail to reject Ho. Conclude: A = A There is no significant difference between the opinions of the experts and the opinions of the CPS'S on the importance of the in-basket items. 237 TABLE 50 A ONE-SAMPLE T TEST OF THE DIFFERENCES IN OPINIONS ON THE IMPORTANCE OF THE IN-BASKET ITEMS AS SHOWN BY ANALYZING RESPONSES 0F NSA SECRETARIES AND THE PANEL OF EXPERTS In-Basket Experts ' NSA' s 2 . . Item Opinion Opinion d HypotheSis Testing l-A 1 2 —1 1 Ho:/¢=./¢._ 1-B 2 2 0 0 _ 2—A 1 1 0 0 H1 'A’éfl‘ 2-B 2 2 0 O Asspmptions: 3-A 2 2 0 0 a. Random sample of differences 3-B 2 2 0 O b. Normality of the differences 4—A 3 4 -1 1 . . , 4-3 2 2 0 0 W' i i 33:17.84 M .212 _ =__T._(__12 _. 6-A 1 1 0 0 sd “ n 1) 6-B 1 l 0 0 7-A 2 2 0 0 J n 2:}: i i 8 8 Decision Rule: ‘- = . 5 Reject Ho if t6 -2.02 or if 8-B 1 2 —1 1 t , +2 02 9-A 2 2 0 0 "' ' ' 9-B l 1 0 0 Compute the Value of the Statistic: lO—A 1 l 0 0 A 2 _ 2 _ . 2 lO-B 2 2 0 0 sd ‘ “2‘1 n n_ d1) ll-A 2 2 0 0 A 2 _ 2 ll-B 2 2 +1 1 5d ‘ 40(8) ' (‘4) 40i39i 12—A 1 l 0 0 A 2 A lZ—B 2 3 —1 1 Sd = .1948 Sd = .4413 13—A 1 1 0 0 _ 13-3 1 1 0 0 It'd-A =-.1 -0=1.4326 14—A 2 2 O 0 ————————— .4413 14-B l 1 O 0 5d ’4-0- 15-A 2 2 0 0 —-— 15—B 3 4 —1 1 Jn 16-A 1 l 0 0 Decision: Fail to reject HO . 16-B 1 1 0 0 —-——-—-- 17-A 4 4 0 0 Conclude: ~/‘I= J‘z. 1:2: i i (0) 3 There is no significant difference 18-B 2 2 0 0 between the opinions of the experts 19-A 2 3 -l 1 and the opinions of the NSA secre— 19-B 2 1 +1 1 taries on the importance of the 20—A 1 1 0 0 in-basket items. ZO—B l 1 _0__0 f 8 238 £_test shows that the value of £_was computed at +1.4326. The decision rule at the .05 level of significance stated that the null hypothesis would be rejected if the value of £_were equal to or greater than +2.02. Therefore, the decision was made not to reject the null hypothesis; and the conclusion resulted that there is no significant difference between the opinions of the panel of experts and the opinions of the NSA secre- taries on the importance of the in-basket items. The mean differences between the opinions of the Michigan Bell secretaries and the panel of experts on the importance of the in—basket items were tested by means of a one—sample p_test to see if a signifi- cant difference occurs between these two groups. The null hypothesis for this test was stated: "The MBT secretaries and the panel of experts do not differ significantly in their opinions as to the importance of the in-basket items.” This null hypothesis may be represented in this way: (11) H0: MBT Secretaries = PANEL 0F EXPERTS The alternate hypothesis was stated: "The MBT secretaries and the panel of experts do differ significantly in their opinions as to the importance of the in-basket items." This alternate hypothesis may be represented in this way: (He) H1: MBT Secretaries 7e PANEL OF EXPERTS Table 51 shows the comparison of the opinions of the panel of experts on each of the in-basket items and the opinions of the Michigan Bell secretaries on the same in-basket items. A one—sample p_test of the differences between the two groups shows that the value of the 3. statistic is -.9025. The decision rule stated that H0 will be rejected 239 TABLE 51 A ONE-SAMPLE T TEST OF THE DIFFERENCES IN OPINIONS ON THE IMPORTANCE OF THE IN-BASKET ITEMS AS SHOWN BY ANALYZING RESPONSES OF MICHIGAN BELL SECRETARIES AND THE PANEL 0F EXPERTS In-Basket Experts' MBT's 2 . . . . t Item Opinion 02}n10n d HypotheSis Tes ing .- ‘ “03”,: It - ”19‘9’9“; : Assumptions: a. Random sample of differences b. Normality of the differences Test Statistic: t d&42_ 2- .22 d n n-l d R" Decision Rulemc = .05 Reject H if :6 —2.02 or if C>C>C>P‘F‘C>C>C>C>C>F‘F‘C>F‘C>C>F‘C>F‘C>C>C>C>C>C>hIC>C>C>C>C>C>P‘F‘C>C>C>C>C>h‘ hiF‘CJC>C3K>a>a>\l\l?‘O\UIUlh-h-Ulblk)k)hihi u!>-u1>-u1>-u1>-ul>-u:>-u!>-u!>-u1>»ai>-ua>- HI—‘NNNHN-AHHMNHNt—Ic—aNHOGNNHHNHHHNHHHHNLNNNNHNH bib!h)oahlhak>h-h‘h‘n301hih-hihaoih‘n)k)h)h'hik)hik)hik)h*h-hih'uah-k)k)k)hlk3k) C>C>C>h‘P‘C>C>C>C>C>F‘F‘C>H*C>C>F‘C>h‘C>C>C>C>C>C)F*C>C>C>C>C>C>h‘F‘C>C>C>C>C>F‘ : t i+2.03. 1 - Compute the Value of the Statistic: l - J\:2 2 2 1 - 5c1 = -————-M ' i)— 1 + n(n-1) - I\:Z 2 lZ-A sd = 40(11)p; (-3) lZ-B - 1‘ 40(39) 13-A 2 ‘ A _ 13-B 5d - .2762 5d _ .5255 14-A + _. l4-B t = d 741d = “0:35; 0 = -.9025 lS-A - T—S - 15-B + d [4'0" 16-A l6-B J“ l7-A Decision: Fail to reject Ho. 17-B 13"A Conclude : fl, = A 13:: : There is no significant difference 19-B between the Opinions of the experts ZO-A and the Opinions of the MBT secre- 20-B taries on the importance of the in-basket items. 2". ‘— 240 iflp is less than or equal to -2.02. Therefore, the decision was made not to reject the null hypothesis; and the conclusion is made that there is no significant difference between the opinions of the panel of experts and the opinions of the Michigan Bell secretaries on the importance of the in-basket items. The mean differences between the opinions of the students in the office block program and the panel of experts on the importance of the in-basket items were also tested by means of a one—sample E_test to see if a significant difference occurs between these two groups. The null hypothesis for this test was stated: "The students in the office block program and the panel of experts do not differ significantly in their opinions as to the importance of the in-basket items." This null hypo- thesis may be represented in this way: Block The alternate hypothesis was stated: "The students in the office (12) Ho: STUDENTS = PANEL OF EXPERTS block program and the panel of experts do differ significantly in their opinions as to the importance of the in—basket items." This alternate hypothesis may be represented in this way: (12a) H1: STUDENTSBlock # PANEL 0F EXPERTS Table 52 illustrates the experts‘ opinions as to the importance of each in-basket item as compared with the opinions of students in the office block prOgram and shows the differences per item. The one—sample £_test of the differences shows a computed E_statistic of +2.1572. The decision rule at the .05 level of significance shows that the null hypo- thesis will be rejected if E.is equal to or greater than +2.02. The computed p_statistic is greater than +2.02. Therefore, the null hypothesis ——* 241 TABLE 52 A ONE—SAMPLE T TEST OF THE DIFFERENCES IN OPINIONS ON THE IMPORTANCE OF THE IN-BASKET ITEMS AS SHOWN BY ANALYZING RESPONSES OF STUDENTS IN BLOCK PROGRAMS AND THE PANEL OF EXPERTS In-Basket Experts' Students' 2 . . Item Opinion Opinion d d HypotheSis Testing l-A 1 1 0 0 H0:/€=./‘1. l-B 2 2 0 0 2-A 1 1 0 0 H1 'A75 A 2‘8 2 2 0 0 Assumptions: 3-A 2 2 O 0 . 3-B 2 2 0 0 g. Randomsample pf giggerences 4-A 3 2 +1 1 . Normality of t e 1 erences 4-B 2 3 -l 1 Test Statistic: 5-A 1 l 0 0 _ S-B 1 1 0 0 t = d -/(d =_"ZE_____2' 6—A 1 l 0 O T—4 n——-——(—-—-n_ l)i-—— 6—B 1 l O 0 5d 7-A 2 1 +1 1 7-B 1 1 0 O J n 3:: 11 g 3 1) Decision Rule: 0C: Reject H0 if t6-2.02 or if 9—A 2 1 +1 1 ta- +2. 02. 9-B 1 1 O O lO—A 1 l O 0 Compute the Value of2 the Statistic: lO—B 2 1 +1 1 2 2 ii) ll—A 2 2 0 0 Gd: rid n(n—1)i— Il-B 3 2 +1 1 2 2 l2-A 1 1 0 0 9d = 4009) - (9) 12—B 2 1 +1 1 40(39)A 13-A 1 1 0 0 A 2 _ _ lS-B 1 1 0 0 5d - .4352 5d — .6596 l4-A 2 2 0 0 _ - _ _ 14‘B 1 l 0 O t - d ”d — £236.59 — +2.1572 lS—A 2 2 0 O ’5‘ ' lS-B 3 2 +1 1 d 340 16—A 1 1 0 O ‘r‘ l6—B 1 1 o 0 “ l7-A 4 1 +3 9 Decision: Reject Ho. l7-B 2 2 O 0 18-A 1 l 0 O Concludez/ll fl; 3;}: g ; +3) (1) There is a significant difference 159-8 2 2 O 0 between the Opinions of the experts 20-A 1 1 0 0 and the opinions of the students in ZO-B 1 1 O O the office block prOgram on the —— —— importance of the in-basket items. if a ..n— ._ .-.-.1.-1—-u.-.-u...'e..‘u__.. - ...;i. IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIlII___________________________________________________' 242 is rejected; and the alternate hypothesis that the students in the office block program and the panel of experts do differ significantly in their opinions as to the importance of the in-basket items is accepted. The conclusion is reached that there is a significant difference between the opinions of the experts and the opinions of the students in the Office block program. By taking a look at the specific differences existing within specific item comparisons on Table 52, one can state that the students were systematically overrating the experts on the importance of the items; that is, the students tended to give a greater degree of importance to specific items than the panel of experts. The mean differences between the opinions of the students in the single-period classes and the panel of experts on the importance of the in-basket items were also tested by means of a one~sample £_test to see if a significant difference occurs between these two groups. The null hypothesis for this test was stated: "The students in the single—period classes and the panel of experts do not differ Significantly in their opinions as to the importance of the in—basket items.” The null hypo— thesis may be represented in this way: (13) H : STUDENTS = PANEL OF EXPERTS 0 Classes The alternate hypothesis was stated: "The students in the single- period classes and the panel of experts do differ significantly in their Opinions as to the importance of the in-basket items.” This alternate hypothesis may be represented in this way: ' EXPERTS (13a) H1. STUDENTSClasses # PANEL OF Table 53 compares the experts' opinions on each in-basket item to the students' opinions, and the differences between the two groups on __——.fl 243 TABLE 53 A ONE-SAMPLE T TEST OF THE DIFFERENCES IN OPINIONS ON THE IMPORTANCE OF THE IN-BASKET ITEMS AS SHOWN BY ANALYZING RESPONSES OF STUDENTS IN SINGLE-PERIOD CLASSES AND THE PANEL 0F EXPERTS In-Basket Experts' Students' 2 CI. Item Opinion Opinion Hypothesis Testing I I I I I I I I I u + + + c>h+c>hmo 4- HH w>w>w>w>w>w>w>w>w>w>w>w> HHNNNHNgHHMNI-‘NHHNHMNNHHNI—‘HHNHHHHNWNNNHNH HI—INNI—‘HNHHHNHHHHHHHNNHHHNHHHI—‘I—‘HHHUJHNNHHHH OOOOHCO‘OOOHHOHOOF‘OHOHOOOOOOHOOOOHhOOF-d0H0 O 2 1 0 O O 0 1 O O O O 0 1 - O l - +1 1 - 0 1 - +1 - 0 - +1 l3-A 0 13-B O 14-A +1 14-B 0 lS-A +1 lS-B +1 l6-A 0 16-B O 17-A +3 17-B 0 18-A 0 18-B +1 19-A 0 19-B O ZO-A 0 ZO-B 0 “+3. 91 HozaA§=./‘L H1 #5 fla- Assumptions: a. RandOm sample of differences b. Normality of the differences Test Statistic: t=H'/"c1 Az_nle- (fill: 25: 5d ' n(n-l)1 a in Decision Rule: :13: .05 Reject H if t 64.02 or if tin-2.02. Compute the Value of the Statistic: /\ 2 5d2 = 7:312 ’ 1L " 2 nIn-l; 2 sd = 40(24) - (14 ‘A 40:395 2 _ I\ _ 5d - .4897 5d - .6997 .35 - O = +3.16l6 .6997 Decision: Reject Ho. Conclude: A 3‘ A; There is a significant difference between the Opinions of the experts and the opinions of the students in the single-period classes on the importance of the in-basket items. 244 each item were calculated. The computed E_statistic was +3.1616. The decision rule at the .05 level of significance stated that the null hypothesis will be rejected if the E_statistic is equal to or greater than +2.02. Therefore, the decision was made to reject the null hypo— thesis and to accept the alternate hypothesis that the students in the single-period classes and the panel of experts do differ significantly in their opinions as to the importance of the in~basket items. The conclusion reached in this one—sample E_test was that there is a sig- nificant difference between the Opinions of the panel of experts and the opinions of the students in the single—period classes on the impor- tance of the in—basket items. A look at the differences between the two groups shows that all but one of the differences are positive. This indicates that the students were systematically overrating the importance of the in—basket items as compared with the opinion of the panel of experts on the same items. The magnitude of the differences among all five groups was also examined by means of a two—way analysis of variance to determine if the magnitude of the differences among the groups on the importance of the in-basket items were about the same when compared with the panel of experts. Table 54 charts the obtained differences when each of the five groups was compared with the panel of experts (data from Tables 49, 50, 51, 52, and 53). The plus and minus signs were disregarded in this analysis so that the question could be investigated as to the degree of agreement among the grOups with respect to each group‘s difference with the panel of experts. Table 55 presents the summary table for the two—way analysis of variance. 245 TABLE 54 THE MAGNITUDE OF THE DIFFERENCES AMONG THE GROUPS ON THE IMPORTANCE OF THE IN-BASKET ITEMS Row Totals Students NSA Students MBT (Class) Secy (Block) Segy_ CPS's In-Basket Item 2102005500002012100205040020340060032200 1000001100000010000001010010110000011000 0000001100001001100101010000010030010000 1000001000000001000001010000010000001100 0101002100001000000101010010110030010000 0001000200000000000001000000100000000100 ABABABABABABABABABABABABABABABABABABABAB 112233445.5667788990011223344556677889900 1111111111111111111122 54 11 13 16 Column Totals 246 mumfi. oma MN.mN u wm.vH + ON.mm I mH.oH 1 00.0h 2 no A he and www» a“... flow a waive. - was > VWW 2530235 NAM C m U M Name. 2 am Nc.em n wm.eH - om.am C. 2 , a o a .3 - .H a «Q SWWWv Nd. s NNVW ...zom m u e u m OH.N+ mmmm. v hm.H n wm.¢a I mH.©H 2 gm 530.335 a x n . .m: or: - a n u > .N “N Q: 2538 8 was: m: N c m ow N s Weva oamzcm Eopoohm m can: mo moonmoo 8mg mohosvm mo 55m ooasom memmmxm mzk :HHR QmmOz< >m<223m mm mqm<9 247 TABLE 55 (Continued Hypothesis: Ho‘J'Idl 7‘82 f/(ds =fld4 =Jld5 H : The magnitude of the differences for the groups are not all identical. Decision Rule: MS . . columns Reject H if F = ———-—————— F o MSint. > 4, 156 (.95) (134,”: +2.37) Decision: MS lu co mns = +2.10, Fail to reject Ho. M . . interaction Conclude: The magnitude of the differences of the groups when compared with the experts is about the same. There is no significant difference in the magnitude among the five groups. We cannot say that one group is contributing more than another group to the magnitude of the differences. 248 The null hypothesis that the mean differences among the five groups (CPS, NSA, MBT, ST STCL) do not differ significantly on the degree to BL’ which they differ from the Opinions of the panel of experts on the impor— tance of the in—basket items was tested by means of a two-way analysis of variance. The null hypothesis may be represented in this way: (14) Ho:‘/(d =J£d =_/(d =fld =fld 1 2 3 4 5 The alternate hypothesis may be stated in this way: ”The magnitude of the differences for the five groups are not all identical." Table 55 presents the summary table for the two—way ANOVA of the magnitude of the differences among the groups on the importance of the in-basket items. The computed F statistic was +2.10. The decision rule for the testing of the hypothesis stated that the null hypothesis will be rejected 1f F4, 156 (.95) 15 more than or equal to +2.37. Therefore, the decision is not to reject the null hypothesis and to conclude that the magnitude of the differences of the groups when compared with the experts is about the same. We cannot say that one group is contributing more than another group to the magnitude of the differences. Item Analysis Based on Mean Scores on_guality Responses No statistical item analysis was designed into this in—basket study. However, a simplified method was used to analyze the group means for each in—basket item on the quality reSponses (action responses) given to each in—basket item attempted to see if, in fact, the means for the groups appear to differ on the quality response scores. The number of subjects responding to specific items varies because of the nature of the in—basket itself. Some subjects completed all of the items; others completed as many as possible in the time limit. 249 Table 56 shows that one of the three secretary groups had the high- est group mean on 39 of the items. The students in the office block program had the highest group mean on one item, Item l7-B, with a group mean of 3.56. Using the five-point scale of quality responses as a guide, one can determine what the general response was to the item. The office block students responded to Item l7-B (a confidential letter to the personnel director is being sent and the secretary is asked her opinion of the letter) by indicating to the president that the letter needed few or no revisions. The Michigan Bell secretaries had a group mean of 3.00, which indicated that they felt no revisions were necessary. The CPS's (2.67) and the NSA secretaries (2.40) felt that they would read the letter and perhaps ask another secretary for her opinion on the letter and its contents. The students in the single-period classes (1.77) would read the letter before sending it. Perhaps there is an indication with this particular item of carefulness on the part of expe- rienced secretaries to make a conclusion about a situation they may not be familiar with. Since this is the first day on the job, the secre- taries may feel that there are some judgments that should remain with the employer. By examining the number of secretaries and students who attempted Item l7-B (which was located in the last enve10pe of THE IN-BASKET), one may notice that a total of 22 out of 51 secretaries attempted the item while 16 out of 54 students in the office block program and 22 out of 65 students in the single-period classes attempted the item. The question may be raised as to the possibility of a selection bias with regard to this item: Do the better students get to this item since 250 . O-OH OOO N~.N OO OH.N OO NO.N ON OO.N O OO.M O <-OH OOO OO.H mO am.N NO OO.H OM OO.~ OH OO O O . -O wao OO.H mm OO.m OO OO.N ON OO.O OH OO.O OH m-O Omz mm. OO HO. NN OO.H ON OO.H m On O OOO OO.N mm OO.N mO OO.N ON O0.0 HH OOHO OH mum Omz OO.H OO OO.H NO OO.N om OO.m OH OO O O Oau HO. am BN.H HO OO.H RN BO.H O OHHN 5 “HM Om: OO.H OO OO.H ON OO.N mm OO.H a Om H O m3 ONO mO Om.m mm om.m om OO.m HH 9...". OH Mum OOO OO.H Om OO.H mm OO.H OH OO.N a O0.0 m OOO OO.H Om OO.H NO OO.N ON OO.O HH OO.O O mum OOO OH.H OO NO.H NO HN.H Om OO.N OH OO.N O HO: OO.H ON OO.H OH OO.N OH OO.N O OO.H O ONO OOO HO.H OO OO.H NO OO.H am OO.H O OH.N O < O OOO OH.N NO OO.H mm Om.m am OO.N HH On.m O ONO OOz OO.N Om OO.N OO OO.O ON OH.O HH Ha.N a O m Omz OO.N Hm OH.N OO BO.N Hm ON.m OH NN.N O ONO Omz OO.H OO OO.H mm OO.H Om Oa.m OH Om.m O < N OOO OH.N Om OO.H HO HH.N ON OO.m HH NN.m O O-H OOO HO.H Om OO.N Hm OO.N ON nm.m HH Om.m O <-H O .m z x z x z .m z .m z “Onesz :oyo mommmHu .l .I HOO OH - EOHOOHO OOOHO HOOZO HOOzO HOOOO amp : .: OOHHOO OHOOHO OH OOOOOOHO OOHHOHOHOOO OOHHOHOHOOO OOHHOHOHOOO H ca mucmwsum mzmqmomm memHHAMH Om m4mH Two basic assumptions were made in designing and implementing the procedures for the study: 1. Even though the groups of secretaries (CPS'S, experienced secretaries in other categories) and the groups of high school students (secretarial practice and/or advanced shorthand classes and vocational office block programs) may be employed or enrolled in different institutions, their scores on the in-basket can be grouped together so that a total score for a specific group of subjects may be obtained. The samples of secretaries and students will want to participate in the study, based upon the background of the study and the preparation for the experimental phases of the study. This research in the development of in-basket simulation required the researcher to work under restricted conditions at certain Stages of the study and to face the following limitations: 1. The in—basket deve10ped in this research study represented terminal evaluation of the secretary's decision-making ability in setting work priorities, not intermediate evaluation. The in-basket was designed in such a way that the participant indicated responses on a special answer sheet. Within the time limit set for the in-basket it would have been impos- sible to have the participant actually produce typewritten problems or perform other actions in addition to making appropriate work—priority decisions. The in—basket itself was based upon a time limitation of 45 minutes. Therefore, the number of in-basket items was limited to 40 items, two in—basket items for each of 20 problem categories. The samples of secretaries contacted during the critical incident study were drawn from the membership rolls of the Michigan Division of the National Secretaries Association (International) and the Institute for Certifying Secretaries. The samples of secretaries selected for field testing in— cluded: (a) Certified Professional Secretaries residing in the State of Michigan, (b) secretaries who are members of the Michigan Division of the National Secretaries Association (International) and residing in the State of Michigan, and (c) secretaries to divisional managers of the Michigan Bell Telephone Company, Detroit, Michigan. 267 High school students participating in the field testing were selected from high schools in the State of Michigan, selected on the basis of whether there were single-period secretarial practice and/or advanced shorthand classes or vocational office block programs. The number of tryouts for the in-basket was limited to two, the first tryout followed by a revision stage. Theoretical Principles for the Development of In-Basket Simulation Basic theories relevant to the design and development of in-basket simulation require the integration of sound and innovative concepts from the real business world with meaningful and relevant methods of instruc- tion in the classroom environment. There has been little published on the actual design of in-basket simulation; therefore, this summation of the theoretical principles used as the basis for THE SECRETARIAL IN- BASKET provides a rather comprehensive analysis of those principles that are applicable to the deve10pment of in-basket simulation: 1. 3. Basic Approaches to Test Development: The rational hypo- thesis approach provides the basic approach to the deve10p- ment of in-basket simulation because it produces tests of typical performance in real-life situations. Philospphical Theory_for In-Baskct Development: In—basket simulation must represent as closely as possible a situa- tional excerpt from the real world in its attempt to bridge the gap between the business world and the conventional classroom. ngchological Theopy for In-Basket Develppment: The stimulus-response-feedback theory and the identical elements theory are prominent in the deve10pment of in- basket simulation in order that transfer of learning occurs and motivation for learning enhanced. Sociological Theopy for In-Basket Develppment: The deve10pment of an in-basket simulation authenticating a given occupational role requires a study of the social role arising from the job classification under study with specific relevance to the human relations required in the work to be performed on the job. lulu!“ . ll ...ili If 268 5. Integrated Theopy for In-Basket Deve10pment: An integrated approach to inibasket simulation appears essential in order to tie the foregoing philosophical, psychological, and sociological elements to the mechanistic theory essential for control in design. 6. Measurement of Individual Performance: Accurate measurement of What the individualfihas Iearnedfi(Specific skills and knowl- edges, behavioral changes in attitude, awareness of self, and motivation) is essential in evaluating and making decisions about that individual's performance on the test. 7. Standardization of Testipg Procedures: Every detail of the testing situation must be identical for all subjects tested if any form of standardization is to occur. 8. Maintenance of Test Reliability: Test reliability is the consistency with which the in-basket simulation measures what it is intended to measure from one time to another. 9. Establishment of Test Validipx: The validity of in-basket simulatIonTTCOntent, construct, concurrent, and predictive) is prOportional to the degree of correspondence between performance on the in-basket and performance on the job. The Design and Development of THE SECRETARIAL IN-BASKET One of the questions arising in this research study asked if in- basket simulation can be used to evaluate the ability of an individual to handle specific decision-making required in a particular position, that of secretary. The design and development of THE SECRETARIAL IN-BASKET into an instrument that could, in fact, evaluate an individ- ual's ability to handle these kinds of decisions was dependent upon the following operational phases: Preliminary Planning. Preliminary preparation for the in-basket research involved two basic Operational stages: the establishment of initial contact with the National Secretaries Association (International) and the Institute for Certifying Secretaries. A number of eXperts in the areas of personnel management, management training and deve10pment, 269 testing and evaluation, manpower planning and research, and teacher education were contacted because of their work in the development 0 in-basket simulations. Arrangements were made whereby samples of secretaries who were NSA members in the State of Michigan and samples of Certified Professional Secretaries who reside in the State of Michigan could be contacted for purposes of the study. Review of Related Research and Literature. Four specific areas of study were included in the review of related research and literature pertinent to the study: 1. Design and Developmental Theories: Selected reviews of research and literature pertinent to the development of in—basket simulation; design and developmental theory from areas of education, tests and measurements, psychology, philosophy, and other areas of concentration. 2. The Historical Development of In—Basket Simulation: The use of in-basket simulation in research, management train— ing, and secretarial education as an instructional, training, and/or testing procedure. 3. Analyses of the Secretarial Role: Research pertaining to the role of the secretary and delineation of personal traits, duties, responsibilities, behaviors, and/or tasks performed. 4. Research in Curriculum Deve10pment: Application of instructional models, implications for instructional materials and evaluative instruments in the development of secretarial training programs. The Critical Incident Study. Flanagan's critical incident tech— nique was modified for use in the preliminary search for the kinds of typical but significant decisions the secretary faces in setting work priorities during the routine business day. Two samples of 50 secre- taries each, one representing members of the Michigan Division of the National Secretaries ASsociation (International) and one representing Certified Professional Secretaries residing in the State of Michigan, 270 were contacted in order to obtain samples of the kinds of decisions made by the secretary in her job. Each critical incident was examined and classified according to the Specific problem and decision factors inherent in the incident. The 181 critical incidents yielded 415 prob- lem factors, and those 20 problem factors receiving the largest number of tallies were utilized in the development of the in-basket items. The classification of decision factors within a specific incident was based upon the Huffman et a1 taxonomical framework for analyzing office activities and elicited the kinds of decisions secretaries make in solving given problem situations. The Deve10pment of THE SECRETARIAL IN—BASKET. The actual develop- ment of the in-basket simulation involved the following operations that were essential for the completion of the study: I. The planning, writing, and sequencing of the in—basket items in the in—basket simulation. Reviewing the in-basket items as a reality measure. 3. Developing the procedures to be used in administration of the in-basket to individuals and/or to groups. 4. Administering THE SECRETARIAL IN—BASKET to a panel of experts to determine the experts' responses to the items. 5. Developing the scoring procedure to be used in the evaluation of individual performance. N THE SECRETARIAL IN—BASKET was prepared in such a way that the 40 items were included either in the form of business papers or in the form of taped interruptions. The business papers were organized in a portfolio, and the audio tape of planned interruptions was included as a part of the administrative set of materials. Elelleesting THE SECRETARIAL IN—BASKET. The field testing of THE SECRETARIAL IN—BASKET included two tryouts with a revision stage follow— ing the first tryout. The ultimate goal of the field testing phase of 271 the research was to collect data for validation of the instrument by comparing the performance of eXperienced secretaries with the perfor- mance of students who are preparing to be secretaries and are enrolled in secondary education programs. In order to meet this goal, experi— enced secretaries (Certified Professional Secretaries, secretaries who are members of the National Secretaries Association (International), and secretaries to divisional managers of the Michigan Bell Telephone Company) and high school students (students in office block programs and students in secretarial practice and/or advanced shorthand classes) were administered the in-basket and the results analyzed. The field testing phase of the study was conducted as a means of validation for the instrument. It was hypothesized that the in-basket would be sufficiently sensitive to be able to detect differences in the decision-making abilities of experienced secretaries (CPS's, NSA secre- taries, and Michigan Bell Telephone Company secretaries) and potential secretaries (students in office block programs and students in secre- tarial practice and/or advanced shorthand classes). Group comparisons were not intended to serve as a vehicle for comparing types of in-school instructional techniques nor as a measure of comparing CPS's with other groups of experienced secretaries and cannot be validly used to serve this function. Analysis of the Data. Three types of statistical analyses of the data derived from the field testing phase of the study depicted subjects' performance on THE SECRETARIAL IN-BASKBT: 1. Performance on the In-Basket Simulation: A multivariate analysis of variance was used to compare the groups on total number of problems attempted in the entire in-basket, the total quality response scores, and the mean scores for 272 quality responses. The total problems attempted in the three separate envelopes within the in—basket and the percentage of inversions in the setting of work priorities were also evaluated by means of a multivariate analysis of variance. Scheffé’post hoc comparisons were used to evalu— ate individual univariate differences between means. 2. Qplnions on the Importance of the In-Basket Items: Each of the subjects within the five groups was asked to respond to the question of the importance of each in—basket item and to include this information on the answer sheet as decisions were made on in-basket items. NO formal item analysis of the importance of the in—basket items, based upon the opinions of the subjects participating in the study, was planned for the in—basket study. However, an informal analysis of the opinions expressed by the groups as well as the testing of hypotheses concerned with the comparison of each group with the panel of experts served to further clarify the effect of the importance ratings on the setting of work priorities. A one—sample t test was used to test the differences between each of The five groups and the panel of experts. 3. Performance on Individual In—Basket Items: No statistical item analysis of the in—basket items was designed into this study. However, a simplified method was used to analyze the group means for each in—basket item on the quality responses (action responses) given to each in—basket item attempted to see if, in fact, the means for the groups appear to differ on the quality response scores. Findings of the Study Three general areas of concentration are represented in the findings that result from this in—basket study: 1. Findings that relate to the actual design and development of THE SECRETARIAL IN-BASKET. 2. Findings that relate to the validation of the in—basket simulation. 3. Findings that relate to the development of a measurement schema for the in-basket simulation. Specific findings related to these three general areas follow in the next subsection. 273 Findings Related to Design and Development of the In-Basket. During the entire design and developmental process it was evident that one must follow certain theoretical and practical guidelines in developing an in-basket simulation such as THE SECRETARIAL IN-BASKET. The following findings emphasize those essential elements derived from this study for developing in—basket simulations for use in secretarial education: 1. The occupational role to be represented in the in-basket simulation (i.e., secretary) must be analyzed by the researcher in order to assure that the in—basket developed is representative of the occupation. 2. The in—basket simulation must contain problems which are verified by practitioners in the field (secretaries) as cases of real business practice. 3. Each in—basket item must be designed and deve10ped initially in isolation from all other in—basket items; later, these items may be sequenced and ”clustered” into the in-basket simulation. 4. A control mechanism must be a part of the in-basket design in order that the situation, the time element, and the individual's performance on the simulation may be standardized. S. The involvement of both business educators and experi— enced secretaries is essential at all stages of in-basket development so that intermediate as well as terminal review of the in-basket items and, finally, the entire in-basket itself becomes an essential part of the developmental process. 6. The scoring procedures and devices developed for evalu- ating an individual's performance on the in-basket simulation must be objective in nature so that the scorer does not deviate from the established scoring pattern. 7. The in-basket simulation must be designed and developed in such a way that it may be administered individually or in groups, independent of a group leader's presence or a researcher's presence. Findings Related to the Validation of the In—Basket. Although statistical tests of differences were made between the three groups of 274 secretaries (CPS's, NSA secretaries, and Michigan Bell Telephone secre- taries) and the two groups of students (office block students and students in secretarial practice and/or advanced shorthand classes) in terms of performance on this particular in—basket, the reader should not interpret the differences indicated in the findings as meaning that one group of subjects is ”better" in any way than another group or that one type of instructional program is "better" in any one than another program. These specific types of comparisons cannot be made as a part of this study for two reasons: (1) There is no way of estimating that any one group of experienced secretaries received more training in decision—making skills or had a higher mean intelligence quotient or functioned at a higher level than any other group of experienced secre- taries, and (2) there is no way of estimating that one type of instruc— tional program (in comparison with other programs) provided more training in decision making or had students of higher mean intelligence enrolled in the program or had students who were functioning at higher levels. One of the main purposes of the study was to determine whether the in—basket did, in fact, distinguish decision-making differences between experienced secretaries and potential secretaries. The use of statis- tical tests of differences provided the opportunity to examine each group's performance on the in—basket in comparison with other groups in order to substantiate the discriminating power of the in—basket. The following findings, significant at the .05 level of signifi— cance, were based on the second tryout and relate gnly_to the validation of the instrument and are not to be seen as a measure of the type of in—school instructional technique. The eXperiment was conducted in 275 order to determine variations in secretarial behavior among groups of secretaries and students on THE SECRETARIAL IN—BASKET. Performance on the In-Basket Simulation 1. The secretaries combined (CPS, NSA, MBT secretaries) scored significantly higher than the students (the office block program and the single—period classes) on the total response score. Each individual group of experienced secretaries (CPS, NSA, MBT secretaries) scored signifi- cantly higher than each individual group of potential secretaries (the office block program and the single— period classes) with which it was compared. The secretaries combined (CPS, NSA, MBT secretaries) scored significantly higher than the students combined (the office block program and the single—period classes) on the mean response score. Each individual group of experienced secretaries (CPS, NSA, MBT secretaries) scored significantly higher than each individual group of potential secretaries (the office block program and the single-period classes) with which it was compared. The only difference on the percentage of inversions found between experienced secretaries (CPS, NSA, MBT secretaries) and potential secretaries (the office block program and the single—period classes) was the following: The Michigan Bell Telephone Company secre— taries scored significantly higher than the students in the single—period classes on the percentage of inversions in the in-basket envelope. Opinions on the Importance of the In—Basket Items 1. There is a significant difference between the Opinions of the experts and the opinions of the students in the office block program on the importance of the in—basket items. There is a significant difference between the Opinions of the experts and the opinions of the students in the single—period classes on the importance of the in—basket items. The magnitude of the differences of the groups when com— pared with the experts is about the same. There is no significant difference in the magnitude among the five groups. 276 Performance on Individual In—Basket Items 1. The analysis shows that one of the three secretary groups had the highest group mean on 39 of the items. 2. The analysis shows that the students in the office block program had the highest group mean on one of the items. Findings Related to the Development of a Measurement Schema. One question which ultimately results when a testing instrument such as THE SECRETARIAL IN—BASKET is developed and field tested relates to the use of those results in future testing situations. Both the researcher and the classroom teacher need to know how the results of this research may be used in administering THE SECRETARIAL IN—BASKET to another indi- vidual. Three basic ways of evaluating the subject's performance on the in-basket, in comparison with all other subjects who have taken the in- basket during this study, have been developed; and these findings result in a suggested measurement schema: l. A subject's total quality responses on THE SECRETARIAL IN—BASKET may be compared with all secretaries and/or all students who have previously taken the in—basket by using the percentile ranks for total quality responses in Table 57. 2. A subject's performance as it relates to the total number of problems attempted may be compared with all secretaries and/or all students who have previously taken THE SECRE- TARIAL IN— BASKET by using the percentile ranks for total number of problems attempted in Table 58. 3. A subject's performance on the ordering of the in- basket items may be compared with all secretaries and/or all students who have previously taken the in— —basket by using the frequencies for percentages of inversions within the three in-basket envelopes contained in Tables 59, 60, and 61. The measurement schema is an attempt to relate the field testing that has been done in this study to the reality of using the same in— basket in additional research and classroom settings. 277 B. Conclusions The three basic conclusions drawn from the analyses of the data in the foregoing experimental study emphasize the theory of decision making as it must be combined with the career ladder concept, the importance of quality of performance, and the need for levels of secretarial education. Hierarchy of Decision Making + Career Ladder Concept = Skill Development The 20 problem factors requiring decision making that were devel- oped in THE SECRETARIAL IN-BASKET were contributed by experienced secre- taries averaging 19.25 years of secretarial experience. The kinds of tasks in their respective positions require a high level of decision making with a range from simple to complex problems, but the majority of the tasks were of the latter variety. Those students who are potential secretaries should not be expected to display decision making abilities at the same levels as the eXperienced secretaries. First, a student in an office block program, a secretarial practice class, or an advanced shorthand class, for example, may not have been exposed to the complete range of decision—making activities that were included in THE IN-BASKET. However, if a student had been eXposed to decision- making principles and methodology as they pertained to the specific kinds of tasks in this in—basket, she would no doubt perform at a higher level than if she had not. Secondly, decision making is a progression skill that moves from the simple to the complex, from the concrete to the abstract, and from the immediate to the remote. That is, as an individual pregresses up the career ladder of office occupations, i.e., from clerk-typist to 278 stenographer to secretary, the complexity of the decisions she must make on the job may be in direct proportion to the complexity of the task requirements. Therefore, in the development and evaluation of decision-making skills, it is first of all essential that educators know and understand (a) the career ladder concept as it relates to the office occupations, (b) the relationship of careers within the office occupations, (c) the job requirements--the specific skills, knowledges, and attitudes needed to perform in specific jobs. Secondly, we need to know what the hierarchy of decision making is within each step on the office occupations career ladder in order to prescribe the appropriate "dosage" of learning in the development of decision- making skills necessary for given entry and exit points on the career ladder. é-anlity + %-guantity + %-Time = 100% Efficiency of Performance Significant differences among the groups participating in the experimental study occurred, for the most part, on the quality (action) reSponses of the subjects. No significant differences occurred among the groups on the total number of problems attempted in THE IN-BASKET. A conclusion to be drawn from these data indicates that quality of action responses is very important in determining the status of the performance of an individual, but quality alone is not the only criterion. The total quality responses of an individual on THE IN—BASKET, for example, reflects not only the quality of given responses but also the quality in relation to the quantity of items produced. Therefore, quantity of responses is considered in the individual's performance. —_——" 279 By examining what the individual has accomplished either in his training or on the job, one can determine the quality of the action taken for each task performed. One of the causes for the students not performing as well as the secretaries on the in—basket items may be that the students lack competency in skills and understandings as well as confidence in themselves to make those kinds of judgments that secretaries averaging 19.25 years of experience might make in their jobs. Experienced secretaries are confident in their work, have experienced success on the job, possess skill competencies, and have the desire to be promotable even when they are at the top of the career ladder. Time is another factor to be considered in relation to decision making. In teaching a student what it is like to work in an office, we often use time as a factor: timing the students for a given number of minutes to see how fast they can type, giving students dictation at a given rate for three or five minutes, administering a 30—minute production test of typing problems, and so on. THE SECRETARIAL IN— BASKET is an example of another timed test, in this case for 45 minutes, on the kinds of decisions in the setting of work priorities experienced in the secretarial position. There may be a need to develop proficiency so that students will be able to handle routine decision—making activi- ties so well that the time taken in making a routine decision is momentary. Certain other kinds of decisions, however, may require unlimited time so that the student can analyze what would be done in a real situation. A student in the process of learning requires time to determine the steps and procedures that may be necessary or to 280 decide if she should go to the files and look something up or to finish typing the letter she is working on before she delivers a message to her ”boss.” Three factors come into focus when efficiency of performance is the ultimate goal: quality, quantity, and time. The conclusion derived from the study is that quality is the most important of the three and deserves additional attention in the determination of perfor— mance; however, quantity and time should not be overlooked in their contribution to the overall efficiency of performance needed to accom~ plish office tasks. Therefore, the basic conclusion is that quality of performance coupled with quantity of items produced plus time as it is applicable to a specific task will lead to greater efficiency of performance on the job. Setting Work Priorities = In-School Education + Training_Within Industry The secretaries participating in this study were from three popu- lations: (1) Certified Professional Secretaries, (2) secretaries who are members of the National Secretaries Association (International), and (3) secretaries to divisional managers of the Michigan Bell Telephone Company. The Michigan Bell secretaries were the only group of secre- taries to perform significantly higher than the students in the single— period classes on the percentage of inversions in the in—basket envelope (one of three envelopes in THE SECRETARIAL IN—BASKET). There were no significant differences among the other groups of secretaries and the students in the office block program. 281 In in-school education (secondary and post-secondary) students are to develop a general range of skills, understandings, and attitudes needed to perform within a career range on the ladder. It could be concluded that potential secretaries (students) need training and practice in how to set work priorities as they might relate to specific secretarial responsibilities. The setting of work priorities requires decision-making skill that is practiced in a systematic way in order to develop simple as well as complex decision-making skill. Decision-making training conducted 2:33; the individual obtains a secretarial position within an organization provides her with the specific requirements of her occupation as it relates to the entire company and the given work priorities within that organization. For example, the Michigan Bell Telephone Company has an extensive training program for their secretaries and encourages them to participate in these training programs on company time at specific intervals in their careers. Their training prOgram shows people how to work in a specific company. Michigan Bell has identified what the work priorities are for secretaries to divisional managers, the group of secretaries who partici— pated in THE SECRETARIAL IN-BASKET field testing. Training for Michigan Bell means that they are identifying for their secretaries what these work priorities are and how they are related to the entire operation of the organization. The amount of company training provided to the CPS's and the NSA secretaries participating in the other two secretarial groups is unknown. 282 C. Implications for the Study Implications resulting from this in-basket research are evident in the areas of curriculum deve10pment (secondary and post-secondary), research, teacher education (pre-service and in-service), and training within industry. One research study like this one is a beginning, not an ending; and, hopefully, others will glean ideas from it and be able to add to the information that has already been obtained. Curriculum Development Instructional programs now being deve10ped on the secondary and post-secondary levels will require a learning environment which teaches students how to make decisions and then provides opportunities to practice making decisions--decisions about their personal lives, decisions about their careers, decisions about given tasks to be performed, decisions which affect their futures. Teachers have an obligation to provide the kind of learning environment that will assist students in making appropriate vocational and personal choices, and thus both the teacher and the student should have a thorough under- standing of the decision-making process. In order to develop vocational curricula which are relevant and meaningful to the students, teachers need to understand the requirements of given occupations either by means of task analyses, job surveys, and other types of occupational studies. Curriculum deve10pers need to be aware of the ways in which information obtained from business can be utilized in updating and revamping the current occupational curricula. Instructional materials, whether they are for the secretarial program or one of the other vocational programs, will need to be realistic to 283 the students. The tasks in the materials need to be representative of the kinds of tasks that are actually required on the job. One of the most important implications arising from this study is the importance of evaluating performance in the classroom according to on—the-job business standards. If a student is performing an office task in the classroom and receives a grade of C, what does that represent? That same student as a secretary in the business office will type a business letter that will either be mailed out or not mailed out, depending upon whether it was acceptable or unacceptable to the employer. Also, evaluation of student performance in decision making in a laboratory situation must be related to the performance standard that is evident in the real job situation. If there is a progression of decision—making skills required at various entry and exit points on the office occupations career ladder, it is essential that vocational programs include this progression of decision—making skills (with appropriate instructional materials) in the curriculum. These skills must range from simple to complex, from concrete to abstract, and from immediate to remote. Students must be given an opportunity to practice making the kinds of decisions they will make later on the job. Too often teachers have not incorporated the simplest decisions in the development of instructional materials; too often they "jump in” at the more complex levels. Time should also be built into the instructional materials for the learning of decision— making skills. The student must be given time to analyze given prob- lems and situations to determine how best to handle them and what steps and procedures might be necessary. Real or simulated experiences in 284 the classroom or laboratory will provide students with the skills, understandings, and attitudes needed on the job. Research A number of possible research studies could stem from the basis of this study: 1. A replication of this study with larger samples in order to make the measurement more reliable; a replication with additional samples (students in area center programs, secretaries in training programs within industry). 2. A study to determine the correlation between the progres— sion an individual makes on a given career ladder and his equivalent progression on a learning hierarchy, especially as it relates to decision-making skill. 3. A study of the levels of years of experience of secretaries to determine the kinds of decisions which are, in fact, required at various points on the secretarial career ladder. 4. Formal item analysis of items contained in in-basket simulations or other situational tests to see what levels of decision—making might be required. 5. A study of the theory of decision making as it relates to the means by which experienced secretaries (at different rungs of the career ladder) do actually make decisions. 6. Analysis of items contained in in—basket simulations and other situational tests to develop evaluations for these items based upon the tasks to be performed and the decision making required. 7. A study of the opinions of secretaries as to the importance of tasks inherent in their positions in comparison with students in secretarial programs and their opinions on the same tasks. Teacher Education In order to adequately prepare students for the world of work, a teacher must be acquainted with the career ladder concept and its relationship to the vocational curriculum. An understanding of the 285 learning hierarchy is essential in develOping apprOpriate instructional materials for the teaching of decision-making skills to students, and this basic understanding can best be obtained through a teacher educa— tion program that is geared to the deve10pment of these types of basic competencies. Teachers not only need to know how to develop instruc- tional materials, but they also need to know how to implement these materials in the classroom situation. Teachers also need assistance in knowing how to evaluate the effectiveness of instructional materials in the classroom or laboratory. In addition, teachers need to be aware of the close ties with business and industry that are essential in developing vocational programs to effectively meet business needs for future employees. These "needs" of the vocational teacher can be answered best through teacher education, both pre-service and in-service programs. The times change very rapidly, and the teacher education program can help the local teacher change and adapt his secondary or post-secondary program to meet the needs of the times. Trainingzgithin Industry A secretarial program cannot do all things for all people. There- fore, a secretarial program, whether it is on the secondary or post- secondary level, is not a substitute for nor in lieu of industrial training. The secretarial program may Speed a student on her way to the development of skill and knowledge competencies required for a given career on the ladder and provide her with a basic, general understanding of the nature of secretarial work. If, at the end of her program, she enters employment with a particular company, it will 286 be necessary for her to be trained in the rules and regulations for the company, the work priorities that exist for her specific position, and any other specific information on the entire operation of the business vital to her smooth functioning. Therefore, there is a need for a coordinated effort between education and business to provide a complete secretarial education for the secretary. BIBLIOGRAPHY BIBLIOGRAPHY Books Anastasi, Anne. Psychological Testing. 2nd ed. New York: The Macmillan Company, 1961. Anderson, Lee F., et al. A Comparison of Simulation, Case Studies, and Problem Papers in Teaching Decision Making. Evanston: Northwestern University, 1964. Anderson, Ruth 1., et al. The Administrative Secretary: Resource. New York: McGraw-Hill Book Company, 1970. Becker, Esther R., and Rollason, Peggy Norton. The High-Paid Secretary. Englewood Cliffs: Prentice—Hall, Inc., 1967. Bruner, Jerome S. The Process of Education. Cambridge: Harvard University Press, 1960. Bursk, Edward C., and Chapman, John F. New Decision—Making Tools for Managers. Cambridge: Harvard University Press, 1963. Calhoun, Calfrey C., and Hillestad, Mildred, ed. Contributions of Research to Business Educationinational Business Education Yearbook, Number 9. Washington, D. C.: National Business Education Association, 1971. Caplow, Theodore. The Sociolggy of Work. New York: McGraw—Hill Book Company, 1964. Charters, W. W., and Whitley, Isadore B. Analysis of Secretarial Duties and Traits. Baltimore: Williams and Wilkins Company, 1924. Cook, Fred 8., ed. Secretaries on the Spot. Kansas City: The National Secretaries Association (International), 1967. Crawford, T. James. Production Typewriting, Monograph 97. Cincinnati: South—Western Publishing Company, 1960. Cronbach, Lee J. Essentials of Psychological Testing. 2nd ed. New York: Harper 5 Row, 1960. 287 288 "Evaluation for Course Improvement.” Current Research on Instruction. Edited by Richard C, Anderson et al. Englewood Cliffs: Prentice-Hall, Inc., 1969, pp. 36 — 373. Cronbach, Lee J., and Suppes, Patrick, ed. Research for Tomorrow's Schools: Disciplined Inquiry for Education. Toronto: The Macmillan Company, 1969. Ebel, Robert L. Measuring Educational Achievement. Englewood Cliffs: Prentice—Hall, Inc., 1965. Ebel, Robert L. ”Must All Tests Be Valid?” Principles of Educational and Psychological Measurement: A Book of Selected Readings. Edited by William A. Mehrens and Robert Ebel. Chicago: Rand McNally and Company, 1967, pp. 219-232. Ewing, David W. The Managerial Mind. New York: The Free Press of Glencoe, 1964. Fattu, Nicholas A., and Elam, Stanley. Simulation Models for Education. Phi Delta Kappa: Fourth Annual Phi Delta Kappa Symposium on Educational Research, 1965. Frederiksen, Norman; Jensen, Ollie; and Albert E. Beaton. Organizational Climates and Administrative Performance. Princeton: Educa- tional Testing Service, 1968. Fromm, Erich. ”Personality and the Market Place." Man, Work, and Society. Edited by Sigmund Nosow and William H. Form. New York: Basic Books, Inc., 1962, pp. 446—452. Furst, Edward J. Constructing Evaluation Instruments. New York: Longmans, Green and Company, 1958. Gagné, Robert M. The Conditions of Learning. New York: Holt, ' Rinehart and Winston, Inc., 1965. Ghiselli, Edwin E. Theory of Psychological Measurement. New York: McGraw—Hill Book Company, 1964. Glaser, Robert. Training Research and Education. New York: John Wiley 8 Sons, Inc., 1965. Goslin, David A. Teachers and Testing. New York: Russell Sage Foundation, 1967. Hardaway, Mathilde. Testing and Evaluation in Business Education. 3rd ed. Cincinnati: South—Western Publishing Company, 1966. Hays, William L. Statistics for Psychologists. New York: Holt, Rinehart and Winston, 1963. 289 Hemphill, John K.; Griffiths, Daniel E.; and Frederiksen, Norman. Administrative Performance and Personality: A Study of the Principal in a Simulated Elementarnychnni. New York: Teachers College, Columbia University, 1962. Hilgard, E. R., and Bower, G. H. Theories of Learning. New York: Appleton—Century—Crofts, 1966. Horst, Paul. Psychological Measurement and Prediction. Belmont: Wadsworth Publishing Company, Inc., 1966. Howes, Virgil M. Individualization of Instruction. New York: The Macmillan Company, 1970. Humphreys, Lloyd G. "The Normal Curve and the Attenuation Paradox in Test Theory.” Principles of Educational and Psychological Measurement: A Book of Selected Readingg, Edited by William A. Mehrens and Robert Ebel. Chicago: Rand McNally and Company, 1967, pp. 33—39. Klausmeier, H. J. Learning and Human Abilities. New York: Academic Press, 1963. Kneller, George F. Educational Anthropology: An Introduction. New York: John Wiley 6 Sons, Inc., 1965. Lindeman, Richard H. Educational Measurement. Glenview: Scott, Foresman and Company, 1967. Lopez, Felix M., Jr. Evaluating Executive Decision Making: The In—Basket Technique, AMA Research Study 75. American Management Association, Inc., 1966. Mednick, Sarnoff A. Learning. Englewood Cliffs: Prentice—Hall, Inc. 1964. ’ Mehrens, William A., and Ebel, Robert, ed. Principles of Educational and Psychological Measurement: A Book of Selected Readings. Chicago: Rand McNally and Company, 1967. Moore, Charles G. ”Simulation of Organizational Decision—Making: A Survey." Simulation in the Study of Politics. Chicago: Markham Publishing Company, 1968. National Education Association. Toward Better Evaluation of Learning. Washington, D. C.: Council on Instruction, 1962. Nichols, Frederick G. The Personal Secretary. Cambridge: Harvard University Press, 1934. Nosow, Sigmund, and Form, William H. Man, Work, and Society. New York: Basic Books, Inc., 1962. 290 Office of Strategic Services Assessment Staff. Assessment of Men: Selection of Personnel for the Office of Strategic Services. New York: Rinehart and Company, Inc., 1948. Oppenheim, A. N. Questionnaire Design and Attitude Measurement. New York: Basic Books, 1966. Osgood, C., et al. The Measurement of Meaning, Urbana: University of Illinois Press, 1957. Snyder, Helen 1. Contemporary Educational Psychology. New York: John Wiley 5 Sons, Inc., 1968. Spindler, George D. Education and Culture——Anthropological Approaches. New York: Holt, Rinehart and Winston, 1965. Stephenson, W. The Study of Behavior: Q—Techniqne and Its Methodology. Chicago: University of Chicago Press, 1953. Taylor, C. W., and Ellison, R. L. ”Predicting Creative Performance from Multiple Measures.” Widening Horizons in Creativity. New York: John Wiley 6 Sons, Inc., 1964. Tyler, Leona E. Tests and Measurements. Englewood Cliffs: Prentice— Hall, Inc., 1963. U. S. Department of Labor. Dictionary of Occupational Titles, Volume 11. Washington, D. C.: Government Printing Office. U. S. Office of Education. Vocational Education and Occupations. Washington, D. C.: U. S. Department of Health, Education and Welfare, 1969. Vance, Stanley. Management Decision Simulation. New York: McGraw— Hill Book Company, Inc., 1960. Zoll, Allen A., 111. Dynamic Management Education. 2nd ed. Reading: Addison—Wesley Publishing Company, 1969. Reports Behavioral Research Service. The In-Basket Test as a Measure of Foreman Performance. Crotonville, New York: Management Development and Employee Relations Services, General Electric Company, 1963. Behavioral Research Service. The In—Basket Test as a Measure of Managerial Aptitude. New York: Relations Services, General Electric Company, 1961. 291 Bentz, V. Jon. The Sears Experience in the Investigation, Description and Prediction of Executive Behavior. Chicago: Sears, Roebuck and Company. Calhoun, Calfrey C., et al. Development of Performance Goals for a New Office and Business Education Learnings Sygtem, Final Report, Project 91E, Grant No. OEG-8-084l4. Washington, D. C.: U. S. Department of Health, Education and Welfare, 1970. Cook, Fred 8. Detroit Studyyfor the Effectiveness of High School Education for Entrance into the World of Work. Report to the Michigan Department of Education, Division of Vocational Education, June, 1963. Cook, Fred 8.; Brown, Francis J.; and Lanham, Frank W. Senior Year Intensified Demonstration Curriculum Pro ram, Final Report, Project No. 6-1968, Grant No. DEG—061968—1993. Detroit: Wayne State University, April, 1970. Cook, Fred 8., and Lanham, Frank W. Opportunities and Reqnirements for Initial Employment of School Leavers with Emphasis on Office and Retail Jobs, Project No. 2378. Detroit: Wayne State University, June, 1966. Cook, Fred 8., and Shapiro, Edward Gary. Factors Associated with Successful Adaptation to the Secretarial/Stenographic Role, Project No. 6—1281, Grant No. OEG—3—6-062181-2079. Detroit: Wayne State University, 1968. Huffman, Harry, et al. A Taxonomy of Office Activities for Business and Office Education, Interim Report, Project No. 7—1223, Grant No. OEG—1—7—O71223—5134, Research 12. Columbus, Ohio: The Center for Vocational and Technical Education, July, 1968. Kersh, Bert Y. Classroom Simulation: A New Dimension in Teacher Education, Project No. 886, U. S. Office of Education, NDEA Title VII. Monmouth: Training Research Division, Oregon State System of Higher Education, June, 1963. Lanham, Frank W., et al. Development of Performance Goals for a New Office and Business Education Learnings System, Final Project Report, Project No. 8—0414, Grant No. OEG—O—O- 080414-3733 (083). Washington, D. C.: U. S. Office of Education, 1970. Perkins, Edward A., Jr., and Byrd, F. Ross. A Research Model for Identification of Task and Knowledge Clusters Associated with Performance of Mnjor Types of Office Employees' Work, Final Report Number 5, Project No. ERD-257—65, Contract No. OE—5—85-109. Pullman: Washington State University, December, 1966. 292 Perkins, Edward A., Jr.; Byrd, F. Ross; and Roley, Dennis E. Clusters of Tasks Associated with Performance of Mnjor Types ofygffice Work, Final Report, Project No. 7-0031, Grant No. OEG-4-7- 070031—1626. Pullman: Washington State University, January, 1968. Poland, Robert, and Haines, Peter. A Studyyof a Block Time Schedule for Teaching_Vocationa1 Office Practice, Final Report,'__ Project No. 201, Grant No. OEG-3-7-070211-2679. East Lansing: Michigan State University, 1969. Shulman, Lee 8.; Loupe, Michael J.; and Piper, Richard M. Studies of the Inguiry Process. East Lansing: Michigan State University, 1968. Articles and Periodicals Byham, William C. "Assessment Centers for Spotting Future Managers." Harvard Business Review (July-August, 1970), pp. 150-167. Casebier, Virginia. ”A Time Study of Activities and Responsibilities of Secretaries with Implications fer the Training of Prospective Secretaries." Dissertation Abstracpgp Ann Arbor: University Microfilms, 1957, pp. 2875-2876. Cook, Fred 8., and Shapiro, Edward Gary. "Factors Affecting Secretarial Success." BySiness Education Forum, Volume 24 (February, 1970), pp. 33-34. ‘ Crawford, T. James. "Typewriting and the Vocational Education Act of 1963." Selected Reading; in Business and Offico_0ccupations, National Business Education Yearbook, No. S (1967), pp. 95-99. Cross, W. Ray. "Relationships Between Elementary School Principals' In-Basket Performance and Their On-the-Job Behavior." The Journal of Educational Research, Volume 63, Number 1 _——' (September, 1969), pp. 26-30. Drenth, J. A. "Simulation = Stimulation, Realism = Relevance." The Balance Sheet (October, 1966), pp. 61+. Elstein, Arthur S., et al. "Methods and Theory in the Study of Medical Inquiry." Office of Medical Education Research and Development, College of Human Medicine, Michigan State University (Summer, 1971). Elstein, Arthur 8.; Loupe, Michael J.; and Erdmann, James B. "An Experimental Study of Medical Diagnostic Thinking." Journal of Structured Learning, Volume 2, Number 4 (1971), pp. 45-53. 293 Fiske, Donald W. "Why Do We Use Situational Performance Tests?" Personnel Psychology, Volume 7 (1954), pp. 464—469. Flanagan, J. C. ”Some Considerations in the Development of Situation Tests." Personnel Psychology, Number 7 (1954), pp. 461~464. . "The Critical Incident Technique." Psychological Bulletin, Volume 51, Number 4 (July, 1954), pp. 327-358. Frederiksen, Norman. ”Factors in In-Basket Performance.” Psychological Monographs: General and Applied, Volume 76, Number 22, Whole No. 541 (1962), pp. 1-25. Frederiksen, Norman; Saunders, D. R.; and Wand, Barbara. "The In-Basket Test." Psychological Monographs: General and Applied, Volume 71, Number 9 (1957), pp. 1-27. Greenlaw, Paul S. "The In-Basket as a Training Instrument." Marketing Keys to Profits in the 1960's. Chicago: American Marketing Association, 1960, pp. 452—459. Hanson, Garth A., and Stocker, H. Robert. "Mobile Simulation in Office Education.” Business Education Forum (October, 1968), pp. 17— 18. Helfer, Ray E., and Slater, Carl H. "Assaying the Process of Reaching a Clinical Diagnosis.” University of Colorado Medical Center, pp. 79—87. Iannizzi, Elizabeth. ”Preparing Office Clerical Workers Through Experience in Making Decisions.” Business Education Forum (February, 1967), pp. 9—13. Jackson, Sharon McNutt. ”Applications of the New Media in Secretarial Office Practice." New Media in Teaching the Business Subjects, National Business Education Yearbook, No. 3 (1965), pp. 149- 160. Kallaus, Norman P. "Hardware: A New Man—Machine Interface." A Taxonomy of Office Activities for Business and Office Education, Interim Report, Project No. 7—1223, Grant No. OEG—1-7-071223—Sl34, Research 12. Columbus, Ohio: The Center for Vocational and Technical Education, July, 1968. Kelly, E. Lowell. ”The Place of Situation Tests in Evaluating Clinical Psychologists.” Personnel Psyghology, Number 7 (1954), pp. 484-492. Klein, Joseph F. "Adequately Trained Secretaries——A Follow—Up Study." The Balance Sheet (March, 1969), pp. 292—3+. Lanham, Frank W. "Electronics and the General Clerical Program.” Business Education Forum, Volume 16 (February, 1962). 294 Lanham, Frank W., and Weber, Cathryn P. "NOBELS Action Verbs and Synonyms, Part I.” The Journal of Business Education (April, 1970), pp. 284—290; "NOBELS Action Verbs and Synonyms, Part II," The Journal of Business Education (May, 1970), pp. 329—330+. _______________._________.________._. Lehnert, Marie, CPS. ”P. S. for Private Secretaries.” Volume 14, Number 7. Waterford, Connecticut: Bureau of Business Practice, Inc. (April 15, 1971), pp. 1—2. Leles, Sam. ”Using the Critical Incident Technique to Develop a Theory of Educational Professionalism: An Exploratory Study." The Journal of Teacher Education, Volume XIX, Number 1 (Spring, 1968), pp. 59—69. McGuire, Christine H. ”An Evaluation Model for Professional Education—- Medical Education." Proceedings of the 1967 Invitational Conference on Testin Problems. Princeton: Educational Testing Service (1968), pp. 37-52. Meyer, E. Dorothea. "The Secretary to the Engineer: A Study of the Personal Qualifications, Training, and Experience Expected or Desired of Secretaries Who Work for Mechanical Engineers in the State of Connecticut.” Dissertation Abstracts. Ann Arbor: University Microfilms, 1954, p. 2242. Mickelson, Harold. ”Knowledge, Skills, and Personal Qualities of Medical Secretaries.” The National Business Education gnarterly (Fall, 1957), p. 49+. Mitchell, William, and Olson, Adelyn. ”Who Needs Shorthand! Who Needs Secretaries!” Business Education World, Volume 50, Number 6 (May—June, 1970), pp. 6—7. Myers, Charles. ”Secretarial Education in an Office—Like Environment.” The Balance Sheet (May, 1970), pp. 396—397+. Oliverio, Mary Ellen. ”Projections for the Future." Business Education: An Evaluative Inventory, National Business Education Yearbook, No. 6 (1968), pp. 118—127. "Teaching Clerical Practice Students How to Work.” . Selected Readings in Business and Office Occnpations, Nat1onal Business Education Yearbook, No. 5 (1967), pp. 76—78. Paddock, Harriet L. ”Personnel Needs for High—Level Secretarial P051— tions.” Delta Pi Epsilon Journal, Volume XI, Number 2 (February, 1969), pp. 17—22. Patrick, Alfred. ”Strengths of the Past and Present.” Business Education: An Evaluative Inventory, National Bu51ness Education Yearbook, No. 6 (1968), pp. 5-15. 295 Perkins, Edward A., Jr., and Byrd, F. Ross. "Identifying Office Activities by Clusters of Tasks." The Emerging Content and Structure of Business Educationl_National Business Education Yearbook, No. 8 (1970), pp. 104-112. Rimoldi, H. J. A. ”Rationale and Applications of the Test of Diagnostic Skills." Journal of Medical Education, Volume 38, Number 5 (May, 1963)——, pp .‘Térsss. "The Test of Diagnostic Skills." Journal of Medical Education, Volume 38, Number 5 (May, 1963), pp. 73-79. Robertson, Leonard. "Everything You Always Wanted to Know About Preparing for Part V." The Secretary (March, 1971), pp. 12- 13. Shulman, Lee 8. "Reconstruction of Educational Research." Review of Educational Research, Volume 40, Number 3 (June, 1970), pp. 371-396. Stierwalt, Irma Lee. "Help Prospective Secretaries Develop Decision- Making Ability." The Balance Sheet (May, 1969), pp. 401-402. Travis, Dorothy L. "Developing Decision-Making Ability in Office Employees." Business Education Forum, Volume 19, Number 6 (March, 1965), pp. 24-26. Weislogal, Robert L. "Development of Situational Tests for Military Personnel." Personnel ngchology, Volume 7 (1954), pp. 492- 499. Weitman, M., and Coisman, F. "Medical Student Pathways to Diagnosis." Journal of Medical Education, Volume 40 (1965), pp. 166-179. Welling, Mary. "A Survey of the Duties and Activities of the Secre- taries of High School Principals." The National Business Education Qparteriy, Volume XXVI (Fall, 1957), p. 69. West, Leonard J. "Research on Teaching Business and Commercial Subjects." Research Report 7l-l, Research and Evaluation Unit, Division of Teacher Education, The City University of New York (January, 1971). Wink, Barbara Ann. "An Analysis of the Duties of an NROTC Secretary." The National Business Education Quarterly, Volume XXV (Fall, 1956, p. 62. "You Too Can Be a Certified Professional Secretary." The Secretary_ (October, 1970), p. 39. 296 Booklets and Pamphlets ”The In-Basket Exercise." Training Guide TG 5-19. Washington, D. C.: Office of Assistant Administrator for Personnel, Veterans Administration (July, 1961). Personnel Assessment Program. Detroit: Michigan Bell Telephone Cempany. U. S. Department of Labor. Adjustments to the Introduction of Office Automation, Bulletin 1276. Washington, D. C.: Government Printing Office. Instructional Materials ”All in a Day's Work.” The Secretary (October, 1970), pp. 26-29. Center for Educational Development. Patient Manngement Problems. University of Illinois College of Medicine. ”In—Basket Project.” Executive Secretarial Seminar. Chicago: Montgomery Ward. ”The Secretary's In-Basket.” The Secretary (June—July, 1971), pp. 28— 31. Witherow, Mary. The Secretary on the Job. New York: McGraw—Hill Book Company, 1967. Zoll, Allen A., III. The In—Basket Kit: Materials for the Creation and Use of In—Basket Materials. Reading: Addison-Wesley Publishing Company, 1971. Unpublished Material Beaufore, Jerry Ann. ”Four In—Basket Projects for the Secretarial Practice Class Based on Follow—Up Study and Job Analysis of the 1965—1967 St. Johns High School Business Graduates." Independent Study. East Lansing: Michigan State University, 1968. Kosy, Eugene J. "The Critical Requirements for Private Secretaries Based upon an Analysis of Critical Incidents." Unpublished doctoral dissertation. University of Wiscon51n, 1958. Noyes, Honora M. ”The Role of the Executive Secretary as a Member of Management.” Unpublished Ed. D. dissertation. UniverSity of Maryland, 1960. 297 Ogunniyi, Omotosho. ”The Methodology of Educational Simulation and Design of a Simulated Instructional Model for Occupational Education.” Unpublished Ph. D. dissertation. Michigan State University, 1969. Place, Irene. ”A Study of Personal Secretaries in Sixteen Communities in the State of Michigan." Unpublished Ed. D. dissertation. New York University, 1945. Ringeisen, Carolyn Jean. "A Model Office for Secondary Schools." Independent Study. East Lansing: Michigan State University, 1969. Siefert, Dolores. "The In—Basket and Its Application to Office Education.” Independent Study. East Lansing: Michigan State University, July 31, 1967. Wagoner, Kathleen P. ”The Role of the Secretary in a Changing World: An Analysis of the Duties and Functions Performed by and Understandings Required of the Secretary." Unpublished Ph. D. thesis. University of Iowa, 1967. Wilms, Lynda Britt. ”Critical Requirements for Beginning Office Workers Based upon an Analysis of Critical Incidents.” Unpublished M. A. thesis. The University of Georgia, 1969. Young, John R. ”A Study of Secretarial Duties with Special Attention to Those Duties Which Emphasize Bookkeeping or Accounting Func- tions.” Unpublished Ph. D. dissertation. State University of Iowa, 1954. Other Sources Bentz, V. Jon, Director of Psychological Research and Services, Sears, Roebuck and Company, Chicago, Illinois; letter dated December 3, 1969. P., Training Services Manager, Montgomery Ward; letter dated November 25, 1969. O Breen, Frederiksen, Norman, Director, Educational Testing Service, Princeton, New Jersey; letter dated November 25, 1969. Greenlaw, Paul 8., Professor of Management, The Pennsylvania State University; letter dated November 21, 1969. Huck, James H., Personnel Assessment Center, Michigan Bell Telephone Company, Detroit, Michigan; interview on September 29, 1971. 298 Meyer, Herbert H., Manager, Personnel Research, General Electric Company; letter dated December 2, 1969. Popham, Estelle L., Chairman, Department of Business Education, Hunter College of the City University of New York; letter dated December 18, 1969. Scott, Hugh M., M. D., Royal Victoria Hospital, Montreal 112, P. Q.; letter dated March 3, 1970. Wisner, Roscoe W., Supervisor, Personnel Testing, The Port of New York Authority; letter dated December--, 1969. 2011, Allen A., 111, Management Education Adviser, The Boeing Company; letter dated November 24, 1969. 1,; E .32; . '7 .1 5w . APPENDICES DEVELOPERS OF IN—BASKET SIMULATION FOR SELECTION, TRAINING, AND EVALUATION CLASSIFICATION OF SECRETARIAL PROBLEMS: Table 1: Classification of Secretarial Problems (by Day of Week) Table 2: Classification of Secretarial Problems (Frequency) Table 3: Classification of Secretarial Problems (by Years of Experience) Table 4: Classification of Secretarial Problems (by Time of Day) 299 APPENDIX A DEVELOPERS 0F IN-BASKET SIMULATION FOR SELECTION, TRAINING, AND EVALUATION The following individuals were contacted during the preliminary planning of the study because of their experience in develOping in— baskets for selection, training, and evaluation: 1. V. Jon Bentz Director of Psych010gica1 Research and Services Sears, Roebuck and Company, Chicago, Illinois 60607 2. C. P. Breen Training Services Manager, Montgomery Ward 619 West Chicago Avenue, Chicago, Illinois 60607 3. Norman Frederiksen Director, Educational Testing Service Princeton, New Jersey 08540 4. Paul S. Greenlaw Professor of Management, The Pennsylvania State University, University Park, Pennsylvania 16802 5. Herbert H. Meyer Manager, Personnel Research, General Electric Company 570 Lexington Avenue, New York, New York 10022 6. Estelle L. Popham Chairman, Department of Business Education Hunter College of the City University of New York 695 Park Avenue, New York, New York 10021 7. Hugh M. Scott, M. D. Royal Victoria Hospital 687 Pine Avenue West, Montreal 112, P. Q. 8. Roscoe W. Wisner Supervisor, Personnel Testing, The Port of New York Authority, 111 8th Avenue, New York, New York 10011 9. Allen A. 2011, 111 Management Education Adviser The Boeing Company, Seattle, Washington 98124 300 HH Ou—cOO MNHN em OH NH ”NCO V'Nv-OLD ma AH HHHN NHHN mooH COOL!) l OOOO Nd'r-Om om HHv—ifr V we onHnuonHmmme .m N N o o o oH mmcHuOOz paw moocoaomcou mancmHH< .N v N m N H NH mu:oEu:Homm< mmochsm mcHHspocom .H o v m w H NN mHZMmezom pew wchHpm .m m m N H N HH mcHHOOHHQDQ paw mcHAmou .N H H H m o e mfimomaoo . H MN mH VH ON mH mm mmmm0Haa< .m o N o H o m mcoHumoMMSm no mcoHumpcoEEooom wcHnommo .N N N H N H w mEoHnonm o>HpmechHEp< mcHNNHmc< .H m e N v H vH mzmqmomm MHHHmmaDm .m HpoacHocouv mmHHHHHmHmzoammm qummmuoma .HH mEOHQOHd as 2 H 2 mo Honesz moHHowoumu EOHpoam HeoscHocooV H mHmHHOQ Hmcomhom . mommmmoz N OQNHOHOHIEwHwOHok mH HHmz LnMI—ir’) MMHN NHHm NOr—(N‘) M000 ‘2' HNMsr 3()3 NH m m o m mm omamoummuuonHuuonH1120HhNNN=N .N o H N H oH mmcHuooz pee uneconomcou mchcenu< .N N m m N NH mugoEucHoma< mmochzm mcHHsvonow .H N N N N NN NNZNZNNZNNNN NNNzHNNN NzHHNZN: .o NH N N N NN «NNN oz< onNNN New NcHNHem .N N N N N HH NcHNNoHHasN Nee NeHNaoo .N N N o H N NeHNoaeou .H NN NH NH NN NN NNNNNN oza Nszzsooo NNNzHNNN NzHNNNNNN .< NNHNHHHNHNzoaNmN NzHNNNooNN .HH mEoHponm NH.ouo Nngcoz Nonoz NHHmo mo nonssz NOHNOMONNU aoHnoum HNosercooN N NHNNN m xHszmm< 306 vm em mm em N o o o H o o o N H o o o H H 0 HH N H o mHN mmHHHHHmHmzommmm qummmuomm HOHmm< .m m m:OHumomm:m Ho mcoHpmpcoEEooom wcHNommo .N w NEOHQOHQ o>HumHuchHEp< wcHNNHmn< .H NH mZmHmomm m>HHmmmDm .m mmHHHHHmHmzommmm Uszwmuomm .HH NH.ouo NHcpqoz NHHNNz NHHNN mEoHponm Ho Nopssz NOHHomoumu EOHDONN HeoseHNeouN N NHNNN N xHazmaa< 307 NN NH ON HN NHH onNNOHzazzoo quooNbo HHHNN Hmeomsoa .N m H o o v mommmmoz .m N N H o N NNNHNHNN-ENNNNHNN .N N N N N NH HHNz .H NN N N N NN NNNNOUNN--onNNN NNN NNHHHNN .N H H N H H H H H HH NNHHNNHHNNN NNN NeHNNoo .N N H N H N NNHNoNeoo .H mm mammHm one: oocoHuomxo mo memo» on HNHNNHNcN NHnu scum popsHoxo one maoHnonm NH « NH NH NH NH m mm mH NH 0H m NN m 0H mHN mmHNHHHmHmzommmm oszmmuomm Honmg< .m H H H m mcoHumommzm Ho NcoHumpcanOOom mcHHommo .N m H H H H H m NEOHHONQ O>HumuuchHEp< mcHNNHmn< .H NH m2mHmomm m>HNzmmzm .m m N H H N N HH son :oHumEHomeH mcHHHoaunou .m H H H N H H H NH NNNNN NH HoeeoNHNN NNHNNHNNN .N. H N m H N m N H m N NN onm Human wchoonHo .H we monNmmm3m .m mmHHHHHmHmzommmm oszmmUQmm .HH Nm om mN VN mN 0N mH NH mH HH OH 0 m mEoHnonm oocoHHomxm mo Name» mo Nonezz NoHNomoumu EoHnoam HNNNNHHNNNN N NHNNN N xHNZNNNN 311 NH N N N H NN N NH N N NH H NH NHH onNNNstzzNN NzHNNNNN HNNNN H1. Ha. H1. 11. H1. m1. 11. 11. 11. 11. H1. 11. 11. N onNNNstzzoN NzHNNNNN NNNNZNHHNNNHE .o H H N N H H N NH NHN>HHNN HNeoNNNN .N H N H N mowmmmoz .m H H N N H N NNNHNHNN15NHNNHNN .N H H N H N H H NH HHNE .H NN NNNNNNNN--ZNHH11onHmmm2m .m N H N v oH mmchomz can mmocoymmcou mchchn< .N H H m H v NH manoEucHomg< mmmchnm wcHHsvmcom .H NN mHZmmezma cam Nchme .m H N N v HH mcHumoHHmsa nan msHNmou .N N H N o maHmONEOU .H mm mmmmHm mmz xmw mo oEHp on HmmeHmcw mHsp Scum novsHUXm mum mEoHnoum NH N o mH HH NH wH 0 mm mN ow mv mHN mmHHHHHmHmzomwmm oszmmuoma H0Hmm< .m H N m mcoHpmmmwsm No mcoprowEEoomm wcHhommo .N m mEmHQONQ m>HNMNuchHEw< mcHNmec< .H vH mZmHmomm m>HHmmmsm .m mmHHHHHmHmzommmm oszmmuomm .HH w v m N H NH HH 0H m w mEmHDOHm Ame mo mEHH mo Honfisz mmHHommumu EoHnonm $25388 v Em: m xHazmanE ‘ mawm‘nfi-d-HHEH‘.‘ w—V' 315 0 MH NH m N m mH OH mN 0H mHH onNHHoo Hmcomhom .v H H H H v momwmmoz .m H N H H N N mmNpmHoH-amammHoN .N N m N H N N H mH HHmz .H mm omamoumm--onN--onN