,V',~‘_~ . THE PREPARATION OF CIVIL SERVICE EXAMINATIONS : A CASE HISTORY OF THE ~ EMPLOYMENT SERVICE EXECUTIVE SERIES ThesIs ”for the Dogma of M. A. MICHIGAN STATE COLLEGE Norman Berkowifz 1953 , 7- - Man-5| -. l l"-.'..";=~1.""._ _,_. WT“ ‘Iumjnmmnn ’Ii 12mm: I L 93 008 124 i This is to certify that the thesis entitled "The Preparation of Civil Service Examinations: A Case History of the Employment Service Executive Series." presented by Norman Berkowitz has been accepted towards fulfillment of the requirements for Master's degree in Political Science f <%/ I Major professor Date June 1, 1°53 0-169 PLACE IN RETURN BOX to remove this checkout from your record. TO AVOID FINES return on or before date due. DATE DUE DATE DUE DATE DUE APR “9191?» ' MSU Is An Affirmative Action/Equal Opportunity Institution c:\c|rc\dnsduer pma-p. 1 THE PREPARATION OF CIVIL SERVICE EXAMINATIONS: A CASE HISTORY OF THE EMPLOYMENT SERVICE EXECUTIVE SERIES by Norman Berkowitz A Thesis Submitted to the School of Graduate Studies of Michigan State College of Agriculture and Applied Science in partial fulfillment of the requirements for the degree of MASTER OF ARTS Department of Political Science 1953 6 80:1 '1‘. .no "I. ‘0 "'év A! -«.io ‘96 QWVAB 0i J AC KN O i v LEDGEL’LEN TS The author wishes to acknowledge the guidance and assistance of Dr. R. Vance Presthus, Department of Political Science, under whose general supervision this work was prepared. He wishes to express a particular indebtedness to Mr. Paul T. Anderson, Chief of the Examination Section, Michigan State Civil Service Commission for his contribu- tion toward the writer's knowledge of the field. The writer further wishes to express appreciation to Mr. Milton.Lavine, Chief, Classification and Pay, City of Philadelphia Civil Service Commission, for his review and advice on the sections of the thesis dealing with classifi- cation. 'M 'i ’LJ C [3‘ \v I‘ x. t) 'J‘i " cog- ”j *' .025 ‘al u-:-g A? .. dd. A - 7:1“? LA. “an: . -V" ‘1 k. ‘5‘ .l O. nonv— 5.0 3.1.3 A< P Lula. “. a“ .25. 71937 .-, ‘._‘ O... 2_ _.g Do" U. 9‘ 1" be; "1‘5 o’. ‘-~:‘ W :em I A Pro: Os.-. u—I .VV‘ . w ASS 3' I) t, 'm T'r} m l O ‘l-{J (r) [’1 J: O - r O h .9..‘ i TABLE OF CONTENTS LIST OF TABLES .OOOOOOOOOOOOOOOOOO0.00.00.00.00...O. LIST OF NGURES 0.0000000000000000000000000000000000 CHAPTERS I. II. III. VI. INTRODUCTION ooooooooooooooooooooocooocococo. THE PROBLEM OF CLASSIFICATION ooooocooooooooo Classification Factors ...................... Definitions or Class Levels ooooooooooooocooo Review of Class Specifications .............. MININUM AND DESIRABLE REQUIREMENTS coco-coco. Definition of Minimum Requirements .......... Definition of Desirable Requirements ........ TIE PUBLIC AA‘NOUIJCEI'EIJT OOOOOOOOOOOCOOOOOOOOO General Considerations in Drafting the Public Announcement .............................. PromOtional PrOblemS ooooooocoooooooooooooooe Open Competitive Considerations ............. Assigning Relative weights .................. EDUCATION AND EXPERIENCE..................... IntrOdUCtion cooooooooooooooooooooooooooooooo quality Ratings 00.00.0000....OOOOOOOOOOOOOO. Types of Rating Charts ...................... Appeals from Ratings ........................ {Ill-IE VJRITr-EEN TEST 00......OOOOOOOOOOOOOOOOOOO. Early Testing Procedure ..................... Bases for the hritten Test .................. Types Of Items 00000000000000oooooooooooooooo completion Items coco-00000000000000.0009coco MUltiplS‘ChOice Items ooooooooooooooooooooooo True-False Items coooooooooooocoocoooooocooo. 'Sources Of Items oooooooooo0.0000000000000000 Time Allowances ............................. Weighting the Parts of the Test ............. StfltiStiCal Procedure 000.000.000.00000000000 Setting the Passing Point ................... TeSt Analy318 cocoooooooooooooooooooooooooooo -1- Page iii iv aflhvl U1 i4 C) l 10 10 13 w -‘ 5 v.50" no. he ‘— l.l. U..s~bn -V‘" ‘0‘ J5 Staf 11-..”, ‘Y..u ' o I I ‘ ”‘- L.“‘ It... a... 53?: I:TQ¢‘- 3-...:._X D ‘.k‘ mo L-w Y c. .1; (r““T-— .N'V .‘LD (1) CHAPTERS VII. ORM-l INTERVIEV’U'S 0....OOOOOOOOOOOOOOOOOOOOOOOO Introduction .C...‘.......................... Staffing the Oral Boards .................... Type Of Examining Process 00.000000000000000. Limitations coco...coo.00000000090000.0000... VIII. WRIT RATINGS 0.0.000000000000000000000000... Service Ratings.............................. PromOtional POtential Ratings 00000000000000. Restrictions on a Promotional Program ....... IX. SUI‘MARY OOOOOOOOOOOOOOOOOOOOO0.0.0.0....0.... APPENDIX A. ”Committee" protest of Employment Ser- vice Executive Examination Results .... APPENDIX B. Special Report on Service Rating Program 000.00.000.0000000oooeooooooooo APPENDIX C. Derivation of Multiplying Factors Em- ploying an Average of the Effective Range and Inter-Quartile Range Methods. SELECTED BIBLIOGRAPHY 0.00.0000...OOOOOOOOOOOOOOO... Page 92 92 98 98 100V}. 102 / 10h 106 111 118 120 o o p O ' Q C I I I . a o O O C . . n o o o 0 O O O l I o v 0 O O C o a - a O O Q 9 t o . ~ I I I O I I Q l C o C I c I v ., O a 0 o - O O I O l O O I O I I C O I I I O O I I O o o e O O o o O I 0.. , Q ‘_. '7’ ‘7 I.‘. I)! in] a fun LIST OF TABLES TABLES Page I. RELATIVE WEIGHTS OF PARTS OF THE OPEN- COhTPETITIVE EXMEIN'ATION 00000000000000000000 20 II. RELATIVE WEIGHTS OF PARTS OF THE PROMOTIONAL EXAPJINATION 000......OOOOOOOOOOOOCOOOOOOOOOO 20 III. EMPLOYMENT SERVICE EXECUTIVE PROMOTIONAL EXAMINATION EXPERIENCE AND TRAINING TABULA- TIONS 0.......00....0.0.0.000000000000000... A9 IV. EMPLOYMENT SERVICE EXECUTIVE OPEN COMPETI- TIVE EXPERIENCE AND TRAINING TABULATIONS.... 50 V. RESULTS OF POOR SECTION WEIGHTING TECHNIQUE.. 77 VI. VARIOUS METHODS USED TO DERIVE MULTIPLYING FACTORS OOOOOOOOOOOOOOOOOOOOOOO0.00.00.00.00 79 VII. ENGLISH USAGE SECTION STATISTICS ............ 82 VIII. RESULTS OF EFFECTIVENESS OF ITEMS FOR THE ENIPLOYIIENT SERVICE EXECUTIVE II 0 0 0 0 0 0 0 0 0 0 0 0 90 IX. DISTRIBUTION OF SERVICE RATINGS IN THE FIVE PERFORMANCE RANGES COOOOOOOOOO00......000... 100 -iii- is: ‘o-“ p I‘ ..—~—0 0.. n. .4.“ 94 «A w. P F .3 .5 O l A By n; r. by .1 . 1‘ “me r1 Train 0 O O O O O ' O a c t C a... .8 x a. S B n. . .. 9v n C. I. I Tu 55 m I .. . 0‘“ 6b » a. n Y“ Y... Pu flue w.“ lh« u 0 b C his T Q" LIST OF.FIGURES Figures 1. 2. 3. 9. 10. 11. 12. Table of Weights Used on Examinations in l9h6.. Experience and Training Chart for Game Farm- handc .00....00.0.0000...OOOOOOOOOOOOOOOOOOOO Experience and Training Chart for Liquor Store mmager A2 .0O0.00......OOOOOOOOOOOOOOOOOOOOOO Experience and Training Chart for General Clerk B 0.000.000.0000.00000000000000000000000 Experience and Training Chart for Liquor Law Enforcement EXGCUtive III000000000000000000000 Experience and Training Chart for Budget Ex- minei’ III OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO Experience and Training Chart for Buyer Trainee I .0.000000000000000000000000000000000 Experience and Training Chart for Unemployment Claims Examiner I ............................ Experience and Training Chart for Employment SBrVICe EXECUtive Series 000000000000000000000 Tentative Experience and Training Chart for Screening Purposes Only for Employment Service Executive V - VI 00000000000000000000000000000 Experience and Training Chart for Insurance Examiner IIIa 00000000000000000000000000000000 Examination Outline for the Employment Service EXGCUtive Ia‘II-Ila‘III-IVo0000000000000000000 Sample Examination Item Source Card............ IBM Answer Sheet Marking Procedure 000000000000 Michigan Civil Service Answer Sheet Scoring methOd O...OOOOOOOQOOOOOOOOOOOOOOQOOOOIOOOOOOO Tabulation of Hypothetical Total Weighted Scores 000000000000000000000000000000000000000 -1v- Page 19 26 27 28 29 30 31 32 33 37 S9 60 63 6h 80 O O C O 0 O O O C O I . O . Q :23” ~02. “SEE P7:':clog-s .;‘.i:udes 1 mt tows: :znsmctig is 16.5231 Gover Pisaeers 1 :.le Star.- CHAPTER I INTRODUCTION Constructing tests to predict potential and actual job competence is still a comparatively new field. Psychologists have developed tests in the fields of basic aptitudes and intelligence and they are making a notable start toward the development of personality tests; but the construction of tests to predict success on specific jobs has lagged considerably behind. Governmental civil service agencies have become the pioneers in this field. Most civil service jurisdictions have staffs employed to develop tests pertaining to specific positions. Historically, civil service agencies in the United States have deveIOped narrow job specifications and have recruited personnel to fill these narrow specifications, but this development has not been universal. In England, for example, recruitment has been for broad general classes. British examinations are in liberal and cultural studies and while they might be conducted to secure men with parti- cular qualifications, they are not framed for special appointments to specific positions.1 By contrast, in the United States, because of narrow class specifications examinations have been pointed toward a person with a very specialized group of experiences, knowledges and abilities. The recent trend has been toward agadual r' 1:: by 1:: sin bac’ggr 25:13; by :is trend being disc: ezi since 1 \ sizinis :e “e ~::A . 8 0&9! ‘.Q I“ ‘ . r911116 " d- a gradual reversal. More and more classes have been come bined by public personnel agencies to allow for more exten- sive backgrounds on the part of applicants and for broader . testing by the examining agency. A specific example of this trend in action may be found in connection with the jobs being discussed in this paper. Since this paper was begun and since the examination to be discussed in detail was administered and employed to refer qualified applicants to Jobs, several other classes have been integrated into the Employment Service Executive class. Two of these are Employ- ment Placement Specialist and Employment Counselor. Posi- tions in these classes are now known as Employment Service Executives. In spite of this trend, the large majority of classes found in many agencies are still narrow in concept and civil service testing must accordingly accomodate itself to the techniques of examining for comparatively "narrow" classes. Civil service testing Operates in an area restricted by arbitrary rules and in some cases by the whims of politicians. The civil service movement is comparatively new in this country. The rise of the modern merit agency dates back to the enactment by the federal government of the Pendleton Act in 1883.2 From that time it has gradually extended itself to various states, cities, and counties. In each instance, the authority has been created by legislative stor C033 23121911 is bassible i. 3:;L Va . G.1 v63 . N- «31’; :7-14 act or constitutional or charter amendment. Also, the degree of authority has varied and as a result agencies vary in their "purity” or degree of adherence to the strict merit principles. Within the framework of his own agency, the technician can only attempt to prepare as adequate a test as is possible. The Employment Service Executive series is found in Michigan only in the Michigan.Employment Security Commission. This series is the promotional line for employment service interviewers. The employees of the Michigan Employment Security Commission have had a singular experience with civil service. Civil Service originally came to Michigan employees in 1938 -- examinations were held and civil service staffs were built. The following year, however, the state legislature passed what was known as the “Ripper Act," which removed more than fifty per cent of the employees from state civil service protection. Within two years a constitutional amend- ment was passed which returned civil service to state employees. In l9h2 employees of the Employment Security Commission, whose duties were connected with the employment service functions of the department were federalized and became part of the United States Employment Service. Then in late 19h6 the employment service was returned to the state and the stage was set for the ultimate examination with which this paper mainly deals. One ms tea: state $.11". serv ic a: the pie 9. ‘00 .. 5,. . . Dvuadae Cf One must keep in mind now a group of employees who first became state civil service employees, were then shorn of civil service protection and either were layed off or served at the pleasure of a spoils system, were returned to civil service, were federalized, and finally were returned to state civil service protection. This group of employees had had no examinations for the five years of federal service, therefore all promotions were temporary; and in many cases, because of manpower shortages, promotions were made above the capacity of the employee. Then, on March 7, 19h9, an examination for their class was announced. l. Hiram M. Stout, Public Service in Great Britain, Univer- sity of North Carolina Press, Chapel Hill, 1938, p. 3h. 2. William.E. Mosher and J. Donald Kingsley, Public Personnel Administration, Harper and Brothers, New York, l9hl, p. 2h. To un sit in RC .1... “.48, a s. mst be pr In t1". .p—w O to U1 0 (I) n: CHAPTER II THE PROBLEM OF CLASSIFICATION Classification Factors To understand fully the type of testing program carried out in Michigan, as well as in other civil service jurisdic- tions, a short resume of the Michigan classification plan must be presented. In the public service a practical classification plan is the basic tool for carrying out the functions of testing, transactions and pay determination. The State of Michigan classification plan consists of twenty-one class levels in- tegrated with the compensation plan. Positions are allocated to class by levels in terms of the following factors which may be present in any one case in varying combinations: 1. Extent of important public contacts. 2. Responsibility for policy determination. 3. Conditions under which work must be performed. A. Supervision exercised. S. Supervision received. 6. Place in organization structure. 7. Intrinsic difficulty of work, including variety. 8. Qualification requirements in terms of education and experience. 9. Exercise of independent judgment. lO. Consequence of error. he f0: anti in 1 twin thesw armed d3 3:23.319 . Routine in QR 1e 0" vi or c: O I :1 (D ,i—l “(1 Arrinl-gO’Da? 7‘5: b“ Definitions of Class Levels The following definitions of work levels are used as a guide in the original determination of class level, and within these the particular position or class is further narrowed down in terms of the allocating factors previously mentioned. Routine Worker, Beginner or Learner (D-C-Cl) An employee who does simple, repetitive work re- quiring relatively little attention after it is learned. He works under close supervision and follows detailed directions as to methods. Super- visory responsibilities are limited to patients or inmates, where there is usually a negligible consequence of error. Examples: Food Service Helper D General Clerk C Typist Clerk Cl Helper or Working Supervisor (B-AZ) An employee who helps another with greater knowledge of tools, materials or work processes, subject to frequent checks and controls; or one who reviews or checks the work of others for accuracy or for come pliance with instructions, instructs them in the details of the work according to plans or rules laid down by a superior, and preserves order and industry through personal presence, While at the same time performing part of all of the same work processes. He may have nominal supervisory responsibilities. He has some latitude for the exercise of independent judgment. Only minor consequences of error are present. Examples: Janitor B Cook A2 Unit Head or Journeyman (A-Al) An employee who allots tasks or assignments, gives instructions or advice, lays down methods, plans programs as they are given him and Who is generally responsible for the work produced by the employees under his guidance; or an employee who performs work in a trade, or sub-profession. He receives general and periodic reviews of his work. He will have considerable latitude for the exercise of independent judgment but only in unvarying and well defined Oil"! of« h ks circumstances. There will be a moderate consequence of error present. Examples: Farmer A Carpenter Al Section Head or Beginning Professional (I-II) An employee who follows prescribed plans, programs or general instructions, executes minor details on his own responsibility with only occasional reference to a superior for advice and instructions. He may have considerable supervisory responsibilities. There will be only general and infrequent review of his work. He may have rather broad latitude for the exercise of independent judgment, in varying and occasionally unpredictable circumstances. A substan- tial consequence of error is involved. Examples: Stenographer Executive I Civil Engineer I Division Head or Advanced Professional (III-IV-V) Top An employee who reviews the plans and decisions of assistants and subordinates, makes recommendations on matters of policy affecting the work of subordi- nates and interprets departmental policy. He may be responsible for technical or professional judg- ment, has control over objectives such as purposes of the work or characteristics of the finished pro- ject, and plans for accomplishing the objectives. He receives only irregular review of his work and has a wide latitude for the exercise of independent judgment. Consequence of error may be of extreme significance. Examples: Executive III Hospital Physician IV Buyer IV Management - Specialists (VI-VII) An employee subject to general administrative policies only, not otherwise controlled as to the general administration of his office. His account- ability is primarily for results. He makes decisions on major matters of policy governing the conduct of the work, plans programs and the flow of work. Authority in the case of a specialist may be recom- mendatory only. Extensive supervisory responsibili- ties may be present. He deals with a wide variety of professional and/or administrative activities. He has comprehensive latitude for the exercise of independent judgment in varying and unpredictable circumstances. Consequence of error may have the most serious significance. Public contacts are ex- tensive and important. Examples: Public Utilitias Engr. VI Accountant VII magmatic: features of :Lasses of WT Jute 3‘, I .._l ‘- L- d‘ -36 .“ ‘3‘: aha ‘0 '“-'=rLe“.l sA Fr: _. .U‘. \“ ~~f§§ - ”~- —\ :9... $.— RE 5“- I; ‘ w te‘ \3 "Pr. \ 5 4 “c: “- :‘E w The use of the intermediate levels, Ia, IIa, etc., is generally reserved for those positions in an agency which require additional recognition due to their place in the organizational structure or because of special or unusual features of the work. Employees are allocated to specific classes of positions on the basis of the above factors. Review of Class Specifications The specifications for a class under the Michigan system are divided into five parts: (1) A title; (2) A general defi- nition of the work characteristics of the class; (3) Distin- guishing features of the class; (A) Typical examples of the work characteristics of the class; and (S) Desirable or minimum qualification requirements in terms of education and experience, knowledges, abilities, personal characteristics, and physical attributes. Before formal announcement of the examination is made the class specifications are reviewed by the test examiner. The test examiner reviews the specifications for all of the five factors listed above, but concentrates mostly on the fifth factor. If an entire series is being given, the examiner must check for internal consistency between levels, and for adequate knowledges and abilities for the various levels. He must understand completely the tenninology used. There is a decided lack of definition of terms at this time and conse- quently the examiner must obtain clear definitions in the mien 329.; '3. :rs exa: 393319 0:; .evel new :3? CCllc N kaiser ~ 37%- «=~.»erie . - I5“ ‘ g U—Q -“; \ .‘e \:n ‘ " Sdz'e ? ‘3 .3133 c review stage so that applications can be rated fairly later in the examination process. The biggest problem, however, in reviewing the class specifications at this point is the check for internal con— sistency. A classification plan is not a static instrument, it is subject to constant change. Consequently, it often happens that parts of one specification within a series become out of line with the rest of the series. While one level may call for high school and a higher level may call for college, an even higher level may call for high school. Another illustration would be the requirement for years of experience. Types of experience required might change as one went higher in the series. In some cases the required years of eXperience might become out of line from level to level. The test examiner must analyze all of the requirements to be sure that an equitable experience and education rating is possible. 1. Michigan Civil Service Commission, Memorandum, April, 19h7, p. l. 2’ Ib1d0, pp0 2‘30 Unzer "5.5350. on 13:21:: re eminatio it: exazp; c‘ if.“ P?" ' “<11; 1“; 5‘ 2‘ .. “‘:t* 0‘] 4 «5:1 . 2" Cu i 1e Nuts f, “Le CHAPTER III MINIMUM AND DESIRABLE REQUIREMENTS Definition of Minimum Requirements Under the rules of most merit systems, the requirements listed on the class specifications are considered to be minimum requirements. The applications of candidates for examination who do not meet 2&1 the requirements are rejected. For example, an applicant who had graduated from college and who had obtained three years of experience in an apprOpriate field would have his application rejected if more than three years of experience was required. Similarly, if he had sufficient experience but was lacking the necessary education his application would also be rejected. If there was an age requirement and he did not meet it, his application would be rejected. In other words minimum requirements indicate absolute standards required to compete in the examination. Definition of Desirable Requirements Some jurisdictions have adopted various modifications of the minimum requirement rule. These organizations operate under either a "desirable requirements" system or some com- bination of desirable and minimum requirements. Under the desirable requirements applicants need not meet the require- ments for any particular part of the examination providing they can do sufficiently better than the desirable requirements such as ex 1: the ill 23: De QC: 22:5. {:3 Tzie‘.‘ this sued for 11 on some other part of the test to compensate for the original lack. In practice, this means that a person with an inade- quate amount of education may compensate for his deficiency by adding extra years of experience. However, there are many variations of the desirable and minimum.requirements systems. Some agencies have lifted the minimum requirements on only certain parts of the examination, such as experience and education. If this is the case, then in the illustration above the deficiency in education could not be compensated for except by additional years of experi- ence. Other agencies, have made all requirements desirable. Under this system the applicant could not only have compen- sated for his educational deficiency by adding years of ex- perience, but also by scoring more than the necessary 70% on other parts of the test, such as the written or the oral interview. Similarly, if he did not meet the requirements for one of the other parts of the examination (if he failed the written test) he could compensate for it by having extra amounts of education and experience. In spite of a tendency recently to return at least par- tially to minimum requirements, the Michigan Civil Service Commission in most cases utilizes the desirable requirement system. Since, under this system the scores from any com- bination of parts may substitute for one another so long as the final score totals up to a passing grade of 70%, it is apparent that the applicant is not held rigidly to passing ,“al a 5" d. v; mad the to the 5:: 21323 bef aid he v :cxletei gassed t1: v~..2 ”9-13.16 .‘rce ’h ‘0 -. h‘. N -.,= ‘sp + 12 any particular part of the examination. It should also be noted that in Michigan, veterans preference points are added to the score of the candidates on open competitive examina- tions before the testing procedure begins. 'Most agencies add the veterans preference points after the test has been completed and then only to those who have successfully passed the examination. In Michigan, under the desirable requirement system, with the addition of veterans preference points it is possible to fail all parts of an examination and still have sufficient points to pass the total test. This procedure severely impairs the ability of the agency to carry out its program of employing only qualified personnel. While the agency is not to be commended for initiating the procedure, veteran's groups have constituted a strong enough force in the past to inhibit its desire to change to a more adequate policy. CHAPTER IV THE PUBLIC ANNOUNCEMENT General Considerations in Drafting the Public Announcement After the preliminary reviews have been completed by the examiner, work is begun on drafting the public announcement. While a more or less standardized form is established by most jurisdictions, much tailoring must be done to fit the announcement to the specific examination under consideration. Some of the problems that confront the examiner in the pre- paration of the public announcement are: 1. Levels to be announced. 2. Open-competitive or promotional. 3. If promotional, what departments? h. If promotional, which employees (level with status, etc.)? 5. If open-competitive what residence requirements are to be employed? 6. Weights of the various parts of the examination. Provisional Employees In Michigan the announcing of examinations is condi- tioned by the number of persons employed provisionally within a class; the length of time that provisionals have been in a class; the length of time since the examination was last announced, and the potential future need of the class. Ideally, of course, there should be no provisional em- ployees under a merit system, but the Michigan Civil Service E:&.:“D“ W "000-! a , 0,~ Ow 4“." Luv. --:t :I.“: ’- "“ ,4. -3. "v—. ~“"~-' e..-v} 3339 of >5 I \. ‘ ‘3 K‘ .\ ‘ -‘O‘l 5‘ q q ‘ ‘ s— “v—- ‘1 ‘\ v ‘ “\~ ~ vq‘ ~ “ t \- -“ l‘ “‘l CHAPTER IV THE PUBLIC ANNOUNCEMENT General Considerations in Drafting the Public Announcement After the preliminary reviews have been completed by the examiner, work is begun on drafting the public announcement. While a more or less standardized form is established by most jurisdictions, much tailoring must be done to fit the announcement to the specific examination under consideration. Some of the problems that confront the examiner in the pre- paration of the public announcement are: 1. Levels to be announced. 2. Open-competitive or promotional. 3. If promotional, what departments? h. If promotional, which employees (level with status, etc.)? 5. If open-competitive what residence requirements are to be employed? 6. Weights of the various parts of the examination. Provisional Employees In Michigan the announcing of examinations is condi- tioned by the number of persons employed provisionally within a class; the length of time that provisionals have been in a class; the length of time since the examination was last announced, and the potential future need of the class. Ideally, of course, there should be no provisional em- ployees under a merit system, but the Michigan Civil Service ”Mission “Quad . -v.visional ’0' . 1129 C: mi: I: not t ‘_ “V".‘n tuna... a w .A . .L 1k. HGt C Inter of :rizerion a“? fact: 3f 2'83}, 35 affect the 11%- Commission.has been unable to eliminate them. The number of provisional employees is one of the generally accepted mea- sures of the success of a public personnel agency. In a recent report on Michigan government, the Michigan Civil Ser- vice Commission was severely attacked on this score.1 While I cannot tell Whether the number of provisional employees in Michigan government is excessive, I would like to say that I do not believe a simple statement of fact concerning the number of provisional employees is, in itself, a valid criterion of the effectiveness of a personnel agency. Too many factors such as maintenance of adequate standards, rates of pay, geography, and war-caused manpower shortages may affect the number of provisional employees. Retaining a large number of employees on the payroll for extended periods of time on a provisional basis, however, can be used as a mechanism to subvert the merit principle. A constant struggle is being waged in Michigan, and if a normal labor market exists for the next few years, it is hoped that this problem can be reduced to an insignificant proportion. Promotional Problems When considering which classes to announce, it is de- sirable to announce continuous levels within a series. However, even here, considerations must be given to the cost factor. If no vacancies exist on the III level, but there are many vacancies on both the II and IV levels, the examiner 3y decide if a large sic: may 1 L; the I? that, in a 5’ H. P! ‘ 6-:Ie . cerzein ef t“ ‘9 15 may decide to announce the III level anyway. To the contrary, if a large number of applications is anticipated, the deci- sion may be to eliminate the III level. The cost of announc- ing the III level might be quite high, and if one considers that, in all likelihood, no vacancies in the class will occur during the life of the eligible list, then the basis for eliminating the intermediate level can be seen. The duration of eligible registers in Michigan is three years unless specifically extended by the Commission. The life of eligible registers in other agencies varies; usually it is one, two, or three years. Some agencies vary the life of certain eligible registers, depending on the class. In the series specifically selected for consideration in this paper, all levels were announced. It should be re- membered that no examinations had been given in this series for several years and large numbers of employees were working either promotionally or otherwise on a provisional basis. Since the Employment Service was under federal jurisdiction for several years many employees were hired who had no status, therefore, the decision in this case was to announce the entire series on both an open competitive and a promo- tional basis. The promotional examination was restricted to employees of the Michigan Employment Security Commission. It was further restricted to employees who had status at least at the A level. Limit: 22?. and to ‘ tended to service to Efrem :ec 2:13:15. a .72: exezple scriction c exited fa: mimetic 51:15“ re ti~ 16 Limiting the promotional examination to the one depart- ment and to persons working above the clerical level was intended to confine the number of candidates from within state service to those who were apt to be best qualified. This is a frank mechanism to discourage unqualified candidates from applying, and it can be employed in a combination of ways. For example, in the Employment Service Examinations, the re- striction could have been set at one level below the one applied for, or it could have been set so that only employees working within the specific series (Employment Service Execu- tive) instead of anyone in the department at a certain level could apply. A combination of the two methods described above, or a number of other combinations could have been em- ployed. The current practice in Michigan is to generally restrict promotional examinations to anyone within a depart- ment who is working at no lower than two levels below the examination. This method is used, subject, of course, to mdnimum requirements where they might exist. Class specifica- tions are being written more and more so that persons two levels below may qualify and the reason for this trend is the time lag in giving examinations. In many cases, examina- tions for a series are not given for two or three years. It is felt that restricting employees to only one level above their present position either restricts promotional possi- bilities for extended periods of time, causes an increased number of provisional appointments, or both. :21 a basis, €15 azeuncezm mztoffi ;3ssibl “"C'mCeLs here-var, 17 Open Competitive Considerations In announcing the examination on an open competitive basis, as broad a sample as possible was desired. Conse- quently, besides arranging for the normal distribution of announcements to all county buildings, colleges, and employ- ment offices, the examiner was required to discover other possible sources of qualified applicants and to distribute announcements to them. One major restriction applies, however, to the distribution of announcements. Almost without' exception, public merit agencies Operate under residence requirements. The Michigan Civil Service Commission requires V; six months residence in Michigan to qualify for examinations and this rule is broken only in classes where the manpower shortage is serious enough to cause positions to remain un- filled for long periods of time. Examples of classes where manpower shortages exist at the present time and where resi- dence requirements are waived are Physician and Psychiatric Social Worker. In the Employment Service Executive Series, residence was restricted to Michigan. Assigning Relative Weights The distribution of weights to the various parts of the examination is still one of the unsolved problems of civil service examining. There is hardly any uniformity from agency to agency on this problem. In Michigan weights have followed a zig-zag path. During the time of the administration of the :..J 1"" ‘2‘ e} '13 tile '«V vari‘“ " was r 18 Employment Service Executive Series the following memorandum shows the formula employed to distribute weights to the various parts of the examinations. (See Figure 1, page 19.) At the present time, however, an almost complete reversal has taken place. The emphasis has shifted to the written part of the examination. Weights are now assigned to each examination after a complete review of the various factors involved and there is no longer a formula such as the one presented above which pre-determines the weights of the vari- ous parts. A trend has set in and it indicates that for the near future, at least, the written part of the examination will be assigned the heaviest weight. FUrther, When there are additional parts added to the examination, the weight of the experience and training (education) part will drop rather than the weight of the written part. An interesting develop- ment recently has been the complete elimination of the experience and education factor. This has been done in the Liquor Enforcement Officer II promotional examination, where the assigned weights were 75% for the written test and 25% for the service rating. This class had minimum education and experience requirements. Since the memorandum shown above was used as a guide to distribute weights when the Employment Service Executive Series was announced, the actual breakdown of weights was that shown in Tables I and II (page 20). has: ( -.. .'. & blb.=c U: limit... less £61 I 11‘ (“1 .1 c .— l r—e—coovcp-b—p-K—‘muvtnl 19 State of Michigan CIVIL SERVICE COMMISSION Memorandum To: Recruiting Staff June 20, 19u6 From: Chief of Examination Section Subject: In preparing examination announcements, the following weights should be used on the various parts of the examination, effective as of the date that this memorandum is received. Open Competitive Examinations Oral Inter- Class Written E & T view or Perform! Level Test Applic. Examination ance Test Total 0 60% 110% 100% 01 60% u0% 100% 01 20% h h0% 100% B 60% no% 100% B 201 h0% u0% 100% A2 55% 115% 100% A2 20 h5% 35% 100% A 55% 15% , 100% A 20% 115% 35% 100% A 20% 115% 35% 100% I 50% 50% 100% I 20% 50% 30% 100% I 20% 50% 30% 100% II 1676 55% 100% II 20% 55% 25% 100% II 53; h5% 100% III h0% 6 ‘ 100% III 20% 60% 20% 100% III 60% h0% 100% IV,V,VI,VII u0% 60% ' 100% IV,V,VI,VII 20% 60% 20% 100% IV,V,VI,VII 100% 100% Promotional Examination Class Written E & T Oral Intv. Perf. Service Total Level Test Applic. or Exam. Test Rating, All Levels 35% AO% 25% 100% All levels 15% 35% 25% 25% 100% All levels 15% 35% 25% 25% 100% All levels 50% 25% 25% 100% All levels 75% 25% 100% Figure 1. Table of Weights Used on Examinations in l9h6 '|"' 218 new.) I CI: “-0- _ r- : I t 6 I a S r; A I! . M. s .fi 9% hm: .3 .2 20 TABLE I RELATIVE WEIGHTS OF PARTS OF THE OPEN-COMPETITIVE EXAMINATION Written Exp. and Oral Oral Examination Test Training Intv. Exam. Employment Service Executive Ia 20% 50% 30% II 20% 55% 25% 110 20% 55% 25% III 20% 60% 2q% Iv 20% 60% 20% v* 60% hcz TABLE II RELATIVE WEIGHTS OF PARTS OF THE PROMOTIONAL EXAMINATION Written Exp. and Oral Oral Service Examination Test Training Intv. Exam. Rating_ Empl. Serv. Exec. Ia 15% 35% 25% 25% 11 15% 35% 25% 25% IIa 15% 35% 25% 25% III 15% 35% 25% _ 25% Iv 15% fi,% 25% 25% V* . 35% 25% VI* 10% 35% 25% - *The oral examining board rated the eXperience and training of the candidate also on the basis of the interview. However, the civil service staff con- structed a tentative experience and education chart whichrwas used to choose candidates to appear before the oral examining board. 1. Louis L. Friedland, Personnel Administration in Michigan Government (Lansing, Michigan Joint Legislative Committee on Reorganization of State Government, 1951), p. AS. l a: exec-.29. .I l ‘ 353-1153 , ‘Lfl five “10".“ , .'_ _ A '45 5% 3Q- ‘- c.1320. uh I 1 CHAPTER V EDUCATION AND EXPERIENCE Introduction The basic components of civil service examinations have been the written test and the rating of education and experi- ence. Refinements in the examining procedure have been added hOping to increase the validity of the procedure. Some of these refinements have been the addition to the examination of ratings of promotional potential, service ratings, oral interviews, oral examinations, physical exami- nations, performance tests, seniority, and others, depending on the specific examination under consideration. Some of these additional parts have been weighted and incorporated into the test score, while others, such as the personal characteristics investigation, have been incorporated on a non-weighted basis; that is, either the candidate passes the particular test or he is eliminated from the examination. No points are allowed for having various degrees of the factor under consideration.’ An interesting variation of this unweighted technique was the Detroit Civil Service Commis- sion's administration of an unweighted personality test in the Student Technical Assistant Examination in l9h5. The results of the personality test were used later only as a guide in the oral interview. of civil se me the sin Tne p: is to oval specific w I is stil tea uncert 22 In any case, and in spite of the modifications of the examining procedure listed above, the basic component parts of civil service examinations still remain the written test and the education and experience rating. The primary purpose of rating education.and eXperience is to evaluate the background of the candidate in terms of specific work assignments and educational courses attained. It is still recognized that the validity of written tests is too uncertain at the present time to chance a total evalua- tion on the basis of that one factor. Written tests still remain verbal samplings of the field under consideration. There remains too much change of candidates with recent edu- cational contacts having acquired sufficient vocabulary and knowledge in a particular field to pass a written test at any level. The education and experience part of the examina- tion compensates for this defect by demanding actual experi- ence as well as education and ability to pass a written test. Quality Ratings The rating of education and experience has been a difficult problem in civil service agencies, especially in those employing desirable rather than minimum requirements. Charts have been constructed which quantitatively measure education and experience, but when an attempt has been made to qualitatively rate education and experience the results have not been too satisfactory. The problem, of course, is “so! to obj mier desi zative nee :‘tiiiates mitten s: perience 5 other hen: the Irittt Iith prac 23 how to objectively rate quality. Jurisdictions which operate under desirable requirements are much more in need of quali- tative measurements. Without the qualitative measurement, candidates become able to pass examinations with very low written scores. This is possible where the education and ex- perience part of the examination is heavily weighted. On the other hand, if education and experience is lightly weighted, the written-test-wise candidate, may pass the examination with practically no eXperience. One obvious answer would be to disallow long years of experience if the written test has been failed. This could conceivably be done because apparently the candidate gained nothing from his experience as evidenced by his low written score. But here again, the assumption must be made that the written test has a high validity. With minimum requirements, this problem is greatly reduced, since those candidates who fail the written are eliminated from the test. However, even here, there are factors, such as low passing points and inadequate samples, which do not completely eliminate the problem. To the writer's knowledge, the major attempts to rate education and experience qualitatively have been based on level of experience and progression of experience. Some class specifications have even been written with statements such as "experience of an increasingly difficult nature." This is an attempt at a qualitative approach, but it still has the serious shortcoming of forcing the examiner, usually 0 the beef I I R? areas .231 1 in e: I: fee to emerienc ca: grew 55:60:11 5&3 be tire of 2h on the basis of written statements only, to evaluate the in- creasingly difficult nature of the experience. Types of Rating Charts An education and experience chart is constructed wherever the factor is weighted in an examination. An education and experience chart is used to evaluate and rank the various backgrounds of the candidates. This chart is always based on the requirements of the class as listed in the class specifi- cation.and it is presumably a measure administered fairly and objectively to all candidates. While candidates are rated objectively and uniformily on the basis of the chart, there may be inherent factors in its construction which favor one type of background over another. In practice, all charts, whether by design or accident, tend to show partiality to cer- tain groups. Listed below are a number of charts used by the Michigan Civil Service Commission (pages 26-h0). Each points out a different emphasis, as well as illustrating different types of construction. Each chart is fair and equitable and, administered fairly and objectively by the merit agency, is beyond reproach, legally or otherwise. Yet, the group favored by the particular construction of the chart will be seen at a glance in each case. Before studying the charts it would be well to state some basic principles followed in Michigan: 1. The desirable or minimum requirements will always equal 70% of the weight assigned to education and eXperience.l ”To 2. ” .AOV .~. g. nigh: ass.e 30 1‘31 a‘"ez;ents I m 40 la ‘I'ct be tfic 'mli‘iate 0 Tier: b "i ‘. “hit? 01 . vi) 80339011 3,; PE 25 2. The maximum allowable will never exceed 100% of the weight assigned to education and experience. 3. The fewest points attainable under desirable re- quirements could be as low as zero. ha The minimum attainable under minimum requirements would be 70% of the assigned weight. Candidates not meeting the minimum requirements would be rejected from the examina— tion and receive no score. Figure 2 (page 26) illustrates a simple construction. A decision was made that experience should carry the bulk of the weight, and that a lack of the necessary education would not be too serious in evaluating the qualifications of the candidates. As a result, twenty-six of the forty points were allocated to experience and only fourteen to education. At- tention should be given to the fact that the requirements of high school (10.5) and a background in general farming (17.5) equals twenty-eight points or 70% of the points assigned to education and experience. The lack of emphasis on education can be shown by the score obtained with only the ninth grade plus the necessary experience background; the score received, 25.5, would equal about 63 per cent. On the other hand, a drOp of only one year in experience, with the necessary educa- tion, would produce a score of only 10.5 or about 26 per cent. An attempt at a qualitative approach is represented in the chart by the statement allowing two points for recentness and quality of experience. These two points would not be given to someone who had farmed sometime in his background, but had no recent experience. (Text continues on pagelfl..) p3 ~- 5 LEA. Eminetien Q‘H‘—-fi ‘. I \‘ ~< In Educ--M—l a Eizher (1) 000m £11511 S experience “so“.np— I I 5 . .9 - .3 _ 1‘: :2 _ l - 2 - Must % 211959 I Era; fie 26 Figure 2. Experience and Training Chart for GAME FARMHAND C Examination Date: October 15, 19h9 Weight: h0% DESIRABLE QUALIFICATION REQUIREMENTS: Either (1) a background in general farming, and graduation from high school; or (2) an equivalent combination of the experience and training described above. EDUCATION Grade Points u - 5 - fl 6 - 5 7 - 6 8 - 7 9 - 8 10 - 9 11 - 10 12 - 10.5 1 - 12 2 - 13 Allow 1 point for 16 weeks short course in agriculture. Deduct 5 credit per year if college was not in agricul- tural field. EXPERIENCE Years Points 0 - O 5 - 12 l or farm- 17.5 background 2 - 19 21 E, 22 5 2 6 2:? Allow farm background up to l6 years of age. After 16 years, allow as farm experience. Allow a maximum of 2 points for recency and quality of experience. 1 +1 3:21.115 “031 . :1 J r. 15': v ~-.v'a-d-v.‘ nave azi- traini: ‘ szzcel .Ira A'fl' all .0 s} 27 Figure 3. Experience and training chart for LIQUOR STORE MANAGER A2 Examination Date: August 5, 19111;. Weight: 110% DESIRABLEggUALIFICATION REQUIREMENTS: (1) Three years of retail store experience, one year of Which shall have been as manager and completion of the twelfth school grade, or (2) an equivalent combination of experience and training. EDUCATION EXPERIENCE EDUCATION EXPERIENCE Years Points Years Points Completion 0 10 Completion O 1% of 8th 31; 12 of 10th 5— 1 grade 1 1% grade 1 18 2 1 2 2O 18 a 22 i 20 2; 5 22 5 28 6 25 6 32 7 28 7 36 8 32 8 38 9 36 9 £9 10 38 10 0 ll 9 12 0 Completion 0 l8 Completion O 22 of l2th in 20 of 2 yrs. 3- 25 grade 1 22 college 1 28 2 25 2 32 i 28 36 32 g, 38 5 36 £9 6 38 6 O 7 9 8 O Interpolate for 9th and llth grades and one year college. Allow full credit only for retail store experience as MANAGER above 2 years of regular retail store experience. Deduct h points for lack of any retail store experience as manager but do not go below points shown for no experience for a given amount of training. Give } credit for general retail store experience. Sue year- o :15}; schoo Rate: 28 Figure A. Experience and training chart for GENERAL CLERK B Examination Date: December 17, 1989 Weight: h0% DESIRABLE QUALIFICATION REQUIREMENTS: One year of general office experience, and graduation from high school. Note: One year of general office experience may be substituted for one year of high school up to a maximum of two years. EDUCATION EXPERIENCE Q fiYr. 1 Yr. 2 Yrs. §Yrs. Yrs. Yrs. Less than 8th grade 1 2 3 h S 10 1E 8th 3 6 9 12 16 2o 2 9th 8 12 16 20 2h 28 32 10th 13 16 19 23 28 ho llth 18 21 23 28 80 12th 23 26 28 no 1 year general college 2h 27 28 ho 2 years general college 28 35 he 1 year business college equals 2 years general college. Four years investigat efsuch e) eiinistra Eli train{ a I “ 1: P3“ .29 Figure 5. Experience and training chart for LIQUOR LAW ENFORCEMENT EXECUTIVE III Examination Date: February 18, 1950 Weight: 20% DESIRABLE QDALIFICATION REQDIREMENTS: Four years of experience in liquor law enforcement as an investigator, and graduation from high school; or two years of such experience and graduation from college in police administration: or an equivalent combination of experience and training. EDUCATION EXPERIENCE General Specialized Years Points 10 or less 1 Less than 1 0 ll 1 l 12 7 2 g 1 8 i 2 8.5 7 g 8-75 9.5 5 8 9 10.0 Allow up to 2 points for Specialized education equals over-all quality of com! police administration major. bination of experience and education. Eg. EXperience on a higher level than an investigator tOgether with a college degree in an appropriate speciality, or other similar combinations. engineeri. 5:2”: ation in; .d 1‘ i: amublic 1 college, a: public aimf experience N O~U1F*w n.) H n: J 30 Figure 6. Experience and training chart for BUDGET EXAMINER III Examination Date: October 23, 1950 Weight: 60% DESIRABLE QUALIFICATION REQUIREMENTS: III—Four years of experience in accounting, budget examining, engineering, public health, economics or social work, and graduation from an accredited college with courses in account- ing and finance; or (2) three years of experience in private or public financial administration, graduation from an accredited college, and possession of a master's degree in business or public administration; or (3) an equivalent combination of the experience and training described above. EDUCATION Any major with courses in Major in Acct., Bus oAdo , PO]. 0 Any major Accounting and Sci., Econ., 23 Graduate with no Finance, 23 Major in Engr. major in courses in Major in Social or Social Work, any field Accounting Work or Engr. with courses in of Public and Finance with no courses Accounting and Ad. or in Acct.&Finance Finance Bus. Ad. 12 3 1 7 7 7 2 l2 l2 12 17 2h 27 a 21 28 30 b 32 6 6 7 0 EXPERIENCE Years Points Less than 1 year - O 1 to 2 years 8 l 2 to 3 years - 6 to years - 10 to 5 years - 1h . Add 1 point per year's experience as a budget examiner or procedures analyst or equivalent in a large public or private organization (but no more than 6 points). "d f—‘u (If. Exazination Ev‘rvu ‘91 r 3-3.; .h 3- :1 Graduation in busine s s 901185-43 j Nears 2 3981‘s 0 Years years 310! u: a“ . ‘JP OYJeI‘. 3533 0f ( p‘irfif‘as‘, d 31 Figure 7. Experience and training chart for BUYER TRAINEE I Examination Date: November 19, 1919 Weight: 110;: MINIMUM QUALIFICATION REQUIREMENTS: Graduation from an accredited college with specialization in business or hotel administration or institutional management, or equivalent, preferably with one or more courses in purchasing. EDUCATION EXPERIENCE College in speciality 6 months - l h years 28 2 year - 2 5 years 31 2 years - 3 6 years 33 7 years 36 Experience must be in purchasing. Allow up to one point for over-all apprOpriate- ness of courses including purchasing. lav-I mall have ‘1 payment of 1 m greduet accredited labor relat 1 “3' 90253111 ' 533 to base CtdCt S p: «e ‘ess thz 5.: 's 1""cdbs 351‘ 21‘ T5: Schoo. Eyeer Een; . 3951‘s 5"“ ‘ VJ 5385133 :1." ‘ - 58: 7 ‘8 8‘31 2 ' or 1 I I‘if‘a a . 32 Figure 8. Experience and training chart for UNEMPLOYMENT CLAIMS EXAMINER I Examination Date: November 5, 19h9 Weight: hq% DESIRABLE QUALIFICATION REQUIREMENTS: . Either (I) TEur years of ofTice experience, one year of which shall have been of a responsible nature in the processing or payment of unemployment compensation claims or its equivalent and graduation from high school; or (2) graduation from.an accredited college preferably with courses such as economics, labor relations and public unemployment insurance. Education General Special 12 Li 1 l 3 l 2 2 l i l l 0 0 Any combination on above chart equals 28 points. Add to base of 28 points: 2 points if applicant has A years of college with courses in econo- mics, labor relations, and unem- ployment insurance. 3 points for 5 years of above special education. Deduct 5 points for each year less than high school offered. If less than required number of years is offered, deduct: General experience - h points per year Special experience - 10 points per year For special experience in addition to required amount, add 2 points per year up to maximum allowable with educational equivalent. Special Experience Maximums Allowable High school or less fl 1 year general college 2 years general college 5 Note: Special college equals years general college 6 courses in economics, years general college 8 labor relations and i years special college 10 unemployment insurance. years special college 12 Spec ial Experience Equal s . Claims Examiner A or better experience from 3/h6 General Clerk A experience prior to 3/h6 either in a local office of the Michigan Unemployment Compensation Commission working on claims or in the claims determination section of the state offices. 33 Figure 9. Experience and training chart for EMPLOYMENT SERVICE EXECUTIVE SERIES Examination Date: May 7, 19h9 Weights: OC PX Ia - 35% 35% II - IIa - 55% 35% III - IV - 60% 35% DESIRABLE QUALIFICATION REQUIREMENTS: Ia Level (1) Five years of responsible experience in employment or personnel office procedures, and graduation from high school; or (2) one year of such experience, and graduation from an accredited college with courses in public or business ad- ministration, vocational rehabilitation, vocational education and guidance, or related fields; or (3) an equivalent combina- tion of the experience and training described above. II Level 1 Six years of responsible experience in employment or personnel office procedures, one year of which shall have been in a supervisory capacity and graduation from.high school; or (2) two years of such experience, one year of which shall have been specialized, and graduation from an accredited college with courses in public or business admi- nistration, vocational rehabilitation, vocational education or guidance, or related fields; or (3) an equivalent combina- tion of the experience and training described above. IIi Level 1 S x years of responsible experience in employment or personnel office procedures, two years of which shall have been in a supervisory capacity, and graduation from high school; or (2) two years of such specialized experience, and gradua- tion from an accredited college with courses in public or business administration, vocational rehabilitation, vocational education or guidance, or related fields; or (3) an equivalent combination of the experience and training described above. III Level (1) Six years of experience in employee placement processes including registration, counseling and selective placement, or employer contacts, two years of which shall have been in a supervisory or technical capacity and graduation from high school; or (2) three years of such experience, two years of which shall have been specialized, and graduation from.an accredited college with courses in public or business admi- nistration, vocational rehabilitation vocational education or guidance, or related fields; or (35 an equivalent combi- nation of the experience and training described above. excer‘ie A I 7‘ AU 1. eve ’1- b inner f: .4" \A“ .N u‘ .. UCIU 1 S k; 01' 1‘8 Q 1 1’0 § 369(0 \ l ( 1 PC: U ab C Ce,‘ 3. 3h Figure 9 (continued) IV Level (1) Seven years of experience in employee placement processes including registration, counseling and selective placement, or employer contacts, two years of which shall have involved procedures evaluation, and graduation from high school; or (2) three years of such experience, one year of which shall have been specialized, and graduation from an accredited college with courses in public or business administration, vocational rehabilitation, vocational education or guidance, or related fields; or (3) an equivalent combination of the experience and training described above. All Points used on this chart based on PX - Use conversion factor fer OC Ia II IIa III IV Gen.Spec. Gen.Spec. Gen.Spec. Gen.Spec. Gen. Spec. High School or Less 5 O 6 l 6 2 6 2 7 2 1 Yr. Gen. h 0 5 l 5 2 5 2 6 2 2 Yrs. Gen. LL 0 5 1 5 2 5 2 6 2 3 Yrs. Gen. 3 O h 1 h 2 A 2 5 2 3 Yrs. Spec. 2 0 3 1 3 2 3%; 2 h 1 h Yrs. Gen. 3 0 L1 1 Li 2 11 2 5 2 11 Yrs. Spec. 1 o 2 1 2 2 3 2 3 1 5 Yrs. Gen. 2 O 3 l 3 2 3% 2 h 2 5 Yrs. Spec. % O 2 5 2 1 2 l 2 1 Deduct 3 points for each year less than high school offered. Combinations above equal 2h.5 points. If less than required number of years is offered deduct: lg, II, IIa levels III level IV level 3 points per er—Tor gen. points for gen. 3 points for gen. 3 points per yr. for spec. points for spec. 12 points fer spec. For special experience in addition to required amount, add points according to attached chart. 35 Figure 9 (continued) Added Special Experience Chart EMPLOYMENT SERVICE EXECUTIVE SERIES Examination IV III II - IIa Ia Class level of Experience IV Special IV Related Special 2 III Special 1 III Related Special 2 II-IIa Special .5 1.5 II-IIa Related Special 1.5 2 Ia Special 2 I Special Starting from zero toward Ia but only if minimum no. of yrs. required with par- ticular educational combi— nation is met. 1 1 I Related Special Same as I Special .5 A Only fer more than minimum numbers of years required with particular educational combination. Allow above points per year for all experience above desirable requirements. Special experience equals Employment Service Executive Series. Relating special equals Employment and Claims Branch.Manager, etc. In adding additional points according to above chart, the follow- ing maximums apply: High School or less 5 l to 2 yrs. of gen. college 6 3, h or 5 yrs. of gen. cell. 7 yrs. special college 8.5 yrs. special college 9.5 yrs. special college 10.5 7":11‘6 9 (C II - 128 .~’ 1-; Figure 9 (continued) 36 Allowances for Minimum Requirements EMPLOYMENT SERVICE EXECUTIVE SERIES Level General Special Ia Responsible experience in employment or personnel None office procedures equals "A" level or above in em- ployment work. II - IIa Same as Ia Experience in a supervisory capacity in employment or personnel office procedures equals Employment Service Executive Ia experience or equivalent in some other agency, which could be on the I level. III Experience in employee placement processes in- Same as above cluding registration, counseling, and selective placement, or employer contacts equals Employment Service Interviewer I experience or above. IV Same as III Procedures evaluation experience in employee placement processes equals Employment Ser- vice Executive III experience or above. P: a”?! .bafia n 1 I simmer. 13 / at. ,1: .n m (J 1 4’7wwrur—I’Iz {DAY} 135m MAD—(c? 4 (.0 muu H: r Figure 10. 37 Tentative experience and training chart for screening purposes only for EMPLOYMENT SERVICE EXECUTIVE V - VI Examination Date: May 7, 19h9 Weight: 00 - PX - 1007; DESIRABLE QUALIFICATION REQUIREMENTS: Employment Service Executive V Experience and Training: (1) Eight years of experience in employee placement processes including registration, coun- seling and selective placement, and employer contacts, four years of which shall have involved procedures evaluation, and graduation from high school; or (2) four years of such experience, two years of which shall have been specialized, and graduation from an accredited college with courses in public or business administration, vocational rehabilitation, vocational e ducation or guidance, or related fields: or (3) an equivalent combination of the experience and training described above. Employment Service Executive VI ExperIEnce and Training: (1) Ten years of experience in em- ployment services, five years of which shall have been in an administrative capacity, and graduation from high school; or (2) five years of such experience, three years of which shall have been specialized, and graduation from an accredited college with courses in public or business administration, vocational rehabilitation, vocational education or guidance, or related fields; or (3) an equivalent combination of the experience and training described above. Note: In order to receive a minimum rating of 70% on experience and training, candidates must possess the desirable qualific High School or Less 1 Year College 2 Years College 3 Years General Years Years h'Years 5 Years 5 Years Special General Special General Special ations listed above. V VI General Specialized General Spec. 8 )1 1o 5 7 LI- 9 S 7 1+ 9 S College 6 h 8 College 5 6 i College 6 fl 7 5 College 11 2 5 College 5 h 6 fl College 3 2 h 2 Deduct 3 points Combinations above equal 70 points. for each year less than high school offered. Far specie paints ac Allow “a ~~‘ ' u I n ‘ Q J”. 54- .011?” 38 Figure 10 (continued) If less than required number of years of experience is offered deduct: General - 3 points per year Specialized - 10 points per year For special experience in addition to required amount, add points according to attached chart. Added Special Experience Chart for EMPLOYMENT SERVICE EXECUTIVE V - VI Examination VI V Class Level of Experience 7 7 VII Special 7 7 VI Special 2 3 VI Related Special 5 7 V Special 1 2 V Related Special .5 5 IV Special 1 IV Relatédg Special .5 III Special Allow above points per year for all experience above desirable requirements. In adding additional points according to above chart, the following maximums apply: High School or less - 15 l to 2 yrs. Gen. College 18 3-b,5 yrs. Gen. College 21 yrs. Spec. College - 2h yrs. Spec. College - 27 yrs. Spec. College - 30 :r S e v». Q a: 71 V. C. p ‘93. “SrU. tau Ffiw ~m fink RAN. Figure 10 (continued) 39 Allowances for Minimum Requirements EMPLOYMENT SERVICE EXECUTIVE V - VI Level General Special V Experience in employee place- Procedures evaluation ment processes including experience in employee registration counseling, and placement processes selective placement, or equals Employment Ser- employer contacts equals Emr vice Executive III ex- ployment Service Interviewer I perience or above. or above. VI Same as V. Employment service experience in an ad- ministrative capacity equals Employment Service Executive IV experience or above. Specific Examples Top Training Officer Top Procedures Analyst Employment and Claims Field Supervisor Employment and Claims District Manager Employment Security Exec. OK, related special for V OK, related special for V Related special - III for V - IV for Related special - III for V - IV for Related special - III for V - IV for V - IV for Related special - III for VI VI VI VI .1 “L31 PC U“ , 1 ) se la 3858 the s m C o.» In r\ r‘\ 110 Figure 11. Experience and training chart for INSURANCE EXAMINER IIIa Examination Date: June 17, 1950 Weight: 20% DESIRABLE QUALIFICATION REQUIREMENTS: I17 one year of experience in accounting or auditing, plus two years of insurance company accounting, auditing or examining, plus one year of experience as a senior examiner: and gradua- tion from an accredited college in business administration: or (2) seven years of experience in accounting or auditing, four years of which shall have involved insurance company accounting or auditing or examining, plus one year of experience as a senior examiner and graduation from high school. Senior Insur. Insur. Education Acc't. Acc't. Exam. NOTE: All figures Less than 10 3 7 h on chart additive. I? g g 3 Insurance account- 12 3 h 1 ing may be substi- 1 2% u 1 tuted for general 2 2 u 1 accounting only if yrs. Gen. College 1% u 1 maximum number of 8 yrs. Gen. College 1 h 1 years allowed for yrs. Spec. College 1 3 1; insurance account- 8 yrs. Spec. College 1 2 1 ing is exceeded. Deduct 5 points for each year less than 10th grade offered. For lack of required experience deduct: Accounting - .75 points per year; Insurance Accounting - 3 points per year; Senior Insurance Examiner - 10 points per year. Candidates having additional experience are to have 1 point per year added for Senior Insurance Examining up to a maximum of three years. Candidates meeting maximum experience allowances are to receive additional educational points for: One and two years College 1 point Three and four yrs. Gen. College 2 points Three and four yrs. Spec. College 3 points each grail. college. tisn a co reduced I 1118 adie: shown by cational minim the C2315 m 1:111:21 Figure 3 (page 27) illustrates a different type of con- struction. In this chart an even further bias against educa- tion is illustrated. Separate charts have been made for each grade level from the eighth grade through two years of college. As each chart progresses up the ladder of educa- tion a corresponding number of years of experience is reduced from the amount necessary to obtain a maximum score. The added bias against education in this type of chart is shown by the fact that in Figure 2 a candidate with an edu- cational attainment of the eighth grade could receive a maximum score of only thirty-three points, while in Figure 3 the candidate with an eighth grade education could receive a maximum score of forty points, the same as the candidate with two years of college. Figure h.(page 28) represents a combination of Figures 2 and 3 in that additional years of experience compensate for a lack of education.and allow for a maximum attainable score, but only up to a certain limit, for below the tenth grade no absolute substitution of experience can be made for education. Figure A further illustrates an opposite bias in favor of education. In spite of the qualification require- ments of one year of experience, a candidate with one year of business college and no experience receives 70 per cent of the education and experience allowance or twenty-eight points. This chart also illustrates a type employed where extremely large numbers of applicants are encountered. Only 112 one figure need be noted to obtain the score rather than the totalling of a separate education and separate experience score. Figure 5 (page 29) illustrates a quick drOp away for a lack of education. A candidate lacking only two years of education with the necessary experience would receive eight points or forty per cent. Because of a careful balance between education and experience and the addition of an attempt at a qualitative rating, the maximum score the can- didate with a tenth grade education could receive would be nine points, or forty-five per cent. This shows the balance necessary to allow for both the college and the high school graduate since the specification lists both as desirable requirements. Figure 6 (page 30) illustrates a heavy bias in favor of education. Of the total sixty points, forty are allocated to education and only twenty to experience, thus explaining the refinement possible in rating education. Four different categories have been established which allow a fine differen- tiation between candidates. Another feature incorporated in the chart shows that while the specification lists many related types of experience which qualify an applicant for the position, the chart is constructed so that the person with experience of the specific nature of the position being tested for receives an advantage over the others. This is a good illustration of the effective use of quality of experi- ence in rating education and experience. L13 Figure 7 (page 31) points out the relative simplicity that can sometimes be achieved when minimum requirements are employed. Since graduation from college was listed as a minimum requirement, the chart provides for no educational background with less than the full educational requirements; applications from candidates without such backgrounds are rejected. Figure 7 also shows just how far the examiner can go in biasing a chart toward education. Only three of forty points were allocated to experience, and it is debatable whether even those points should have been so used in the examination. Figure 8 (page 32) is rather complicated and shows a triple bias toward the college trained individual. At the same time it allows the high school candidate a fair oppor- tunity to compete in the examination. The sub-chart within Figure 8 represents the various combinations that will give seventy per cent or meet the desirable requirements. The person who has graduated from.high school needs four years of office experience, one year of which shall have been of a responsible nature in the processing of, or payment of unemployment compensation claims. The person with two years of college needs only two years of the first type, or general, experience, one year of which must have been specific or specialized. And finally, this chart shows that the college person need have no eXperience whatever. every Ci this b8 a? ‘1 more, I: ment.’ -( that a eXperie; 'ith 811 hi; In using this type of chart the examiner begins by giving every candidate seventy per cent or twenty-eight points. From this base figure he then adds or subtracts points to arrive at the final score. Any candidate having exactly the require- ments indicated above would receive twenty-eight points; no more, no less. The chart shows that to the base score of twenty-eight points, five points would be deducted for each year of school less than high school. It also points out that a lack of either the general or specialized types of experience causes points to be deducted. Thus, the candidate with eleven years of school, three years of general experience but no special experience would receive the base score of twenty-eight points minus five points for the lack of one year of school, plus the subtraction of four points for the lack of one year of general experience, and a further deduc- tion of ten points fbr lacking the necessary year of spe- cialized experience. His score, then, would be twenty-eight minus five, minus four, minus ten, or nine points. The first bias in the chart toward the college candidate is represented by the heavy deduction for a lack of the re- quired year of specialized experience. The college candi- date needs no experience and is therefore not subjected to this deduction. The second bias in favor of the college trained candidate is shown in the paragraph adding two points to the twenty-eight point base to candidates with an appro- priate major, and three points to candidates with an bias is 3 amount of w-.. ALHJ V sizilar t Ont appropriate graduate school major but with no experience. This bias is justified by the statement in the desirable requirements, "graduation from.an accredited college preferably with courses such as economics, labor relations and public unemployment insurance." The final education bias is shown by the lower chart setting limitations to the amount of additional points that can be obtained. Figures 9 (pages 33-36) and 10 (pages 37-39) are very similar to Figure 8. Figure 9 was used to rate the candi- dates while Figure 10 illustrates how a chart may be used for purely screening purposes. It was employed to select candidates to appear for the oral examination. Once the can- didates were chosen, the scores were given by the oral examining board, and these scores were then translated into percentages and used for the final ranking process. On the second page of Figure 9 is a sub-chart con- structed in the same manner as the one in Figure 8. The difference in appearance is caused by the large number of classes combined in the one chart. Again, as in Figure 8, the figures on the chart represent the various combinations that equal the desirable requirements. Any of these combi- nations, then, give seventy per cent of the weight allotted for experience and education for the particular class under consideration. The deduction factors for a lack of the re- quired years of education or experience are quite different than in Figure 8. There is a difference of emphasis. A lack ’46 of education assumes a lesser importance. where five points were deducted from twenty-eight for each year less than the twelfth grade in Figure 8, only three points were deducted from 2h.5 for the identical reason in this chart. In the experience deduction factor, the same thinking holds. For a lack of the necessary experience, a lesser number of points was deducted than in Figure 8. This holds for all except the IV level where a basic change was ef- fected in the type of experience needed. The thinking behind these deduction factors was based on the knowledge that practically all (if not all) of the positions would be filled from within the agency on a promotional basis. It was ex- pected in constructing Figure 8, however, that all of the positions would be filled by new employees to the state service. Page three of Figure 9 illustrates a definite attempt at a quality evaluation of the candidates beyond the seventy per cent figure. Page three also includes an added chart limiting the number of points that could be gained by the amount of education, the same type of device employed in Figure 8. However, it has since been modified because of criticism leveled by failing candidates. Page four of Figure 9 shows a concrete definition of the somewhat intan- gible description of the requirements listed on the announce- ment. Ln Figure 11 (page hO) represents current thought as of July, 1950. It can be easily seen that no two charts pro- vide the same emphasis. Discussion is already evident in the agency to provide for alterations and modifications of the philOSOphy embodied in Figure 11. There has been no final answer to the problem of rating experience and educa- tion. A constant striving for improvement continues day by day. At the end of Figure 11 a modification caused by criti- cism of Figure 9 can be seen. No longer is the total amount of experience points limited by the degree of education. All candidates can receive the same amount of experience points. However, in this chart, additional points can be gained by possession of college credits. The net effect of the alteration is the same. But the hOpe is to alleviate criticism aimed at the thought that additional years of ex- perience are of no consequence without additional years of education. The basic change in philosophy incorporated into Figure 11 was the theory that all candidates should have a conceivable Opportunity of passing the examination in spite of their educational background. Constructing the chart of combinations which equal seventy per cent caused the educa- tion factor to drop to the tenth grade. This was a shift from previous charts where the twelfth grade was the base figure. This lowering of the base served two purposes: us first, it allowed candidates with lesser degrees of education an ample opportunity to compete in the examination, and second, it made possible the requirement of additional years of experience to such candidates. Thus, it can be seen that the candidate with a tenth grade education or less needs ten years of experience more than the candidate with four years of specialized college. Further, three of those extra years mustjhave been of the highest type and quality of experience. In short, the object of the chart was to allow a conceivable Opportunity for anyone to meet the requirements, but only with the addition of many years of compensating experience. Appeals from Ratings Education and experience ratings are subject to more controversy than any of the other devices employed in public personnel testing. Opinion as to the value of particular experiences is often subject to intense discussion and the question of whether a senior bookkeeper is doing work equi- valent to a junior accountant illustrates this problem in a lucid manner. To the personnel examiner the bookkeeping experience has a different value than that Of the junior accountant. However, to the senior bookkeeper, perhaps work- ing side by side with the junior accountant, possibly doing identical work in the main, the fairness of the rating may be, and often is, sharply questioned. Because of the need Of depending on Opinion in'rating education and experience and the Objectivity of most of the 119 other testing devices such as the written test, most of the appeal problems connected with the Employment Service Execu- tive examination involved ratings of education and experience. Before discussing some of the specific appeals, some statistics will be presented concerning the results of the education and experience ratings. (See Table IIL.and Table IV, page 50.) TABLE III EMPLOYMENT SERVICE EXECUTIVE PROMOTIONAL EXAMINATION EXPERIENCE AND TRAINING TABULATIONS Passing Papers NO. Re- NO. Rec. E&T NO. ceiving NO.Rec. NO.Bec. NO.Rec. Less Than Level Wt. Exam. max.E&T 31.5:35 28:31.5 2ug5-28 Minimum . IA 35 235 9 58 131 13 33 II 35 206 3 3h 86 21 65 IIA 35 150 a 29 72 19 30 III 35 138 31 75 18 1h IV 35 33 1 5 15 10 3 All Papers IA 35 266 9 58 132 13 63 II 35 276 3 3h 86 21 135 IIA 35 2u5 fi 29 72 19 125 III 35 227 31 75 18 103 Iv 35 137 1 5 15 10 107 The problem of appeals in the Employment Service Executive examination was unique. Most candidates who appealed passed the examination and there were practically no protests from failing candidates. The reason for these appeals can be found in the background information of the examination. Most of SO mm s m NH m 0 9: oo B mm 4 «a om 4: m m: on HHH m3 m ma m: I: m 93 mm 3H 03 s 2 om ma m :mm mm H 3a 6 mm o: m emu mm 5 mnommm Had o m m «H m o 3 on E a m S cm I: m 5 66 HS 8 m 3 +3 1: m mm mm «H mm m I: a: ma m a: mm HH 0: o 8 on w 2: om «H mmoq .oz .02 .oz .02 .02 .Oz .oom.oz mouwufincmo wcfimmwm mZOHBHBHBmm200 zmmo m>HBDomNm WOH>mmm BzmSMOAmEm >H mum48 51 the applicants held only provisional or temporary appointments at their current level, their permanent status was one, two, or even three levels below their current rating. The appeals, then, were not an attempt to gain promotion, but a fight to retain the rank they held. To pass the examination and be subject to future appointment was Of no interest to them. Actually the fact that they did pass and in effect were shown able to perform successfully made their readjustment even more bitter. Finally, demotion in the agency often meant moving from one city to another. In any case, a combination of factors led a large number of passing candidates to protest their final ratings. Ap- parently they felt their best opportunity for success lay in the evaluation of education and experience and appeal after appeal was received directed to the education and experience rating. To illustrate the type and temper of the appeals en- countered, a number of them are presented below.2 Example 1: ”My experience with the Nichigan Unemployment Compensation Commission, first as a Labor Supply Tech- nician III, and later as a Personnel Methods Technician III should have been appropriate for specialized experi- ence on the IV level examination; that is, procedures evaluation experience of employment service functions.“ Example 2: "It is my opinion that I was given insuffi- cient points for my experience and training." 3.913.]. A? llrlr 1 . ,.......?_. 3. C 9.......« 52 Example 3: "I am basing this appeal on the over-all re- sults of the examination, which appears to eliminate about ninety per cent of the incumbents in this (Detroit) area, and especially in the matter of the formula used to evaluate experience and training." Example h: "I am protesting the evaluation of my experi- ence. My contention is that during the period that I was classified as an Employment Service Executive I and as an Unemployment Claims Examiner A, I was, in fact, working as an Employment Service Executive Ia." Example 5: "I have received the ratings I Obtained . . . and feel they are undoubtedly without prejudice and are reasonably accurate. However, since the evaluations on experience and training are apparently computed on some mathematical equation based on information furnished by the individual being rated, I feel that I have not been given full credit for experience as against others with far less experience in the classification.” Example 6: "I am dissatisfied with the evaluation of my educational background." Example 7: "The examination announcement stated that any candidate should Obtain a maximum of 35 per cent for eXperience and training. After the scoring, I am ad— vised that the maximum in my case, because Of the lack of college education was 29.5 per cent, and five years of my experience as an Executive I was entirely disreo garded as a result Of this method of scoring. "Further, it sets up a discriminatory condi- tion whereby one group Of individuals are limited in the possibility of Obtaining maximum scores and another group being favored in that respect. My contention is not to disregard or limit the credit for education, but to permit all experience to balance out any educa- tional shortage." ,..| , ,. .... 5.4., 5...;- Lu Pvnfu! I L 4-H» 53 Example 8: "The undersigned are a representative group, authorized to speak for all persons in the Detroit area who wish to protest the evaluation of the recent Employment Service Executive examination. Authoriza- tion was given at a meeting held on March 18, 1950. The following was resolved: ”Whereas, on March 9, l9h9, the State of Michigan Civil Service Commission announced a promo- tional examination for Employment Service Executive Ia, II, IIa, III, IV, V, and VI, and "Whereas, on page M thereof they advise that in order to receive a minimum rating of 70% on experi— ence and training, candidates must possess the desirable qualifications listed, and ”Whereas in evaluating eXperience and train- ing they allow a maximum of 35% of which a person who has 11 years of experience with the agency, and meets the requirements of admission to the examination and who has graduated from High School, only 29.5% Of this as credit to their final score, and "Whereas, in evaluating experience and train- ing they allow a maximum of 35% which a person who has 11 years of experience with the agency, and.meets the requirements of admission to the examination and has in addition a college degree 33.5% to 35.0%, and "Whereas, this inflicts a penalty on many of the employees Of the Michigan Unemployment Compen- sation Commission by reducing their final score by failure to allow credit for supervisory and administra- tive experience in private employment. "Now, therefore be it resolved that one full credit be given to Offset college education for all years of service within the agency, especially within a supervisory capacity." Each appeal had to be answered individually. In the main, the appeals fell into two groups. The first consisted of those who felt that their experience was not given a proper evaluation. In some cases they felt that the content of their 51L jobs had been misinterpreted and in other cases they felt their agency experience had been wrongly evaluated by an in- sistence on rating according to the job classification they held. They insisted that, in many cases, they were doing work outside Of the classification they held. The second group consisted of those candidates who, lacking a college education, questioned the validity of the education and ex- perience rating chart. The paragraphs below contain the essence Of the answer given to the applicants questioning the validity of the ex- perience and education chart. ”It was the feeling of the examination section that only the best qualifications in all respects should receive as much as 100». Others, who other- wise met the requirements, should receive scores somewhere between 70% and 100%. Obviously, on a ranking basis, only a very few could possibly hope to reach 100%. EXperience and training consists Of two factors: Education and experience. Therefore, in constructing a rating chart to rank employees Who have more than the desired requirements it is felt that both factors have to be taken into consideration. Thus, the chart provides that additional years of experience by itself would not rank a person at 100%. Nor would additional years Of education by itself rank a person at 100%. To receive a tOp rank each candidate would have to possess education and experience in addition to the required basic amounts.” 3 Appendix "A" (memo of h/2l/50) illustrates a comprehen- sive and effective answer to many of the appeals. Those persons who protested on the basis of their having worked out of classification received little attention. Any department head, line supervisor, or employee can request of 55 the Civil Service Commission a re-evaluation of a job. In all cases brought before the review board no prior notice had been given to the Civil Service Commission of any supposed error in classification. In actual practice, both the Civil Service Hearing Board made up of men from without the agency, and the Civil Service Commission itself, to which appeals from the Hearing Board are submitted, have all too often reversed the decisions Of the technical staff. In the present situation, however, solid backing was received and all appeals concerning basic policy were ultimately unsuccessful. l. Seventy per cent is an arbitrary figure used by most juris- dictions including Michigan to represent a passing grade. If some other figure such as fifty or eighty represented passing, then passing on the education and experience chart would represent one of those figures. While the above practice is widespread, it is not universal. For example, the Detroit Civil Service Commission uses mini- mum requirements on all examinations. However, they interpret minimum as equaling only 65% where 70% is passing. Thus, a person who met the basic required edu- cation and experience requirements in Detroit would receive only 65% of the assigned weight for that part of the examination and would have to compensate by attain- ing more than 70% Of the other parts of the examination to receive a final passing grade. If, however, the candidate did not meet the minimum requirements, he would be rejected from the examination. 2. All of the illustrations were taken from confidential transcripts of Michigan Civil Service Appeal Board pro- ceedings. ' 3. Taken from a sampling of cases prepared for the Michigan Civil Service Hearing Board. .453... - .. CHAPTER VI THE WRITTEN TEST Early Testing Procedure Present day civil service examinations reflect exami- nations given in the school system. In England, the first attempt at filling positions through tests was in the higher professional and technical classes. Since most of the pro- fessional personnel were recruited from students at select colleges, it was normal to test the applicants for what they learned in'gchool. Consequently, broad examinations were utilized in England, first for the higher level positions and then later in a someWhat modified form for the clerical positions. It can readily be seen that examinations developed to test for the professional and technical classes would not necessarily be useful in testing for clerical positions. Fortunately, tests of knowledge have been found to have a significant correlation with intelligence. Further, it has since been discovered that the correlation between intelligence and clerical aptitude is in the neighborhood of {.80.1 As a result of the chance relationship, early civil service written tests were fairly successful in predicting later success on the job, but the procedure was not of much value when extended to the mechanical and administrative positions. 5I‘5’lu'unllr .7. . . I... 55.. Iktll; _ f 57 Since the early deveIOpment of written tests much re- search has been conducted by the psychologists. They have demonstrated that individual differences have considerable influence on occupational attainment. It has been shown that there is a best combination of knowledges, abilities, and other traits for any one job, so the work of the civil ser- vice agency, as well as personnel offices of private in- dustry is to find the peculiar combination of traits required and to validly test for them.2 Bases for the Written Test Job analysis is performed by the classification staffs in the larger public personnel agencies.3 In the smaller jurisdictions, civil service technicians perform the func- tions of classifier and then later the function of examiner or test constructor. Because of the extremely close rela- tionship, some agencies have attempted to coordinate the knowledges Obtained by other combinations of job duties. In the Detroit Civil Service Commission, a deliberate policy has been followed of transferring technicians for extended periods of time back and forth between the classification and examination sections. In the Michigan State Civil Ser- vice Commission, examiners accompany classifiers on audits of positions they will later examine. The job analysis results in the preparation of a formal class specification which lists, among other things, the 58 knowledges and abilities required for minimum acceptable work performance. The class specification also includes a list Of typical duties and assignments as well as a list of the more common aptitudes required for satisfactory performs ance on the job. They are not meant to be an inclusive list, nor are they usually listed in order of importance. They are merely samples of what can generally be expected of the employee on the job. The examiner's problem is to study carefully the knowledges and abilities, analyze the educa- tion and experience section of the class specification, carefully review the list of typical duties, and from an analysis of the various parts of the class specification, develop an outline of the topics to be used in the examina- tion.“For example, a statement in the class specification such as ”above average intelligence" justifies the use Of an intelligence examination. The statements "ability to get along well with others” or "meets the public . . .,' make feasible the use of a personality testA\ A final example will amply illustrate the freedom given the examiner in con- structing examinations even though he is subject to justify- ing each individual section. A statement such as ”graduation from college with major course of study in business administra- tion" would be sufficient reason for the inclusion of any subject apt to be taken by the business administration major, such as accounting, economics, or mathematics. Each section used in an examination demands justification from the class 59 Specification, but it can be written to give the examiner a broad area in which to test. In constructing a written test, one starts with the develOpment of the outline. The outline used for the Employ- ment Service Executive Series is illustrated in Figure 12. Section No. #— Number Section Headings Items Weight I Interviewing 35f l 2 Vocational Guidance 65 l 3 Occupations 65 1 Total - Ia 165 % General Personnel Management 30 l . Job Analysis 30 l 6 Labor Relations 20 1 Total - II, IIa 2L15 7 Placement Services 35 2 8 Office Practices, Procedures, and Techniques 10 1 Total - III 9 Business Information 55 2 10 Management Practices 10 1 Total IV 355 Figure 12. Examination outline for the Employment Service Executive Ia-II-IIa-III-IV. Types Of Items After the outline is prepared listing the sections to be used, the next problem is the securing Of individual items. Most items are written by the examiner using as source material reliable and standard books in the field. As the items are constructed they are fully documented like the sample item card shown in Figure 13 (page 60). 6O In a geographical filing system, material to be filed is first classified according to the name of the state. True Doris and Miller Complete Secretary's Handbook N.B. p. 25 §-;6 Figure 13. Sample examination item source card. Because of the need of highly reliable test items that can be easily and quickly scored, written test items in civil service examinations have tended to be of the objective type. Essay items tend to be used only Where insufficient material exists to properly prepare items in objective form. Hard work and sufficient preparation can usually translate a good essay section into a group of the objective type of questions covering the same material. The following kind of items are most Often used in civil service examining: 1. Completion. 2. Multiple-Choice. 3. True-False. (A) Yes-No. (B) Correct-Incorrect Completion Items Completion items are most closely related to the essay form. This kind of item offers an incomplete statement and the candidate must complete it either with a word or two, or 61 a short phrase. Examples are listed below: Example: The chemical formula for water is: Ex. Example: A circular chart having segments representing various percentages Of the whole is known as a: Ex. The answer to the first example is "H20" and the answer to the second is "pie chart." The completion item is also called a ”recall item.” These questions must be answered through direct recall of the specific knowledge required. They present to a great degree some of the problems found in using the essay type item. In most cases, no matter how carefully the examiner may prepare the item, more than one correct answer may be found. The most common variant will be a shift in the tense or degree of the answer. The answer to a question may be "race," but the candidate may say "racing," or "raced." Depending on the group being examined, all Of the answers listed above could be considered correct. In actual practice, the most desirable method of scoring completion items is to keep a tabulation of all answers submitted, with the examiner indi- cating which choices are to be scored correct. This tabula- tion can be built up from examination to examination until a point is reached where only occasionally will the examiner have to decide whether new alternatives are acceptable. In large agencies that utilize test-scoring clerks, the procedure can be developed so that the test checker need bring only new answers to the examiner. However, even with an efficient :(fi, In... (71" Nil ['04. 3..., - , - . 3.1.5..“ .L. 62 set-up a great deal of time must be spent by the exaniner in scoring completion items. Also, with more than one possible answer, the problem always exists Of accuracy of the scoring key. Multiple-Choice Items To get around just such problems as those listed pre- viously, but to retain the best elements Of the completion item, the Multiple-Choice item has become more common. Example: The capital city of the State of Michigan is: (A) Detroit. (B) Grand Rapids. (C) Lansing. (D) Flint. (E) Kalamazoo. The answer to the above item is Lansing. In this type of item, the candidate must show recognition of the correct answer rather than recall it. There are major advantages in employing the multiple-choice questions. The foremost advantage is the ease of scoring. Multiple-dhoice items can have but one correct answer and the candidate must choose from one of the choices listed. The examiner can reduce the entire scoring Operation to a clerical procedure. No tech- nical knowledge of the subject is required. Given a scoring key the test scorer can rapidly and accurately score exami- nation after examination. 63 The multiple-Choice type of item as well as the true- false, to be discussed later, lend themselves to the use of answer sheets and mechanical scoring. The most widely known mechanically scoring device is the IBM scoring machine, which makes use of the recording of electrical impulses. These impulses are produced in the following manner: the candidate uses a soft lead pencil to mark his answers in predetermined locations on the answer sheet. Usually he must blacken in one Of a series of pairs of lines such as those illustrated below: Figure 1h. IBM answer sheet marking procedure. The answer to the sample multiple-Choice item shown previously would have been choice (C) Lansing. The candidate would therefore blacken the (C) choice. The answer sheets are placed in the IBM machine Where electrical contact is made. Disadvantages in using IBM scoring machines, however, have kept many jurisdictions from adopting them. The major consi- deration that discourages all but the largest jurisdictions from utilizing these machines is the cost involved. Another consideration is inaccuracy caused by the lack of electrical contact in the machine when candidates do not press hard enough with their pencils while recording their answers. Inaccuracy is also caused by the machine picking up sweat 611 marks and dust particles. However, the largest cause of error is the sensitivity of the machine to atmospheric condi- tions. Under humid temperatures the machine will simply not score accurately. Finally, the problem and expense of re- wiring the machine for each different answer sheet precludes its use on all but the largest examinations. The Michigan Civil Service Commission makes use of semi- mechanical scoring, but without the use of the IBM scoring machine. Michigan prints the answers on the answer sheet .33323 the candidate has marked his answers. The test scorer then compares answer and key to score the examination. Note the illustration shown below: Figure 15. Michigan Civil Service answer sheet scoring method. Under this system an answer stencil is typed and the answer sheet is re-run with.the key being super-imposed on each candidate's paper. Most of the disadvantages of the IBM 65 scoring machine are overcome. The cost is low and the answers are always legible. Besides it is adaptable to much smaller examinations volume-wise than is the IBM system. Also, it has some distinct advantages Of its own. The candi- date can see the answer key super-imposed on his own answer sheet when reviewing his examination. That is not possible when the IBM scoring machine is used. The major advantage is the ability to tailor make the answer sheet. In the illus- tration shown above, multiple-choice items are interspersed with true-false items. Using the IBM scoring machine confines the examiner to a number of pre-set patterns unless expensive individual effort is expended. multiple—choice items can be constructed with any number of possible choices, though.usually from three to seven choices are found. Probably four or five choices are the most common.5 The greatest difficulty in constructing multiple-Choice items is the employment Of suitable distrac- tors. If a four choice item is constructed and only two of the choices are obvious, then the candidate is actually con- fronted with the equivalent of a true-false situation. He can take one Of the two remaining choices and by aiswering it true or false decide on the answer to the multiple-choice item. The difficulty of constructing multiple-choice items with satisfactory distractors causes sufficient concern to seriously question their use in any, but very carefully con- structed and validated sections. The above statement is made 66 in the face Of a strong and marked trend toward the use of multiple-choice items almost to the exclusion of other types. It is felt that a weak multiple-choice item is less desirable than a weak true-false item. A true-false question employs penalties to account for guessing, but a multiple-choice item usually presupposes enough choices to offset the guessing factor and consequently no consideration is usually given to it. In tabulating scores of material used in Michigan of apparently the same difficulty and covering the same subjects, results have been consistently higher on multiple-choice items than on true-false. Multiple-choice presents problems in construction because a much greater knowledge is required of the field under consideration. A multiple-choice item is made up of a premise and.the alternative answers. The information the candidate needs in order to answer the question should be set in the premise. When the applicant has read the premise, he should know the general type of answer for which he is to search among the various choices. Many examiners prefer an indicative form of the premise to an interrogative form. In the treatment of incomplete answers, however, it is frequently difficult to set up a pre- mise except in the interrogative form. Attempts to construct all multiple-choice items in the indicative form are apt to result in cases Of stilted expressions or expressions that will not constitute a grammatically accurate sentence when coupled with the expected correct answer. 67 Some writers like to compose items that require a compe- titor to find something that is 223 a reason or tO find a practice or principle that is 333 a good practice or principle. Although it is not being suggested that all items should be positive statements, it is believed that this shOuld apply to the great majority. This is because a negative item among a group Of positive items tends to act as an irritant to the candidate. In developing the correct choice, the examiner must make it long enough to formulateidequately the expectedresponses, but it should not be unnecessarily so. The examiner must bear in mind that if the correct choice is long then the in- correct choices should also be long, and that as the correct answer becomes longer and longer he can less and less readily build up each incorrect choice to the same length. Many item.writers tend to protect the correct answer by adding so many qualifying phrases that they cannot hope to develop their incorrect choices to the same degree Of complexity and length. Candidates have occasionally been able to Obtain good scores merely by marking as correct the longest of the possible choices. The item as a whole should be so constructed that no one is able to ignore the premise and to select the expected answer by the way it is worded. The writer should also attempt to avoid the "nice" type of item in which the differentiation between the expected answer and the incorrect choices can be made entirely on the basis of the expected man, ,J I .5 e 59". II . If . m c 68 answer containing many "nice” words, while the incorrect answers contain only "bad" words. An incorrect choice Ob- viously so absurd in relation to the premise that no candi- date will consider it is an absolute waste from an examina- tion point of view. An efficient incorrect choice is one that has pulling power for the poorer qualified candidates. In developing items as a whole, the examiner should attempt to construct correct and incorrect alternatives so that the better qualified candidates choose the expected answers as a result of superior insight. The incorrect choices should be so stated that in selecting one of them the less qualified candidates can demonstrate that, because of this lack of in- sight, they think and react differently from.those persons better qualified. The attraction of the incorrect answers for the less qualified candidates should be that these in- correct alternatives are statements of the same kinds of incorrect concepts, illogical conclusions, and generally erroneous ideas that they tend to hold. Perhaps the greatest difficulty in constructing incorrect answers is to have choices that appear plausible to the less qualified candi- date, but not so plausible that they are chosen by the better applicants. One final work of advice should be given to the appren- tice item writer. Experiments have shown that even well constructed items must be placed in a certain order to be 'most effective. It has been found that among candidates who null. ..... 4“. ,.. J .u...'Il.'l '1 ‘3. a a h 1. 69 have no knowledge Of the answer in a five choice item the first choice is the one most often guessed. This is followed by the fifth choice and then by the third, fourth and 8 finally the second choice. True-False Items The true-false item is the final type that will be dis— cussed in detail in this paper. Other items such as matching questions are sometimes used, but, in the main, the items described in this paper contribute the major portion to written civil service tests. As in the completion item, the use of appropriate wording can become the difference between a good true-false question and a poor one. Truss-false items, more than other types must be pointed closely to particular examinations. For many positions only a small amount of education is required; for others, college or more is needed. An item.that is appropriate for one group is not necessarily satisfactory to the other. For example, the following item is apt to be answered differently by various groups: "Water freezes at 32°." To the average person this would be a true statement. To students of high school chemistry or physics a query would be raised as to whether Fahrenheit or Centigrade scale is referred to. Adding the symbol ”F", the average person familiar with high school chemistry or physics would consider the item true; however, to the physicist, the expert in this field, this statement would be unqualifiedly false even if 70 the symbol "F" were added. In the first place water does not freeze at 32°, but begins to freeze at that temperature and then only when certain conditions are present as purity of water, atmospheric pressure at sea level, etc. This illus- trates the point that not only must language be used suitable to the occupational group to be tested, but that suitable qualifying statements must be included in the items depending on the knowledge of the group.9 The true-false item may be used in testing the appli- cant's knowledge of a concept, fact, principle, reason, inference, meaning application, etc. A series of principles can be expounded concerning the construction of true-false items. These are listed below: I. There should be no ambiguity. 2. Each word should be commonly understood. 3. The statements should not be excessively long. A. Such words as "no,” "not," ”always," and "never" should be avoided insofar as is possible. 5. Questions with Obvious answers should not be used. 6. DO not include items Which may be partly true and partly false. 7. Questions which express Opinionated views, without giving the source, should also be avoided. The true-false item is used in several modified forms. Note the two examples. Diseases that may be transmitted by water are: 71 l. Typhoid Fever T F 2. Diphtheria T F 3. Diarrhea T F A. Cholera T F 5. Tuberculosis T F This type of item is particularly adaptable to testing the applicant's knowledge of several facts all of which have something in common. A certain office has a mailing list which now con- tains about 1000 names. An average of 60 new names are added each week. An average Of 20 Of the Old names are discarded each week. The names to be added and removed from the mailing list come to the senior clerk who has charge of the list partly on typed sheets of paper containing from 3 to 30 names and partly on carbon copies of letters. 1. Cards are superior to loose leaf books for keeping such a list. T F 2. TO prevent the loss of some names, the list should be kept in a bound book with all names beginning with the same letter on one or two pages. T F This type of item is adaptable to testing the applicant's judgment in a given situation, or his interpretation of case histories, experiments, etc. This form of test should include 5 to 15 questions on each situation, case history, or experiment. 72 Sources of Items Item construction is one of the most difficult and time consuming factors in civil service examining. Fortunately, sources are available on which the test constructor can rely to supplement his own efforts in this field. Supplementary sources become an indispensible factor in the smaller agency, where it is physically impossible to construct all the necessary items. The Midhigan Civil Service Commission makes use of the Federal Security Administration test service for items in certain fields. Wherever the federal government grants monetary aid for personnel services to state governments, it demands flaat those governments employ that help on the basis of merit examinations. Since many states do not have exist- ing merit agencies, the federal government has aided the organization Of Merit System Councils. These councils are then provided with examination material by the test service of the Federal Security Administration. States with existing civil service commissions, however, are also given the benefit of the work produced by this agency. The Michigan Civil Service Commission makes use of examination material prepared by the Federal test service in areas such as Social Work and Employment Security. Perhaps the single most useful source Of examination material is the Civil Service Assembly of the United States and Canada. This professional organization of public .1 a4!!!" Ill. V I \ a i. 73 personnel administration maintains a test exchange service for the use of its members. All members send copies Of their examinations to the Assembly and in return borrow examination material sent by other agencies. Civil service agencies, including Michigan, make use of city and state agencies. Experts in various fields are available and their aid is utilized in constructing examina- tion items. Colleges are another source of which civil service agencies make a great deal of use. Classroom exami- nations are often obtained and with very little adaptation can be fitted into examinations. In many other cases, very generous cOOperation is obtained from specialists in certain areas who will give freely of their time and construct items for civil service agencies. These items can usually be used with very little editing. Finally, many, if not all, civil service agencies maintain item card files in.which they file items concerning most of the fields in which they must pre- pare examinations. At the present time the Michigan card file consists of more than 100,000 items. These items range over most any topic conceivable from "Ability to Get Along with Others" to "Zoology.“ Examination items are reviewed after construction by subject matter experts. Since, in spite of the sources available, a majority of examination items are written by the specialist in test construction rather than the subject matter specialist, it usually is advisable to have the questions art! .3! ..r.lu.'..l. . .4- 7.!Ibu aunt. I I ”.51 .a. I. 7h reviewed for adequacy, and correctness. A further review is usually made in Michigan by the supervisors of the departmental personnel involved. This review is to check not so much for accuracy as for pertinancy for the job. Time Allowances After the examination items are compiled and reviewed, they are assembled into subject matter sections although some agencies do not attempt to do this. The sections to be em- ployed had been decided earlier, of course, when reviewing the class specification knowledges and abilities. sections are then assigned time allowances. _As in many other areas of civil service administration, the setting of the time allow- ance may depend on the general policy of the agency. In Michigan time allowances are generally nine minutes for 25 true-false items and a minute for each multiple-choice item. These times are considered sufficient for all but the very slowest. This policy is deliberate and is aimed at keeping at a minimum criticism of the agency. Detroit Civil Service, on the other hand, because of its longer period of existence and consequent greater acceptability, follows a policy of timing sections so that only those who know the area will finish within the allotted time. With true-false items, for example, their time allowance is 25 in six minutes or one- third faster than in Michigan. The time allowance for multiple-choice is correspondingly faster in Detroit. 75 Weighting the Parts of the Test . After sections have been organized and time allowances set, the examiner must then "weight" the various sections. Weighting involves an determining Of the relative weight or importance Of each section of the examination in relation to all other sections. Thus, if an examination was prepared for Senior Personnel Technician, the following sections might be used. 1. Intelligence Test 2. General Background 3. Personnel Techniques The problem for the examiner would be to determine (guess) of what importance each of the sections is to the total test. (See previously displayed copy Of Employment Service Executive examination which includes weighting of the various sections.) Weighting is one Of the vital devices employed by the test technician. By prOper weighting, sections can be assured of their proper importance. This follows both for the sections that are well constructed and pertinent and for the sections that are weakly constructed and test only fringe knowledge. A section of the latter kind can be assigned a low weight by the trained examiner and can be put into examinations because it is expected and lends face validity. Yet the examiner can, at the same time, by an adjustment of the weight, negate the value of the section in terms of really contributing to a passing score. 76 Weighting is by far one of the most useful and important appliances utilized in test construction. It is in this as- pect of the examination that it is most important for the test constructor to know intimately the duties and functions of the job for which.he is preparing an examination. Statistical Procedure After an examination is carefully prepared, assembled, and administered much paper work still remains; first, the examination must be scored. Earlier in the paner scoring methods were described; however, the actual scoring for right and wrong answers is only part of the task, as a great deal of statistical tabulation and analysis still remains. The specific statistical techniques vary slightly from agency to agency. Some omit various parts; others, with the time and money required, spend time experimenting with and analyzing results. In Michigan, the following statistical methods are em- ployed as routine on all examinations. Each individual sec- tion is tabulated according to the number of candidates that receive varying numbers of right answers. High, low, medium and quartile scores as well as an effective range is then computed so that from the statistics obtained, comparison can be made from section to section. A large number of agencies, at this point in the process, determine the standard deviation of the various sections for 77 comparison purposes. Michigan, however, has adopted a proce- dure develOped by the Detroit Civil Service Commission which is shorter and easier to compute and produces substantially the same result. The basic problem, whether using the stan- dard deviation method or the one to be described in this paper, is to make sections count that part Of the examination that they were intended to do. The most simple illustration of the problem under consideration would be a free answer exami- nation consisting Of two questions, one of which the test constructor feels is worth twice as much as the other. The problem is how to treat scores so that the ratings on one question will have twice the final effect (have double the weight) on the total rating as the other.10 The examiner may approach this problem from several different paths. He may rate both questions on a basis of 100; then multiply one rating by two and combine the scores or he may allow only 66 2/3% for one question and 33 1/3% for the other. However, it is an accepted fact in statistics that the distribution of scores affect the weights assigned. Notice the hypothetical scores shown in Table V. TABLE V RESULTS OF POOR SECTION WEIGHTING TECHNIQUE Paper Question 1 Times 2 gggestion 2 Total 1 100 200 60 260 2 98 196 70 265 i 96 192 80 272 9A 188 90 278 __5 92 18A 100 28k 78 Thus, the total score is largely determined by the scores on the second question. This is because the second question shows a range of scores 5 times the range of the unweighted scores on the first question and 2% times the range of the weighted scores.11 ”Statistical research has shown that the standard devia- tion is more truly representative of the form and shape of a distribution than is any other single statistical measure."12 However, Detroit has developed and Michigan is using, with a slight modification, a substitute method of producing the same results. An illustration will show the relationship that exists between the results Obtained employing the method described in Appendix‘ C. and the standard deviation method. (See Table VI, page 79). The similarity between the multi- plying factors derived using an average of the Effective Range Method and the Inter-quartile Range Method and the Standard Deviation Iethod can be seen by studying Table VI. Detroit has found that, "if the total weighted scores of a test battery for a group should be computed by these two methods and the results correlated, correlations of from .96 to .99 would be secured.“13 Whatever the method employed, whether standard deviation or some other accepted method, the multiplying factor that is derived is multiplyed by the raw score of the sectiOn to obtain the weighted score. The weighted section scores are 79 TABLE VI VARIOUS METHODS USED TO DERIVE MULTIPLYING FACTORS Multiply- Multiply- Multiplying Multiply» ing fac- ing factor factor de- ing factor tor de- derived rived using derived rived us- using the an average using the ing the Inter- of the effec- Standard Effective Quartile tive Range & Deviation Test Relative Range Range Inter-Quartile Method Section. Weight Method Method Ragge Methods 1 ' 3 1. 068 059 061.]. .68 2 l 1100 1.00 1.00 ' 1.00 l .25 .18 .22 .20 fl 2 1.70 1.60 1.65 1.66 5 l .13 .ll .12 .ll 6 l .73 .73 .73 .83 7 12 1.10 1.15 1.13 1.08 added to derive the total weighted score for the examination, these scores are tabulated, and the examiner is then ready to set a passing point for the examination Setting the Passing Point A passing point for a more or less unstandardized civil service test is a difficult problem to solve. There are several methods employed. One of the first decisions to be made is whether or not it is desirable to pass a large number of candidates. Most agencies determine the answer to this on a policy basis. The agency may commit itself to a policy of passing just enough to fill the existing vacancies or it can adOpt a policy of passing most candidates, making 80 the assumption that most Of them will never receive appoint- ments, but having passed the examination, they will harbor less resentment against the agency. It is an unanswered question whether more ill will is produced by passing a can- didate who will never receive an appointment or by failing him. In either case, the civil service agency can expect some degree of bad feeling as a result of the examination process. Passing points are sometimes set on the basis of breaks in the final tabulations. That is, a tabulation is made of all the final total weighted scores. A typical tabulation of 175 candidates might look like the one produced in Figure 16. Class Interval Number of Cases 96 - 100 91 - 9S 1 86 - 90 111 81 - 85 sees 11 76 - 80 1111 111 71 - 75 e111 sees 66 - 7o 31111'1111 same 61 - 65 S6 - 60 £1 - 55 sees 11am sees 1111 sees ease-sees 6 - 5o -1111'1111'1111'1111‘1111‘1111 bl - A5 vases-seas ease 1111 36.. 110 magma-13:11 31 - 3S sass~ee1a mess 26 _ 30 see} 1 21 — 25 1111 16 _ 2O 11 11 - 15 1 6 - 10 1 - 5 1 o - Figure 16. Tabulation of hypothetical total weighted scores. 81 A quick glance shows a significant break in the scores at the 66 to 70 group. It can be deduced that the Ah candidates with scores of 66 or better show a distinct and definite edge in knowledge over the balance of the group. From the picture given by the tabulation one can guess that the average candi‘ dates will cluster together at scores below 55, while superior candidates will achieve scores Of at least 66. In setting a passing point based upon a break in the tabulation, a score of 66 would be set. Setting a passing point upon this basis ignores several factors that may or may not be important. One Of the factors not considered was the number of jobs to be filled. Given a situation Where only two positions were vacant, passing all uh of the candidates in the upper group would indicate a policy of passing more candidates than can possibly be appointed. On the other hand if 60 or 70 posi- tions were to be filled, the question of provisional appoint- ments would have to be met. TO fill all vacancies the h passing point should be set at A6 or 51,1 but would this pass unqualified persons? Since most agencies are working with more or less unstandardized tests, it is very difficult for them to answer this question. A great deal depends on whether the group in question is above average, average, or below average. The same problems exist if passing points are fixed on the basis of a set percentage of the candidates. Many jurisdictions decide beforehand, as a matter Of general policy to pass perhaps 25% or maybe 50% of those competing. 82 Some agencies have attacked this problem by standardiz- ing sections, even though the examinations as a whole are not standardized. Thus, Michigan maintains a section card file. Information listed includes: the examination in which it has been previously used, the level of the groups, the number of candidates in the groups, the highest scores achieved by each group, the third quartile, median, first quartile, and lowest scores, as well as the effective ranges for the groups. From the information listed on the section card, the examiner can estimate what scores the new groups should probably achieve. For example, a section entitled English Usage might show the following results. TABLE VII ENGLISH USAGE SECTION STATISTICS Third First Effec- NO. High Quartile Median Quartile Low tive Exam Use Cases Score Score Score Score Score Range Personnel Technician 225 2A 19 15 1o 2 23 Sr 0 Admini' strative Ash» 115 2A 22 17 13 2 2 Social Worker 300 25 18 1h 8 2 2 Sr. Social Worker 85 25 23 18 1A A 22 Sr. Librarian 100 25 23 17 1h. 3 23 From inspecting Table VII the examiner can ascertain what to expect from a new group. Apparently those persons who reach the senior level score three or four points better than candidates for induction levels. The examiner can then decide {w ”,0. j I! '1'?! Win. 71*." . . .. 1': I . . . f d 83 that if the class under consideration is a senior level that the successful candidate should get twenty-three or seventeen depending on what percentage of the group it is desirable to pass. This figure of twenty-three, for example, can be multiplied by the multiplying factor for that section in the examination under consideration, and a passing score for that section established. This procedure may be followed for every section with the passing point becoming the total of the passing points of each of the sections. This procedure guarantees a maintenance of minimum standards even when the examination itself is not standardized. The procedure could tell us that in the tabulation listed previously it might have been possible to lower the passing point to MS or 50 to allow for a large number of job vacancies. This could have been done if the total of the section passing points fell at MS or less since it would tell the examiner that the group under consideration was above average. On the other hand, if the total Of the section passing points was 65 or 75, it would indicate that under no conditions should the passing point be dropped to AS or 50-—perhaps it should even be raised to 65 or 75. After a passing point has been set it is necessary to convert the weighted scores into percentage scores. The number of weighted points fixed as the passing point become 70% of the percentage points allowed for the written test.15 The most common procedure is then to subjectively guess on 8h the basis of past performance or the performance of similar groups the highest and lowest percentages to be granted. It is then a simple matter to fit the intermediate scores into intermediate percentages. It can and probably is usually done on a straight line basis. If more refinement in technique is desired, the percentage scores may be so scaled that under any part Of the total scores would exist the same percentage of cases as would be found under a normal distri- bution curve. A refinement to this degree, however, is hardly worth the effort. Test Analysis NO report on the preparation of civil service examina- tions would be complete without some reference to an analysis of test results. While thorough and extensive analysis of test material is an impossible task for the small agency and only a possibility for the larger agency, each office should attempt to do everything possible within its budgetary position to improve examination techniques and material. [1% / [Item analysis studies and validity studies are the two best known statistical devices employed in improving an examination progran. But even here, a word of caution must be used. Validity studies, in spite of the demand for them are almost non-existent in civil service testing.§ To explain this one must understand the basic difference between more or less unstandardized civil service examinations and the well 85 standardized test developed by the psychologist to test for a specific interest, ability, or other function. The psycho- logist can take a set number of items and try them out again and again. He can shuffle, change, revise. He can set up very exact control situations. And finally, he can obtain thousands and even millions of cases to document his studies. His end product after an enormous amount Of effort is usually a near perfect instrument that can be utilized for a specific purpose. Almost none of the conditions described above exist in civil service testing. Groups cannot be controlled. Exten- sive numbers of candidates are not always available. Criteria is most difficult to obtain. And finally, the basic require- ment for a validity study cannot be fulfilled in civil if service testing. That is the employment of a control group, some of whom.do well on a test and some of whom do not do well. Civil Service must employ only those candidates who do well on a test constructed for the job in question. XkConsequently, in spite of repeated criticism of civil service agencies for their apparent apathy in failing to validate their work, not much will be found along this line among them, either now or in the near future.f However, there does remain an area in which they work toward improving their test material. Improvement lies along the path Of item effectiveness. Without too much additional labor an agency can find out Whether items are too easy, too hard, or whether 86 they discriminate between the groups that ultimately doéwell on the total test and those that do not score as well.1 They can also isolate specific subject areas by means of item homogeneity studies. Michigan does carry on studies in these areas. For the Employment Service Executive examina- tion under consideration in this paper, a study of the effec- tiveness of the items was prepared. This study together with its conclusions is shown below after an explanation of the mechanics. Since many different methods are available the method and mechanics used in this particular situation will be ex- plained in detail. The tetrachoric correlation coefficient was employed. The tetrachoric correlation can be used to advantage if one is measuring only the presence or absence of certain traits but not their scaled measurements. For purposes of this analysis the traits to be correlated were whether a candidate passed or failed the written test and whether he answered a particular item correctly or incorrectly. For purposes of this study a passing score on the examination was considered to be a weighted score of 110.5. The actual passing point was 110. However, scores of 110.5 or above included exactly 25% of the candidates and so were used. Once the passing score was determined, the material was ready for tabulation. This type of correlation can be obtained only for groups which possess or do not possess certain traits. In this study those candidates who had scores of 110.5 were 87 considered to possess a trait and those persons who success- fully attempted to answer an itemvmderconsidered to have another trait. Therefore, the necessary pass, fail situation was set up. One of the biggest advantages of the tetrachoric correla- tion coefficient is the simplicity of the method with which it is Obtained. It is only necessary to run a tally accord- ing to the following illustrations: x I z y [ k pass first but fail second trait Where: I fail both traits Y z I pass both traits k fail first but pass second trait After the tallies are obtained it is necessary to find what proportion of the total number of cases fall in each square. The following diagram is then constructed: ' (c)' _ (bl _' '_ _. 181' X, y, z, and k now represent proportions of the total which, of course, must add up to a hundred percent. A and b are 4 III . . H» r. .i. .13. J. 1‘1 ct.— .M. . I. if! . Inn... .lv .d ll u m .H .H .H Hm.m Hm.m .H mm mm H manoHaaoecH H .m.s - .m.2 Adv .oa; madam A.m.mq .oaa. omaam .a; caez .oz .w>< .a.2 omaom oHHaeaso .a.z omaam .mnm oHHpaosu unopaH .aam unoHMH» . .>H .HHH anH mHH .HH .mH 0>H9500xm N ms open Goapmcaswxm ooabnom pcthOHmEm sands Goapmcufidxm om Mmmw.oz GOHHGCHEdEm meomemz mozHaommam mme mo mu¢ zd oszoqmzm mmoeoHmma CoaumaEEoo ooabuom HH>HU nwmfinoaz SELECTED BIBLIOGRAPHY 121 Books Adkins, Dorothy C., et a1. Construction and Analysis of Achievement Testg. Washington, D.C., United States Government Printing Office, 19h7. Baruch, Ismar. Position Classification in the Public Ser- vice. Chicago, Civil Service Assembly, 19h1. Bingham, Walter Van Dyke, Aptitudes and Aptitude Testing. New York, Harper and Brothers, l9h2. Buros, Oscar K. (Editor). The Third Mental Measurements Yearbook. New Brunswick, Rutgers University Press, 19h9. Burtt, Harold E. Principles of Employment Psychology. Revised Edition, New York, Harper and Brothers, 19h2. Cronbach, Lee J. Essentials of ngchological Testing. New York, Harper and Brothers, 19H9. Freeman, Frank S. Theory and Practice of Psychological Testing. New York, Henry Holt and Company, 1950. Friedland, Louis L. Personnel Administration in Michigan Government. Lansing, Michigan, Joint Legislative Committee on Reorganization of State Government, 1951. Gouldner, Alvin W. (Editor). Studies in.Leadershyp. New York, Harper and Brothers, 1950. Guilford, J. P. Fundamental Statistics in Psycholggy and Education. New York, McGraw-Hill Book Company, Incorporated, 19h2. Hoover, Herbert, et a1. Personnel Management. Washington, D.C., United States Government Printing Office, 19h9. Hoslett, Schuyler D. (Editor). Human Factors in Management. Parkville, Park College Press, l9h6. Lindquist, E. F. (Editor). Educational Measurement. Wash- ington, D.C., American Council on Education, 1951. Lytle, C. W., Job Evaluation Methods. New Yerk, Ronald Mosher, William E., Kingsley, Donald J., and Stahl, O. G. Public Personnel Administration. 3d edition, New York, Harper and Brothers, 1950. 122 Mursell, James L. Psychological Testing. New York, Long- mans, Green and Company, l9h7. Scott, Roberta, The Group Oral Test in Selecting Public Employges. Chicago, Civil Service Assembly, 1950. Shartle, Carroll L. Occupational Information. New York, Prentice-Hall, Incorporated, l9u6. Stout, Hiram M. Public Service in Great Britain. Chapel Hill, University of North Carolina Press, 1938. Sublette, Donald J. Notes on Statistical Procedures in Con- nection with Personnel Testing. Detroit, Detroit Civil Service Commission, 1936. . Pencil and Paper Testing as a Selectipn Technique in Public Personnel Administration. Detroit, Detroit Civil Service Commission, 19h5. - Sues, Otto L. A Guide for Preparing Personnel Tests. Chi- cago, Civil Service Assembly, 1951. Super, Donald E. Appraising Vocational Fitness. New York, Harper and Brothers, 19E9. Thorndike, Robert L. Personnel Selection. New York, John Wiley and Sons, Incorporated, 19E9. Thurstone, Lewis L. Computing_Diagrams for the Tetrachoric Correlation CoeffIcient. Chicago, University of Chicago Press, 1933. Tiffin, Joseph. Industrial ngchology. New York, Prentice- Hall, Incorporated, 19h7. Yoder, Dale. Personnel Managgment and Industrial Relations. New York, Prentice-Hall, 19h2. Periodicals Edwards, Austin S. The Myth of Chronolo ical Age. Journal of Applied Psychology, 1950, Vol. 3 , pp. 316-318. Flanagan, John C. General Considerations in the Selection of Test Items. Journal of Educational Psychology, 1939, V01. 30’ pp. 6714.3‘6800 Galston, Samuel H., and Hoberman, Solomon. Testing for a High Level Position. Public Personnel Review, 1952, Vol. 13, pp. 2u-27. Jig |1ilI_ - , , .1! 123 Gulliksen, Harold. Intrinsic Validity. American Psycholo— gist, 1950, Vol. 5, pp. 511-517. Jurgensen, C. E. Overall Job Success as a Basis for Emplo ee Ratings. Journal of Applied Psychology, 1950, Vol. 3 , Pp- 333-337- McGilvery, Francis E. The Problem of Position Evaluation. Public Personnel Review, 1952, Vol. 13, pp. 12-17. Rechetnick, Joseph, and Levine, Harold. Selecting Admini- strators. Public Personnel Review, 1952, Vol. 13, pp- 172-175. Viteles, Morris S. A Dynamic Criterion. Occupatiop;,1936, V010 lLl-D pp- 953-9570 2111, Anthony. A Further Inquiry into the Group Oral. Public Personnel Review, 1953, Vol. 1h, pp. 55-63. Public Documents Detroit Civil Service Commission. The Classification Plan of the City of Detroit. l9h2- Michigan State Civil Service Commission. Examinations, Certificationsl and Appointments in the MIchigan State Civil Service l9H6-l951. 1951. Michigan State Civil Service Commission. Analysis of the Supplementary Comments of Loren B. Millernyirector of the Michigan Joint Legislative Committee on Reorggni- gation of State Government Concerning-“Personnel AdminI- stration in Michigan Government."_1951. Michigan State Civil Service Commission. Minimum Entrance Standards for Admission to Civil Service Examinations. 1939. United States Congress. 8lst. Classification Act of 19h9. Public Law M29, Washington D.C., United States Govern- ment Printing Office, l9h9. 121; Private Communications Feinberg, George, Sacramento, California. Information on classification techniques in the State of California Civil Service Board. '[Private communicationi7 l9h8. Meyer, Charles A., Detroit, Michigan. Letter to Civil Service Assembly concerning longevity pay and its re- lationship to classification practices in public agencies. ‘ZPrivate communicationi7 19h8. BUM USE ONLY _ _ . ’2’? 8881:: USE my 03:7 ”9%? M71 W5 1ICHIGRN STATE UNIV. LIBRQRIES llllllllllllllllllllllllllllllllllllllllllllllllllll 31293008522124