{A ‘1 ”4:1 ‘ :rui‘m; ‘mw 3. "1 txa‘..;1!§ih'n:m':?.‘3.e«Jaw M ‘ 5 ' "4 :n1‘-.‘~-‘vw“"L‘~"‘ ~"L w '-’ ( u .vq .A .- Y. ‘ 14.4. fl‘ 1 *f '1“! L ! I W .~,\ «.9. u , | . - :A l’ ,1 ta .L ‘ 1 ~ Maw: .--‘E~_*: L ~ .3“ “ '“I,315;},‘7qlffifigfifigz1‘5t‘g1%,“_, 99:9 ~ .k V‘ S :u 1‘: x, A “1161‘s!“ “1:13: . v, <_ - «I ,. *- ul‘éjar. ! t ,2" I.ifixle;&;§$;;zfig§a£§fi3%? ‘51:!“ N. “v 12'," ‘ 11‘ 1‘11”. v I. A . ,\ 9 « MM ‘M x. xi ~ ‘ , ‘. .23 WM «9 ,‘ a. WM?! 1; . .NL x '- u ‘ .4 ,3‘,‘ :1 . .. gay i? {r55 m: W"‘3’“"1'4W'fi‘5‘1' 31%;??? db! ' a. ‘ ‘ , ‘14!‘{(\J(r}'!1.ihlqflfnfi ML} ‘Wlw «macaw. . -~ *u-duy‘tdmsu. 'v; (mm-a 3 Mi}. . N . ‘ ‘1,|)'u‘:l:q': |., thiimhlw “4d”. . A k!“ ‘:§‘1“i ‘1! 1‘ k ' 1‘Lfiii't'ilfil‘ ‘v . ,1" )g‘hg‘h‘gq ‘Nyfll' y ‘ ‘9‘ ‘ 4' :*:x$w«“5mz='*:‘fi‘~zs‘ " H 4‘ QH :: - «disadw, ) |(. {w‘QQ‘ES‘:E:“111‘:‘C§'\‘ " “94’ .‘fi h a 'u ”WW%M3““WWFH%%@MQ 'f'fl'j'WI" I ‘x " $4. «@fi‘ C v‘ ‘ 9 ‘ ' " ‘ ‘ g , 3 ' 4 _J 9%” l.- l L ’ _ . “Fulfil L x I him: ,1” ‘t , n } ‘ "(11%: t w.‘ j .‘ 0.; ’3 ‘ “W“ 1:“ "‘ "i h «n M > ‘ 1}“ P51? “We 'w-M ., ~- Mp4 ‘ K a ‘ ‘ WM balsa-".1; wgm 4 a? 23— : in; 4 ,. O , ' r". '9' . L v 1. 4 .‘z ,,y. fiat y ‘p‘. .. ,IM‘ ‘1” ‘ ,. ”1‘. I . “N .V: y. 3-9.; 9‘ ‘c ’r ' i'1i:.1&““‘ {my}: 1R “531$, }\ fin“ '. 30': ckéffii‘é‘x‘aziégé‘hfi Wmiuh ‘ I Y ’3 '& 1“ V‘M‘afi“ ‘ .p ' ’1! ‘- H fsd‘ H», *r‘ A. -M.-.... - 4 mg '1“ mm H '5‘!“ z:(‘ 9 (It ,1 V l I 11 ’ ‘J‘A‘ a? ‘ v-n —. n—E —‘.—- ' ' *m 1 '“~“—E~=:— 31—2: 9 .. .. ‘5" .4; 5:33:14 F" 71325-” ~ 9: '. “fir? .: 3&9».- w “1% I 4’ ‘ 155“»? 31%;; n 3' 3‘ mmw‘rgqtg‘h; t 8"qu u “ 3%: 3%“ - . 1"” 2.35133 '3 WWA.‘7.JJ' -..V-.. in} “”1513"? HI (I ("51, J . in, 4 {iii ‘1‘ I'vz‘7'tg . . WM“?! 1‘ .. 5.41% M :‘ .z' ‘ V . .1- b " " NI" ! . '5443‘31 ‘ ‘4 l l} l ;...::2:. a: 551’" h f ‘1' f? . ' . a, «1.23%.?- .5 RI L 2?. 4w? am“ 5 3- «1», 3%} L ‘l l )i ‘ ‘Lfifif‘gqukfi'r‘g " - t :15} ‘4‘ “RF: 7: 2&1,“ ”L‘L‘éiggiméi‘i". - NW, , I "" . 9‘. 3' L L. ; “”1eg s; :11? J33 “til-:Wfikgk ‘58)“; 'S ‘ mmmdfim’hm a; ”99:3“ ‘ ‘fisfif‘a '3‘! ‘iifié'ifighfiimmfififiu.qu' % Mm gems; , fig ‘3‘?”% ’ ’i ”w * “if “”3 ”if 33‘ ' fig‘dfii'i‘ am“; y ‘. m tafiifi‘ligifip L? W3? ‘ 'T - . ‘~ 31.. _ '1' ‘ .3“ 2:: 431.; V “2.. :f ‘ ‘ h; . 9,“ m9, {1 N s 9.55: 1“ .. «a? A W. - 372: *3?“ Hz: .9... - ".r‘rz‘ . tax—2‘32. "5‘.— . ' ‘é‘ifim 355459”: a .__.... A_ 1.3 “M ' .41; <3", 1...__..§ '3 N “"3? ' '2. afiC-r‘ c.- ‘ ""- 333-: .. - 5. ”.19” ' 5. ."faa‘nv'a 1.5???- (gégy'ihéai 1' ‘r—v' ;~asa§f§‘f"‘"""5%§r r , "- -2“; 3’“: a .. ‘ x 1. ‘2’ is" . h 4. I W i 1 if. _.: .‘1' H. 1 a ... h 5"»: ' .;.;(M'1:d) unsungx‘figfi ' h ‘- I v p- r 3’ ”33257:: 3 mac‘s.“ ,V E:— :‘ 543i ' :Jfifimg’ ' I: ‘ Wu 0- "x . « A. -. “:5” _r_ ."'-‘.-',A ‘53:" ' " "z A «frag '— " ". 4A 1 A , ._ . ,;-Y“" ‘ . E v; ‘ _ « (”-33-4 .3! 35$ '5 .. I 1 v v . . $35. 9452— H ; Li.“ W“ N eff} WI h “:1- u‘ . \k» V A??? In ‘Fkthfgzfi‘ék ha} a ) _l='£— ‘13:“ '"d k r , . ‘ u ‘. g‘x“ ‘u hr" 9%.. ‘ ’fl Lfl'n z, ,7" .5' n ’ .. . 9% we?» MM 3,2 “9 ' new-919: zwa‘ Ma “w? 5:! ~ 3 99 . : I "1341‘“ xi fist“? kg“. Lihdfl' I .T'Asni‘ . .> :51 ‘9 1’! RN Lghéfit‘mkfl r, a» w. x“ (lb. 32%;“ 33:99" 45mw‘dvt‘ ' ‘7'; F’s} ugf .93»: ‘ -... ¢ V Fr. Tic—t: '99 3 ' A. ‘ _ ‘ -:,4~_ A -A . z, a.” V .D _, _' "'_ .. .JQ' 4' ‘Wr" .- A» _ 33.12.: ,, ..:—_V, ‘ fl *3; 4.. - v” w ‘ 7‘3.3.37-;1- 9.3; _ :- .. 4?”-9“ .. ‘7 ..V a ,< ._ -13?" I: . ‘3.) ”.32..“ it? 3"“ «1— .2.. ‘ .4" «:r“ “ as“? £3.35: x 39‘" w {Lgsk 1 - u‘ r‘ “M: at” “J? y a » ‘-"“‘xh~'i‘§éhh‘i‘rfiw‘§$§€m' w .gq my , Tfinmqm. Mm VT!- r‘i 1.4““ . ‘Lmiyfi #3 53’»? 9. rug-hi mm M. . . {‘1 l h .; ‘Bl’vfl . '4 be L ‘ x' 5%” 131W“? 1 %{ Tr' ”-3;me v. ‘1. 1-.- 'E‘gr.‘ .1. 72:“ .rftzy. v: viz-Ii??? 5,331? a: 1‘55 L353 ' ‘11-” W :2. ‘1: "2.5.11 :12? '5 N ggg‘" r.‘ 1:27.}: 3; ma _ M .321; .3 .. 2' 33% C- 3’ F... If: JET {Li ... - as? a ‘ 9 'V~ 9 '4'; 3 =1 "'1". 1.: 95‘ 2:3 1’56Q$‘3948?3= .55 . ' ”1%. mm g‘m' '9 ‘ Fa ij‘hn‘knhéfififlr, , ‘ ‘1 ‘t‘ié'w' x. u é» )--a- ‘ .1- ‘1» ”£51” 3:??? 9.2.; ‘ *3: 1": m 592%: ’ @533 .5." «53“ 5—3 .359 art 2337?; :47; r“ 55537 2f,- ‘ .z-vi- .3 ~57 Jr “ _ ‘ new 33:3 d?! ,— F’Ltf-v : < ‘77 v ’" u -—.:~ . ; “’:":f' . argn." ,9 j -2, ‘ , .. I: ,. <. 322‘ - rcr .L; ‘ _ ‘ ~ M. 3:33“ . Jr... , 9.33:“ ”J?“ ‘4". .25.; . t) i h? fit? i‘t " €51; _u‘...:.‘=:.~;:: 5:3: wdfl—w ‘23-‘- A Scrv "iv-:52- 2"” Eggs-5;: . ‘ JESS: ‘ '7'“: " -. . 9 49:23 9 .__ r w 5?; fig: ’13:,» Jyfi “$5223 .. r , .4953” mines“ 13.53" 9-?“ 12:3 L” . .y 9-.....5..- a. A 7 . ’- ram... .— ':t'.‘.~:.L',5-'>~:z - ___._:'.r: r). ‘ l - Kilt): " ‘ U. l L 51 3‘ l ‘ ‘m. NH: {9' ,, I“; 9‘. ”HA" :1 "W‘swfii‘im ‘ m ; 3g; gar??? 9*? W >.. 4.1:: : ,. 9 WW1“ ”319% ‘ifiw‘z ma ‘ ‘3 ‘V‘ "“‘5 ‘ ”1 Mz- 8 U .. 'h ' a . {Mt ‘. {E E: “I'M,“ 19:53:13.} fig)?“ figfig 39$ *, ”1’1. :. 1H..H 9. J 1‘; l! # ffitquJK -.-z;«;"‘ - V . 'éfir. ; _ ‘ 1....“41f'g'3 $3 ‘ ., ..~J=-£%§~==:r 79 raw 4 93‘.” Z'Lfirfi" M“- V 9253.- :-;~: {53:21, , A. ‘ “ .. 'Zz‘f "x... ‘_. g:.'..:"........ E 1“”- - arr-o— ‘21: " " r942; ""r-‘w’fAET‘ZiLrg‘. 9.532.523- 51- Jfl‘, “gr-4... fiat—3 ‘ «5:3; _. A ” 4915-39 a .r $§~ 9.32: ‘ ii."‘..}..‘:‘ .A “M. "raga-am... ‘ a 1 ,_ ~ 93;“? «95:. 3 ‘V‘ ‘ . 2 31 “ Mafia-as 13:: em"? M “3:5? $32.21 A ’L‘ $3w§9‘£.» (“.7 " 1y, y We..." _ fir“: 3:39: “-35% L.- -.9‘" A ‘ ....J-' ‘_'. . . ‘ : _‘ 723‘; ‘13! '3;— .31.. M ”as?" “3 ‘&$’, :3,” . 73 . :3' N q~ M. 31.3333: . «3135:; 3?. «0.... -gé“ U _ . 1‘ a: ~35 -: "'7..:c:~?§£‘ ‘ x M...- 504—" TC; 3:;th . n .52 A; ‘.—.. [33' “L 3.? ‘ 4 <3: as? :33? ‘ ~-— “" so} “.41.“. _-;.::_3? 3 A m. {53. 39:"? 'fEi"%"f“ Ny.“ w...-....‘,.:. 13-.-»-.. “3*” ; ~- 746' gang: 5. 1-... ., ~._‘ IHESIS um" ’ will i mg Michigan State 31293 University ll llllfllllllllll fl This is to certify that the thesis entitled COGNITIVE CATEGORIES OF RATERS IN PERFORMANCE APPRAISAL AND THEIR RELATIONSHIP TO RATING ACCURACY presented by Cheri Lee Ostroff has been accepted towards fulfillment of the requirements for M.A. degflmin Psychology Daniel R. Ilgen Majos’ professor Date 7/30/85 0-7639 MS U is an Affirmative Action/Equal Opportunity Institution MSU LIBRARIES RETURNING MATERIALS: Place in book drop to remove this checkout from your record. FINES will be charged if book is returned after the date stamped below. new will COGNITIVE CATEGORIES OF RATERS IN PERFORMANCE APPRAISAL AND THEIR RELATIONSHIP TO RATING ACCURACY By Cheri Lee Ostroff A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of MASTER OF ARTS Department of Psychology 1985 {3 _,.— . .4] u. g..' ..‘ ‘1 ’ / ( ABSTRACT COGNITIVE CATEGORIES OF RATERS IN PERFORMANCE APPRAISAL AND THEIR RELATIONSHIP TO RATING ACCURACY By Cheri Lee Ostroff This research investigated cognitive processes affecting the accuracy of performance evaluation. One hundred and twenty-five hospital nurses completed questionnaires measuring their cognitive categorization processes and then rated a videotaped nurse's performance. Results indicated that rating accuracy was related to the match between raters’ cognitive categories and rating scale dimensions. Little support was found for the effect of "miscategorization" of behaviors and dimensions irrelevant to job performance on accuracy. Raters' ability to differentiate between dimensions was related to accuracy and to halo in ratings. While raters' tendency to describe people in terms of behaviors or personal characteristics (traits) showed little relation to accuracy, relationships to raters' cognitive categorization processes were found. Finally, work experience and job position were related to raters' cognitive systems while prior rating experience was important for accuracy. Implications, limitations of the study and future directions for use of cognitive processes related to rating accuracy are discussed. ACKNOWLEDGEMENTS First, my thanks are extended to my committee members. My sincere appreciation and gratitude go to my chair, Dan Ilgen, for his guidance, clear thinking, and support. Neal Schmitt and Steve Kozlowski are also thanked for their insightful comments and willingness to help whenever needed. All three committee members have been an integral part of this thesis, but more importantly, they have contributed to my knowledge, understanding, and dedication to I/O psychology. Secondly, I am grateful to the nurses, directors of nurses and hospitals for their participation and cooperation in this study. Many of my fellow graduate students deserve thanks for their performances in the videotape developed for this study and for their willingness to complete pretest questionnaires. Finally, I would like to dedicate this thesis to my grandfather, Rabbi Alexander E. Levin, who has always been an inspiration to me and who has constantly supported my endeavors. ii TABLE OF CONTENTS Page LIST OF TABLESI-II-IIIIIOII-IIIIII-IIIIIIIIOIIIIIOIIII V H INTRODUCTION-I...IIIIII...-.IIIIIIOIIIOIIOCCUOIIIIIII. Overview............................................ 3 Impact of Categories on Cognitive Processing........ 6 Attention......................................... 8 Organization and Storage.......................... 9 Recall............................................ 11 Summary........................................... 13 Theories Related to Categorization.................. 14 Implicit Personality Theory....................... 14 Summary......................................... 18 Personal Construct Theory......................... 18 Empirical Evidence of Categorization in Performance Ratings............................... 21 Objectives of the Present Study..................... 27 Accuracy Hypotheses............................... 28 Possible Correlates of Category Systems........... 31 METHODUIOICIICICUCUDI.‘IIIOIIIIICICIIIIIIIIIIIII-III.- 33 Research Setting.................................... 33 Sample.............................................. 33 Development of Stimulus Materials................... 34 Role Grid......................................... 35 Behavior Grid..................................... 37 Reliability of Grids.............................. 41 Background Ouestionnaire.......................... 42 Videotape......................................... 43 True Scores..................................... 44 BARS Performance Rating Scale..................... 51 Trait Performance Rating Scale.................... 53 Procedure........................................... 53 Data Analysis Measures ............................. 56 Category Orientation.............................. 56 Degree of Match to Rating Scale................... 57 Categorizing Job and Non-Job Related Behaviors and Dimensions............. .......... . SB Cognitive Differentiation......................... 59 Accuracy.......................................... 60 Halo.... ..... ............ .......... . ....... ....... 61 Job Experience Variables.......................... 61 iii RESULTSI-III-IICCCIIIOIIIDII.III-IIOIIIIII IIIIIIIIIIII 63 Accuracy Measures.......................... ..... .... 63 Cognitive Measures.................................. 66 Hypotheses..... ...... . .......... .................... 70 Category Orientation.............................. 70 Hypothesis 1.................................... 70 Dimensionalizing Behaviors........................ 74 Hypothesis 2............................... ..... 74 Hypothesis 3.................................... 74 Hypothesis 4.................................... 76 Hypothesis 5.................................... 7B Experiences of the Rater.......................... 78 Hypothesis 6......................... ......... .. 78 Additional Findings................... ............. . 84 Category Orientation and Category Dimensionalization.............................. B4 DISCUSSION-U...COCO-IIIIOI-I IIIIIIIIIIIIIIIIIIIIIIIIII 87 Category Orientation................................ 88 Cognitive Categorization. ................ ........... 89 Dimensionalizing.................................. B9 Miscategorization................................. 91 Cognitive Differentiation........................... 93 Experiences of the Rater............................ 96 Category Orientation and Categorization............. 98 BARS versus Trait Accuracy.......................... 99 Limitations of the Research............. ....... .....100 Summary and Future Directions.......................103 APPENDICESICIIIIIDI. IIIIIIII 0.. ...... III-IIOIIOOIIOIIIIOS A. ROLE GRID.......................................105 B. BEHAVIOR GRID...................................109 C. RETRANSLATION PERCENTAGES FOR BEHAVIOR GRID.....117 D. BACKGROUND OUESTIONNAIRE........................123 E. SCRIPTS FOR VIDEOTAPED PERFORMANCE .............. 125 F. BARS SCALE................................. ..... 144 G. TRAIT RATING SCALE................. ..... ........151 H. LETTER SENT TO HOSPITALS ..................... ...154 I. CONSENT FORM....................................159 J. EXPERIENCE VARIABLES FOR NURSES.... ....... ......161 K. RAN DATA AND CODING INFORMATION ................. 164 FOOTNOTEDIIIIIIII IIIIIIIIIIIIIIIIIIIII IIIOUIIIO ..... .I181 REFERENCES..... ........... . ................. ..........182 iv 10. 11. LIST OF TABLES Videotape Scenes Representing Each Performance Dimension............................ Videotape Scenes Representing Each Trait DimenSion-.I.......II.I..III-IIII-IOIIIIIIIIII... True Score Ratings of Performance for BARS scale--IIII....I-OIUIICI-'IIII...‘.I.I..IIII'D... True Score Ratings of Performance for Trait scale-IIIIIICIIII-IIIIIIIIIIIIOOIIIIIIIIII. Means, SDs and Intercorrelations of Accuracy measures-IOIIIIIIIIDII...III-IIIIIIIIIIOIIIIOIIII Results of Analyses of Variance for Accuracy Means by Hospital Group.......................... Means, SDs and Intercorrelations of Cognitive measures.IIIIIICIIIIIIIIIIIIOCIOIIIIIIOIIOIOIIII. Correlations for Behavior and Trait Category Orientations by Accuracy......................... Analyses of Variance of Behavior or Trait Category Orientation by Accuracy.......... ...... . Correlations for Cognitive Measures by BARS Accuracy.................................... Correlations for Cognitive Differentiation by Halo-IIICC-II.II.I.COCO-CIIICIIICIIIIIII-IIIII Correlations for Experience Variables by Cognitive Variables.............................. Correlations for Experience Variables by accuracy Measures-IO...-IIIIDII-IIICOIIIIIIIIIII. Page 45 46 SO 64 67 68 71 79 81 CD I24 INTRODUCTION The measurement of performance plays a major role in many personnel decisions. Performance related information is used for promotion decisions, selection of employees, compensation, as criteria for evaluating training and selection, and for feedback and employee counseling. These decisions are based on distinguishing between satisfactory and unsatisfactory employee performance (Ilgen & Feldman, 1983). The performance rating scale is by far the most ubiquitous form of performance measurement utilized (Borman, 1979; Landy & Farr 1980). Yet, despite its widespread use, numerous authors have expressed dissatisfaction in the lack of success organizations experience with performance ratings (Carroll & Schneier, 1982; Devries, Morrison, Shullman & Gerlach, 1981; Ilgen & Feldman, 1983). Most problems center on the many rating biases that permeate ratings (Landy & Farr, 1980) resulting in low to moderate inter-rater reliabilities (e.g. Borman, 1974; Zamuto, London & Rowland, 1982) and questionable validity and accuracy (Bernardin & Pence, 1980; Borman, 1981; Carroll & Schneier, 1982). In an effort to improve the quality of performance ratings, researchers have tended to focus on 1) the psychometric characteristics of rating formats (e.g. h.) Kingstrom & Bass, 1982; Latham & Wexley, 1977); 2) training raters to minimize errors (e.g. Bernardin & Pence, 1980; Borman, 1975; Fay & Latham, 1982); and 3)increasing observational skill (e.g. Bernardin & Walter, 1977; Borman, 1978). For the most part, research has neglected the issue of accuracy, defined as the degree to which ratings match or correlate with true criterion scores (Dunnette & Barman, 1979). Further, prior research has tended to overlook the process of appraising performance. The performance appraisal process refers to the cognitive and behavioral actions that occur within the performance evaluator as he or she observes the behaviors of persons being rated and translates these observations into decisions about and evaluations of ratees (Feldman, 1981). In the last few years, researchers have concluded that a model of the performance appraisal process is necessary before any significant advances can be made in understanding performance measures (e.g. DeCotiis, 1977; Devries, et.al, 1982; Kane & Lawler, 1978; Landy & Farr, 1980). The result has been the emergence of a new focus in performance rating literature - a focus on analyses of the rating process and a consideration of rating accuracy. Several models of the rating process have been proposed (e.g. Borman, 1978a; Cooper, 1981; Feldman, 1981; Landy & Farr, 1980) but Ilgen & Feldman's (1983) model is the most IN comprehensive in illustrating the various factors that can influence performance ratings. In this model, the performance appraisal process is construed as a function of three interacting systems: the organization within which the appraisal takes place, the appraiser's information processing system, and the behavioral system of the appraisee. While many factors are influential in the rating process, in order to understand the determinants and effects of performance appraisal, one must begin with a realistic conception of the evaluator as an information gatherer and processor operating in a complex system (Ilgen & Feldman, 1983). This notion of understanding the cognitive processes of the evaluator underlies all performance appraisal process models proposed. Thus, in the rating process, all information must pass through a cognitive filter represented by the rater, and the factors influencing the rater cannot be fully understood until the cognitive processes of the rater are understood (Feldman, 1981; Landy & Farr, 1980). Qxecxieu The present study focuses upon cognitive issues in performance appraisal with specific attention given to individual differences in cognitive processing and their influence on performance ratings. While individual differences in performance rating have been investigated occasionally, these studies have tended to focus on demographic characteristics rather than influences of cognitive operations. Recently, borrowing largely from social, cognitive, and personality psychology, many theories and methodologies have been used to represent the judgmental processes of the rater. More specifically, the cognitive processes involved in rating have been viewed as those of gathggigg, gtggigg, and :ggalligg information about others. Research directed toward studying the rating process has done so from 1) an attributional perspective (e.g. Green & Mitchell, 1979); 2) a policy capturing framework (e.g. Zedeck & Cascio, 1982); and 3) a focus on interpersonal constructs and category systems (e.g. Bernardin, et.al, 1982; Borman, 1978; Nathan and Lord, 1983). Research in these areas has demonstrated that individual differences in the way people categorize or store information affects rating errors and accuracy; yet, an individual differences approach analyzing the categories involved in performance ratings and their relationship to rating accuracy has not been undertaken. The present research proposes that an understanding of the nature and content of the way individuals categorize the work behaviors of others is necessary in order to understand the performance appraisal process.) It is argued that a comparison of good to poor raters will yield differences in the nature and content of their cognitive categories. Accurate raters (i.e. good raters) are those people whose ratings of others' performance match the true criterion performance ratings while inaccurate raters (i.e. poor raters) provide ratings which do not match the true criterion scores. It is hypothesized that accurate raters have cognitive structures related to job dimensions while inaccurate raters possess more general, global category systems which apply across a variety of situations, rather than specifically to the job in question. To the degree that raters possess category systems consistent with those of the rating scale, performance appraisal accuracy should increase. Further, the view taken here is that people tend to possess either trait-based or behaviorally-based category systems; that is, information is placed in "storage bins" based primarily on trait (personal characteristics) dimensions or on behavioral (explicit actions) dimensions which focus their perceptions of others. Thus, it is hypothesized that the raters' category tendency, trait-based or behaviorally-based, will affect the accuracy with which he/she rates performance when using trait—based or beha- viorally-based rating scales. Finally, factors such as job experience, rating experience, and job position, should correlate with the category systems raters develop and with their rating accuracy. If the rating process of accurate versus inaccurate raters is delineated, these findings can be applied to format development, rater training, the investigation of rating biases and environmentallsituational influences on raters, and interrater agreement in the rating process. Before addressing these issues and presenting the hypotheses of the current study, a framework of the cognitive processes in performance rating and the overriding influence of cognitive categorization is presented to aid in the understanding of these processes. Imeaet 9i QEQOiEixe Categocies 9O QGQOitive Processing Performance appraisal is a specific case of more general cognitive processes and to fully understand it, these general processes must be considered (Feldman, 1981). Several models have been posited, but Feldman's (1981) model best represents this approach by delineating four basic components raters use in making performance ratings. The rater must 1) attend to relevant information about ratees; 2) organize and store this information; 3) recall relevant information and 4) integrate the information into a summary judgment. The processes are complex and cyclical; processes at any one point are influenced by the state of the system at all previous points. The task of attending, storing, recalling, and integrating all the information presented in a perfect and completely accurate manner is very demanding, if not impossible, for an observer to accomplish. Individuals have highly limited spans of attention and limited storage capacities in memory (Miller, 1956; Taylor & Fiske, 1978). Thus, as individuals are forced to operate within the limitations of their cognitive capabilities in processing information, they seek strategies to reduce the enormous effort these four processes require. One of the most common strategies for simplifying this task is the use of categories (Rosch, 1975) or "storage bins" (Wyer & Scrull, 1980), by which information is stored in memory. Analogous to the category notion is that of the "schema", a cognitive structure representing some group of stimuli (Taylor & Crocker, 1981). Schemas, or categorization - the process by which stimuli are grouped into clusters - is central to all four of the above-mentioned cognitive processes. The structure and interrelationships of those clusters are regarded as the key concept linking the processes of attention, storage, retrieval and integration. To appreciate the impact of category systems on judgmental processes it is first necessary to discuss the processes of attention, storing, and recall in more detail. Research on these processes is extensive; but for the purpose here, only a brief discussion of each is presented (For a more complete review, see Ilgen & Feldman, 1983). attention Considering the notion that individuals have limited attention spans and memory storage capacities (Taylor & Fiske, 1981), information selectivity is a requirement of attention. This selectivity can be evidenced in two manners. First, aspects of the situation may make a given person or a given feature of a stimulus person more salient. Attention may then be automatically directed to these salient people and/or attributes (Taylor & Fiske, 1978). Secondly, individual differences in the cognitive categories raters use for evaluating others may influence the salience of particular stimulus attributes. In the performance appraisal setting, the category systems raters use in evaluating others in the work context may lead them to attend to different aspects of work behavior. Markus (1977), Kelly (1955), and Sechrest (1968) have shown that people differ in their category systems for evaluating others. Further, these studies suggest that the category system used to interpret and store information directs attention to the different aspects of the stimulus environment (Ilgen & Feldman, 1983). A better understanding of the organizing and storing of information will aid in the development of these notions. In order to make sense of their environment, individuals organize and store information in categories enabling them to process large amounts of information despite limited cognitive capabilities (Taylor & Fiske, 1981). The relationship of attention to categorization is delineated by Bruner (1958): Perception of the stimuli implies the placement of a stimulus in a category along with similar stimuli that share certain of its features. Thus, category memberships, assignment of people or stimuli to category groups, follow from whatever category system the individual has developed to organize the world. Rosch and her coworkers (Rosch, 1977; Rosch, Mervis, Gray, Johnson & Boyes-Braem, 1976) assert that categories are "fuzzy sets"; the possesion of every relevant attribute in the category is not necessary for membership. To the extent that characteristics of a stimulus overlap with those of a particular category prototype, that stimulus will be placed into that category. A category prototype refers to sets of the most widely shared features of category members (Rosch, 1975). Category schemes or prototypes (Tversky, 1977) represent the most widely shared features of category members and thus summarize resemblances among category members. Not only do these category systems affect the 10 information attended to, but once a stimulus is categorized, recall of that stimulus is biased toward the general characteristics of the category (e.g. Cantor & Mischel, 1977, 1979; Tsujimoto, 1978). Thus, evaluations of others are colored by their assignment to a category prototype. Of most importance to the present proposal is the evidence reporting that characteristics of perceivers affect the nature of the category systems these persons hold. People differ in terms of the number and nature of the categories they possess (Ashmore & DelBoca, 1979; Feldman, 1981). Research by Markus (1977), Rosenberg (1977), and Sechrest (1968) has revealed substantial individual differences in category constructs. Not only do the categories differ across people, but the salience or importance of any particular category varies across people. When a given person or stimulus is immediately or automatically assimilated into a category prototype, further information about the individual will be automatically interpreted and stored in terms of the most prominent features of that category. In turn, the category becomes more accessible, or more readily available for use, than others (Wyer & Scrull, 1980) and may further bias evaluations. Features of the situation may also increase the salience of a particular category, making the perceiver's use of that category more likely and focusing attention on stimulus features related to the protoype and away from other features (Taylor & Fiske, 1978). It has also been shown that cultural factors (Triandis, 1964) and individual difference variables, such as prejudice and cognitive complexity (Feldman & Hilterman, 1975), make different categories salient for different people. Relevant to performance appraisal, to the extent that information about employees is stored and recalled in terms of category prototypes, the actual category system used by an evaluator will play a role in subsequent evaluations. The most recent research related to categorization in the performance appraisal area has been in the cognitive complexity of raters (e.g. Bernardin, Cardy, & Carlyle, 1982; Sauser & Pond, 1981; Schneier, 1977), which will be discussed later. Resell Due to limited cognitive capabilities, judgments of a person are seldom based on all the information one has acquired about the person; rather, they are often based on a subset of this information that can most easily be recalled (Wyer & Gordon, 1982). Presumably, the interpretation and organization of new information about persons is based on "categories" one has stored in semantic memory. Categories of the rater may influence and bias subsequent recall of the information. Once a stimulus is categorized, recall and recognition of that stimulus are biased toward the general characteristics of the category and may include recognition of information that was never presented (Cantor & Mischel, 1977, 1979; Wyer & Scrull, 1980). For example, Cantor and Mischel (1977) found that presentation of information consisting of prototypical characteristics of a person led to recognition errors that were consistent with this assumed prototype. Stimulus characteristics may be more difficult to recall over time as they become colored in such a manner to be consistent with the characteristics of the prototype to which they were matched (Wyer & Scrull, 1980). Further, information consistent with a prototype is most likely to be recalled, while information inconsistent is least likely to be remembered (Hamilton, 1979). Encountering a stimulus which matches already established categories increases the recall of category relevant information. The previously existing category systems of persons influenced recall of information consistent with categories already in use by the indiviudal (Taylor, Livingston & Crocker, 1982). Thus recall of information is intimately related to the category systems of the perceiver. The following section summarizes the role of categorization in attention, organizing and storing, and recall of information. Summacx The processing of information involves 1) attending to stimuli in the environment, 2) organizing and storing the information gathered in categories and 3) recalling this information for later consideration. The perceivers' category systems influence each stage of this process: categories direct attention to certain information or attributes; categories affect how this information is organized; and categories affect the manner, amount, and type of information recalled. The task of the rater in appraising the performance of others involves all of these cognitive processes: attending to relevant work behaviors of the ratee; storing and organizing the information acquired; and finally recalling the stored information to produce an evaluation of the employee. Therefore, it follows that the category systems raters employ in this process will affect their evaluations and the accuracy with which they appraise others' performance. A rater’s category system may cause him or her to attend to attributes and behaviors irrelevant to job performance (i.e. race, sex, dress), to store this information in category prototypes irrelevant to job dimensions, and to recall and integrate this information in 14 a biased or inconsistent manner. A better understanding of the role of these categories would certainly aid in understanding and improving the rating process. Ibeeclee Belated t9 Eelegecleetlee The role of individual differences in category systems is well documented, but an examination of the nature and content of these categories in relation to performance appraisal has not been undertaken. Do people differ in the category systems used in the work context? Do the category systems of certain raters yield more accurate evaluations? Two bodies of research are relevant in this respect, implicit personality theory and personal construct theory. Both of these theories and their associated research examine the categorization schemas people use in constructing their environment and in evaluating others. These two areas will be discussed with special attention given to differences in category systems of raters. Also, the research evidence applying these notions in performance appraisal will be presented. Finally, the use of these theories will be delineated with respect to the objectives and hypotheses of the current study. lmellele EeceeoellEX Ibeecx Implicit personality theory is concerned with how people construe traits to covary in others, or in other words, 15 persons' perceptions of relationships between traits (Bruner & Taguiri, 1954; Schneider, 1973). Research in implicit personality theory has demonstrated that raters use their own categories, or implicit theories, to judge others and that these categories relate to trait dimensions. Thus, the rater's perception of how trait dimensions in his or her category systems covary will effect the evaluation of others. Focusing on relationships among dimensions, factor structures of personality ratings were found to be relatively unaffected by how well raters knew ratees (Koltuv, 1962; Passini & Norman, 1966). Thus, Passini and Norman (1966) concluded that based on superficial information about others, people draw upon their own category systems to yield structures of rating people they do not know that are similar to those of people with whom they are intimately acquainted. Further, it has been demonstrated that the correlation among trait dimensions could be accounted for through constructs of the rater rather than through co-occurances of behavior patterns of ratees (Hakel, 1969; Norman & Goldberg, 1966; Passini & Norman, 1969). Multidimensional scaling techniques have been used to discover individual differences among groups of persons who differ in their patterns of perceived similarities between 16 traits (e.g. Pederson, 1965; Walters & Jackson, 1966). Essentially in these studies people were asked to judge the similarities between pairs of traits. The persons were then grouped according to the similarity of their judgments and personality variables were correlated with these different "points of view" based on their groupings. Reliable group differences in implicit personality theories of individuals were found; however, these differences were not related to personality variables (Walters & Jackson, 1966). In relation to performance appraisal, performance ratings may represent specific instances of implicit personality theories of raters. Values raters provide on performance scale dimensions may be independent of the actual behavior of the ratees on these dimensions (Landy & Farr, 1980); raters may possess "theories" composed of trait dimensions and intercorrelations between these dimensions that may or may not match actual conditions. Nathan and Alexander (1985) argue that raters' implicit theories have important implications for rater accuracy. Raters whose implicit theories about performance represent the ratee's actual performance will provide accurate ratings, while those whose implicit assumptions about behavior are incorrect will yield inaccurate ratings. Further, the degree to which a rater assigns stimuli persons to the proper category prototype will influence accuracy of 17 the ratings such that correct assignment promotes accuracy and incorrect assignment yields inaccurate ratings. Finally, in the absence of sufficient observation, proper association between raters and correct prototypes can result in ratings more accurate than would have been possible based on otherwise limited behavioral observations. Implicit personality theory has been used to explain two rating errors, halo and systematic distortion. Briefly, halo represents inflated correlations between dimensions, that is, a higher degree of correlation among all traits than is warranted by actual intercorrelations. Halo has been found to exist in studies when comparing intercorrelations of rating to actual levels of relationships suggesting that individuals distort the magnitude of relationships between dimensions of personality and job performance (Borman, 1975; Nisbett & Wilson, 1977). Systematic distortion relates to distortions in correlations between dimensions in the direction of semantic relationships between those dimensions. Research by Shweder and D’Andrade (1980) has shown that in the absence of relevant information about ratees or when sufficient time delay has elapsed between observation and rating, inter— dimension correlations of ratings are likely to be biased in the direction of semantic similarity. Further, Borman (1983) posits that both halo and systematic distortion 18 operate simultaneously; both inflation in correlations and distortion toward semantic similarity occur in personality and behavior ratings. ngmggy. Implicit personality theory has focused on individuals’ evaluations of others based on the covariation of traits contained in their category systems. Research using implicit personality theories identifies the importance of individual differences in raters' conceptions about person dimensions. Yet, the theory simply describes the effects of individuals' implicit theories, rather than evaluating the nature of its process. What is needed is a more thorough investigation of the individual differences in the category systems comprising implicit theories in the work context. Personal construct theory provides a manner to investigate these issues by exploring individual differences in category systems in person perception. Personal Construct Theory Kelly (1955) proposed a psychological theory of personal constructs describing how people function and view the world. As part of this theory, individual differences in person perception were described which have some relevance to performance appraisal rating processes. Kelly asserted that each individual formulates, in his own way, constructs through which s/he views the world of events. That is, 19 individuals develop personal construct systems, or categories, which they use to judge events and make predictions about future events, and they impose these categories on person perceptions. While similar to implicit personality theory in that both conceptions relate to the interpersonal "filter" of information by perceivers, personal construct theory examines the individual differences in these filters in terms of their structure and content, while implicit personality theory focuses on the covariance of traits in raters' category systems rather than on real differences in the categories (Borman, 1983). Most research in personal construct theory has used the Role Construct Reporatory Test (RCRT). The test requires the respondents to record names of persons who fit a number of roles. The repondent is then asked to consider various triads of these role persons, and, for each triad, identify an important way in which two of the persons are alike and yet different from the third. Taken together, the responses constitute measures of the person's personal constructs. Studies utilizing the RCRT have consistently shown that when raters use their own personal constructs as the dimensions by which to rate others, larger variances across ratees result than when rating using scales provided by the experimenter (e.g. Adams-Webber, 1979; Isaacson, 1966). Specifically, it appears that individuals prefer to use 20 their own constructs to rate others (Bonarius, 1965) and they differentiate more finely between ratees when employing their own dimensions. Further, Sechrest (1968) and Rosenberg (1977) found substantial individual differences in the content of categories used by perceivers when using this type of free—response method. Personal constructs have to do with both relationships between interpersonal dimensions, as in implicit personality research, and in the content of those dimensions. Individual differences in content appear to have additional implications for performance ratings. The personal construct systems and the content of these constructs may influence what the rater looks for in observing persons at work. As an example, if one rater possesses the construct "getting along with others" and feels this construct is important in judging others at work, but a second rater does not, the first rater may focus on behavior related to that aspect of performance, whereas the second rater may focus on behaviors related to his/her own constructs (Borman, 1983). Borman (1983) proposed several promising areas of research involving personal constructs, which may serve as potential sources of inter-rater disagreement and accuracy in ratings. Research is needed 1) to determine if raters possess and can report meaningful constructs related to observing work behavior; 2) to evaluate the stability of these constructs in differing situations and contexts related to observing work behavior; 3) to examine individual differences in such constructs; and 4) to assess the impact of these differences on observations of work behavior and performance ratings. This research addresses issues related to one, three, and four above. Research in the performance appraisal area has applied the concepts of implicit personality theory, personal construct theory, and categorization to raters’ cognitive processing demonstrating that individuals process information differently and that category systems affect this process. For example, Cooper (1981) examined the concept of conceptual similarity in relation to halo in job performance ratings, focusing on raters' beliefs that rating categories are conceptually similar and covary thereby inflating observed correlation matrices. He concluded that conceptual similarities among job dimensions represent a potential source of covariation and will appear to be halo error in performance ratings. A recent study by Nathan and Lord (1983) compared Borman's (1978) notion that raters store information in independent dimensional schemata and retrieve this dimensionally independent information when making performance evaluations, with Feldman's (1981) view |’--J N that information is automatically stored and integrated as part of a prototype-based, global category. Results indicated that Borman's model was useful in demonstrating raters' ability to differentiate between performance dimensions, however a large halo effect was found consistent with Feldman’s model. The halo effect was evidenced by rating in accordance with an overall, general impression and by cognitively distorting stored observations, or by distorting the recall of behaviors consistent with their overall rating. The authors concluded that the support for both models may be due to individual differences in cognitive styles of raters. One approach to studying the rating process was devised by Banks (1979) who examined the behavioral cues raters use in making ratings. Subjects viewed each of Borman’s (1977) videotapes and rated each performer on one dimension at a time until all six performance dimensions were evaluated. Each time subjects viewed a behavior they thought was relevant to the performance dimension under consideration, they pressed a button on a computer console which corresponded to a seven-point rating scale. The buttons were attached to a timing device that provided an exact record of where in the tape each button was pressed. Subjects also provided a verbal description of the behavior they were attending to each time they pressed a button. M M Results of this study showed that different raters identified different behaviors as relevant for making performance evaluations. Further, "rating style" variables were investigated based on the verbal reports. Raters with effective styles of rating (defined as using specific vs. global information, using many vs. few behavioral cues, and attending to both effective and ineffective performance of the ratee) exhibited fewer rating errors than raters with ineffective styles. Finally, the individuals' rating styles were relatively consistent across dimensions and across ratees. This research supports the notion that individual differences in rating exists, and people focus on different job behaviors when rating; however, accuracy was not investigated in this study and the internal processes of the rater were not specifically investigated. It seems plausible that the category systems of individual raters may have guided their attention to behavioral cues and directed their rating styles. Numerous authors have cited the importance of investigating the cognitive complexity of raters in an attempt to understand individual difference characteristics in the appraisal process (Feldman, 1981; Kane & Lawler, 1979; Landy & Farr, 1980). Cognitive complexity was defined by Schneier (1977) as the “degree to which a person possesses the ability to perceive behavior in a multidimensional manner" (p. 541). He proposed and supported a cognitive compatability theory of performance appraisal in that cognitively complex raters, in comparison to cognitively simple raters, exhibited psychometrically superior ratings when using complex rating scales. Recent empirical evidence has failed to replicate Schneier's findings in relation to rating errors, or in relation to acceptance of the format, or confidence in ratings (Bernardin, Cardy & Carlyle, 1982; Borman, 1979; Lahey & Saal, 1981; Sauser & Pond, 1981). Bernardin et.al (1982) extended this concept to include its relationship to accuracy, but failed to show any support. Bernardin, et.al (1982) proposed that if cognitive complexity is the ability to view behavior in a multidimensional manner, then, in an appraisal situation, complexity would be reflected in the ability to conceptualize performance into multiple dimensions. Investigating the relationship of cognitive complexity to the development of schema using a free-response type format, Bernardin, et al. found that student subjects were better able to generate dimensions for rating instructors as opposed to rating managers, and subsequently committed less halo in rating instructors. The authors concluded that it is not a general trait of complexity that is important, but the complexity of raters' schema specific to the situation 25 that may be important in rating effectiveness. Thus, evidence implies that individuals have different schemas for different jobs, are better able to generate dimensions for jobs they are more familiar with, and that the dimensions generated are related to behaviors exhibited on the job. At this time, these notions are interesting but need more empirical support. Cognitive complexity is related to personal construct theory such that both concepts rely on the notion of individual category systems in evaluating others. While personal construct theory assesses the nature of the category systems individuals possess, cognitive complexity attempts to assess the level of development of a construct system in terms of the degree of differentiation between dimensions in the system (Adams—Webber, 1981). The cognitive complexity notion suggests who will use complex or simple systems; the cognitively complex rater uses contructs independently and utilizes more dimensions than the cognitively simple rater. Another area of research is concerned with the assimilation of specific job-related behavioral information into categories and trait-like judgments. Rating scales provided in the appraisal task typically require the rater to observe and evaluate specific job behaviors. However, evidence suggests that this behavioral information is integrated into categories which are global and/or trait- based, rather than based on the specific behaviors observed (Murphy, Martin & Garcia, 1982). A study by Phillips and Lord (1982) suggested that judgments of others stem from a mapping of stimulus characteristics into pre—existing cognitive categories or prototypes which affect subsequent information processing. Raters were found to distort performance descriptions consistent with their prototypical category. Subjects in this study were less able to distinguish between observed and unobserved behaviors when the behaviors were characteristic of the prototypical category, than when the behaviors were uncharacteristic of the prototype. These results indicate that focusing on the raters’ observation of behaviors alone is not sufficient to increase accuracy in ratings; the information processing mechanism of the rater also needs attention. Murphy, et a1. (1982) found behavioral information to be organized around general impressions. Recall of behavior was determined by the degree to which certain behaviors are representative of general judgments made about ratees. Rating scales asking raters to measure simple frequencies of behavioral observations (i.e. BOS scales, Latham & Wexley, 1977) were found to measure trait-like judgments. Thus, while performance rating instruments typically require 27 raters to focus on job behaviors, the effect of observing these behaviors and then incorporating them into the category systems of raters may seriously bias the ratings. Recall of behaviors appears to reflect judgments about the frequency of behaviors based on a structurally organized general impression. Taken together, this research indicates that individual differences in raters' category systems seem to exist and that the categories themselves influence the performance ratings given to ratees. It is not clear exactly what these category systems are, how they are structured, how consistent they are with the rating scales provided in performance appraisal, or how they impact performance evaluations, other than to suggest that all of these issues affect the degree to which accurate ratings of employees can be made. Obieetixee ei the Eceeeot Stud! This research proposes to explore the nature and effects of the category systems used by employees in evaluating the work performance of others. From a pragmatic standpoint, the effect of these category systems on the degree to which raters aggugatgly rate the work performance of others is the most important concern. Several hypotheses are delineated which relate aspects of raters' category systems to rating accuracy. A further area of interest is simply a description of the nature and types of category systems used in evaluating employees. At this stage, there are few a priori expectations about the nature of the category systems utilized by the selected sample, thus no specific hypotheses can be offered. Finally, it is of interest to explore possible predictors or correlates of specific category systems. One hypothesis deals with this issue. eeeeceex Bxeetbeeee One conclusion that can be drawn from the literature reviewed is that raters should provide more accurate ratings to the extent that their personal category systems match that of the rating forms used in appraising performance. Recent work in performance ratings has emphasized the superiority of rating scales anchored with job behaviors over scales using trait-oriented anchors (e.g. Carroll & Schneier, 1982). Thus it follows that: Hyggthgsig gag: When using behaviorally-based rating scales, raters possessing behaviorally- oriented category systems will yield more accurate ratings than raters with trait-like categories. For trait—based scales, raters possessing trait— oriented category systems will yield more accurate ratings than raters with behaviorally—based categories. Hypothesis one was concerned with raters' overall tendency to categorize elements in a behaviorally—based or trait—based system. However, given that performance rating forms generally contain behavioral anchors, raters typically are required to incorporate behavioral information into rating forms. One approach to increase rating accuracy on such scales in this process has been to train raters on the definitions of dimensions and on the behaviors contained within each dimension that are presented on the rating scale. Raters are given a common "frame of reference" by which to evaluate workers using the scales (Bernardin & Pence, 1980; McIntyre, Smith, & Hassett, 1984; Pulakos, 1984, 1985). Underlying this approach is the assumption that accuracy is increased because raters are trained to use a category system that matches the performance rating scale. A similar notion is reflected in Hypothesis Two: Hyggthgsig 159: To the extent that raters are able to dimensionalize job behaviors in a manner consistent with that of the rating scale, ratings will be more accurate. In appraising performance, the rater must determine which of the ratee’s behaviors are job-related and which should be observed and evaluated. Yet, considerable evidence suggests that non—performance related characteristics and behaviors of the ratee (i.e. sex, race, etc.) are observed and included in raters' category systems which may serve to bias ratings (e.g. Landy & Farr, 1980; Ilgen & Feldman, 1983). This implies that: Hyggtnggig Igggg: Accuracy in ratings will be related to the degree to which a rater is able to distinguish between behaviors and dimensions that are gglgyagt to job performance and behaviors and dimensions that are iggglgxggt to job performance. One aspect of cognitive complexity is cognitive differentiation, i.e the extent to which different constructs a rater uses are applied differentially to other persons or elements (Bieri, 1955). In other words, a rater does not apply every construct or category to refer to the same group of people, elements, or behaviors. Results of studies investigating cognitive complexity in relation to performance rating errors and accuracy have been conflicting (e.g. Bernardin, et al., 1982; Lahey & Saal, 1981; Schneier, 1977). However, Bernardin, et al. (1982) suggested that an investigation of raters' category systems specific to the job, rather than his or her general category system may be fruitful in examining rater accuracy. Thus, it follows that if raters are able to differentiate job behaviors into job dimensions with little degree of overlap, their ratings will be more accurate. Hypothesis Four follows: flyggthggig Eggc: More accurate raters have highly differentiated category systems for the job such that low intercorrelations exist between category dimensions, while less accurate raters are unable to differentiate clearly between dimensions. Similarly, the cognitive differentiation of raters should be related to the degree of halo in ratings. Raters who are unable to differentiate clearly between dimensions may form more overall general impressions of ratees. These overall impressions may serve to inflate the intercorrelations between dimensions when making ratings, thereby producing halo in the ratings. Empirical investigations of the relations between cognitive differentiation and halo have shown mixed results (e.g. Bernardin, et al., 1982; Schneier, 1977). However, if as Bernardin et al. suggest that situation specific cognitive differentiation, rather than general cognitive differentiation, is important for examining the rating process, cognitive differentiation for the job should be related to halo. Hypothesis 5 reflects this notion. Hyggthggig Eigg: Raters with more highly differentiated category systems for the job will exhibit less halo in their ratings than those with less differentiated systems. Given that the category system a rater possesses influences his rating ability, what are some of the determinants of these category systems. One such factor may be job experience? As workers gain experience on the job, they may develop category systems that incorporate important job dimensions and job behaviors. Secondly, the 32 hierarchical position or job position may cause raters to develop different category systems and attend to different aspects of job performance. Other factors which may determine aspects of the category system adopted are experience with evaluation, professional training, and educational experience. Thus Hypothesis Six follows: Hyggthggig Six: Experiences of the rater will be correlated with the category system he/she utilizes in evaluating the job performance of others and with rating accuracy. METHOD Beeeeceb Setting Nurses from three large midwestern hospitals participated in the study. All nurses were registered nurses working during one of three eight—hour shifts. The study was conducted on site at the hospitals. Semele Four groups of nurses participated in the study. The primary group consisted of 129 nurses in three hospitals, 125 females and 4 males. Data were collected from 17 nurses in hospital one, 61 from hospital two, and 51 from hospital three. Ninety-two percent of the participants had worked as a nurse for five or more years, and 872 reported having previous experience in rating nursing performance. Only 32 of the nurses were staff nurses whose primary duty was providing patient care, whereas 97% were charge nurses, head nurses, supervisory or nursing directors whose primary responsibilities included supervision of nurses and nursing acitivies. The nurses worked in a variety of units in the hospital. Due to some nurses' failure to complete assessments, usable data were collected from 125 nurses. The remaining groups of participants aided in the development of stimulus materials. One group of nurses 7 -"i 34 served as "expert raters" to provide true score ratings for the videotape, described below in the development of stimulus materials. Ten graduate students in nursing, all of whom were registered nurses, provided true score ratings for the Behahviorally Anchored Rating Scale. An additional ten nursing students, all of whom were licensed practical nurses, provided true score ratings for the Trait Rating Scale, described below. A third group of 15 graduate students and 12 hospital nurses provided information necessary for the development of two stimulus measures, the Role Grid and the Behavior Grid, described below. Finally, 11 head nurses completed some of the stimulus materials necessary to assess test—retest reliabilities. These nurses worked full-time but were also students in a management class where they volunteered to complete the measures. Qexeleement ei Slimelee deteciele Several measures were developed to evaluate the hypotheses described previously. Two measures were developed to assess the cognitive categorization processes of raters ~ the Role Grid and the Behavior Grid. In order to assess some potential correlates of category systems (such as sex, education) a Background Questionnaire was developed. Finally, 1) a videotape of a nurse performing job duties, 2) two performance rating scales for the 35 videotape, and 3) "true score" ratings for the nurse’s performance on the videotape were developed to assess rating accuracy. The development of each stimulus material is described in detail below. Bele field A Role Grid was developed based on Kelly's (1955) reporatory grid technique in order to determine the degree to which nurses possessed trait—based or behaviorally-based category systems. In this grid, participants were presented with a list of job role titles, for example, football player, salesman, or doctor. Nurses were asked to 1) consider specific triads of these roles, 2) pick two of the three roles in a triad and 3) write down a way in which the two were similar. A total of eight triads were presented. Thus, based on the similarity of two members in the triad, each nurse provided eight written responses representing behavior or trait "constructs." For example, consider a triad of roles, artist, comedian, and cartoonist. The artist and cartoonist both draw or use art mediums to create a picture; they share the behavior of drawing. The comedian and cartoonist both possess the trait of humor. Thus, the triad was scored as a behavioral construct if the person wrote a behavior such as "draws" or as a trait construct if a response such as "humor" was provided (See 36 Appendix A for the Role Grid). It was important that in each triad, a behavior could be identified for two of the roles and a trait could be identified for two of the roles. Selection of the role titles and triads was such that in each triad, two of the roles were alike in a behavioral action and two were similar based on a common trait. To develop this grid, a large sample of 39 roles and 13 triads was pre—tested on a group of eight graduate students and five hospital nurses. First, a set of two roles from each triad, which the experimenter believed possessed a common behavior, were presented. Persons were asked to write the behavior the two roles have in common. Next, they were presented with a different set of two roles from each triad which the investigator believed possessed a common trait. Participants were asked to provide a common trait that they perceived each of the two roles to possess. Triads for which at least 702 of the people identified a trait for two roles agg a behavior for the other two roles in the triad were retained. The ten remaining triads were pretested on a sample of 15 hospital nurses. Nurses were presented with the 10 triads in the grid, asked to pick two roles in each triad, and write down a way in which the two were similar. Two triads were eliminated from this grid because 33% of the people were unable to identify either a 37 behavior or trait construct for the triad. Thus, eight triads were retained in the final version of the Role Grid. The final grid contains only those triads for which participants were able to identify a common behavior and a common trait for two sets of roles in the triad when presented separately during the pretest and those triads for which persons could identify either a behavior or trait when presented together in the pretest grid. Each triad contains at least one way in which two of the roles are behaviorally similar and two roles are similar based on a common trait. However, it is possible that a person may perceive two roles identified as behaviorally alike to be similar based on a commonly possessed trait, or vice versa. To deal with this possibility, nurses were asked to write down how they perceived the two roles to be similar. Based on the written constructs provided by the participants it was possible to determine if nurses tended to use traits or behaviors when evaluating others. Eebexlec field A second grid was developed to assess 1) nurses' ability to dimensionalize job behaviors in a manner that matched the performance rating form and 2) the extent to which nurses categorized job behaviors into non—job related dimensions and non-job behaviors into job related dimensions. The dimensions listed at the top of the grid consisted of job 38 and non-job related dimensions and traits. The elements listed along the side of this grid consisted of job relevant behaviors and behaviors seen on the job, but irrelevant to job performance. Nurses read each behavior and placed a check mark in the columns under the dimension or dimensions for which they believed the behavior belonged. Appendix B contains a copy of the Behavior Grid. The job related behaviors were extrapolated from a BARS scale developed by Smith and Kendall(1963) specifically for nurses in hospitals. In developing the BARS scale, Smith and Kendall report at least 60% agreement among raters as to the dimension to which each behavior retained in the scale belongs. Further, there was significant agreement of these assignments when compared to an independent set of raters. To select those behaviors from the BARS scale to appear in the Behavior Grid, a very high rate of agreement from the retranslation procedure, at least 85%, was desired for each item. To determine which behaviors from the BARS yielded this agreement level and to determine if any changes in the scale would be necessary for this sample, a group of seven hospital nurses served as "job experts" and followed the retranslation procedure outlined by Smith and Kendall, for the behaviors and dimensions appearing in the BARS. From this retranslation, 20 job behaviors were selected for use in the Behavior Grid. Of the behaviors selected, 14 39 had 1002 agreement and six had 862 agreement from the retranslation procedure. These behaviors were listed on the right side of the grid. Four behaviors for each of five job performance dimensions were selected. The job related dimensions were taken directly from the BARS scale and listed at the top of the grid. These dimensions are: Knowledge and Judgment, Organizational Ability, Skill in Human Relations, Conscientiousness, and Observational Ability. The non—job related behaviors and dimensions used in the Behavior Grid were derived from a procedure similar to that used in developing BARS scales. Five hospital nurses were asked to generate separate lists of behaviors which they felt occurred on the job, but were not directly related to job performance. Such incidents as dresses fashionably, smiles a lot, and calls spouse frequently were included in the set. The lists of behaviors were examined by the experimenter and a list of 46 non—job related behaviors was compiled. Ten non-job related dimensions were developed which appeared to tap the non-job related behaviors listed by the nurses. The non—job related dimensions were: Sense of Humor, Outgoing, Humility, Creative/Artistic, Religious, Health—Conscious, Appearance, Well—Mannered, Communicative and Family Oriented. A group of 15 graduate students were asked to retranslate the non-job related behaviors into 40 these dimensional categories. A definition of each of the ten non-job related dimensions and the five job related dimensions (from the BARS scale) was given along with the list of non-job related behaviors. Subjects identified the non—job related behaviors which they perceived as belonging to each dimension. Five non-job related dimensions were consistently utilized and retained in the Behavior Grid: Health— Conscious, Outgoing/Communicative, Sense of Humor, Appearance and Family Oriented. Twenty non-job related behaviors related were identified for use in the Behavior Grid. Only those non—job related behaviors which had high agreement in this retranslation procedure were retained and only those non-job related behaviors were retained which were seen as belonging to the non—job related dimensions, and not the job related performance dimensions. Of these non-job related behaviors, 15 had agreement of 872 or above and five behaviors had at least 732 agreement in the retranslation procedure. The final format of the grid was in matrix form with behaviors as rows and dimensions as columns. Both rows and columns consisted of two subsets. For behaviors (rows), half of the elements were related to job performance while the other half were non-performance related. For columns (dimensions), half were job relevant and half were non-job 41 related. The placement of items within rows or columns was random. However, the matrix allowed for the determination of the following four scores: 1) degree to which behaviors were dimensionalized in a manner consistent with the BARS scale; 2) number of non-job related behaviors sorted into job-related dimensions; 3) number of job relevant behaviors placed into non-job related dimensions; and 4) ability to differentiate between behaviors and dimensions. Appendix C lists the job behaviors, the non-job behaviors, their respective dimensions, and the retranslation percentage for each behavior in the dimension. Bellebllltx Qfi Beige To determine the stability of the responses to the Role Grid and the Behavior Grid, test-retest reliablities were computed for each grid. Eight head nurses and eleven undergraduate students completed the Role Grid twice, and eleven head nurses completed the Behavior Grid twice, approximately one month between administrations. For the Role Grid test-retest reliability, nineteen complete sets of grids were returned. Three separate scores from the Role Grid were computed for each person: 1) the number of behavior responses; 2) the number of trait responses; and 3) the number of "other" (neither behavior nor trait) responses. Correlations of these scores between 42 the two time periods, across participants, were §=.83 for behavior; ;=.85 for trait; and :=.70 for other. It is important to note that these are not independent reliabilities as the data are ipsative in nature. Eleven head nurses completed the Behavior Grid at both time periods. For each nurse, the score on each item was correlated between the two time periods. Correlations ranged from .40 to .90 for the participants, the average test-retest reliability coefficient was :=.68. Since the variances across items was low for some nurses, it was felt that the correlations may have been underestimates of response stability. Thus, a percentage was computed for each nurse based on the percent of items for which responses remained unchanged over time. The percentages ranged from 85% to 97%; the average percentage of unchanged responses over time was 92%. Obviously, the responses were very stable over the month. In sum, evidence from these analyses of the Role Grid and the Behavior Grid indicated that the measures were adequately reliable over time suggesting that the cognitive categorizations of raters, measured by these grids, remains stable, at least over relatively short periods of time. Eeetgceeoe Qeeetieooelce A Background Questionnaire was developed to measure basic demographic and background variables of the nurses which may affect nurses' schemas and rating accuracy. The items in this questionnaire included years of experience on the job, job position, job title, unit in the hospital, educational experience, highest educational degree, sex and experience with rating. The Background Questionnaire appears in Appendix D. A videotape was developed to assess nurses' ability to rate performance accurately. The videotape featured a female nurse, in a hospital setting, performing work duties. Eighteen short, one to three minute, scenes which depict a variety of examples of ratee job behavior were included in the videotape. The videotape in its entirety lasts for approximately 25 mintues. The scenes in the videotape depict examples of job behaviors from each of the five job performance dimensions (Knowledge and Judgment, Organizational Ability, Skill in Human Relations, Conscientiousness, and Observational Ability) on the Smith and Kendall's BARS scale. In developing the scenes for the videotape, behavioral examples from the BARS scale were modified, and the experimenter and two nurses created scenes based on behavioral examples for each job dimension. Within each dimension, the ratee's behavior was designed to be 44 consistent in performance level, but across job dimensions, the performance level was varied. On the dimensions of Skill in Human Relations, Conscientiousness and Observational Ability, the ratee exhibited examples of good performance; on the dimension of Knowledge and Judgment, the ratee exhibited average performance; and on Organizational Ability, poor performance. The scenes were randomly ordered in the videotape. A complete copy of the script appears in Appendix E. The performance dimensions represented by each scene are indicated in Table 1. In Order to assess rating accuracy using a trait scale, it was necessary to identify the job—relevant trait dimensions depicted by the ratee in each scene. A group of "expert raters" (10 nursing students), described below, viewed each scene and identified which of six trait dimensions (Compassionate, Helpful, Proficient, Perceptive, Communicative and Efficient) were represented in the scene. The trait dimensions represented in each scene appear in Table 2. Igge Sggcgg. Following tape development, ten graduate nursing students, who were unaware of the intended performance levels of the scripts, rated the tape on the Smith and Kendall BARS scale, which is described in detail below. The nature of the study, the rating scale, and some common rating errors were explained to each rater. The 45 Table 1 BARS Performance Dimension Scene Number KJ ORG SHR CON OBS 1 * 2 * 3 * 4 I- 5 * 6 * 7 * a * * 9 * 1o * * 11 * 12 * * 13 * 14 * 15 * 16 * 17 * 1a * * Note. KJ = Knowledge and Judgment; ORG = Organizational Ability; SHR = Skill in Human Relations; CON = Conscientiousness; OBS = Observational Ability. A star indicates that at least 702 of the raters perceived the scene to represent the dimension. 46 Table 2 ElEEQEeee §eeoee Beeceeentieg Eeeb Icelt Qimeneleo Trait Dimension Scene Number COP HEL PRO PER COM EFF 1 * * * 2 * * 3 4 * * 5 a a * * 6 if «ll- 7 l- 8 * * * 9 «l- 10 «I» 4(- 11 -l- at- 12 * * * 13 * * 14 * * 15 * * a» 16 * * * 17 * a * 18 * * * * Note. COP = Compassionate; HEL = Helpful; PRO = Proficient; Perceptive; COM = Communicative; EFF = PER Efficient. A star indicates that at least 702 of the raters perceived the scene to represent the dimension. 47 nursing students served as "expert raters" and were very familiar with the performance demands of the job. Each rater viewed each scene separately, as many times as desired. Raters were instructed to provide ratings for each scene, independently, choosing to make their ratings on the dimension or dimensions they believed were represented in the scene. Originally, 25 scenes were utilized in the videotape. Scenes were eliminated if 1) less than 70% of the raters agreed upon the performance dimension(s) represented in the scene, or 2) the mean level of performance rating for the scene was inconsistent with the intended performance for a dimension and at least 40% of the raters believed the scene represented the dimension. These criteria resulted in the elimination of six scenes from the tape. Several analyses were performed in order to determine the raters' agreement as to which incidents represent which dimensions. Interrater reliabilities were computed for each job performance dimension for the 18 scenes to determine the consistency of raters' choice of dimensions for each scene. Average alphas were for Knowledge and Judgment, C><= .92; for Organizational Ability, C><= .95; for Skill in Human Relations, Ck = .95; for Conscientiousness, C>'= .73; and for Observational Ability, CW = .88. Thus, it appears that within each dimension, raters agreed as to the scenes 48 representing them. An overall measure of raters’ agreement of the assignment of scenes to dimensions was derived based on Cronbach's generalizability formula. A repeated measures analysis of variance was performed for raters choosing or not choosing a scene to represent a dimension by raters, scenes, and dimensions. Assuming scenes and dimensions are fixed and raters are random, the generalizability coefficient is .94. Thus, considerable agreement among raters exists as to dimension representation by scenes. The expert ratings were also used to confirm that the intended performance levels were adequately portrayed on tape and to develop the actual "true scores" for accuracy assessment. For each dimension, those scenes chosen by at least 70% of the raters (ten graduate nursing students) as representing the performance dimension were examined (See Table 1 for scene by dimension representation). The means and standard deviations of the performance ratings for each scene by dimension were computed. For each scene in a dimension, the standard deviations of the level of performance ratings (variables ranging from 0.0 to 2.0) across raters were quite low, ranging from .11 to .45, with means ranging from .125 to 1.93. Thus, not only were raters able to agree on the scenes which represent a dimension, there was little variation in the actual performance level rating given for each scene. As mentioned above, scenes 49 were eliminated if raters did not agree on their dimension representation or if the mean level of performance for the scene was inconsistent with the intended performance level for the dimension. Finally, to compute the "true score" performance level for each dimension, the means of those scenes defined as representing the dimension (if at least 70% of the raters agreed the scene represented the dimension) were averaged. The true scores for each dimension, based on the expert ratings, appear in Table 3. Similar procedures were followed to develop expert rating "true scores" for a Trait rating scale. A second group of ten nursing students viewed each scene in the videotape, and rated the performance of the nurse ratee, choosing to make ratings on the dimension(s) they believed were represented by the scene. The Trait scale originally contained ten trait dimensions of performance. Six of these ten dimensions were consistently agreed upon by the raters as being represented in the scenes, and thus were retained in the final form of the Trait scale. Interrater reliability analyses to assess raters’ agreement of choice of scenes to represent dimensions resulted in the following average alphas for each dimension: Compassionate, cx = .90; Helpful, Ck = .78; Proficient, Ck = .87; Perceptive, C) = .86; Communicative, CX = .74; Efficient, C) = .91. Using Cronbach's generalizability formula, the reliability is .96. Table 3 Icee Eeece Bethge ei Eeciecmeoee i9: the EBBS 50 Performance "True Score“ Dimension Mean SD Knowledge and Judgment 1.38 .189 Organizational Ability 0.19 .088 Skill in Human Relations 1.66 .223 Conscientiousness 1.59 .249 Observational Ability 1.56 .125 Note. Means and standard deviations are based on a nine- point rating scale ranging from 0.0 to 2.0 in units of 0.25. 51 Again, considerable agreement exists among raters as to dimension representation of the scenes in the videotape. True scores for the Trait dimensions were developed in the same manner as those for the BARS scale. A scene was said to represent a dimension if at least 70% of the raters indicated so (See Table 2 for scene by dimension representation). For each of these scenes by dimension, the standard deviations ranged from 0.0 to 1.19; the means ranged from 1.0 to 4.8. Thus, there was adequate agreement for the performance level rating given for each scene as the standard deviations were fairly low. The true scores for each dimension were computed by averaging the means of those scenes defined as representing the dimension (if at least 70% of the raters agreed the scene represented the dimension). The true scores for the Trait scale are indicated in Table 4. BARS Performance Rating Sgglg The BARS scale developed by Smith and Kendall (1963) was used by the nurses to rate the videotaped nurse’s performance. This instrument identifies five dimensions (Knowledge and Judgment, Organizational Ability, Skill in Human Relations, Conscientiousness, and Observational Ability). The rating form includes a separate page with a nine-point scale, ranging from 0.0 to 2.0, for each dimension. The dimension title and definition are listed at Table 4 52 Icee Seece Ratings of Eeciecmeoee fie: tbe Icelt Seele Trait "True Score" Dimension Mean SD Compassionate 4.18 .459 Helpful 4.01 .369 Proficient 3.78 .382 Perceptive 4.01 .283 Communicative 3.54 .693 Efficient 1.05 .071 Note. Means and standard deviations are based on a five point rating scale ranging from 1.0 to 5.0 in units of 1.0. 53 the top of each page; on the right are about eight behavioral examples of high, average, or low job performance. On the left of each page are general descriptions of high, average, and low job performance for that dimension. Appendix F contains the BARS scale. Igait Performance Rating Sgglg A second rating scale was developed for subject nurses to use when rating the videotaped nurse’s performance. This rating scale was trait-based. The traits used in the scale were culled from previously developed rating scales for nurses. The scale contains six trait dimensions which were consistently represented in the videotape mentioned above. Each dimension was listed with a short definitional description and a five point Likert—type scale ranging from exceptional to unsatisfactory. The Trait scale is presented in Appendix G. EceeeEEce Approval of the hospitals was obtained via a letter and follow—up phone calls before conducting this research. A letter was sent to each hospital explaining the study and asking for their participation (Appendix H contains a copy of the letter). When hospital staff indicated an interest in participating, the experimenter met with interested staff, in particular,the Director of Nursing or supervisory 54 personnel, to explain the details of the study and to establish schedules for data collection. The study required nurses to participate in a one and a half hour long session.1 Nurses were assessed in groups of three to fifty members per session depending on work scheduling. At the start of each session, a brief description of the project was given explaining that the project involved studying the processes of raters in evaluating the work performance of others. Nurses were told they would be completing several questionnaires and then would rate the videotaped performance of a nurse. Participation in the research was voluntary. Nurses signed a consent form agreeing to participate which is presented in Appendix I. Nurses were given a packet of material containing the Background Questionnaire, the Role Grid, the Behavior Grid, the BARS scale and the Trait scale. Each nurse was asked to respond to the items on the Background Questionnaire. Upon completion of the form, nurses were asked to complete the Role Grid. Nurses were instructed to look at the list of role titles presented at the top of the grid. Then, looking at the first row, nurses were asked to identify the three roles which corresponded to the three circles on the first row. For example, in row one on the grid, the circles corresponded to the roles of Social Worker, Nurse, and Pharmacist. Paricipants were instructed 55 to think about how the three roles were similar, decide upon two of the roles which they believed had something in common, and place a check mark in the two circles corresponding to the two roles chosen. Finally, they were asked to write down, on the line to the right, the way in which they perceived the two roles they chose to be similar. Nurses were instructed to repeat this process until the grid was completed and eight "constructs" had been provided in writing. Nurses were then asked to complete the Behavior Grid. Nurses were instructed to look at the first behavior listed on the right side of the grid and place check mark(s) in that row corresponding to the dimension(s) which they believed were related to that behavior. They then considered each of the behaviors, one at a time, and checked the dimension(s) to which they believed the behavior was related until each behavior in the grid was rated. A description of each dimension was also provided so that the dimension definitions were clear. When all nurses had finished these measures, they were collected by the experimenter. An explanation of the rating scales was given before viewing the videotape. Each of the rating dimensions and the behaviors contained within each dimension were described, and nurses were given time to look over the 56 scales and familiarize themselves with them. Finally, nurses viewed the videotape and rated the performance of the nurse featured in the tape using the BARS scale and then the Trait scale. The experimenter collected the rating scales, thanked the nurses for their participation, and answered any questions. Qele aoelxele neeeecee Qelegecx QElEOEéElQO Items on the Role Grid were used to determine the degree to which nurses possessed behavior or trait category systems. For each person, each of the eight written responses was coded as either a “behavior," a "trait," or "other" by the experimenter. The number of behavior constructs and the number of trait constructs were counted for each nurse to derive the behavior and trait category orientation scores. The greater the behavior score, the more behaviorally oriented the person's category system; the greater the trait score, the more trait oriented the category system of the rater. To ensure objectivity and reliability of the coding of written responses by the experimenter, an independent scorer coded two samples of Role Grids. For the first sample of 15 Role Grids, the experimenter and the independent scorer agreed on 90% of the codings of written responses as either 57 behavior, trait or other. A second random sample of 10 Role Grids was also independently scored with 89% agreement of the codings between the scorers. Due to the high level of agreement, the codings made by the experimenter were utilized in the data analyses. Qegcee 9i fleeeb te Betleg §eele A measure of the degree to which subjects dimensionalized job behaviors in a manner consistent with that of the rating scale was assessed from the Behavior Grid. Only one sector of the grid was used for this measure - the job-related behaviors corresponding to the job related dimensions. Twenty job related behaviors were presented on the grid - four behaviors for each of the five job dimensions. For each nurse, the responses to each job related behavior on the grid were given a score ranging from one to six, depending on the degree to which the response matched the BARS scale. A perfect match, a score of six, occurred for the behavior if the person placed the behavior in the same job dimension from which the behavior was taken from the BARS scale egg the s/he did not place this behavior in any other job related dimension. A score of five indicated the behavior was placed in the "correct" job dimension and in one other job dimension; four indicated "correct" placement and placement in two other job dimensions; a three 58 was given for behaviors placed in the "correct" dimension and three other job dimensions; a score of two indicated the behavior was placed in the "correct" dimension and four other job dimensions. Finally, a score of one was given if the nurse failed to place the behavior in the appropriate job dimension, indicating no match to the rating scale. The scores for each behavior were totalled to derive the degree of match score for each person. Thus, if a nurse matched the scale perfectly, he or she received a score of 120; if the nurse did not match the scale at all, s/he received a 20. The higher the total score for the person, the more s/he was able to categorize job behaviors into job dimensions in a manner consistent with the rating scale. Celegecieleg leg and Neozlee Belated Bebexiece eod Qimeeeleee Using the Behavior Grid, two different measures were derived for each nurse to determine how well subjects distinguished among job related behaviors and dimensions and non-job related behaviors and dimensions. The first measure assessed the degree to which nurses categorized the non—job related behaviors, behaviors seen on the job but irrelevant to job performance, into dimensions related to job performance. The score was derived by totalling the number of the 20 non-job related behaviors which the person placed 59 into job related dimensions. The higher the total score on this component, the more a person allowed non—job behaviors to enter into job related dimensional categories, which in turn may serve to bias ratings. A second measure for distinguishing between job and non- job behaviors and dimensions was computed to determine the extent to which a nurse categorized job behaviors into non- job related dimensions. The number of behaviors related to job performance which were placed into dimensions which were irrelevant to job performance was totalled for each person. A higher score denoted a nurse categorized job behaviors into non-job related dimensions to a greater extent than a nurse with a low score on this component. Finally, participants with a higher score on this measure may be missing important job information when rating performance; these nurses may not observe this job behavior when viewing others at work as they may not perceive the job behavior to be part of the job-related dimension. Cognitive Differentiation From the Behavior Grid, three measures of cognitive differentiation were computed to assess the extent to which different dimensions a rater used were applied differentially to behaviors. The general measure of cognitive differentiation was computed by totalling the number of check marks (or the number of times behaviors were 60 placed in dimensions) each rater placed in the grid. Nurses yielding low total scores placed few behaviors in more than one dimension; thus, they differentiated more between dimensions than those with higher total scores. In other words, those with low total scores were better able to differentiate behaviors into dimensions with little degree of overlap; and these people did not apply every dimensional category to refer to the same behaviors. The second measure of cognitive differentiation indicated the raters' ability to differentiate between dimensions and behaviors which were related to job performance. The number of check marks each nurse placed in the grid was totalled only for the behaviors which were job related in order to derive the score for job behavior cognitive differentiation. Finally, non-job behavior cognitive differentiation was assessed by totalling the number of check marks each person placed in the grid only for those behaviors irrelevant to job performance. These separate measures were computed to determine if differences in cognitive differentiation exist for job versus non-job related behaviors. eeeuceex A total of four accuracy measures, two for the BARS scale and two for the Trait scale, were calculated for each 61 rater. Cronbach's (1955) component of overall accuracy was used to assess the accuracy with which nurses rated the videotaped nurse's performance. Overall accuracy referred to the squared difference between the rated and true scores summed over all dimensions. True scores were from the "expert raters" and the observed scores were those provided by nurses on the rating scales. Two measures of overall accuracy were computed for each rater: accuracy using the BARS scale (BOA) and accuracy when using the Trait scale (TOA). Lower overall accuracy scores indicated greater accuracy. Accuracy was also assessed using a correlational accuracy measure which was the correlation between the true scores and the observed scores. For the BARS, correlational accuracy (BCA) was calculated by correlating the observed scores with the true scores across the dimensions for each nurse. The same method was used to calculate correlational accuracy for the Trait Scale (TCA). Here, higher correlational accuracy scores indicated greater accuracy in terms of the pattern of performance levels across dimensions for the ratee. Belg Two measures of halo were computed for each rater, one for the BARS scale and one for the Trait scale. Halo was assessed as the standard deviation of the ratings, across 62 rating dimensions, provided by each nurse. lee Eeeecleoee YecleElee Measures from the Background Questionnaire were used to determine possible correlates of category systems. Specifically, the number of years of nursing experience was derived; job position of the nurse was assessed as either staff nurse, head nurse, charge nurse, supervisor, or other; job title was assessed as either Licensed Practical Nurse, Registered Nurse, Nurse Practioner, or other. Nurses were classified according to the unit in the hospital in which they work. The type of educational training was determined as either community college, hospital, or college; and the highest educational degree received was reported. Nurses were classified into groups based upon whether or not they had previous rating experience of nurses and upon the number of years of previous rating experience. Finally, nurses were grouped into categories based on sex. RESULTS This section is divided into three major parts. First discussed are the two sets of variables of interest, a) the accuracy measures and b) the cognitive processing indices. Secondly, results for each of the hypotheses presented earlier are discussed, in turn. Finally, additional findings, not hypothesized a priori, are presented. Beeeceex neeeecee The means, standard deviations and intercorrelations for the nurses’ accuracy scores - BARS overall accuracy (BOA), BARS correlational accuracy (BCA), Trait overall accuracy (TOA) and Trait correlational accuracy (TCA) — are presented in Table 5. The two BARS accuracy measures and the two Trait accuracy measures are highly intercorrelated (E = —.70 and g = -.61, respectively). Small to moderate intercorrelations were found between BARS and Trait accuracy scores (5’s ranged from —.18 to .47). To test the relative accuracies of the BARS versus the Trait scale, it was first necessary to standardize the scale scores. The observed scores and the true scores for each dimension on each scale were transformed to z-scores before computing overall accuracy scores. Mean comparisons using t—tests revealed significant differences in accuracy when using the BARS scale versus the Trait scale for overall 63 64 means. §Qe eOe Intercorrelatieee ei eeeeceex neeeucee Variable Mean SD (1) (2) (3) (4) 1. BOA 8.74 8.57 2. BCA 0.94 0.25 -.70 3. TOA 10.92 6.16 .47 -.18 4. TCA 0.95 0.24 -.14 .20 -.61 Note. Means and standard deviations for BOA and TOA were computed for z-scores of these variables; for BCA and TCA, means and standard deviations were computed after Fisher’s r-to-z transformation of the scores. BOA = overall accuracy (sum of squared differences between observed and true scores) for BARS scale; BCA = correlational accuracy (correlation between observed and true scores) for BARS scale; TOA = overall accuracy for Trait scale; TCA = correlational accuracy for Trait scale. 65 accuracy. Nurses were significantly more accurate, measured by overall accuracy, with the BARS scale than with the Trait scale (§(1,125) = 2.82, g < .01). In addition, the correlational accuracy score for each scale was transformed using Fischer's r-to-z transformation and a t-test was computed between the two means. No significant mean differences in accuracy were found between the BARS and the Trait scale for correlational accuracy scales (t(1,123) = 0.6, Q = .5). Thus, it appears that participants were more accurate in discerning performance levels across dimensions when using the BARS scale than when using the Trait scale, but no difference in accuracy existed in nurses' ability to reflect the pattern of performance levels across dimensions when using the BARS or Trait scale. As mentioned earlier, the two measures of BARS accuracy, BOA and BCA were highly intercorrelated (; = -.70). Moreover, the pattern of correlations of these two measures with the cognitive measures was very similar. Thus, the two measures appeared to be assessing raters' accuracy using the BARS scale in a similar fashion; although, the BARS correlational accuracy measure appeared to be more sensitive, indicated by a greater number of significant correlations with the cognitive measures, shown later. One-way analyses of variance were performed for each of the accuracy measures by hospital groups to ensure the data 66 could be collapsed across the hospitals from which it was drawn. Results of these analyses, shown in Table 6, indicated no difference by hospital grouping for three of the measures of accuracy, BOA, BCA and TCA. However, there were significant differences in accuracy for TOA by hospital group. Closer examination of these data using Newman-Kuel's tests revealed that the difference was due to hospital one in which accuracy scores were significantly lower than hospitals two and three. Degeitlxe beeeecee The means, standard deviations and intercorrelations for the cognitive processing indices of raters, measured by the Role Grid and the Behavior Grid, appear in Table 7. Specifically, the cognitive measures include: 1) the behavioral orientation of the raters' category system, measured by the number of behavioral constructs elicited on the Role Grid (BEH); 2) the trait orientation of the raters' category system, measured by the number of trait constructs elicited on the Role Grid (TRAIT); 3) the degree to which raters matched the BARS scale in the assignment of job behaviors to job dimensions (MATCH); Beeeitel 9:922 67 Accuracy Measure BARS BARS Trait Trait Dependent Overall Correlational Overall Correlational Variable (BOA) (BCA) (TOA) (TCA) Effect df F F F F * Hospital 2 0.63 0.36 3.10 1.48 Groups Subjects 121 (.35) (.03) (3.63) (.02) within groups Note. Numbers in parentheses are the mean square error asssociated with the F tests. *- p < .05. 6 8 deeoei §Qe EOE loteceecceletleoe ei Qegollixe beseecee Variable Mean SD (1) (2) (3) (4) (6) (7) (8) 1. BEH 4.40 2. 18 ** 2. TRAIT 3.04 2.12 -.93 3. MATCH 89.08 16.35 .05 .09 a ** ** 4. NJJ 5.43 4.90 -.14 .19 .60 * ** ** 5. JNJ 2.48 3.02 -.10 .13 .57 .73 i ** ** ** 6. COG 76.67 36.21 -.11 .13 .69 .85 ** ** ** ** 7. JCOG 44.26 21.35 -.06 .10 .72 .80 .97 ** ** ** ** ** ** ** 8. NJCOG 32.51 16.19 -.16 .18 .60 .84 .95 .86 Note. The BEH and TRAIT intercorrelation is based on ipsative data. For a complete description of each variable see pages 66 and 69. * p .07; p < .05, for one-tailed tests. 69 4) the extent to which behaviors irrelevant to job performance were included in the raters' dimensional category system for the the job (NJJ); 5) the extent to which job related behaviors were included in the raters' non-job related dimensional category system (JNJ); 6) the overall cognitive differentiation of behaviors and dimensions (COG); 7) the cognitive differentiation for job related behaviors (JCOG); 8) the cognitive differentiation for behaviors observed on the job, but unrelated to job performance (NJCOG). For the most part, the cognitive measures are highly intercorrelated. The two measures from the Role Grid which assess raters' category orientation, BEH and TRAIT, are not independent and are ipsative data; thus, their intercorrelation was, as expected, quite high (5 = -.93). Note, the BEH and TRAIT intercorrelation was not 1.0 as some responses were coded as "other" representing neither the behavior nor trait category. The BEH and TRAIT measures, however, revealed fairly low intercorrelations with the remaining cognitive measures (5’s ranged from .06 to .19). The six cognitive measures derived from the Behavior Grid - MATCH, NJJ, JNJ, COG, JCOG, and =' NJCOG - were all highly intercorrelated (5's ranged from .57 70 to .97). These results suggest that the two grids are measuring two separate constructs. The Role Grid measures category orientation of the raters while the Behavior Grid assesses the categorizing of behaviors and dimensions for the raters cognitive category system. It is also possible that method bias may have served to produce the high intercorrelations within the Behavior Grid. To ensure that no differences in the cognitive processing indices of raters existed based on hospital groups, analyses of variance were performed for each of the cognitive measures by hospital groupings. Results indicated no significant differences for any of the cognitive assessments based on the hospital groups. Bxeetbeeee Qetegecx cheotelieo Hypothesis one stated that raters will yield more accurate ratings to the extent that their category orientation, behavior or trait based, corresponds to the rating scale, behaviorally or trait-anchored scales. Correlations between BEH, TRAIT and the four accuracy measures were computed to test this hypothesis and are presented in Table 8. Specifically, for each nurse, the number of behavioral constructs elicited in the Role Grid was correlated with the 71 eeeeceex Accuracy Measure BARS BARS Trait Trait Category Overall Correlational Overall Correlational Orientation (BOA) (BCA) (TOA) (TCA) BEH -.02 .07 .07 .06 TRAIT .05 -.06 -.05 -.10 Note. BEH = number of behavioral constructs in category system; TRAIT = number of trait constructs in category system. 72 four accuracy measures, BARS and Trait scale overall and correlational accuracy, and the number of trait constructs elicited in the grid was correlated with the four accuracy measures. No significant correlations resulted from these analyses indicating that the raters' cognitive orientation, as measured in this study, had little relationship to accuracy in rating performance. Both the behavioral and trait measures in Table 8 were treated as continua; yet, it was possible that some nurses had category systems that were equally behavior or trait in their orientation. Since the hypothesis was based on a more extreme either-or condition, three subgroups were formed. Raters' cognitive systems were classified as: 1) behaviorally based - if 75% of their responses were behavior constructs; 2) trait—based - if 75% of their responses were trait constructs; and 3) mixed - if the majority of their responses were both behaviors and traits. Four separate one-way analyses of variance were conducted for each of the four accuracy measures by the category orientation classification identified above. Results of these analyses are depicted in Table 9. Again, no support was found for the effect of raters category orientation on accuracy in ratings. 73 Accuracy Measure BARS BARS Trait Trait Dependent Overall Correlational Overall Correlational Variable (BOA) (BCA) (TOA) (TCA) Effect df F F F F Category 2 0.80 1.94 0.55 1.82 Groups Subjects 110 (.33) (.02) (3.85) (.02) within groups Note. Levels of Category Orientation are 1) behaviorally- oriented category system; 2) trait-oriented category system; and 3) mixed category orientation. Numbers in parentheses are the mean square error associated with the F tests. 74 Dimensionalizing Eebexlece The remaining cognitive indices of raters were derived from the Behavior Grid and dealt with differences in the categorizing of behaviors and dimensions in relation to rating accuracy. These measures were based on the BARS scales - MATCH, NJJ, JNJ, COG, JCOG and NJCOG - hence only the BARS accuracy measures, BOA and BCA, were the appropriate accuracy measures to consider. Table 10 presents the correlations between the cognitive measures and the BARS accuracy measures. Oxeetbeeie 3 - Qegcee ei meteb t9 Eating Seale- Hypothesis two predicted that accuracy in ratings would be greater to the extent that raters are able to dimensionalize job behaviors in a manner consistent with the rating scale. In support of this hypothesis, correlations between MATCH and both BARS accuracy measures, BOA and BCA, were significant and in the predicted direction. There were significant correlations between the degree to which raters dimensionalized job behaviors in a manner consistent with the job dimensions and behaviors on the rating scale and accuracy in ratings using the BARS scale. Neozlee Belexeot Eebexlece eOe Qimeoelene- Hypothesis three stated that accuracy in ratings will be related to the degree to which a rater is able to distinguish between 75 Accuracy Measure BARS BARS Overall Correlational (BOA) (BCA) Cognitive Measure E E ** ** MATCH -.22 .27 NJJ .10 -.10 JNJ .10 -.06 * COG .09 -.13 JCOG .07 -.08 ** NJCOG .10 -.18 Note. For a complete description of each variable, see pages 66 and 69. * ** p < .10; p < .05, for one-tailed tests. 76 behaviors and dimensions which are relevant to job performance and those that are irrelevant to job performance. Small, but non-significant correlations, were indicated for BOA and BCA with NJJ (see Table 10). Consistent with Hypothesis three, the extent to which job relevant behaviors were categorized into dimensions unrelated to job performance (JNJ) should have been related to the two BARS accuracy scores (See Table 10). No significant correlations resulted from these analyses. Here, the accuracy of ratings was not related to the degree to which raters "miscategorized" job related behaviors into dimensions irrelevant to job performance. Bxeetbeele & - Cognitive Diffeceotletieo- Hypothesis four posited that raters who are more accurate in their ratings will have more highly differentiated category systems for the job such that little overlap exists between category dimensions while raters who provide less accurate ratings will be unable to differentiate clearly between dimensions. Cognitive differentiation assessed the extent to which different dimensions the rater used were applied differentially to behaviors. To test this hypothesis, the three measures of cognitive differentiation were derived - 1) overall differentiation for both behaviors and dimensions relevant and irrelevant to job performance (COG); 2) differentiation for the behaviors related to job 77 performance (JCOG); and 3) differentiation for behaviors observed on the job, but irrelevant to job performance (NJCOG) - and each were correlated with the two BARS accuracy measures (See Table 10). High scores on the cognitive differentiation measures indicated less differentiation between behaviors and dimensions. Correlational results showed that overall cognitive differentiation was marginally significant and positively related to accuracy in terms of the pattern of ratings (BCA). The more a person differentiated between behaviors and dimensions, the more accurate his/her ratings. Interestingly, this effect was dependent on the type of behavior dimensionalized by the rater. There were no significant correlations when differentiation was assessed for behaviors related to job performance (JCOG); however, when subjects dimensionalized behaviors seen on the job but unrelated to job performance (NJCOG), a signficant correlation resulted for correlational accuracy using the BARS scale. This finding suggests that the better the rater was able to dimensionalize non-job related behaviors into dimensions with little degree of overlap, the more accurate his/her ratings. It is also interesting to note that there were no significant correlations found between any of the above cognitive measures of dimensionalizing behaviors and/or 78 match to the rating scale and either of the Trait accuracy measures (E’s ranged from .01 to .10). As expected, cognitive processes of raters assessed in the manners mentioned above appear unrelated to accuracy in ratings when using a trait-based rating scale; the cognitive processing indices of raters focused on behaviors rather than traits. Hypothesis five proposed that raters with more highly differentiated category systems for the job will exhibit less halo in their ratings than those with less differentiated systems. The three measures of cognitive differentiation (COG, JCOG, and NJCOG) were each correlated with the two measures of halo - halo for the BARS scale and halo for the Trait scale. Results of these analyses are reported in Table 11. Significant correlations were found between each of the cognitive differentiation measures and each of the halo measures. Thus, raters who do not clearly differentiate between dimensions in their category systems exhibit more halo in their ratings. EEOECiEheee oi the Bate: Hypothesis six stated that experiences of the rater will be correlated with the category system s/he utilizes in evaluating the job performance of others and with rating 79 Table 11 Qgggglgtions for Cognitive Differentiation gy Halg Cognitive Diffrentiation Halo COG JCOG NJCOG . * a * Halo -.19 -.18 -.19 BARS Scale * * a Halo -.19 -.18 -.19 Trait Scale * Note. p < .05, for one-tailed tests. 80 accuracy. Raters were asked to report their sex, the number of years they had worked as a nurse (WORK), their job position (POS), job title, unit in the hospital in which they worked (UNIT), where they received their primary education (SCHOOL), the highest educational degree received (DEGREE), whether or not they had previous experience rating nurses' performance (RATING), and the number of years of prior rating experience (YEARS). Appendix J presents the percent of nurses falling in each category for each of the experience variables. Because 97% of the nurses were female and because 91% were Registered Nurses, no analyses were performed based on sex of the nurse or job title. Correlations were computed between each of the experience variables (except UNIT) and each of the cognitive measures for the raters and are reported in Table 12. Only three variables related to prior experiences of the rater were significantly correlated with the cognitive processes of the rater. Specifically, the number of years worked as a nurse (WORK) was negatively correlated with the degree to which nurses dimensionalized behaviors in a manner consistent with the rating scale (MATCH). The job position of the nurse (POS) was significantly related to several cognitive processing variables. The higher the job position of the nurse, the less likely was the nurse to "miscategorize" behaviors by placing non-job related 81 Cognitive Variable Experience BEH TRAIT MATCH NJJ JNJ COG JCOG NJCOG ** WORK .00 .00 -.16 -.01 .07 -.01 -.03 .00 ** ** * POSITION .08 -.03 -.04 -.19 -.20 -.14 -.12 —.15 * SCHOOL —.09 .13 -.05 .07 —.04 .02 .07 —.04 DEGREE -.08 .11 -.01 .04 -.03 .07 .10 .03 RATING .00 —.01 -.03 -.08 -.09 —.09 -.08 -.10 YEARS -.01 .01 -.02 .00 .05 .03 .02 .03 Note. WORK = number of years worked as nurse; SCHOOL = type of educational training based on years required; DEGREE = educational degree; RATING = prior rating experience; YEARS {- p < .10 number of years of rating experience. ** ; p < .05, for one tailed tests. 82 behaviors into job dimensions (NJJ) or job behaviors in non- job related dimensions (JNJ). Marginally significant correlations were found between job position and the cognitive differentiation measures of COG and NJCOG, such that the higher the job position, the more differentiated the raters' category system. Finally, the type of educational experience of the rater (SCHOOL) yielded a marginally significant correlation with the number of trait constructs elicited by the rater (TRAIT). Nurses in programs which required more years of schooling before receiving a degree produced more trait constructs in their category systems. Additional correlational analyses were performed to determine if any of the prior experiences of the rater were related to rating accuracy. Results of these analyses are presented in Table 13. Only prior rating experience (RATING) was signficantly and positively correlated with any of the accuracy measures. Raters who had prior experience in rating nursing performance were more accurate in their ratings using the BARS scale (BOA and BCA) than those without such prior rating experience. No significant results emerged for experiences of the rater in relation to rating accuracy using the Trait scale. A one-way analysis of variance was also performed to determine if differences in rating accuracy were related to Table 13 Accuracy Measure BARS BARS Trait Trait Experience Overall Correlational Overall Correlational Variable (BOA) (BCA) (TOA) (TCA) WORK .06 .03 .11 -.01 POSITION -.02 -.04 .06 -.03 SCHOOL -.09 .05 .03 -.02 DEGREE -.02 .03 .02 .07 * * RATING .15 -.17 .10 -.04 YEARS —.06 .10 -.01 .06 Note. WORK = number of years worked as nurse; SCHOOL = type of educational training based on years required; DEGREE = educational degree; RATING = prior rating experience; YEARS = number of years of rating experience. * p < .05. 84 the unit in the hospital in which the nurse worked. Results indicated that differences in accuracy, measured as BCA, existed by hospital unit (E(7,116)=3.57, p (.01). Closer examination of the data revealed that nurses working in the surgery unit in the hospital were less accurate in their ratings for BARS correlational accuracy than those persons in any other hospital unit. No other differences by hospital unit were found for any of the other accuracy measures. Overall, it appears that the variables of years worked as a nurse and job position are the important variables to consider for the cognitive processing of raters, while prior rating experience is important for rating accuracy using the BARS scale. The following results do not bear directly on any hypothesis. However, these data add insight for understanding some of the cognitive processes involved in the rating process. Qateaecx QCiEhtatieh aha Categecx theheionalization While category orientation, behavior or trait based category systems, was not found to be related to rating accuracy, there were some significant correlations between the behavior or trait scores and the cognitive indices 85 measuring the categorization of behaviors and dimensions (See Table 7). Specifically, scores for both BEH and TRAIT were signficantly correlated with NJCOG, the cognitive differentiation for behaviors seen on the job but irrelevant to job performance. Raters with poor cognitive differentiation were more likely to possess trait based constructs in their category systems than those who were better able to differentiate between dimensions; nurses with better ablility to differentiate between dimensions were more likely to elicit more behavioral constructs than those poor in differentiation. The number of non-job related behaviors placed in job related dimensions (NJJ) revealed a marginally significant correlation with BEH and a significant correlation with TRAIT. Here, raters who allowed more non-job related behaviors into job related category dimensions were less likely to elicit behavioral contructs in their category systems than those who placed few non-job related behaviors into job dimensions. Similarly, higher TRAIT scores were related to placing more non-job related behaviors into job dimensions. Finally, the number of trait constructs elicted produced marginally significant correlations with overall cognitive differentiation (COG) and with the number of job related behaviors placed in non- job related dimensions (JNJ). The less cognitive differentiation between dimensions was 86 related to a greater number of traits in nurses' category systems. The greater the number of job related behaviors placed in non-job dimensions, the greater the number of traits in raters' category systems. Overall, category orientation, behavior or trait based, was related to many of the subjects' categorizations of behaviors and dimensions. Significant correlations were evidenced between category orientation scores and some of the category dimensionalizing scores. More specifically, BEH was related to NJJ and NJCOG; TRAIT was related to NJJ, JNJ, COG, NJCOG. Significant correlations were also found between some of the category dimensionalizing measures and the accuracy measures mentioned above. Specifically, those of MATCH, NJJ, COG, and NJCOG were related to BCA. However, no significant relations were found between category orientation and accuracy in ratings. Thus, it appears that a causal linkage between cognitive categorization processes of the raters, raters’ category orientation and accuracy in ratings is a likely explanation for these results. This relationship needs more empirical support through causal modeling methods. DISCUSSION This study responds to recent suggestions that the cognitive processes of raters need investigation in order to fully understand the performance appraisal process (Borman, 1978; Feldman, 1981; Ilgen & Feldman, 1983; Landy & Farr, 1980). From a pragmatic standpoint, the relationship of cognitive processes of raters to the degree to which raters important concern. More specifically, this study focused on raters’ general category orientation as behaviorally or trait based, several indices of raters' categorization of job relevant and job irrelevant behaviors, and on raters’ cognitive differentiation of dimensions as they relate to rating accuracy. To some extent, results supported the notion that cognitive categorization processes of raters are related to accuracy in ratings. Also investigated were the prior experiences of raters in relation to their cognitive processes and rating accuracy. Results indicated that years on the job and job position were related to the cognitive categorization processes of raters while prior rating experience was related to rating accuracy. These results, their implications and suggestions for future research are discussed in detail below. 87 88 Eategecx QElEhaaaleh In general, the performance appraisal literature contends that raters should provide more accurate ratings to the extent that their personal category systems match that of the rating forms used in appraising performance. Specifically, it was hypothesized that raters possessing behaviorally oriented category systems would yield more accurate ratings using a behaviorally based scale while raters with trait oriented category systems would be more accurate with a trait based scale. Results from this study generally failed to support this view. Failure to find support for the relationship between trait category orientation and accuracy using a trait scale may be due to the fact that, in general, raters were less accurate with the trait scale than with the behavior scale. Traits tend to focus on more general overall impressions, rather than specific information about the ratee. The formation of overall impressions when using the Trait scale may inhibit making accurate ratings. Further, the videotape developed for the study focused on specific behavioral examples, rather than traits, of the ratee. Thus, raters may have had more difficulty making trait ratings. In addition, raters always rated on the BARS scale first which may have either focused raters attention to behaviors or served as a response set bias. 89 Qeahltixe Categorization Three cognitive categorization processes of raters were investigated in the present study: 1) the degree to which raters dimensionalized behaviors in a manner consistent with the rating scale; 2) the "miscategorization" of behaviors unrelated to job performance into job performance dimensions; and 3) the "miscategorization" of behaviors related to job performance into dimensional categories irrelevant to job performance. Qimggsionglizing. Results supported the notion that raters' ability to dimensionalize behaviors in a manner consistent with the rating scale is related to accuracy in rating using that scale. That is, when raters were given a list of job behaviors from a BARS scale and asked to categorize them into the job dimensions from the BARS scale, the more their dimensionalization of behaviors matched those of the BARS scale, the more accurate were their performance ratings. This finding held for both accuracy measures used in the study: 1) overall accuracy indicating the ability to accurately report the ratee's level of performance and 2) correlational accuracy indicating the ability to accurately reflect the pattern of performance levels for the ratee. Behaviorally anchored performance rating forms, such as the BARS scale used in this study, require the rater to incorporate behavioral information into category systems in 90 the appraisal process. If the rater's category system matches the category system of the rating scale, the rater need not transpose his category system to make ratings using a rating scale with a different categorization of behaviors and dimensions. Thus, it follows that when the rater’s category system matches the rating scale, the process of making ratings will be easier and ratings will be more accurate. This finding is not surprising when viewed in light of the assumptions made in the training of raters to yield more accurate ratings. One approach to rater training has been to train raters on the definitions of dimensions and on the behaviors contained within each dimension presented on the rating scale. In this way, raters are given a common "frame of reference" by which to evaluate workers using the scales (Bernardin & Pence, 1980; Pulakos, 1984, 1985). The underlying assumption is that rating accuracy is increased by training individuals in a manner that is consistent with the cognitive demands required by the particular rating format (Pulakos, 1985). Hence, once raters are trained to use category systems which match the rating scales, ratings are more accurate. The present findings directly support this notion; the more raters' category systems matched the BARS rating scale, the more accurate their ratings. 91 fliggétggorization. The hypothesized relationship of the "miscategorization" of non-job related behaviors into job dimensions and job behaviors into non-job related dimensions to rating accuracy received considerably less support in this study. Results indicated only small relationships between the number of behaviors related to job performance which were categorized into non-job related dimensions and rating accuracy. No other significant relationships resulted. There are several reasons why this hypothesis failed to recieve support for these notions. First, the videotape of the nurse which nurses observed in order to make their performance ratings was specifically designed to include examples of job related behaviors. No effort was made to include behaviors which were not related to job performance or to include examples representative of non-performance related dimensional categories. Therefore, even though raters may have had category systems which included behaviors and dimensions unrelated to job performance, there were no explicit examples of such depicted by the ratee. Raters with such category systems may have been able to accurately rate the peformance of the ratee as no or few non-job related behavioral examples were exhibited to interfere with the rating process. Secondly, the measure used to assess raters' 92 "miscategorization" of non-job related behaviors and dimensions may not have adequately tapped this process, Looking at the means for these variables, few non-job related behaviors were miscategorized into job dimensions and very few job behaviors were miscategorized into non—job related dimensions. Perhaps the distinction between the job relevant and non-job relevant behaviors and dimensions was too salient in this measure. For example, raters categorizing the job behavior "would expect this nurse to give meticulous back care to those patients for whom it was ordered" and the non-job related behavior "would expect this nurse to come to work dressed sloppily" may have had no trouble distinguishing between the job relevant and job irrelvant behaviors as the distinction is fairly obvious. This poses a problem for measurement development to assess this "miscategorization" process of non-performance related behaviors and dimensions. The non-job related behaviors and dimensions chosen reflected high levels of rater agreement as to their dimensional representation. It may be difficult to find non-performance related behaviors and dimensions which yield a less salient distinction from job related behaviors and dimensions and still maintain high levels of dimension agreement. These categorization processes are important considerations for rating accuracy. Several authors have 93 contended that non-performance related characteristics and behaviors of the ratee (i.e. sex, race, etc.) are observed and included in raters' category systems (Ilgen & Feldman, 1983; Landy & Farr, 1980). Perhaps better indices to evaluate ratees and/or better measures to assess this categorizaton of job relevant and irrelevant behaviors and dimensions would enhance empirical support for this categorization process in relation to rating accuracy. Cognitive Diffecehtlatleh A great deal of controversy exists in the literature as to the importance of cognitive complexity for rating accuracy and for rating errors (Bernardin, et al., 1982; Lahey & Saal, 1981; Schneier, 1977). One aspect of cognitive complexity, cognitive differentiation (the extent to which different constructs the rater uses are applied differentially to elements), was assessed in this study. Results indicated that raters higher in cognitive differentiation were somewhat more accurate in their ratings (for correlational accuracy) and exhibited less halo in their ratings. Although cognitive differentiation was measured for both job relevant and non-job performance related behaviors, the behaviors were specific to the situation of nurses’ performance on the job, lending some support to Bernardin et al.’s (1982) notion that cognitive complexity specific to the situation may be more useful for 94 examining rating accuracy and halo. The relationship between cognitive differentiation and halo in ratings was consistently supported in this study. Overall cognitive differentiation and cognitive differentiation for both job and non-job related behaviors revealed significant relationships, in the predicted direction, to halo in ratings for both the BARS scale and the Trait scale. When raters are unable to differentiate clearly between dimensions, they may form an overall impression of the ratee which is reflected by halo in their ratings. Interestingly, the relationship of cognitive differentiation to rating accuracy appeared to depend on the type of behaviors dimensionalized. Cognitive differentiation for job-related behaviors showed little relation to rating accuracy; yet, for non-job related behaviors, more differentiation was significantly related to greater accuracy in ratings. One possible explanation for this finding is that raters unable to perceive non-job related behaviors in a multidimensional manner may be integrating behaviors into an overall general impression that serves to bias ratings (Feldman, 1981). Although the relationship between cognitive differentiation and rating accuracy and halo is not elucidated much by this study, these results are consistent 95 with other findings suggesting that cognitive differentiation specific to the situation is more promising than general cognitive complexity (Bernardin, et al., 1982). Moreover, it would be interesting to investigate the relationship of cognitive differentiation to alternative information processing models. Nathan and Lord (1983) investigated Borman's (1978) model in which observed behaviors are integrated into dimensional schemata and Feldman's (1981) model in which behaviors are integrated into a general impression that serves as the basis for performance ratings. Results supported both theories such that subjects were able to distinguish between performance dimensions, however a large halo effect was found resulting from a strong general impression. Examination of these cognitive processing notions with regard to cognitive differentiation specific to the situation may help elucidate the rating process further. It may be that raters unable to differentiate well between dimensions form more overall general impressions, whereas raters who cognitively differentiate are better able to integrate behaviors into dimensional schemata. It is also important to note that support for this hypothesis was found only with the correlational accuracy measure. Correlational accuracy reflects the ability to discern the pattern of performance across all dimensions, 96 perhaps reflecting more discrimination between dimensions than the overall accuracy measure. It may be that the overall accuracy measure was not sensitive to the differentiation process of raters. Eaeecieheee oi the Bate: Experiences and education the rater has may have important influences on their category development (Rosch, et al., 1976). In the present study, results indicated a negative relationship between the number of years of job experience and the ability to dimensionalize behaviors in a manner consistent with the rating scale. This finding is seemingly incongruent with what one might expect, however it is not unreasonable. As workers gain experience on the job, they may formulate their own personal category systems of job dimensions and job performance. The more time the rater spends on the job the more "rooted" this category system becomes for the person and the less likely s/he would automatically adopt a new category system. Further, in this study, the rating form used was not the one used by the hospital. Raters with several years of job experience may have formulated category systems specific to the rating form used by the hospital, while raters newer on the job may not have done so yet. Hence, raters with less job experience may be more able to dimensionalize behaviors in a manner 97 consistent with the rating form imposed on them in the study as their category systems are less formalized to a specific rating scale. It is also possible that experience of the rater results in a more highly differentiated category system (Rosch, et al., 1976). Yet, the BARS scale has only five dimensions. Raters with more highly differentiated category systems may have difficulty in applying their category systems to a rating scale with only five dimensions. Further, the procedures for development of a "good" BARS scale mitigates against differentiation. That is, only those dimensions and behaviors which everyone agrees upon are retained in the BARS scale. This "agreement" leads to a common, very usable set of dimensions, rather than allowing for complex highly differentiated systems to be integrated into the scale. Again, experienced raters who develop highly differentiated systems may have difficulty in applying their cognitive system to the rating format. Job position of the raters was related to several cognitive categorization processes in this study. Raters in higher job positions "miscategorized" fewer behaviors and also differentiated more between dimensions. The higher job positions may influence raters to develop different category systems, attend to different aspects of job performance and/or enable them to better distinguish between performance 98 and non-performance related dimensions. Surprisingly, no other experiences of the rater (schooling, educational degree or prior rating experience) were related to cognitive processes of the rater. Apparently, these type of experiences have little impact on the cognitive processes raters use in evaluating performance. It is also possible that the measure used here! was not sensitive enough to delineate these relationships. Another interesting finding was that the nurses’ prior rating experience was related to rating accuracy using the BARS scale. Nurses who had previous rating experience were more accurate in their ratings than those with no prior experience. Perhaps, simple practice in making ratings enchances accuracy . Qategecx chehtatiah aha Categorization Interestingly, the category orientation of raters was related to the cognitive categorization processes of raters which, in turn, were related to rating accuracy. There is some supportive evidence to suggest that a causal linkage exists between the cognitive orientation of raters, the categorization processes of raters, and rating accuracy, at least for behaviorally anchored scales. While this was not hypothesized a priori, it seems reasonable. Raters with more behaviorally oriented category systems "miscategorized" non-job related behaviors into job dimensions to a lesser 99 extent, while those with more trait oriented systems allowed non-job behaviors into their job dimensional categories to a greater extent. Further, to some degree, trait oriented people also miscategorized job behaviors into non-job dimensions to a greater extent than those with less trait orientation. The behavioral orientation of raters' category systems may serve to focus their attention on behaviors allowing for better distinction between job relevant and irrelevant behaviors and dimensions, while trait oriented people are unable to do so as their attention is focused on more general traits. Moreover, similar relationships were found between behavior and trait category orientation and cognitive differentiation such that the more trait oriented, the less differentiation and the more behaviorally oriented, the more differentiation. Again, raters who are more trait oriented may form more generalized overall impressions and hence differentiate to a lesser degree than behaviorally oriented people. BBBS xeceee Icalt eeeecaex Although this study was not designed to examine the relative rating accuracies when using behaviorally anchored or trait anchored scles, results indicated that raters were more accurate in determining the ratee's level of 100 performance when using a BARS scale than when using a Trait scale. This finding reinforces the contentions of several authors (e.g. Carroll & Schneier, 1982) that behaviorally anchored scales are superior to adjective or trait anchored scales. However, there were no differences in accuracy with different formats in relation to the pattern of performance levels. Raters were able to discern patterns of performance levels equally well with the two formats, but were better at determining the level of performance with a BARS scale. tlhitatlghe at the Beeeaceh Although, in general, results from this study supported the hypothesis that cognitive processes of the rater influence rating accuracy, there are limitations to the research that should be considered in drawing conclusions from this data. Many of the correlations supporting the hypotheses were low, albeit significant. This may have been due, in part, to the inability of the measures to adequately tap the constructs of raters' cognitive processes. In addition, only a few of the large range of potential cognitive processing influences to rating accuracy were examined in this study. For example, observational processes of raters such as which behaviors raters focused on and recall processes such as which behaviors were recalled or how raters arrived at their ratings during recall were not examined in this study. While the cognitive 101 categorization processes examined in this study were important in relation to accuracy, there are yet a great number of other potentially important cognitive processes to consider. Further, only one component (cognitive processes of the rater) of the entire appraisal process was examined in this study. The use of a videotaped ratee precluded consideration of many factors that may influence performance appraisals in the “real world". Important components of the appraisal process such as ratee characteristics, rater/ratee interaction, environmental factors of the work setting (Ilgen & Favero, 1985) or purpose of rating (Zedeck & Cascio, 1982), all of which may affect the judgmental processes of the rater, were not able to be investigated in this study. Additional problems are also concerned with the videotape used in this study. As noted earlier, the videotape focused specifically on job relevant behaviors and did not include the full range of job relevant and job irrelevant behaviors observed at work. Moreover, only one ratee was used in this study due to time limitations. Additional ratings for several ratees may provide a better indication of raters' accuracy. It would also be interesting to vary some ratee characteristics such as sex and/or race to determine their influence on rating accuarcy 102 (Feldman, 1981; Ilgen & Feldman, 1983). In general, it also appeared that the behaviors and performance levels exhibited by the ratee were very salient making the rating task easier for the raters. Raters were, for the most part, fairly accurate in their ratings and expert raters commented the the task was a fairly easy one. Use of the videotape with ratings made immediately following observation precluded any rating errors due to memory decay of the raters. It has been demonstrated that raters are less able to recall specific behaviors over time (Heneman & Wexley, 1983). As no time lag was incorporated between observation and rating, accuaracy could be maximized, thereby reducing variation in the accuracy measures. Another potential limitation concerns the rating scale used. This study, or other studies examining rating accuracy, did not investigate the reliability of the accuracy measures derived from ratings provided on the scales. It is possible that the ratings providied were unreliable which may have contributed to the low correlations found in this study. Further, although the study used "real world” subjects, the rating forms used were not those used by the hospitals. Rather, each subject used the standardized BARS scale as it was applicable to all hospitals and allowed for greater control in the study. There is the possibility that the cognitive indices measured 103 here and the accuracy using the BARS scale are specific to these measures and not generalizable to other formats. In addition, in assessing the cognitive categorization processes of the raters, the category dimensions were imposed on the rater. Imposing the category dimensions on raters, rather than assessing raters' personal constructs in a free—format manner (Borman, 1983), may limit the assessment of raters' cognitive processes. However, determining the nature and content of raters' category systems in a free-response mode may inhibit the determination of reliability and validity of these constructs. Batman and Estate Qiceetlahe This study could be replicated and extended using variants of the cognitive categorization measures, including more behaviors and/or ratees in the videotape and possibily utilizing formats specific to the particular field setting. Other components of the raters’ cognitive processes or the appraisal process in general could also be examined to gain further understanding. With the delineation of the cognitive categories of raters in relation to rating accuracy, the next logical step would be to use this information to increase the accuracy of poor raters. Training programs could be developed focusing 104 on the weaknesses in raters' category systems or on the cognitive demands placed on raters by the type of rating scale. Since the degree to which raters' category systems matched the rating scale was most related to rating accuracy, a useful approach would to train raters who reallocate behaviors differently from the consensus of those in the organization and therefore whose idiosyncratic category systems are different from the normative implicit theories held by the organization (Nathan & Alexander, 1985). This approach may correct any erroneous impressions raters may have had and ensure a common frame of reference for performance ratings by all organization members. Further, it is necessary to determine if training actually can alter raters' category systems and if this effect remains stable over time. The present study delineated some important cognitive processes related to rating accuracy and also defined the relationship of experiences and education of the rater to raters’ cognitive processing and rating accuracy. This information is useful in order to understand fully the performance appraisal process and to point direction for future research in this area. APPENDIX A ROLE GRID 105 106 Name Hospital RUL-E ER 9 Instructions: At the top of this grid is a list of job role titles such as dancer, football player, etc. On the side of the grid, are eight blank rows. Look at the first row on the grid. There will be three circles which correspond to three of the job titles listed at the top of the grid. Consider these three job roles, pick Egg of these three and write down a way in which the two are similar on the blank to the right. Also, place a check mark in the circles corresponding to the two job roles you have chosen. Now, move on to the second row and find the three circles. Identify the three job roles that correspond to the circles, pick two of them and write down a way in which these two are similar on the second blank. Place a check mark in the circles corresponding to the two roles you have chosen as similar. Continue until you have completed all eighttflanks. Below is an example of a completed grid using foods rather than job titles. Look at the grid below. From the first three foods (coffee, soup and salad), coffee and soup were chosen as the two food with similar characteristics. An X was marked in the circles corresponding to coffee and soup, and "hot food“ was written on the blank to the right. From the second group of three foods (ice cream, candy and cheese) ice cream and cheese were the two foods chosen with something in common. An X was marked in the circles corresponding to ice cream and cheese, and "milk products" was written in the blank. Please use this procedure as you complete the grid on the following pages. ® ® 0 Hot «food @ 0 ® LAP-glue.“ ® ® 0 Red JOB TITLES COMMON CHARACTERISTIC 107 oloo GOO OOO GOO CONTINUED JOB TITLES COMMON CHARACTERISTIC 108 aloe GOO GOO GOO APPENDIX B BEHAVIOR GRID 109 110 Name Hospital BEHAVIOR GRID The following pages contain a list of behaviors that one might see when observing a nurse on the job. The purpose of this grid is to determine how you categorize these behaviors. Along the top of each page, 10 categories or dimensions are listed. Along the right side of each page, the behaviors are listed. For each behavior, please place a check mark in the box corresponding to the dimension or dimensions which you feel the behavior belongs in. If you do not feel the behavior belongs in any of the dimensions listed, leave that line blank. Also, not every dimension needs to be used, and you may place a behavior in more than one dimension. Below is a sample of a completed grid. The first behavior, "this person chews gum noisily," was placed in the category "well-mannered." Although this behavior represents poor manners, it is placed in the "well-mannered“ dimension. The second behavior, "this person spends time needlepointing," was placed in the "creative" category. This person chews gum noisily. \/ This person spends time needlepointing. On the following page are descriptions of the dimensions. Please read these before beginning the questionnaire and use this page of descriptions when categorizing the behaviors into the dimen51ons. b) 7) 8) 9) 10) 111 DIMENSION DESCRIPTIONS Observational Ability - sees actions and changes which might affect nursing care. agglth-congcioug - concern with own health and fitness. Qutggingiggmmsnisatixs - talkatiVO. open. effusive and friendly with others. Qggggigggiggggggg - maintains high standards of nursing care and fulfills nursing responsibilities. Sense Qfi figmg; - ability to appreciate or express what is amusing or comic. , Orgégizgtional agility - uses time, equipment, and personnel effectively in providing nursing care. Knowledge and Judgmggg - needed to meet nursing needs of patients. eggggcgggg - concern with looks and dress. Eémilngciggggg - concern with own family matters. 551;; Lg figmgg Belgtiggg - skillful in handling difficult emotional and social situations with patients, families, and coworkers which affect nursing care. 112 .ntmzao gust Donuts ocuqu UCI aca£OSIu cmflu >—_I:u: Ia mats: was» .woauuua use LIuc uOUIna uCI tucaIaao c008 ta: Laazu .Uuzz I Itasca sac; uco Iiou Ou LIJLOIOU I JII ca touuuoxu on u—30u UILfl: Iazu .LaIsu .0053 case ale 0» I—oooa aka *0 net; Ina BIL—JUIL as: unlauIa I nu alcoalnI ~— .Ifiau 00L. L0 IJIILa I«£\Lfl£ azaLat duo coastal Lo .acaooon oucul«ULle OILS: saga Sta» 0a uuuuxl t-aoz .CIILt area on: coo—a as» Logan .suc: .Laoan too—a ocsauaw I I>I£ a» II: on! atlaqu I Du ca—3ICa LI-flOUL oca>au >I—flt nu sauce. 30:; Du titan saga uuuuxu tuna: .Izucoe 3m» >L0>I uu>aILuIs IazxLlc OOCIzu Du Dita: Iazu uuuoxu snub: .-. ..oc.uc~cnn. an 0a untousucou as as: unlaaIa LIUCIUI—Icultuu I :0“: x—Iu uCI stop aa- OJ .Iunamuoa LO>ICU£3 .otIuUInxl In u—aou Data: 0.3» .uJUIcn ucI .IU-quIDfl) IIL .IcuILo quzz I—ozl loco» zuItauICx >~c0 IuII 005:: much I—lfl! to» ..Uat .ataoo> .l.a. .600» to fln>u LI—SUaaLIa I BI£U=Oucs IU>IU~ >~u¢lauqlcou atlueIa I uIzu m>Lmuoo Ou umuumaxfl in an: b.30u Data: mash m¢O~><=un wza_mzmz_: 113 .wE.u UDL. LD I.IDE .mxuwtn ocuLDD awntac nguD zuux x-uu LD uuzu >‘uLuL Du DULDC l¢£u uumeD t-DD: . .>LU Du uD: CUEDI uzu _—Du Du twuumaxu on v.30: Data: I.£u ._~u >—w:DLUOCID mu ucmamaz Luz muflfluua aca>Lu I« at! :MED: I so Ducmnfltn uzu :— .nLUuumE ~uu.uwilcoc uDDDm mLDuamu> nus: zuocu_ u. xmlu Du UMLDC nuzu uuwaxu D—DD: .Dmn nu: CD >~uuuaa acu>— Bozo. fin :Iu cauuo Dca >uu>uuUI Dummutuuu luuuLuncolwo >ID Ucu so umDE own .0 u:D cums mm: on: >u3uu LD» :« ucDuuua >LDuu—DnEI :I uuau m>LumaD Du Dauuuuxo an u—aou ammo: mush .uxnttn LD UEuu 00L» acuLDD mtuzuD Du mmxoq ecu—uou Duch n—zu ucuw Du uumaxfl o~303 .zu:3~ LuuwI lawn >qum Dame uca .U—aumeou vu .Dc—DLDE uzu :— Iucuuuun own xul LuzuD Law ULuu Du uqu uzu uuaLumcu ucI .nu:UEunwLu Dem mcDuuNquwE D>uo .Iucmqua ——ul>_m:Dume wzu Du uLlu fl>.a Du Dauuunxl an u-aDU DIL5: muzu .umummI Du Dual UCD zuu: ._~u >—u:DuLam ULI eat: so LaD.u mucuwuua two o. »D ULIU Law >uu—mn—nconqu mzu cw>uw .>mu I wuco uMMU— uI UMDDQM uuzxLflc nn—ou Data: was» .tnaeue #usuu It: I Du *uumE_£\Lm£ DunflnLucm fits a: x-ux Du DMLDC muzu uumnxu D_:D3 mxc~><=Mn mz:_mzu2_: 114 .ucoiuuaatm >LDu0—JULuu cum: ucuquu I wD Dun Dzu ~D zuuao Dmcx Dcu mauIL .uumaamL n.u:fl«uIa Dcu uI .ucoal DILDC luzu ulzu uuunxl n—3D3 .ucvauID DCD LDu DLIU ocuCLDI to» DDBDI: CDcu- ulo Du uumD-u cucu_ Dzu Du IDaLu LDD~ IxIt Du DDuUDDxD In 3—303 .>—uDDD~n bunnnto 3L0: Du DIDU Du Data: Ducu uumeD D—303 .eucu CD a: JUDcu Du ~DD:UI lDLw DOD: uDo >D£u c033 >ID>LU>D DILD—uzu luzxtoz uaIu Du uuuuunxu In D—DDU Data: ouch .uucmquD cu IUDUDLD Du luau ILDquu> LI—DuuuLIu zuucz uumyuu ~IcDuuDIU nzu D>LDIDD Du quUDaxD Ia t—DDU Data: much .>-u:D:UILy IDxDEI Data: mush .DDLDDLD u« u« [083 Law lucuuuaa Duccu Du DLIU xUIn “DD—Dauuti D>uo Du DILD: quu uuluxu D—DDI .ULDCuD Du oeux-Iu :03: CD zit-anon: Data: lush .CDulu>LInDn uaozuux on DD Du DD Dug-IL In D—DDU oauDCDLDuLI>l—. DU£QD>DJ ocu>uIUUL uCDqus I :D ufluscui n >Lu>l IDLDuIILa DOD-n DLDUDL DCI lulu Du DDJMI ~— .>—omc0u£mab mumMULD LD DDIMILD >—nquuafiu I>I3—I Ia main: maze wzou><=mn m23-n222—2 115 .xLDI Du mono—DU LD DELDwLmo .DDIDxIE uD uD- I atom: Data: mush .ucmu cnm>xD CI cu ucuuuau UIuDLIu uuul>~musua :I Du tuna quuLID I >~CD D>uo Du Data: nunu uUUDxU D~DD3 .mCDquNLI>cDu DDDLD >~D:D.L+ ocuLDD >~uDuDD uun Du DmLD: lazu uuwnxu v.30: .IDLI 8&0: nu£\LU£ cu sui>qu~IL .cutbauzu .umzoum. >~uEI¢ nmzxtmz ~D IDLDuuuD I>I~Duut DmLD: mush .cmuuu~ DCI DDumDLnucu xDD— Du Douuaaxu In D—DDU .>LDunu£ uauutue umID DCI nDDuDE>u L0: *0 >~nDDCuucDu DDI xuuunat mu—Iu Dz: u:flqua I ocuuuusuI OLD: lmLD: nusu ~— .nD:DuL» DCI IleLDIDU zuul Iluuu>uuUI quUDI DDUCILLI LD u—IWE D>I£ Du nDDCILLI >-IDuD unLDc much .IDLI 8L0: uuz\Lm£ ca ODI: Du LD nLuzuD Donn Du IDDDuLIu LD nuuDDu cu ocutn >-Icounauuo Du DULDD luzu uuuaxu D—DD3 .Lfluflzqu OCu—uuxtcu DI IDL» DCuLD wD tout wD DDuumnnIu I DNuCODUDL Du DmLDc luau uumnxfl ch D-DDZ .DEDc um cumEDL Den nu. uLDDDL DCI ~quunD£ ozu -IU Du >uuxu~ 0a D—DD! DmLDD muzu .Dmuuuebu uLm mucuquD -ul>~muau0 wD tunes: uIDMDCD CI .>uDD usuu: I acuLDD .*— mmo~><=mn mzo_mzm2.: 116 .mouI :I >2 >_CD uuucaueouum >ML1x Du ucu—uua Dzu Dcum Du DmLDC nucu uuwnxw DuDD: .>Mle Du Du ucuuuaa u~uI>—_quutu I umzu umuucmmmw m. uu *— .umtuw umE an Du U>Iz nucuquD umuuu—w wzu so moot: lzu Dom mumtac .0 uLD£m mu nuuuumnz tzu uuzu ucuuumn Dzu __uu Du >_mxu— an D—DDI nuts: luzu .ulu—Dmoc Dcu Du Duu>Lwl uzu uDDnu DD:—I—DIDU ucw—uID ucuuuu_I>CDu I *— .)ID I—zxLuz uDDnl nLuzuD —_Du Du UuDDUDCI ocuIDlI cm In: >-_I:ID DmLDD m—Dp .DID— JLDI .D mum—DLMUUL .uu ocuLuaumL lucmuula LDw Dav—csuuu uuuomma MDD—DUuumE uDD >LLMU Du DILDC lucu uumaxl D_:D3 .ONue um eta: mgu D>MD— Du >DNWL DD! Duzunn ucmuuun Dzu m>mz Du Dwuumuxu ma D—DDU .tc onuo um >ma >LD>U >DMLmzuDum>zD Du muDu Dz: ucuuumu I Du Ducoummm mLuz meDc muzu v— wmo_><=mn sz-wzm=~: APPENDIX C RETRANSLATIDN PERCENTAGES FDR BEHAVIOR GRID 117 118 RETRANSLATIDN PERCENTAGES FDR BEHAVIOR GRID JOB RELEVANT BEHAVIORS AND DIMENSIONS Eneelegge and Judgment Behavior Would expect this nurse to know enough to delay giving regular insulin to a patient who was to have a fasting blood sugar, until after the blood had been drawn. Would expect that this nurse might, at the patient's request, raise the knee gatch of the bed of a patient with circulatory impairment. Would expect this nurse to give only a partial bath to an acutely-ill cardiac patient in an oxygen tent. If it is essential that a critcally-ill patient go to x-ray, would expect this nurse to send the patient to x-ray accompanied only by an aide. Organizational Ability If assigned to a patient who required the help of two people to get into wheel chair, this nurse could be expected to ask a coworker to come and help before a wheel chair had been obtained and placed near the bedside. Given the responsibility for care of 10 bed patients (four of whom are seriously ill) with one aide to assist, this nurse could be expected to give care to the seriously—ill patients, give medications and treatments, and instruct the aide to care for other six bed patients in the morning, if possible, and make empty beds after lunch. Would be expected to make four trips to the linen closet to get linen needed for morning care for one patient. Percent 100 86 100 86 100 100 119 Eebaxiec If this nurse were assigned to a patient who goes to physiotherapy every day at 9:30 AM, could be expected to have the patient bathed and ready to leave the ward at 9:20. §till in Human Relations This nurse could be expected, whenever possible, to sit down and talk with a terminal- cancer patient who is considered to be "demanding." In the presence of a woman who is crying because her husband is dangerously ill, this nurse would be expected to tell the woman not to cry. If this nurse were admitting a patient who talks rapidly and continously of her symptoms and past medical history, could be expected to look interested and listen. If a convalescent patient complained about the service in the hospital, this nurse would be likely to tell the patient that the hospital is short of nurses and the needs of the sickest patients have to be met first. Eggssientieusoess If asked to take and record blood pressures every 3 minutes on a patient receiving Levophed (levarterenol), could be relied on to do so without supervision. Would expect this nurse to give meticulous back care to those patients for whom it is ordered. If, during a night duty, an unusual number of acutely-ill patients are admitted, this nurse would be likely to call the hospital and report ill and remain at home. Would expect this nurse to carry out meticulous aseptic technique for patients requiring it, regardless of work load. Percent 86 100 86 100 100 100 100 86 120 Qaeecxaiienal ability Eebexioc This nurse could not be expected to observe that a patient consistently leaves untouched a particular type of food. This nurse could be expected to observe that an ambulatory patient in for study who has been out of bed most of the day demonstrates decreased activity and often can be found lying quietly on his bed. This nurse could be expected to observe the emotional effect which particular visitors seem to produce in patients. Would not expect this nurse to recognize a cessation of flow of urine from an indwelling catheter. NON-JOB RELATED BEHAVIORS AND DIMENSIONS flgalth-Congcioug Behavior This nurse eats only "natural" foods (i.e. yogurt, whole wheat grains, raw vegetables, etc.) for meals and snacks. Would expect to find this nurse exercising, jogging, or working out during her/his breaks or free time. This nurse smokes frequently. Outgoing/Communicative Would expect this nurse to walk up and introduce her/himself to a new staff member. Percent 100 86 100 Percent 100 80 121 Eebaxis: Would expect this nurse to talk at length with visitors about non-medical matters. Would expect this nurse to rarely chat or talk with other nurses during breaks, meals or free time. This nurse "babbles" on when talking to others. This nurse usually arranges to have meals or arranges social activities with coworkers and friends. Would expect this nurse to sit quietly during friendly group conversations. Sense of Humor This nurse is usually seen laughing and joking around with others. Would expect to find this nurse telling jokes to others during free time or breaks. Would expect this nurse to occasionally bring in comics or cartoons to show others or to hang in her/his work area. This nurse usually has an amusing anecdote to tell others about her/his day. Baoeacanse Would expect this nurse to change her/his hairstyle every few months. This nurse is always impeccably dressed or dresses fashionably. Would expect this nurse to come to work dressed sloppily. This nurse wears a lot of make-up, perfurme or cologne to work. Percent 93 80 93 100 0 (.4 100 122 Eamilx Qcientsg Esbaxiec Eecseot This nurse calls her/his spouse at least once a 100 day. This nurse could be expected to call her/his 100 children everyday when they get home from school to check up on them. This nurse displays pictures of her/his family 100 (spouse, children, relatives) in her/his work area. APPENDIX D BACKGROUND QUESTIONNAIRE 123 1) 2) 3) 4) 5) 6) 6) 8) 124 Name Hospital BACKGROUND QUESTIONNAIRE How many years have you worked as a nurse? less than 1 year 11 to 20 years 1 to 4 years 21 to 30 years 5 to 10 years over 30 years What is your position? staff nurse charge nurse head nurse supervisor other What is your title? RN LPN Nurse Practitioner other In what unit in the hospital do you work? intensive care psychiatric emergency obstetrical/gynecology geriatrics medical surgery children's other Where did you receive your primary professional training? college/university (4 year program) community college (2 year program) hospital training (3 year program) other What is your highest educational degree? A.D. B.A. M.A. Ph.D. What is your sex? male female Have your previously had any experience in rating or evaluating other nurses? YES no If yes, how many years have you done so? 5 to 10 years over 10 years less than 1 year 1 to 4 years APPENDIX E SCRIPTS FOR VIDEOTAPED PERFORMANCE 126 Marilyn is the nurse ratee in the videotape. Scenes were filmed in a hospital setting. (Marilyn is standing at desk reading charts when Doctor walks up) Doctor 1: Marilyn: Doctor 1: Marilyn: Doctor 1: Marilyn: Doctor 1: Hello. Hi Dr. Adams. I was just in Jane’s room. She’s really jittery and she's very anxious. Yesterday she was so quiet. We didn't have a problem with her. Now she's just walking up and down the room... She's usually pretty quiet, pretty subdued? She always has been. I’m wondering is it the medication or is it the illness that’s causing this. We did change the medication. Some of the other people are showing the same thing. That was a good point - good for you to notice that. We might have to change it or change the dosage. Maybe we should check the other people and see what's going on. Okay, would you like to change the order then? Okay. (signs form) (Marilyn is reading chart at desk when Nurse walks up) Nurse 1: Marilyn: Nurse 1: Excuse me, Marilyn. Can I interupt you for just a moment? Sure Jan. Uh, I have a problem in that, uh, I’m always having to pick up after Cathy. Now, I know that she’s new and everything, but she doesn’t have the work load the rest of us do; she doesn’t have 127 as many patients. And I'm having to go in and help her make her beds, help her finish up her baths, uh, she doesn’t watch the schedule to know when her patients are to go to x-ray or anything. And I really think it's getting to be a problem, uh... Marilyn: Okay, Jan. I’m glad you came to me. I have been observing this and I have set up a meeting with you and Cathy and myself. Okay, so this’ll be tomorrow at three o’clock. Nurse 1: Oh, that'll be great. I just didn't know what to do. I try to be nice in talking with her and be... Marilyn: I'm sure you have and I appreciate you coming to me. And I'm sure this can be worked out. Nurse 1: Okay, great. I’ll see you tomorrow then. Bye. Marilyn: Okay, bye. (Marilyn and Nurse are at desk looking at a chart) Nurse 1: Marilyn, now you know about this, uh, new patient that was brought in late yesterday, this Peter Humphries. Uh, he was an overdose, uh...taking valium and well, you can see a number of different things here. Uh, he is under control now, he is stable, but we do need to watch him. We're still taking his vital signs through today. The doctor wanted us to check him yet, well, still on every two hours yet this morning and then, uh, by lunch time we can go on to every four hours schedule...And just kind of monitor to make sure he's eating right. Marilyn: Okay, the Vitals are ready to be taken now? Nurse 1: Yeah, so why don't you get started and I’ll let you go with that. We’ll check with you later. (Marilyn leaves; Doctor walks in and talks to nurse) Doctor 2: Nurse 1: Doctor 2: Nurse 1: Doctor 2: Nurse 1: Doctor 2: Nurse 1: Doctor 2: Nurse 1: Doctor 2: Nurse 1: 128 Hi, Jan. I'm a little worried about Pete Humphries in 122. How’s he doing? Well, I was just talking to Marilyn. He is stable this morning and we've still been checking his Vitals and he's doing pretty good. Is someone keeping an eye on him? Yeah, Marilyn, the new nurse. She just started last week...she's real good. Yeah, um, she is good though? Yeah. I don't have some...well, I know she's new here and I don't want to say that just because she's new, she's not good, but...I just want to make sure there's someone who knows what they're doing, in case things get a little... Well, I watched her when she was...well, she did her nursing training her too, and so, um, I had seen her then and I'd talked with her quite a bit then...And she's real conscientious, she just uh...you know, I trust her to keep an eye on him, even though she’s new. Uh...and she's got the regular load like everybody else, too. So, I know that she'll do a good job. I'll take your word for it then. We're going to check together at noon and compare notes and see how he’s doing. So, uh... Okay, real good. Thanks, Jan. Yeah, we'll see ya later. (Marilyn is at desk when Doctor walks up) Doctor 3: Marilyn: Hi, Mrs. Smith. Hi, Doctor Daniels. 129 Doctor 3: I just checked on, uh, Mr. Vancouver. He seems to be doing better. Marilyn: Yes. Doctor 3: What are his Vitals? Marilyn: Okay, his blood pressure is 116 over 70; his pulse is 68 and regular, and his respirations are 18. His temperature is a little high...99.2. Doctor 3: Okay. Well, I just, uh...I understand that you did the CPR this morning for him. Marilyn: Yes. And I'm glad the hospital instituted that training program for us. It went well. We’re real pleased. Doctor 3: Good. Well, thanks alot. We really appreciate it. Marilyn: Your welcome. (Patient is lying in bed when Marilyn walks in) Marilyn: Hi, Jeff. How are you doing? Patient 1: Oh, I'm a little worried about the surgery. Uh...Am I going to be able to walk again? Marilyn: Sure you are. I noticed that you had been quite quiet and a little bit restless, though. The doctor will come in tonight and explain to you about the procedure. And, it's called an arthroscopy. Okay. You mind, I'll have a seat and talk to you a little bit about it. (pulls up chair and sits down) Patient 1: What about water? I heard that water builds up and every once in a while you can't walk, and you have to have crutches, and... Marilyn: No, the doctor will follow you real closely and at first you will keep your leg elevated. And you'll have a wrap on it - a dressing. And for the frist 24 hours, you will be in the bed with your leg elevated. Now, if you have to go to the bathroom or something like that, we'll help you. You've already been trained to use crutches, correct.... (Marilyn is at desk counting paper clips when Nurse walks up) Nurse 2: Marilyn: Nurse 2: Marilyn: Nurse 2: Marilyn: Nurse 2: Marilyn: Nurse 2: Marilyn: (Doctor is Marilyn: I just talked to Ruth. a real busy morning. She said it's going to be Oh. Um, Sue's not going to be able to get in until about noon, so we're supposed to just get our high priority stuff done and, uh, get that taken care of. Okay. And worry about other things until later in the day. Okay. Well, you know we need all these paper clips for the chart. I was just going to count these out. And then I'll check the supplies in the closet. Uh...Marilyn... What. We need to get high priority stuff done...bed baths, make sure everybody gets their breakfasts in and that kind of stuff. And then we'll worry about that later. Well, okay, if you say so. at desk looking at chart when Marilyn walks up) Hi, Dr. Adams. Doctor 1: Hi. Marilyn: I was talking to Mr. Smith, and we had an order for theolair. I noticed in his history that he has had heart problems. I’m wondering if you want this drug used on him. Doctor 1: Uh, it's a good point. Yeah, and it has been generally not recommended for heart patients. But, I've checked it out and it won't be a problem with this particular patient. But thanks for bringing it up and noticing it. Marilyn: Your welcome. Okay. Thank you. I’ll go get on it. Doctor 1: Okay. (Marilyn and Nurse standing by desk talking) Nurse 2: I can't believe what a busy day it's been. It’s been so hectic...two admissions over dinner hour and then having Sally out sick has just make it impossible....Boy, oh boy. Marilyn: You know, I talked with Ruth and I told her that I would stay a couple hours after my shift if she needed me. Nurse 2: Boy, that's nice of you. I've got a big dinner party tomorrow night and I got to get home and start getting some stuff situated for that. Marilyn: Oh, well, I'm glad I could stay then. Nurse 2: Yeah. (Marilyn is at desk writing. Nurse walks up) Nurse 2: Hello. Marilyn: Hi Cathy. Have you noticed on Jim Smith, the Nurse 2: Marilyn: Nurse 2: Marilyn: Nurse 2: Marilyn: Nurse 2: Marilyn: SCENE 10 diabetic in room 234 that he hasn't been eating very well lately? Oh, I haven't been taking care of Jim so I haven't noticed. You know, we better alert the nurses to watch what he's eating. His blood sugar is up to 460. Oh my. I don't know if his friends are bringing food in, or relatives, things that he shouldn't be having. So... Hmm, that's a possibility. So, let's just keep a watch on him. Okay. Okay, thanks. (Patient is in bed. Marilyn walks in room with pill cup and water) Marilyn: Patient 2: Marilyn: Patient 2: Marilyn: Patient 2: Marilyn: Patient 2: Hi, Peggy. Hi. How are you doing? I was fine 'til you came. You don’t like these, do you? No, I don't. Well, you only will have to be taking them three more days. Dr. Jones says that, uh, you’ll be fine by then and ready to go home. That'll be nice anyway. Can I -- can I have milk instead. I haven't had any milk today, and I'd -- I'd rather have milk. Marilyn: Patient 2: Marilyn: Patient 2: Marilyn: Patient 2: Marilyn: Patient 2: Marilyn: Patient 2: Marilyn: SCENE 11 133 Peggy, this is tetracycline. And we really do not like to give milk when you're taking it. Oh, why is that? Because, it...the drug itself will not absorb in properly in the gastrointestinal tract if you drink milk with this drug. Now, you doctor says it's okay that you can have milk two hours after you take these pills. Oh. Okay. So, we will bring you some. Would you like chocolate or white? Oh...just plain white. Just plain white? Yeah. Okay. please? So would you like to take these now Not really, but I guess I have to. Okay. (gives patient pill) (Nurse is at desk when Doctor walks up) Doctor 2: 2: Nurse Doctor 2: Nurse 2: Hey, Jan. You got, uh, Bob Morrison's chart there for me? Yeah. doctor) I got it right here. (gives chart to Thanks. He’s down in x—ray right now. (Marilyn rushes in and bumps into doctor) Marilyn: Doctor 2: I'm sorry. No problem. (walks off) Marilyn: Nurse 2: Marilyn: Nurse 2: Marilyn: Nurse 2: Marilyn: Nurse 2: Marilyn: Nurse 2: Marilyn: Nurse 2: Marilyn: Nurse 2: Marilyn: 134 (exasperated) Jan, can you help me? I've got six patients to feed in room 232. I've got to exercise Mr. Jones who had that myocardial infarction, and I haven't had any time, and the patients.... Wait a minute. Wait a minute. Slow, slow down, Marilyn. Sighs. Uh, you've got six patients to feed?...it's been 45 minutes since they brought the trays up. What, uh.... I know, I know. But Mrs. Jones came and she brought her household things. I had to inventory them. I had to do everything...l had to count the supplies...patients are... Yeah, I know. Wait a minute. The patients are really getting upset with me. Now, slow down. You know that we have to name label all those things, but they didn't have to be done before lunch here. You could set those aside and we could have done that after lunch. You've got to set your priorities. I know. But they're mad at me. They're hungry.... Well, yes, they're hungry...we're going to have to...- Would you help me? Yeah, I'll help you get things going today. But I think we’re going to start writing a list of priorities of what needs to be done first...and get a timetable down so that we can do it. Okay? I know. I’m really sorry. Okay. Let's go and we'll work on getting the meals out. Thanks I appreciate it. (They walk off) SCENE 12 135 (Patient is in bed with bandage on arm talking to Marilyn) Marilyn: Patient 3: Marilyn: Patient 3: Marilyn: Patient 3: Marilyn: Patient 3 Marilyn: Patient 3: Marilyn: Patient 3: Marilyn: Patient 3: Marilyn: Hi, Lynn. How are you doing today? I'm pretty good. How are you? Fine thank you. We're going to change the dressing. Okay? Okay. How has the wound been feeling? Well, it seems to be a little bit more painful, but...not, not really too much. Not too much. Okay, have you been able to sleep in that okay, or no? Well, I can't of course lay on it or anything, and there's a little bit of redness, but beside that, no. Okay, I'm going to cut the gauze off. Okay? I realize the abcess is on this side. So this shouldn't hurt. (starts to change dressing) Remember when we talked before about signs of infections as far as redness and swelling, pussy drainage? Um hmm. Okay. So you said the pain has increased somewhat? Somewhat. (takes off old dressing) Okay, now don't touch that. Okay. (starts to put on new dressing) This is a sterile field I'm going to use. And I'll put this under your arm... SCENE 13 (Marilyn and Doctor are at desk looking at charts. walks up) Nurse 1: Marilyn: Doctor 2: Marilyn: Nurse 1: Marilyn: Nurse 1: Marilyn: Nurse 1: Marilyn: Nurse 1: Marilyn: Nurse 1: Marilyn: Nurse 1: Marilyn: Nurse 1: 136 Nurse Hi Marilyn. Hi Jan. Hi Jan. (walks off) How are you? Oh, just fine. Good. Weather's nice out. Oh, sure is. (reading chart) Oh, look! What. John has a birthday tomorrow...you know, the patient in room 228 that has a broken leg. Oh, that, that's nice. You know, his parents are gone for the weekend. I think I'll bake him a birthday cake. I think I'd like that. (surprised) You’re going to bake the patient a birthday cake? Sure, I mean he’s going to be alone and he's going to be 21 and he's depressed he's in the hospital. I realize that he's on a general diet and everything but...you know, if we start making birthday cakes for everybody that, uh... Well, I really don't mind making this one. Remember that pretty card you got for Susan one time, your daughter? Yeah. Marilyn: Nurse 1: Marilyn: Nurse 1: Marilyn: Nurse 1: Marilyn: Nurse 1: Marilyn: Nurse Marilyn: §Q§H§ lfl (Marilyn Nurse 2: Marilyn: You think you could buy a card and bring it in and I'll have some of the other nurses bring in cups and streamers. Well, I think we ought to check with them first. I mean...this is really nice of you to want to do this for the patient but...you know maybe we don't want to start something that's going to get out of hand, uh... Well... Okay, if you really don't want to. But, you were such a help before. I was just wondering if, you know, perhaps you would like to think about it. Well, I suppose it would pick him up. He's got the broken leg. He's been in bed for...two months now. I know that's what I was thinking when I saw that. Well, that is a pretty good idea. Okay. He'd enjoy that. That would pick him up. Okay, thanks alot. 1: You check with the other ones and we'll work on it. Okay. I will do that. Thank you. and nurse are at desk talking) That Mrs. Miller sure is a sweet little old lady, isn't she? You know, she really is. Have you noticed the patient in room 224, Sally? She's always so jovial. 'She laughs and talks with us and walks all around. Lately, she's just been sitting in her room for the past like four hours. Nurse 2: Marilyn: Nurse 2: Marilyn: Nurse 2: Marilyn: Nurse 2: §QEBE 15. (Patient Marilyn: Patient 4: Marilyn: Patient 4: Marilyn: Patient 4: Marilyn: Patient 4: Marilyn: Patient 4: Marilyn: Patient 4: 138 Oh, really? Have you talked with her at all or...? No, I haven't seen her. I think we better watch and see what might be the problem. Hmmm. Because, it's just such a drastic change in her behavior. Hmm...has anybody been in to visit her or.... with eye patches is in bed when Marilyn walks in) Hi Jeff. I'm Mrs. Smith. Oh nurse. I thought you were supposed to be at lunch? I ate a little and then I remembered that I promised you that I'd come and read the newspaper. You probably ate the hospital food. I’m sorry, what? You were probably eating the hospital food. No, I actually don’t like the hospital food. So, I brought my own sandwich. I can understand that. How's your day been going? I just can’t see anything. Oh, it’s perfect. Do you feel okay, as far as you don't have any pain? Oh, no pain, no. Marilyn: Patient 4: Marilyn: Patient 4: Marilyn: Patient 4: Marilyn: Patient 4: SCENE 16 139 Oh. The doctor'll be in this afternoon. I understand he's going to take the eye patches off. Good! I'm looking forward to it. Okay. No pun intended. (both laugh) I have the paper here and if you don't mind, I'll sit right here by your bedside. I'd love it. Okay. This is the Grand Rapids Free Press. Today's issue. And the leading story is Reagan clobbers Mondale. Sounds like a football game... (Nurse is reading at desk. Marilyn walks in exhausted) Marilyn: Nurse 1: Marilyn: Nurse 1: Marilyn: Oh, hi Jan. Hi. You look really beat today. You going to make it 'til the end of the shift? I think so. This has been really a tough case. Larry Jones, you know, in 231? Yeah. Well, you know, he's had leukemia. He's in for treatment and it doesn't look good. They called his family in the middle of the night last night. You know, he has an 11 year old and a 13 year old. And they live, oh...about 100 miles from here, because they don't have the money to stay here. So, they called the family in and I had to talk with them and talk with the patient...He's going to die. Nurse 1: Marilyn: Nurse 1: Marilyn: Nurse 1: Marilyn: Nurse 1: Marilyn: Nurse 1: Marilyn: Nurse 1: SCENE 17 140 I know that's a drain on you. And I don’t know which is worse. We take care of the patients, but it takes a lot out of you. I know, I did arrange... And you're really good at that too...talking with the family and... Thank you. It's really hard though. I arranged for the wife to stay and the children to stay with Red Cross. They have housing available on an emergency basis when they don't have the money. The wife really wants to stay in the room, but the husband and I don't blame her. I would want to do that too. So, I've looked into what, uh, to do with the children. Because they really shouldn't be in the room all the time with the father...But, of course, they want to be too. So, I think we have that worked out. That sounds like something we've never done before. I mean, you've made arrangements for the children to stay too. I think they will be allowed to stay. I think it's really important too. Because I don't believe that he'll live through the next 24 hours. Yeah. So, but I just feel really bad. I just, you know, some of these patients you just don’t know what to do. Well... You try your best. We sure do appreciate your spending the time you have... (Nurse is on phone as Patient walks by) Nurse 1: ...Oh, seven o'clock. Let’s get it through the Patient Nurse 1: Patient Nurse 1: Patient Nurse 1: 3: 3: 3: 141 dinner hour, okay? (hangs up phone) Hello, Mrs. Harding. How are you doing? Fine. I'm a little tired though. Well, I'm just coming on. What did you do today? Did you do too much in physical therapy? Maybe. I don't know. I just seem to be tired, though. I thought I'd read a magazine. Yeah, why don't you go lie down and put your feet up a little. But we've got an hour here before dinner and then you'll feel like exercising again after supper. Oh, maybe you're right. I'll do that. Okay. We'll see you later. (Marilyn walks in with coat and purse in hand) Marilyn: Nurse 1: Marilyn: Nurse 1: Marilyn: Nurse 1: Marilyn: Nurse 1: Marilyn: Nurse 1: Hi Jan. Yeah. I'm just getting ready to leave. But remember before we were talking about Mary in 232, the one that's been real depressed? Yeah. Well, today, I really don't know. She's been happy. She’s been telling people to have a nice day, and she's been walking around... That's pretty unusual for Mary. It really is. I wish that...would you keep an eye on her, maybe check her every half hour or so? Yeah. I'm a little afraid that...She's just too happy... She's just been so into herself for, well, ever since she's been here. Yeah, I think I'll, uh, have Marge kind of keep an eye on her for us too. Marilyn: Nurse 1: Marilyn: Nurse 1: Marilyn: Nurse 1: SCENE IQ (Doctor is Marilyn: Doctor 4: Marilyn: Doctor 4: Marilyn: Doctor 4: Marilyn: Doctor 4: Marilyn: 142 You know, when I go off to supper or something, I'll make sure that somebody's watching her. You know, as well as I do, that somebody who's been so depressed for so long and then to be picked up like that right now... That's right. We don't have any suicide precautions in the room. Not in this wing...yeah. I'd appreciate it. I'll be sure and tell Marge. Thanks a lot. Yeah, have a nice evening. in office reading when Marilyn walks in) Hi Doctor. How are you? Okay, and you? Fine, thank you. Mr. Wilbur, the patient in room 224...Remember he had surgery early this morning? He came in as an emergency. Right. His catheter was taken out and he hasn't voided. It's been now about eight hours. He seems real...he’s distended and he's uncomfortable. I'm wondering if you would write an order that we could catheterize him p.r.n. Yes, that's the correct procedure. (writes order) Okay. Otherwise, he's doing real well. Good. His vital signs are fine and his temperature is down. Doctor 4: Marilyn: 143 Good. Thank you for noticing that. Your welcome. Thank you very much. APPENDIX F BARS SCALE 144 145 .>uu~wa¢ mIcoqu>LDuoU Dom .mmmcmaowucwuumcou .lcoqu—Dm CIIDI cw uuuxm .>uw—uo¢ uIDDquNuCIoLD .ucueuoodo Dom Ioouuzocx .DLI Icowlceiut nan Dab .DOCquL >cm mouxms ILDwuo IchuucwwDo couecuiwo Dan lcu 4D was suaotzu taut Dummum .cDuucufiuD uucu DDwCLIUDDu IUDIELDwLua I.IIILD: Izu nucmnmLomL umun uIzu LIDID: Dzu D—ULuu .aCDuICDEuD can: :UII LDC umcoquULmo mmmnz ml¢hm J¢¢uzmm J¢hnmm01 Nth can m4¢um m2~b¢¢ woz¢2¢0ummm .auwamox 146 .guoou ecI Iqu .IocI; I“; zen) cu ecucuol 0;. :u a: uIe on er- I III: on I. 0:: acoque I ab—uI e—ooa IIus: Iuau “Ia. nous-I o—ao: .ueIquIalu mucqusuuuu gnu: ucquIa I no one Is» .0 :que 00:; I:. IIqu .uoooeo. I.ueequa Isa uI .ugeul IIuac I—gu uIsu uuoauo v.90: .Ieua cI ha knee oeueIanqu AI» I cu accuses Ia» ecoa 0» I32. I32 Duos-I 3:0: .59. I3 on uouqua awash—«quuuuo I :5 quueuIII Id 3 .— .ucuu IIean :I In acoqua quvqu uuuus—IusuI :I On auIa ququa I huge I>ue cu Ions: Iago unease ease: .o—OD acuou uooauua IaIOAlougu qusosoo IusuI sou: esoqua I vIIu see IguIa vase: IIuaa Idea uIau uoIeII vase: .oaIuo eILa vIe soc—a Isa qu-I aqua: .uIese woo—a eequIu I III: In II: as: acuque I I» eu—oIIu uI—aeou e¢u>ue uI—Io Ia seaoeo soeu cu IIuaa Ida» uueexe ease: .ooocoucu II: onus—oust quueaa eququII ocI saunas ecu .qu cu vouuuexe IA euaou Ions: Inga .eoqu-quua«. qusouusI and: aeoque I sue Ioueuae so. :quuua In nee-o II an .Ioo» use cu uII couuuau uuquIe I III: ocI Iasu Icu>Ia uI‘uocI no IouuaIIcu so. IuIaoua .uoauoo In» nuance cu souuoauo Ia e—eoo IIuso Iueu .Iaau Icuaoa Is. no ~I>oleu Isa Isaac—«on soocoquo Iu eeI Ions-c no Ichaalou :ouuuoIIu autuIIe I 5..) acoqua I nu .Ioeuqu I; soon II: ecu soIIu accquu«oul IquueouaaI nozuo ocI oqucouoI I>Ig ocI s—oqueollu sauces Isa quu on quuIaxI In e—oou «Isa: Iago .IuceIse II: ocI :uooov ecu—Ion: so Icquelou ou—uuUuooa no ocuuuoneu :I oo>uuuou II: as: usoqua I u— mm.o 0o.- n~.. e: A: .2. .I:0uuuooou sol-cu auua nocouuea .0 Isaac ecueuso ouIIa Isa eo>o ace: 0. coca-III mega-ac IquuaoueEI lacunae ocI uuu_oa cu ensue: ucoaeosn ocI «moo—socu Is. «a souaauuuoe IDquI=0Ioo Do: as Icouuo< .ItOuuuocou callbaca aunt IuequIe Iloa eoI Icouuuvcou gallon saw: Iuooque u—I .0 I100: ecuauoo ouIIa Isa uIII cu IouoeIol oedema: OquueODeaI Incubus ooI uuouoe cu scone: useleoaa woe Ieoouaoou Isa mo ecueaoeeoa Iquuoeu Ieouuus .Ioouuuocou cal-cue: gnu: IuquuIa use! ooI. occuuuooou col-ow so.) oucoque -I so none: "IsIocs osI ouIIa Izu Dual 0. IouseIII ecueuso oquueouaeI ~ Isoeuea ocI ..uo—oo .IuIauUuucI I» saved: acoleoaq ooa Inca—30c; as. .o :OuIIIIIoa oquuocu accuuoe ill". mucossc; Co meow: mcumsoc Dose so cosmos I quzusoo oz< muomuzozz _ 147 .ucoqua «so use Iqu ecu=u9l ace voodoo cucuu Due cu uIIO—D :Iouu Isa cu Ieunu soo- IuII o» oouuoewu In oases .ueoque Isa On Iqu ecueu0I ecueue Iuouoa IIII» nose". Ia» ea quIa Ian Iecho cu Ieuao Iugu uuoaII ease: .vuaqu Isa nuance—~00 so. uIcquoOD I qu>o.e cu acumen cu quuIaxI Ia v—oou cans: easy .IuIIuoIquIe ~I=qIDVAI ecu aeoquIa IuIeous cu quII «— .Iouaooa Isa nIIo oIUIua eoI.vI=qupo :Ioa oIa queu goes) I Incuua ego: oeI 0.50 on nouuoalcu I u.» o» vouuoaue In v—IOD «Ina: ousu .qusu uouga case you cu I—eoue can go e—Ia Isa ouuuaeou as: uooquIa I on soc-«III ~— .suqueIo uuqu>uIa5I I ou oeuuuesu use IoouI scI IIIuao quuuoeue o» Incoqua -I neuIII cu souuoeuo Ia e—sou IIaae Inga .IIuI ch Icons: ~ququue on» no oucIuIaoaI on» sea: Anna unnquuuuu Icov Iueoque an no. saddens-eoeIIu eopue «— .quIa and: eueoqucou :Ieuu .IeuI_ noun—«u Isusowl o. on» cause ocI IouI Isa uIe cu oeuuosuo In ouoou IIus: Iago .qumou hues-solo use Iuooz oau new «no as:- Ia enact unease quIa quuAIo: Is» .oue o— on uIsu Icosaoaou as sequence Ibo: IIuso Iago .Ioue II sud) sues eouoI>I so oIsa .m— .Ouue uI ouIa 0;» I>Iou cu noIIu voI ouaqu Danae-a Isa I>I£ cu vouooaxo In o—300 ..x.< en.o uI qu >DI>I AaIuI£DOuIA£a cu Icon 0:) ueoqua I On voceuaII «no: IIuo: Iagu «- .quuI use an II: you ocI; so IanquII ooouoo noguo ooI .Ieouo .oocuu no names. uoououuuae I o>Is cu oouuoaxI In o—aou .IuoIIAI I.IIu:o oII: use :« ouIa In» no oeuIso cu Inca cuss: Inc. an .guoou noqu Ivan Aueeo IuIl ooI .I—Auaeoe mu .esucuol 0;» cu Iucoqua can sue Duane Dog Iqu On IouI Isa uuouueou oeI .Iuculueouu ooI IcOuDquool I>ue .Iucoqua mud nauaooquI use an Iqu I>ue cu oouuuexo on Queen cease Ida»..uI_III cu IouI Ioo sud: A—uu s~e=0uuou IuI loss Co macaw Iuooque one o- no Iqu no. sun—unuecoauuu Isa :o>uu z_o>.soouuo _o::owuoc oco A, . .aouuuuruum no IIoII uaoguua :.1II£ . tan! - no: no ooouocu I—II: nos: ecuos 8 o I .95—=32. 0333-». no age—I93. _ uoosuu: IuculceuIII s—qu Iu£0Iouaa< «no and mud 11! .ueolevan :uu) ecqueoqu ocI Iowuuuouae o» Axu.. AW :oququuacou e=u>ue .IucoeoeuIII uquo IdocIa cu que uncququuII I use Iouquu ocI IouII uuquI0uI30 mm.- on; Q... .ucoleooa sun) ecqueoueo .uaauqu>I aoccoouoa ocI Iluu Isa CON A i no In: ~55}:- ouII ecu 33:32. Duo—II ,9 one .u>uumouxo Ia oIo~ use: cog) oo>w goose—=79 .osuu mom: I resusm< oao Isa :I>I Iu—oeouou cu 8.O IIII1II 3:32: :I Ion—seq Inca “33::qu «IquOI no _Ioo_u0II uasuuuuuo IuIqu no oaqu—elou o» cocoa u« use» ease I« quauo gnu: nou>Ison ..eIoIII uoo II: IucoauIa Ia“ ecu acqqu :u can; Ian uIAu acoqua uuouI—:AII Isa uuou cu wouuoexo Ia o—sOu .eeugqusa cu uuouuuuuo nN.Au ecuaIs III suIa Ia. Ia uIquIe I uIAu unuqua uuouI_:aII II we o—ou Inc: cause Iugu an . . .uuu on an: :IIIa Isa uuou Iu quuIaII Ia snob: «Ins: Iago .nnu s—IaouoeeIo Iu ooIAIa: no: IIaIuoa eouhuu Iu Is) eIloa I no Inca-Ins Is» on .uquuIa nouoIoI—rsqluou Isu In Iqu ecu>ue Inns) .IquIe Hague . 2.. .2; :oquIuIZ-ou I so 3:0 3 ‘30:: In 3:00 35:35.02... 3: VII-«uses Iu Om. O as) .IOOD vIaIu I cu .ucoqua souaeuI-Ioulsou I new Iqu In voeeuIII IuIa quoe Iaau mu .uIqu uII Ia I» III: Incoqus quuu~I Isa no IsIIo Isa oII IIIDII no cause Ia quuana Iau uIsu uquuIe Iau auou cu anon—a Ia suaoa IIuas Iuau .quuana Iau on IuathII Isa u:¢AI oIoweneIoo uooque uoIoII—I>o00 I u— _ mud :.vooe In ecuuuveou nos: .Iao—~0u II u—eou o» vquoaII In suzou IIusc Iusu .qua - .Iuoguo ocI e Ia: u304I IuII .Iouuuecou oooe cu weI IpquuIeOque Iu as: IIIo: I no osIAIsa Isa nu unoI cu I¢0uu0qu ~Icouuolo quIA .9 IuanoeIoo Axu._ adequao Incl Isa onus-cue» cu nuuuuaI IA» IqueIw Idea «Icoquuauue quDOI ope ~I=0uu9II uqsuuuuuo Iquuuanu no: Iooo an uIsu ease In IuIauo new: quIIaua .uIaueuI Iu u=0I0>~o>=uuu~II «Inauuolo an .uu uaoae IoIIu oI ecquI usoguua Ioaeua uquuIe Isa II as In QIDUIAII Ia oaoou .quuI eququ I Ia Icon ucIIuIqu mes ecu>Ia cos: IquIIu oz: DIIuDIa I 0» Iluu ququ Isa sou ooceuIII Inn: IIuao Iuau an WN. . .Ioueuu see quIIquow 4004 cu caucus-I Ia finaou .ADOqus «quoul uIIs woe Iloualhe no: no uneoooouuoou ocI huoueIu IAqu 0:: uooque I eouuuuloI Ina: Ieuao ease an .Ioau ou enunquuII Ie.vq:oa sous: IoeoIau eququ IuII ooI IIouI 0;» soul Iouaoua Isa IIsquv cu IIuos Iuou uuusxo ouao: .ouoeuIII IuoouaIa Isa gnu: III—none uuoaaIu no IIquIa Insole-«III . oecIauxo ousou hose nu .quII~ III» II eouu0I .IIuoo Iuau sou-I IovuI oeu nu Ayn _ :.e:uocaluo: In cu vouuouacou I. oz: acouuea sous-u I—Icuauou I saw) u—Iu ocI once Dun cu .ouounuoo uo>0=usa .vouuoexo In o—aou quoo quh m\._ .A.u.o .uuecI .cOququouu .DIIu .nquon ....Iv qu:uo ocI an». :« Icouu0IIu quowuolI quIn no monococIOU I:Ou>ao Incl Isa II nae) II mxu.m AWIIIIIII InuaaI 0;. uuu=woouu on sou—uan 0:» euuneeu Iva» “Icoqusqu ~quoa scI ~I:0uu01o unauuuuuo IuIIIIue . .P gonna use nausea» s—co ac: «a uIau nous-«Ian oeI flag—3):— :ouuauue OI I. qup—DD 4:3 sofa—.0: 360.2503: m,_coE>—o:u 1:... .5355? 0;”. so odour—0.37.... ..o:::=_ ... .3: 753.3... was-332.3 I. 305 :2.) I03. .1 III" I... :— wo>=;o; I Amcoxuo3Icu 3:: .mou__Ec_ .mucmuac: ;s_3v m;_:%z:_%<4:z zII~ cu Ieuoo Iusu uuoano snap: and .uog ueIu IuI IAIOI Isa uIsu IssI us ca .Iouo cu :IuIOI ac: IaooquIOu: Iaue cu oooeqeee «a IIuso Iusu IIu>uIeoI ou I>I= v—ooa and .IIuo: .IIIuoo co InouIceuI Iva—uou cu nquBOuIquo IIuoo Iuau oculuu cu I>Ig vase: mud .28 I53 “sauna: okay-o Icuuaou I no IouquADUI Axu.. ATIIIIIIIIIIIIIII quOuuuooI IquuIvos on voI .eoueupueaso III—u usoauu: Iqu Iguanas no IoququI - snoququuII IqueaII ch quuleeuooI I=I Inovuo use squu cu vouuoaxo In eIu .ooueouo Iu Au. uu log) new Iuooque Iuogu cu IDIu qua Isaaauuuol I>ue cu IIuao Iago uuoauo swoon 4 . .1 mm. .couI.>uoa:I goose.) OI oo cu :o oouuou Ia euaou .A-ocouuqu>Iav ooseo>oa ecuauouou acoque I so quooul n nuo>o Iousaaoue soc—a Queue» scI Iqu cu sou-I um on; .oIo~ use: no IIIuouIeIu ..u ecuuusvou Iuooque no. ooeuezuou uuuaIII quueII IvanauquI «so hquu cu IIuao Iuau uuueuI eases .oluuuo>o hose so eousIuI IoIIl nu ma cI>I sIo uIsu ea 09 cu oouuuexo n\._ on snoou .sav DIA:Uuaqu I so qu: I.ucouuas I coolesI cu song-cue ecu>qs .IIuaa Iush . .o—OD ecuoa usosuu: Inoqu seque> I _o «saga—unuocoanou ~I=0uuuvoI Ila-II ou axu.m AmIIIIIIIII.I . scI .:ouuu>.oaso uaoguua Iqu eequso no Iouooceue and: :«IuouIl oeI Iuooloeueee h. ace anyone “so squu cu so oouuou Ia eIu m._ss.usumccamac m.____:_ as: matcczaum sue; msscscsms I mmszmsosezusumzou 150 . \ -1- .3232; 0:3. I: >2: :2: Axe O I Iucouuea cu ouncecu Isa—>90 I>hoeuo on In—IL ecu—«Iaoeu :I lean Isak: 0 so .quoaqu mm A. . u an no ooquIIIu I queeooou cu II-oo Iago uoIexI no: case: .89- 92.125 I Iu Iueuque 0.5 oIIauIa IIuooeIuII no Ice: Iu-uoo .5 II»:... I...» 32:; «o: 3:03 and ..voo. no Iehu uI~:u«u-Ia I 92.2525 II>II— :25qu25“. 2.0:!— I uIau ICIIAI. cu quuaII In no: 325 II»...- Iuah _ mud 0.: on; I a .uelowuB-I 8.— 2.0qu cu Ive—3:0 25:50 .Ufl.dfl.§ 0‘“ I. “H03 .- 0~u£30 ”nan HO '9. .0.U..8 DIuvnIu I In III—VI 233.053. .0 Iva-III: I3» IPIIIJI 0» I93: Idea :0st 3.53 11 . 4.9: .23qu 3:3 .IIIII—e .Iouuuvel :35: Iva—LEI 2.33: no IDs-SDI uIIouuuooI sou QIII I£u IIDIIAI 0» I95... I13 noel-II 3.53 mm .. on; .3; I; so :uuuoe 9:: .253 on .30 c3... 9.3 :14qu oIIququ IIquuIeolIo 51. I... «0 Ion sea .0 use :02. II... 9.: :5: to. ca 2.0:: CauI—aiI ...I :5 0:02... 3 oouuoexo I.- 3:00 093.. IZR Q. ._ .Iucoque cu cusses; cu IIII IDODqu> sonsuuuune :02... 30:0 .Icouuolo 0.2 03:30 o» 039093 I: Esau 09:... I13. Jon—Ian)!— I .30. .3 3:2: 32.0qu.— quuuaaoc 3:929 I o. Inca—3 I 0:02... 0» $33....» In 3:09 00.5: It... 1! .uoulcoau>=o I... to... eucuqum c.- Iona—Isa 8.N AI 35:36 .22 I: :0) I: 03...; I...» IIEIIAO a. I . .Isou usumc:: .;;__: .::_5 zoucs moucoco oz: m::_s;: moon I rk.4_=< 4mmmmo .APPENDIX G TRAIT RATING SCALE 152 Name Hospital Date PERFORMANCE RATING SCALE For each quality listed below, please make a rating of the videotaped nurse's performance by circling the appropriate number on each scale. Consider and rate each trait independently. 1) 2) 3) 4) QQMEQSSIQNAIE - for patients and their families 5 4 ~ 3 2 1 exceptional superior average satisfactory unsatisfactory HELEEQL - to doctors, nursing staff, and patients 5 4 3 2 1 exceptional superior average satisfactory unsatisfactory EBQFICLENI - well-versed in knowledge and skills needed to meet nursing needs of patients 5 4 3 2 1 exceptional superior average satisfactory unsatisfactory 5 4 3 2 1 exceptional superior average satisfactory unsatisfactory 5) 6) 153 CDMflflfilQfiIlgfi - in interactions with nursing, health care staff, social service professionals, and patients 5 4 3 2 1 exceptional superior average satisfactory unsatisfactory EEBEEEIIEE - in observing changing in patients and circumstances which eight affect nursing care S 4 .3 2 1 exceptional superior average satisfactory unsatisfactory APPENDIX H LETTER SENT TO HOSPITALS 155 MICHIGAN STATE UNIVERSITY DEPARTMENT OF PSYCHOLOGY EAST LANSING - MICHIGAN - 488244 I I7 PSYCHOLOGY RESEARCH BUILDING December 1, 1984 Nurse Title Hospital Name Street City, State Zip Dear Nurse, We are involved in a large project which addresses understanding and evaluating performance appraisal systems. As part of this project, we are seeking volunteers from hospitals in Michigan. Participation in this research project requires nursing staff members to attend two separate sessions, each approximately one and one-half hours in length. These sessions are designed to collect data on performance appraisal and to train nurses in the use of a performance appraisal instrument. The advantages accrued to participating hospitals are performance appraisal training for personnel and perhaps the implementation of an evaluation instrument if the hospital is not currently using one. All sessions will be conducted on site at the hospital at a time of mutual convenience. The costs of conducting the actual sessions will be covered by the researchers. Attached is a more detailed description of the research project. If you have any interest in participating in this project, we would like to discuss it further with you. We have discussed this project with the Michigan Hospital Association and have received their permission to contact you. They have expressed an interest in the project and believe it will contribute to their ability to provide personnel services to associated hospitals. We are providing them with a final report of our research which we feel will be of interest to the association. MSU is an Affirmative Action/Equal OpporIum'n' Insumn'un 156 We hope you will agree to help us by participating in this project. We will be calling you in a week or two to get your reaction and discuss this further. Thank you for your consideration. Sincerely, 5AM Daniel R. Ilg n John A. Hannah Professor of Psychology & Organizational Behavior 2122 Graduate Student . Industrial/Organizational Psychology 157 PERFORMANCE APPRAISAL ACCURACY Purpose Rationale In many organizations, supervisors are required to rate the performance of their subordinates on some standard form. These ratings play an important role in providing develop- mental feedback for employees and in guiding personnel decisions. Underlying the use of such ratings, termed performance appraisals, is the assumption that people can provide accurate ratings of the performance of others. Yet, we know that some people can rate accurately while others cannot. The primary goal of this research is to identify those who can rate performance accurately and determine why they can do so when others cannot. Hypothesis. One reason that some people may be better at rating performance than others is that the better raters may use more appropriate standards for judging the performance of others. Specifically, we believe that good raters judge workers on the job against job-relevant standards whereas poor raters use more global standards that apply primarily to people in general, rather than primarily to people on the job in question. Thus, the purpose of this research will be to identify tha.nurses who rate performance accurately, and then examine the types of standards they use when appraising nurses. One implication of this study is that if we are able to delineate the processes underlying making accurate ratings, we may be able t0' train inaccurate raters to become more accurate. . CONDUCT OF RESEARCH The research will require each participant to provide two relatively separate types of responses. First, using question- naires, nurses will be asked to report the basis on which they describe and evaluate other people. These responses will be used to identify nurses who describe others in terms of job-related standards and those who tend to use general standards. Next, the nurses will be asked to watch some video-tapes of nurses working on the job and evaluate the performance of the nurses seen on the video-tape. A standard performance appraisal form developed for nurses in hospital settings will be used for rating the performance of the nurse on the video-tape. Nurses will be identified as accurate or inaccurate raters based on their ratings of the video-taped nurses’ performance. Nurses’ accuracy in rating the video-tape will be compared to their responses to the questionnaire items. Data will be collected by the researchers from Michigan State University on site in the hospital. Approximately one and ‘one-half hours of the participating nurses’ time is needed to complete the study. Nurses participating in the study can do so individually or in small groups. The size of the groups and the 15.8 times for participating are flexible; we will be willing to work around the work schedules of the participants. RESOURCE NEEDS All costs for conducting the research, questionnaire preparation, data analyses, and report writing will be covered by the Michigan State University research group. Hospital staff will be needed to help the researchers identify nurses to participate in the study and to aid in scheduling them for the research sessions. It is our aim to identify a sample of 100 or more nurses for the study. PRODUCTS A report will be prepared by the research group specifically for use by the participating organization. This report will discuss issues related to performance appraisal accuracy. In addition, the researchers will be willing to present a workshop on general performance appraisal issues as well as on issues related to making accurate performance appraisals. Prepared by: Daniel R. Ilgen and Cheri Ostroff Department of Psychology Michigan State University 129 PRB E. Lansing, MI 48824 (317) 355-7503 APPENDIX I CONSENT FORM 160 WEBB! This session is part of a large project investigating issues in performance evaluation conducted by Ms. Cheri Ostroff and Dr. Daniel R. Ilgen from Michigan State University. The purpose of this study is to examine some ways in which nurses evaluate the job performance of others. Performance ratings play an important role in providing developmental feedback for employees and in guiding personnel decisions. Underlying the use of such ratings, termed performance appraisals, is the assumption that people can provide accurate ratings of the performance of others. Yet, we know that some people can rate accurately while others cannot. The primary goal of this research is to identify those who can rate performance accurately and determine why they can do so when others cannot. ‘ During this session, you will be asked to complete three questionnaires which investigate the basis by which you decribe and evaluate other people. You will also view a videotape of a nurse performing her job duties and you will be asked to rate the videotaped nurse's performance on a standard performance rating form. 7 All information you provide will be kept in strictest confidence. No one outside the Michigan State staff will see your individual responses. Only overall information will be summarized in a report: no indivudal's responses will be identified under any circumstances. You are free to withdrawl from this study at any time without penalty. Partigpag§;g Stagemggt: “I agree to participate in this study as described above. My participation will involve filling out the questionnaires and performance rating forms. I do this realizing that the researchers agree to keep confidential any data I provide by not releasing it to anyone in a form that would allow it to be identified as mine.“ Signature of Participant Date APPENDIX J EXPERIENCE VARIABLES FOR NURSES 1&1 162 PERCENT OF NURSES IN EACH CATEGORY OF EACH EXPERIENCE VARIABLE Variable Percent Years worked as nurse: less than 1 year 12 1 to 4 years 82 5 to 10 years 30% 11 to 20 years 352 21 to 30 years 18% over 30 years 92 Position: Staff Nurse 32 Charge Nurse 35% Head Nurse 22% Supervisor 16% Other 24% Title: Licensed Practical Nurse 1% Registered Nurse 91% Nurse Practioner 1% Other 52 Sex: Male 32 Female 97% Unit working in Hospital: Intensive Care 112 Emergency 4% Geriatrics 0% Surgery 122 Psychiatric 32 OB/GYN 17% Medical 13% Children 22 Other 372 163 Educational Training: Community College (2 years) Hospital (3 years) College (4 years) Highest Educational Degree: Associate Degree Bachelor's Degree Master's Degree Ph.D. Prior Rating Experience: Yes No Years of Rating Experience: No experience less than 1 year 1 to 4 years 5 to 10 years over 10 years 33% 47% 20% 60% 32% 8% OZ 13% 52 332 25% 23% APPENDIX K RAW DATA AND CODING INFORMATION 164 Column 10 - 11 14 15 16 18 - 25 165 Coding Information for Ba! Data Cece yecieele and Description 1 1 Subject Number (001 - 129) Hospital (2 = Hospital 1; 3 = Hospital 2; 4 Hospital 3) Years worked (1 = less than 1; 2 = 1 to 4; 3 = 5 to 10; 4 = 11 to 20; 5 = 21 to 30; 6 = over 30) Position (1 = staff nurse; 2 = charge nurse; 3 = head nurse; 4 = supervisor; 5 = other) Title (1 = LPN; 2 other) RN; 3 = Practioner; 4 = Hospital Unit (01 = intensive care; 02 = emergency; 03 = geriatrics; 04 = surgery; 05 psychiatric; 06 = OB/GYN; 07 = medical; 08 children; 09 = other) Training (1 = Community College; 2 = Hospital; 3 = College; 4 = Other) Degree (1 = A.D.; 2 = 8.8; 3 = M.S.; 4 = Ph.D) Sex (1 = Male; 2 = Female) H ‘< I'D III Rating Experience (0 = no; 1 Years Rating (1 = less than 1; 2 = 1 to 4; 3 = 5 to 10; 4 = over 10) Role Grid Scores Items 1 - 8 (1 = Behavior; 2 = Trait; 3 = Other) 166 thgvior Grid Scores Behavior 1 (0 = item not placed in dimension; 1 = item place in dimension) 27 1 Observational Ability 2B 1 Health-Conscious 29 1 Outgoing 30 1 Conscientiousness 31 1 Sense of Humor 32 1 Organizational Ability 33 1 Knowledge and Judgment 34 1 Appearance 35 1 Family-Oriented 36 1 Skill in Human Relations 38-47 1 Same as Col. 27-36 but for Behavior 2 49 — 58 1 Same as Col. 27-36 but for Behavior 3 60 - 69 1 Same as Col. 27-36 but for Behavior 4 1 - 3 2 Subject Number 5 - 14 2 Same as Card 1, Col. 27-36 but for Behavior 5 16 - 25 2 Same as Card 1, Col. 27-36 but for Behavior 6 27 - 36 2 Same as Card 1, Col. 27-36 but for Behavior 7 3B - 47 2 Same as Card 1, Col. 27-36 but for Behavior 8 49 - SB 2 Same as Card 1, Col. 27-36 but for Behavior 9 61 - 70 2 Same as Card 1, Col. 27-36 but for Behavior 10 1 - 69 3 Same as Card 2, but for Behaviors 11 - 16 1 - 69 4 Same as Card 2, but for Behaviors 17 - 22 1 — 69 5 Same as Card 2, but for Behaviors 23 - 2B 1 — 69 6 Same as Card 2, but for Behaviors 29 — 34 17 21 25 26 27 69 11 15 19 EU woman m to CD [I] m (I) m 167 Same as Card 2, but for Behaviors 35 - 40 Subject Number Knowledge and Judgment Organizational Ability Skill in Human Relations Conscientiousness Observational Ability Compassionate Helpful Proficient Efficient Communicative Percpetive 8888 a? §§ ii 88 1 MN” 8 ‘ §§ §§§§§§§88888888 168 0100000000 0000000001 0000000100 00000000 0100001000 0010000001 0000000100 100000 8 10000 0010 0010000000 0000000010 1 0010000000 0000001001 0010000001 10000001 0001001000 0100000000 1000000001 0000 0000001000 0001 1001000000 1 0010000010 0010000001 0000000100 0000010000 300%010000 0010100000 0001001001 1001000000 0100000000 0010001001 0000000100 00000 0000 0011100001 0010000001 0001000010 1001001000 0011000011 0000001001 0010000001 0000000001 0001010000 0100000000 1001001001 0000010000 1001001000 0001000000 1001001000 1 0000000010 0010000001 0001001000 1000 0000100001 0000000001 1001001000 1000 0100000000 0011001001 0000000100 0000 0001100001 0010000001 0000000010 1000000000 0010010000 0000001001 0010000001 0000011000 01 1 0000010000 0000001000 0001000000 0001001000 0000000010 0010000001 0001001000 0000 0000100000 0000000001 0000001000 1001000000 0100001100 0010001001 0000000100 169 .WWW-W ..-,WWW W.-W WW W. WW W W.W WW W.--WWW ..WWWWW W.WWWWW W.W WW W-WWWW .. WWWWW WWWWWWW WWWW WWWWWW WWWW WWWWWWW WWWWWWW WWWW-WW WWWWW WWWW W m-me W-mmm WW1WW WWW m W--WW WWWWW W-Wmmmm WW-W-W W--WW W-mmm W -- , - --- -- --- - --- - - - - , - ,- -,-.--- , - , - - , ------- -- WWW WWW WWW W W WWW WWW . WWW WWW WWW W W WWW? WWWWWWW W WWW WWW-WWW. W WWW WWWWWWW WWWWWWW WWWWWWW WWWWW -WWWWWW ..W WW -.wm WWW..- , W..- WWW- W, WWW-W ..W. WWW WWW-W WWW .W. WWW WW .WW WWW .WW WWW .WW W W .WWWWWW .WWWWWW W.WWWWW WWWWWWW .WWWWWW .WWWW W W. WW-WWW WWWWWW WWWWW W WWW WWW WWWWW-W WWWWWWW WWWWWW. W -W-W WWWWWWW WWW WWW WW W.WWWW. WWWWWWW-WWWWWWWWWWWWWWW1W-WWWWWWWWWWW.WWWWWWWWW,WW.WWWW,WWWWWWWWWWWWWWWWW WW WHWW. , ---WW,W-W,WW -.-.WWWWW-W,WW ,HW WWWWWWWWWWW,- W- WWWW-WWWWHW-WWW-WHWW W.W-... - .W.WWW -WWWWWWW..W.WWWW.WWWWWW.-.WWW.WW.WWWWWmmW W.w WW .WWWWW. WWW WW.M W. W... WW... WWW. WWW W. W .W W... .WW WWW-W. W WW W WWWW W WW W WWW WWWW W WWWWW W WWWWW W WWHWWWWWWWWW W WW WW W WWWW WW WWW 11.111112222222233333333444444445555555566666666777777778888888899999999000 2 00000000000000000.000000000000000000000000000000000000000000000000000000000 170 1WWWWWW WW WW Wm ““ W-.WWW. WWWWWWW 1001 1001 211 1000000000 0100000000 0010000001 00000001 0100000000 0000010000 0000100000 0010000001 000000001 02410GXH1GX)OGXHOGXD KXDGXDGDOOKKDGXD KXDGXD0100KDGXWO 11 1 11 1 11 0 W00 0 ...OW. 1.1. 1 “WW1 1 1 1 n W1W W... WWW. ..-WWW WWWWg WW nu WWWWWWW WWWWWWW ‘11W111 11‘1i“ 1101011000 0010100000 0010000000 0000000010 111100000 1100000000 0001001001 0000000000 0000000000 0001000000 0000000000 1000000000 0100000000 0000000000 1001000000 0100000000 1001001001 ooooooom 028 0000000010 0000000000 0000010000 0000010000 0010100000 0010000001 000000 1000000000 0010000001 0000000001 001 @1000 0010000001 0001000000 0100000000 1001M 100 0000010000 0001001000 0001000000 1001m 10000000 0010000000 0000000010 0010000000 0001 00000 a0001011000 0000100000 0000000001 0000001000 W10000 0001000000 0000100000 0000000001 0000001000 00000122 0001000000 0100000100 0010010001 0000000100 W13 1 4 4 1 3 4 0401 1 104 4 1W 1 021 5 2m1wwmm4m1 1WW52 WW ..WWW W... W... . ... owmm1zom1m11 22 024 0000000100 0000010000 0000001000 0010100000 0000000001 0000001000 024 0000000010 0000000100 0000010000 0000001000 0001000000 1000001000 024 0010100000 0010000000 0000000001 0000000010 0010000000 0000001000 027 0000100000 0010000000 0000000001 0000000010 0010000000 0000001000 024 0000000100 1000001000 0010000000 0001000000 0100000000 1000000001 W... ...W.. W.. W .W....-. W... W... . W . 0200 1 02 0001 12 11 02 04 0 .m1nm1mmm11 cum-m. m ou111nmnm an «.11-Hmmm.... 5W 1 5 1 7 11 15 7 WW 0231 W 1.11 W 13 2 fW OOWW 1.11.. .1. -3 W 111111222222223333333344 4555555556666666677 778888 222222222222222222222222 2222222222222222222 222222 000000000000000000000000 0000000000000000000 000000 3889 2222 0000 10 .....W..... WW... 0000000100 0001000001 0010000001 100001000 0011000001 0100000 0010000001 1 1 0mm 111““ WW... WWW WWWmWW-W1W1 “mowommwun W .‘1111111fl1 A 43W1111 42 2 31 O31 1 151 11 o1 “ WW-.. WWW. o1111om .12nXXUO1m 999999900000 oooooooooooooooooo 171 WWW W-W-WWW WWWWW. W. WWWWW W... WWWWW WWWWW. WWW-WW. W---WWW WW-WWWW WWW. ...W- W. WWW WWW W WWW W W. WWW WWW-W WWW WWW .. .WW 1 1 1 1 1 1 1 111m111 1111 11 11H 11 1 1 1 111 11 111w1 1 111m111 W WWW WWWW W. W. WWW... WWW WWW. W... .W .WW WWW WW-WWWW WWWWWWW WWW-WW W-HWWW- -.WWWW-H WW WWW-W WW WWW WWW WW-WWW. W W --W W W--- - W--- WWn-W-W -W W--W W-.--W W W---W WWWW-WW WWWWW-W WWWWWWW WWWW W - - - - W - --W- - W, - WW - - W- - H - - -W - WWWW - - - WWWW WWWWWWW WWWWWWW WWWWWWW WW-WW WW WWWWW-WWWWWWWW WWWWWWW WWWWWWW WWWW-WW WWWW WW.WWWWWW......WW......W,.WW. WWW WWW...-WWWWWWWW WW-WW-WWWWWWWWWWWWWWWWW WWWWWWWWWWWWHWWWWW- WWWWWWWWWWWWWWWWm WWW WWWWWWWWWWWWWW W-WWWW-W W WWmm-WW WWW WWWWWWW- WHmW-WW WWW-W WWW -WWWHWW WWWW -Wm-W WWW W WWWWW-WWWW-W WWWWW WWWWWW-.WWWWWW WWW WWWW m-WWW W W.n-WWWW.n- W.WW WW WW WWW W -- -WW ---WW -W- -W-m--H--WW -W W--WWW W W--WW WW-WWWWW WWWWWWW WWWWWWW W .WWWW WWWWWWW WWW WWW .W-WWWW - WWW W WW WW WWW-WW W WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW 172 m mm1wm 111 mm‘11 1 0 1 1 1 001 00100 00000 00 00 00 0000001000 1 0 1001000 0100000000 1000000001 0100000 0010000000 00000000% 0000001 0010000001 0010100000 0000001000 0001010000 1000001000 000000000 0010000001 0000000108 m mWWM1 mmwwwwm mmmmwm mww1m12 mmmm1nm 11 mmm 11 m m1mflwmmw.0mn’1WOXXUo A912 41.11 111m “1111 11031 2222211....22. 22222.2... 2.2.. 2.2-2.2. 0000000010 0010000000 0000001000 0000100000 0000000001 1000000 0100000000 1011011001 0000000100 1001 1001 1001 1001 1000 1001 100000100 0011000001 ooooooom 011000 0010100000 0000000001 000 000 001 001 11 00001 110 0001 0011 1000 1000 1010100000 00000 2.222 wm1m 11 m 11 mflwm 4AU4 O11 “4'11 1AU41 22222.22 mmmmm .2. W W W. W2 2- 1000000000 0100000000 0010001001 0000000100 100000000 0000010000 0010100000 0010000000 0000000010 0000100000 1000000000 0010000000 0000000001 0010000000 050 0000000100 0001001000 0010000000 0001000000 0100000000 1000000000 134 WW 212 221222 1000000000 0100000000 0000000001 0000000100 @10100000000 0000010000 0010000000 0010000000 0000000010 048 0000010000 0001000000 1000000000 0001000000 0000000001 0010000000 200 554344 71 0010000000 0000000001 0000000010 0010000000 1 50000010000 0001000000 0000100000 0000000001 01 00 00 100 100 050 0000000010 0000000100 0000010000 0001000000 0001000000 0001001000 048 0000000010 0000000100 0000010000 0000001000 0001000000 1000000000 048 0000100000 0010000000 0000000001 0000000010 0010000000 0000001000 041 0000000010 0000000100 0000010000 0000001000 0001000000 1000001000 048 0000000100 0001000000 0000000001 0001000000 0100000000 1000000000 041 0010100000 0010000000 0001000001 041 00000001 %0000010000 0001000000 1m m 0 071 172 1o. m 1222 “MHZ” 14121NAUOO 2112 .11H”.1m MW1M WW2 1 mmmmmmw 2 om 11mmm211m112m mm mmmomm11wom1 m WW1& 1 1 OW1mm1meOW1oo %WOM1m4 52 OW11030 “W 212 2221mmmm2 221111222222 2W1221mammm21m22 21221222WW2£W 2122122 m12 1 dIJI .2 2.22-.. 2-2.... 2 2.12.... .22 22... 22.22 2.22. 22 AYZWOV 1 may 1 n:UO1.n22m1n¥Zonv 1 41.12 fix‘ 0 41 1 m 2M11 m 2211 m smwwnnm 221M11 snwwm m 22W1m2 22 W11W 2211 m 221m n 22mm n2mm ”211222 22 1 mm 2211m m 221mmmm 22 n2 1 mmn2 22mm 2221 m 44444444444444‘444444444444444444444444444444444444 4444444444555 55555555 .OOXXUOAXUOfiXYOnXUOfiXUOnXUOfiXUOfiXXQOnXUOfiXUOfiXUOfiXUOfiXUOfiXUOfiXUOfiXUO AUOfiXUOfiXUOfiXUOfiXU nXUOfiXUOfiXU 173 omam WWWWW ‘11‘ WWWWWW% 11104111 W WWW W W WW- W m-m-m QIOQI 1| WWW 1001000000 0100000000 0010001001 000000000 0000010000 0000100000 0010000000 000000000 0001000000 0010010000 0001010000 0100000000 1000000001 ‘11 111 WW-WWWW WWW. W m m 1132 1000001000 0100000000 1011001001 0000000100 WW w-WW- W w W W W W WWWWWWW WW- W WWWWWWW mnmmo WWWW WW WWWWW m111‘ WWWWW-W '11 th www ‘11 WWWW WW WW WWW W W - -W-W W- W mmmm w mmmm 11‘1 000 0001000000 1000001000 00000 0010000000 1000000001 0000000010 0010000000 0000001000 WWW WW W. W w W mmw W W - WWW WWW , W HmmW m 1 11. m 1m 1 m 1 WW W W WWWWW W WWW WWWWW W-WW WWWWW W WW -W WW WWWWW W WWWWW WW W W WWWWWW WWW WWWWWWW WW-W WWWWWWW WWW WWWW 3W WW WWWWW WWW- W WWWWW--W W WW- WWWW WWWWH WW WWWWWWW WWW WWW WW- WWWWW W WWWWWWW-WW W-WW- W WWW-WWW-WWW WWW WWWWWW-HWWWW WWWWWW WW W-WMWWWWWWWM WWW-WWWWWWW WW-WW W ,WWWWWW WWW WW-W-WWWW% -- -W- WWW-WW--WW-WW - WWWW,W- ,,WW -W m1030m WWW 0mmW01mHm m MAW0H1WNW1W1AMWW mm 11011020 m M10mmm mmmsmmm W-W-WWWWW W-WWWW WWWHWWWWWWWWWWW WW -WWWWW-WWWWWWW WWWWm -W-WWWWWWW 9WWWWW WW WWW WWW- : WW W W W. WWW WWW-WWW. E-WWWWWwWWWHWWWmMWW-WWWWWWWWWWWmWWWWWWWWWW WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW 064 0000000 5 0000000100 0000010000 1 7 100001 174 .s q 0’ M 01 d 43 do. a d“ ‘0. 8 n K §§§E§§§§§§§§ g§::§§§§§§:g§ g 2%; g 2 §§§ 8 § § § 00 100001 175 075 200 175 1 5320921214‘1111 0 0000010000 00001 ..0 0000000010 0000100000 175 075 175 175 4120621220 3223221 0000001000 0100000000 000001 0000010000 0010100000 0000000000 0000000100 0001000000 0010000000 0000000010 0000000100 0000010000 0010100000 10000000 000000000 0000000100 0001001000 0000 175 025 175 17 200 544134 552092 21411 111113 1 1000 0 00000 1001 0011000000 1000001000 1 0010000000 0 figs é 0010000001 000110000 0001011000 75 175 444344 11121221 1 i??? a§§§§3 §§§§§°° i i ‘ § Rig—b § 6"“ ° 3 2§88 g 38§§838 03 a 0 0011001001 0010000001 0000000001 0100000100 0101000000 0010000001 0001010001 1011001001 0000000001 000000 0001001001 0011011001 0010000001 1011001001 01000001 E 175 '0‘ .5 b b 001 0100000100 1011001001 10000000 8 -.Egp 8 §§§ 9 §§ gaé on g? 9 :égéégg é éfié §§§ 3%? $5 33% g .. § 3 9 § § § 0000000100 00000 175 050 200 200 175 444245 3 632042 214 2221 1000000000 0000001000 0100000000 1000001000 1000000000 1 4 111111 1001001000 0100000000 1011011001 0010000001 0010000001 0000000001 0000000100 100 011000 0010000001 0001011000 0100000000 0000000010 100 0000010000 0000001000 0001000000 0010100000 0010000000 1001000001 0000000010 0010000001 00000001 1000000001 ‘ §T§ § § § § 0010000001 0000001001 1000 0100000100 0001011000 0010000001 0000000011 0001011001 0001010001 0010000000 § 3 2 § § § é a ‘ ‘ § i g; g 8§§ 3% 33% §§§ E 100000 1000001000 00 0000000001 010000 0010100000 0001000000 0010100100 0000 1000000000 0100000000 0010100000 0010000001 0000000001 0000000010 0000000001 0000000100 0000010000 0001010000 0010100000 0010000000 175 100 150 175 175 444244 3 4320711220 21121121 1000001000 0100000100 0001001001 0001001000 0100000100 0000011000 0010100000 0010000000 0000011000 0010100000 1 001 0000000000 0000000100 0001001000 0010000000 0001000000 0000000000 0000000010 0000000100 0000010000 0000001000 0000000000 0000100000 0010000000 0001000000 0000000010 0000000000 0000000100 0000010000 1 0000100000 0000000001 175 125 200 175 200 544245 4 6320832214 11121211 1 0100000000 0000000001 0000001000 0100000000 0000010000 0000100000 0000000001 0000010000 0010000000 0000000001 0000000001 0000000100 0001000000 00 0000000 0001000000 0100000000 0000000010 0000000100 0000010000 0000001000 1000000 0000100000 1 000000000 0000000010 0010000000 0000000100 0001 001 1 175 100 175 175 150 444344 4 532082 220 311 1001001000 0100000000 1011001000 1000001000 01 0000011000 0010100000 0010000000 0001010000 0010100 1000001000 0010000000 0010000001 0000000100 0001 0010000001 0001000000 0100000000 0000000010 0000000100 0000010000 0001001000 1000000 0000100000 001 1 0010000000 100 0000010000 0001 0000100000 1 150 075 175 175 175 444344 4 532092 214 23221122 11 0100000000 1001000000 1000 0100000100 000001 0000100000 0010000001 0000010000 0010100000 1000001000 0010000000 0010000001 0000000100 0001 001 1000000 1 0000000010 0000000100 1000010000 1000001000 0000010000 00001 0010000000 0010000001 0 0010000000 1 1000010000 0001001000 0000100000 000000000 175 100 175 175 175 444344 4 4320711213 11111111 0000000000 00000000 $132? 9 § ..u. g E g 5 75 175 175 175 434454 4 4220921212 11112222 1 1000 01 0000010000 0000001000 0000100000 1 0000000100 1000001000 0000100000 10 0000000100 0000010000 0000100000 001 1000000001 100 1000000000 0000000 0000010000 175 075 175 150 175 434454 4 1320411211 11111111 000000 0000000 150 050 175 175 0000000 150 075 175 150 150 433144 4 4320121214 11212222 1 §2 ‘3 E? 3% § E g 2 111% 51% §§§§§§§ 11% §§ §§ §§§ §§§§§§ 3? °§€3§° g §§2§ §§§§§§§ § § § 177 .W WWW WWWWWWWWW W...WWWW WWWWWW W.-WWWW WWWWW WWWWWW WWWWWW WWW-WWW WWW-WWW W. WWW WWW WW WWW. ...WWW WWW WWW WWW-W WWW W... W WWWWWW WW WWWW WWWWWWW WW -WWW WWWW- -WWWWW WW- WWW WWWWWW WWWW WW WWW-W- W W. .. ,- - WW WWWWW-W WWW-W WW WWW WWWW WWWWW. WWW-W W W - W W W. W W. W W... . Ww W. WWWWWW WWWWWWWW -WWWWW WW WWW W WWWWW W WWWWW W WWWWW W WWWWWWW WWWWW WWWWW W WW WWWWWW WWWWWWW WWW. WWW WWWWWWW WWW-WWW WWWWW-W WW WWW WWWWWWW WW WWW WWWWWW WW. WWWW-WW-WWWWWWWWWWWWWW,WWWW-WWWWW WWWW-WWWWWWWWW- WWW-WWWWWWWWWWWWWW-WWW WWWWW W --- -W W ..W W WWWWWWWWWWW-WWW WWW WWWWWW-WWWW -WW WWWWW--WWW-WMWWWW -WWW -W -.W- - -W.W WWW - -.m WW -W -W -W -W WWW - WWW WW W-W WWW WWWWWWW WWW. WW WWWw «WWW W.WWWWWWWWWWW-W-W WW--WWWWWWW WWWW W WWWWWW W-WW. W - WW W W-.W 1 o 2 mmmm mo m 72 m o- o WW-WW-WWWW WW-WWWW W WWWW-W- WWWW-W WW WWWWWW WWWWWWWW -WWWWW W WWWWW- -W-.WWW WW WW- WWW WWWWWW WW WWWW W WWWWW WWWWWWWWWWWWWWW WWWWWWW WWWWWWW W WWWW W WWWWW W WW WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW 178 W W WWWW W WWW%1WWWW W W11 W10 WWWW W-W WW W WW -W WWWW W1W WW 1 11 m W1WWW WM. WW 00 00 0100000000 0011001001 ooooooom w10100000 1000001001 0010100001 0010000001 0000000010 0010000001 0010000001 0010100000 @0010010000 1001001000 0100010000 1000001000 m000001oooo 0000001000 0000000000 1000001000 10000000 1010000000 0000000010 0000000000 1001001000 08000000100004 0000011000 0010100000 0000000001 0000001000 0000000010 0010000000 1000001000 0100000100 0000001001 0000000100 W10000 0001001000 0100001000 0000001000 1000 0010100000 001100100 41000000000 0100000000 1010000001 0000000100 100000 0001010000 0100000000 1000000001 010000 0010100000 0010000001 000000000 $000000 0010000001 0000001001 0010000001 110000010000000010 001000000 W W W W. 000 1000011000 0010100000 0000000001 :1010000 0001010000 0010100000 0000000001 0001001000 m51 m1m M1WW1515201WWW1M10WWW W W 1W OWWWW W WWWWW1W W WW WW WWW WW W - WW WWW W WWWW WWW W W W WW. WWWWWWWWW-WW WWWWW WWW WWWWWWWW W,W WWWWWWWW W WWW WWW. WWWW W... W W... WWW-W .. WW. WWW 1W1 WWWWW W1WWWW 1.13 WON 50! 000000 .WWWWWW- .. WW 102 0000010000 0000100000 1000000000 0010000000 0000000001 0010000000 40 030 WWW WWWW.W% ..WW.Ww WWWWWWWWM WWWWW WWW. WW.WWW WWWWWWW WWW.W- WW WWWWW WWW 1 WWW WWW W W WWWWW 1-WWWW WW1WWW W WWWWWW 1 .. W01WWWWW WWW .WW W WW WW - W.- W. . .WW WW WWWWWWWW. WWW-WW Wm WW W-W WWWWWW. WWWWWWWWWW WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW 179 1“1 1 1 1111 11111111 1111Wmm 11111Wm 11:1- 011:“ 111 1111111111 111 11-111 111 1.... 1 11-1111-11 1111111111111 001000 0010100000 0000001001 00001 001000 001 ‘ @0010100001 1 1001 11.00000 o0000010100 0000100000 0010000000 0000000010 1000000000 0100000000 0000001000 0000000100 1 1 1 11 0000010000100000000 0001001001 00000001 1W m mmmo1W11.mom ‘smmdl m m411u4l Oldl4 1. WW 1 1 1 1 1 1 100000 1000000000 0010000000 0000000001 0010000000 0001000000 0010000000 0001000000 0100000000 1000000000 1.1 1 1 1 1 1 0000000010 0000000100 0000010000 0001000000 0000001000 0000001000 111 1 1 1 1111 1 1 1 11111.1 1 1 111111111111 111 1 111111-11- 1 12 1a 1 1 1 11 m 11 1 1.1 11.1. 1.1. 11111111 1111111. 1-1111 11-11 1111 1 1 111111 1mm 1111111 1111 1111 111111111 1111 1 1 11-1111 1-1111111-111111111111111m111w11 111 1- mm SSWWWOWWSQWWW W1591mmomm1541mwm O4WWWO @541 54 1 OWW SWWWOW 571m57wmw .1 1 1111 1 1111 11111111 1 111111 11 111111 111 1 11 11 1 1111 1 1111 1111-111 1 1 11 11 1 1 1” 111111 1 1111 1 11 1 11 1 222223333333344444444555555556666666677777777888888889999999900000000111111112222 «I111.111:11.‘1‘1‘1‘1‘1‘1‘11‘4-1.“aldl11.11“11.11111.1“...“14-1‘1.11‘1111122222222222222222222 111‘111'4:4'4!«I11-cl411114.114!‘1."1“.l1¢l4l14l4|4u¢l1¢l111‘4!1111-11114-“1‘11-‘11111111114!111111111“ 180 O 1111111H1W1111111m11 111111 11 1111 W111 Wmom 1101 W1 11 W1 001 001 O1 1001OGXDGD1 1001 1001 101KDO1 O1 OGXXHW 00 01 1010111 (XKDO1 mxmom 001 OOflXM 00(311 mxmo1o 01 axm01 OOflM1 0010:} 1 1 1 11 11 m Wmmwmw 111me omo 1o1 1 11111 1 11 1111 111111 111.11. 1 .1111 111111 1.11 1. 131 111 1111111111 1GXKDOoamxnomx>omxmouxm. Bettecmanee meesgcement egg Ibeecx- Hilledale. NJ: Erlbaum, 127-165. Bruner, J.S. (1958). Social Psychology and perception. In E.E. Maccoby, T." Newcomb, & E.E. Hartley (Eds.), Beegiuge in §Qsiél Esxsbelegx. New York: Holt. Rinehart. & Winston. Bruner, J.S. & Tagiuri, R. (1954). The perception of people. (Vol. 2). Reading, Mass: Addison-Wesley. Cantor, N. & Mischel, W. (1977). Traits as prototypes: Effects on recognition memory. Journal of Eecegnalitx egg 5991a; Eexsbglggx. §§. 38-48- Cantor, N. & Mischel, W. (1979). Prototypes in person perception. In L. Berkowitz (Ed.) Advances in Academic Press. Carroll, S.J. & Schneier, C.E. (1982). Eagfiggmaaga agggaiaal egg sexier sxetemei Ihe igentification- meaeecementi egg Qexeleemens Qi eecigcmange in organizatiene- Scott. Foresman, & Co., Glenview, Illinois. Cooper, W.H. (1981). Ubiquitous Halo. anchological Eelletip. 29. 218-244- ‘ 184 DeCotiis, T.A. (1977). An analysis of the external validity and applied relevance of three rating formats. Organizational Eebaxie: age Human Eeciecmaaee. l2. 247- 266. DeVries, D. Morrison, A., Shullman, S., & Gerlach, M. (1981). Performance agggaiaa; 9a gag Liaa. New York: Wiley-Interscience. Dunnette, M.D. & Borman, W.C. (1979). Personnel selection and classification systems. egaaal Ragiag 9f Eexebeiegx. §Q. 477-525- Favero, J.L. (1983). The effects of ratee prototypicality and time on raters’ observation of the ratee and evaluation accuracy. Masters thesis, Purdue University. Fay, C.H. & Latham, G.P. (1982). Effects of training and rating scales on rating errors. Eagagaga; Eaxguglggy, §§, 105-115. Feldman, J. (1981). Beyond attribution theory: Cognitive processes in performance appraisal. angaal Qfi egaliag Eexeneiegx. aa. 127-148- Feldman, J. & Hilterman, R.J. (1975). Stereotype attribution revisited: The role of stimulus characteristics, racial attitude, and cognitive differentiation. Jgacgal Qi Eacaaaaligy, §;, 1177-1188. Green, S.G. & Mitchell, T.R. (1979). Attributional processes of leaders in leader-member interactions. Organizational Eebaxie: ago Human Eeciecmaaee. 2§. 429- 458. Hakel, M. (1969). Significance of implicit personality theories in personality research and theory. Procaading oi the 77th Convention at the fleecieaa Eexcnolooical eeeeeiatiee. 403-404- Hamilton, D. (1979). A cognitive-attributional analysis of stereotyping. In L. Berkowitz (Ed.) egxaggaa la eaeeciaeatal eeeia; eexenelegx (Vol- 12)- New York: Academic Press. Heneman, R.L. & Wexley, K.N. (1983). The effects of time delay in rating and the amount of information observed in performance rating accuracy. egagamy Qfi Maaagamaag Jeeceal. ea. 677-687- 185 Ilgen, D.R. & Favero, J.L. (1985). Limits in generalization from psychological research to performance appraisal processeS- eoaoeox oi management Bexieu. 19. 311-321- Ilgen, D.R. & Feldman, J.M. (1983). Performance appraisal: A process approach. In B.M Staw (Ed.), BEEEQEED in organization behavior, Vol. 2, Greenwich, Conn: JAI Press, Inc. Isaacson, G.I. (1966). A comparative study of meaningfulness of personal and common constructs. Unpublished Ph.D. thesis, University of Missouri. Kane, J.S. & Lawler, E.E., III. (1978). Methods of peer assessment- Eexooolooioai Eolietio. 99. 555-586- Kane, J.S. & Lawler, E.E., III. (1979). Performance appraisal effectiveness: Its assessments and determinants. In B.M. Staw (Ed.), Baggaggb La organizational behavior, Greenwhich, Conn.: JAI press. KeIIY. G-A- (1955)- Ioe oexoooloox oi oeoeooai oooetooote- New York: Norton. Kingstom, P.O. & Bass, A.R. (1981). A critical analysis of studies comparing behaviorally anchored rating scales and other rating format. Personnel Psychology, §fl, 263- 288. Koltuv, B. (1962). Some characteristics of intrajudge trait correlations. Psychological Monographs, ZQ, 552. Lahey, M.A. & Saal, F.E. (1981). Evidence incompatible with a cognitive compatibility theory of rating behavior. Journal at Applied Psychology, 9, 706-715. Landy, F.J. & Farr, J. (1980). Perfomance rating. Eexooolooioai Eoiietio. 92. 72-107- Latham, G.P. & Wexley, K.N. (1977). Behavioral observation scales for performance appraisal purposes. EaEanga; Eexoooloox. 99. 255-268- Markus, H. (1977). Self-schemata and processing information about the self- Qoocoai oi Eeceooalitx aoo §ooiai Eexoooloox. ea. 63-78- 186 McIntyre, R.M., Smith, D.E., & Hassett, C.E. (1984). Accuracy of performance ratings as affected by rater training and perceived purpose of rating. Jgaggal Qfi eoolieo Eexoooloox. oz. 147-156- Miller, G.A. (1956). The magical number seven plus or minus two: some limits on our capacity for processing information. anchologica; Bayiaa, 98, 102-114. Murphy, K.R., Martin, C. & Garcia, M. (1982). Do behavioral observation scales measure observation? Jgagaa; Qfi Boolieo Eexoooloox. oZ. 562-567- Nathan, B.R. & Alexander, R.A. (1985). The role of inferential accuracy in performance rating. Agagamy 91 naoaoeoeot Bezieo. 19. 109-115- Nathan, B.R. & Lord, R.G. (1983). Cognitive categorization and dimensional schemata: A process approach to the study of halo in performance ratings. ggagaa; Qfi Aggliag Eexoboioox. 99. 102-114- Nisbett, R.E. & Wilson, T.D. (1977). Telling more than we can know: Verbal reports on mental processes. ngchological Bayiag, 85 231-259. Norman, W.T., & Goldberg, L.R. (1966). Raters, ratees, and randomness in personality structure. Journal of Eeceooalitx aoo §oeia1 Eexoooloox. 3. 681-691- Passini, F.T. & Norman W.T. (1966). A univeral conception of personality stucture? Journal of Personaligy aag §ooia1 Boxcholoox. a, 44—49. Pederson, D.M. (1965). The measurement of individual differences in percieved personality-trait relationships and their relation to certain determinants. Jaggaal gt §ooial Eexoooloox. go. 233-258- Phillips, J.S. & Lord, R.G. (1982). Schematic information processing and perceptions of leadership in problem- solving groUPS- Joocoal oi Boolieo Eeycholoox. oz. 486-492. Pulakos, E.D. (1984). A comparison of rater training programs: Error training versus accuracy training. Journal of eggliag anchology, in press. 187 Pulakos, E.D. (1985). The interactive effects of format and training on performance appraisal accuracy. Unpublished dissertation, Michigan State University. Rosch, E. (1975). Cognitive reference points. Cognitive Eexooo1oox. 1. 532-547. Rosch, E. (1977). Human categorization. In N. Warren (Ed.), 9tooiee 1o croee-cultura1 oexooo1oox. Vol- 1- New york: Academic Press. Rosch, E. (1978). Principles of categorization. In E. Rosch & B.B. Lloyd (Eds.), Qgggigiya Qagagorizatigg. Hillsdale, N.J.: Erlbaum. Rosch, E., Mervis, C.G., Gray, W.D., Johnson, D.M., & Boyers-Braem, P. (1976). Basic objects in natural categorieS- Qoooiiixe Eexoooioox. 9. 382-439- Rosenberg, S. (1977). New approaches to the analysis of personal constructs in person perception. In Naggaaka Sygggaigg gg Motivation1 1976. Lincoln: Univeristy of Nebraska Press. Sauser, W.I. & Pond, 8.8. (1981). Effects of rater training and participation on cognitive complexity: An exploration of Schneier's cognitive reinterpretation- Eeceoooe1 Eexooo1oox. 93. 563- 577. Schneider, D.J. (1973). Implicit personality theory: A review. Baychologica; Ballagig, 22, 294-309. Schneier, E.C. (1977). Operational utility and psychometric characteristics of behavioral expectation scales: A cognitive reinterpretation. Journal of egaliag Eexooo1oox. oz. 541-548- Sechrest, L.B. (1968). Personal constructs and personal characteristiCS- ioocoa1 oi iooixiooa1 Eexooo1oox. 95. 162-166. Smith, P.D. & Kendall, L.M. (1963). Retranslation of expectations: An approach to construction of unambiguous anchors for rating scales. Jggggal gfi eoo11eo Eexooo1oox. 32. 149-155- 188 Shweder, R.A. & D'Andrade, R.G. (1980). The systematic distortion hypothesis. In R.A. Shweder (Ed.) Ealligla ioooeheht 1h hehaxioca1 :eeeacoh. Vol- 4- San Francisco: Jossey—Bass. Taylor, S.E., & Crocker, J. (1981). Schematic bases of social information processing. In E.T. Higgins, C.P. Herman. & M-P- Zanna (Ede-). 9ooia1 ooohitiohi Ihe Qggagig aymggaigm (Vol. 1). Hillsdale, N.J.: Erlbaum. Taylor, S.E. & Fiske, S.T. (1978). Salience, attention, and attributions: Top of the head phenomena. In L. Berkowitz (Ed-i. eoxahoee 1h etoeoihehta1 eooia1 oexoho1oox (Vol- 11). New York: Academic Press. Taylor, S.E., & Fiske, S.T. (1981). Getting inside the head: Methodologies for process analysis in attribution and social cognition. In J.H. Harvey, W. Ickes, & R.F. Kidd (Eds.), New directions in attributional research (Vol. 3). Hillsdale, N.J.: Erlbaum. Tsujimoto, R.N. (1978). Memory bias toward normative and novel trait Prototypee- 9oo:oa1 oi Eeoeoha1itx aho 9ooia1 Eexoho1oox. 99. 1391-1401- TriendiS. H-C- <197o)- yaciatiohe ih o1aot one white oecoeotiohe oi the eoo1a1 ehxicohheht- Urbana. IL: University of Illinois press. Tversky, A. (1977). Features of similarity. anghglggigal Boxiew. 95. 327-352- Walters, H.A. & Jackson, D.N. (1966). Group and indiviudal regularities in trait inference: A multidimensional scaling analysis. Multivariate Behavioral Baaaacgh, 1, 145—163. Weiss, H.M. & Adler, S. (1981). Cognitive complexity and the structure of implicit leadership theories. Journal of 9221299 Eexeho1oox. oo. 69-79- Wyer. R.G. & Gordon, S.E. (1982). The recall of information about persons and groups. Experimental Social Psychology, 18, 128—164. Wyer, R.G. & Scrull, T.K. (1980). The processing of social stimulus information: A conceptual integration. In R. Hetie. et al- (Ede-i. Eoceoh hehocx aho ehoooiho agggaaaaa. Hillsdale, N.J.: Erlbaum. 189 Zammuto, R.F., London, M. & Rowland, K.M. (1982). Organization and rater differences in performance appraisale- Beheohhe1 Eexoho1oox. 99. 643-657- Zedeck, 8., & Cascio, W.F. (1982). Performance appraisal decisions as a function of rater training and purpose of appraisal. Journal of Qagliag anggglggy, QZ, 752-758. MICHIGAN STATE UNIV. LIBRARIES m\IHWIWIWIWWWWIIWHIWWNWWI 31293010972382