i I "mm!IWL‘HIWW/Wmnmflm ‘ L3 3 1293 01010 0695 )V1531.J RETURNING MATERIALS: Place in book drop to LJBRARJES remove this checkout from Ailing-III. your record. FINES will be charged if book is returned after the date stamped be10w. 53m 9 6 100/! 3:971; .w.’ ' ' . £300 3 z 2003-" * ' THE EFFECTS OF APPRAISAL PURPOSE AND PRIOR KNOWLEDGE: AN INFORMATION PROCESSING PERSPECTIVE BY Susan Lynn Schechtnan A THESIS Submitted to Michigan State University in partial fulfillment: of the requirements for the degree of MASTER OF ARTS Department of Psychology 1987 \l ‘7'DI;; ’II’» V -. (V ABSTRACT THE EFFECTS OF APPRAISAL PURPOSE AND PRIOR KNOWLEDGE: AN INFORMATION PROCESSING PERSPECTIVE By Susan Lynn Schechtman Recently, it has been suggested that research in performance appraisal should focus on the human judgmental processes involved in appraisal decision making. Most of the research has utilized policy capturing and focused on the integration component of information processing models. The current study employed the process tracing technique of the information board to monitor the process by which subjects accessed information under different appraisal purpose (promotion, training) and prior knowledge (high, low, none) conditions. Results indicated that subjects searched far more extensively for ratees purported as 'good' performers when a promotion decision was to be made. Conversely, subjects searched far more extensively for ratees considered 'poor' for training decisions. Implications and future research directions are suggested. ACKNOHLEDGEMENTS Wonders never cease. It's hard to believe, but the margins have finally responded to my pleas and I can now place the finishing touches on this project. I would like to thank Steve Kozlowski for serving on my committee and sharing the usefulness of 3-D diagrams with me. I would also like to thank Neal Schmitt for serving on my committee and securing the resources necessary for developing the computer software for conducting the present experiment and for purchasing the computer on which the study was run. Lastly, I would like to thank Kevin Ford for serving as chair of my committee and providing the appropriate frustrations and threats that contributed to the quality of this project. The experimental sessions for this study could not have been completed within the desired time frame without the assistance of several key individuals. I am grateful to the late Phil Marcus of the Social Science Research Bureau for providing the financial input for the equipment necessary to complete the project. I am deeply indebted to Hsiang-Li Chiang for developing the computer software for conducting process tracing studies on a personal computer. Also, I am very grateful to Michael P. Kirsch and Kathy Karl for providing the incentives necessary for their students to participate in the experiment. I would also like to thank my summer class for taking advantage of the incentives I offered them for participating. Finally, iii I would like to express my appreciation to Mary D Zalesny for letting me use her research space for conducting subject sessions. Several members of the Michigan State I/O program deserve special recognition. My stability could not have been maintained without the support and patience emitted by Scott Cohen during those crucial proposal completion and data collection months. You deserve more than Denny’s and miniature golf. Additionally, I would like to thank Mary Doherty and the other summer residents of Baker for staying clear of my path as I stomped through the basement between subject sessions in search of my sanity. Finally, words cannot express how much I appreciate the unending encouragment and emotional support continuously given by Judy and Allan Schechtman. What can I say? This project would not have been completed without ybur unconditional positive regard. This one's for you. iv TABLE OF CONTENTS LIST OF TABLES . . . . . . . . . . LIST OF FIGURES . . . . INTRODUCTI on I I I I I I I I I I I I I I I I I I Methodologies for Studying Cognitive Processes Policy Capturing . . . . . Process Tracing . . . . Explicit information search Verbal protocols . . . . . . . . . General Issues Surrounding Information Processing Methodologies Application of Process Techniques to Performance Appraisal Policy Capturing . . . . . . . . Process Tracing Verbal protocols . . . . . . . Explicit information search Limitations of Process-Oriented Research in Performance Appraisal Information Gathering Process Appraisal Purpose . Prior Knowledge . . Search Variables . . . Depth of Search . . Sequence of Search . . Appraisal Purpose and Prior Knowledge Framework for the Current Research . Factors Affecting the Attention or Operationalised and Hypotheses Presented Appraisal Purpose Prior Knowledge . . . . . Appraisal Purpose X Level of Performance . “THOD I I I I I I I I I I I I Sample . . . . . . . General Procedure . Ratees . . . . . . . . . Design . . . . . . . Page viii ix WQQUIQD 11 12 12 13 13 14 15 18 19 19 21 24 24 24 25 25 26 26 28 28 28 29 29 Independent Variables . . . Appraisal purpose . . . . . . Prior knowledge . . . . . . . . Level of performance in impression . . Task . . . . . . . . . . . . . . Protocol and Stimuli . . . . . . Introduction and practice . . . . Introduction to experimental session Prior knowledge . . . . . . . . . . Search procedure . . . . . . . . . . Performance stimuli . . . . . . . . Dependent Variables . . . . . . . . Depth of search . . . . . . . . . . . Sequence of search . . . . . . . . . Post search questionnaire . . . . . RESULTS I I I I I I I I I I I I I I I I I I I I Manipulation Checks . . . . . . . . . . ...... Descriptive Summary of Search Variables . . Intercorrelation among search variables Amount searched . . . . . . . . . . . Sequence of search . . . Overall Effects . . . . . . Analyses of Hypotheses . Appraisal purpose . . Prior knowledge . . . . . . . . Appraisal purpose X prior knowledge Supplemental Analyses . . . . . . . . . Other search indices . . . . . . . . DISCUSSION . . . . . . . . . . . . . . . . . . . Summary of Results . . . . . . . . . . Alternative Explanations . . . . . . Limitations of the Study . . . . . . . Guidelines for Process Tracing Studies Implications and Future Directions . . APPENDICES . . . . . . . . . . . . . . . Appendix A . . . . . . . . . . . . . . . . Introduction: Administrative (Promotion) Condition . . . . . . . . . . . . . . . . Introduction: Developmental (Training) Condition . . . . . . . . . . . . . . . . Appendix B . . . . . . . . . . . . . . Promotion: Practice Evaluation Task . Training: Practice Evaluation Task . Appendix C . . . . . . . . . . . . . . . Information Values for Practice Session vi 83 83 98 113 113 115 117 117 Page Appendix D . . . . . . . . . . . . . . . . . . . . . . . . 119 Promotion: Police Officer Performance Dimenions . . . . . 119 Training: Police Officer Performance Dimensions . . . . . 121 Appendix E . . . . . . . . . . . . . . . . . . . . . . . ... 123 Promotion Background Information Packet . . . . . . . . . 123 Training Background Information Packet . . . . . . . . 131 Appendix F . . . . . . . . . . . . . . . . . . . . . . . . . 139 Matrix Contents for Version 1 . . . . . . . . . . . . . 139 Matrix Contents for Version 2 . . . . . . . . . . . . . . 143 Matrix Contents for Version 3 . . . . . . . . . . . . . . 148 Matrix Contents for Version 4 . . . . . . . . . . . . . . 152 Matrix Contents for Version 5 . . . . . . . . . . . . . . 156 Matrix Contents for Version 6 . . . . . . . . . . . . . . 161 Matrix Contents for Version 7 . . . . . . . . . . . . . . 166 Matrix Contents for Version 8 . . . . . . . . . . . . . . 171 Matrix Contents for Version 9 . . . . . . . . . . . . . . 176 Appendix G . . . . . . . . . . . . . . . . . . . . . . . . . 180 Promotion: Post Search Questionnaire . . . . . . . . . . 180 Training: Post Search Qustionnaire . . . . . . . . . . . 182 Appendix H . . . . . . . . . . . . . . . . . . . . . . . . . 184 Descriptive Summary of Sample Representation per Version . . . . . . . . . . . . . . . . 184 Appendix I . . . . . . . . . . . . . . . . . . . . . . . . . 185 Comprehensive Summary of the Depth of Search for the Officers by Appraisal Purpose . . . . . . . . . . 185 Appendix J . . . . . . . . . . . . . . . . . . . . . . . . 186 Verbal Rationale for the Influence of Appraisal Purpose on the Amount of Information for which Subjects Searched . . . . . . . . . . . . . . . . . . . . 186 Appendix K . . . . . . . . . . . . . . . . . . . . . . . . . 187 Verbal Statements Regarding the Categorisation of Background Information . . . . . . . . . 187 Appendix L . . . . . . . . . . . . . . . . . . . . . . . . . 188 Possible Reasons for Searching More/Less for Officers Having Background Information . . . . . . . . . . 188 Appendix M . . . . . . . . . . . . . . . . . . . . . . . . . 189 Extent to Mhich the Impressions Formed from the Background Information Influenced the Attention Paid the Different Officers . . . . . . . . . . . . . . . 189 App.ndi x N I I I I I I I I I I I I I I I I I I I I I I I I I 1 91 Strategy Used to Gather Information for Making the Promotion/Training Selection Decision . . . . . . . . 191 Appendix 0 . . . . . . . . . . . . . . . . . . . . . . . . . 194 How Items of Information Here Chosen During the Search Process . . . . . . . . . . . . . . . . 194 Appendix P . . . . . . . . . . . . . . . . . . . . . . . . . 195 How Subjects Determined When to Conclude Search . . . . . 195 FOOTuOTE I I I I I I I I I I I I I I I I I I I I I I I I I I I 197 LIST OF REFERENCES . . . . . . . . . . . . . . . . . . . . . . 198 vii Table 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. LIST OF TABLES Perceived Desirability of Profile Incidents for Effective Performance (N=30) . . . . . Manipulation Check for Appraisal Purpose . . Manipulation Check for Prior Knowledge . . . Analysis of Variance for the Amount of Information Searched by Version . . . . Analysis of Variance for the Officer Selected by Version . . . . . . . . . . . Intercorrelation Among Search Variables . . Descriptive Summary of Search Depth by Appraisal Purpose . . . . . . . . . . . Descriptive Summary of Search Depth for each Performance Dimension by Appraisal Purpose Descriptive Summary of Search Pattern by Appraisal Purpose . . . . . . . . . . . Analysis of Variance for the Amount of Information Searched by Appraisal Purpose and Level of Background Knowledge . . . . . . . . Simple Effects Analysis for the Amount of Information Searched by Appraisal Purpose and Prior Information . . . . . . . . . . Analysis of Variance for the Pattern of Search by Appraisal Purpose . . . . . . . T-test between the Amount of Information Searched for Officers Having Background Information Presented Versus those that Had No Background Information Presented . . . . . . . . Analysis of Variance for the Amount of Information Searched for High and Low Performers by Appraisal Purpose . . . . . Analysis of Variance for the Officer Selected by Appraisal Purpose . . . . . . . . . . . Cell Frequencies for the Number of Times the Officers were Selected for Different Appraisal Purposes . . . . . . . . . . . viii Page 33 42 44 46 46 47 48 51 52 54 56 57 57 60 66 66 LIST OF FIGURES Figure Page 1. Experimental Design for the Present Study . . . . . . . . . . . . . . . . . . . 30 2. Illustration of Performance Information Acquisition Task . . . . . . . . . . . . . . . . . . 35 3. Relationship between Appraisal Purpose and Prior Knowledge for the Amount of Information Searched . . . . . . . . . . . . . . . . 55 4. Mean Number of Information Acquisition for High and Low Performers (Background Information) by Appraisal Purpose . . . . . . . . . 61 ix INTRODUCTION The significant impact of the performance appraisal system on the organisation (i.e., human resource planning) and its members (i.e., promotion, salary increase) has resulted in a substantial body of empirical literature in industrial[organizational (I/O) psychology. During the 1950s, performance appraisal research tended to focus on methodological factors that fostered inaccuracy and biased appraisal results. Researchers assumed that rating scale development would directly affect the accuracy of the performance description: consequently, most of the early work concentrated on engineering the performance appraisal instrument to develop the most effective 'mousetrap' (Landy A Zedeck, 1983). Sophisticated engineering in the 1960s and 1970s led to the development of various behavioral oriented scales, i.e., Behavioral Anchored Rating Scales (Smith & Kendall, 1963) and Behavioral Observation Scales (Latham, Fay, & Saari, 1979; Latham & Mexley, 1981). Although the behavioral approach enabled l/O psychologists to measure performance in particular occupations better than the “engineers“ (Landy 6 Zedeck, 1983), it has done little to reduce commonly found errors (e.g., halo, leniency, central tendency) characteristic of the more traditional graphic scales (Bernardin, 1977; Bernardin, Alvares, & Cranny, 1976; Landy & Farr, 1980; 1983). In general, research addressing the superiority of behavioral scales for measuring performance has found that the rigor with which the scale is developed is more critical to reducing error than behavioral versus nonbehavioral terminology. A second approach to increasing appraisal accuracy has been the growth of rater training programs for reducing errors. Most typically, raters receive a lecture-based discussion regarding the performance appraisal system and the avoidance of errors such as halo and leniency (Landy, 1985). Although results indicate that raters do engage in a response set characterized by lower average ratings (reduced leniency) and greater variability in ratings across dimensions (reduced halo) as a result of training, the long term impact on rating accuracy is minimal (Bernardin A Pence, 1980). Implicit in the rater training research is the notion that an effective performance evaluation system cannot be obtained by changes in rating format or by lectures on how to avoid certain rating tendencies. The result is often a forced response set in which the rater produces numbers with particular statistical characteristics rather than accurate descriptions of behavior (Landy, 1985). A third approach suggests that research attention should be directed to aspects in the appraisal process that directly or indirectly deter or enhance the appropriateness of the final performance judgment (DeMisi, Cafferty, a Meglino, 1984). Interest in investigating issues related to the processes in the appraisal context developed in the late 1970s. DeCotiis A Petit (1978) presented a literature-based model of the appraisal process that focused on factors affecting the accuracy of performance ratings. The authors proposed that appraisal accuracy is determined by (1) the rater's motivation to accurately evaluate performance, (2) the rater's interpretive abilities regarding observed job behaviors, and (3) the rating standards used by the rater. DeCotiis A Petit (1978) hypothesised that the rater must be motivated to rate accurately, use appropriate rating standards, and have the ability to correctly interpret ratee behaviors in terms of these standards. Landy A Barr (1980) have also constructed a process model of the appraisal context. These authors highlighted the need for research that focuses on the cognitive processes by which human judgments are made. Raters are information processors who must observe, classify, store, and retrieve information when assigning a performance judgment (Wherry, 1952). To understand the factors leading to bias and inaccuracy, attention needs to focus on these judgmental issues rather than redesigning the appraisal form or training raters to conform to a particular response set. Several models focusing on the rater's cognitive processes during judgment have been proposed (Cooper, 1981; DeNisi et al., 1984; Peldman, 1981; Ilgen A Feldman, 1983). Similar to Landy a Farr (1980), these models have adopted an information processing perspective to the rating process. Rather than passively responding to an appraisal instrument, the rater is viewed as an information gatherer and processor actively engaging in a social perception exercise (DeNisi et al., 1984). The information processing models are based on the social cognition literature and assert that raters must: (1) gather information from the appraisal context by attending to and recognizing relevant information; (2) categorize that information and 4 integrate new information in some organized fashion; and (3) recall/integrate the appropriate information when task demands require a formal performance judgment (Feldman, 1981; Ilgen A Feldman, 1983). Although these information processing perspectives have stimulated theoretical discussion about the rater's cognitive activities during the appraisal process (Banks, 1982), few direct assessments of the linkages between the attention, storage, retrieval, or integration stages, nor systematic investigations within any particular phase have been conducted. The purpose of this research is to examine the attention or information gathering phase by investigating the influence of factors in the appraisal context on the strategies in which raters engage when collecting information to form evaluative judgments. To achieve this goal, techniques relevant to examining cognitive processing variables such as attention are examined. Next, factors affecting attention or information gathering issues are explored and hypotheses presented. Finally, the current focus and resulting experiment are presented. Methodologies For Studying Cognitive Processes The decision making literature within cognitive psychology has identified two methodologies for investigating the cognitive processes of individuals as they engage in decision tasks. These techniques are structural analysis/policy capturing and process tracing. Policy capturing statistically igggrg the decision process from parameters surrounding the decision situation through regression analysis (Svenson, 1979). Process tracing observes the predecisional behavior in analysing a situation by tracing the steps leading to a decision (Payne, Braunstein, & Carroll, 1978). Each methodology is introduced and both techniques are reviewed in terms of general issues surrounding these information processing methods. This review is followed by a brief summary of research applying each technique to performance appraisal issues. Finally, limitations in the current use of these techniques in process-oriented research in performance appraisal is presented and the present theoretical framework is described. We Policy capturing determines what judges 'do' (i.e., what information is used) when presented a series of stimuli representing a decision task (Slovic A Lichtenstein, 1971). Through regression analysis, the subject‘s final decision is statistically examined to create a mathematical description of the decision policy (i.e., the actions of the judge). The extent to which the outcome of the judge's decision can be predicted based on the characteristics of the stimuli is the extent to which the judge's policy has been 'captured' (Naylor A Mherry, 1965). In a recent review of the policy capturing literature, Robson a Gibson (1983) identified three advantages to the utilization of policy capturing results. First, the unique information processing strategy of each judge can be determined and compared to other judges viewing the same stimuli. Second, the rating policy gathered through the mathematical description can be compared to a subjective estimate of the same process. Finally, judges/raters can be trained regarding the consistent use of a particular rating strategy. In the typical experiment, subjects/raters are presented profiles (i.e., paragraph descriptions) representing hypothetical stimuli (e.g., graduate school applicants, clinical patients) and are asked to provide a quantitative evaluation for each 'individual.‘ All profiles are constructed to reflect the level of performance on several criterion elements (i.e., GRE's, sociability) and the level of the different criterion elements is varied among profiles. In regressing the overall evaluation onto the different criterion elements and examining the beta weights and corresponding R squared, the judge’s implicit, integrative process is inferred without the decision maker subjectively determining the actual process (Zedeck A Kafry, 1977). Policy capturing has been used to investigate a wide variety of research issues. Brady a Rappoport (1973) examined experts' policies regarding nuclear regulation; Rynes & Lawler (1983) used policy capturing to examine the role of expectancies in the search for job alternatives. Other topics utilizing policy capturing include (1) graduate school acceptance process (Dawes, 1971). (2) attitudes in organizational diagnosis (Madden, 1981), (3) transportation for the handicapped (Allen & Muchinsky, 1984), and (4) executive acquisition decisions (Stahl & Zimmerer, 1984). For example, Allen A Muchinsky (1984) used policy capturing to determine the desirability of providing special transportation services for the physically handicapped. Subjects (students and transportation employees) were given 36 bus proposals that contained four dimensions of transportation services and equipment (entrance, safety, 7 comfort, and mobility with limited funds). The levels of these dimensions varied among the different proposals. Based on the overall assessment of desirability, regression equations were constructed to reflect the degree of importance for the different dimensions. The policy equations revealed that the evaluations were based on entrance and safety (functional necessities) to a greater extent than comfort and mobility with limited funds (comfort/convenience). Wain Unlike policy capturing which infers cognitive activities from the structural decomposition of the decision situation, process tracing techniques focus on observing the predecisional behavior in analyzing and evaluating a situation. This methodology directly assesses what information the rater/decision maker had available and how that information was used to form a judgment (Payne et al., 1978; Pits, 1976). Two of the most relevant process tracing techniques are: (l) verbal protocols and (2) explicit information search. V o o . The collection of verbal protocols provides a sequential record of the subject’s ongoing predecisional behavior. In progressing through the decision situation, the subject is asked to state aloud his or her line of reasoning in completing the decision task (e.g., Payne, 1976). It is assumed that the subject's utterance at time 2 indicates his or her knowledge state at time t (Newell a Simon, 1972). Through the dissection and coding of verbal protocols, the intermediate stages between presentation of a stimulus and final judgment are revealed. Verbal protocol data have been used for a variety of different purposes such as (1) exploratory investigations of new 8 decision behaviors (e.g., Carroll 4 Payne, 1977; Svenson, 1974), (2) supplementing information obtained from other process tracing methods (e.g., Payne, 1976), (3) testing hypotheses about a particular phenomenon (e.g., Bettman & Park, 1980; Montgomery, 1976), and (4) building and testing computer models of decision behavior (e.g., Payne et al., 1978). As a method for exploring a new area in decision research, Carroll & Payne (1977) used verbal protocols to search for regularities in thought among parole decision makers. The authors were interested in the role of attribution theory in parole decision making and found that attributional statements comprised the largest category of verbal statements generated. Additionally, this type of statement occurred in a particular stage in the decision process. Attributional statments represented more than one-fifth of all coded statements and tended to occur in the third quarter of the protocol (cf. Payne et al., 1978). Verbal protocols have also been used to supplement information for data obtained from other process-oriented methods. Payne (1976), for example, found that coded protocols provided explanatory information to strengthen the conclusions drawn from data gathered through the explicit search technique. The verbal protocols provided insight as to why subjects used particular strategies in reducing the amount of information that was presented to them. Verbal protocols have also been used to Itest a priori hypotheses. Bettman & Park (1980) used coded protocols to examine the effects of prior knowledge and phases of choice on decision processes. The authors found that consumers with moderate prior knowledge and 9 experience used more of the available information than subjects low or high in prior knowledge. Finally, verbal protocols have been used to systematically model the step by step fashion in which subjects complete decision tasks (Payne et al., 1978). The creation of an information processing model attempts to accurately represent the decision process in the same sequential pattern as the subject. This type of modeling is an important step in developing a generalized understanding of decision behavior. Ixpligit_1;fg;ggtigg_sg§;gh. With this approach, the decision maker's information acquisition patterns are monitored by presenting subjects with decision tasks in which they must explicitly search for information. The information is typically presented on a matrix/board containing alternatives (e.g, apartments, consumer products) and attributes or dimensions (e.g., rent, price) describing the alternatives. This type of display has been labeled an I'information display board" (Jacoby, Chestnut, Fisher, & Neigl, 1976; Nilkins, 1964). Each cell of the alternative x dimension information display board contains an information value representing the corresponding alternative and dimension pair (e.g., the rent for Apartment A is 250 dollars/month). Subjects are usually permitted to search for as little or as much information as they would like in making their choice. To select information, the subject locates the cell of interest, notes the information presented and continues this process until the final choice selection is made. To examine the salient characteristics of an individual’s search process, four types of variables have been utilized: depth of 10 search, content of search, sequence of search, and latency or response time (Jacoby et al., 1976; Payne et al., 1978). Depth of search refers to the amount of information acquired by the decision maker during the particular task under consideration. Content of search refers to the specific nature of the information accessed (e.g, price versus nutrient information in a consumer product experiment). Sequence of information search refers to the temporal pattern in which information was acquired and provides information about the strategies in which subjects engaged while acquiring information. Finally, latency data provides information regarding the amount of time necessary to view the information and complete the decision task. To illustrate, Jacoby, Chestnut, a Fisher (1978) collected depth, content, sequence, and latency data in an investigation of perceived risk, product importance, and prior knowledge on the choice of breakfast cereals. An information display board was constructed to simulate the information environment for purchasing cold breakfast cereals. Brand names and product attributes comprised the matrix/board and subjects were instructed to search for information values until ready to choose a cereal. The authors found that product importance was related to the depth of search, but that perceived risk and brand loyalty were not related to any search variables. 11 General Issues Surrounding Info tion oces ’n Methodolo ' Most of the research addressing human judgment processes in decision making has employed the policy capturing/regression approach and relatively few have used process tracing. Billings & Marcus (1983) compared policy capturing (rating task) and process tracing (information board) techniques regarding their convergence in measuring decision behavior. The authors found little agreement between the two measures and concluded that separate processes were being tapped by each method. In mathematically representing what the rater sight gg_when faced with a similar array of stimulus information, policy capturing describes the input-output relationship characterizing the decision situation. Policy capturing examines the influence of the stimulus variables on the response variables (Simon, 1976), i.e., how the information was combined to form a judgment. This procedure does not, however, provide insight into what the rater actually gig. Research that has compared the judge's stated policy with that produced from the regression analysis has found discrepant results. Raters tend to underestimate the weight of important cues and overestimate the weight ascribed less important cues (e.g., Robson, Mendel, & Gibson, 1981). Mhen the goal for a piece of research is to explore the processes by which a decision is reached, the more appropriate technique would be one specifically designed for examining ongoing process issues. Simon (1976) noted that the correlational paradigm is relatively useless for discovering process models which 12 delineate 'what goes on between the appearance of a stimulus and performance of a response“ (p. 261). For instance, the regression approach interprets a high multiple R as indication of an additive, linear combination of variables; nevertheless, the multiple R does not necessarily indicate that the judge used an additive combination of variables (Oskamp, 1967). In addition, two or more structural models of the judgment process may be algebraically equivalent, yet suggestive of different underlying processes (Hoffman, 1960). Thus, for examining process issues, it is necessary to use a technique capable of documenting predecisional events. Billings & Marcus (1983) suggested that the decision behavior tapped by the process tracing method was the prejudgment acquisition of information. Although process tracing techniques may tap the appropriate process, as with policy capturing, care must be taken to avoid mere demonstrations of the technique's usefulness versus examinations of conceptually-driven notions. Appliggtions of Process Techniques to Performance Appraisal Was Several performance appraisal studies have employed policy capturing as a methodology for assessing the cognitive activities of raters while evaluating others (Anderson, 1977; DeNisi & Stevens, 1981; Hobson, Mendel, & Gibson, 1981; Naylor & Wherry, 1965; Stumpf & London, 1981; Taylor 4 Milsted, 1974; Zedeck & Cascio, 1982; Zedeck I Kafry, 1977). The goal of these studies has been to £11912 the unique information processing strategies of raters based on the overall evaluations assigned to ratees. 13 Raters are usually presented profiles (verbal or numerical) representing hypothetical employees and are asked to provide an overall rating of effectiveness for each employee. Profiles are constructed to reflect the ratee's level of performance on several job dimensions and the performance on the different dimensions is varied among ratees (Zedeck a Kafry, 1977). Naylor a Mherry (1965) developed 250 profiles representing scores of hypothetical Air Force recruits on 23 traits. Raters (Air Force supervisors) were asked to determine each recruit's ”worth to the Air Force" based on the information (scores on the different traits) provided. Regression equations indicated that raters weighted and combined information consistently across ratees. Using policy capturing techniques for testing a priori notions regarding performance appraisal issues, Zedeck a Cascio (1982) examined the effect of rater training and appraisal purpose on rating accuracy. A 2 (rater training versus no training) x 3 (development, merit raise, retention) ANOVA with the standard deviation in ratings across ratees as the dependent variable revealed that integrative strategies differed by appraisal purpose. Subjects (college students) in the merit raise condition weighted and combined information similar to each other, yet qualitatively different than those in the other purpose conditions. E!QQS!!.I£§QLDS V;;bgl___p;gtg§gls. The use of verbal protocols for investigating performance appraisal issues has been severely limited. Lewis 4 Layman (1979) analyzed verbal protocols to 14 develop models of information processing strategies involved in the peer rating process. Subjects participated in a simulation of managerial decision making in which groups of 5-7 were videotaped while analyzing a hypothetical company and preparing recommendations for that company. Participants then completed a peer evaluation questionnaire consisting of sociometric items. All questionnaires were missing one item and the individuals were independently presented their missing item and asked to verbally respond to that item. Although verbal protocols were not gathered during the entire decision task, information processing models based on the individual protocol sessions and the videotaped group decision making were developed in order to rank group members for each sociometric question. Spearman rank correlations demonstrated a high relationship between predicted rank orders and actual rank orders. That is, protocol-derived models appeared to capture the information being processed by individuals while evaluating their peers. 't i ation search. Matte (1982) used the information board to investigate factors affecting attentianal/observational processes in performance appraisal. Matte examined factors that might result in some ratees being observed more closely than others (i.e., more extensive information search for some ratees). The following three factors were addressed: (1) level of ratee performance (high versus low); (2) consistency of performance (high versus low); and (3) appraisal purpose (promotion plus feedback administration versus feedback only). Subjects (college students) were presented with six 8 x 6 matrices (boards) representing the performance of eight grocery clerks 15 on six performance dimensions over the course of six months. Participants were instructed to use as little information (i.e., a numerical rating reflecting the employee's performance on a particular dimension during the month being investigated) as necessary to provide a summary rating for each employee and for each dimension. Additionally, four ratees representing the promotion plus feedback condition were rated concerning their eligibility for promotion. Matte used the depth/amount of information searched as the major dependent variable. Raters engaged in a higher rate of information search when the ratee's performance was high, inconsistent and for promotion plus feedback purposes. Secondly, although not part of the original hypotheses, Matte also examined the content of information searched for different appraisal purposes. Results indicated that subjects sought Attendance and Punctuality and Courtesy to Customers more than the other dimensions across the different purposes, but that Reordering Stock received significantly more attention for the promotion condition. Although the 'information board" was not operationalized in terms of the prototypical ratee by dimension array, Milliams et al. (1985) examined information search behavior for different appraisal purposes (salary increase, promotion, remedial training) and outcomes (designation, deservedness) using a computer-controlled search procedure. Designation outcomes referred to those situations where the supervisor must designate one individual to receive the particular outcome. Deservedness outcomes referred to those situations where the supervisor must assess how deserving each employee was of the 16 particular outcome. Subjects (college students) were instructed to evaluate four different ratees in terms of the particular Purpose X Outcome they were assigned. Subjects were seated at a computer console and presented an item describing the performance of one ratee performing a particular task (e.g., Anne, Budget). To gather sufficient information about all ratees to make a deservedness/designation rating, subjects could request one of four types of information content from the computer display: (1) the same person performing the same task; (2) the same person performing a different task; (3) a different person performing the same task; and (4) a different person performing a different task. Results indicated that subjects searched more extensively for information regarding the 'same person performing a different task" for deservedness decisions and 'a different person performing the same task' for designation decisions. t' of r c ss-Oriented Research ip Perfprmance Appraisal Information processing models in performance appraisal have identified four subphases to the rater's judgment process: attention, categorization, recall, and integration. Most of the research exploring 'process' has utlized policy capturing, a technique thch has been shown to appropriately assess issues revolving around post-decision information integration. To assess ongoing process, the appropriate starting point would be when social perception commences, during the first subphase, or attention. DeNisi et al. (1984) defined the attentional stage of the 17 performance evaluation process in terms of observation or information gathering. The authors suggested that the observation process could be defined by the amount of information search that is conducted prior to the assignment of summary performance ratings. This view of attention is most similar to Billings a Marcus' (1983) recommended use of process tracing methodologies in information acquisition research. There are three limitations to the process tracing literature in performance appraisal. First, research that has attempted to directly address attentional issues through process tracing has been quite sparse. Only three studies could be located that employed process tracing techniques to the evaluation of employees. Second, in performance appraisal research, the measurements of information search have been limited. As noted earlier, the decision making literature has identified four types of search variables for use with the information display board: depth, content, sequence, and latency. Matte (1982) primarily examined the depth of search and analyzed content on a post hoc basis. Milliams et al. (1985) examined information type in terms of depth of search. To gain a comprehensive understanding of search behavior and attentional processes, it is important to not only consider the amount and content of information searched, but also the sequence in which that information was sought. Latency information can provide supportive data to the other search variables or can be used independently to provide information regarding the importance of particular items or alternatives. Third, previous process-oriented techniques for addressing 18 attentional issues involved the reliance on naive raters as the experimental subjects. For the most part, not only are the subjects unknowledgeable about the individuals whom they are rating, the subjects are also typically blind as to the dimensions being rated. For example, Matte used the job of grocery clerk because it was assumed to be a job of reasonable familiarity. Subjects may merely be searching to find out more about the job than searching because of the particular manipulation. Thus, process-oriented approaches to performance appraisal suffer from focusing primarily on indicants of information processing (e.g, policy capturing), using decision making techniques for measuring information search/attentional processes without fully examining search variables, and relying on naive subjects. Framework for the Current Research Two assumptions are guiding the current approach to investigating information processing issues in performance assessment: (1) human beings are limited in their information precessing capabilities (Newell & Simon, 1972) and (2) raters cannot observe every behavior emitted by each employee in the particular rating period. Given that all behaviors emitted cannot be observed and all observations cannot be processed, it is important to examine how factors in the appraisal context influence the attentional component of the information processing models. In the present investigation, the attentional component of information processing models of performance assessment is operationally examined in terms of information acquisition. Similar 19 to the decision making literature, information acquisition is operationalized as search behavior and is measured in terms of sequence and depth of search.1 In the next section, factors in the appraisal context that may influence attentional/observation issues are presented and the search variables are defined. Finally, the independent variables are operationalized and hypotheses incorporating the search variables are provided. Factors Affecting the Attention or Information Gathering Process The information processing perspective posits that the rater is an active seeker and processor of information (Ilgen a Feldman, 1983). Even when the appraiser is passively responding to the presentation of information, he/she must still choose which 'pieces' of information should be attended to and coded for later retrieval (cf. Bargh, 1982). DeNisi et al. (1984) developed a social-cognitive model of the performance appraisal process that incorporated several factors that might affect the rater‘s attentional processes and subsequent information search. Two of these proposed factors are appraisal purpose and prior knowledge (preconceived notions). Both of these factors present implications for information processing issues and subsequent phases in information processing models; yet, only two studies have focused on the information processing aspect of purpose and no studies have investigated the additional contribution of prior knowledge. Appggipgl Purpose Traditionally, appraisal purpose (i.e., the use for which the ratings are being collected) has been examined in terms of the 20 rater's motivation or willingness to provide accurate ratings. Research has demonstrated that ratings collected for administrative purposes (promotion, raise) tend to be more lenient than ratings collected for research or developmental purposes (e.g., Meyer, Kay, 0 French, 1967). For example, Kirkpatrick, Ewen, Barrett, 4 Katzell (1968) found that ratings for research purposes did not differ between blacks and whites, although ratings on the same sample of employees for administrative purposes were higher for whites than for blacks (cf. Landy 4 Farr, 1983). Recent research has demonstrated that appraisal purpose may also affect the information that is sought prior to assigning performance judgments (Matte, 1982; Milliams et al., 1985). Matte (1982) used the information display board to investigate the influence of appraisal importance (purpose) on the extent to which subjects searched for information about ratees (i.e., depth of search). The two purposes investigated were: (1) promotion decision plus the administration of feedback and (2) only the administration of feedback. It was hypothesized that making a promotion decision plus administering that information to the ratee would be perceived as more important to the rater (i.e., greater accountability) and result in greater information search than appraising an individual's performance only for the purpose of giving feedback. Results supported this hypothesis; raters searched for information more extensively in preparation for an important task (e.g., promotion plus feedback) than for a task that was lower in importance (e.g., feedback only). Milliams et al. (1985) also investigated the influence of appraisal purpose on the depth of information search. Three 21 purposes were examined (1) salary increase, (2) promotion, (3) remedial training for one of two possible outcomes (deservedness, designation). Subjects were either instructed to rate how deserving each individual was or designate one individual to receive the particular outcome. No main effects on information search were found; however, the greatest amount of information search occurred in the most important Purpose x Outcome condition (i.e., salary raise, designation). Prio e One of the common criticisms given laboratory investigations of performance appraisal issues is the overreliance on naive subjects in the place of supervisors (Ilgen 8 Favero, 1985). In the current context, both Matte (1982) and Nilliams et al. (1985) used subjects having no knowledge about the ratees to be evaluated. However, it has been suggested that prior knowledge or preconceived notions (DeNisi et al., 1984) structures and possibly reduces the flow of incoming information. Prior knowlege has been shown to operate as a type of schema that reduces cognitive chaos by influencing the information that is sought by the cognitive system (Cohen, 1981). Hansen (1980) suggested that individuals approach an attributional setting with tentative hypotheses regarding the witnessed events. These tentative hypotheses may then guide the perceiver's search for information on which to support formal explanations for the observed events. In the performance appraisal context, supervisors enter the performance evaluation process with an existing knowledge base about the ratees. Thus far, 22 little research has addressed the influence of prior knowledge on the extent to which supervisors will search for information. Prior knowledge may develop from a variety of different origins including (1) earlier interactions with the ratee, (2) prior evaluations, or (3) information about the ratee that is gathered from other sources (DeNisi et al., 1984). Prior knowledge results in the development of a schema that reduces cognitive chaos by influencing the information that is sought by the cognitive system (Cohen, 1981). For example, a rater who develops a 'good' or "bad'I impression/schema may not feel the need to collect new information about the worker or may desire just enough information to reinforce the schema. Research addressing the influence of prior knowledge has focused on the outcomes resulting from the similarity between the existing knowledge base and subsequent observations. Using a performance appraisal task, Murphy, Balzer, Lockhart, & Eisenman (1985) examined the impact of a ratee's previous performance level on later observations and corresponding ratings of performance for that individual. Subjects rated three videotapes of a graduate student delivering lectures. Half the subjects viewed two performances that were 'poor' followed by an 'average' performance videotape and half the subjects viewed two 'good' performance videotapes followed by the same “average“ videotape. Manipulation checks indicated that the 'good' performance tapes were perceived as representing significantly better performance than the first two 'poor' performance tapes. However, ratings for the third tape (average performance) indicated a significant contrast 23 effect. Ratings from subjects having first viewed the 'poor' tapes were higher than ratings from individuals in the "good'' performance condition. The authors hypothesized that the contrast effect may be due to (1) biases in memory for ratee behaviors or (2) biases in attention to and encoding of ratee behaviors. Murphy et al. conducted a second experiment to test the notion that contrast effects are due to memory biases. Based on recent memory research which found that general impressions have an increasingly strong effect as the time lag between observation and judgment lengthens (e.g., Hastie, 1980), the authors hypothesized that the contrast effects should be accentuated if increased memory demands (i.e., time lag) are imposed on the participants. To test this hypothesis, subjects viewed the first two tapes and returned the following day to view the third tape and provide behavior and performance ratings. The authors did not find a contrast effect; no significant differences were found between the “good“ and 'poor' performance conditions regarding the rating of the “average“ performance. The authors concluded that the contrast effect found in the first study was not due to memory biases and suggested that these biases may enter during the attention-encoding process and recommended that research address such a bias. The present research explores 393 the presence of a knowledge base affects the attention-encoding process. More specifically, prior to determining the impact of a 'good“ impression when faced with subsequent ”poor“ performance, it is critical to examine the influence of the “good“ versus 'poor' 24 impressions on attentional processes (i.e., information search). The next section describes the search variables employed. Following this discussion, the independent variables are operationalized and the hypotheses are presented. Search Variables W The decision making literature has defined depth of search in terms of the total amount of information an individual acquires while completing the decision task. The extent to which a subject searches for information regarding a particular decision problem provides information indicating the relative importance of the different dimensions and alternatives in completion of the task. For example, Matte (1982) used the amount of information searched to index the differential reliance on available information depending on the importance of the decision to the rater. The present investigation measures depth of search in terms of the total number of information pieces acquired. We The sequence of information search provides information regarding the temporal pattern by which the information was acquired. The decision making literature has delineated three types of search patterns that can be examined through the information board (e.g., Payne, 1976). These search patterns are categorized based on the relationship between the pth and pth + 1 pieces of information acquired. Interdimensional search indicates a search pattern characterized by successive information requests across 25 dimensions within a particular alternative. Intradimensional search represents a search strategy involving greater numbers of information requests within a dimension and across alternatives. Finally, shifts reflect noncategorical search patterns in which the pth and pth + 1 pieces of information requested do not represent the same dimension or the same alternative. Appraisal Puzpcge apg Prio; Knpwlgdgg nggptipnalizgd and Hypotheses Prgsentgd App;§i§pl__ppppp§g. Based on previous research with appraisal purpose, two uses for collecting appraisal information were investigated: either an administrative decision (e.g., promotion) or a developmental decision (e.g., remedial training). A promotion decision has greater consequences for the rater and the ratee and is expected to be perceived as more important than a recommendation for training. In addition, this perceived importance is expected to affect the process by which the information is acquired. Thus, based on examinations by Matte and Hilliams et al., the following hypotheses are suggested regarding appraisal purpose and information search variables: Hypothesis 1: Depth of search will be greater for subjects making administrative decisions than subjects making developmental decisions. In addition, two exploratory hypotheses are examined regarding search sequence. It is assumed that an administrative decision more directly involves a comparison among ratees and a developmental decision involves an examination of each ratee's strengths and weaknesses. 26 Hypothesis 2a: Mhen assigning performance judgments for an administrative decision, the sequence of search will be characterized by an intradimensional search strategy. Hypothesis 2b: Mhen assigning performance judgments for a developmental decision, the sequence of search will be characterized by an interdimensional search strategy. £§ip§_kppglpggg. Two aspects of prior knowledge are investigated: the presence/absence of that knowledge and the level of performance (high versus low) within that scheme. Hansen (1980) hypothesized that evaluators within a social context follow the rule of cognitive economy when faced with incoming information. Perceivers simplify the amount of information to be processed by relying on mechanisms that group information together. One method for grouping information is to utilize one’s preconceived notions based on prior knowledge and/or experience regarding the incoming information. Thus, the hypothesis regarding prior knowledge and information search is: Hypothesis 3: Raters will search more extensively for ratees about whom they have no information than ratees about whom they have knowledge. Due to the lack of research regarding the influence of prior knowledge on information search variables in performance appraisal, no a priori hypotheses are offered for search sequence. Appraisgl pppppsg X levpl of performance. Appraisal purpose and prior knowledge may affect the amount and/or type of information that is searched when evaluating others. In addition, the level of performance (good versus bad) in the prior impression may interact with appraisal purpose in affecting information search. Matte (1982) found that the importance of the decision and the level of 27 performance emitted by the ratee interacted such that raters in the promotion plus feedback condition searched for more information for the high performers than for the low performers. Consequently, the following hypotheses are offered: Hypothesis 4a: Performance appraisals conducted for administrative purposes will result in greater information search for high than for low performers. Hypothesis 4b: Performance appraisals conducted for developmental purposes will result in more information search for low than for high performers. METHOD 5.141211 Subjects in the present investigation included 100 male and female undergraduate students who participated voluntarily or as part of a course requirement. A power analysis revealed that this sample size provided appropriate statistical power (e.g., .89) to detect a 'moderate' effect size with the design employed (Cohen, 1979). W Subjects were tested individually and randomly assigned to an experimental condition. After entering the experimental room, the subject was seated in front of a Zenith personal computer. On the computer screen, the subject was welcomed to the experiment and acquainted with the procedure for accessing information from the computer. Following this tutorial, the subject was given an opportunity to ask questions and to practice his/her new skill by conducting a practice performance evaluation. The subject then engaged in the information search task until ready to select the appropriate ratee. Upon completion of this selection, a post search questionnaire was administered. The subject was debriefed and thanked for his/her participation. 28 29 Rgtepp The job whose performance the subjects rated was that of a police officer. The job was chosen because it was believed to be reasonably familiar to most individuals and critical incident information based on a performance scale development project (Landy 8 Farr, 1975) was available. The job dimensions used were chosen based on their adequacy in providing behavioral examples of good, average, and poor police officer performance. Consequently, the following six dimensions were used in this study: (1) Job Knowledge; (2) Judgment; (3) Dependability or Reliability; (4) Communication; (5) Mork Attitude; and (6) Dealing with Others. Denise The design of the present study was a 2 X 3 mixed analysis of variance design with repeated measures on the last factor. The two independent variables were appraisal purpose (administrative versus developmental decision), and performance level in the prior impression (high, low, none). The design of the experiment is illustrated in Figure 1. Subjects were randomly assigned to an appraisal purpose condition. Each subject experienced all three levels of prior knowledge. I de V ' es A a l u se. Appraisal purpose was examined in terms of the type of decision to be made from the performance appraisal information (i.e., administrative versus developmental decision). In the present study, subjects assumed the role of a police sergeant who must evaluate his/her officers for a particular purpose. Half the 3O Appraisal Purpose Administrative Development I I I I High I I I I Prior I -------- I I I Knowledge I I I I Impression I Low I N=50 I N=50 I I I I I I -------- I I I I I I I I Home I I I I I I I I I I I Figure 1 Experimental Design for the Present Study 31 subjects were instructed to make an administrative decision and half were instructed to make a developmental decision. Those in the administrative condition selected one officer to be promoted and subjects in the developmental condition selected one officer to send to remedial training. £119;_§ggg;gggg. Prior knowledge was operationalized as the overall impression raters possessed regarding their officers' past performance (i.e., 'good' versus "poor”). To determine the impact of prior knowledge/impressions on attentional processes, subjects were provided with background information concerning a subset of the officers being evaluated. This background information consisted of paragraph descriptions representing behavioral examples of the performance dimensions used in the evaluation task (see Appendix E for profiles). Research by Tversky & Kahneman (1974) has demonstrated that individuals have a bias towards information that is based on personal or subjective experience rather than abstract statistical data. Nisbett & Ross (1980) noted that statistical information may be too abstract and/or dry for people to use (of. Kirsch, 1985). In addition, statistical information may conjure up images other than what was intended (Hogarth, 1980). For example, 'rarely checks files for a suspect’s friends and favorite hangouts" may be cognitively evaluated and/or represented much differently than a '2' on 'Initiative' for police officer performance. Consequently, verbal statements were chosen to create the global performance impressions. 32 Lgvgl 9f pgrformgnce in impression. Profiles were composed of behavioral incidents collected for the development of a performance evaluation instrument (Landy & Farr, 1975). Using a 9-point scale (1-very undesirable performance and 9=very desirable performance), 30 college undergraduates rated whether these behavioral incidents reflected above average, average, or below average performance. To maximize the impression of "good' versus “poor" performance, incidents representing above average performance (M87.0-9.O) or high-average performance (H=6.0-6.9) were used to construct the “good“ performing profiles. Similarly, statements representing low (MI1.O-3.O) or low-average performance (M=3.1-4.0) were used to construct the "poor'I performing paragraphs. The means and standard deviations for the incidents used in the background profiles are presented in Table 1. In the current study, three officers represented poor performing officers and three officers represented good performing officers. Profiles A, B, and C represented Good performers, profiles D, E, and F represented Poor performers, and officers 8, H, and I were not described in the background packet. In! In this investigation, the information board from the consumer psychology (e.g., Jacoby et al., 1976) and decision making (e.g., Payne, 1976) literatures was adapted to a performance evaluation task involving ratees (alternatives) and dimensions of performance (dimensions). The 'board“ was presented and controlled by a Zenith personal computer. Subjects were presented a list of officers and a list of job dimensions and were asked to select 33 Table 1 Perceived Desirability of Profile Incidents for Effective Performance (N=30) Profile Dimension JK J D/R C WA DO A 7.33 7.33 8.00 7.83 8.20 7.70 (1.56) (1.81) (1.49) (1.58) (1.16) (1.23) B 7.00 7.90 7.87 7.93 7.93 7.20 (1.93) (2.02) (1.76) (1.46) (1.41) (2.41) C 8.07 7.93 7.87 7.90 8.00 8.23 (1.34) (1.41) (1.50) (1.26) (1.26) (1.19) D 1.73 2.50 1.60 2.03 1.83 2.80 (1.11) (1.43) (1.10) (1.07) (1.46) (1.13) E 3.07 1.97 1.10 1.97 2.13 1.70 (2.69) (1.19) (0.30) (1.33) (1.11) (1.05) F 1.97 2.50 1.40 2.00 1.30 2.13 (1.43) (1.50) (1.17) (1.29) JK=Job Knowledge JsJudgment DIR=Dependabilitleeliability C=Communication NAaflork Attitude DOaDealing with Others Numbers in first row represent means and numbers in parentheses represent standard deviations. (0.62) (1.34) 34 information by accessing the officer and dimension of interest. Figure 2 contains a typical screen of information the subject encountered. After selecting the desired alternative and dimension, the corresponding information value was presented to the subject on the next screen. The subject’s task was to gather as much information as needed to select one officer to be promoted/trained. Protggg; and Stimuli Introduction gag practice. At the beginning of the session, subjects were welcomed to the experiment and told that they would be assuming the role of a police sergeant who must evaluate his/her employees in terms of their promotability or need for remedial training. Additionally, the subjects were acquainted with the procedure for accessing information from the computer (see Appendix A for the introduction messages for both purpose conditions). Following this introduction, participants were given an opportunity to practice the search procedure (see Appendix B) by selecting one nurse to be promoted/trained (depending on the appraisal purpose assigned). Four dimensions of performance could be searched in evaluating these nurses. These performance dimensions included Patient-Family Teaching, Professional Nursing, Basic Nursing Care, and Documentation and are defined in Appendix B. The information values (i.e., behavioral incident for a particular nurse on a particular performance dimension) were constructed to represent average performance across dimensions for each nurse and to minimize the intercorrelation among dimensions (see Appendix C for the information values comprising the practice task). Subjects were instructed to look at as little or as much information as necessary to select one 3S ALTERNATIVE DIMENSION 1: OFFICER A : JOB KNOWLEDGE : OFFICER B 2: JUDGMENT 3: OFFICER C 3: DEPENDABILITY/RELIABILITY 4: OFFICER D 4: COMMUNICATION 5: OFFICER E 5: WORK ATTITUDE 6: OFFICER F 6: DEALING NITH OTHERS 7: OFFICER G B: OFFICER H : OFFICER I ENTER NO OF ALTERNATIVE FROM 1 TO 9 THEN RETURN ? ENTER NO OF DIMENSION FROM 1 TO 6 THEN RETURN Figure 2 Illustration of Performance Information Acquisition Task 36 nurse for promotion or remedial training (depending on appraisal purpose assigned). Hhen the subject completed the selection for the practice session, the police officer performance evaluation began. Intgggggtigg t9 ggpegigggtal agggio . Subjects were reintroduced to the type of decision to be made and were presented definitions of the performance dimensions on which the officers would be evaluated. A hard copy of the dimension definitions was made available to the subject, but the subject was instructed to familiarize him or herself with the dimensions prior to beginning the search process. Performance dimensions used are defined in Appendix D. Ezig;_§pgglgggg. After familiarizing themselves with the dimensions, subjects were presented a packet containing paper profiles representing the previous performance for six of their nine subordinates (see Appendix E for background packets). As noted earlier, profiles A, D, and C were always 'good' performers and profiles D, E, and F were always “poor' performers. Attached to each packet was a cover letter. This letter was designed as a memo from the previous sergeant who was just informed about a transfer and had only a few days to gather together materials for the new sergeant. Consequently, there was only time to provide background information for some of the officers. Sggggh proceggre. There were nine stimulus persons contained in the list titled ALTERNATIVES and were described as officers of the subject. Chestnut (1977) found that stimuli labeled by first names generated differential levels of affect (of. Matte, 1982); consequently, officers were identified by an alphabetic letter (A-I). The order of alternative/officer and dimension presentation on the 37 computer screen was randomized for each subject. Pgrfgggggge stimgli. The information values corresponding to alternative-dimension pairs were behavioral statements (similar to those used in profile generation) representing the officers’ level of performance on the six different dimensions (see Appendix F for matrix contents). Incidents used represented high, average, and low performance on the different dimensions. The matrix of information values was constructed to minimize the correlation among cue values (mean intercorrelation 8-.19; SD 8 .07) depicting the level of performance on the different dimensions and the level of performance across dimensions for each ratee was approximately average (i.e., 5.0 on a 9-point scale). In addition, the position of the matrix’ contents was ordered across subjects to minimize the possible confirmatory/disconfirmatory relationship with the items in the prior impressions. The six statements representing an officer were always presented together. All subjects were instructed to search for as little or as much information as necessary to select the appropriate officer. Following the completion of the search task, a hard copy of the subject’s search process and the officer selected printed on the line printer next to the computer. A post-experimental questionnaire was administered and the session was concluded (see Appendix G). e V i There were two primary dependent measures used in the present study: the depth of information search and the sequence of information search. In addition, a post experimental questionnaire was administered to assess the subject's search strategy during the 38 decision task and to provide manipulation checks for appraisal purpose and prior knowledge. Dgpth gf search. Depth of search was operationalized as a frequency count of the total number of information pieces (i.e., ratee-peformance dimensions) requested. In addition, this total was broken down into the mean amount of information acquisitions for the different levels of prior knowledge: (1) the high performers (A, B, C), the low performers (D, E, E) and the information-absent or neutral performers (G, H, I). 5:99:39; of searc . Sequence of search was operationalized in terms of the temporal pattern in which information items were accessed. Payne (1976) has identified three major search movements that characterize the sequence of search. These movements compare the alternative and dimension associated with the nth and the nth + 1 pieces of information acquired and are labeled interdimensional search, intradimensional search, and shifts. A global measure of search pattern which incorporates the number of interdimensional and intradimensional singlestep transitions is given by the following formula: Pattern Ner _ "ra of = ---- Search Her I "ta "er I total number of interdimensional singlestep transitions "I. 3 total number of intradimensional singlestep transitions 39 A pattern of search characterized by mostly interdimensional transitions would result in a value close to +1.00. Conversely, a pattern of search characterized by mostly intradimensional searches would result in a value close to -1.00. Mixed patterns (e.g., shifts) would result in a value approximating zero. Eggt sgggch ggestionpaire. The post-experimental questionnaires are located in Appendix H. The purpose of these measures was to determine whether the manipulations had their intended effects and to gain greater insights into the search process than is available from monitoring only external search activities. In this questionnaire, participants were asked to indicate the perceived importance of promotion/training recommendations (depending on purpose assigned) (see questionnaire item 1). Additionally, subjects were asked to compare the importance of promotion recommendations versus training recommendations (see questionnaire item 2). Second, the subjects were asked to describe if (and how) the purpose for which the appraisal information was being collected influenced the amount of information for which they searched (see questionnaire item 3). These items provided the only distinction between the questionnaires. For prior knowledge, subjects were asked (1) if they attempted to categorize the background information in any way; (2) if so, how they grouped the information and if not, how they used the information (see questionnaire item 6); and (3) whether they searched differently for officers having background information than those that did not (see questionnaire item 7). To supplement information regarding the interaction of prior knowledge and appraisal purpose, subjects were asked to indicate whether the impressions formed from the background 40 information influenced the attention paid the different officers (see questionnaire item 9). Lastly, subjects were asked to recall the ratings given the background profiles (see item 8). Three additional open-ended questions were asked to examine the strategies raters used to gather information. These items were not included to define traditional decision making strategies (i.e., additive differnce, elimination by aspects), but were included to provide additional insight into the information gathering process. Consequently, three questions pertaining to the process of gathering information were: (1) how the information items requested were determined (see questionnaire item 5); (2) what strategy was used to gather information (see questionnaire item 4); and (3) what determined that enough information had been colllected and to terminate the selection process (see questionnaire item 10). RESULTS The results section is divided into four subsections. First, the results regarding the manipulation checks for appraisal purpose, level of performance in the prior knowledge, and order effects for stimuli presentation are provided. Second, descriptive summaries for the amount of information searched and the sequence by which subjects acquired information are presented. Third, the overall analysis for the influence of appraisal purpose and prior knowledge on search variables is described. This discussion is then followed by a presentation of the supplemental analyses conducted to further understand the results found. Manipulation Checks After the task was completed, subjects were asked to rate the perceived importance of using appraisal information for promotion (training) decisions using a 7-point scale (1=of no importance and 7-extremely important). Results indicated that both appraisal purposes were perceived as very important. As described in Table 2, the mean importance rating for promotion was 6.08 (SD=.66) and the mean importance rating for training was 5.86 (SD=1.01). These mean importance ratings were not significantly different (331.29, df(98), ns). 41 42 Table 2 Manipulation Check for Appraisal Purpose MEAN MEAN STANDARD COMPARATIVE STANDARD egggggg, IMPORTANCEa DEVIATION IMPORTANCEb DEVIATION Promotion 6.08 0.66 4.66 1.36 (n-SO) Training 5.86 1.01 3.74 1.16 (n850) g 1.29 3.54 p .20 .oo 'Scale for importance was anchored with 1=of no importance and 7=extremely important bScale for comparative importance was anchored with 1=training decisions are more important and 7=promotion decisions are more important 43 Subjects were also asked to compare the importance of using appraisal information for promotion (training) versus using the information for training (promotion) on a 7-point scale (1=training decisions are more important, 4=equally important, and 7=promotion decisions are more important). Results suggested significant differences in the comparison ratings (t=3.64, df(98), p(.001). Subjects in the promotion condition perceived using appraisal information for promotional purposes as more important than for training purposes (M=4.66, SD=1.36). Conversely, subjects in the training condition perceived using appraisal information for training purposes as more important than promotional purposes (M-3.66, SD81.16). For the manipulation check on prior knowledge, subjects were asked to rate each paper profile on a 9-point scale (1=ineffective performance, 9-effective performance, NA-not applicable). As presented in Table 3, the 'high' performers (Officers A, B, C) were perceived as effective performers (M=8.08, SD=.82) and the "low" performers (Officers D, E, F) were perceived as ineffective (M=1.82, 508.74) officers. The perceived difference between the high and low performers was significant (t=48.55, df(99), p<.001). An attempt was made to control for the possible confirmatory/disconfirmatory relationship between the background information and the computer screen by changing the placement of alternative-dimension pairs across subjects. Only the order of items was changed; the statements representing a particular officer were always presented together. Nine versions were created so that each series of statements would be used to represent each officer. OFFICER SUBGROUP Higha Lowb 44 Table 3 Manipulation Check for Prior Knowledge MEAN EFFECTIVENESS STANDARD RATING DEVIATION g p 8.08 0.82 48.55 .00 1.82 0.74 aHigh=average effectiveness rating given the profiles for Officers A, 8, and C on a 9-point scale of effectiveness where 1=very ineffective and 9=very effective. bLow-average effectiveness rating given the profiles for Officers D, E, and F on a 9-point scale of effectiveness where 1=very ineffective and 9=very effective. 45 The number of subjects per version is presented in Appendix H. To determine whether the particular placement of items influenced search amount or the officer selected, two analyses of variance were conducted and are presented in Tables 4 and 5. As described by the tables, the amount of information searched (F=.697, df(8,91), ns) did not differ by version. The officer selected differed marginally by item placement (F=1.96, df(8,91), p(.10). Thus, we can be reasonably confident that the subjects were responding to similar information content. Descriptive Summary of the Search Variables Igtggcgggelgtion among search variables. Table 6 contains the intercorrelation among the search variables. The amount of information searched by the three officer subgroups were moderately related to each other and strongly related to the total amount of information searched. The sequence of search, computed by Payne’s index, and the number of interdimensional movements correlated .61; the correlation between sequence of search and the number of intradimensional movements was -.68. Amogpt gggrched. The depth of information search for the officers and dimensions are presented in Tables 7 and 8, respectively. Table 7 describes the amount of information acquired for the high performers, the low performers, and the 'neutral' performers (i.e., those for whom no background information was provided) for the total sample and for each appraisal purpose subgroup. The total number of information pieces acquired for the two appraisal purposes was quite similar. Subjects were instructed to look 46 Table 4 Analysis of Variance for the Amount of Information Searched by Version 59939.5 55. £1; 85. E 2 Version 6370.0090 8 796.251 0.697 .693 Error 0.104E+06 91 1142.661 Table 5 Analysis of Variance for the Officer Selected by Version 5.92.891 5.5 if. E E 2 Version 99.907 8 12.488 1.96 .061 Error 580.683 91 6.38 47 Table 6 Intercorrelation Among Search Variables 1 2 3 4 5 6 7 1. Total Amt - 2. High-Amt 78 - 3. Low-Amt 84 43 - 4. Neutral-Amt 86 53 63 - 5. Pattern-Amt 95 26 33 37 - 6. Interdim-Amt 95 72 80 83 61 - 7. Intradim-Amt 22 27 18 10 -68 -08 - Tgt§1_figt=total amount of information—searchedE-Hggh:Am-;--- amount of information searched for the high performers from the background information; Low-Amt=amount of information searched for the low performers from the background information; Neutral-Ag§=amount of information searched for the background-absent officers; Pattern-Amt=Payne's index for search sequence; Lnterdim-Amt=number of interdimensional movements; and Int;adim-Amt=number of intradimensional movements. 48 Table 7 Descriptive Summary of Search Depth by Appraisal Purpose Total Number of Information Acquisitions High Performers Low Performers Neutral Performers PROMOTION (n850) Ma SDb 65.60 31.64 7.90 4.51 6.28 3.69 7.71 4.12 TRAINING (n850) M SD 66.58 35.37 6.10 4.27 8.07 5.44 8.01 4.45 TOTAL (n=100) M SD 66.09 33.39 7.00 4.46 7.18 4.71 7.86 4.27 'Msaverage number of information acquisitions for each officer subgroup and total bSD-Istandard deviation of information acquisitions for each officer subgroup and total 49 at as much information as necessary during the decision task and consequently, each piece of information. could be accessed as many times as necessary. Therefore, although the matrix contained 54 'pieces,‘ an infinite number of acquisitions could have occurred. The mean number of total information pieces acquired for subjects in the promotion condition was 65.6 (SD=31.64), which ranged from 7 to 142 pieces for the 50 participants in this condition. Overall, subjects in the training condition collected an average of 66.58 (SD-35.37) pieces of information, which ranged _from 8 to 152 total acquisitions. To examine each officer subgroup, three composites were formed. Each composite was based on the average amount of information searched for all the officers in the subgroup. Officers A, B, and C formed the HIGH group, Officers D, E, and F represented the LON officers, and Officers 6, H, and I formed the NEUTRAL officers. Nhen broken down by officer subgroup, subjects in the promotion condition averaged 7.90 (SD=4.51), 6.28 (SD=3.69), and 7.71 (SD=4.12) pieces of information for the High, Low, and Neutral performers, respectively. Information search for the High performers ranged from zero to 25.00 pieces collected, search for the Low performers ranged from 0 to 13.00 information items, and total acquisitions for the Neutral performers ranged from 0 to 18.67. Subjects collecting information for a training decision averaged 6.10 (SD84.27), 8.07 (SD=5.44), and 8.01 (SD=4.45) pieces of information acquired for the High, Low, and Neutral performers, respectively. The ranges surrounding these means are zero to 16.67 for the High perfomers, 0.67 to 27.67 for the Low performers, and 0 to 50 19.67 for the Neutral performers. Officer by officer breakdown for the amount of information searched is located in Appendix I. Based on the total amount of information searched for the total sample and by each appraisal purpose subgroup, Table 8 contains a breakdown of the depth of information searched for each performance dimension. From an examination of ranges of acquisition amount, subjects generally accessed each performance. dimension between zero and 29 times during the search process. Averaged across all six performance dimensions, subjects in the training condition searched a mean of 11.10 (SD=1.45) pieces of information per dimension which ranged from a mean of 9.28 (SD=5.92) pieces for Communication to a mean of 13.52 (SD=8.33) pieces of information for Job Knowledge. Similarly, subjects in the promotion condition accessed each performance dimension between 0 and 27 times while completing the search task. Averaged across all six dimensions, these subjects searched an average of 10.93 (SD=1.0B) pieces of information per performance dimension which ranged from a mean of 9.3 (SD=5.18) pieces for Communication to a mean of 12.08 (SD=6.69) acquisitions for Job Knowledge. Seggence of search. Table 9 presents the results for the sequence of information search in terms of interdimensionality, intradimensionality, and pattern (Payne, 1976) of search. The sequence by which information was acquired was quite similar across the two appraisal purposes. Overall, subjects averaged 45.75 (SD=27.42) interdimensional movements (44.94 for promotion and 46.56 for training). The number of intradimensional movements was substantially lower. Subjects in the promotion condition averaged 7.78 intradimensional movements and those in the training condition 51 Table 8 Descriptive Summary of Search Depth for each Performance Dimension by Appraisal Purpose PROMOTION TRAINING TOTAL (N=50) (N=50) (N=100) M3 sob M SD M so Total Number of Information Acquisitions 65.60 31.64 66.58 35.37 66.09 33.39 Job Knowledge 12.08 6.69 13.52 8.33 12.80 7.56 Judgment 12.00 6.23 11.94 6.13 11.97 6.15 Dependability/ Reliability 11.62 5.39 11.32 6.78 11.47 6.10 Communication 9.30 5.18 9.28 5.92 9.29 5.53 Hork Attitude 10.20 5.83 11.08 6.45 10.64 6.13 Dealing with Others 10.40 6.48 9.44 6.12 9.92 6.29 IMtaverage number of information acquisitions for each dimension and overall bSDI-standard deviation of information acquisitions 52 Table 9 Descriptive Summary of Search Pattern by Appraisal Purpose PROMOTION TRAINING TOTAL (N=50) (N=50) (N=100) “3 SD” M 90 M 90 Mean Number of Search Movements Interdimensionalc 44.94 24.75 46.56 30.08 45.75 27.42 Intradinensionald 7.78 9.92 7.10 7.76 7.44 8.32 P133213. 0.62 0.45 0.63 0.46 0.63 0.45 aM-average number of acquisitions per movement bSD'standard deviation of acquisition number per movement cInterdimensional=movement type where the nth and nth + 1 pieces of information are from the same alternative but a different dimension dIntradimensional=movement type where the nth and nth + 1 pieces of information are from the same dimension but a different alternative ePattern=the number of interdimensional movements minus the number of intradimensional movements divided by the sum of the number of interdimensional movements plus the number of intradimensional movements 53 averaged 7.10. Pattern of search, which takes into account both interdimensional and intradimensional movements, reflected an interdimensional search pattern. The mean pattern of search for individuals in the promotion condition was .62 (SD=.45) and .63 (SD=.46) for those in the training condition. Overall Effects A 2 x 3 repeated measures analysis of variance was conducted to assess the impact of appraisal purpose (promotion, training) and prior knowledge (high, low, none) on depth of search. The results for the amount of information searched are presented in Table 10 and illustrated in Figure 3. Results indicated a nonsignificant effect for appraisal purpose (F=.00, df(1,98), ns), a marginally significant effect for prior knowledge (F82.79, df(2,196), p<.10), and a highly significant interaction between appraisal purpose and prior knowledge (F310.4O, df(2,196), p<.001). These results suggest that the depth of information for the officer subgroups varied as a function of the purpose for which the appraisal information was being collected. A simple effects test, presented in Table 11 revealed that information search for the high performers was significantly different for the promotion purpose than for training (F=4.17, df(1,98), p<.05). Although similar differences were found for the low performers (F83.72, df(1,98), p<.10), the amount of information searched for the neutral performers did not differ by appraisal purpose (F=.12, df(1,98), ns). Since the pattern or sequence of information acquisition is SOURCE Purpose Error Level of Knowledge Knowledge x Purpose Error 54 Table 10 Analysis of Variance for the Amount of Information Searched by Appraisal Purpose and Level of Background Knowledge g 0. 081 3985.041 48.895 181.932 1714.499 g 98 196 g 0.081 40.664 24.448 90.966 8.747 I") 0.00 2.79 10.40 .9636 .0636 .0001 SS I l 8 'I ‘0 training I I promotion mean I amount I of 7 -I information I searched I I I 6 -I I I I I 5 -I I I I I‘- High Low None Level of Prior Knowlege Figure 3 Relationship between Appraisal Purpose and Prior Knowledge for the Amount of Information Searched 56 Table 11 Simple Effects Analysis for the Amount of Information Searched by Appraisal Purpose and Prior Information SOURCE §§ g; MS 3 2 Knowledge High X Purpose 80.4609 1 80.4609 4.17 .0440 Error 1893.1274 98 19.3176 Low X Purpose 80.3712 1 80.3712 3.72 .0566 Error 2115.9823 98 21.5917 Neutral X Purpose 2.2410 1 2.24101 .12 .73 Error 1803.6978 98 18.4050 57 Table 12 Analysis of Variance for the Pattern of Search by Appraisal Purpose 9031363 99 41 14.5. 6 p. Purpose 0.002 1 0.002 0.01 .9197 Error 20.283 98 0.207 Table 13 T-test between the Amount of Information Searched for Officers Having Background Information Presented Versus those that Had No Background Information Presented I! I9 k? a. H: In Background Information Provided 7.08 3.88 2.40 99 .02 Background Information Not Provided 7.86 4.27 58 a function of the pth and pth + 1 pieces of information accessed, pattern of search (interdimensional movements, intradimensional movements, and computed pattern) cannot be meaningfully applied to a repeated-measures analysis and is not analyzed in this fashion. Results concerning pattern of search and appraisal purpose are discussed below. WEE-i Appgpigpl purpose. Hypothesis 1 proposed that the amount of information searched would vary by appraisal purpose. Specifically, this hypothesis stated that subjects in the promotion condition would search more extensively than those in the training purpose condition. Since the overall analysis suggested no effect for appraisal purpose on search depth, this hypothesis was not supported. The second hypothesis posited that subjects in the training condition would search more interdimensionally than those in the promotion condition and those in the promotion condition would search more intradimensionally. To test this hypothesis, an analysis of variance was first conducted using Payne’s index of search pattern as the dependent variable in order to determine if further dissection into interdimensional and intradimensional movements seemed warranted. Table 12 presents the results of this analysis which suggests that the pattern of search did not vary by appraisal purpose (F=.01, df(1,98), ns). Consequently, the second hypothesis was not supported. Prip; kpowlgdge. The overall 2 x 3 repeated measures analysis suggested a marginal effect for prior knowledge for the amount of 59 information searched. Hypothesis 3 proposed that information search would be more extensive for those officers having no background information than those officers having background information presented. A t-test was conducted to assess the mean difference in amount of information sought and is presented in Table 13. Results indicated a significant effect for the presence versus the absence of prior knowledge (232.40, df(99), p<.05). Appgpisp; purpose and prior knowledge. A 2 (promotion, training) X 2 (high, low) repeated measures analysis of variance was conducted to directly assess the influence of appraisal purpose and high versus low performance impressions on the search process. Results are presented in Table 14 and depicted in Figure 4. In terms [of the amount of information searched, there was a significant interaction between the level of effectiveness portrayed in the background information and appraisal purpose (F317.05, df(1,98), p<.001). Hypothesis 4a proposed that the amount of search would be greater for the high performers when a promotion decision was being made. Similarly, hypothesis 4b stated that information search would be more extensive for the low performers when a training decision was being made. As illustrated in Figure 4, these hypotheses were supported. Subjects in the training condition searched more extensively for the low performers and subjects in the promotion condition searched more extensively for the high performers. 60 Table 14 Analysis of Variance for the Amount of Information Searched for High and Low Performers by Appraisal Purpose 5.9% 5.5. if. MS .1: P. Purpose 0.505 1 0.505 0.02 .896 Error 2865.489 98 29.240 Level of Knowledge 3.734 1 3.735 0.36 .553 Knowledge H Purpose 179.267 1 179.267 17.05 .0001 Error 1030.352 98 10.514 8 mean amount 7 of information accessed 6 5 Mean and 61 training I I I I I l I I I I I promotion I I I I I I I I I High Low Level of Prior Knowlege Figure 4 Number of Information Acquisitions for High Low Performers (Background Information) by Appraisal Purpose 62 Supplemental Analyses On the post-search questionnaire, subjects were asked to respond to a series of questions regarding the processes in which they engaged during the search task. After rating the importance of promotion/training and comparing it to training/promotion, subjects were asked to indicate whether the appraisal purpose affected the amount of information for which they searched (Yes/No) and to provide a rationale for their response (see questionnaire item 3). Appendix J contains the range of responses to this item and is organized in terms of the specific rationales given the Yes or No response and the number of subjects providing a similar response. The numbers next to the Yes and No represent the number of subjects that did not provide a specific rationale for their Yes or No. The Other category represents responses that did not conceptually fit into the affirmative ‘or negative categories. Overall, approximately three-quarters of the subjects were influenced by the purpose for which the appraisal information was being collected. More than half the subjects in the promotion condition and roughly 40 percent of the subjects in the training condition stated that the particular purpose was important and led them to gather sufficent information to make a well-informed decision. Only ten percent of the subjects in the promotion condition and 32 percent of those in the training condition stated that they were not influenced by the appraisal purpose. Subjects were also asked to indicate whether they attempted to categorize the background information and to state the manner in which this grouping took form (see questionnaire item 6). The range of 63 categorizations and the frequency of similar responses is listed under the ”Yes“ category in Appendix K. In addition, reasons for not categorizing are listed under 'No" and other uses that were made of this information are listed under “Other.“ Again, the numbers next to the 'Ves' and 'No' represent the frequency with which only "Ves‘I or “No“ was provided. Two-thirds of the subjects attempted a grouping scheme and one-third did not. Of those individuals that did categorize, 70 percent grouped the information in terms of those officers that received positive appraisals and those officers that received negative appraisals (i.e., high versus low performers). Other methods of grouping included utilization of the performance dimensions or in terms of the individual officers. Those individuals that did not categorize were either overwhelmed by the wealth of information between the paper information and the computer screen or were attempting to compensate for biases and stereotypes that may enter subjective evaluations. Regardless of whether subjects grouped the background information in any coherent fashion, questionnaire item seven asked subjects to indicate whether their search process for the officers represented by the background information differed from their search process for the officers without background descriptions. The list of rationales for this item are listed in Appnedix L and are also summarized in terms of 'Ves,‘ 'No,‘ and 'Other.‘ Although the t-test presented earlier showed that the amount of information searched for background-absent officers was greater than for background-present officers, more than half of the subjects stated that they did not search differently for the two sets of officers. Of those 64 subjects who did pay differential attention, most first examined the officers with no background information and searched in more detail than for the other officers. A small percentage investigated the officers they liked from the background information. Ouestidnnaire item 8 asked subjects to recall the ratings given the six officers from the background information packet. Subjects experienced great difficulty in completing this item without the use of probes from the experimenter. Consequently, the reliability of this data is questionable and the item was not analyzed. The last question addressing the independent variables asked subjects to discuss if, and how, the impressions formed from the background information influenced tha attention paid the different officers for making the promotion/training decision (see questionnaire item 9). The responses to this question appear to support the interactions reported earlier and are located in Appendix M. Those subjects who were influenced by the background information paid more attention to the more deserving officers (i.e., A, B, C for the promotion condition and D, E, F for the training decision). Subjects that did not search for information based on the prior information and appraisal purpose tended to become too involved with the computer information and consequently forgot the background information. Lastly, some subjects perceived the background information as disconfirmatory to the computer information and therefore discounted the background information. In addition to the open-ended assessment of the interaction between appraisal purpose and prior knowledge, an analysis of variance was conducted to determine if the officer selected at the end of the task 65 differed by appraisal purpose. Results of the analysis are summarized in Table 15 and the cell means are presented in Table 16. As indicated in the tables, the officer selected did vary by appraisal purpose (F=5.81, df(1,98), p<.05). Officers represented as good performers from the background information were more likely to be selected for promotional decisions and those officers characterized as poor performers from the background information were more likely to be selected for remedial training. Othpr pgprch issues. As an exploratory measure for examining decision behavior, subjects were asked to indicate the strategy used for gathering information during the decision task (see questionnaire item 4). Appendix N contains the variety of strategies offered and the number of responses per strategy broken down by appraisal purpose. For both promotion and training, three general strategies were employed: (1) evaluate every piece of information; (2) gather information on the performance dimensions deemed important; or (3) gather information in order to compare the background-present versus the background-absent officers. The primary difference in strategy between the two purposes was that subjects searched for positive information for promotion and negative information for training. Both sets of subjects searched across performance categories for each relevant officer until the decision was clear. In addition, subjects were asked to state the method by which they identified items of information to request (see questionnaire item 5). These responses are summarized in Appendix 0 by appraisal purpose. More than half of the subjects based their item selection on a self composed hierarchy of dimension importance. Most of the remaining subjects simply 66 Table 15 Analysis of Variance of the Officer Selected by Appraisal Purpose 592395 95 3: 89 E 2 Purpose 3.61 1 3.61 5.81 .018 Error 60.90 98 0.62 Table 16 Cell Frequencies for the Number of Times the Officers were Selected for Different Appraisal Purposes SUBGROUP PROMOTION TRAINING High. 26 11 Lowb 11 24 Neutralc 13 15 ‘High-Officers A, a, or c bLow=Officers D, E, or F cNeutralSOfficers G, H, or I 67 requested all the information with no set strategy. Lastly, subjects were asked to indicate the mechanism by which they concluded searching and could identify the appropriate officer (see questionnaire item 10). Appendix P contains the range of possibilities separated by appraisal purpose. In general, two strategies were prevalant: (1) all the information had been examined or (2) sufficient information on the important performance dimensions had been investigated to lead to a confident decision. Subjects who searched all the information engaged in a process of elimination. Both these subjects and those subjects searching a subset of dimensions looked for poor performance for the training decision and good performance for the promotion decision. DISCUSSION In this section, three basic issues are discussed. First, the findings are described and possible explanations for the ocurrence of the results obtained are provided. Second, limitations of the present study are presented. Finally, implications for the study are provided and directions for future research suggested. Summary of Results The present investigation examined the influence of two factors in the appraisal context on judgmental issues involved in appraisal decision making. This study assessed the impact of appraisal purpose (promotion, training) and level of performance within prior impressions (high, low, absent) on the manner in which raters searched for information for the subordinates being appraised. Results indicated that information search for the police officers being evaluated was affected by the joint function of the purpose for which the performance information was being collected and the evaluative tone surrounding the background information used to create the prior impressions. Overall, subjects required to select one officer to be promoted concentrated their search effort toward those officers purported as ''good" performers in the background information and those officers about whom no information was available. Similarly, subjects required to select one individual 68 69 to be sent to a remedial training program focused their search energies toward those officers depicted as “poor” performers in the background information and those officers about whom the evaluators were naive. No main effects were found for appraisal purpose on depth or sequence of search and marginal main effects were found for prior knowledge on depth of search. Altpgnative Explanations It was hypothesized that the two appraisal purposes would differentially impact the depth and sequence of information acquisition. Neither hypothesis was supported. Subjects in the promotion condition and subjects in the training condition both searched a similar amount of information during the evaluation task. As indicated in the post-questionnaire, both sets of subjects perceived the purpose for the performance evaluation as important, thereby resulting in extensive information search. In addition, the participants in the promotion (training) condition considered the purpose they were assigned as significantly more important than the opposite purpose. It is interesting to note, however, that during . the introductory computer session, subjects in the training condition were informed of the low priority typically given training programs in the organization and subjects in the promotion condition were apprised of the negative implications surrounding the advancement of the wrong individual. Nevertheless, subjects’ overall information search did not vary by appraisal purpose. The difficulty in detecting differential information search by appraisal purpose may be more a function of the design employed than the attempted manipulation of purpose importance. In the present 7O investigation, appraisal purpose was manipulated as a between-subjects variable. Thus, although both sets of subjects were made aware of the possible decisions regarding the use of appraisal information, neither set of subjects was necessarily faced with operating under both conditions. Williams et al. (1985) also utilized a between-subjects design for purpose and did not find a purpose main effect. Matte (1982), however, used a within-subjects design and found differential search for two appraisal purposes: promotion eligibility rating plus the administration of feedback versus feedback alone. Regarding the order in which information was acquired, it was proposed that search sequence would reflect an interdimensional pattern for subjects in the training condition and an intradimensional search pattern for subjects in the promotion condition. Results did not support this contention; the sequence of information search was identical for the two conditions and reflected an interdimensional search strategy (i.e., subjects searched across dimensions within alternatives). Open-ended responses collected from the post questionnaire regarding acquisition strategies supported this result. A number of participants reported information gathering strategies in which each dimension was examined for the relevant officer, followed by an elimination of officers having inappropriate behaviors on particular performance dimensions. Even when subjects were "comparing'I officers during the elimination process, it appears that they continued to gather information across dimensions and use that information to then compare across officers. 71 One factor that may have influenced the use of an interdimensional search strategy was the relatively small, and cognitively manageable, number of performance dimensions representing the nine officers. Research examining variation in the numbers of alternatives and/or dimensions (e.g., information load or task complexity) have found that the number of dimensions per alternative plays an important role in the processing of information. When presented different levels of alternative numbers (2, 4, 8, 12) and different levels of dimension numbers (4, 8, 12), Payne (1976) found a somewhat larger percentage of information searched across the different levels of alternatives when the dimension number was held constant than the counterpart products across different levels of dimensions with the alternatives held constant. For example, the mean percentage of information searched was greater for 2, 4, 8, 12 alternatives by 4 dimensions (M=.732I than for 4 alternatives by 4, 8, 12 dimensions (M=.590). Further support for the role of dimensions in information processing is available in the consumer behavior literature. Russo (1974) argued that it is psychologically irrelevant to define the complexity of the task situation or the perceived information load in terms of the multiplicative relationship between the number of alternatives and the number of dimensions. Russo stated that it is more likely that less important dimensions are overlooked during an evaluation task than entire alternatives. Therefore, the number of dimensions and the number of alternatives do not compensate for one another. As noted above, 12 alternatives and 16 dimensions would not result in the same decision process as 16 alternatives and 12 72 dimensions. Since subjects in the present study were provided only one piece of information per performance dimension for each officer, they could evaluate all the relevant information. In fact, subjects searched each performance dimension an average of two times and some individuals accessed the dimensions more than three times. As a result of the repeated acquisitions for each dimension in order to gain an overall picture for each officer and each surviving subset of officers, subjects accumulated an interdimensional search strategy and minimal intradimensional movements. Or, more simply, interdimensional strategies have been suggested in the decision making literature as being more cognitively simple to use. Given that there were only six dimensions in the present task and some alternatives may have been eliminated based on the prior information, subjects could process the 36-54 pieces of information and did not need to utilize a more complex, simplifying strategy involving intradimensional movements. Also, research has demonstrated that the format of the information display board may produce a particular type of information processing. Bettman & Kakkar (1977) conducted a series of studies in which the information matrix promoted (1) alternative processing, (2) dimension processing, and (3) mixed processing. Results indicated that information search varied by display. The present task was officer-driven and thereby resulted in significant interdimensional information acquisition. There was supporting evidence for the third hypothesis which proposed greater information search for prior knowledge-absent 73 than prior knowledge-present officers. Both the direct comparison between those officers described by background information and those that were not and the overall analysis of variance demonstrated more extensive search for officers not described in the background information packet. It is interesting to note that the effect for level of prior knowledge dissipated when looking only at the information search for high performers versus low performers. Overall, similar amounts of information were accessed for the low and high performers; however, these amounts were significantly less than that searched for the remaining officers. Support for Hypothesis 3 provides some evidence for the use of prior knowledge as a chaos-reducing mechanism in social environments (Cohen, 1981). Additionally, the examination of the relationship between prior knowledge and appraisal purpose provides additional insight into this process. When presented with background information describing several officers as Good performers and several officers as Poor performers, subjects prioritized their search effort in terms of the decision to be made regarding these officers. As stated earlier, significantly more attention (i.e., search) was afforded to the Good performers for the promotion decison and to the Poor performers for the training decision. Although this finding may lend support to the organizing system brought about through the prior knowledge, it also lends support the possibility of bias in attention to and encoding of ratee behaviors suggested by Murphy et al. (1985). These authors proposed that a contrast effect in ratings, i.e., lower ratings given an average 74 performance when it follows a good performance than the same average performance when it follows a poor performance may be due to the selective attention and encoding of the follow-up performance. From the current study, it appears that the background information set the stage for attending to the subsequent information on the computer screen. Based on the positive or negative impression formed from the background descriptions, subjects determined the relevant information pieces to search. All the information contained within the computer screen represented nine average police officers. In addition, each group of six performance statements appeared at each officer position an approximately equal number of times. Thus, each police officer could have been accessed and selected an equal number of times. However, search did vary by appraisal purpose. Murphy et al. (1985) described two processes that may occur when evaluating the effect of past performance on present performance evaluations. Raters can attempt to assimilate or integrate previous perceptions of a ratee with current observations. On the reverse side, evaluations may be biased in a direction opposite to the previous perceptions of performance. An examination of the process by which information was acquired for this study would indicate the former: raters had formed an opinion for six of the officers and interpreted the subsequent information in terms of these opinions. This phenomenon resembles the hypothesis-confirmation research of Darley and Fazio (1980) and Swann and Read (1978). These authors posited that evaluators seek information that will confirm their impressions and discount information that contradicts the impression. 75 The extent to which a confirmation or discounting process took place would have to be addressed by future research. Limitations of the Study The present study was an exploratory investigation of the decision making processes underlying performance evaluations. Several limitations to the study’s outcomes resulted. First, as mentioned previously, the design employed hindered the impact of appraisal purpose on search behavior. The relative influence of collecting information for a promotion/administrative decision versus a training/developmental decision needs to be examined when one individual is performing both sets of decisions (as is typical in an organizational setting). In addition, it is necessary to assess the constructs behind any differential search behavior. Although subjects in the present study reported that the purpose for the performance evaluation was important, it is unclear whether this construct led to information search or whether the demands of the task to select the best suited individual led to search. Second, in designing the study, an attempt was made to control for the confirmatory/disconfirmatory relationship between the background information and the computer information. Information content was placed in nine different locations for roughly an equal number of subjects. Although across all six dimensions, officer performance was average, some dimensions reflected slightly above average performance and some dimensions reflected slightly below average performance. Consequently, based on the information accessed, some subjects perceived a discrepancy between 76 the background information and the computer information, which may have resulted in search behavior in other than the hypothesized directions. Third, the number of dimensions used in the present manipulation was not sufficient to create variance in search depth or sequence. In addition to making the purpose manipulation stronger through a within-subjects design, the expansion of dimension number may force the subject to appreciate the differences in appraisal purposes and search accordingly. Also, the addition of dimensions may result in a more operationally complex information gathering strategy, which may help us understand mechanisms underlying information processing in the appraisal context more completely. Related to this issue, instructions used in the current experiment told subjects that they could search for as little or as much information as necessary to complete the performance evaluation. Matte (1982) instructed his subjects to search as little information as possible and Nilliams at al. (1985) placed a ceiling on the amount of information that could be searched. These authors found search differences among the subjects. Possibly, the ability to search as much information as necessary may actually defeat the purpose and manipulations of interest. Finally, two cautions need to be discussed pertaining to the present investigation and process tracing studies using the information board. First, as each piece of information is accessed from the manual or computerized information board, the researcher assumes that it has been attended tolrecognized and encoded. However, as the 77 technique presently operates, process tracing via the information board does not examine the encoding process or the influence of short term memory on subsequent acquisitons. Specifically, although it is assumed that all pieces gathered were cognitively processed, the information board cannot detect which pieces of information were accessed but ignored and which pieces of information were gathered and encoded. Lastly, the present study utilized a computerized information board for examining the search process. Although the computerized method allows the experimenter to unobtrusively monitor predecisional information acquisition, the subjects’ accountability reduces to zero. Consequently, the subject may become lost in the process of (reIaccessing information and lose sight of the decision to be made. Guidelines for Process Tracing Studies Based on the latter two observations and the experience resulting from many experimental sessions, several guidelines are offered for researchers interested in using process tracing techniques, particularly the information board. First, this technique should be used only for 'research examining the pre-solution activities of decision makers. Process tracing allows the researcher to investigate what information was examined and in what order it was accessed. One issue of particular importance is the number of dimensions used in an investigation. The optimal number of attributes for particular decision problems have not yet been determined; however, since dimensions are more likely to be ignored than alternatives (Russo, 78 1974), it is necessary to select dimensions that simulate the actual environment and are sufficiently independent to represent distinctive information. A second group of issues to be considered concerns insurances regarding the usefulness of the data that is collected from information board studies. Often, data gatherd from process tracing studies are used to infer decison making strategies (i.e., what dimension were eliminated, how the alternatives were evaluated). Therefore, the validity of these inferred strategies depends on the quality of the data gathered from the information acquisition process. Safeguards include creating an artificial source against which the individual’s decision is to be compared. In addition, it is necessary to prevent the tendency for subjects to randomly acquire all the available information. This randomness typically results in a search process where the subject attempts to overcome his/her confusion rather than access meaningful information. One solution to this difficulty involves the instructions for searching information provided at the beginning of the experiment. Instruction requiring subjects to search I'as much information as necessary“ to complete the task often falls prey to the overly extensive search process just mentioned. Changing the instructions to convey “searching sufficient information" may alleviate these problems and insure the quality of the data. It is important, however, to use instructions that coincide with the goals of the task. The effects of instructions on the search process has not been empirically tested. For example, a researh project assessing the influences of a satisficing versus optimizing strategy may result in qualitatively different outcomes if the instructions were to 79 ”search as much' versus 'search as little.‘I Empirical research need to be conducted for assessing the differences that may develop as a function of these task attributes. Finally, on the conceptual side, the information board provides an excellent mechanism for monitoring the external information acquisition activities of a decision maker. If resources permit, researchers should attempt to supplement this information with a procedure such as verbal protocols that can assist in understanding "why'I particular alternatives or attributes were attractive/important. Although process tracing techniques provide insightful information pertaining to the influence of task attributes on decision outcomes, the combination of techniques will allow the researchers to provide a rationale for the behaviors being monitored. Implications and Future Directions Overall, this study provided evidence that the evaluation of present performance is evaluated in terms of past performance and the reason for collecting the performance information. It is important to consider the strength of the prior information. The influence of appraisal purpose on search behavior became relevant only in relation to the level of ratee previous performance. When raters felt that an individual performed poorly in the past, those individuals were no longer considered viable candidates for promotion (vice versa for training). In actuality, raters may not even attempt to observe new behaviors to see if performance level has improved. Future research needs to explore why prior information had such a 80 pronounced effect in the present study and/or if the effect continues over time. A useful starting point might be a partial replication of the present study with two modifications. First subjects in the current study realized the linkage between Officer “A“ in the background information and Officer “A" on the computer data base. However, the overly positive statements used to produce the prior impression conflicted with the overall “average“ performance depicted on the computer. Consequently, efforts should be made to insure a consistent relationship between the background information and the currently available information (i.e., use statements for the computer task that reflect high average or high performance for the 'good' performers). Additonally, the second modification would involve removing the "no information“ option and increasing the number of poor performers to four and the number of good performers to four. To examine the influence of the prior information over time and future information gathering, subjects from the replication should return for a follow-up session. Initially, subjects should provide an assessment of each candidate and possibly recall items representing each candidate. In this follow-up, two of the four previously good performers are depicted in terms of low-average to low statements and two of the four previously poor performers are depicted in terms of high average and high statements. This manipulation attempts to examine the extent to which the prior information that is stored in memory continues to operate as an organizing scheme when faced with new information. Secondly, this task attempts to examine whether raters seek to confirm what is known (e.g., assimilate the new information into a presently existing scheme). If 81 the prior information that was entered into memory during the initial task no longer exists, information search should be relatively equivalent across ratees. If raters attempt to confirm that which is known, information gathering should focus in the ratees that are consistent with the prior information. The question then becomes the following: given a decision task for a particular purpose, will raters continue to focus mostly on those thought to be good (poor) performers and ignore the rest? In other words, can an initially bad impression be overcome and/or what happens to a good impression when faced with contradictory information. Other possibilities for assessing information processing issues is open for inquiry. Previous research has demonstrated the motivational role of appraisal purpose in performance ratings. In addition, the current study and the research by Williams et al. (1985) have provided preliminary evidence regarding the role of appraisal purpose in information processing issues. It is now important to address appraisal purpose as both a motivational and cognitive variable to gain a fuller understanding of accuracy issues. Second, an issue related to prior knowledge is the amount of information one has regarding a group of ratees. For instance, at what point would a rater feel confident in the information possessed and not need to search for new information. Finally, further development of information processing methodologies is warranted. Limited research has utilized the process tracing methodologies. Consequently, the boundaries surrounding 82 the interpretations from data collected from such a method have not been established. Yet, the wealth of information provided by this method provides new energy for the delineation of decision making and information processing behaviors. Ideally, process tracing should be supplemented with other methodologies for assessing attentional issues (e.g., verbal protocols) to gain greater confidence in the information provided by this method. In addition, research needs to bridge the linkages between the phases proposed in information processing theories to fully appreciate the antecedents and consequents of the gathering, storing, and integration cycles. APPEN DI CES APPENDIX A 83 Appendix A Introduction: Administrative (Promotion) Decision W E L C D N E This exercise is a simulation of a particular type of decision faced by supervisory personnel in most organizations. Today, you will be taking the role of a police sergeant and will be required to make a series of decisions regarding the performance evaluation of your subordinate police officers. Organizations utilize performance evaluation information for a variety of different purposes. From the organization's standpoint, the most important use of performance appraisal information is to make administrative decisions such as promoting an individual to a higher level in the company. Awarding a promotion to an individual who does not deserve it often results in negative consequences for the individual (dissatisfied subordinates) and the organization (reduced efficiency and effectiveness). Today, you will be required to evaluate the performance of nine police officers in order to select one officer to be promoted to police detective. Be careful. Wrong promotion decisions reflect extremely poorly on the individual providing the recommendation. ‘ PRESS THE RETURN BUTTON FOR FURTHER EXPLANATION 84 Appendix A (cont’d) When a supervisor determines the promotability of his/her employees to a higher level (e.g., production worker to foreman; resident nurse to head nurse), several factors may be taken into account. For example, the decision to promote an individual may be based on attendance records, performance on important dimensions of job performance, or scores on job knowledge exams. As seargent, your task will involve examining the performance of your nine officers on different performance dimensions and then to select the one officer that most deserves the promotion to police detective. If you have any questions, please ask the experimenter for assistance. If you do not have any questions, press the RETURN button and you will receive more specific instructions about your task. 85 Appendix A (cont’d) To aid you in the search process, you will be presented two lists. One list contains the employees you presently supervise and is labeled ALTERNATIVES. The second list contains a number of different factors that you might want to consider in evaluating your subordinates and is labeled DIMENSIONS. For example, you might encounter a screen of information such as: ALTERNATIVES DIMENSIONS 1=EMPLOYEE N 1=INITIATIVE 2=EMPLOYEE X 2=DEMEANOR 3=EMPLOYEE Y 3=ABSENTEEISM 4=ENPLOYEE Z 4=JOB KNOWLEDGE SCORE As you can see, each alternative and each dimension are identified by a number. To begin searching for information, you will be asked two questions: (1) the alternative number about which you would like information and (2) the dimension number about which you would like to receive information. Using the number keys on the row above the typewriter keypad, simply type the number corresponding to the alternative you would like and then type the number corresponding to the dimension you would like. PRESS THE RETURN BUTTON TO CONTINUE 86 Appendix A (cont’d) CONFUSED? Let's go through the evaluation process in detail. ALTERNATIVES DIMENSIONS 1=EMPLOYEE w 1=INITIATIVE 2=EMPLOYEE X 2=DEMEANOR 3=EMPLOYEE Y 3=ABSENTEEISM 4=EMPLOYEE Z 4=JOB KNOWLEDGE SCORE To begin the search process, you will choose one alternative and one dimension of information describing that alternative. You will continue this procedure until you have gathered enough information to choose one employee to be promoted to police detective. PRESS THE RETURN BUTTON TO CONTINUE 87 Appendix A (cont’d) To see how this procedure works, let’s begin with the following lists: ALTERNATIVES DIMENSIONS 1=EMPLOYEE N 1=INITIATIVE 2=EMPLOYEE X 2=DEMEANOR 3=EMPLOYEE Y =ABSENTEEISM 4=EMPLOYEE Z 4=JOB KNOWLEDGE SCORE The following message will appear below the alternatives and dimensions: ENTER THE NO. OF THE ALTERNATIVE AND HIT RETURN ? w." ENTER THE NO. OF THE DIMENSION AND HIT RETURN Let's assume that you are interested in EMPLOYEE N'S DEMEANOR. You would press -1- for EMPLOYEE w, the RETURN button, and then -2— for DEMEANOR and the RETURN button. The present screen will disappear and the requested information will be shown on the next screen as follows: PRESS THE RETURN BUTTON TO CONTINUE 88 Appendix A (cont'd) Employee N reports for duty with uncombed hair and an obvious hangover. PRESS THE RETURN BUTTON TO CONTINUE" 89 Appendix A (cont’d) At this point, the computer will print the following message: ENTER 1: IF YOU NEED MORE INFORMATION 2: IF YOU ARE READY TO MAKE THE FINAL PROMOTION DECISION Let's assume that you are not ready to select the employee to be promoted and would like additional information. You would press a -l- and the RETURN button. The computer will reprint the original menu of employees and dimensions on the next screen. PRESS THE RETURN BUTTON TO CONTINUE 90 Appendix A (cont’d) ALTERNATIVES DIMENSIONS 1=EMPLOYEE N 1=INITIATIVE =EMPLOYEE X =DEMEANOR 3=EMPLOYEE Y 3=ABSENTEEISM 4=EMPLOYEE Z 4=JOB KNOHLEDGE SCORE ENTER THE NO. OF THE ALTERNATIVE AND HIT RETURN ? ENTER THE NO. OF THE DIMENSION AND HIT RETURN ? Now let's suppose you want to know the ABSENTEEISM rate for EMPLOYEE Y. You would type in a 3 for EMPLOYEE Y and a 3 for ABSENTEEISM. PRESS THE RETURN BUTTON TO CONTINUE 91 Appendix A (cont’d) Now the computer prints the following message: The personnel file indicates that Employee Y has averaged 3 absences per month over the last 12 months. PRESS THE RETURN BUTTON TO CONTINUE 92 Appendix A (cont’d) At this point, the computer will print the message: following ENTER 1: IF YOU NEED MORE INFORMATION 2: IF YOU ARE READY TO MAKE THE FINAL PROMOTION DECISION Again, let's assume that you are not ready to select the candidate for promotion. After pressing -1- for more information, the computer will reprint the original menu on the next screen. PRESS THE RETURN BUTTON TO CONTINUE 93 Appendix A (cont’d) ALTERNATIVES DIMENSIONS 1=EMPLOYEE W 1=INITIATIVE 2=EMPLOYEE X 2=DEMEANOR 3=EMPLOYEE Y 3=ABSENTEEISM 4=EMPLOYEE Z 4=JOB KNOWLEDGE SCORE ENTER THE NO. ENTER THE NO. OF THE ALTERNATIVE AND HIT RETURN ? OF THE DIMENSION AND HIT RETURN ? Now let's assume that you want to see the JOB KNOWLEDGE SCORE for EMPLOYEE X. You would type a 2 for EMPLOYEE X and a 4 for JOB KNOWLEDGE SCORE. PRESS THE RETURN BUTTON TO CONTINUE 94 Appendix A (cont’d) Employee X scored in the 87th percentile on the Job Knowledge Exam. PRESS THE RETURN BUTTON TO CONTINUE 95 Appendix A (cont'd) Again, the computer will print the following message at this time: ENTER 1: IF YOU NEED MORE INFORMATION 2: IF YOU ARE READY TO MAKE THE FINAL PROMOTION DECISION At this point, let's assume that you have searched a sufficient number of different dimensions of performance for each of the employees. You are ready to select the candidate for promotion. You would type a -2- and the RETURN button. PRESS THE RETURN BUTTON TO CONTINUE 96 Appendix A (cont'd) The computer will now reprint the set of alternatives as follows: ALTERNATIVES 1=EMPLOYEE =EMPLOYEE 3=EMPLOYEE 4=EMPLOYEE N-<><£ ENTER IF YOU ARE NOT READY TO MAKE A DECISION: ENTER IF YOU READY TO MAKE A DECISION: Since you are ready to make a decision, you would press the -Y- key. The computer will then ask you to enter your decision in the following manner: ENTER THE NO. REPRESENTING THE OFFICER YOU WISH TO PROMOTE AND HIT RETURN: Let's assume that you have decided to promote EMPLOYEE X. You would type in a 2 and hit the RETURN button. PRESS THE RETURN BUTTON TO CONTINUE 97 Appendix A (cont'd) Your final decision will appear on the printer next to the computer. Are you ready to continue and make your own responses? If you are ready, press the RETURN button to continue. If you are not ready, ask the experimenter to clarify an questions you may have. GOOD LUCK! PRESS THE RETURN BUTTON TO CONTINUE 98 Appendix A (cont’d) Introduction: Developmental (Training) Decision W E L C O M E This exercise is a simulation of a particular type of decision faced by supervisory personnel in most organizations. Today, you will be taking the role of a police sergeant and will be required to make a series of decisions regarding the performance evaluation of your subordinate police officers. Organizations utilize performance evaluation information for a variety of different purposes. One of the uses of performance appraisal information is to identify employees who are lacking important knowledges and skills necessary for job performance. These individuals may then be recommended for remedial training. From the organization’s standpoint, these recommendations for training programs are noted only after more important budgetary concerns are determined. Today, you will be required to evaluate the performance of nine police officers in order to select one officer to send to remedial training. Although sending the wrong individual to remedial training is not considered a disaster, do a decent job. PRESS THE RETURN BUTTON FOR FURTHER EXPLANATION 99 Appendix A (cont'd) When a supervisor evaluates the strengths and deficiencies of different employees in order to determine whether any of these individuals need to be sent to a remedial training program to improve knowledges, skills, or abilities, several factors may be taken into account. For example, the decision to send an individual to remedial training may be based on noticed deficiencies in important aspects of job performance, poor job knowledge exam scores, or attitudinal diffuculties such as poor attendance. As sergeant, your task will involve examining the performance of your nine officers on different performance dimensions and then to select the one officer who is most in need of remedial training. If you have any questions, please ask the experimenter for assistance. If you do not have any questions, press the RETURN button and you will receive more specific instructions about your task. 100 Appendix A (cont'd) To aid you in the search process, you will be presented two lists. One list contains the employees you presently supervise and is labeled ALTERNATIVES. The second list contains a number of different factors that you might want to consider in evaluating your subordinates and is labeled DIMENSIONS. For example, you might encounter a screen of information such as: ALTERNATIVES DIMENSIONS 1=EMPLOYEE W 1=INITIATIVE 2=EMPLOYEE X 2=DEMEANOR 3=EMPLOYEE Y 3=ABSENTEEISM 4=EMPLOYEE Z 4=JOB KNOWLEDGE SCORE As you can see, each alternative and each dimension are identified by a number. To begin searching for information, you will be asked two questions: (1) the alternative number about which you would like information and (2) the dimension number about which you would like to receive information. Using the number keys on the row above the typewriter keypad, simply type the number corresponding to the alternative you would like and then type the number corresponding to the dimension you would like. PRESS THE RETURN BUTTON TO CONTINUE 101 Appendix A (cont'd) CONFUSED? Let’s go through the evaluation process in detail. ALTERNATIVES DIMENSIONS 1=EMPLOYEE W 1=INITIATIVE 2=EMPLOYEE X 2=DEMEANOR 3=EMPLOYEE Y 3=ABSENTEEISM 4=EMPLOYEE Z 4=JOB KNOWLEDGE SCORE To begin the search process, you will choose one alternative and one. dimension of information describing that alternative. You will continue this procedure until you have gathered enough information to choose one employee to send to remedial training. PRESS THE RETURN BUTTON TO CONTINUE 102 Appendix A (cont‘d) To see how this procedure works, let's begin with the following lists: ALTERNATIVES DIMENSIONS 1=EMPLOYEE W 1=INITIATIVE 2=EMPLOYEE X 2=DEMEANOR 3=EMPLOYEE Y 3=ABSENTEEISM 4=EMPLOYEE Z 4=JOB KNOWLEDGE SCORE The following message will appear below the alternatives and dimensions: ug‘ ENTER THE NO. OF THE ALTERNATIVE AND HIT RETURN "$1 ENTER THE NO. OF THE DIMENSION AND HIT RETURN Let's assume that you are interested in EMPLOYEE W'S DEMEANOR. You would press -1- for EMPLOYEE W, the RETURN button, and then -2— for DEMEANOR and the RETURN button. The present screen will disappear and the requested information will be shown on the next screen as follows: PRESS THE RETURN BUTTON TO CONTINUE 103 Appendix A (cont'd) Employee W reports for duty with uncombed hair and an obvious hangover. PRESS THE RETURN BUTTON TO CONTINUE" 104 Appendix A (cont'd) At this point, the computer will print the following message: ENTER 1: IF YOU NEED MORE INFORMATION 2: IF YOU ARE READY TO MAKE THE FINAL TRAINING DECISION Let's assume that you are not ready to select the employee to be promoted and would like additional information. You would press a -1- and the RETURN button. The computer will reprint the original menu of employees and dimensions on the next screen. PRESS THE RETURN BUTTON TO CONTINUE 105 Appendix A (cont’d) ALTERNATIVES DIMENSIONS 1=EMPLOYEE W 1=INITIATIVE 2=EMPLOYEE X 2=DEMEANOR 3=EMPLOYEE Y 3=ABSENTEEISM 4=EMPLOYEE Z 4=JOB KNOWLEDGE SCORE ENTER THE NO. OF THE ALTERNATIVE AND HIT RETURN 7 D‘;| ENTER THE NO. OF THE DIMENSION AND HIT RETURN Now let’s suppose you want to know the ABSENTEEISM rate for EMPLOYEE Y. You would type in a 3 for EMPLOYEE Y and a 3 for ABSENTEEISM. PRESS THE RETURN BUTTON TO CONTINUE 106 Appendix A (cont'd) Now the computer prints the following message: The personnel file indicates that Employee Y has averaged 3 absences per month over the last 12 months. PRESS THE RETURN BUTTON TO CONTINUE 107 Appendix A (cont'd) At this point, the computer will print the following message: ENTER 1: IF YOU NEED MORE INFORMATION 2: IF YOU ARE READY TO MAKE THE FINAL TRAINING DECISION Again, let's assume that you are not ready to make the final selection for the remedial training program. After pressing -1- for more information, the computer will reprint the original menu on the next screen. PRESS THE RETURN BUTTON TO CONTINUE 108 Appendix A (cont’d) ALTERNATIVES DIMENSIONS 1=EMPLOYEE W 1=INITIATIVE 2=EMPLOYEE X 2=DEMEANOR 3=EMPLOYEE Y 3=ABSENTEEISM 4=EMPLOYEE Z 4=JOB KNOWLEDGE SCORE ENTER THE NO. ENTER THE NO. OF THE ALTERNATIVE AND HIT RETURN ? OF THE DIMENSION AND HIT RETURN ? Now let‘s assume that you want to see the JOB KNOWLEDGE SCORE for EMPLOYEE X. You would type a 2 for EMPLOYEE X and a 4 for JOB KNOWLEDGE SCORE. PRESS THE RETURN BUTTON TO CONTINUE 109 Appendix A (cont’d) Employee X scored in the 87th percentile on the Job Knowledge Exam. PRESS THE RETURN BUTTON TO CONTINUE 110 Appendix A (cont’d) Again, the computer will print the following message at this time: ENTER 1: IF YOU NEED MORE INFORMATION 2: IF YOU ARE READY TO MAKE THE FINAL TRAINING DECISION At this point, let's assume that you have searched a sufficient number of different dimensions of performance for each of the employees. You are ready to select the individual for remedial training. You would type a -2- and then hit the RETURN button. PRESS THE RETURN BUTTON TO CONTINUE 111 Appendix A (cont'd) The computer will now reprint the set of alternatives as follows: ALTERNATIVES 1=EMPLOYEE 2=EMPLOYEE 3=EMPLOYEE 4=EMPLOYEE h4<><£ ENTER IF YOU ARE NOT READY TO MAKE A DECISION: ENTER IF YOU READY TO MAKE A DECISION: Since you are ready to make a decision, you would press the -Y- key. The computer will then ask you to enter your decision in the following manner: ENTER THE NO. REPRESENTING THE OFFICER IN NEED OF TRAINING AND HIT RETURN: Let's assume that you have decided to send EMPLOYEE X to training. You would type in a 2 and hit the RETURN button. PRESS THE RETURN BUTTON TO CONTINUE 112 Appendix A (cont’d) Your final decision will appear on the printer next to the computer. Are you ready to continue and make your own responses? If you are ready, press the RETURN button to continue. If you are not ready, ask the experimenter to clarify an questions you may have. GOOD LUCK! PRESS THE RETURN BUTTON TO CONTINUE APPENDIX B 113 Appendix B Promotion: Practice Evaluation Task Now that you are familiar with the search procedure, you will be given an opportunity to practice your new skills prior to conducting the actual performance evaluations. For this practice task, you will be evaluating the performance of four nurses in order to select one nurse to be promoted. These four nurses will be described by four performance dimensions. Search enough information to make a good promotion decision. The alternatives and dimensions are as follows: ALTERNATIVES DIMENSIONS 1=NURSE L 1=PATIENT-FAMILY TEACHING 2=NURSE M 2=PROFESSIONAL NURSING 3=NURSE N 3=BASIC NURSING CARE =NURSE O =DOCUMENTATION Remember to choose one alternative and one dimension at a time. Type in the number corresponding the desired alternative, hit RETURN, and then type the number corresponding to the desired dimension and hit RETURN. Continue this procedure until you are ready to select one nurse to be promoted. PRESS THE RETURN BUTTON TO CONTINUE 114 Appendix B (cont’d) Listed below are the definitions of the dimensions you will be using in the practice session. Please note that you have been given a paper copy of this information. Patient-Family Teaching. Activities involving teaching, advising, giving directions, and demonstrating to patients and families. Professional Nursing. Activities directed at treating patients for the illnesses they were admitted for and any complications. Basic Nursing Care. Activities necessary for the comfort and well-being of patients, to maintain their health and fight infection, REGARDLESS OF THE ILLNESS FROM WHICH THEY ARE SUFFERING. Dogumentation. Activities necessary for record keeping and documenting staff actions; activities involving written communication. PRESS THE RETURN BUTTON TO BEGIN THE PRACTICE 115 Appendix B Training: Practice Evaluation Task Now that you are familiar with the search procedure, you will be given an opportunity to practice your new skills prior to conducting the actual performance evaluations. For this practice task, you will be evaluating the performance of four nurses in order to select one nurse in need of training. These four nurses will be described by four performance dimensions. Search enough information to make a good promotion decision. The alternatives and dimensions are as follows: ALTERNATIVES DIMENSIONS IBNURSE L 1=PATIENT-FAMILY TEACHING 2=NURSE M 2=PROFESSIONAL NURSING =NURSE N 3=BASIC NURSING CARE 4=NURSE O 4=DOCUMENTATION Remember to choose one alternative and one dimension at a time. Type in the number corresponding the desired alternative, hit RETURN, and then type the number corresponding to the desired dimension and hit RETURN. Continue this procedure until you are ready to select one nurse in need of remedial training. PRESS THE RETURN BUTTON TO CONTINUE 116 Appendix B (cont'd) Listed below are the definitions of the dimensions you will be using in the practice session. Please note that you have been given a paper copy of this information. Patient-Family Tgéching. Activities involving teaching, advising, giving directions, and demonstrating to patients and families. Professional Nursing. Activities directed at treating patients for the illnesses they were admitted for and any complications. Basic Nursing Care. Activities necessary for the comfort and well-being of patients, to maintain their health and fight infection, REGARDLESS OF THE ILLNESS FROM WHICH THEY ARE SUFFERING. Qgggmgntation. Activities necessary for record keeping and documenting staff actions: activities involving written communication. PRESS THE RETURN BUTTON TO BEGIN THE PRACTICE APPENDIX C 117 Appendix C Information Values for Practice Session THE CONTENT OF THE DATA FILE CURRENT DATA VERSION : 0 NO OF ALTERNATIVES : 4 NO OF DIMENSIONS : 4 NO NAME OF ALTERNATIVE 1 NURSE L 2 NURSE M 3’ NURSE N 4 NURSE O NO NAME OF DIMENSION l PATIENT-FAMILY TEACHING 2 PROFESSIONAL NURSING 3 BASIC NURSING CARE 4 DOCUMENTATION NO OF A. NO OF D. CONTENT l 1 Nurse L sends patients home without teaching them what they need to know after discharge (i.e., diabetic diet or giving themselves insulin injections). 1 2 Nurse L orders test for patients that the physician has not recommended because patients show symptoms of complication (i.e., poor color or respiration). 1 3 Nurse L asks patients if they want something different to eat if they are not eating the food that is served to them at meals. 1 4 Nurse L does not document completely (i.e., documenting symptoms but omitting treatment and omitting the response to treatment. 2 1 Nurse M gives patients a brief surface explanation for the purpose of a medication rather than explaining in more detail (i.e., 'for your heart'). 2 2 Nurse M sees patients beginning to show signs of complication and pre-prepares in anticipation of physicians' orders (such as having supplies set up). 118 Appendix C (cont'd) 2 3 Nurse M removes call signal light from patients' reach so that patients will not signal for nurse. 2 4 Nurse M writes down on chart particular steps which physician has specified nurses should go through in order to have reference the next time procedure is done. 3 1 Nurse N does library searches to get information for teaching patients. 3 2 Nurse N does not realize patients are in shock when they are pale, unresponsive and have low blood pressure. 3 3 Nurse N refuses to change patients' sheets when a spot is on them. 3 4 Nurse N writes down incorrect information on consent form (such as wrong procedure). 4 1 Nurse O sets up meetings with patients and/or families and then does not show up. 4 2 Nurse 0 does not call for help when having a problem repeatedly (i.e., having to poke patients three times to start an IV). 4 3 Nurse O gives back care two or three times a night to patients who are totally bedridden. 4 4 Nurse O keeps documentation on what has been taught to patients and families in order to remember what needs to be taught. APPENDIX D Appendix D Promotion: Police Officer Performance Dimensions Now that you have done an example, you should be ready to begin the experiment. You will be presented with a list of your nine subordinate police officers (alternatives) who are described by six dimensions of officer performance (dimensions). Your task is to examine a sufficient amount of information in order to select one officer to promote to police detective. Listed below are the six performance dimensions that you will be using in this task. Please familiarize yourself with these dimensions and note the paper copy next to the computer. Job Knowledge. Awareness of procedures, laws, techniques related to the patrol function (including the application of prior training), court rulings, and changes in any of them. Judgment. Observation and analytic assessment of the situation and taking appropriate action after consideration of alternative approaches. Qggggggbilitx/Rgliability. Predictable job behaviors (including job attendance, promptness, acceptance of responsibility, accuracy in work, and reaction to boredom, stress, and criticism). PRESS THE RETURN BUTTON TO CONTINUE 120 Appendix D (cont d) Communication. Ability to make oneself understood and gather and transmit information both in oral and written fashion. Work Attidug. General orientation toward the law enforcement profession and the department--satisfaction in doing a good job & the fair and objective enforcement and administration of the law. Dealing with Others. Ability to deal with people with whom the officer comes into contact during daily performance (including the public, fellow officers, and supervisors). FAMILIARIZE YOURSELF WITH THESE PERFORMANCE DIMENSIONS. STOP AND NOTIFY THE EXPERIMENTER WHEN YOU ARE DONE. YOU WILL BE PRESENTED BACKGROUND INFORMATION REGARDING SOME OF THE OFFICERS YOU WILL BE EVALUATING. DO NOT PROCEED UNTIL YOU HAVE READ THIS INFORMATION CAREFULLY!" PRESS THE RETURN BUTTON WHEN YOU HAVE FINISHED READING THE BACKGROUND INFORMATION AND ARE READY TO BEGIN THE TASK. ASK THE EXPERIMENTER TO CLARIFY ANY QUESTIONS YOU MAY HAVE. 121 Appendix D (cont’d) Training: Police Officer Performance Dimensions Now that you have done an example, you should be ready to begin the experiment. You will be presented with a list of your nine subordinate police officers (alternatives) who are described by six dimensions of officer performance (dimensions). Your task is to examine a sufficient amount of information to select one individual to send to the remedial training program. Listed below are the six performance dimensions that you will be using in this task. Please familiarize yourself with these dimensions and note the paper copy next to the computer terminal. Job Knowledge. Awareness of procedures, laws, techniques related to the patrol function (including the application of prior training), court rulings, and changes in any of them. Judgment. Observation and analytic assessment of the situation and taking appropriate action after consideration of alternative approaches. Degendability/Reliability. Predictable job behaviors (including job attendance, promptness, acceptance of responsibility, accuracy in work, and reaction to boredom, stress, and criticism). PRESS THE RETURN BUTTON TO CONTINUE 122 Appendix D (cont’d) Communicgtion. Ability to make oneself understood and gather and transmit information both in oral and written fashion. Work Attidug. General orientation toward the law enforcement profession and the department--satisfaction in doing a good job & the fair and objective enforcement and administration of the law. Dealing with Others. Ability to deal with people with whom the officer comes into contact during daily performance (including the public, fellow officers, and supervisors). FAMILIARIZE YOURSELF WITH THESE PERFORMANCE DIMENSIONS. STOP AND NOTIFY THE EXPERIMENTER WHEN YOU ARE DONE. YOU WILL BE PRESENTED BACKGROUND INFORMATION REGARDING SOME OF THE OFFICERS YOU WILL BE EVALUATING. DO NOT PROCEED UNTIL YOU HAVE READ THIS INFORMATION CAREFULLY!" PRESS THE RETURN BUTTON WHEN YOU HAVE FINISHED READING THE BACKGROUND INFORMATION AND ARE READY TO BEGIN THE TASK. ASK THE EXPERIMENTER TO CLARIFY ANY QUESTIONS YOU MAY HAVE. APPENDIX E 123 Appendix E Promotion Background Information Packet To: New Sergeant From: T. J. Barnes Re: Quarterly Performance Evaluations NELCOME!! I wasn’t informed of my transfer until two days ago. Unfortunateer I have to rely on you to perform the evaluations. Due to the rush with which I had to exit from the office, I only had time to write up a brief description for 6 out of the 9 officers. I hope these help! Additionally, promotions are very important to this precinct. Detectives and sergeants who make mistakes are very costly. Make sure you evaluate these people carefully. 124 Appendix E (cont’d) Officer A This officer seldom has to ask others about points of law. Officer A takes the appropriate action with little supervision. This officer remains cool under most circumstances. Officer A submits extremely neat, in depth reports and is at ease when speaking with the public. Officer A shows interest in the job and continually seeks ways of improving self. Finally, Officer A commends citizens for heroic acts. 125 Appendix E (cont’d) Officer B In drug cases, Officer B collects evidence that will be admissable in court. This officer withholds fire in situations where gunfire would endanger innocent bystanders. Even on cold and rainy nights, Officer B checks the back doors of businesses on the beat. This officer talks to people of lower educational levels without talking down to them. Officer B devotes some off-duty hours to establishing sports activities and programs in lower class areas. Lastly, This officer works willingly with other officers having trouble adjusting to various duties. 126 Appendix E (cont’d) Officer C This officer follows correct procedures for evidence preservation at the scene of a crime. Officer C notices potentially dangerous situations before anything actually occurs. This officer always arrives at work in time to check the latest reports and to receive other assignments. Officer C speaks slowly and ,clearly when testifying in court. Additionally, this officer is very polite and understanding while in contact with the public. Finally, Officer C relates well with both older and younger officers on the shift and has good relations with the public. 127 Appendix E (cont’d) Officer D Officer D is frequently charged with making false arrests. This officer requests a second opinion in many instances where the action is clearcut. While frisking a suspect, this officer overlooks a concealed weapon, even though the officer is perfectly aware of the procedures for searching a suspect. Officer D talks in an unintelligible manner over the radio. This officer smiles, waves back, and continues driving when a derelict calls for assistance because “drunks” are not worthy of attention. Officer D sometimes irritates other officers by asking their advice and then criticizing that advice. 128 Appendix E (cont’d) Officer E When straightening up a homocide scene, Officer E handles a knife that had not been previously observed. When hearing a report of a robbery in progress nearby, this officer continues to write a traffic violation. Officer E drinks alcoholic beverages on duty which causes performance to deteriorate. Having extreme problems with the English language, written reports are seldom acceptable and this officer’s speech is often hard to understand. Officer E refuses to meet minimum departmental standards and bucks authority. Finally, this officer antagonizes superiors and fellow officers by starting rumors about them. 129 Appendix E (cont’d) Officer F Officer F is sometimes not aware of the motor vehicle code for some less frequent traffic violations. This officer evacuates a building threatened by a bomb but not the building next to it. Officer F disppears when dangerous situations arise. Having unsatisfactory grammar, this officer is unable to interact well with other officers or the public. Officer F refuses to participate in a study of police officers’ opinions being conducted by the local college. Finally, Officer F insults and bullies a father in front of his family. 130 Appendix E (cont’d) Rating of Profile Effectiveness Using the following 9-point scale, please indicate how effective you feel each individual performs as a police officer (1=very ineffective and 9=very effective). Use any number along the scale that corresponds to appropriate value. Mark your answer to the right of the Officer's name. If you did not view any background information about an officer, write in NA (Not Applicable). very very. ineffective effective OFFICER A __ OFFICER B __ OFFICER c __ OFFICER n OFFICER E OFFICER F __ OFFICER G _ OFFICER H __ OFFICER I _ 131 Appendix E (cont’d) Training Background Information Packet To: New Sergeant From: T. J. Barnes Re: Quarterly Performance Evaluations WELCOME!! I wasn't informed of my transfer until two days ago. Unfortunately, I have to rely on you to perform the evaluations. Due to the rush with which I had to exit from this Officer, I only had time to write up a brief description for 6 of the 9 officers. I hope these help! Additionally, training programs are Offered in this precinct when the budget sees fit. Do a decent job evaluating these people, but realize that training is not the most crucial reason for collecting performance information. 132 Appendix E (cont’d) Officer A This Officer seldom has to ask others about points of law. Officer A takes the appropriate action with little supervision. This officer remains cool under most circumstances. Officer A submits extremely neat, in depth reports and is at ease when speaking with the public. Officer A shows interest in the job and continually seeks ways of improving self. Finally, Officer A commends citizens for heroic acts. 133 Appendix E (cont’d) Officer B In drug cases, Officer B collects evidence that will be admissable in court. This officer withholds fire in situations where gunfire WOUld endanger innocent bystanders. Even on cold and rainy nights, Officer B checks the back doors of businesses on the heat. This officer talks to people Of lower educational levels without talking down to them. Officer B devotes some off-duty hours to establishing sports activities and programs in lower class areas. Lastly, This Officer works willingly with other Officers having trouble adjusting to various duties. 134 Appendix E (cont’d) Officer C This officer follows correct procedures for evidence preservation at the scene of a crime. Officer C notices potentially dangerous situations before anything actually occurs. This Officer always arrives at work in time to check the latest reports and to receive other assignments. Officer C speaks slowly and clearly when testifying in court. Additionally, this officer is very polite and understanding while in contact with the public. Finally, Officer C relates well with both Older and younger officers on the shift and has good relations with the public. 135 Appendix E (cont’d) Officer D Officer D is frequently charged with making false arrests. This officer requests a second Opinion in many instances where the action is clearcut. While frisking a suspect, this officer overlooks a concealed weapon, even though the Officer is perfectly aware of the procedures for searching a suspect. Officer D talks in an unintelligible manner over the radio. This Officer smiles, waves back, and continues driving when a derelict calls for assistance because “drunks" are not worthy Of attention. Officer D sometimes irritates other officers by asking their advice and then criticizing that advice. 136 Appendix E (cont’d) Officer E When straightening up a homocide scene, Officer E handles a knife that had not been previously observed. When hearing a report of a robbery in progress nearby, this officer continues to write a traffic violation. Officer E drinks alcoholic beverages on duty which causes performance to deteriorate. Having extreme problems with the English language, written reports are seldom acceptable and this officer’s speech is often hard to understand. Officer E refuses to meet minimum departmental standards and bucks authority. Finally, this officer antagonizes superiors and fellow officers by starting rumors about them. 137 Appendix E (cont’d) Officer F Officer F is sometimes not aware of the motor vehicle code for some less frequent traffic violations. This officer evacuates a building threatened by a bomb but not the building next to it. Officer F disppears when dangerous situations arise. Having unsatisfactory grammar, this officer is unable to interact well with other officers or the public. Officer F refuses to participate in a study of police Officers’ Opinions, being conducted by the local college. Finally, Officer F insults and bullies a father in front of his family. 138 Appendix E (cont'd) Rating of Profile Effectiveness Using the following 9-point scale, please indicate how effective you feel each individual performs as a police officer (1=very ineffective and 9=very effective). Use any number along the scale that corresponds to appropriate value. Mark your answer to the right of the officer’s name. If you did not view any background information about an officer, write in NA (Not Applicable). I - I I l I ------- I ------- I ------- I ------- I 1 very very ineffective effective OFFICER OFFICER 0313’ OFFICER OFFICER D OFFICER E OFFICER F OFFICER G OFFICER H OFFICER I APPENDIX F 139 Appendix F Matrix Contents for Version 1 CURRENT DATA VERSION : 1 NO OF ALTERNATIVES : 9 NO OF DIMENSIONS : 6 NAME OF ALTERNATIVE OFFICER OFFICER OFFICER OFFICER OFFICER OFFICER OFFICER OFFICER OFFICER nmummnumrs “101111710an 2 O‘UIJBUMF‘D NAME OF ALTERNATIVE JOB KNOWLEDGE JUDGMENT DEPENDABILITY/RELIABILITY COMMUNICATION WORK ATTITUDE DEALING WITH OTHERS NO OF A. NO OF D. CONTENT 1 1 Officer A arrests suspects for misdemeanors not committed while the officer was present. 1 2 Officer A issues warnings instead of tickets to traffic violators in particularly confusing intersections. 1 3 Officer A logs an unusual number of sick days and is often late for work. 1 4 Officer A turns in reports which are neat, accurate, and well written. 1 5 Officer A participates in exercises that contribute to a better quality of life both on and off the job. 1 6 Officer A is sometimes patient with citizens who speak little English. 140 Appendix F (cont'd) 2 1 Officer B usually abides by and implements the latest court rulings after being informed of them. 2 2 Upon arrival at the scene of a huge fire, Officer B sees the need for additional help and calls for it. 2 3 Depending on the work, Officer B sometimes shows signs of boredom. 2 4 Officer B speaks slowly and clearly when testifying in court. 2 5 Officer B perceives self as an expert and usually refuses training. 2 6 Officer B usually puts fellow officers at risk by refusing to share relevant information with them. 3 1 Officer C understands the difference between civil and criminal matters in practical applications on the streets. 3 2 Officer C calls for assistance and clears the area of bystanders when confronting a barricaded, heavily-armed suspect. 3 3 Officer C responds to calls, but often takes the long way to get there in the hopes that other officers will arrive first. 3 4 Officer C uses unsatisfactory grammar and thus is unable to communicate well with the public. 3 5 Officer C does the work assigned, yet also initiates own activities. 3 6 Officer C works well with others, but is not a real extrovert. 4 1 Officer D usually functions on the streets with minimal legal problems. 4 2 When out on a call, Officer D waits to complete a physical arrest until securing assistance. 4 3 Officer D panics upon receiving an emergency call. 4 4 Officer D has a highly developed vocabulary and uses it appropriately. 141 Appendix F (cont'd) 4 5 Officer D fails to recognize and correct own deficiencies without prompting by others. 4 6 Officer D is usually patient with citizens who speak very little English. 5 1 Officer E is consistently unaware of the general orders and departmental policy. 5 2 Officer E exhibits good judgment except under stressful situations. 5 3 Officer E uses only a minimum of sick days per year. 5 4 Officer E sometimes has to be asked to repeat a message sent over the radio. 5 5 Officer E is willing to work on investigations other than the current assignment. 5 6 Officer E usually gets along well with others, but has a tendency to 'snap'answers during conversations. 6 1 Officer F searches citizens' vehicles with probable cause which occasionally results in discovering smuggled narcotics. 6 2 Officer F calls for a backup in ALL situations. 6 3 Officer F sometimes reports for duty with a cold if trouble is expected in the city. 6 4 Officer F submits reports that are neatly done, but not always grammatically correct. . 6 5 Officer F attends special training schools during off-duty hours to improve efficiency and effectiveness as an officer. 6 6 Officer F often calls fellow workers 'misguided social workers' for showing interest in other people's problems. 7 1 Officer G is often unaware of the important facts in a case which results in losing a good case. 7 2 Officer 6 radios in position and discontinues a high speed chase when entering areas of high vehicle and pedestrian traffic. 142 Appendix F (cont'd) 7 3 Officer G answers calls from other precincts when necessary. 7 4 Officer G tries hard, but often stutters when excited and talks too much over the radio. 7 5 Officer G violates departmental rules and regulations when it is in the best interest of the public. 7 6 Officer G is a little overbearing once in a while--probably trying to be 'just like everyone else.’ 8 1 Officer H profits from past mistakes in court by appropriately correcting arrest and search procedures. 8 2 Officer H enters buildings with broken doors and windows instead of guarding the exits and calling for a backup. 8 3 Officer H has average attendance, handles stress situations as well as others and is sometimes excitable. B 4 At times, Officer H does not probe far enough and seems embarassed asking questions. 8 5 Officer H uses racially-toned language in front of minority groups. 8 6 Officer H maintains friendly relations with the civilians in the patrol area. 9 1 Officer I keeps a well organized file containing extensive information concerning the various duties of a police officer. 9 2 Officer I considers all the facts before making a decision and then sticks to it. 9 3 Officer I reports minor damages to the patrol car even though the officer will be held responsible. 9 4 Officer I includes trivial, irrelevant information in written reports and radio communication. 9 5 Officer I will go 'out of the way' to defy departmental regulations. . 9 6 Officer I takes pleasure in embarrassing a rape victim by concentrating unnecessarily on the particularly lewd details of the statement. 143 Appendix F (cont’d) Matrix Contents for Version 2 CURRENT DATA VERSION . 2 NO OF ALTERNATIVES : 9 NO OF DIMENSIONS : 6 NO NAME OF ALTERNATIVE 1 OFFICER A 2 OFFICER B 3 OFFICER C 4 OFFICER D 5 OFFICER E 6 OFFICER F 7 OFFICER G B OFFICER H 9 OFFICER I NO NAME OF ALTERNATIVE 1 JOB KNOWLEDGE 2 JUDGMENT 3 DEPENDABILITY/RELIABILITY 4 COMMUNICATION 5 WORK ATTITUDE 6 DEALING WITH OTHERS NO OF A. NO OF D. CONTENT 1 1 Officer A usually abides by and implements the latest court rulings after being informed of them. 1 2 Upon arrival at the scene of a huge fire, Officer A sees the need for additional help and calls for it. 1 3 Depending on the work, Officer A sometimes shows signs of boredom. 1 4 Officer A speaks slowly and clearly when testifying in court. 1 5 Officer A perceives self as an expert and usually refuses training. 1 6 Officer A usually puts fellow officers at risk by refusing to share relevant information with them. 144 Appendix F (cont’d) 2 1 Officer B understands the difference between civil and criminal matters in practical applications on the streets. 2 2 Officer B calls for assistance and clears the area of bystanders when confronting a barricaded, heavily-armed suspect. 2 3 Officer B responds to calls, but often takes the long way to get there in the hopes that other officers will arrive first. 2 4 Officer B uses unsatisfactory grammar and thus is unable to communicate well with the public. 2 5 Officer B does the work assigned, yet also initiates own activities. 2 6 Officer B works well with others, but is not a real extrovert. 3 1 Officer C usually functions on the streets with minimal legal problems. 3 2 When out on a call, Officer C waits to complete a physical arrest until securing assistance. 3 3 Officer C panics upon receiving an emergency call. 3 4 Officer C has a highly developed vocabulary and uses it appropriately. 3 5 Officer C fails to recognize and correct own deficiencies without prompting by others. 3 6 Officer C is usually patient with citizens who speak very little English. 4 1 Officer D is consistently unaware of the general orders and departmental policy. 4 2 Officer D exhibits good judgment except under stressful situations. 4 3 Officer D uses only a minimum of sick days per year. 4 4 Officer D sometimes has to be asked to repeat a message sent over the radio. 145 Appendix F (cont'd) 4 5 Officer D is willing to work on investigations other than the current assignment. 4 6 Officer D usually gets along well with others, but has a tendency to 'snap'answers during conversations. 5 1 Officer E searches citizens' vehicles with probable cause which occasionally results in discovering smuggled narcotics. S 2 Officer E calls for a backup in ALL situations. 5 3 Officer E sometimes reports for duty with a cold if trouble is expected in the city. 5 4 Officer E submits reports that are neatly done, but not always grammatically correct. 5 5 Officer E attends special training schools during off-duty hours to improve efficiency and effectiveness as an officer. 5 6 Officer E often calls fellow workers 'misguided social workers' for showing interest in other people’s problems. 6 1 Officer F is often unaware of the important facts in a case which results in losing a good case. 6 2 Officer F radios in position and discontinues a high speed chase when entering areas of high vehicle and pedestrian traffic. 6 3 Officer F answers calls from other precincts when necessary. 6 4 Officer F tries hard, but often stutters when excited and talks too much over the radio. 6 5 Officer F violates departmental rules and regulations when it is in the best interest of the public. 6 6 Officer F is a little overbearing once in a while-—probably trying to be 'just like everyone else.‘ 7 1 Officer G profits from past mistakes in court by approPriately correcting arrest and search procedures. 7 2 Officer G enters buildings with broken doors and windows instead of guarding the exits and calling for a backup. 146 Appendix F (cont'd) 4 5 Officer D is willing to work on investigations other than the current assignment. 4 6 Officer D usually gets along well with others, but has a tendency to 'snap’answers during conversations. S 1 Officer E searches citizens' vehicles with probable cause which occasionally results in discovering smuggled narcotics. 5 2 Officer E calls for a backup in ALL situations. 5 3 Officer E sometimes reports for duty with a cold if trouble is expected in the city. 5 4 Officer E submits reports that are neatly done, but not always grammatically correct. 5 5 Officer E attends special training schools during off-duty hours to improve efficiency and effectiveness as an officer. 5 6 Officer E often calls fellow workers 'misguided social workers' for showing interest in other people’s problems. 6 1 Officer F is often unaware of the important facts in a case which results in losing a good case. 6 2 Officer F radios in position and discontinues a high speed chase when entering areas of high vehicle and pedestrian traffic. 6 3 Officer F answers calls from other precincts when necessary. 6 4 Officer F tries hard, but often stutters when excited and talks too much over the radio. 6 5 Officer F violates departmental rules and regulations when it is in the best interest of the public. 6 6 Officer F is a little overbearing once in a while--probably trying to be 'just like everyone else.’ 7 1 - Officer G profits from past mistakes in court by appropriately correcting arrest and search procedures. 7 2 Officer G enters buildings with broken doors and windows instead of guarding the exits and calling for a backup. 147 Appendix F (cont'd) 7 3 Officer 6 has average attendance, handles stress situations as well as others and is sometimes excitable. 7 4 At times, Officer 6 does not probe far enough and seems embarassed asking questions. 7 5 Officer 6 uses racially-toned language in front of minority groups. 7 6 Officer 6 maintains friendly relations with the civilians in the patrol area. 8 1 Officer H keeps a well organized file containing extensive information concerning the various duties of a police officer. 8 2 Officer H considers all the facts before making a decision and then sticks to it. 8 3 Officer H reports minor damages to the patrol car even though the officer will be held responsible. 8 4 Officer H includes trivial, irrelevant information in written reports and radio communication. 8 5 Officer H will go 'out of the way' to defy departmental regulations. 8 6 Officer H takes pleasure in embarrassing a rape victim by concentrating unnecessarily on the particularly lewd details of the statement. 9 1 Officer I arrests suspects for misdemeanors not committed while the officer was present. 9 2 Officer 1 issues warnings instead of tickets to traffic violators in particularly confusing intersections. 9 3 Officer I logs an unusual number of sick days and is often late for work. 9 4 Officer I turns in reports which are neat, accurate, and well written. 9 5 Officer I participates in exercises that contribute to a better quality of life both on and off the job. 9 6 Officer I is sometimes patient with citizens who speak little English. 148 Appendix F (cont’d) Matrix Contents for Version 3 CURRENT DATA VERSION : 3 NO OF ALTERNATIVES : 9 NO OF DIMENSIONS : 6 NO NAME OF ALTERNATIVE OFFICER OFFICER OFFICER OFFICER OFFICER OFFICER OFFICER OFFICER OFFICER OmVD‘UI-AMM" HIGDMDUmD NAME OF ALTERNATIVE JOB KNOWLEDGE JUDGMENT DEPENDABILITY/RELIABILITY COMMUNICATION WORK ATTITUDE . DEALING WITH OTHERS Z O‘Ulhwl’flho NO OF A. NO OF D. CONTENT 1 1 Officer A understands the difference between civil and criminal matters in practical applications on the streets. 1 2 Officer A calls for assistance and clears the area of bystanders when confronting a barricaded, heavily-armed suspect. 1 3 Officer A responds to calls, but often takes the long way to get there in the hopes that other officers will arrive first. 1 4 Officer A uses unsatisfactory grammar and thus is unable to communicate well with the public. 1 5 Officer A does the work assigned, yet also initiates own activities. 1 6 Officer A works well with others, but is not a real extrovert. 149 2 1 Officer B usually functions on the streets with minimal legal problems. 2 2 When out on a call, Officer 8 waits to complete a physical arrest until securing assistance. H 5? ‘ 5' Officer B panics upon receiving an emergency call. 2 4 Officer B has a highly developed vocabulary and uses it appropriately. 2 5 Officer 8 fails to recognize and correct own deficiencies without prompting by others. 2 6 Officer B is usually patient with citizens who speak very little English. 3 1 Officer C is consistently unaware of the general orders and departmental policy. 3 2 Officer C exhibits good judgment except under stressful situations. 3 3 Officer C uses only a minimum of sick days per year. 3 4 Officer C sometimes has to be asked to repeat a message sent over the radio. 3 5 Officer C is willing to work on investigations other than the current assignment. 3 6 Officer C usually gets along well with others, but has a tendency to 'snap'answers during conversations. 4 1 Officer D searches citizens' vehicles with probable cause which occasionally results in discovering smuggled narcotics. 4 2 Officer D calls for a backup in ALL situations. 4 3 Officer D sometimes reports for duty with a cold if trouble is expected in the city. 4 4 Officer D submits reports that are neatly done, but not always grammatically correct. 4 5 Officer D attends special training schools during off-duty hours to improve efficiency and effectiveness as an officer. 4 6 Officer D often calls fellow workers 'misguided social workers' for showing interest in other people‘s problems. 150 Appendix F (cont'd) 5 1 Officer E is often unaware of the important facts in a case which results in losing a good case. 5 2 Officer E radios in position and discontinues a high speed chase when entering areas of high vehicle and pedestrian traffic. 5 3 Officer E answers calls from other precincts when necessary. 5 4 Officer E tries hard, but often stutters when excited and talks too much over the radio. 5 5 Officer E violates departmental rules and regulations when it is in the best interest of the public. 5 6 Officer E is a little overbearing once in a while-probably trying to be 'just like everyone else.’ 6 1 Officer F profits from past mistakes in court by appropriately correcting arrest and search procedures. 6 2 Officer F enters buildings with broken doors and windows instead of guarding the exits and calling for a backup. 6 3 Officer F has average attendance, handles stress situations as well as others and is sometimes excitable. 6 4 At times, Officer F does not probe far enough and seems embarassed asking questions. 6 5 Officer F uses racially-toned language in front of minority groups. 6 6 Officer F maintains friendly relations with the civilians in the patrol area. 7 1 Officer 6 keeps a well organized file containing extensive information concerning the various duties of a police officer. 7 2 _ Officer 6 considers all the facts before making a decision and then sticks to it. 7 3 Officer 6 reports minor damages to the patrol car even though the officer will be held responsible. 7 4 Officer 6 includes trivial, irrelevant information in written reports and radio communication. 151 Appendix F (cont’d) 7 5 Officer 6 will go ’out of the way’ to defy departmental regulations. 7 6 Officer 6 takes pleasure in embarrassing a rape victim by concentrating unnecessarily on the particularly lewd details of the statement. 8 1 Officer H arrests suspects for misdemeanors not committed while the officer was present. 8 2 Officer H issues warnings instead of tickets to traffic violators in particularly confusing intersections. 8 3 Officer H logs an unusual number of sick days and is often late for work. 8 4 Officer H turns in reports which are neat, accurate, and well written. 8 5 Officer H participates in exercises that contribute to a better quality of life both on and off the job. 8 6 Officer H is sometimes patient with citizens who speak little English. 9 1 Officer I usually abides by and implements the latest court rulings after being informed of them. 9 2 Upon arrival at the scene of a huge fire, Officer I sees the need for additional help and calls for it. 9 3 Depending on the work, Officer I sometimes shows signs of boredom. 9 4 Officer I speaks slowly and clearly when testifying in court. 9 5 Officer I perceives self as an expert and usually refuses training. 9 6 Officer I usually puts fellow officers at risk by refusing to share relevant information with them. 152 Appendix F (cont’d) Matrix Contents for Version 4 CURRENT DATA VERSION : 4 NO OF ALTERNATIVES : 9 NO OF DIMENSIONS : 6 NO NAME OF ALTERNATIVE l OFFICER A 2 OFFICER B 3 OFFICER C 4 OFFICER D 5 OFFICER E 6 OFFICER F 7 OFFICER G 8 OFFICER H 9 OFFICER I NO NAME OF ALTERNATIVE 1 JOB KNOWLEDGE 2 JUDGMENT 3 DEPENDABILITY/RELIABILITY 4 COMMUNICATION 5 WORK ATTITUDE 6 DEALING WITH OTHERS NO OF A. NO OF D. CONTENT 1 1 Officer A usually functions on the streets with minimal legal problems. 1 2 When out on a call, Officer A waits to complete a physical arrest until securing assistance. 1 3 Officer A panics upon receiving an emergency call. 1 4 Officer A has a highly developed vocabulary and uses it appropriately. 1 5 Officer A fails to recognize and correct own deficiencies without prompting by others. 1 6 Officer A is usually patient with citizens who speak very little English. 2 1 Officer 8 is consistently unaware of the general orders and departmental policy. 153 Appendix F (cont'd) 2 2 Officer B exhibits good judgment except under stressful situations. 2 3 Officer B uses only a minimum of sick days per year. 2 4 Officer B sometimes has to be asked to repeat a message sent over the radio. 2 5 Officer B is willing to work on investigations other than the current assignment. 2 6 Officer B usually gets along well with others, but has a tendency to 'snap'answers during conversations. 3 1 Officer C searches citizens' vehicles with probable cause which occasionally results in discovering smuggled narcotics. 3 2 Officer C calls for a backup in ALL situations. 3 3 Officer C sometimes reports for duty with a cold if trouble is expected in the city. 3 4 Officer C submits reports that are neatly done, but not always grammatically correct. 3 5 Officer C attends special training schools during off-duty hours to improve efficiency and effectiveness as an officer. 3 6 Officer C often calls fellow workers 'misguided social workers' for showing interest in other people’s problems. 4 1 Officer D is often unaware of the important facts in a case which results in losing a good case. 4 2 Officer D radios in position and discontinues a high speed chase when entering areas of high vehicle and pedestrian traffic. 4 3 Officer D answers calls from other precincts when necessary. 4 4 Officer D tries hard, but often stutters when excited and talks too much over the radio. - 4 5 Officer D violates departmental rules and regulations when it is in the best interest of the public. 154 Appendix F (cont‘d) 4 6 Officer D is a little overbearing once in a while--probably trying to be ’just like everyone else.’ 5 1 Officer E profits from past mistakes in court by appropriately correcting arrest and search procedures. 5 2 Officer E enters buildings with broken doors and windows instead of guarding the exits and calling for a backup. 5 3 Officer E has average attendance, handles stress situations as well as others and is sometimes excitable. 5 4 At times, Officer E does not probe far enough and seems embarassed asking questions. 5 5 Officer E uses racially-toned language in front of minority groups. 5 6 Officer E maintains friendly relations with the civilians in the patrol area. 6 1 Officer F keeps a well organized file containing extensive information concerning the various duties of a police officer. 6 2 Officer F considers all the facts before making a decision and then sticks to it. 6 3 Officer F reports minor damages to the patrol car even though the officer will be held responsible. 6 4 Officer F includes trivial, irrelevant information in written reports and radio communication. 6 5 Officer F will go ’out of the way’ to defy departmental regulations. 6 6 Officer F takes pleasure in embarrassing a rape victim by concentrating unnecessarily on the particularly lewd details of the statement. 7 1 Officer 6 arrests suspects for misdemeanors not committed while the officer was present. 7 2 Officer 6 issues warnings instead of tickets to traffic violators in particularly confusing intersections. 155 Appendix F (cont'd) 7 3 Officer 6 logs an unusual number of sick days and is often late for work. 7 4 Officer 6 turns in reports which are neat, accurate, and well written. 7 5 Officer 6 participates in exercises that contribute to a better quality of life both on and off the Job. 7 6 Officer 6 is sometimes patient with citizens who speak little English. 8 1 Officer H usually abides by and implements the latest court rulings after being informed of them. 8 2 Upon arrival at the scene of a huge fire, Officer H sees the need for additional help and calls for it. 8 3 Depending on the work, Officer H sometimes shows signs of boredom. B 4 Officer H speaks slowly and clearly when testifying in court. 8 5 Officer H perceives self as an expert and usually refuses training. 8 6 Officer H usually puts fellow officers at risk by refusing to share relevant information with them. 9 1 Officer I understands the difference between civil and criminal matters in practical applications on the streets. 9 2 Officer I calls for assistance and clears the area of bystanders when confronting a barricaded, heavily-armed suspect. 9 3 Officer I responds to calls, but often takes the long way to get there in the hopes that other officers will arrive first. 9 4 Officer I uses unsatisfactory grammar and thus is unable to communicate well with the public. 9 5 Officer I does the work assigned, yet also initiates own activities. 9 6 Officer I works well with others, but is not a real extrovert. 156 Appendix F (cont’d) Matrix Contents for Version 5 CURRENT DATA VERSION . 5 NO OF ALTERNATIVES : 9 NO OF DIMENSIONS : 6 NO NAME OF ALTERNATIVE OFFICER OFFICER OFFICER OFFICER OFFICER OFFICER OFFICER OFFICER OFFICER y.‘ QEDVOUIAHN HIGJTINUUIDD Z ombMNF‘D NAME OF ALTERNATIVE JOB KNOWLEDGE JUDGMENT DEPENDABILITY/RELIABILITY COMMUNICATION WORK ATTITUDE DEALING WITH OTHERS NO OF A. NO OF D. CONTENT 1 1 Officer A is consistently unaware of the general orders and departmental policy. 1 2 Officer A exhibits good judgment except under stressful situations. 1 3 Officer A uses only a minimum of sick days per year. 1 4 Officer A sometimes has to be asked to repeat a message sent over the radio. 1 5 Officer A is willing to work on investigations other than the current assignment. 1 6 Officer A usually gets along well with others, but has a tendency to ’snap’answers during conversations. 157 Appendix F (cont’d) 2 1 Officer B searches citizens’ vehicles with probable cause which occasionally results in discovering smuggled narcotics. 2 2 Officer B calls for a backup in ALL situations. 2 3 Officer B sometimes reports for duty with a cold if trouble is expected in the city. 2 4 Officer B submits reports that are neatly done, but not always grammatically correct. 2 5 Officer B attends special training schools during off-duty hours to improve efficiency and effectiveness as an officer. 2 6 Officer B often calls fellow workers ’misguided social workers’ for showing interest in other people’s problems. 3 1 Officer C is often unaware of the important facts in a case which results in losing a good case. 3 2 Officer C radios in position and discontinues a high speed chase when entering areas of high vehicle and pedestrian traffic. 3 3 Officer C answers calls from other precincts when necessary. 3 4 Officer C tries hard, but often stutters when excited and talks too much over the radio. 3 5 Officer C violates departmental rules and regulations when it is in the best interest of the public. 3 6 Officer C is a little overbearing once in a while—-probably trying to be ’just like everyone else.’ 4 1 Officer D profits from past mistakes in court by appropriately correcting arrest and search procedures. 4 2 Officer D enters buildings with broken doors and windows instead of guarding the exits and calling for a backup. 4 3 Officer D has average attendance, handles stress situations as well as others and is sometimes excitable. 4 4 At times, Officer D does not probe far enough and seems embarassed asking questions. 158 Appendix F (cont’d) 4 5 Officer D uses racially-toned language in front of minority groups. 4 6 Officer D maintains friendly relations with the civilians in the patrol area. 5 1 Officer E keeps a well organized file containing extensive information concerning the various duties of a police officer. 5 2 Officer E considers all the facts before making a decision and then sticks to it. 5 3 Officer E reports minor damages to the patrol car even though the officer will be held responsible. 5 4 Officer E includes trivial, irrelevant information in written reports and radio communication. 5 5 Officer E will go ’out of the way’ to defy departmental regulations. 5 6 Officer E takes pleasure in embarrassing a rape victim by concentrating unnecessarily on the particularly lewd details of the statement. 6 1 Officer F arrests suspects for misdemeanors not committed while the officer was present. 6 2 Officer F issues warnings instead of tickets to traffic violators in particularly confusing intersections. 6 3 Officer F logs an unusual number of sick days and is often late for work. 6 4 Officer F turns in reports which are neat, accurate, and well written. 6 5 Officer F participates in exercises that contribute to a better quality of life both on and off the job. 6 6 Officer F is sometimes patient with citizens who speak little English. 7 1 Officer 6 usually abides by and implements the latest court rulings after being informed of them. 159 Appendix F (cont’d) 7 2 Upon arrival at the scene of a huge fire, Officer 6 sees the need for additional help and calls for it. 7 3 Depending on the work, Officer 6 sometimes shows signs of boredom. 7 4 Officer 6 speaks slowly and clearly when testifying in court. 7 5 Officer 6 perceives self as an expert and usually refuses training. 7 6 Officer 6 usually puts fellow officers at risk by refusing to share relevant information with them. 8 1 Officer H understands the difference between civil and criminal matters in practical applications on the streets. 8 2 Officer H calls for assistance and clears the area of bystanders when confronting a barricaded, heavily-armed suspect. B 3 Officer H responds to calls, but often takes the long way to get there in the hopes that other officers will arrive first. 8 4 Officer H uses unsatisfactory grammar and thus is unable to communicate well with the public. 8 5 Officer H does the work assigned, yet also initiates own activities. 8 6 Officer H works well with others, but is not a real extrovert. 9 1 Officer I usually functions on the streets with minimal legal problems. 9 2 When out on a call, Officer I waits to complete a physical arrest until securing assistance. 9 3 Officer I panics upon receiving an emergency call. 9 4 Officer I has a highly developed vocabulary and uses it appropriately. 160 Appendix F (cont’d) 9 5 Officer I fails to recognize and without prompting by others. 9 6 Officer I is usually patient with citizens who speak very little English. correct own deficiencies 161 Appendix F (cont’d) Matrix Contents for Version 6 CURRENT DATA VERSION : 6 NO OF ALTERNATIVES : 9 NO OF DIMENSIONS : 6 NO NAME OF ALTERNATIVE 1 OFFICER A ' 2 OFFICER B 3 OFFICER C 4 OFFICER D S OFFICER E 6 OFFICER F 7 OFFICER G 8 OFFICER H 9 OFFICER I NO NAME OF ALTERNATIVE 1 JOB KNOWLEDGE 2 JUDGMENT 3 DEPENDABILITY/RELIABILITY 4 COMMUNICATION 5 WORK ATTITUDE 6 DEALING WITH OTHERS NO OF A. NO OF D. CONTENT 1 1 Officer A searches citizens’ vehicles with probable cause which occasionally results in discovering smuggled narcotics. 1 2 Officer A calls for a backup in ALL situations. 1 3 Officer A sometimes reports for duty with a cold if trouble is expected in the city. 1 4 Officer A submits reports that are neatly done, but not always grammatically correct. 1 5 Officer A attends special training schools during off-duty hours to improve efficiency and effectiveness as an officer. 1 6 ‘ Officer A often calls fellow workers 'misguided social workers’ for showing interest in other people’s problems. 162 Appendix F (cont’d) 2 1 Officer B is often unaware of the important facts in a case which results in losing a good case. 2 2 Officer B radios in position and discontinues a high speed chase when entering areas of high vehicle and pedestrian traffic. 2 3 Officer B answers calls from other precincts when necessary. 2 4 Officer B tries hard, but often stutters when excited and talks too much over the radio. 2 5 Officer B violates departmental rules and regulations when it is in the best interest of the public. 2 6 Officer B is a little overbearing once in a while-—probably trying to be ’just like everyone else.’ 3 1 Officer C profits from past mistakes in court by appropriately correcting arrest and search procedures. 3 2 Officer C enters buildings with broken doors and windows instead of guarding the exits and calling for a backup. 3 3 Officer C has average attendance, handles stress situations as well as others and is sometimes excitable. 3 4 ' At times, Officer C does not probe far enough and seems embarassed asking questions. 3 5 Officer C uses racially-toned language in front of minority groups. 3 6 Officer C maintains friendly relations with the civilians in the patrol area. 4 1 Officer D keeps a well organized file containing extensive information concerning the various duties of a police officer. 4 2 Officer D considers all the facts before making a decision and then sticks to it. 4 3 Officer D reports minor damages to the patrol car even though the officer will be held responsible. 4 4 Officer D includes trivial, irrelevant information in written reports and radio communication. 163 Appendix F (cont’d) 4 5 Officer D will go ’out of the way’ to defy departmental regulations. 4 6 Officer D takes pleasure in embarrassing a rape victim by concentrating unnecessarily on the particularly lewd details of the statement. 5 1 Officer E arrests suspects for misdemeanors not committed while the officer was present. 5 2 Officer E issues warnings instead of tickets to traffic violators in particularly confusing intersections. 5 3 Officer E logs an unusual number of sick days and is often late for work. 5 4 Officer E turns in reports which are neat, accurate, and well written. 5 5 Officer E participates in exercises that contribute to a better quality of life both on and off the job. 5 6 Officer E is sometimes patient with citizens who speak little English. 6 1 Officer F usually abides by and implements the latest court rulings after being informed of them. 6 2 Upon arrival at the scene of a huge fire, Officer F sees the need for additional help and calls for it. 6 3 Depending on the work, Officer F sometimes shows signs of boredom. 6 4 Officer F speaks slowly and clearly when testifying in court. 6 5 Officer F perceives self as an expert and usually refuses training. 6 6 Officer F usually puts fellow officers at risk by refusing to share relevant information with them. 7 1 Officer G understands the difference between civil and criminal matters in practical applications on the streets. 164 Appendix F (cont’d) 7 2 Officer G calls for assistance and clears the area of bystanders when confronting a barricaded, heavily-armed suspect. 7 3 Officer G responds to calls, but often takes the long way to get there in the hopes that other officers will arrive first. 7 4 Officer G uses unsatisfactory grammar and thus is unable to communicate well with the public. 7 5 Officer G does the work assigned, yet also initiates own activities. 7 6 Officer G works well with others, but is not a real extrovert. 8 1 Officer H usually functions on the streets with minimal legal problems. 8 2 When out on a call, Officer H waits to complete a physical arrest until securing assistance. 8 3 Officer H panics upon receiving an emergency call. 8 4 Officer H has a highly developed vocabulary and uses it appropriately. B 5 Officer H fails to recognize and correct own deficiencies without prompting by others. 8 6 Officer H is usually patient with citizens who speak very little English. 9 1 Officer I is consistently unaware of the general orders and departmental policy. 9 2 _ Officer I exhibits good judgment except under stressful situations. 9 3 Officer I uses only a minimum of sick days per year. 9 4 Officer I sometimes has to be asked to repeat a message sent over the radio. 165 Appendix F (cont’d) 9 5 Officer I is willing to work the current assignment. 9 6 Officer I usually gets along well with others, but has a tendency to 'snap’answers during conversations. on investigations other than 166 Appendix F (cont'd) Matrix Contents for Version 7 CURRENT DATA VERSION : 7 NO OF ALTERNATIVES : 9 NO OF DIMENSIONS : 6 NO NAME OF ALTERNATIVE l OFFICER A 2 OFFICER B 3 OFFICER C 4 OFFICER D S OFFICER E 6 OFFICER F 7 OFFICER G 8 OFFICER H 9 OFFICER I NO NAME OF ALTERNATIVE 1 JOB KNOWLEDGE 2 JUDGMENT 3 DEPENDABILITY/RELIABILITY 4 COMMUNICATION 5 WORK ATTITUDE 6 DEALING WITH OTHERS NO OF A. NO OF D. CONTENT 1 1 Officer A is often unaware of the important facts in a case which results in losing a good case. 1 2 Officer A radios in position and discontinues a high speed chase when entering areas of high vehicle and pedestrian traffic. 1 3 Officer A answers calls from other precincts when necessary. 1 4 Officer A tries hard, but often stutters when excited and talks too much over the radio. 1 5 Officer A violates departmental rules and regulations when it is in the best interest of the public. 1 6 Officer A is a little overbearing once in a while--probably trying to be ’just like everyone else.’ 167 Appendix F (cont’d) 2 1 Officer B profits from past mistakes in court by appropriately correcting arrest and search procedures. 2 2 Officer B enters buildings with broken doors and windows instead of guarding the exits and calling for a backup. 2 3 Officer B has average attendance, handles stress situations as well as others and is sometimes excitable. 2 4 At times, Officer B does not probe far enough and seems embarassed asking questions. 2 5 Officer B uses racially-toned language in front of minority groups. 2 6 Officer B maintains friendly relations with the civilians in the patrol area. 3 1 Officer C keeps a well organized file containing extensive information concerning the various duties of a police officer. 3 2 Officer C considers all the facts before making a decision and then sticks to it. 3 3 Officer C reports minor damages to the patrol car even though the officer will be held responsible. 3 4 Officer C includes trivial, irrelevant information in written reports and radio communication. 3 5 Officer C will go ’out of the way’ to defy departmental regulations. 3 6 Officer C takes pleasure in embarrassing a rape victim by concentrating unnecessarily on the particularly lewd details of the statement. 4 1 Officer D arrests suspects for misdemeanors not committed while the officer was present. 4 2 Officer D issues warnings instead of tickets to traffic violators in particularly confusing intersections. 4 3 Officer D logs an unusual number of sick days and is often late for work. 168 Appendix F (cont’d) 4 4 Officer D turns in reports which are neat, accurate, and well written. 4 5 Officer D participates in exercises that contribute to a better quality of life both on and off the job. 4 6 Officer D is sometimes patient with citizens who speak little English. 5 1 Officer E usually abides by and implements the latest court rulings after being informed of them. 5 2 Upon arrival at the scene of a huge fire, Officer E sees the need for additional help and calls for it. 5 3 Depending on the work, Officer E sometimes shows signs of boredom. 5 4 Officer E speaks slowly and clearly when testifying in court. 5 5 Officer E perceives self as an expert and usually refuses training. 5 6 Officer E usually puts fellow officers at risk by refusing to share relevant information with them. 6 1 Officer F understands the difference between civil and criminal matters in practical applications on the streets. 6 2 Officer F calls for assistance and clears the area of bystanders when confronting a barricaded, heavily-armed suspect. 6 3 Officer F responds to calls, but often takes the long way to get there in the hopes that other officers will arrive first. 6 4 Officer F uses unsatisfactory grammar and thus is unable to communicate well with the public. 6 5 Officer F does the work assigned, yet also initiates own activities. 6 6 Officer F works well with others, but is not a real extrovert. 169 Appendix F (cont’d) 7 1 Officer G usually functions on the streets with minimal legal problems. 7 2 When out on a call, Officer G waits to complete a physical arrest until securing assistance. 7 3 Officer G panics upon receiving an emergency call. 7 4 Officer G has a highly developed vocabulary and uses it appropriately. 7 5 Officer G fails to recognize and correct own deficiencies without prompting by others. 7 6 Officer G is usually patient with citizens who speak very little English. 8 1 Officer H is consistently unaware of the general orders and departmental policy. 8 2 Officer H exhibits good judgment except under stressful situations. 8 3 Officer H uses only a minimum of sick days per year. 8 4 Officer H sometimes has to be asked to repeat a message sent over the radio. 8 5 Officer H is willing to work on investigations other than the current assignment. 8 6 Officer H usually gets along well with others, but has a tendency to ’snap’answers during conversations. 9 1 Officer I searches citizens’ vehicles with probable cause which occasionally results in discovering smuggled narcotics. 9 2 Officer I calls for a backup in ALL situations. 9 3 Officer I sometimes reports for duty with a cold if trouble is expected in the city. 9 4 Officer I submits reports that are neatly done, but not always grammatically correct. 170 Appendix F (cont’d) 9 5 Officer I attends special training schools during off-duty hours to improve efficiency and effectiveness as an officer. 9 6 - Officer I often calls fellow workers ’misguided social workers’ for showing interest in other people’s problems. 171 Appendix F (cont’d) Matrix Contents for Version 8 CURRENT DATA VERSION . 8 NO OF ALTERNATIVES : 9 NO OF DIMENSIONS : 6 NO NAME OF ALTERNATIVE 1 OFFICER A 2 OFFICER B 3 OFFICER C 4 OFFICER D 5 OFFICER E 6 OFFICER F 7 OFFICER G 8 OFFICER H 9 OFFICER I NO NAME OF ALTERNATIVE 1 JOB KNOWLEDGE 2 JUDGMENT 3 DEPENDABILITY/RELIABILITY 4 COMMUNICATION 5 WORK ATTITUDE 6 DEALING WITH OTHERS NO OF A. NO OF D. CONTENT 1 1 Officer A profits from past mistakes in court by appropriately correcting arrest and search procedures. 1 2 Officer A enters buildings with broken doors and windows instead of guarding the exits and calling for a backup. 1 3 Officer A has average attendance, handles stress situations as well as others and is sometimes excitable. 1 4 At times, Officer A does not probe far enough and seems embarassed asking questions. 1 5 Officer A uses racially-toned language in front of minority groups. 1 6 Officer A maintains friendly relations with the civilians in the patrol area. 172 Appendix F (cont’d) 2 1 Officer B keeps a well organized file containing extensive information concerning the various duties of a police officer. 2 2 Officer 8 considers all the facts before making a decision and then sticks to it. 2 3 Officer B reports minor damages to the patrol car even though the officer will be held responsible. 2 4 Officer B includes trivial, irrelevant information in written reports and radio communication. 2 5 Officer B will go ’out of the way’ to defy departmental regulations. 2 6 Officer B takes pleasure in embarrassing a rape victim by concentrating unnecessarily on the particularly lewd details of the statement. 3 1 Officer C arrests suspects for misdemeanors not committed while the officer was present. 3 2 Officer C issues warnings instead of tickets to traffic violators in particularly confusing intersections. 3 3 Officer C logs an unusual number of sick days and is often late for work. 3 4 Officer C turns in reports which are neat, accurate, and well written. 3 5 Officer C participates in exercises that contribute to a better quality of life both on and off the job. 3 6 Officer C is sometimes patient with citizens who speak little English. 4 1 Officer D usually abides by and implements the latest court rulings after being informed of them. 4 2 Upon arrival at the scene of a huge fire, Officer D sees the need for additional help and calls for it. 4 3 Depending on the work, Officer D sometimes shows signs of boredom. 173 Appendix F (cont’d) 4 4 Officer D speaks slowly and clearly when testifying in court. 4 5 Officer D perceives self as an expert and usually refuses training. 4 6 Officer D usually puts fellow officers at risk by refusing to share relevant information with them. 5 1 Officer E understands the difference between civil and criminal matters in practical applications on the streets. 5 2 Officer E calls for assistance and clears the area of bystanders when confronting a barricaded, heavily-armed suspect.~ S 3 Officer E responds to calls, but often takes the long way to get there in the hopes that other officers will arrive first. 5 4 Officer E uses unsatisfactory grammar and thus is unable to communicate well with the public. 5 5 Officer E does the work assigned, yet also initiates own activities. 5 6 Officer E works well with others, but is not a real extrovert. 6 1 Officer F usually functions on the streets with minimal legal problems. 6 2 When out on a call, Officer F waits to complete a physical arrest until securing assistance. 6 3 Officer F panics upon receiving an emergency call. 6 4 Officer F has a highly developed vocabulary and uses it appropriately. 6 5 Officer F fails to recognize and correct own deficiencies without prompting by others. 6 6 Officer F is usually patient with citizens who speak very little English. 174 Appendix F (cont'd) 7 1 Officer G is consistently unaware of the general orders and departmental policy. 7 2 Officer G exhibits good judgment except under stressful situations. 7 3 Officer G uses only a minimum of sick days per year. 7 4 Officer G sometimes has to be asked to repeat a message sent over the radio. 7 5 Officer G is willing to work on investigations other than the current assignment. 7 6 Officer G usually gets along well with others, but has a tendency to ’snap’answers during conversations. B 1 Officer H searches citizens’ vehicles with probable cause which occasionally results in discovering smuggled narcotics. 8 2 Officer H calls for a backup in ALL situations. 8 3 Officer H sometimes reports for duty with a cold if trouble is expected in the city. 8 4 Officer H submits reports that are neatly done, but not always grammatically correct. 8 5 Officer H attends special training schools during off-duty hours to improve efficiency and effectiveness as an officer. 8 6 Officer H often calls fellow workers 'misguided social workers’ for showing interest in other people's problems. 9 1 Officer I is often unaware of the important facts in a case which results in losing a good case. 9 2 Officer I radios in position and discontinues a high speed chase when entering areas of high vehicle and pedestrian traffic. 9 3 Officer I answers calls from other precincts when necessary. 9 4 Officer I tries hard, but often stutters when excited and talks too much over the radio. 175 Appendix F (cont’d) 9 5 Officer I violates departmental rules and regulations when it is in the best interest of the public. 9 6 Officer I is a little overbearing once in a while--probably trying to be ’just like everyone else.’ 176 Appendix F (cont’d) Matrix Contents for Version 9 CURRENT DATA VERSION : 9 NO OF ALTERNATIVES : 9 NO OF DIMENSIONS : 6 NO NAME OF ALTERNATIVE 1 OFFICER A 2 OFFICER B 3 OFFICER C 4 OFFICER D 5 OFFICER E 6 OFFICER F 7 OFFICER G a ‘ OFFICER H 9 OFFICER I NO NAME OF ALTERNATIVE 1 JOB KNOWLEDGE 2 JUDGMENT 3 DEPENDABILITY/RELIABILITY 4 COMMUNICATION 5 WORK ATTITUDE 6 DEALING WITH OTHERS NO OF A. NO OF D. CONTENT 1 1 Officer A keeps a well organized file containing extensive information concerning the various duties of a police officer. 1 2 Officer A considers all the facts before making a decision and then sticks to it. 1 3 Officer A reports minor damages to the patrol car even though the officer will be held responsible. 1 4 Officer A includes trivial, irrelevant information in written reports and radio communication. 1 S ‘ Officer A will go ’out of the way' to defy departmental regulations. 1 6 Officer A takes pleasure in embarrassing a rape victim by concentrating unnecessarily on the particularly lewd details of the statement. 177 2 1 Officer B arrests suspects for misdemeanors not committed while the officer was present. 2 2 Officer B issues warnings instead of tickets to traffic violators in particularly confusing intersections. 2 3 Officer B logs an unusual number of sick days and is often late for work. 2 4 Officer B turns in reports which are neat, accurate, and well written. 2 5 Officer B participates in exercises that contribute to a better quality of life both on and off the job. 2 6 Officer B is sometimes patient with citizens who speak little English. 3 1 Officer C usually abides by and implements the latest court rulings after being informed of them. 3 2 Upon arrival at the scene of a huge fire, Officer C sees the need for additional help and calls for it. 3 3 Depending on the work, Officer C sometimes shows signs of boredom. 3 4 Officer C speaks slowly and clearly when testifying in court. 3 5 Officer C perceives self as an expert and usually refuses training. 3 6 Officer C usually puts fellow officers at risk by refusing to share relevant information with them. 4 1 Officer D understands the difference between ciVIl and criminal matters in practical applications on the streets. 4 2 Officer D calls for assistance and clears the area of bystanders when confronting a barricaded, heavily-armed suspect. 4 3 Officer D responds to'calls, but often takes the long way to get there in the hopes that other officers will arrive first. 4 4 Officer D uses unsatisfactory grammar and thus is unable to communicate well with the public. 178 Appendix F (cont d) 4 5 Officer D does the work assigned, yet also initiates own activities. 4 6 Officer D works well with others, but is not a real extrovert. 5 1 Officer E usually functions on the streets with minimal legal problems. 5 2 When out on a call, Officer E waits to complete a physical arrest until securing assistance. 5 3 Officer E panics upon receiving an emergency call. 5 4 Officer E has a highly developed vocabulary and uses it appropriately. 5 5 Officer E fails to recognize and correct own deficiencies without prompting by others. 5 6 Officer E is usually patient with citizens who speak very little English. 6 1 Officer F is consistently unaware of the general orders and departmental policy. 6 2 Officer F exhibits good judgment except under stressful situations. 6 3 Officer F uses only a minimum of sick days per year. 6 4 Officer F sometimes has to be asked to repeat a message sent over the radio. 6 5 . Officer F is willing to work on investigations other than the current assignment. 6 6 Officer F usually gets along well with others, but has a tendency to 'snap’answers during conversations. 7 1 Officer G searches citizens’ vehicles with probable cause which occasionally results in discovering smuggled narcotics. 7 2 Officer G calls for a backup in ALL situations. 179 Appendix F (cont’d) 7 3 Officer G sometimes reports for duty with a cold if trouble is expected in the city. 7 4 Officer G submits reports that are neatly done, but not always grammatically correct. 7 5 Officer G attends special training schools during off—duty hours to improve efficiency and effectiveness as an officer. 7 6 Officer G often calls fellow workers ’misguided social workers’ for showing interest in other people’s problems. 8 1 Officer H is often unaware of the important facts in a case which results in losing a good case. 8 2 Officer H radios in position and discontinues a high speed chase when entering areas of high vehicle and pedestrian traffic. 8 3 Officer H answers calls from other precincts when necessary. 8 4 Officer H tries hard, but often stutters when excited and talks too much over the radio. 8 5 Officer H violates departmental rules and regulations when it is in the best interest of the public. 8 6 Officer H is a little overbearing once in a while—-probably trying to be ’just like everyone else.’ 9 1 Officer I profits from past mistakes in court by appropriately correcting arrest and search procedures. 9 2 Officer I enters buildings with broken doors and windows instead of guarding the exits and calling for a backup. 9 3 Officer I has average attendance, handles stress situations as well as others and is sometimes excitable. 9 4 At times, Officer I does not probe far enough and seems embarassed asking questions. 9 5 Officer I uses racially-toned language in front of minority groups. 9 6 Officer I maintains friendly relations with the civilians in the patrol area. APPENDIX G 180 Appendix 6 Promotion: Post Search Questionnaire SSN 1. Performance appraisal information is collected for variety of reasons. How important is using appraisal information to promote an individual to a higher level? Please circle the appropriate number on the following 7-point scale. 1 2 3 4 5 6 7 of no extremely importance important Please elaborate 2. Compared to selecting an individual for a remedial training program, how important are promotion decisions? Please circle the appropriate number. 1 2 3 4 5 6 7 training decisions equally promotion are more important important decisions are more important 3. Did the promotion decision influence the amount of information for which you searched? Please circle YES or NO. If so, how? 4. What strategy did you use to gather information for making the promotion decisions? 10. 181 Appendix G (cont’d) How did you determine which items of information to request? Did you attempt to categorize the background information representing the officers' previous performance? Please circle YES or NO. If so, how did you group the information? If you did not categorize, how did you use this information? Did you search for information differently for officers about which you had information and those you did not? Please report the ratings you provided for the different officers after reading the background information about them. For those individuals who were not described, simply write NA. Officer Officer F H226) Did the impressions you formed based on the background information influence the attention you paid the different officers in order to make the promotion decisions? Why or why not? How did you determine that you had searched enough information and were ready to select the candidate to be promoted? 182 Appendix G (cont'd) Training: Post Search Questionnaire SSN 1. Performance appraisal information is collected for a variety of reasons. How important is using appraisal information to send a deficient individual to a remedial training program? Please circle the appropriate number on the following 7-point scale. I I I --------- I --------- I --------- I --------- I 1 2 3 4 S 6 7 of no extremely importance important Please elaborate 2. Compared to selecting one individual to be promoted to a higher level, how important are training recommendations? Please circle the appropriate number. l 2 3 4 5 6 7 training decisions equally promotion are more important important decisions are more important 3. Did the training decision influence the amount of information for which you searched? Please circle YES or NO. If so, how? 4. What strategy did you use to gather information for making the training selection decision? 183 Appendix G (cont’d) 5. How did you determine which items of information to request? 6. Did you attempt to categorize the background information representing the officers' previous performance? Please circle YES or NO. If so, how did you group the information? If you did not categorize, how did you use this information? 7. Did you search for information differently for officers about which you had information and those you did not? 8. Please report the ratings you provided for the different officers after reading the background information about them. For those individuals who were not described, simply write NA. Officer A Officer F Officer B Officer G Officer C Officer H Officer D Officer I Officer E 9. Did the impressions you formed based on the background information influence the attention you paid the different officers in order to make the training decisions? Why or why not? 10. How did you determine that you had searched enough information and were ready to select the candidate that needed training? APPENDIX H 184 Appendix H Descriptive Summary of Sample Representation Per Version Version 3222;; Promotion Training 1 6 S 2 S 5 3 7 7 4 6 6 5 6 S 6 6 4 7 S 3 B 4 2 9 5 1 APPENDIX I 185 Appendix I Comprehensive Summary of the Depth of Search for the Total Number of Information Acquisitions Officer Officer Officer Officer Officer Officer Officer Officer Officer A B C D E F G H I PROMOTION (n=50) Ma snb 65.60 31.64 8.22 5.93 7.48 4.65 7.92 6.29 7.00 5.37 5.96 4.37 5.88 4.09 8.08 5.49 7.72 4.40 7.34 4.85 Officers by Appraisal Purpose TRAINING (n=50) M SD 66.58 35.37 5.94 4.78 5.88 4.60 6.50 4.88 8.24 6.17 8.72 6.21 7.26 6.71 7.68 5.43 7.60 5.26 8.76 6.62 aM=average number of information acquisitions for each officer subgroup and total bSD=standard deviation of information acquisitions for each officer subgroup and total TOTAL (n=100) M SD 66.09 33.39 7.08 5.48 6.68 4.67 7.21 5.65 7.62 5.79 7.34 5.51 6.57 5.58 7.88 5.43 7.66 4.83 8.05 5.82 APPENDIX J QUESTION 3 I. II. 186 Appendix J Verbal Rationale for the Influence of Appraisal Purpose on the Amount of Information for which Subjects Searched Did the PROMOTION decision influence the amount of information for which you searched? A. YES 1. This is a very important job and I wanted to make a well-informed decision. 2. Promotions are for the high achievers in all areas and I needed to know everything in order to make a decision NO OTHER 1. I looked for areas that would be necessary for job performance at a higher level Did the TRAINING decision influence the amount of information for which you searched? A. YES 1. I thought training is important and I wanted to make sure the right person got it. 2. I looked for the most deficient individual. NO OTHER 1. I looked primarily at job knowledge and attitude. 26 10 21 16 APPENDIX K Question 6 I. YES A. II. III. 187 Appendix K Verbal Statements Regarding the Categorization of Background Information I split the group into those receiving positive appraisals and those receiving negative appraisals. B. Categorized in terms of particular dimensions. C. Whether previous performance violated criteria I had chosen. D. Grouped according to individual officers. E. Important or not important. NO A. Letters of evaluation are usually biased so I negated all of it. B. I used the information to see if it matched the computer information. C. Categorization is not always good--stereotypes may enter into the decision process. D. It was too much to remember, so I relied on the screen information. E. It represented one person's appraisal to be used as additional information. OTHER A. Just to give me an overall idea of past performance. 8. Things they could control versus things needing training. C. Grouped it randomly--no set schedule. 47 14 16 APPENDIX L 188 Appendix L Possible Reasons for Searching More/Less for Officers Having Background Information Question 7 II. III. YES A. I went right to the officers with whom I did not have previous knowledge and examined their information. 8. I feel I paid more attention to the six with previous information. C. I examined all dimensions for the officers I liked from the background information. D. I examined all six dimensions for the officers I didn't have any background information for. For the others, I used the computer to refresh my memory. E. The background information helped eliminate some mentally almost immediately. F. I tried to get a rating in my mind of how G, H, & I would compare with the others. NO A. The background information only aided in the final decision process. 8. I searched the same amount of information for all officers. OTHER A. I tried not to. B. At first, but then I realized the computer had either more or less to tell me. C. Not at first, but after reading all the information, I was a bit partial to those officers. 11 10 44 APPENDIX M 189 Appendix M Extent to Which the Impressions Formed from the Background Information Influenced the Attention Paid the Different Officers Question 9 I. II. Those I was initially impressed with I paid a little more attention to initially. The background information was put together by the previous seargent who had experience with them in the past. The more deserving officers received more information requests. The background information gave me a better overall view of the kind of person the officers represented. I used the first three {promotion} as a comparison tool against the rest. After a consideration of all officers, I used the background information as the deciding factor. The background information and the computer were consistent. The information seemed relevant. I felt I knew a little more concerning the ones I read about. I wanted to make my own evaluations, so I put little emphasis on the background information. I didn’t have background information for all officers. I was too involved with the computer information and forgot the paper information. [\1 18 12 III. 190 Appendix H (cont’d) D. All the officers had faults. E. I collected all the information on all the officers. F. The background information was too general. G. I realized that I was missing some important information. H. If someone had a specific problem, I was attentive to it regardless of the background information. OTHER A. The background information was not reflective of the factual details contained on the computer. B. In order to make a good, sound decision, I would have more information in front of me and more of an idea of the evaluation process. C. Yes, because the dimensions seemed impersonal. 0. Somewhat, I kept in mind what I rated them. E. These things can be deceiving. F. I wanted to see if everything matched. 10 APPENDIX N 191 Appendix N Strategy Used to Gather Information for Making the Promotion/Training Selection Decision Question 4 I. PROMOTION A. I started with the first officer and gathered information using all performance categories. 1. 2. 3. 4. I looked at all the information, narrowed it down, and rechecked to be sure. I gathered as much positive information relevant to the job. I looked at each officer on all dimensions, chose the 'best' and then compared them dimension by dimension. I evaluated each officer and then chose the best in the most important things. I rated the performance dimensions from most important to least important and evaluated from there. 1. 2. I concentrated on a few dimensions as the main focus and then used some of the other dimensions. I started off numerically, but disregarded those officers with really bad items. I evaluated the nine officers in two chunks of three dimensions each, picked out the three best, and promoted based on the most important criteria. I evaluated all officers on the most important dimensions, narrowed it down, and evaluated them on all the dimensions. I gathered information on Officers G-I and then rechecked my previous decisions on Officers A-F. 1. I eliminated D-F, ranked A-C from the background information, searched on the computer to eliminate two from A-C. I then compared the remaining one with G-I on the most important factors and selected. 15 ll II. 192 Appendix N (cont’d) Z. I first searched for A-C, then G-I, and then D-F. 3. I compared A-C and G-I using all six dimensions. 4. I eliminated D-F from the background and searched the remaining officers on Judgment. 5. I first eliminated D-F based on the background, but the discrepancy between the background and the computerled me to search all the information. 6. I tried to eliminate the worst early to have fewer to search more thoroughly. D. I picked an officer that I thought might be right and compared the other officers against him. If one was better, I replaced the old standard with him and continued. E. I compared each officer on the six dimensions. If I felt they could be trained in that area, I did not disqualify them. TRAINING A. I considered all possibilities. 1. I looked at each individual first, then compared them across to select the worst. 2. I found the best officer first and compared him to the worst. 3. I first looked over all the data on each officer and then decided that only certain dimensions are relevant to whether the officer needs training. 4. I evaluated each officer in each dimension then I picked the one who exhibited the major flaws in the crucial areas. 5. I looked at each officer, counted the negative points and made my decision. 6. I compared dimensions using some randomness. 7. I looked at how officers compared in the same categories and the number of categories one officer rated poorly in. 10 B. 193 Appendix N (cont’d) I selected the areas I thought were most important for the job and evaluated the performance in those areas. 1. I used what I believed made a good officer, compared two, and eliminated the better. 2. I looked at 2-3 dimensions for each officer--if the officer did not have favorable information in those dimensions, then I looked at all six. I weeded out the good officers and concentrated on D-I. 1. I considered the evaluations for A-F, and then searched for information about G-I and compared it to A-F. 2. I concentrated on G-I because I did not have any prior information for them. 18 APPENDIX 0 194 Appendix 0 How Items of Information Were Chosen during the Search Process Question 5 I. PROMOTION A. I concentrated on what I thought was important II. for police work. 1. I started with the most important quality and compared each officer on down the line. 2. I used random selection after I decided what I thought was important. 3. I used items that were directly involved in the job. B. I requested all the information. 1. I requested all items and then double checked the most relevant or crucial items. C. I looked at all the items for A-C and G-I. TRAINING A. I looked at the dimensions most important for police work. 1. I put myself as the officer’s partner and superior. 2. I examined whether the officers answered my criteria in what I was evaluating for. 3. I chose the dimensions most important, then chose more if these were I'bad." B. I requested it all. 1. I requested it all at first, then the ones I felt to be crucial. C. I searched for a motivated, low performing person to give training. D. It depended on the previous information given. E. I eliminated positive factors to isolate negative factors. 25 1 1 18 22 16 APPENDIX P 195 Appendix P How Subjects Determined When to Conclude Search Question 10 I. II. PROMOTION A. I had looked at all the information. 1. I felt I had a good overall view of each officer and was in a position to make a well-informed decision. 2. I kept narrowing the prospects until it came to 2 and then I reviewed both again and made a decision. 3. I went through all the officers comparing them to the officer I chose. 4. I narrowed down the selection to two, compared them to the background knowledge and chose. B. When the information was satsifactory in a number of categories for one officer, I chose him. C. I compared the best of A-F with the best of G-1 and selected. 1. By elimination of the top choices and further elimination with officers G-I. TRAINING A. After I examined the most important points and the candidates did poorly in these. B. I searched all the information before deciding and felt confident. 1. Checked all the candidates in every category and eliminated the good ones in the categories thought most important until one was left. 2. I found one officer who was poor in areas that could be improved with training. 14 12 11 196 Appendix P (cont’d) 3. I narrowed it down, looked them over again and chose. 4. I compared the background information with the computerinformation and chose. When I felt I was at the very bottom of the list. Looked at all dimensions for the last six officers and at least three dimensions for the first three officers. 19 197 FOOTNOTE 1In addition to depth and sequence of search, latency data was also collected. However, the latency measurement was discovered to be problematic and is therefore not reported. LIST OF REFERENCES 198 LIST 07 REFERENCES Allen, J. S., & Muchinsky, P. M. (1984). Assessing raters‘ policies in evaluating proposed services for transporting the physically handicapped- l_uraal_nf_hnnlied_£srsbelegr £2. 3-11- Anderson, 8. L. (1977). Differences in teachers‘ judgment policies for varying numbers of verbal and numberical cues. ' ' e vi e o , 12, 68-88. Banks, C. G. (1982). Behaviqggl indiggs gf gatggs' cognitive plgggggigg. Unpublished manuscript, The University of Texas. Bargh, J. A. (1982). Attention and automacity in the processing of self-relevant information. o na ‘ e o lit d Esrsbelosr. 13. 425-436- Bernardin, H. J. (1977). Behavioral expectation scales versus summated ratings: A fairer comparison. 1g;;ngl_g{_§ppl;gg Ellfihnlflllt 51: 422‘427- Bernardin, H. J., Alvares, K. M., A Cranny, C. J. (1976). A recomparison of behavioral expectation scales to summated scales. gougpgl of Applied Psychglogy, 6;, 564-570. Bernardin, H. J., & Pence, E. C. (1980). Effects of rater training: Creating new response sets and decreasing accuracy. Jo a f A i P ho , 65, 60-66. Bettman, J. R. & Kakkar, P. (1977). Effects of information presentation format on consumer information acquisition strategies. Journal of Consumer Research, a, 233-240. Bettman, J. R., & Park, C. W. (1980). Effects of prior knowledge and experience and phase of choice process on consumer decision processes: A protocol analysis. ggggg;;__91__ggg§gggg Releases. 1. 234-248- Billings, R. 8., 4 Marcus, S. A. (1983). Measures of compensatory and noncompensatory models of decision behavior: Process tracing versus policy capturing. Organizatigggl Bebazior..sd.flueee.£er£2rseess 31 331- 352- 199 Brady, 0., & Rappaport, L. (1973). Policy-capturing in the field: The nuclear safeguards problem. Qggggiggtigggl WW. 2. 253-266- Carroll, J. S., & Payne, J. H. (1977). Judgments about crime and the criminal: A model and method for investigating parole decisions. In B. 0. Sales (Ed.), P e iv ' pgyggglogy: Thg gyigingl jugtige system (Vol 1). New York: Plenum. Chestnut, R. W. (1977). I m tion ac ' ' n i nsura ce ic s ction: Mon'torin the im act f t be 'e af ect tow r nt and externa m mor . Unpublished doctoral dissertation, Purdue University. Cohen, C. E. (1981). Person categories and social perception: Testing some boundaries of the processing effects of prior knowledge. Jogggal gf Personality gng Socigl Psychology, 5g, 441-452. Cohen, J. (1979). Stgtistical powg; gpglysis for the bghgvioya; 521122;; (Revised Edition). New York: Academic Press. Cooper, W. H. (1981). Ubiquitous halo. Psychglggiggl 3.4113333. 9.0. 219-244- Darley, J., & Fazio, R. (1980). Expectancy confirmation processes in the social interaction sequence. Algziggg W: 14 957491 - Dawes, R. (1971). A case study of graduate admissions: Applications of 3 principles of human decision making. Aggriggn Psycholggist, 26, 180-186. DeCotiis, T. A., & Petit, A. (1978). The performance appraisal process: A model and some testable hypotheses. Acadggy g; W. A. 635-646- DeNisi, A. 8., Cafferty, T. P., & Meglino, B. M. (1984). A cognitive view of the performance appraisal process: A model and research propositions. Orggnigatignal Behavigr and Human ngformange, 31, 360-396. DeNisi, A. S., 4 Stevens, G. E. (1981). Profiles of performance, performance evaluation, and personnel decisions. Agaggmy of Magagement Journal, 24, 592-602. Feldman, J. M. (1981). Beyond attribution theory: Cognitive processes in performance appraisal. nal of A ied mm. 5.6: 127-149- Hansen, R. D. (1980). Commonsense attribution. Journa of Eggsgggligy ggd Social Psychology, 39, 996-1009. 200 Hastie, R. (1980). Memory for behavioral information that confirms or contradicts a personality impression. In R. Hastie et al. (Eds.), ngggg ggggyy: Th; gggniyivg baggy gf gogigl pgrggptiog (pp. 155-178). Hillsdale, NJ: Erlbaum. Hobson, C. J., 4 Gibson, F. W. (1983). Policy capturing as a approach to understanding and improving performance appraisal: A review of the literature. Acadggy of Manggegeny Rgvigg, 8 640-649. Hobson, C. J., Mendel, R. H., 9 Gibson, F. H. (1981). Clarifying performance appraisal criteria. gygggjygyigggl §2D3Z1QI.!D§_EBEIB_ESIIQIEIBSSp 23. 154'133- Hoffman, P. J. (1960). The paramorphic representation of clinical judanent- _arshelegisal_§ulle&ia.Iii. 116-131. Hogarth, R. H. (1980). Judgment agd Choige. New York: Wiley and Sons. Ilgen, D. R., & Feldman, J. H. (1983). Performance appraisal: A process focus. In B. H. Staw and L. L. Cummings (Eds.). Rgggaggh ig nggniyatigga; fighavioy, Vol. 5. Greenwich, CT: JAI Press. Ilgen, D. R. & Favero, J. L. (1985). Limits in generalization from psychological research to performance appraisal processes. Agingly W. 19. 311-321- Jacoby, J., Chestnut, R. W., & Fisher, W. A. (1978). A behavioral process approach to information acquisition in nondnrable purchasinq- _2urael_ef_Narke&ias_Besee_sb.11. 532-543. Jacoby, J., Chestnut, R. W., Weigl, K. C., 4 Fisher, W. (1976). Pre-purchase information acquisition: Description of a process methodology, research paradigm, and pilot investigation. In B. B. Anderson (Ed.), Ag_ancgs_in_ggn§gmgr_rg§ga;g_ (Vol 3) Chicago: Association for Consumer Research, University of Illinois. Kirkpatrick, J. J., Ewen, R. 8., Barrett, R. S., & Katzell, R. A. (1968). Iggtigg 33d Fair Egplgyggg . New York: New York University Press. Kirsch, H. P. (1985). e ef o i ormation ualit a d t e f at o ' formatio int ra i str te ies. Unpublished masters thesis, Michigan State University. Landy, F. J. (1985). Psychology of Work Behavior (3rd edition). Homewood, IL: Dorsey. Landy, F. J., A Farr, J. L. (1975). Pglice Pgrformagcg Apprggi ;. University Park, PA: Pennsylvania State University. 201 Landy, F. J., 4 Farr, J. L. (1980). Performance rating. E§1§b219§l£§l_42112412. 91. 72-107- Landy, F. J., 4 Farr, J. L. (1983). The meagppgment of work pgpfpppangg: Methodg, mggggpgment and theory. New York: Academic Press. Landy, F., 4 Zedeck, S. (1983). Introduction. In F. Landy, 5. Zedeck, 4 J. Cleveland (Eds.), Pgrformangg Megggygpept gpg Thgpyy. Hilsdale, NJ: Erlbaum. Latham, G. P., Fay, C., 4 Saari, L. (1979). The development of behavioral observation scales for appraising the performance of foreman. Pprspppel Ppygpology, 3;, 299-311. Latham, G. P. 4 Wexley, K. N. (1981). Increasing ppodgppivity thpgugh pgpfogpgpcg appraisal. Reading, Mass: Addison-Wesley. Lewin, A. Y., 4 Layman, S. S. (1979). Information processing models of peer nominations. Personnel Ps cholo , 3;, 63-82. Madden, J. M. (1981). Using policy capturing to measure attitudes in organizational diagnosis. Peysonnel Egycpplogy, 31, 341-350. Matte, W. E. (1982). erim a ve ° a ' n f ipggympyipp pgpygh 1p pgrformppgg gpprpisal. Unpublished doctoral dissertation, Purdue University. Meyer, H. H., Kay, E., 4 French, J. R. (1965). Split roles in performance appraisal. Harvard Business Bgview, 43, 123-129. Montgomery, H. (1976). A study of intransitive preferences using a think aloud procedure. In H. Jungerman 4 G. de Zeeuw (Eds.), Procegdipgg of the Fifth Cpnfergnce on Subjeptive P b ' it tilit a Dec'sion Mak‘n . Murphy, K. R., Balzer, W. H., Lockhart, M. C., 4 Eisenman, E. J. (1985). Effects of previous performance on evaluations of present performance. Jppypal of Applied Psychology, 19, 72-84. Naylor, J. C., 4 Wherry, R. J., Sr. (1965). The use of simulated stimuli and the "JAN'I technique to capture and cluster the policies of raters. Edugptional and Psychological Measureaeal. Zi. 959-995- Newell, A., 4 Simon, H. A. (1972). prap problep solving. Englewood Cliffs, NJ: Prentice-Hall. Nisbett, R., 4 Ross, L. (1980). Human Inference: Strategies and spgrtcomipgp. Englewood ciffs, NJ: Prentice-Hall. 202 Oskamp, S. (1967). Clinical judgment from the MMPI: Simple or complex? Jgu gpg1 gf Q11p1gg1 ngcho1ogy,22, 411-415. Payne, J. W. (1976). Task complexity and contingent processing in decision making: An information search and protocol analysis- Draw. 15. 366-387. Payne, J. W., Braunstein, M. L., 4 Carroll, J. S. (1978). Exploring predecisional decision behavior: An alternative approach to decision research. Orgapizational Behavio; apg flueea_Eerfermaase. 22. 17-44- Fits, G. F. (1976). Decision making and cognition. In H. Jungerman 4 G. deZeeuw (Eds.). ££Q§g2912g§__2£_ing_£1£§h 'e iv P obabil't Ut'l' a ci ’ Making- Russo, J. E. (1974). More information is better: A reevaluation of Jacoby, Speller, 4 Kohn. Journal of Consumer Research, 1, 68-73. Rynes, S., 4 Lawler, J. (1983). A policy-capturing investigation of the role of expectancies in decisions to pursue job alternatives. Jgugpgl of App1ieg flsyghology, 22, 620-631. Simon, H. A. (1976). Discussion: Cognition and social behavior. In J. S. Caroll 4 J. W. Payne (Eds.), ' ’ b v' r. Hillsdale, NJ: Erlbaum. Slovic, P., 4 Lichtenstein, S. (1971). Comparison of Bayesion and regression approaches to the study of information processing theory in judgment. Organizational Behgviop app Huaan.£erferaanse. 5. 649-744- Smith, P. C., 4 Kendall, L. M. (1963). Retranslation of expectations: An approach to the construction of unamiguous anchors for rating scales. Journal of A lied P hol , 21, 149-155. Stahl, M. J., 4 Zimmerer, T. W. (1984). Modeling strategic acquisition policies: A simulation of executives’ acquisition decisions. Ac dem f Mana ement Journal, 21, 369-383. Stumpf, S. A., 4 London, M. (1981). Capturing rater policies in evaluating candidates for promotion. Academy of Management J rnal, 11, 752-766. Svenson, O. (1974). 2 pot; on think aloud protocols obtaineg during the choice 91 a home. Unpublished reports from the Psychological Laboratories 4421. Stockholm: University of Stockholm. Svenson, O. (1979). Process descriptions of decision making. nggp1pat1opp1 gehavior and Human Performance, 22, 86-112. 203 Swann, W., 4 Read, 5. (1981). Acquiring self-knowledge: The search for feedback that fits. Journal of Personality and §oc1g1 P3ygho1ogy, 21, 1119-1128. Taylor, R. L., 4 Wilsted, W. D. (1974). Capturing judgment policies: A field study of performance appraisal. Aca e of W. 1.2. 440-449- Tversky, A. (1969). Intransitivity of preferences. Egygpolog1ca1 Revigg, 12, 31-48. Tversky, A., 4 Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Sc’ence, 12;, 1124-1131. Wherry, R. J. (1952). The control of bias in ratings: A thgory of rgtingp. Columbus: The Ohio State Research Foundation. Wilkins, L. T. (1964). Social Deviance. London: Tamistock. Williams, K. J., DeNisi, A. S., Blencoe, A. G., 4 Cafferty, T. P. (1985). The role of appraisal purpose: Effects of purpose on information acquisition and utilization. Orgpnizational Behaviop gpg ngpp Egrformange,‘2§, 314-339. Zedeck, 8., 4 Cascio, W. F. (1982). Performance appraisal decisions as a function of rater training and purpose of the appraisal- W. 15.1. 752-758- Zedeck, 8., 4 Kafry, D. (1977). Capturing rater policies for processing evaluation data. Organizational Behavior and Hpman W. L. 269-294-