vwvfi '— w “‘ “MMWMW . Mm.-. ‘WM'VW p n 7"”. M.‘“g*“‘«““.“.W‘am*W Mflmmsfi . . . ~ . ‘ . . l - . . - I _ 5 ' a . . - Q . - Q . - K I O u o > - o } ‘ I ' . . - I . O . a . . . I ~ APPLICATION or THE mam _ - p or SIGNAL DETECTION 10A _ . I _ .- . - . .~ smacnou DECISION. MAKING {PROBLEM . - 2 _ - _ Q . | I .. . . . . b . . . . 0; . - . . I I . . , . ‘J. . ‘ I' - 2 . n 0' . . ~ I . . . - (I O y _ -.. ' O-- . _. . ~' 0 --. l t '3 . ‘ Thesis for the Degree (If M. A. . ‘ MICHIGAN STATE IIIIIVEIISITI _ . _ . 0 - n..- V , -‘-. . -o.". . . " .' - ' I x~.-‘ ' fl . . ~ . _ - . ‘w. . ._ _ " . . .- v "" a -- .- -Y A . ‘ ' .. V ¢. D' ‘ o - -v v? ' . . . o ' . It ' ' . a. I" - o - I ' > , ‘ .2--¢.n . . ‘ " . .. . - _- . ‘ . . H-f . ‘ ' o -- " - ' v ' o ‘ . . . . I . . . . l . - . . . ' . , ' — a ' ‘ .. ‘ i . O f . o . . , ‘_ o o . ' _' . ’ ' .. . . t ' ' . . - _ - - . o o ' ' _ . . . o . . . . . ' ' ' ' ' o o 4». ¥ - . ' - . ‘ “ _ ‘1 G' .' ' .. 'o - ’ n. . I u y - - ' - l . .‘I .- . .. . ’c. .. .‘ .- . p - . _ ' .- . o o o ’ °' - . ° . ° ' ' ' " ' . a .4! ‘Ior ‘ ' " ' ' ' ° .' .'. . ‘ ~ I o ‘ ’ ‘ _ _ . , 4". I it - v -- I.( . - --- o—‘ . o-l .. ,- . ' I - - ' - . o g. c. A .- o u 0. ‘~ I ‘ , . - o -- l ‘ ' "' . , . .0». a . 0 0'0. " ‘ ' i ' " " " "" ‘7 "' ' ’ ' . '0'- - . . . cw - ' ‘._. . ,_ . '. . ...—4 . . . . ,‘ ..... r!' -0- .ng— - a 0' .a o ’ . ' ' . . ...I r.. . . o . .- ‘_ .c: o n ’ . . v ~ . . o ' ‘ ' .. / ..- , .. 'a -- n- .994 9. ".V" -I‘ ’ - 1- , , , . u _ . I . .. .. - _ . , , . ' n . . ' ,, . ..- .a. . - _ . . "’...-l- .3. . -o . o’lo- I ' " . . -.-u - ' . I - ' _ ’ , . _- . -. ' o . ‘ . ' . . . - t- ' . . .' .. . . ‘ . .. ‘ ' I _ '. . o g r u ' ‘ .9 v , n " . "(.'J . , . . , _ . _ * I ' . . - . c . .n . ’0 - " .i. . .‘ o .. .' . . I4. 4 ‘ . . .. , i . O a . . a '5- ‘0‘ . - °v - - ." . l c . . . o . ,!0 ' ' a . -’o , _ ‘ ".--‘.". ‘l"_'.' _ - I—'.'>.rf ‘ , . ’ l - ’1 " " .. A». III! "_ C.’l._'.. ‘ ... ’...l‘q4~oc’.-- ' - . ‘ - 1'.v”¥:-o- - - - - U I ' . v - 10".4". . Ii p- .Zloal’ffl'Pf .. "I‘"./ o: I I 1.; _¢..'. I . ”l, V .f'. Vic-.- o - ' I .I'll'. ' ~ . III“;'MI:'.. ’, .- " r~.'rvo'a"' o '5 - .f'o- . .- v ° 9 v (v ' - ‘-9.! I . ' l ' r'; r o - fln' " .fl’.‘ ' ' ‘ ' ' ' ‘ ' - . .6 .. -...on.,: f l I? . - ~0' -v a In - ‘ ~-- ~ -o"‘_‘ 0.! If a“. ‘ 'l‘ ... .. ,.,..;' na-f f’...‘l‘.[~,l tr: 0 0.1-; . . . . . . .. o. ,. .. _ .. 'l I 1."".‘;”..'."’J"Q ’OIJOPIOlo-p "Y . . . 9' *":’A-.. ’m"'lf'"‘ ' 9,. o vo- O 'JO‘!’ ' . I oU'-»'o {trap/1‘77 ‘ 34". .9". ‘ ' . 0 ‘O ' .' . '1 ‘." 'r.0 1" A, 0' "' .r‘ 'r. 'o‘: ‘ I . ¢ ’1' .':;p) a) . . -.. ' O ' s o ' ‘ “ "I 0" 'Jll. ,' po‘flulr’O-‘Il’rn‘t 'fl' """ "l/IW‘.‘“ ' ‘ ’1 ‘ ' v. -,.I‘ o. ‘5’ '. "r0!’[l’l/OOHO.""""I'"' 1 rr'va”:" ' O I'r - o a, .,,v .o ;' -.p i 'r 10" ‘ l! . ' 0‘ _ IL A'. n - -n..fl”A-'Dl A; r lflflllllfllfllflljllflWNW!HHHIIHIUIHWNW 10391 5223 * ABSTRACT APPLICATION OF THE THEORY OF SIGNAL DETECTION TO A SELECTION DECISION MAKING PROBLEM BY William H. Greenwood, III An experiment was designed which allowed applica- tion of the Theory of Signal Detection (TSD) to a graduate admissions selection problem. TSD has the desirable feature of separating an individual decision maker's sensitivity (d') from his inherent response bias (B). Of particular interest to this study were the correlates of these TSD parameters. Forty-four undergraduate psychology students (18 male, 26 female) were administered the Marlowe-Crowne Social Desirability Scale (M-C Scale), the Pettigrew Category Width Scale (C-W Scale), the Employee Aptitude Survey verbal comprehension test (EAS), and were asked to report SAT scores (SAT-V, SAT-Q). At a later date, the gs were given four predictor scores (three Graduate Record Exam- ination scores and undergraduate grade point average) for 216 previous applicants to graduate school in industrial psychology, 38% of whom had been accepted by the faculty. William H. Greenwood, III From these predictors, gs were asked to guess the faculty's decision for each applicant on a six-point scale of certainty. The same predictor data were also applied to the linear discriminant function (LDF). The specific hypotheses tested were as follows: 1. The LDF will outperform all 85 in determining actual faculty judgments. The TSD parameter d' will be higher for the LDF than for any of the subjective judges. 2. B and d' will behave as theorized in perceptual experiments. A Receiver Operating Characteristic (ROC) curve of expected shape will define a sensitivity level for each g. A correlation of zero between B and d' is hypothesized. 3. (3.1) Need for approval, as measured by the M-C Scale, will correlate negatively with the TSD parameter, B. (3.2) There will be no significant correlation between M-C scores and d'. 4. (Null hypothesis) There will be no significant correlation between the C-W Scale and either of the two TSD parameters. 5. (Null hypothesis) There will be no significant correlations between the mental ability measures (EAS, SAT-V, SAT-Q) and TSD parameters. Findings showed confirmation of Hypotheses l, 2, 3.1, and 3.2. The null hypotheses were also supported for Hypotheses 4 and 5. Further results, indirectly related to the hypotheses, investigated the reliability of d' and B (moderately high) and sex differences on all variables (found only for the C-W Scale). Discussion was offered with reSpect to the new information provided by TSD in the evaluation of William H. Greenwood, III mathematical models, the adaptability of TSD to a specific selection situation, the correlates of TSD parameters, and the usefulness of TSD to psychologists. In addition, the shortcomings of the research are discussed, along with recommendations for future research. Approved by Thesis Committee: Dr. Frank L. Schmidt, Chairman ’ Dr. John H. Wakeley Dr. David Wessel APPLICATION OF THE THEORY OF SIGNAL DETECTION TO A SELECTION DECISION MAKING PROBLEM BY ( .- William H€VGLeenwood, III A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of MASTER OF ARTS Department of Psychology 1973 ‘3'“. ACKNOWLEDGMENTS A single page cannot really do justice to the debt I now find myself owing. To my thesis committee, par- ticularly to its chairman, I remain indebted for its long- lasting support and guidance. Dr. Frank L. Schmidt was my constant source of suggestions, feedback, constructive criticism, and help. Most of all, he was a patient and understanding teacher. Dr. John H. Wakeley, in the midst of an extremely busy period for him, was a willing and very able advisor. His foresight in suggesting an analysis of sex differences is especially appreciated. Dr. David Wessel provided an expertise in the area of the Theory of Signal Detection without which this author most surely would have suffered. Others, both directly and indirectly, deserve credit for their thoughtful contributions. Miss Elaine Bulgozdy offered her time, interest and hard work in the running of subjects, experimental design and data analysis. Fellow graduate students were also there when called upon to lend a hand, act as guinea pigs, or simply offer moral support. Finally, a special debt of gratitude is reserved ii for my parents and wife, Leslie, whose unselfish dedica- tion, understanding, and love will never be repaid. iii LIST OF TABLES . LIST OF FIGURES INTRODUCTION TABLE OF REVIEW OF THE LITERATURE . HYPOTHESES Hypothesis Hypothesis Hypothesis Hypothesis Hypothesis METHODS . Data Source for Subjects Procedure RESULTS . Reliability of TSD Parameters Hypothesis Hypothesis Hypotheses Additional anN L” 3: Predictors 4 and 5 . . Analyses . . iv CONTENTS Criterion Page vi vii 21 21 21 22 24 26 27 27 28 28 30 30 31 35 39 43 Page DISCUSSION 0 O O O O O O O O O O O O O 48 Confirmation of Mathematical Models . . . . . 49 Applicability of TSD to a Specific Applied Setting . . . . . . . . . . . . . 51 Relationship of Personality Traits to TSD Parameters . . . . . . . . . . . 51 Usefulness of TSD to Psychologists . . . . . 55 APPENDIX . . . . . . . . . . . . . . 59 LIST OF REFERENCES . . . . . . . . . . . 68 Table 1. LIST OF TABLES Average d' Values for 83' Dichotomized Ratings Compared witH d' for the Linear Discriminant Function (d'LDF=1.230) . . . Bs for Ss' Dichotomized Ratings Compared With Bopt (1.62) o o o o o o o o o d' as Determined by Reference to Tables Versus the Sensitivity Index, d" . . . . Intercorrelation Matrix for All Variables when the Entire Subject Sample is Included . . Intercorrelation Matrix for Variables Corrected for Attenuation for the Entire Sample (N’= 44) . . . . . . . E_Tests for Sex Differences on All variables 0 O O O I O O O O O O Intercorrelation Matrix for All Variables When Only Males are Included . . . . . Intercorrelation Matrix for All Variables When Only Females are Included . . . . vi Page 33 36 38 40 42 45 46 47 'it'l'll'l‘ [[f‘I [I LIST OF FIGURES Figure Page 1. B and d' for Theoretical "Signal" and "Noise" Distributions . . . . . . . . 12 2. A Typical ROC Curve . . . . . . . . . 16 ' I I 3. Comparison of d LDF Versus 3 SS . . . . . 34 vii INTRODUCTION Much research time and effort in applied psychology over the last twenty years has been Spent investigating the human decision making process. This research has developed largely in two, and possibly three, broad areas. The atten- tion of theorists has been held the longest by the problem of how best to consistently reproduce or predict decisions. Increased psychometric SOphistication has led to relatively simple mathematical models which often do a better job of decision making than man himself. Linear regression models, for example, have been employed in hundreds of successful applications (Slovic & Lichtenstein, 1971). A second area of interest has focused on the cog- nitive processes that comprise thinking and decision making. Although psychologists can claim significant advances in their understanding of physiological and mental phenomena, unanswered research questions continue to abound. Little has progressed beyond the purely theoretical stage (Hayes, 1968; Newell, 1968). Finally, a less pursued, but potentially rewarding research vein has flourished recently and stands ready to make a significant contribution to decision making science. Its emphasis is on individual differences and their corre— lates (Cleary, 1966; Gulliksen, 1964; Rim & Cohen, 1968). Just as no two men are expected to possess identical per- sonalities, intelligence, strength, etc., neither can their decision making capabilities be considered equal under all conditions. The responsibility of psychologists is, there- fore, to develop the ability to identify those individuals with special decision making talent in specific environments. The research to be reported here introduced a decision making model not unfamiliar to the experimental laboratory, but one which has received limited attention in applied psychology. Unlike other models, its usefulness is underscored by the ability to describe individual subjects' decision making behaviors along two independent dimensions, response bias and sensitigity; The advantages of this model Pr. WM 9"” should have particular appeal to applied psychologists in personnel selection settings. In the past, the bulk of selection research has dealt with the reliability, validity, and best combination of paper and pencil tests. The present research, on the other hand, focuses on the decision maker himself with emphasis on the correlates of his sensitivity and bias when faced with data on a large number of appli- cants. Understanding of these correlates should aid in placement of the most qualified decision makers in selection environments. After a review of the literature, the two dimensions are discussed in detail as part of a comprehensive dis- cussion of the model. Later, hypotheses are offered with respect to the model's behavior in an applied setting. Further hypotheses relate to the correlates of decision making as measured by the aforementioned dimensions. Empirical data are then presented to test the hypotheses, followed by a discussion of the model and recommendations for further research. REVIEW OF THE LITERATURE Criticism of the human as decision maker has received widespread attention in the psychological litera- ture. Man has been credited with characteristic defi- ciencies in his ability to problem solve and process information. Notable among these deficiencies are his tendencies to ignore information, alternatives and goals (Churchman & Eisenberg, 1964). Furthermore, his alleged gross misaggregation of data is blamed on conservatism (Edwards, 1968; Edwards & Phillips, 1964), unreliability, and low validity (Goldberg, 1968). As a result, new strategies involving statistical decision theory have been sought for application to problems previously attempted by man. Much of the research along this vein has centered around the debate between clinical versus statistical decision making. Meehl (1954), a clinician and noted writer in this area, concludes in his summary of the evidence that predictions made actuarially, i.e., those involving statistical decision theory, are either equal or superior to subjective judgments. Adding that the record of clinicians' validity is poor, he goes on to suggest that perhaps the clinician should limit himself to therapy, rather than involve himself with prediction (or judg- mental) problems. Hoffman (1960) echoes Meehl in calling for rigorous investigation and description of the judg- mental process in controlled settings. Without attempting to uncover the underlying cognitive processes involved in decision making, he suggests "paramorphic" representations, i.e., "black box” models that will accurately predict human judgments based on how they have been made in the past. Goldberg (1965, 1968, 1970) in a series of articles follow— ing up on Hoffman's suggestion has dramatically demonstrated how models, constructed from a judge's previous decisions, will outperform the judge himself in future decisions. Arguing that the cost of professional time is becoming prohibitive, he adds, " . . . only rarely-~if at a11--wi11 the utilities favor the continued employment of man over model of man (Goldberg, 1970, p. 432]." The actuarialists, having agreed on the benefits inherent in models, have directed their efforts toward determining the most apprOpriate and successful "para- morphic" representation. One school of thought has argued for the development of configural models, based on the intuitive notion that man as information processor surely must perform in a more complex manner than a simple linear model would suggest. For example, Sheridan and Carlson (1972) used an analysis of variance approach. Five measures for 288 hypothetical life insurance agents were presented to 26 insurance executives and agents, who were asked to decide on an agent compensation policy. According to the authors, the predictors selected were as close to "real" data as possible (1) new premium income for the current year; (2) new premium income for the preceding year; (3) average new premium income for the third through fifth preceding years; (4) average new premium income for the sixth through fifteenth preceding years; and (5) the first year persistency of the agent's sales. Discussions with the 26 judges revealed that they would be inclined to reward the same agent performance differently depending on whether an increase or decrease had occurred from previous years. The proposed hypothesis, therefore, was that this configural decision making would be reflected by significant interaction effects in the ANOVA. Results showed, however, the number of significant interactions per judge to vary considerably. In addition, the amount of variance accounted for by these interactions was small, ranging from .5 to 5 per cent. Hoffman (1968) also reports minimal success in disclosing configural effects in deci- sion making. Hammond, Hursch, and Todd (1964) are unsure whether research of this type has potential payoff, sugges- ting that in order to hypothesize man as a configural information processor, it must be proven that nonlinear .‘Q. . "bit? 13'. A“; - . ( - h-..1‘ i; Wag—5...... _ . < 1 combinations of cues (1) exist, (2) are detectable and (3) are worth the trouble to detect them.1 The overwhelming bulk of literature supports application of linear regression. Reiterating Hoffman's (1960) call for paramorphic models, Dawes (1971) maintains that the linear model is the best representation, empha- sizing that even though it may not necessarily accurately describe how individuals make decisions, it yields the best prediction of those decisions. Others (EinHorn, 1970; Fricke, 1956; Goldberg, 1965, 1968, 1970; Meehl, 1954; Slovic & Lichtenstein, 1971; Ynetma & Torgenson, 1961) have presented convincing arguments and support for this claim. Returning to the clinical-actuarial debate, Meehl (1954) suggests in rebuttal that even though a regression equation may be solved on what appear to be purely statis- tical bases, at a much lower level, i.e., in the develop- ment and scoring of predictors, human judgment is likely to enter in. Actuarial methods are also seen by clini- cians as gross simplifications of cues, particularly those encountered in clinical and selection interviewing. 1One of the issues that writers in this area fail to address is the question of the subjective impression that an objective datum makes on a judge. This relation-I ship between subjective impressions and objective data ' may be related by some psychophysical function which is configural in nature, even though in the combination of cues, linear models are the most predictive of judges' decisions. Rigby (1964) adds that actuarial combination of variables may not always be possible. When rapid fire decisions must be made, or when information is either incomplete or biased, the subjective judge has the flexibility to respond accord- ingly. Intuition and bias cannot be programmed given the present state of the art, Rigby says. Both sides of the controversy maintain that they offer convincing support for their theses. The statis- ticians have shown without doubt the power of actuarial models, particularly linear regression. They are careful to explain, however, that these models may tell us little about how decisions are made. Clinicians, interviewers, and others emphasize the role of man as intuitive and flexible information processor, but offer little research evidence. As suggested by several authors (Sawyer, 1966; Shelley & Bryan, 1964; Shepard, 1964), the controversy is probably best resolved by a synthesis of the two points of view. More specifically, statistical models can be pro- grammed in instances where cues are reliably quantifiable and where cost of professional time becomes prohibitive. Man, on the other hand, will be called upon to perform what really is the more difficult task, that of evaluating information of a more subjective nature. As discussed above, psychometricians have fever- ishly investigated the appropriateness of certain decision models. It has been shown that the linear model can be applied to almost all situations. Evaluating individual subjective decision makers has received somewhat less attention, however. Naylor and his associates (Naylor, Dudycha, & Schenck, 1967; Naylor & Schenck, 1966; Naylor & Wherry, 1965) have demonstrated how, with their "policy capturing" technique, models of individual judges can be compared and grouped according to similarities. This approach contrasts from previous comparisons of judges in, that it correlates models (or 'policies") of judges rather than their actual decisions and/or predictions. Policy capturing research should make a significant contribution to better understanding differences between judges. More such analysis is needed, however, with particular attention paid to individual differences among judges and the corre- lates of these differences. It is the goal of the re- search to be reported here to develop a new model capable of differentiating judges on two parameters, sensitivity’ and response bias, and to determine correlates of these 3 parameters. I The model employed is borrowed from psychoPhysics and was applied to a problem in graduate student admissions. More specifically, the theory of signal detectability (TSD) as first described by Peterson, Birdsall, and Fox (1954) has been selected for its ability to differentiate decision makers along two dimensions: (1) sensitivity and (2) response bias. Before developing a. -W. V‘N x the application, some explanation of TSD is necessary. 10 Several excellent summaries (Coombs, Dawes & Tversky, 1970; Green, 1960; Green & Swets, 1966; Swets, 1961; Swets, Tanner, & Birdsall, 1961) have been written and the reader is referred to these for a full treatment. Briefly, the theory evolved as an alternative to classical threshold theory in perception. Threshold theory concerns itself with at what minimal level of stimulation a sensory event, or signal, is perceived by S. Basic to the theory is the assumption that there is some absolute fixed level below which S’is unable to detect a ”signal" from a no- signal, or "noise" condition. TSD, on the other hand, hypothesizes that there is no such fixed sensory level and that whether or not S responds "signal" to a sensory event will depend on variables like the prior probability: of signal occurrence and the so-called "payoff matrix" : associated with making correct detections or committing: false alarms. Both of these variables will define S's I decision making rule, or as it is called in TSD, his criterion level (B) of performance. To backtrack briefly, TSD is applicable in cases where S must choose between two hypotheses, "signal" versus "noise," on the basis of an observed sensory event. In addition it is assumed that since all signals contain noise, both events can be represented as probability distributions (ideally normal and of equal variance) along a_unidimensional axis. The addition of a signal sensory ‘\ -—.—.—-“"~+~—W- a, ‘ I‘ . A , ‘9‘", . f 93" \ .\ g, oflaxqvx a. X“ 11 event simply shifts the signal distribution to the right as a function of signal strength. Figure 1 shows how these two distributions might theoretically appear. It should be noted that the distributions may exist a_priori when there is only one cue,1 but are created by S's } weighting of cues when he must integrate more information.f §fs choice of response ("signal" or "noise") will depend . on three variables: (1) the prior probabilities of the K two events; (2) the payoff matrix involved in making ‘- acorrect and wrong responses; and (3) the overlap of the 3'idistributions themselves as g perceives them. (Note: (1) and (2) determine the criterion cut-off point and (3) is a measure of §fs sensitivity.) Fundamental to TSD is its assumption that g trans- \ forms sensory events to a decision axis known as the “Min-‘- likelihood ratio, 1 (x). Simply stated, S will decide on}; the basis of the observed event from which of the two 1‘ distributions the event was mere likely to have come. If i this likelihood ratio exceeds a certain value, namely the criterion level (B) as defined below, S will respond ”signal." The decision rule then can be defined as follows: 1Although the signal and noise distributions may exist a_ riori, they will vary in their separation in standard Heviation units across individuals. 12 maoausnfluunao .mmaoz. can .Hucmamg anoauououss “on .6 6:6 m .h P “'13...- CD I)? 373.27... .4422? .332. .a musmwm sz>m >¢Omzmmuu 13 S Will reapond "Signal" When 1 (x) 1 a or, when 2 (x48) th (N) an + VSn p (x/N) p (S) VSs + VNs 1 (x) where p (x/S) is the probability of the observed event given that it is from the "signal distribution"; p (x/N) is the probability of the observed event given that it is from the "noise distribution"; p (g) are the prior odds; Nn + VSn represents the payoff values associated VSs + VN8 with the two responses based on which event actually occurs; < V = the "value" of responding "noise" given a noise stimulus; V = the "value" of responding "signal" given a noise stimulus; V = the "value" of responding "signal" given a signal stimulus; V = the "value" of reaponding "noise" given a signal stimulus. It is important to note that B is a variable parameter and will fluctuate as either the prior odds or §fs perception of the payoff matrix is altered. The payoff matrix, incidentally, need not represent material reward or cost (objective matrix), but can be changed as a function of how important g considers hits, false alarms, misses, and false negatives to be (subjective 14 matrix). Therefore, it is highly likely that §fs person- ality will affect the payoff matrix. Returning to Figure 1 should demonstrate how B would be determined in the simplest case, i.e., where p (N) = p (S) and where all values in the objective payoff matrix are equal. In this case the criterion cutoff would be at the point where the probabilities for the two dis- tributions are equal, namely at their intersection (1 [x] = 1). This point is displayed in Figure 1 by labeling it B. Any fluctuation a giveng demonstrates from this point reflects personal meanings imposed by him} .. .-_V._.‘ -._ _ -.—~ M . to create a subjective payoff matrix, since it is assumed QEmEBEY§JFh§lPrior_odds. Of course, it is also possible_ that S may have difficulty grasping the meaning of the prior odds, particularly if they are not 50:50. This deficiency in information processing could also affect B. TSD describes how B can be computed for a given g. Relatively large B indicates a rather strict criterion, whereas lower B reflects more leniency, i.e., a greater willingness to respond ”signal." The Bparameter provides -a rather powerful measure of §fs response bias. A Also of interest is gfs basic sensitivity to the sensory stimulus. As described by TSD, it is conceivable that a decision maker will have a very difficult or perhaps easy time discriminating the two distributions, regardless of his response bias. Fortunately, TSD provides an 15 independent measure of sensitivity, known as the sensi- ...- a z' - l-Q-i-‘H'w fl --‘ tivity index (d:). The sensitivity index, unlike B, is fixed since it is based entirely on the mean difference between the distributions along st sensory axis. d' is represented in Figure l as the distance between the means, of the two distributions. I A method has been developed in TSD to represent §fs total decision making performance along a single curve, known as the receiver Operating characteristic (ROC). Only two parameters are necessary to define a point on the curve, the hit and false alarm rates at a given criterion level. By forcing §.t° manipulate his criterion level (by altering prior odds, the payoff matrix, or by a method to be dis- cussed below), several points and ultimately the entire ROC are defined. B is the lepe of the ROC at a given point; , d' is the area under the ROC. A given 8' 3 ROC will repre-l w ~.—__.._ ..-...~......_.- - a '1‘. .J" Mv—o-o sent a single sensitivity level and the different B's under W.” various conditions. Figure 2 offers a sample ROC curve. Green and Swets (1966) suggest that a minimum of 200 ob- servations are necessary to generate a single ROC point, and that an entire curve will often require 2000-3000 subject trials. Obviously, this sample size can be dis- couraging to even the most ambitious (and wealthy) experi- menter. To alleviate this problem, rather than having gs respond "signal” or "noise" for 200 observations at each criterion level, it is suggested that they be asked to 16 HIT RATE .5 I FALSE ALARM RATE Figure 2. A Typical ROC Curvea aThe slope at each point along the ROC represents B for a given prior odds and payoff matrix; the area under the ROC is d'. . 17 maintain multiple criteria for each observation. This can be accomplished by employing a rating method, i.e., asking the subject to rate the degree of certainty he feels about the sensory event having occurred. The number of rating categories will define the number of defined points on the ROC, thus saving considerable time. Experimentation with the rating method has produced strikingly similar ROCs to those obtained under "signal-noise" conditions (Emmerich, 1968; Nachmias, 1968; Shipley, 1970). In summary, TSD offers three representations of a decision maker's performance: B, d' and an ROC curve. Critical to acceptance of TSD as a valid model is the ques- tion of whether these measures perform as theorized. For the most part, the literature offers substantial support, particularly in perceptual experiments (Broadbent & Gregory, 1965; Davenport, 1969; Galantar & Holman, 1967; Mackworth & Taylor, 1963; Markowitz & Swets, 1967; Schulman & Greenberg, 1970; Swets, 1959, 1961; Swets, Markowitz & Franzen, 1969; Swets & Sewall, 1963; Weintraub & Hake, 1962). DeSpite the widespread interest of TSD in percep- tion and psychOphysics, the model has not been limited to these areas (Peterson & Beach, 1967). One study (Ulehla, Canges, & Wockwitz, 1967), for example, applied TSD to conceptual information. Using word samples as stimuli, gs were asked to guess from which of two magazines a given 18 sample was extracted. Results showed that regardless of experimenter imposed objective payoff matrix or prior odds, d' remained stable within gs. In another paper, Price (1966) discusses TSD in the context of its super- iority over traditional threshold theory in the study of size constancy differences between schizophrenics and normals. Whereas traditional methods yield conflicting results, TSD shows without doubt that nonparanoid schizo- phrenics tend to have lower sensitivity (d'). Price criticizes traditional threshold methodology for (1) method-specific nature of results, (2) arbitrariness of threshold definition, (3) inadequacy of corrections for guessing, and (4) the confounding of sensitivity and criterion level. All of these problems are eliminated by TSD, he adds. Siegel and Pfeiffer (1969) report an interesting study where they gave appropriate true (signal)-false (noise) achievement tests to two samples; one comprised of Navy S8 at various levels of training and/or experience, the other made up of introductory psychology students of diverse majors. d' was calculated and found to correlate significantly with GPA in the college sample and exper- ience in the Navy sample. When applying d' to a multiple; regression analysis with the usual predictors of success’ in the Navy and academic environments, the increase in explained variance was 51% and 40% respectively. 19 Schuck, Cross and Mills (1970) used TSD to deter- mine if nonperceptual factors enter into response on the Rod-Frame Test (RFT). gs were asked to respond on a four category scale of certainty that the rod was vertical after experimenter manipulations. Hits and false alarms were computed for the 17 S3. Findings showed RFT scores to correlate highly with d'. The notion that field dependence (as tested by the RFT) can be explained by response bias (B) was rejected. The authors note that no such evidence could be produced without TSD and its unique capacity to differentiate between sensitivity and bias. Applications of TSD have also been made in a lie detector study (Ben Shakar, Lieblich, & Kugelmass, 1970), eyelid conditioning (Rees & Fishbein, 1970), and_evalua- tignfigfmtherapy (Chapman & Feather, 1971). It would seemf that TSD could be useful in a wide variety of settings. 7 The present research applied TSD to a decision making problem involving the selection of students to graduate school. Making use of previously collected criterion and predictor data (Schmidt & Marshall, 1972), gs were asked to judge whether given applicants were accepted (8) or rejected (N) by the faculty. These judgments were based solely on four predictors: GRE-V, GRE-Q, GRE-Advanced, and undergraduate GPA. The predictor and criterion values (acceptance or rejection) were applied in the earlier study to the linear discriminant 20 function (LDF) to yield the optimal linear policy-capturing (or paramorphic) equation. The LDF should providewthe Optimal linear TSD_function.since by definition LDF maximizes mean difference between groups while minimizing within group variance (Davies, 1970; Day & Kerridge, 1967; Overall & Klett, 1972; Tatsouka, 1971; Van de Geer, 1971). As a result, its d' will tend to be relatively high. By programming a criterion cutoff point based on calculation of prior odds and use of the objective payoff matrix, the LDF will always perform at optimal TSD levels. ‘ngwgnd vv-a—u‘...‘ the LDF are theoretically very similar. Both, for instance,i ”mu-- m. ‘ . Maw.-.', .. r1 .H—n .- . suggest Optimal decision making strategies based on maximum' distinction between groups. Among noteworthy differencesfi is the LDF's assumption of linear combination of cues, % whereas TSD makes no such assumption. This difference % may be more apparent than real, however, as discussed above ' in the section on nonlinear models. Another difference is that TSD does not allow the researcher to determine st cue weights (but this can be overcome by use of the LDF if linearity is assumed). HYPOTHESES The hypotheses to be investigated in the present study are listed below. Hypothesis 1 Based on previous research with subjective versus statistical decision making, the LDF will outperform all subjective judges in determining actual faculty judgments. The TSD parameter, d', will be higher for the LDF than for any of the judges. If the judges are superior to the LDF, it can only be interpreted as indicating a nonlinear combination of cues which characterized faculty judgments as well as a comparable strategy that the gs use in their judgments. Hypothesis 2 B and d' will behave as theorized in perceptual experiments. Criterion level manipulations (by use of the rating method) will result in an ROC curve of expected shape defining a sensitivity level for each judge. Furthermore, a correlation of zero between B and d' is 21 22 hypothesized since they are theoretically independent measures . Hypothesis 3 It has been argued (Carlson, Thayer, Mayfield & Peterson, 1971; Rowe, 1963) that merely describing decision makers' models will not necessarily yield much information about the variables influencing individual decisions. Moreover, McGee (1962) argues for independent behavioral measures of personality traits in decision making settings. Some research has been initiated. For example, Scodel, Ratoosh and Minas (1959) correlated intelligence tests with decisions under conditions of risk, finding intelli- gence not related to degree, but to variability in risk taking. Liverant and Scodel (1960) used a test purportedly measuring the degree to which an individual feels he con- trols his environment (as opposed to luck) and applied it to risk taking decisions. Need for achievement and mani- fest anxiety were correlated with probability learning, decision making and risk taking by Gentile and Schipper (1966). Results overall have generally been inconclusive. Investigations into the personality correlates of TSD parameters have been few. In one such study, Rappa- port and Hopkins (1967) proposed to determine if TSD parameters of schizophrenics deviated significantly from the optimal parameters and normals. This study also was to investigate the effects of drugs on detection 23 performance. No results were reported since actual research had not yet begun. In another TSD application (Strickland & Rodwan, 1964), a correlation of -.64 was found between the Marlowe-Crowne Social Desirability Scale (Crowne & Marlowe, 1960) and the ratio of hits to false alarm rate in a TSD task. It was concluded that high need for approval as measured by the scale motivates gs to guess a signal has occurred with greater frequency; in; other words, their criterion levelfiis’lower. I The present research utilized the Marlowe-Crowne Scale by investigating two hypotheses: (3.1) need for approval, as measured by the M-C Scale, will correlate ‘ negatively with the TSD parameter, B. gs who adopt a relatively strict criterion (high B) will score lower on the scale than those with lower criterion levels; and (3.2) there will be no significant correlation between M-C scores and d'. This second hypothesis is based on the TSD assumption that d' is invariant with respect to purely personality variables (as opposed to cognitive variables).1 Reliability and Validity of the M-C Scale Crowne and Marlowe (1960) in their study which led to the development of the M-C Scale report reliability 1It also assumes an orthogonal relationship be— tween cognitive and personality space, i.e., the Marlowe- Crowne Scale is not correlated with other cognitive variables that might correlate with d', thus producing an indirect relationship between M-C and d'. 24 data on 39 gs. Internal consistency measurement, using Kuder-Richardson formula 20 was .88. Thirty-one §s took the scale on two occasions one month apart; test-retest correlation reached .89. A larger scale reliability study was conducted by Fisher (1967) on 650 Peace Corps volun- teers. While internal consistency was not tested, a test-retest correlation of .84 was found using a one week interval. Validation of the scale, as with most personality scales, was achieved by its correlations with scales of other theoretical variables. In this case, intercorre- lations of the M-C Scale with the Edwards Social Desir- ability Scale (.35, p<.01) and with various MMPI scales were inspected and the conclusion reached that the M-C Scale was a useful index of need for approval independent of psychOpathology. Hypothesis 4 Gardner (1953) observes, "(it) is an everyday clinical observation that persons vary widely in the 'span' or 'realm' of objects, qualities, and so on, which they are willing to subsume under one conceptual rubric as being 'the same' in the sense of being 'not different' [p. 214]." This observation refers to the well-known psychological concept of category width. Pettigrew (1958), in his discussion of the concept, explains that there have been several interpretations of just what is being tapped 25 with category width measurement instruments. It may be they are measures of g'srisk‘t‘aking tendencies (TSD’s B) 3 or, on the other hand, merely measures of 8's concept of; equivalence range (TSD's d'). An instrument to measure J category width has been developed by Pettigrew (1958) which was used in the present research to investigate the following null hypothesis: There will be no significant correlation between the Category Width Scale (CW) and either of the two TSD parameters. A significant corre- lation with B would be interpreted as indicating that §fs category width is related to criterion level and, therefore, varies as prior odds and/or the payoff matrix vary. A significant correlation of CW with d' would be interpreted as indicating that st category width is a stable trait related to his basic sensitivity. It was not expected that significant correlations would occur between CW and both TSD parameters, although the possibility exists. This result would be more difficult to interpret. Reliability and Validity of the C-W Scale Like the M-C Scale, reported reliability of the Category-Width Scale is satisfactory, and validity evi- dence is sparse. Split-half forms of the C-W Scale were administered at least six weeks apart to 97 gs, yielding a corrected reliability coefficient of .72. Internal consistency reliability on odd-even forms of the scale 26 for 281 gs was reported at .90. Criterion validity was evaluated primarily through relationships with laboratory measures of category width, e.g., estimating the weight extremes of ostrich eggs using a fixed set of weights. The rank order correlation between the C-W scale and the combined ranking on five laboratory measures was .57_ (p<.01). A review of the literature since the appearance of the C-W scale yielded no further studies of the psycho- metric properties of the scale. Hypothesis 5 Since it may be that mental ability is related to the processing of information, two additional measures will be made on each S: (l) the Employee Aptitude Survey verbal comprehension test (reported alternate forms reliability = .83) and (2) self-reported Scholastic Aptitude Test (SAT) scores. The null hypothesis to be tested is that there will be no significant correlations between the mental ability measures and TSD parameters. METHODS Data Source for Predictors and Criterion A recent study (Schmidt & Marshall, 1972) collected information on 3,808 graduate school applicants in psycho- logy for the period 1967-1971 at Michigan State University. These data were broken down by interest group to yield information on clinical, experimental, social-personality, industrial, developmental, quantitative and ecological applicants. Predictors employed were three GRE scores (Verbal, Quantitative and Advanced) and the undergraduate GPA. The dichotomous criterion was "acceptance" or "rejec- tion" by the faculty. After checking predictor validities, LDFs were generated for the total departmental sample and for the four interest groups where sample size justified computation of separate LDF weights. These weights were then cross-validated. For the present study only the industrial interest group subsample was employed. This decision was based first on the relatively good dis- crimination provided by the LDF for the industrial group data, and because the sample size (Ns216) provided was in keeping with the 200 suggested by Green and Swets (1966) to generate ROC points 27 28 Subjects Forty-eight undergraduate subjects were recruited from introductory psychology classes at MSU, thereby con- trolling their exposure to psychology. Of these, 18 males and 26 females attended both experimental sessions and were included in later analyses. Only 17 were able to remember SAT scores; therefore, results related to these variables should be viewed with caution. Procedure Upon entering the experimental situation, each g was asked to indicate his background in psychology. Then he was administered the 33-item Marlowe-Crowne Social Desirability Scale, the 20-item Category Width Scale, the verbal comprehension test (Employee Aptitude Survey, Form A, revised, 1956), and given instructions to fill in his SAT scores. Approximately one hour was required for this phase of the study. At a later date he was asked back to perform his main task, that of judging faculty acceptance or rejection of the applicants from the four predictors discussed previously. The instructions consisted of five parts: (1) a discussion of the problem inherent in graduate school selection and the importance of good performance in the task; (2) a discussion and hand-out to familiarize S with the interpretation of GRE scores; (3) information indi- cating that the base rate (TSD's prior odds) for the data 29 was 38/100; (4) information that set the values in the objective payoff matrix equally valuable or costly; and (5) instructions requesting §_to rate his degree of confi- dence that each of the applicants in the sample was accepted or rejected by the faculty. The rating scale took the following form: l--Definitely accepted 2--Probably accepted 3--Barely accepted 4--Barely rejected 5--Probab1y rejected 6--Definitely rejected The usefulness of a rating scale of this sort in TSD experiments has been discussed above. After instructions were completed, it was then S's task to rate each of the "applicants." gs were told that their scores on the rating task would be reported to them at a later date and that they would be given a ranking relative to other gs. No further information was reported until all data had been collected. (See Appendix for the complete set of instructions.) RESULTS Reliability of TSD Parameters Of interest prior to formal investigation of the hypotheses were the psychometric pr0perties of d' and B. Failure to determine reliability coefficients would subse- quently lead to possible misinterpretation of relationships between TSD parameters and other variables due to measure- ment error. Determination of the reliability of d' was made easy as a result of the rating scale method used in data collection. Reference to d' tables (Elliott, 1959; Hockhaus, 1972) for the hit and false alarm pairs at each I ROC point allowed determination of five estimates of d' g per subject. Theoretically these d's should have been equal within a given subject, but this was rarely the case. The variance-covariance matrix was generated across gs for d'l - d'5 as a first step toward computing reliability. From this matrix, coefficient a was cal- culated, yielding a value of .68. This coefficient represents the reliability of the average of the 5 d' values; the reliability of a single d' would be 30 31 considerably lower. Nevertheless, the coefficient is appropriate here since Sf was used for each S to define his sensitivity level. Five different B values were also available for each S, but since average B was not meaningful to this study, a different reliability analysis was required, one that would provide the reliabilities of the individual Be as defined by the rating scale. To accomplish this a split-half correlation was calculated for 31' B2, B3, B4, and B5 for the two halves of the rating task. Corrected coefficients were .10, .45, .67, .70, and .21 respec- tively.1 B3 and B4 showed highest reliability; B3 repre- sents the B that would be obtained had the Ss been asked only to accept or reject each applicant (like the faculty). It is this B that was most important for later analyses. Hypothesis 1: The LDF will outperform all subjective judges in determining actual faculty judgments. The TSD parameter, d', will be higher for the LDF than for any of the judges. To compare Ss' subjective decisions with the LDF, their ratings were dichotomized; ratings 1, 2 and 3 represented "accept" judgments, 4, 5, and 6 representing "reject" judgments. This made possible direct comparisons between d' for each judge and the LDF. This approach.is 1Each of these reliability coefficients represents the proportion of variance of the reSpective Bs that is due to personal value changes made in the subjective payoff matrix. 32 chosen over correlating the two sets of dichotomous decisions (Ss--Faculty and LDF--Faculty) because the apprOpriate statistic, the tetrachoric correlation, can be inaccurate by as much as 20 points of correlation, as an estimate of Pearson r (Nunnally, 1967, p. 124). Tables have been provided (Elliott, 1959; Hockhaus, 1972), so only minimal computation of d' was necessary. d' was read from the appropriate table for each point on an S's ROC 3 curve. The resulting five d' values were then averaged . to yield an overall d' for each S. Table 1 gives the d'.. A I value for the LDF (the mean difference in LDF scores I between acceptees and rejectees in standard deviation 1 units) and all 44 Ss. Only one of the Ss demonstrated: higher sensitivity than the LDF. Figure 3 depicts the. significance of the difference between d'LDF and the mean sensitivity of the Ss (p<.05). If the Ss' scores are regressed toward the mean to estimate true scores, making use of the previously discussed reliability coefficient (Nunnally, 1967), not a single S outperformed the LDF in sensitivity. Although not specifically related to Hypothesis 1, there was interest in the obtained B values across Ss. According to the formula on page 13, B is calculated as the product of prior odds and payoff matrix. Given the payoff matrix of 1 assumed in this study, Optimal B (where fewest errors of classification will occur) is 33 TABLE 1 Average d' Values for Ss' Dichotomized Ratings Compared with d‘ for the Linear . . . . , _ Discriminant Function (d LDF-1.230) 5* d' s# d' 2 1.034 26 1.053 3 .737 27 1.055 4 .970 28 .835 5 1.206 29 1.027 7 .753 30 .732 8 .954 31 1.040 9 .721 32 .934 10 1.131 33 .706 11 .923 34 1.252a 12 .816 35 1.141 13 .911 36 .475 14 1.149 37 1.146 15 1.213 38 .902 16 .949 39 .776 17 .983 40 .735 18 .943 41 .989 19 1.034 42 1.042 20 1.025 43 .850 21 .887 44 .943 22 1.168 46 .952 24 .993 47 .922 25 1.030 48 1.066 3‘ = .969 0d' = .148 Note.--All d' values represent mean differences between signal and noise distributions in standard deviation units. aIndicates a higher d' than that calculated for the LDF. 34 mm.w msmum> moq.o mo COmflummEOU hajhv 30.232... .N ---db 9...- mph "----------wor'u .m madman 35 simply the prior odds, or .62/.38 (1.62). To the extent 1 a given S yields a higher B, he tends to be more strict int his judgments, i.e., his criterion level is higher. Likee7 wise, lower B indicates more leniency in "accepting" appli- cants. As demonstrated in Table 2, only 6 Ss set higher criterion levels than the Optimal B (Bopt). Most Ss were more lenient in their judgments than the optimal B would indicate they should be. Hypothesis 2: B and d' will behave as theorized in perceptual experiments. Criterion level manipulations will result in an ROC curve of expected shape defining a sensitivity level for each S. A correlation of zero between d' and B is hypthesized. ROC curves were drawn for each S, making use of the rating judgments to define points. Of particular interest was whether or not these curves would conform to the theoretical model which suggests normality and equal variances for the signal and noise distributions. Investigation of this question involved plotting the ROC curve for each S.on double probability paper. A straight line ROC on double probability coordinates indicates normality; slope of one reflects confirmation of the equal variance assumption. Inspection of the 44.§S' ROC curves supported both assumptions with striking consistency. Consequently, use of d' tables, which assume normality and equal variance, was justified. A Since the validity of distributional assumptions is critical to the determination of d', an additional 36 TABLE 2 Ba for Ss' Dichotomized Ratings Compared Wlth Bopt (1.62) S# B S# B 2 1.577 26 .793 3 1.504 27 .793 4 1.142 28 1.131 5 1.820 29 .881 7 2.069 30 .683 8 1.018 31 .567 9 .445 32 2.206 10 1.151 33 1.989 11 1.877 34 .638 12 .733 35 .914 13 1.378 36 .584 14 1.605 37 .819 15 .836 38 1.277 16 1.345 39 1.336 17 .914 40 1.137 18 .707 41 1.515 19 1.365 42 .474 20 .809 43 .625 21 .885 44 .905 22 1.284 46 1.225 24 .840 47 2.069 25 1.466 48 .805 B'= 1.139 37 analysis involving another index of sensitivity (Richards & Thornton, 1970) was applied to the data. Essentially, the new index, here defined as d", takes into account the ratio of one standard deviation to the other in its calculation. The two formulae are given belowl: d' = z - z N SN 70 = _' = d ZN a ZSN' where a USN/ON It will be observed that d" = d' when 0 = O . A detailed SN N discussion of the a computation, best performed by com- puter,is offered in the paper to which the interested reader is referred. Suffice it to say that if a signi- ficantly high correlation is found between d' and d", one can feel comfortable with his d' values as read from the standard tables. Table 3 gives d' and d" values for all Ss. The correlation between the two sensitivity indices was .92 (p<.001).2 1The Z- values in the formulae above represent the means of the signal and noise distributions on a continuum where the units are in ON form. 2It should be noted here that several writers (Grier, 1971; Hodos, 1970; Pollack & Norman, 1964) over the past few years have offered nonparametric indices of TSD parameters. By and large these methods involve ways of estimating the area of various portions of the unit square used to draw the ROC curve in order to approximate values for d' and B. The advantage of such methods is that there is no need to concern oneself with whether or not distributional assumptions hold true. Several dis- advantages have been noted as well. For example, most pro- cedures are extremely tedious, particularly considering that they provide only very crude estimates of the desired parameters. d' as Determined by Reference to Tables Versus the Sensitivity Index, d" 38 TABLE 3 S# d. d" s# dl d" 2 1.034 1.035 26 1.053 1.114 3 .737 .728 27 1.055 1.183 4 .970 .974 28 .835 .924 5 1.206 1.216 29 1.027 1.033 7 .753 .732 30 .732 .833 8 .954 .987 31 1.040 1.016 9 .721 .857 32 .934 .934 10 1.131 1.149 33 .706 .681 11 .923 .932 34 1.252 1.279 12 .816 .927 35 1.141 1.289 13 .911 .902 36 .594 .784 14 1.149 1.173 37 1.146 1.150 15 1.213 1.348 38 .902 .906 16 .949 .966 39 .776 .827 17 .983 .987 40 .735 .749 18 .943 1.003 41 .989 .990 19 1.034 1.026 42 1.042 1.105 20 1.025 1.149 43 .850 1.061 21 .887 .938 44 .943 .927 22 1.168 1.195 46 .951 .971 24 .993 .985 47 .922 .929 25 1.030 1.025 48 1.066 1.074 = .92 rdld" 39 Further investigation of Hypothesis 2 required computation of the correlation between TSD parameters B and d' in order to ascertain their degree of independence. The B value used for each S was defined by the dichoto- mization of ratings and would be equivalent to the B obtained if Ss had only been required to "accept" or "reject" each applicant (B3). This B was retained for all further analyses. As reported in Table 4, an insignificant correlation (—.11) was found between B and d', as pre- dicted by the theory. Hypothesi§_S: (3.1) Need for approval, as measured by the M-C Scale, will correlate negatively with the TSD parameter, B, (3.2) There will be no significant correlation between M-C scores and d'. H othesis 4: (Null hypothesis) There will be no significant correlation between the C-W Scale and either of the two TSD parameters. Hypothesis 5: (Null hypothesis) There will be no signi icant correlations between the mental ability measures and TSD parameters. Inspection of Table 4 allows determination of the extent to which the two TSD parameters are related to the independent variables--social desirability, category width, verbal comprehension, and Scholastic Aptitude Test scores (verbal and quantitative). With respect to Hypotheses 3.1 and 3.2, confirmation has been achieved. As expected, a significantly negative correlation (-.32) between B and the Marlowe-Crowne scale was found. Ss who 40 TABLE 4 Intercorrelation Matrix for All Variables when the Entire Subject Sample is Included (1) (2) (3) (4) (5) (6) (7) (1) Sex (2) Social Desir- ability .05 (3) Category Width -.35* -.30* (4) Verbal Compre- hension (EAS) -.21 -.31* .12 (6) SAT-.0 -019 -032 -016 061** 065** (7) B .13 -.32* .03 .08 .35 .30 (8) d' .003 -.13 -.02 .10 .09 .26 -.ll Note.--For correlations involving SAT-V and SAT—Q, E,= 17. For other correlations, §.= 44. *p<.05. **p<.01. 41 set high criteria for admissions tended to demonstrate lower need for approval. In addition, the correlation between d' and the M-C scale showed no practical or statistical significance. The null hypotheses has been supported for Hypothesis 4. Absolutely no relationship whatsoever could be determined between category width and either B (.03) or d' (-.02). None of the three measures of mental ability employed in the current study to test Hypothesis 5 yielded significant correlations with TSD parameters. However, given the size of the Es, as well as the magnitude of the correlations, this evidence may not be conclusive. Making use of the earlier reported reliability estimates, several of the correlations appearing in Table 4 were corrected for attenuation. Table 5 reports these corrected coefficients. Caution should be used in interpretation of these new values, particularly given the sample sizes reported in Table 4 (Guilford, 1954, p. 402). For the initially significant correlations, the corrected correlations represent the best estimate of the true relationship between two variables. The proportion of variance accounted for by this true relationship can be obtained by squaring the correlation coefficient. 42 TABLE 5 Intercorrelation Matrix for Variables Corrected for Attenuation for the Entire Sample (E.= 44) (1) (2) (3) (4) (7) (1) Sex (2) Social Desirability .05 (3) Category Width -.39a -.36a (4) Verbal Comprehension (EAS) -.23 -.36a .15 (7) B .16 --.42a .04 .11 (8) d' .004 -.17 -.03 .13 -.16 Note.--Reliability estimates were as follows: (1) Sex = 1.00; (2) Social Desirability = .88 (reported internal consistency and test-retest reliabilities); (3) Category Width = 772 (reported test-retest reliability); (4) EAS = .83 (reported alternate forms reliability); (7) B = .67 (split half reliability, as reported in the text); (8) d' = .68 (internal consistency reliability, as reported in the text). aRepresents a relationship that was statistically significant prior to correction. 43 Additional Analyses Several significant correlations, not directly related to the formal hypotheses, were obtained. Not surprisingly, for example, strong relationships (p<.01) were found between the Employee Aptitude Survey and the two self-reported SAT scores. Furthermore, the correla- tion between SAT-Q and SAT-V was a highly significant .65 (p<.01). Other significant correlations of note include -.30 (p<.05) between the M-C scale and category width. This result suggests that, on the whole, the more a S tended to respond in socially desirable ways on the M-C scale, the more narrow were his categorizations as measured by the C-W scale. Another negative correlation (-.31, p<.05) was reported between the M-C scale and verbal comprehension on the EAS, suggesting a negative relation- ship between at least one measure of mental ability and the need for approval. An additional set of analyses was motivated by suggestion of research advisors, as well as the signifi- cant relationship between the sex variable1 and category width. Large enough numbers of both males and females were Obtained to permit supplementary testing of some of the hypotheses by sex. The results of these analyses are reported below. 1Males were coded "0," females were coded "1." 44 Sex differences are investigated in Table 6, where E tests have been performed on mean differences for all variables. The only significant E_reported in the table reinforces the earlier reported significant correlation between sex and category width. In general, males tended to process information in broader categories than females. Of particular interest is the finding of only slightly different B values. Furthermore, male and female d's were practically identical. Intercorrelation matrices broken down by sex are presented in Tables 7 and 8. Overall, they shed no new light; findings tend to mirror those of the intercorrelation matrix for the entire sample with two exceptions. For females, a significantly negative correlation (-.59) was reported between the verbal segment of the SAT and the Category-Width scale. It has also been demonstrated that the significant relationship between category width and the M-C scale reported for the entire sample was due to females. Table 8 reports a highly significant correla- tion (-.50, p<.01) for females, whereas Table 7 indicates almost no correlation (.01) for the male subsample. Furthermore, it is important to note that with reSpect to the performance of TSD parameters, under both male and female conditions no significant relationship was found between B and d', as predicted by the theory. TABLE 6 45 E Tests for Sex Differences on All Variables Variable Male Mean M Female Mean F T M-C Scale 11.611 6.00 12.192 5.43 .334 C-W Scale 72.167 15.12 59.731 17.95 -2.405* EAS 22.667 2.52 21.577 2.67 -l.36l SAT-V 571.429 89.48 539.000 101.04 - .681 SAT-Q 568.571 51.70 542.000 81.35 - .760 B 1.068 .397 1.190 .510 .853 d' .969 .143 .969 .153 .025 *p<.05. 46 TABLE 7 Intercorrelation Matrix for All Variables When Only Males are Included (2) (3) (4) (5) (6) (7) (2) (3) (4) (5) (6) (7) (8) Social Desirability Category Width .01 Verbal Compre- hension (EAS) -.41 -.03 SAT-V .07 -.50 .87** SAT-Q -.35 -.ll .16 .11 B -.29 -.03 -.24 .22 .13 d' -.40 .32 .13 -.11 -.18 .11 I2 Note.--For correlations involving SAT-V and SAT-Q, 7. For other correlations, §|= l8. **p<.01. 47 TABLE 8 Intercorrelation Matrix for All Variables When Only Females are Included (2) (3) (4) (5) (6) (7) (2) Social Desirability (3) Category Width -.50*# (4) Verbal Compre- hension (EAS) -.23 ..09 (5) SAT—V -.41 -.59* .81** (6) SAT-Q -.51 -.49 .75** .85** (7) B -.37 .13 .29 .50 .42 (8) d. .06 -020 .09 .17 041 -004 Note.--For correlations involving SAT-V and SAT-Q, 10. For other correlations, E.“ 26. I2 ll *p<.05. **p<.01. DISCUSSION The rationale for the use of TSD in an applied setting has been discussed above. As stated previously, the primary motivation behind its use was the ability to differentiate decision makers along the dimensions of sensitivity and response bias. Furthermore, there was research interest in the correlates of TSD parameters in the hOpe that further understanding of them would enhance the placement of decision makers in a wide variety of situations. Bearing in mind these considerations the present research was conducted to investigate the follow- ing questions: 1. What new information can TSD provide to evaluate the use of mathematical models to improve decisions? 2. Will signal detection methodology be adaptable to a specific selection situation? a. Are sensitivity and response bias indicators measurable, reliable and meaningful? b. Will B and d' behave as predicted by the theory? 48 49 3. Can B and d' be related to certain other traits, specifically need for approval, category width, mental ability, and sex? 4. How might TSD be useful to psychologists? a. To what setting might the theory be applicable? b. Where do we go from here? The formal hypotheses were conceived in order to provide preliminary results aimed at answering questions 1, 2, and 3. It is to these questions that we now turn prior to continuing the discussion of TSD's usefulness (question 4) as begun in the introduction and literature review sections. Confirmation of Mathematical Models The strong support for linear regression models in the literature, as well as the ability of TSD's d' to independently extract from a set of decisions a general sensitivity level, led to the development and testing of Hypothesis 1. Siygg the linear discriminant function as the optimal linear decision rule for placement into two groups, 31322 that the LDF maximizes mean difference between groups, and giygg d' as a measure of this mean difference, it was felt that d'LDF would set a sensitivity ceiling beyond which the subject judges would not go. That 43 of the 44 Ss confirmed the hypothesis is extremely 50 convincing support. In this case, however, one must question why any S at all should be superior to the LDF equation. It is possible that the S who outperformed the LDF was more able to parallel the faculty decision-making process. This is an unlikely state of affairs since it would require (1) that the faculty combined applicant information configurally and (2) that the S somehow had insight into this nonlinear decision making. As reported earlier, the bulk of research literature does not support such an hypothesis. Research data reported here also tend to reject it. A second, and more likely answer, relates to the statistical properties of d' itself. As with any other summary statistic based on a relatively small sample, some measurement error in the determination of d' should be expected. ‘Ss' d's were calculated by averaging the values read from tables for the hit and false alarm rates at each of the five ROC points. Ideally, the d' recorded for each point should be identical, but this was never the case. Averaging should help to cancel out some of the error, but only the reliability study clearly evaluated the extent to which Ss' d's were true sensitivity values. As reported earlier, regressing Ss' d' values toward the mean resulted in a situation whereby no single S surpassed the LDF in sensitivity. 51 applicability of TSD to a Specific AppliedTSetting Since the primary emphasis of this study was the generalizability of the TSD model to a new area, Hypothesis 2 is the most critical to its success. The data generated clearly lead one to the conclusion that TSD passed this preliminary test with flying colors. Despite the concern of some critics (Theodor, 1972), as well as pioneers of TSD research (Green & Swets, 1966), that equal variance and normality assumptions might not hold outside the controlled stimulus distributions of the experimental laboratory, the results indicated without a doubt the availability of meaningful, moderately reliable, and measurable TSD parameters in selection research. Further- more, the success of the rating method in systematically varying response bias, as demonstrated by ROC curves, suggests that the once undesirable characteristic of large numbers of trials can be reduced to a level of manageability and practicality. Of still greater impor- tance was the finding that the two parameters are indeed independent, paving the way for further research into their correlates, a more disappointing phase of the current study to which we now turn. Sgiationship_2£_Personality Traits to TSD Parameters Although the results obtained tended to confirm Hypothesis 3, the general feeling after examining the 52 data related to correlates of TSD parameters is one of disappointment. Unfortunately, the present study can do little more than report a replication of the Strickland and Rodwan (1962) finding of a significantly negative correlation between response bias and the need for approval as measured by the Marlowe-Crowne Social Desirability Scale. This in itself is reinforcing. ‘Ss high in need for approval were more lenient, i.e., they were more willing to make Type I errors, or errors of inclusion. They tended to score maximum hits at the risk of increased false alarms. The results relating to category width measurement were not as encouraging. Pettigrew (1958) has suggested that peOple who are broad categorizers are those with a higher tolerance for Type I errors. In TSD language, these are the people who would be expected to set much lower B levels or, as applied to the current study, they would set more lenient standards in admitting the applicants to graduate school. Narrow categorizers, those with high B, prefer more Type II errors whereby false negatives are reduced at the expense of decreased identification of successful applicants. The seeming high degree of relation- ship between B and category width leads to puzzlement in light of the low correlations found in this study. Likewise no relationship between category width and d' could be determined. If the board categorizer 53 makes very gross distinctions between stimuli, he should not be as sensitive as his narrow categorizing counterpart who can make fine discriminations. The difference should be reflected in d' differences; however, no such differ- ences turned up for these data. Given the "face validity" of the relationship of category width to TSD parameters, one must question these results. The category width instrument emits some skepticism with regard to its validity (see earlier comments). Further research is recommended in this area using other measures, a variety of data sets, and manipulation of prior odds and payoff matrix. The failure of any of the three mental abilities measures to significantly correlate with d' or B was discouraging, but in retrospect not as surprising as might be believed upon initial inspection of the results. First, only 17 of the Ss could remember SAT scores, and there is reason to questiOn the accuracy of their memories. Some could remember only the total score for the two tests and, as a result, reported identical scores for the verbal and quantitative sections. Others admitted to being unsure. Based on these findings, little in the way of conclusive evidence forcnragainst the hypothesis can be Offered. The Employee Aptitude Survey, on the other hand, was taken by all Ss and affords more accurate measurement of mental ability. Its shortcoming, however, appeared to Ii. ['1 I [it'lll l l 54 be its inability to make discriminations for the sample employed i3 this stugy. The mean on the 30-item test was 22.02 with a standard deviation of only 2.6. The scores on the survey would certainly have had more variability had they been generated by a sample more heterogeneous than 44 introductory psychology college students. In the future, more heterogeneous pOpulations are recommended in the study of TSD correlates. The added significant correlation found when intercorrelation matrices were constructed by sex should not necessarily be taken as providing important insights into sex differences. Given the 70 correlations reported in Tables 4, 7 and 8, 3.5 significant ones could be expected to have occurred by chance (p<.05). It is con- ceivable, therefore, that the correlation of note falls into this category. The only definite sex difference appears to be on category width, with males tending to be broader categorizers. Although not particularly relevant to the hypotheses, this finding may have ramifications if category width is applied to future TSD studies as recommended above. In summary, the correlational results, although discouraging, did force thought into some of the short- comings of the experimental method. These shortcomings included homogeneity of subject sample, lack of control over accuracy of reporting of SAT scores, and the validity 55 of personality measures. All of these problems can be dealt with in future research. That there is promise is reinforced by replication of the relationship of B to need for approval. Hopefully, researchers will extend their thinking and data to other personality traits. Usefulness of TSD to Psychologists As pointed out in the literature review section, the study of the human decision making process has elicited interest in the area of improving the quality of decisions, as well as understanding the workings of the process it- self. TSD methodology, it would appear, can be a valuable tool in both areas. As useful as equations are in best representing a given decision maker or group of decision makers, they do little more than consistently reproduce those decisions, regardless of quality. TSD, however, by offering measures of sensitivity and reSponse bias, allows the psychometrician the flexibility to modify models in order to attain Optimal values for the parameters as dictated by the situation. As for understanding the underlying decision making process, B and d' make signi- ficant contributions by breaking down decisions into two independent, but critical components. Further investiga- tion of the correlates of these components will permit additional insights with reSpect to the cognitive processes associated with decision making and problem solving. 56 'The application of TSD can be as widespread as psychologists choose to make it. The industrial/ organizational environment, particularly in light of government intervention into the selection process, is demanding tOp quality decisions in the hiring of personnel at all levels. Special problems exist for managerial jobs where few valid predictor instruments have been developed. It will take personnel officers with keen sensitivity, little subjective bias, and a facility for understanding objective payoffs to make decisions affecting the selection of managerial talent. TSD offers a means for identifying these top quality decision makers. Furthermore, given additional research findings relevant to the correlates of sensitivity and bias, early identification of the best peOple for decision making jobs will eventually be possible simply by administering a battery of predictors known to relate to TSD parameters. The industrial situation is not the only one where TSD can be useful. As the nation's universities continue to swell with applications, increasing demands are placed on educational psychologists to suggest ways of improving the quality of admissions decisions. Similarly, the clinician seeks ways to enhance the diagnostic process. Regardless of the setting, however, TSD methodology remains easy to apply and readily understandable. 57 One extension of TSD, the study of the parameters' correlates, has been discussed at length. Another poten- tially fruitful research tangent might involve training peOple to become better decision makers by giving them insight into their own decision making parameters. If it can be demonstrated clearly to an individual that he is inherently biased one way or the other, it may be possible by sensitizing him to this bias to actually move his criterion level to a more reasonable level. Of course, research must provide the final evaluation of such training. To some extent these "crystal ball" forecasts of TSD applicability are a bit premature. The next step is certainly not to be taken in the direction of improving real world decisions. Rather, efforts should be made to gather additional data to substantiate the claims made in this research under similar and modified conditions. For example, for valid conclusions to be made it is suggested that further research investigate the effects of varying prior probabilities and/or the objective payoff matrix on response bias. Despite the limitations of large numbers of trials, at least one study should probably investigate the difference in results obtained between standard "signal-noise" conditions versus the rating method as employed here. Studies of this kind have been conducted many times in perceptual experiments, but caution 58 must be exercised in generalizing these findings to applied settings. The most obvious need, once the above experiments have been carried out, will be the identification of decision making correlates. As demonstrated here, this will be no easy task. Most likely, it will involve the COOperation of psychologists with a wide variety of interests to properly hypothesize, measure, and relate personality variables to decision making parameters. The long-run benefits of such collaboration will hOpefully be great. APPENDIX APPENDIX GENERAL INTRODUCTION AND INSTRUCTIONS TO SUBJECTS Session 1 The experiment in which you have agreed to parti- cipate will be conducted in two parts, both of which in- volve written exercises. Part I will be conducted today; Part II is scheduled for . Be sure you can attend both sessions since no credit may be given for less than full participation. Please be advised, however, that you are not required to participate in 221 experiment in the psychology department. If, therefore, you find you are unable to continue at any time, feel free to discontinue as a subject. You may find that as you progress through the experiment that not all written exercises will appear relevant. Rest assured that every exercise is relevant and extremely important. Although we cannot explain our reasons for each exercise now, full explanation will be provided to you once all subjects have completed the experiment. It is extremely important that you are careful and completely honest in completing the exercises. 59 60 Once the experiment is over and the results tabulated, we will provide you not only a full explanation of the experiment, but also report your own individual performance. (Note: No one else will have access to your scores). If you read and follow directions carefully, the feedback we give you will make more sense and you will have a better understanding of how you stand relative to other subjects. Thank you very much for your participation. [ {11!}.1" lit. 61 If you wish to receive an explanation of the experiment and information about your own performance, we will need certain information. Your name, MSU address and MSU telephone number are requested only so we can provide feedback. This information may also be used to remind you by card or phone of the second part of the experiment. No other use will be made of this informa- tion; work on the written exercises is confidential and will be seen and processed only by the experimenters. You may choose to remain anonymous, but if you do we will not be able to provide feedback. Please fill in the following information if feed- back is desired. NAME MSU ADDRESS MSU PHONE # Important! If you have chosen to remain anonymous, please make a note of the number appearing in the upper right hand corner of this page. This is necessary in order to have some way of identifying you for Part II of the experiment. 29 this now and save the number until _ Session II. Use the 3 x 5 card providedfor this purpose. You should also use this card to record the day and time you are scheduled for Part II. ALL SUBJECTS (whether you remain anonymous or not) are asked to answer the following four questions: 1. What is your sex? (Circle one) MALE . FEMALE 2. What is your current year in college? (Circle) 1 2 3 4 over 4 3. Have you ever been exposed to formal instruction in psychology prior to the class you are now in? YES NO If you circled YES, please explain. 62 4. If you can remember, what were your scores on the SAT (Scholastic Aptitude Test, sometimes referred to as "College Boards")? Verbal (Reading) Quantitative (Math) Please do not turn the page, but wait for further instructions. It: {III-Ill...ll‘ rill-1111 If I 63 Session II This is the second and final part of the experi- ment. At the end of this session, be sure to have the experimenter sign your credit card, indicating that you have received six (6) credits for your participation. During Part I you were asked to keep a record of your subject number for use at this time. Please write that number now in the space provided in the upper right hand corner of this page. Failure to do so will invali- date all of the written exercises you have completed thus far. There is no need to indicate your name unless you had desired to remain anonymous last time, but have now changed your mind in order to receive feedback. If so, please print your name below. NAME Part II of the experiment is concerned with the selection of students to graduate school in psychology at MSU. Each year hundreds of college seniors apply to this department for graduate school, but only a limited number can be accepted. Processing these applications is an expensive and time-consuming procedure. It has been estimated, for example, that in an average year 1700 hours of faculty time costing approximately $19,000 is required to read and evaluate applications. Therefore, we are very interested in how we may make this procedure more efficient while at the same time making accurate decisions about the potential of each applicant. Since many of the applicants can not come to East Lansing for an interview, the faculty must rely on information about applicants' standardized test scores and undergraduate grade point averages in making the decision whether to accept or reject a given person. You as a subject can be very helpful in answering some-Efiestions we have about these faculty decisions. We will present you with three test scores and undergraduate grade point average for some of the applicants to MSU's graduate school in psychology and ask that you guess the faculty's decision for each applicant. In order to do this, it will be necessary for you to have some information about the scores you will see for each applicant. This informa- tion follows below. Each psychology graduate school applicant must take a test similar to the College Boards (SAT) required for admission to undergraduate colleges. These tests are known as the Graduate Record Examinations (GRE) and include three parts: 64 l. Verbal (GRE-V)--simi1ar to the Verbal (or reading andTEhglish) part of the College Boards (SAT). 2. Quantitative (GRE:Q)--similar to the Quantitative (math) part Of the College Boards (SAT). 3. Psychology Advanced Test (GRE-adv.)--an achievement test of“the student's knowledge and understanding of psychology. These tests, like the SAT, are given nationally with possible scores on each test ranging from approximately 300 to approximately 800. You will be given the scores on all three of these tests for each applicant, as well as the applicant's undergraduate grade point average (on a scale identical to MSU's 4-point system). From these four pieces of information on each person who applied, your job will be to decide what the faculty's judgment was. For each applicant, you will be asked to guess the faculty's decision on the following scale: Definitely Accepted (DA) -- an applicant displaying the highest level of qualifications. Probably Accepted (PA) --—- an applicant displaying generally good qualifi- cations. Barely Accepted (BA) ------ an applicant displaying minimally acceptable qualifications. Barely Rejected (BR) ------ an applicant displaying rather limited quali- fications. Probably Rejected (PR) ---- an applicant displaying generally poor quali- fications. Definitely Rejected (DR) -- an applicant displaying the lowest level of quali— fications. It is important that you be careful in making these judgments since we are interested in determining how to select the best new students in the future (and perhaps ygg may someday be among these applicants). Due to the limitations of the graduate program (size of faculty, Office space, classrooms, etc.), not all appli- cants can be accepted. In fact, only 38% (38 of every hundred who apply) receive acceptance notices; the 65 remaining 62% must be rejected. Therefore, you should be very selective. As costly as it is to reject the highly qualified student (who may someday become a great psychologist), it is e uall as costly to accept the unqualified student wHo is likely to experience severe personal failure in graduate school. You should do your best to accept the good students, but reject the un- qualified ones. The information you will be given (three GRE scores and undergraduate GPA) have demonstrated through experience that they are valid predictors of future performance. Be sure to study this information carefully for each applicant. To assist you in making your judgments, a guide sheet will now be handed out by the experimenter to familiarize you with what kinds of scores to expect. .owummoom on coo mucmowammm on» mo Aooa mum>m «0 uso mmv wmm usonm maco "mmmzmzmm "omens was» cflnuw3 Hamw m.m I m.m mum I awe mmm I mow new I mow mucmOflHQO mnu Ham mo m\~ waovfie one 66 0.6 I m.a own I omm cam I can own I oom mucmoaammm Ham How monoom mo mmamm ma.m mam mam «mm mnemoaadam Ham How mmmum>¢ .¢.m.0 Amwoqomuwmmv m>HB¢BHBz¢DO Admmm> madoodmommozo omoz¢>od mmw mmw mmo .muGMOHHmmm mumsHm>m 90% mm Spawn ummsm was» doom .mucmemusn “90» ca so» spasm on cofiquuomcw 080m nufiz macaw 3oamn woumwa mum unmoflammm some MOM 00m Haws so» :OHumEuowcH mo mucosa upon one MBZMZUQDH mom BHmSm NQHDG 67 a: 0 01 Q peqoefeu Ktearurgaa .mwuoum mnu mcHDOSHm>m cw 50> dam: on ammmm moHDw usom mos Ou HOQEOEmm .mucmowammm Ham How moflomo amps so» .mcofimflomo Mao» wumHQEOO Op cows 90» mafia on» flaw O>wn HHH3 so» mocwm swan on cows 0: ma shone mmdowummsq mad mm mm «m «a do a o m a mm mm .«m .3 «o z N w x a. m w m w. m w m w m. Add 90328me EH95 Hammer .m. m. Tm...“ m m m m. m U seepage omozgna -3248 o e o I I a e a u Immozo $8 .58 «4a. 1x" .45 Ana «+1. 8 T. 3 a 8 T. 8 3. PK D. P D..A D. W .A . .3OH0Q mOHmEOxO Ozu may ca mm GEsHoo mumwumoummm on» cw mumuuma 0:9 OHOHHU mmmmam .coflmfiomn m.>uasomm on» no omega Ado» some m>mn so» mono .OHMOm coauomnmu Ioosmummoom on» macaw mvcflom me on» on Haws cowumEHOmsa own» mcw3OHaom .Ouesam>o ou omxmu was so» unmowamam some now ommum>m peace ounsomumumocz can mmuoom pump owns» may cram Hafi3 so» BOHHOm was» amuse ecu so _ mcoauosuumcH LIST OF REFERENCES LIST OF REFERENCES Ben Shakar, G., Lieblich, J., & Kugelmass, S. Guilty knowledge technique: Application of signal detection measures. Journal of Applied Psychology, 1970, S3, 409-413. Broadbent, D. E., & Gregory, M. Effects of noise and signal rate upon vigilance analyzed by means of dec1sion theory. Human Factors, 1965, 1, 155-162. Carlson, R. E., Thayer, P. W., Mayfield, E. C., & Peterson, D. A. Improvements in the selection interview. Personnel Journal, 1971, SS, 268-275. Chapman, C. R., a Feather, B. W. Sensitivity to phobic imagery: A sensory decision theory analysis. Behavior Research and Therapy, 1971, S, 161-168. Churchman, C. W., & Eisenberg, H. B. Deliberation and judgment. In M. Shelley & G. Bryan (Eds.), Human judgment and Optimality. New York: Wiley, 1964. Cleary, T. A. An individual differences model for multipleV, regression. Psychometrika, 1966, Si, 215-224. Coombs, C. H., Dawes, R. M., & Tversky, A. Mathematical ‘x s cholo , an elementary introduction. EngIewood/* CIiffs, N.J.: Prentice-Hall, 1970. Crowne, D. P., & Marlowe, D. A new scale of social desirability independent of psychopathology. Journal of ConsultingyPsychology, 1960, Si, 349-354. Davenport, W. G. Vibrotactive vigilance: The effects of cost and values on signals. Perception and PsychOphysics, 1969, S, 25-28. Davies, M. G. The performance of the linear discriminant function in two variables. Sritish Journal of Mathematical and Statistical Psychology, 1970, SS, 165-I76. 68 69 Dawes, R. M. A case study of graduate admissions: Application of three principles of human decision making. American Psychologist, 1971, SS, 180-188. Day, N. E., & Kerridge, D. F. A general maximum likelihood discriminant. Biometrics, 1967, SS, 313-323. Edwards, W. Conservatism in human information processing. In B. Kleinmuntz (Ed.), Formal representation of human_judgment. New York: Wiley, 1968? Edwards, W., & Phillips, D. P. Man as transducer for probabilities in Bayesian command and control systems. In M. Shelley & G. Bryan (Eds.), Human judgment and Optimality. New York: Wiley, I964. Einhorn, H. J. The use of nonlinear, noncompensatory models in decision making. Psychological Bulletin, 1970, ZS, 221-230. Elliott, P. B. Tables of d'. Ann Arbor, Michigan: University of MicHigan Research Institute, 1959. Emmerich, D. S. ROCs obtained with two signal intensities presented in random order, and a comparison between yes-no and rating ROCs. Perception and Psycho- physics, 1968, S, 35-40. Fisher, G. Normative and reliability data for the standard and cross-validated Marlowe-Crowne Social Desirability Scale. Psychological Reports, 1967, SS, 174. Fricke, B. Prediction, selection, mortality and quality control. College and University, 1956, SS, 34-520 Galantar, E., & Holman, G. L. Some invariances of the iso-sensitivity function and their implications for the utility function of money. Journal of Experimental Psycholggy, 1967, ZS, 3 - . Gardner, R. W. Cognitive styles in categorizing behavior. Journal of Personality, 1953, 22, 214-233. Gentile, J. R., & Schipper, L. M. Personality variables in probability-learning, decision-making and risk- taking. Perceptual and Motor Skills, 1966, SS, 583-591. 70 Goldberg, L. R. Diagnosticians vs. diagnostic signs: The diagnosis of psychosis vs. neurosis from the MMPI. Psychological Monographs, 1965,79, (9, Whole No. . Goldberg, L. R. Simple models or simple processes? Some research on clinical judgments. American Psychologist, 1968, SS, 483-496. ' Goldberg, L. R. Man versus model of man: A rationale, plus some evidence for a method of improving on clinical inferences. Psychological Bulletin, 1970, ZS, 422-432. Green, D. M. Psychoacoustics and detection theory. Journal of the Acoustical Society of America, 1960, SE, 1189-1203. Green, D. M., & Swets, J. A. Signal detection theory and psychOphysics. New York: Wiley,’l966. Grier, J. B. Non-parametric indexes for sensitivity and bias: Computing formulas. Psychological Bulletin, 1971, ZS, 424-429. Guilford, J. P. Psychometric methods. New York: McGraw- Hill, 1954. Gulliksen, H. The structure of individual differences in Optimality judgments. In M. Shelley & G. Bryan (Eds.), Human judgments and optimality. New York: Wiley, 1964. Hammond, K. R., Hursch, C. J., & Todd, F. J. Analyzing the components of clinical inference. Psychological Review, 1964, ll, 438-456. Hayes, J. R. Strategies in judgmental research. In B. Kleinmuntz (Ed. ), Formal representation of human judgment. New Ybrk: Wiley, 1968. Hockhaus, L. A table for the calculation of d' and B. Psychological Bulletin, 1972, 11, 375-376. Hodos, W. Nonparametric index of response bias for use in detection and recognition experiments. Psychological Bulletin, 1970, 11, 351-354. Hoffman, P. J. The paramorphic representation of human judgment. Psyghological Bulletin, 1960, S1, 116-131. 71 Hoffman, P. J. Cue-consistency and configurality in human judgment. In B. Kleinmuntz (Ed.), Formal representation of human judgment. New Yor : Wiley, 1968. Liverant, S., & Scodel, A. Internal and external control as determinants of decision making under conditions of risk. Psychological Reports, 1960, 1, 59-67. Mackworth, J. F., & Taylor, M. M. The d' measure of signal detectability in vigilance-like situations. Canadian Journal of Psychology, 1963, ll, 302-325. Markowitz, J., & Swets, J. A. Factors affecting the slope of empirical ROC curves: Comparison of binary and rating responses. Perception and Psychophysics, 1967, 3, 91-97. McGee, R. K. ReSponse style as a personality variable: By what criterion? Psychological Bulletin, 1962, SS, 284-295. Meehl, P. E. Clinical versus statistical predigtion. MinneapoIis: University of Minnesota Press, 1954. Nachmias, J. Effects of presentation probability and number of response alternatives on simple visual detection. Perception and PsychOphysics, 1968, S, 151-155. Naylor, J. C., Dudycha, A. L., & Schenck, E. A. An empirical comparison of pa and pm as indices of rater policy agreement. Educational and Psycho- logical Measurement, 1967, 27,—7-20. Naylor, J. C., & Schenck, E. A. PM as an "error-free" index of rater agreement. Educational and Psychological Measurement, 1966, ZS, 8I5-824. Naylor, J. C., & Wherry, R. J. The use of simulated stimuli and the "JAN" technique to capture and cluster the policies of raters. Educational and Psychological Measurement, 1965, 2§3T969-989. Newell, A. Judgment and its representation: An introduction. In B. Kleinmuntz (Ed.), Formal representation of human judgment. New YorE: Wiley, 1968. Nunnally, J. C. Psychometric theory. New York: McGraw- Hill, 1967. 72 Overall, J. E., 5 Klett, C. J. A lied multivariate analysis. New York: McGraw-Hill, 1972. Peterson, C. R., & Beach, L. R. Man as an intuitive statistician. Psychological Bulletin, 1967, 68, 29-46. Peterson, W. W., Birdsall, T. G., & Fox, W. C. The theory of signal detectability. Trans. IRE Professional Group on Information Theory, 1954, PGIT-4, 171-212. Pettigrew, T. F. The measurement and correlates of category width as a cognitive variable. Journal of Personality, 1958, 26, 532-544. Pollack, I., & Norman, D. A. A non-parametric analysis of recognition experiments. Psyghonomic Science, 1964, 1, 125-126. Price, R. H. Signal detection methods in personality and . perception. Psychological Bulletin, 1966, 66, i 55-620 Rappaport, M., & Hopkins, H. K. Signal detection and decision processes in schizophrenics and normal subjects. California Mental Health Research Digest, 1967, 5,7187:188. Rees, J. F., & Fishbein, H. D. Tests of the TSD model in human eyelid conditioning: A priori probability and payoff manipulations. Journal of Experimental Psychology, 1970, 83, 291-298. Richards, B. L., & Thornton, C. L. Quantitative methods‘ of calculating the d' of signal detection theory». Educational and Psychological Measurement, 1970, 29, 855-859. Rigby, F. D. Heuristic analysis of decision situations. In M. Shelley & G. Bryan (Eds.), Humanyjudgments and Optimality. New York: Wiley, 1964? Rim, Y., & Cohen, N. Personality variables in making a choice. Ps cholo ia: International Journal of Psychology in the Orient, 1968, 11, 191:197. Rowe, P. M. Individual differences in selection decisions. Journal of Applied Psychology, 1963, 41, 304-307. 73 Sawyer, J. Measurement and prediction, clinical and statistical. Psychological Bulletin, 1966, 66, 178-200. -’ Schmidt, F. L., & Marshall, R. L. Construction and use of-e a paramorphic representation of departmental policies in graduate admissions decision making. Mimeographed c0py. Schuck, J. R., Cross, H. A., & Mills, D. H. A signal detection analysis of the Rod and Frame Test. Perception and Psychophysics, 1970, 1, 276-280. Schulman, A. I., & Greenberg, G. Z. Operating charac- teristics and a priori probability of a signal. Perception and Psychophysics, 1970, 8, 317-320. Scodel, A., Ratoosh, P., & Minas, J. S. Some personalityx correlates of decision making under conditions of risk. Behavioral Science, 1959, 4, 19-28. Shelley, M. W., & Bryan, G. L. Human judgment and Optimality. New York: Wiley, 1964. Shepard, R. N. On subjectively optimum selections among multi-attribute alternatives. In M. Shelley & G. Bryan (Eds.), Human judgment and optimality. New York: Wiley, 1964. Sheridan, A. J., & Carlson, R. E. Decision-making in a performance appraisal situation. MimeOgraphed copy. Shipley, E. F. A signal detection theory analysis of a category judgment experiment. Perception and Psychophysics, 1970, Z. Siegel, A. I., & Pfeiffer, M. G. Predicting academic success through application of theory of signal detectability variables. Proceedin s of the 77th Annual Convention of the APA, I969, a, 145-146. Slovic, P., & Lichtenstein, S. Comparison of Bayesian and regression approaches to the study of information processing in judgment. Organizational Behavior and Human Performance, 1971, 6] 649-744. Strickland, B. R., & Rodwan, A. S. Relation of certain personality variables to decision-making in perception. Perceptual and Motor Skills, 1964, 18, 353-359. 74 Swets, J. A. Indices of signal detectability obtained with various psychophysical procedures. Journal of the Acoustical Society of America, 1959, 3;, 511-513. Swets, J. A. Detection theory and psychophysics: A review. Psychometrika, 1961, 32' 49-63. Swets, J. A., Markowitz, J., & Franzen, O. Vibrotactile signal detection. Perception and Psychophysics, 1969, s, 83-88. Swets, J. A., & Sewall, S. T. Invariances of signal detectability over stages of practice and levels of motivation. Journal of Experimental Psychology, 1963, 62, 120-126. Swets, J. A., Tanner, W. P., & Birdsall, T. G. Decision processes in perception. Psychological Review, 1961, is, 301-340. Tatsouka, M. M. Multivariate analysis techniques for educational and psyEHOIggical research. New York: Wiley, 1971. Theodor, L. H. A neglected parameter: Some comments on "A table for the calculation of d' and B." Psychological Bulletin, 1972, lg, 260-261. Ulehla, Z. J., Canges, L., & Wockwitz, F. Signal detectability theory applied to conceptual information. Psychonomic Science, 1967, s, 221-222. Van de Geer, J. P. Introduction to multivariate analysis for the social’sciences. San Francisco: Freeman, 1971. Weintraub, D. J., & Hake, H. W. Visual discrimination, an interpretation in terms of detectability theory. Journal of the Optical Society of America, 1962, E3, 1179-1184. Yntema, D. 8., & Torgenson, W. S. Man computer operation in decisions requiring common sense. IRE Trans. Human Factors Electronics, HFE-2, 1961, 20-26. MICHIGQN STRTE UNIV. LIBRQRIES ll Illl 9 312 3103915223