..In ‘ I.

413,3. "~ ‘3': I '%:l :4“, ufi’fi‘bu§# ‘5": “In:
J I '
‘ 19h“?
. 1}”
1

1
,9-

v,y, - 1

I,
II"\ 3‘ I" V‘I‘H‘Ej .‘u’: ’{it‘ \' I I 7' 1 ‘va‘ '1. “Ud‘h‘hv

.- .
Irv ,
Y

«‘55
'h’.

(k .
g.
.1 '.

my?!»
3;!" 5453 V

' ' I 3.‘ ‘
N 1;“,33': 1" #9 ( ku.‘ “f" U
liv '31:. '1’ _ ,~ 3‘ . . _ . r.~.-.-.\\.ux3g.- 2:133;

<.- 1

.|}I' in}; H;
(:94. "1:? 7:32;" 16......“ .‘
:mxtuxm' "i153 323? gfiu-y-t,

‘ _!

."HI'EI‘ \'
1H§

:Iu. .I-\".
v

>1?”
?III‘ 7'. 1': I'

"r
L“. . -’ an:
“3‘? .. v R"
29%? 99312;,“ Iiéfigk
L;.\<;::‘ . “NI 3 103
«2‘ a} ‘
7| I ' s"
. .- ” 2 I":~I , '~,
. W '3".:u‘v'.‘1 u ‘ . IL;;"‘ .
\Ir gs} :é‘ (v1'w Rfduilb‘fiyg‘v‘v"
' {XW‘ k . ‘7;ka
mu.“ “:1...

I \ ..

If!“ IV n- 11‘." :L-‘VII-l
:16". fix...
I .

l
' a"

“>5

- 0‘
"W Pi."
v

:”"vw",J:’-""{§(J.
{"{v} ‘~‘..

I
I
11".
1'11:." LIE}

"V
11~

m '73: v 2
\fi‘v‘g .3513; "I:

4.“, '5‘,
‘:0:" ['1,‘r

‘5‘

I ,9}

8,.
4-
‘-
n
.
o

.‘é

13}-
«MI

{=2

:1 I.

4
o

.v..—.W.

1

£5. 5 .

, « 7‘ 9...? u “PW-“*V’

 

I ‘ >1..|
"‘n‘v‘n'II'L' .n. I “It;

 

:uf'fl'i 8.}.

[a

IIIIIIIIIIII IIIIIIIIIII IIII IIIIIIIII

31293 00996 4101

 

This is to certify that the
thesis entitled
SITUATIONAL INFLUENCES ON NONLINEAR
INFORMATION PROCESSING
presented by

John R. Ogilvie

has been accepted towards fulfillment
of the requirements for

Eh. D. degree in PSYChOlogy

Major professor

Date /C}/J)/'/7f

0-7639

 

 

SITUATIONAL INFLUENCES 0N NONLINEAR
INFORMATION PROCESSING

By

John R. Ogiivie

A DISSERTATION

Submitted to
Michigan State University
in partial fu1fi11ment of the requirements
for the degree

DOCTOR OF PHILOSOPHY
Department of Psychology

1978

ABSTRACT

SITUATIONAL INFLUENCES 0N NONLINEAR
INFORMATION PROCESSING

By
John R. Ogi1vie

Previous research indicated that linear models provide a good
approximation of how information is used to make decisions in a wide
variety of judgment situations. In comparison, empirical support fer
nonlinear and curvilinear judgment strategies was rare. Einhorn (1970)
offered the conjunctive model, representing a multiple cut-off decision
strategy, as an alternative to the linear model. He demonstrated that
the conjunctive model represented a more appropriate decision strategy
when the cost of false positive decision errors was high.

This investigation was designed to systematically study effects
of the decision context on nonlinear information processing. In two
separate experiments, factors hypothesized to effect the cost of false
positive decision errors and use of the conjunctive model were varied.
Factors hypothesized to encourage use of nonlinear models were the
decision makers' experience with the judgment task, cue labels, high
decision cost instructions and loss of money for decision errors.
Information use was studied in a 2 (novice, expert judge) by 2 (cue
labels absent, present) by 2 (low, high decision cost) by 2 (linear,

conjunctive model) design in one study and a 2 (no pay, pay) by 2

John R. Ogilvie

(labels present absent) by 2 (low, high costs) by 2 (linear, conjunctive
model) in a second study.

Subjects in both studies were asked to rate the expected level
of job performance for 40 college graduate job applicants. The ratings
were based on six numerical sources of information presented to the
subjects on each applicant. In labelled conditions, sources were
identified with cue labels: dependability, experience, references, oral
communication, class standing, and intelligence.

Multiple correlations, reflecting how well a model accounts for
the subjects' use of information, of the cue values with success ratings
served as the dependent measure. For the conjunctive model, logarithmic
transformations of cues and ratings were performed before the computation
of the multiple correlations. The multiple correlations were transformed
to Fisher 25 and submitted to analyses of variance.

Consistent with previous research, the results indicated that
the linear model provided a better representation of subjects' judgment
in general. The conjunctive model equalled the ability of the linear to
account for judgments in the high cost condition of study one. In study
two, models performed equally in labelled, low cost conditions. The
discrepancy between study one and two in terms of decision cost were
attributed to increased ego involvement generated by modification of the
instructions in study two. Thus while not yielding conjunctive super-
iority, the experiments did demonstrate conditions in which the con-
junctive model was a more accurate representation of subjects' judgments.

The label and judge factors influenced information processing

overall. Labelled cues resulted in more consistent use of information,

John R. Ogilvie

supporting previous research. Expert judges were also more consistent
in their information use than novices. The judge effect was attributed
to greater involvement and familiarity with the task. Pay had no effect
on decision making.

Configural models were developed for each judge as an alternative
to linear and conjunctive models. In both studies, configural models
provided the best fit of information use for a majority of judges.

While the absence of any pattern in configural term usage may suggest

that configural terms were spurious, the magnitude of increments over

linear models demonstrated that some subjects may have engaged in some
form of nonlinear information processing.

It was suggested that future research should develop better
operationalizations of high decision cost to further test the con-
junctive model. Use of learning paradigms to provide additional knowledge
of learning rates for the conjunctive model in comparison to the linear

and other nonlinear models was also discussed.

TABLE OF CONTENTS

Page
LIST OF TABLES . . . . . . . . . . . . . . . . . iv
LIST OF FIGURES. . . . . . . . . . . . . . . . . v
Chapter
INTRODUCTION. l
Early Views of Cognitive Activity . 2
Basic Approach and Terminology. . 3
Recent Approaches to Information Processing: Two Paradigms. 6
Learning Paradigm . . . . . . . . . . 7
Policy-Capturing Paradigm. . . . . . . . . . 9
Learning Studies Hith Nonlinear Tasks. . . . . . . . . 12
A Rational Nonlinear Model . . . . . . . . . . . . 18
METHOD. . . . . . . . . . . . . . . . . . . . 27
Experiment I . . . . . . . . . . . . . . . . . 27
Experimental Design . . . . . . . . . . . . . . 27
Subjects. . . . . . . . . . . . . . . . . . 27
Decision Task. . . . . . . . . . . . . . . . 27
Procedure . . . . . . . . . . . . . . . . 3O
Dependent Variables . . . . . . . . . . . . . . 31
Data Analysis . . . . . . . . . . . . . . . . 32
Experiment II. . . . . . . . . . . . . . . . . 33
Experimental Design . . . . . . . . . . . . . . 33
Subjects. . . . . . . . . . . . . . . . . . 34
Decision Task. . . . . . . . . . . . . . . . 34
Procedure . . . . . . . . . . . . . . . . 34
Dependent Variables . . . . . . . . . . . . . . 35
Data Analysis . . . . . . . . . . . . . . . . 35
RESULTS AND DISCUSSION . . . . . . . . . . . . . . 37
Results: Experiment I . . . . . . . . . . . . . 37

ii

Chapter Page

Model Consistency Analyses . . . . . . . . . . . . 37
Perceived Manipulation Effects. . . . . . . . . . . 41
Configural Models . . . . . . . . . . . . . . . 43

Discussion: Experiment I . . . . . . . . . . . . . 47
Results: Experiment II . . . . . . . . . . . . . . 51

Model Consistency Analyses . . . . . . . . . . . . 51
Perceived Manipulation Effects. . . . . . . . . . . 55
Configural Models . . . . . . . . . . . . . . . 58
Discussion: Experiment II . . . . . . . . . . . . . 63
SUMMARY AND CONCLUSION. . . . . . . . . . . . . . . 66
LIST OF REFERENCES . . . . . . . . . . . . . . . . 71
APPENDICES
Appendix A. . . . . . . . . . . . . . . . . . . 75
Appendix B. . . . . . . . . . . . . . . . . . . 80

Appendix C. . . . . . . . . . . . . . . .. . . . 85

Table

4:500“)

LIST OF TABLES

Means. Standard Deviations and Intercorrelations Among
Cue Values . . . . . . . . . . .

Experiment 1: Means, Variances and Cell Sizes .

Experiment 1: Analysis of Variance Summary .

Experiment 1: Summary of Perceived Manipulation Effects.

Experiment I: Linear, Conjunctive and Configural Model
Multiple Correlations for all Judges

Experiment II: Means, Variances and Cell Sizes.
Experiment 11: Analysis of Variance Summary.

Experiment II: Linear, Conjunctive and Configural Model
Multiple Correlations for all Judges . .

iv

Page

29
38
39
42

44
53
54

59

LIST OF FIGURES

Figure Page
1. Experiment I: Decision Cost by Model Interaction . . . . 4O
2. Experiment II: Label by Decision Cost by Model Interaction. 56

3. Experiment II: Label by Model Interaction . . . . . . 57

INTRODUCTION

As Toffler (l970) pointed out, we live in a technological
civilization inundated with infbrmation. Adapting to this ever-changing
environment requires an ability to obtain information about certain
regularities in the environment and then to use the information to
respond appropriately. Information processing and the subsequent
judgments and decisions made are very important processes. These funda-
mental cognitive processes are essential to the survival of everyone.

The types of decisions required may range from selecting which loaf of
bread to buy at the local store to determining priorities for the funding
of alternate sources of energy. The cognitive activity of individuals in
our civilization has been credited with many significant accomplishments,
but also a number of dismal failures. Such a pervasive, and at times
significant activity, merits careful study. Stated more earnestly

It is only natural that an increasing number of thoughtful and

socially concerned individuals should be coming to believe that

one of the most urgent tasks for our society is the development

of a body of scientific knowledge that will enable us to under-

stand more of the capacities and limitations of human information

processing . . . (Estes, 1975, p. 2).

While decision making and other cognitive processes have recently
been the subject of increased scientific investigation, the study of

cognitive activity is by no means new. The following review discusses

some of the historical influences on modern cognitive psychology.

Early Views of Cognitive Activity

 

Wilhelm Wundt is generally credited with establishing the first
psychological laboratory in Leipzig in 1879. He studied conscious
experiences through the method of introspection. By systematically
reporting experienced sensations in a controlled manner he hoped to gain
insights into the mind. Together with his student, Edward W. Titchener,
they established the first school of thought in psychology, Structuralism,
which attempted to reduce immediate experience to elementary structural
components (Schultz, 1969).

The approach which supplanted structuralism as a theoretical
basis of early psychology was functionalism. One of the leading advocates
of functionalism. James R. Angell, defined their approach (more con-
sistently with modern cognitive psychology) as the study of mental
operations. They were not concerned with elements, but with processes
involved in mental activity. Angell (1907) viewed the role of the mind
as mediating between the environment and the needs of the organism.

The study of cognitive activity was temporarily suppressed by
Watson and the strict behavorists. Their emphasis on studying only
molecular stimulus-response relationships, which are directly observable,
did not allow the study of mental processes, except on an observable,
physiological level. Watson went so far as to suggest never using terms
like consciousness, mental states and the mind. Behaviorists aided
cognitive research through their objective research methods and by
stressing the role of the environment in influencing behavior.

It was a later behaviorist, E. C. Tolman, who began to soften
Watson's position and give some legitimacy again to the study of cogni-

tive processes. Tolman viewed behavior more on a molar level, i.e.,

the response of the whole organism, rather than glands or nerves.
Instead of simple S-R relationships, Tolman placed the organism in
between the stimulus and response to postulate S-O-R relationships. He
believed that a set of inferred, unobserved factors were the actual
determinants of behavior (Schultz, 1969). Tolman viewed the intervening
organism as a purposive, goal seeking creature. It again became acceptable
to study mental processes, but the intervening variables describing what
was going on within the individual, became acceptable in developing a
theory only when they were shown to be empirically related to both
experimental and behavioral variables. Thus, Tolman not only put cogni-
tion into his theory, but also set some rigorous guidelines for the
study of cognitive variables.

This brief review was intended to demonstrate the role that the
investigation of mental activity has played in psychology from the
science's infancy. Early psychologists gave mental operations a central
role in their theories, and later behaviorists established more rigorous
methods for the study of cognitive variables. Modern research on decision

making has its roots in these pioneering areas.

Basic Approach and Terminology,
Before examining current research and theory, a clarification of
terms is necessary. A number of different topics are included under
the aegis of cognitive psychology (e.g., memory, motivation, perception,
language acquisition, etc.). This paper restricts its study of cognitive
activity to the use of information by individuals in making decisions.
It will not focus on processing at a neural level but upon the grosser

processes employed by humans in integrating information in reaching

decisions. In this regard, the terms judgment, decision making and
information processing are used interchangeably.

A question might be raised concerning how the use of information
to make decisions can tell us anything about cognitive activity. Mental
processes are generally considered subjective, and as a private experi-
ence are not observable. Maintaining both a molar perspective on
behavior and an objective level of description requires some assumptions,
which have been discussed by Hoffman (1960). To describe and investigate
judgment, a set of techniques and a theoretical system must be interposed
between two sets of observable entities. In controlled settings where a
quantifiable input (information) and quantifiable output (judgment) are
known, functional relationships, described by mathematical models, can
be used to objectively infer the unobserved mental activity. There is
no certainty that any particular functional relationship is in fact being
used by the judge to make decisions, but to the extent that such a
relationship allows one to accurately predict responses given varying
informational inputs, the procedure serves as a simulation of how infor-
mation is used. This method proposes a means for describing Tolman's
intervening organismic variables, which Hoffman (1960) termed the para-
morphic representation of human judgment.

The functional relationships are by inference assumed to repre-
sent the process followed by each individual's cognitive system in making
decisions. Rappoport and Summers (1973) further explain.

A cognitive system is any minimally organized set of relationships
between an individual's judgments and the information ("cues“) on
which the judgments are based. In connection with judgment

phenomena, cognitive systems can be thought of as policies. That

is, to the extent that an individual finds meaning in a body of
uncertain information, he does so through application of an implicit

or explicit policy concerning (1) the causal relationships indi-
cated by the information, and (2) the relation of his outstanding
goals or purposes to that information. Policy can include any-
thing from the use of certain criteria by a graduate student
selection committee to a political leader's reliance upon recon-
naissance photographs (rather than statements by Soviet leaders)
when making a judgment about the presence of "offensive" missiles
in Cuba. Technically, a policy may be seen as a set of rules for
utilizing available evidence in order to reach a decision in an
uncertain situation (p. 4).

Using function relationships to represent the cognitive system or policy

of judges has been termed policy-capturing (Dawes & Corrigan, 1974).

Another aspect to consider in decision making is the environ-
mental system, or the situation in which the decision is being made.

As the cognitive system is comprised of individuals' perceived relation-
ships between input information and their judgments, the environmental
system or decision task includes relationships (actual, not perceived)
among environmental events and a criterion (if available). In the
previous graduate admissions example, the environmental system is com-
prised of relationships among available characteristics of the applicants,
plus the relationship between the characteristics and an eventual success
(graduate school or career) criterion (Rappoport & Summers, 1973).

In studying decision making, both cognitive and environmental
systems must be considered since certain cognitive systems may be more
functional in some environments than in others. A number of possible
interactions between cognitive and environmental systems could influence
information processing.

Rappoport and Summers (1973) delineated a number of different
types of cognitive-environment relationships with the distinguishing

feature among them being the number of separate cognitive systems (i.e.,

individuals) studied. Single systems focus on one judge making individual

decisions. Multi-judge systems have two or more judges interacting in
making their decisions. Bargaining situations or the three party
attraction paradigms of Heider (1958) are examples of multi-judge
system. The present study restricts itself to the single system type.
In this system, the person makes judgements individually on the basis of
information which is not completely reliable and valid. For example, a
radiologist has to categorize patients on the basis of somewhat ambiguous
symptoms. Even after decisions are made, confirmation of the correctness
of the decision may be difficult because of long time intervals or con-
founding factors present in ambiguous situations. Thus, the primary
focus of the single system research paradigm is upon the cognitive system
of the judges.

The specific methodologies and techniques employed in decision
making research will be discussed in the literature review which follows.

Recent Approaches to Information Processing:
Two Paradigms

 

 

As the earlier historical section indicated, psychologists have
been studying cognitive processes since the very beginning. However,
the recent molar approach to investigating the use of information in
making judgments began in the early 19605. Prior to that point, most of
the research was done on a molecular level of behavior analysis. Advances
in computer technology at that time further aided the development of this
quantitatively oriented area of study.

Modern research on human judgment has been reviewed by Slovic
and Lichtenstein (1971). In their monograph, they concluded that much
of the recent research can be categorized into two basic methodological

approaches. Each approach has certain tasks and types of input

information which are characteristic of that method. One approach was
based on the conditional probabilities of Bayes' Theorem. Although
Bayesian research is not the focus of this discussion, the interested
reader can consult the monograph for more information. The other approach
Slovic and Lichtenstein termed "regression" since the principle analysis
tools are multiple regression and its dummy coded derivative, analysis

of variance. The regression methodology has two variants: the learning

paradigm, and the policy-capturing paradigm.

Learning Paradigm,

 

Recent regression approaches to judgment research are based
heavily on the work of Egon Brunswik. His thinking also provides a link
between traditional schools of thought in psychology, as previously dis-
cussed, and current perspectives. Brunswik, a colleague of Tolman's for
a number of years, combined the behaviorists' stress on objectivity,
Tolman's emphasis on molar, purposive behavior and the earlier function-
alists approach of studying the adaptive interrelationship between the
organism and its environment. He further added the notion that the
environment is uncertain or probabilistic and that research designs
should be representative of that erraticism (Brunswik, 1955). Kenneth
Hammond (1966, p. 22) somewhat panegyrically described Brunswik as "the
first probability theorist in modern psychology as well as the first
psychologist to challenge the precepts of orthodox experimental design."

The concepts of cognitive and environmental systems discussed in
the previous section are derived from Brunswik's work. Although his own
research was concerned primarily with perception, his constructs and

methods have been widely applied (as will be indicated). He noted that

8

in an uncertain environment an individual decision maker should not
place a great deal of trust in the information in any one cue. To make
a more sound decision, multiple cues are accumulated and combined.

Thus, a general feature of judgment research is the presentation of
multiple information cues to serve as the basis for individual decisions.
Consistent with his notion of representative design, the decision
situation and relationship among the cues should be representative of
the settings in which the people typically decide.

By utilizing a paradigm which represented environmental as well
as individual factors, Brunswik was able to vary the type and magnitude
of relationship between information cues and some environmental criterion
(e.g., success in graduate school). In this way he represented the un-
certainty of information in the environment in different ways. The
correlation of each cue with the criterion was termed its'ecological
validity. Nonlinear cue-criterion relationships could also be represented
by using nonlinear (e.g., logarithmic or exponential) functions or con-
figural terms.

Over a series of trials, multiple cues are presented to subjects
and their judgments obtained. The correlation of the subjects' judg-
ments with the criteria values indicates the accuracy of the judgments.
For information on other judgment indices, such as accuracy and matching,
and their mathematical relationships, see Hursch, Hamnond, and Hursch
(1964) and Tucker (1964). Hammond and Summers (1972) have pointed out
that accuracy involves two steps: acquiring knowledge of the system
(i.e., environmental relationships) and then applying that knowledge in
consistently weighting the cues in making judgments. The Brunswikian

paradigm can be used to study the learning of environmental relationships

by presenting the criterion to the subject on each trial after they

have made their decision. By giving outcome feedback (i.e., the
criterion value) to subjects, the researcher can then examine over
trials if the subjects' use of information matched the environmental
relationship. The multiple correlation of cues with subjects' judgments
provides a separate index of how consistently the judge is applying the
knowledge. Through these studies, termed multiple cue probability
learning (MPCL) studies, inferences can be objectively made as to how

subjects learn cognitive tasks.

Poligy7Capturing Paradigm

While the MCPL model has generated much research, the somewhat
different approach of Hoffman (1960) has also had great impact. Hoffman's
paramorphic representations were developed on cliniciansfl judgments of
Minnesota Multiphasic Personality Inventory (MMPI) profiles. In this
and other decision situations (e.g., medical diagnosis) the environmental
criterion may not be known, immediately or for some time in the future.
Therefore, his research has focused on the cognitive response systems of
the judges. As such, Hoffman has placed more emphasis than Brunswik on
the models which account for the most consistency in the use of infbrma-
tion by judges in making decisions.

He proposed the use of linear and configural models. A linear
model describes the judgments in terms of a weighted sum of the informa-
tion values through multiple regression. The resulting multiple corre-
lation indicates how consistently the judge has used those weights with
that model. A configural model describes the judgments in terms of

weighted sums of the products (or interaction terms) of information

10

values. Use of interaction terms represents a strategy where the use
of one source is contingent upon several others, i.e., the values of
several sources are considered simultaneously. Each value is n93_
weighted and summed in isolation.

In his presentation, Hoffman (1960) noted that in some cases,
several models might represent judgment with equal proficiency. In those
cases, where no model is superior, another criterion must be employed.
Models having a rational or theoretical basis may then provide more
explanation than post hoc empirical models.

Following Hoffman's (1960) paradigm, many researchers have sought
to paramorphically represent the processes of decision makers. In these
studies informational cues (ranging in number from two to eleven) are
presented to subjects, and they are asked to make a judgment based on
the information given. Attempts have been made to capture and charac-
terize the cognitive processes of clinicians' MMPI profiles (Goldberg,
1968, 1971; Wiggins & Hoffman, 1968), faculty making graduate admission
decisions (Dawes, 1971; Einhorn, 1971), college students identifying job
preferences (Einhorn, 1971), judges deciding workmen's compensation
cases (Kort, 1968), radiologists diagnosing ulcers (Hoffman, Slovic &
Rorer, 1968), and bank loan officers evaluating an applicant company's
likelihood of future success (Libby, 1976).

Across these many and diverse studies, a linear model with
weights fitted by a multiple regression procedure has provided the best
model of the subjects' actual decisions. In an early review of the
literature, Hammond and Summers (1965) concluded that despite clinical

teaching and manuals emphasizing nonlinear use of information, linear

ll

models were sufficient to account for most of the variation in sub-
jects' judgments. More recently Goldberg (1971) has cogently argued
for the superiority of the linear model as representative of clinical
judgments by comparing linear and nonlinear models, using data from a
study by Meehl (1954). Green (1968) has attributed this "perverse
pervasiveness of linearity" to the ability of the least squared approxi-
mation method to provide a reasonable fit to the data in all situations
where the relationship between the cues and criterion is monotonic.
Dawes and Corrigan (1974) used data from several different decision
situations to demonstrate how linear methods provide good approximations
in situations where the informational cues have a conditionally monotonic
relationship with the criterion. Finally, Slovic and Lichtenstein (1971)
in their information processing monograph also recognized the strong
predictive ability of linear models.

In the face of this rather powerful evidence, nonlinear infer-
mation processing still seems more intuitively consistent with a view
of human decision makers as complex information integrators. The quest
for nonlinearity began with Meehl's (1954) article proposing that
clinical judgment has the advantage over the linear model in the ability
to employ nonlinear judgment policies. While not powerful, there is some
evidence that supports the hypothesis that people do use information in
decision making in a nonlinear fashion. Wiggins and Hoffman (1968)
found that 16 of 29 clinical judges did use information configurally.
In analyzing judicial decisions on workmen's compensation cases, Kort
(1968) found several multiplicative combinations (i.e., configural terms)
which yielded significantly greater predictive power than a linear model.

Hoffman, Slovic and Rorer (1968) used an analysis of variance procedure

12

to identify interactive use of cues which made small but significant
increments in prediction over a linear model. Valenzi and Andrews (1973)
found state employment placement interviewers made configural use of
information in evaluating the likelihood of job applicants being hired.
Also, Libby (1976) hypothesized that the superiority of actual decision
makers over their linear models in a boot strapping study was due to the
nonlinear utilization of some pieces of information.

While there are some studies which appear to document the non—
linear use of information by subjects on judgment tasks, the majority of
experimenters report little evidence that decision makers use information
in a nonlinear or configural manner. In most situations, the linear
model seems to provide the best fit for the data. Furthermore, a number
of the studies evidencing nonlinear use of information have relied on a
variety of statistical techniques to gain small, but significant, incre-
ments in multiple correlations. Such haphazard searches for configurality
may be merely manipulations of the "decimal dust" residuals of linear
models. One philosopher of science has stated,

. . the growth of the theories of science should not be considered
as a result of the collection, or accumulation, of observations; on
the contrary, the observations and their accumulation should be
considered as the result of the growth of scientific theories
(Popper, 1962, p. 2).

A study of fundamental cognitive processes should begin with psychological

theory or research, and then proceed to statistical procedures or models,

which operationalize the theory.

Learning Studies With Nonlinear Tasks
In attempts to understand on a more theoretical level the differ-

ences between linear and nonlinear information processing, several

13

investigators (Brehmer, 1969, 1971, 1974; Hammond & Summers, 1965, 1972)
have taken a different approach to the study of how different functions
are used in cognitive tasks. These studies have relied primarily on the
learning design of the Brunswikian paradigm, which allows the manipula-
tion of cue-criterion relationships and can provide feedback on correct
responses. If subjects are unable to learn nonlinear cognitive tasks,
then that finding would explain the poor performance of nonlinear models
of information use.

Several studies (Brehmer, 1969, 1973; Hammond & Summers, 1965)
have demonstrated that nonlinear cue-criterion relationships can be
learned. In one study Brehmer (1969) presented subjects with pairs of
lines ranging in length from 1 to 20 cm. They were told that the lines
had some relationship to a criterion line (ranging in length from 1 to
250 cm.) and that their task was to find that relationship. After
viewing the two cue lines, they estimated a criterion value and then
received the actual criterion value. Four different functions related
the cues to the criterion: (1) a linear function of the sum of the
values, (2) a linear function of the product of the cue values, (3) a
linear function of the ratio of the values, and (4) a linear function of
the absolute difference of the cue values. The functions varied in
linearity in that multiple correlations of the sum of the cues with the
criterion yielded values of 1.00, .90, .50, and 0.0, respectively. Over
400 trials the subjects were able to predict the criterion equally well
for all cue-criterion relationships, but the rate of learning was slower
for the nonlinear functions.

Hamnond and Sunmers (1965) also found that individuals were able

to learn to use information to predict criteria based on a nonlinear

14

rule. They presented two numerical cues to subjects and asked them to
predict a criterion value. One of the cues was linearly related to the
criterion and the other was based on the sine function. In addition,
one-third of the subjects were given no prior information about the task,
one-third were told that the task involved both linear and nonlinear
relationships with a high degree of accuracy requiring use of both types
.of information, and one-third were not only told of the relationship but
were given information as to which was the linear and nonlinear cue.

The results indicated that subjects were able to learn to use the cues

in a nonlinear (i.e., sinusoidal in this study) manner. Furthermore,
those subjects given more information learned the task better. Those
subjects given no information tended to use the nonlinear cue in a

linear manner (more than the other two groups did). These results seem
to indicate that in the absence of any information about the relationship
between the cues and criterion, a linear combination rule is used. How-
ever, when given information about relationships, people can use infor-
mation nonlinearly, indicating that the nature of the decision task may
be important to information processing.

In another study in which subjects were given information about
the task, Brehmer (1971) showed participants a graph of the relationship
between the cues and criterion, which they were allowed to study as long
as necessary. He employed a positive linear (PL), a negative linear
(NL), a symetrical U-shaped (U) where Y= (X-R)2, and a symmetrical
inverse U—shaped function (IU) where Y= A-(X-X)2. He used the same line
task that was used in his previously cited study. However, in this
study, subjects were not given the criterion values. Results indicated

that the PL and NL functions were more consistently used than the

15

nonlinear ones, with the best performance in the PL condition. In
comparing the PL function with the other conditions, it was found that
there was more irregular use of information in the other conditions.
The most inconsistent use of information was evidenced in the nonlinear
(IU,U) conditions. Thus, it appears that knowing the functional rule
relating cue to criterion is not sufficient to correctly use that rule.
In a subsequent study, Deane, Hammond and Summers (1972) more
explicitly examined the distinction between knowledge and application.
On a three cue task, half the subjects were given cues that were linearly
related to the criterion and half were given cues that were nonlinearly
related. After 20 warm-up trials, all subjects were given knowledge of
how the cues were related to the criterion and how they should be weighted.
This knowledge was presented in two different ways: verbally or pictori-
ally. After each judgment was made, subjects were given outcome feed-
back, i.e., told the correct response. Performance was poorer in the
nonlinear than linear conditions. (Correlations of subject responses
with correct answers were .58 and .84, respectively.). The researchers
also calculated an index of knowledge learned, which indicated how well
the subjects' weights matched those in the task. The knowledge indices
were very similar for both functions. Hence, differences in performance
must be attributed to greater difficulties in consistently applying more
complex (i.e., nonlinear) information combination rules. Hammond and
Summers (1972) termed this application of information, cognitive control.
Other research (Dudycha & Naylor, 1966; Schmitt, Coyle & King,
1976; Schmitt, 1978) had indicated that individuals have difficulty in
applying a linear combination rule as well as a nonlinear one. Their

analyses revealed that the subjects' weighting of information matched

16

the actual environmental weights, but they were unable to consistently
apply those weights. Thus, while cognitive control is not a problem
unique to nonlinear tasks, the research of Brehmer (1969, 1971) and
Deane, Hammond and Summers (1972) has demonstrated that individuals have
more cognitive control on linear tasks than on nonlinear ones.

To try and explain the difficulties in cognitive control with
functional relationships, Brehmer (1974) has studied differences in the
presence of certain functions in individuals' ecologies. His subjects
were shown PL, NL, IU, and U functions (from Brehmer, 1971) and were
given a verbal example of each. In one study, subjects were asked to
indicate the frequency with which these functions occurred and in another
to estimate how difficult they considered finding examples of those
functions in another study. PL function was the most frequent and easiest
to find with the other functions decreasing in frequency and ease in the
same order as above. In a third study he gave students lines and asked
them to assign numbers to the lines according to some functional rule.
After 20 trials, they were asked to illustrate the rule graphically.
They were then given a new set of 20 trials and asked to use a new rule
until they had produced 10 rules or could think of no more. The graphs
were classified into eight categories with the four most frequent in
descending order being PL, NL, IU, and U.

Brehmer (1974) then studied the learning rates of these four
functions without informing subjects of the nature of the relationship
between the cues and criterion. The results again yielded the same
ordering as before. In a fifth study, he used only the positive linear
and inverted U functions, but varied the levels of cue validity (.45 and

.90). He also asked half of the subjects to report which rule they used

17

after every trial. (By comparisons with control groups, reports were

not found to effect performance.) Both cue validity and function form
variables had a significant effect on information processing with high
validity and PL groups more accurately using the information to predict
the criterion. From self-reports, it was revealed that subjects in all
groups began by using the PL function. The frequency of PL rules remained
constant in the linear conditions and decreased slowly in the nonlinear
ones. When the cue validities were low, the nonlinear functions were

much harder to find. In fact, all subjects used a variety of rules in

the low validity condition, being unable to completely disgard any one.

In summarizing these five experiments, Brehmer (1974) concluded
that subjects have response hierarchies about functional relationships
which vary in strength. Linear functions (specifically positive linear
ones) were found to be most pre-potent in that hierarchy.’ When cue
validities were low, the pre-potent response could not be ruled out and
hence persisted in some form. On this basis, he concluded that the
structure of the data (i.e., environmental system) influences the
selection of hypotheses about relationships among the data.

In evaluating Brehmer's research, one critical point must be
noted. Not only is the structure of the data important in the selection
of information combination strategies, but also the nature of the
decision task. All of Brehmer's studies employed a task with lines of
various lengths as cues and criteria. Many of us were taught from early
childhood to pair the length of a line with a linear measure of that
length. Thus, this specific task rather than the subjects' ecology may
be responsible for the indicated response hierarchy. The research of

Hammond and associates (Hammond & Summers, 1965; Deane, Hammond & Summers,

18

1972) utilized a task which was somewhat more generalizable to real life
decision situations. Their results clearly indicate that individuals do
have more difficulty using nonlinear combination rules. This difficulty
may be due to response hierarchies favoring linear functions and/or to
the belief that nonlinear policies are more complex and therefore harder
to execute (Brehmer, 1973).

All previous studies, evidencing the use of nonlinear policies
by decision makers, have done so with the use of configural (i.e.,
interaction) terms. These terms were empirically derived with no g_
prigrj_basis for the prediction of specific terms. Thus, this support
for nonlinear models does not really make a theoretical contribution to
the area of decision making research (Popper, 1962). The MCPL studies
of nonlinear information processing all employed nonlinear functions
which had no rational (i.e., representative) basis. Those functions did
not purport to represent any practical, rational decision policy, which

may account for the perceived complexity and difficulty in using them.

A Rational Nonlinear Model

An approach, which provides a rational strategy for nonlinear
processing, has been developed by Einhorn (1970, 1971). He has proposed
several alternative nonlinear information strategies, which while more
mathematically complex than linear models, represent means of simplifying
the decision task. He termed them conjunctive and disjunctive strategies.
These models, representing specific strategies, may not be as hard to
apply as other nonlinear models (cf., Brehmer, 1973).

The conjunctive model represents a non-compensatory information

combination strategy. The linear strategy is compensatory, i.e., by

19

using a weighted sum, low values on some cues can be compensated for by
high values on others. The non-compensatory or multiple cut-off approach
is derived from selection research, where there are situations in which
a low value on one ability cannot be compensated for by another. For
example, poor vision for airplane pilot applicants cannot be overcome by
high intelligence. Minimal levels of vision are necessary. In selection,
minimal cut-off levels are established on critical predictors and indi-
viduals who do not exceed the cut off are not hired. In a broader con-
text, a judge using the conjunctive model would assign low criteria
values if any cue values are low.

This model is described by a parabolic function. Mathematically

it is operationalized by the equation.

log10 Y = bi log1O (Xi)' (l)
where Y = the subject's criterion estimate;
Xi = the information sources or cues;
b. = the weighting factors for the cues

1
This negatively accelerating curve has the greatest change in slope in

the lower cue value ranges. With logarithmic transformations, the
greatest difference in values occurs between 0 and l, decreasing there-
after. Hence, discrimination is to be maximized at the low values. The
logarithmic transformation then allows for the weights to be fitted by
multiple regression and the resulting multiple correlations can be then
compared with that from the untransformed linear model.

The disjunctive model represents a very different type of decision
strategy. A judge using this combination policy should estimate the

criterion based on the highest single cue value. This strategy is

20

described by a hyperbolic model. The positively accelerating curve of
this model has the greatest change in slope at upper levels of cue values,
and hence differences there have a greater impact on criterion estimation
than at lower cue values. (No mathematical model is presented since this
model will not be used in the current investigation.)

These models simulate simpler cognitive strategies because a
judge need not sum or average all cue values. With the disjunctive model,
one high cue value can serve as the basis of a decision. For the con-
junctive model, one or more low values would suggest a low criterion
estimate, irrespective of the remaining cues. Much earlier, Simon (1955)
suggested that complex decision tasks could be simplified by a cut-off
strategy.

While not stating that these models are most representative of
human judgment, Einhor (1971) has hypothesized specific circumstances in
which these models might be theoretically more appropriate than a linear
combination. Situational factors which might encourage the use of these
models were the amount of information presented and the cost of a false
positive decision. These factors are characteristics of the decision
task.

When large amounts of information are present, a decision maker
may have difficulty summing or averaging all cue values to make a
judgment. Without a mechanical aid, processing large amounts of infor-
mation in a linear manner requires the ability to conceptualize and
weight each of the values in reaching a decision. As stated above, the
conjunctive and disjunctive models allow the decision maker to reduce the
amount of information by basing a decision primarily on a limited number

of information sources.

21

However, empirical support for the greater use of nonlinear
models in high information situations has not been obtained. Einhorn
(1971) reported two studies in support of his theoretical models.
However, in neither study was there greater use of nonlinear models as
the amount of information increased. In one study, graduating M. B.A.s
and industrial engineering students chose the jobs they most preferred.
In this study, the conjunctive model outperformed the linear, regardless
of the amount of information. The correlations of actual versus predicted
decisions, i.e., accuracy, were also higher in conditions where subjects
were given two cues, then when they were given four or six. In the
second study he reported, faculty and graduate students made decisions
about applicants for graduate school. For this task the linear model
performed equally as well (i.e., in terms of accuracy) as the conjunctive
across information levels. Judgments in the two, four, and six cue
information conditions were also significantly different, being less
accurate as the amount of information increased. The increase in infor-
mation did not effect the performance of the models in either study. The
absence of time pressures or constraints in both studies may have limited
the potential effect of the conjunctive model to reduce information.

More recently, Ogilvie and Schmitt (1978) did obtain changes in
model performance as the amount of information increased. Their experi-
mental task required introductory psychology students to rate the likeli-
hood of success for a series of individuals, applying for the job of
purchasing agent. The linear model multiple correlations of the cues
with judgments were clearly superior to the other models in the two cue
condition. In the four and eight cue conditions, the linear and con-

junctive models' values were not different, and the multiple correlations

22

for both were lower than in the two cue situation. The authors con-
cluded that a simple averaging of all cues appeared to be more difficult
in the higher information conditions. Even this finding provides only
minimal support for the hypothesis that nonlinear models reduce the amount
of information and simplify the decision task since the conjunctive model
was not found to be superior in higher information quantity conditions.

The performance of the disjunctive model in a number of studies
(Einhorn, 1971; Goldberg, 1971; Ogi1vie and Schmitt, 1978) has been
uniformly low and therefore will not be given further consideration.

The relatively even performance of the linear and conjunctive
models in the Ogilvie and Schmitt study may in part reflect individual
differences in the ability of decision makers to process information at
higher levels. All judges appear to be able to use all information at
low (two cues) information levels. When four or more cues were presented,
subject differences in ability to simultaneously process information may
have resulted in some individuals resorting to a linear strategy while
others utilized a conjunctive one. Thus, neither model was superior to
the other. Valenzi and Andrews (1973) and Dobmeyer (1970) found con-
siderable individual differences in the weighting of information (both
linearly and configurally) by selection interviewers.

The amount of information presented is a characteristic of the
situation, but the ability to process information may vary with indi-
viduals. In his well-known article, Miller (1956) suggested that span
of immediate memory, limits of absolute judgment and a number of other
upper limits for cognitive factors might be seven units (plus or minus 2).
However, he noted that by heuristics or encoding processes persons can

significantly increase their short term memory. Thus, amount of

23

information each subject can handle may vary widely. Only a few college
students may have to utilize information reduction models to process
eight cues. Another reason for the apparent low usage of the nonlinear
models may have been the subjects' ignorance of how those strategies
reduce the amount of information. A decision task variable which is
less sensitive to individual differences than the amount of information
may provide greater support for the use of the conjunctive strategy in
decision making.

A second factor, which was hypothesized by Einhorn (1971) to
facilitate the use of the conjunctive model, is the cost of a false
positive decision. The multiple cut-off approach was designed to protect
against that very type of decision error. Where the cost of false
positive decision is high, the decision maker searches for negative
information (i.e., low cue values), comparing values with minimal cut-
off levels to reach a final decision. In Einhorn's job preference
study, the cost of choosing a job, which would eventually be unsatis-
factory, was assumed to be rather high for subjects. In this situation
the conjunctive model was found to be superior to the linear. Here,
individual differences in subjects' ability to process information were
not apparent as the conjunctive model provided the best fit for subjects'
responses across all information quantity conditions. In comparison,
Einhorn's (1971) graduate admissions task, where the cost of a false
position decision had less personal relevance for the subjects, resulted
in relatively equal performance of conjunctive and linear models.
Einhorn (1971) further suggested a number of task-related factors which

might increase the cost of a false positive decision: involvement of the

24

decision maker, amount of payoff, and familiarity of the decision maker
with the task.

Increasing the payoff for the decision makers can increase their
involvement in the task. Giving subjects a sum of money and telling them
that they will lose a fixed amount for each error should further increase
the cost of a false positive decision for them, resulting in greater use
of the conjunctive model.

Familiarity is virtually synonomous with experience. All of the
studies previously cited in support of nonlinear models (Wiggins & Hoffman,
1968; Kort, 1968; Hoffman, Slovic & Rorer, 1968; Valenzi & Andrews, 1973;
and Libby, 1976) used experienced decision makers in the typical context
in which they make job-related decisions.

Another study (Miller, 1971) found differences between statis-
ticians and clinicians in their ability to use information in making
academic performance predictions. The statisticians relied more on the
numerical relationships among cues and criterion while clinicians relied
more on verbal labels. Although Miller's study demonstrates that the
type of judge studied can make a difference, the decisions made were not
typically job relevant, i.e., judges could not be categorized as experts
on academic prediction tasks. The removal of labels would most likely
alter the context considerably, so that the judges would be more prone
to use a linear combination instead.

No comparison has been made of information used by experienced
and novice decision makers on the same task. Without any previous
exposure, decision makers may be more likely to average the information
presented (Brehmer, 1974). Specific configural combinations and the use

of nonlinear models would be more likely to occur after the impact of

25

earlier decisions are known. As previously indicated, Hammond and
Summers (1965) found that subjects, presumably naive to the decision
task, were able to combine information nonlinearly when given outcome
feedback and instructions about the nature of the cue criterion relation-
ships. Experienced decision makers get feedback for decisions made on
the job and hence may develop nonlinear policies.

Since nonlinear models and the use of specific configural cue
combinations have been demonstrated in contexts familiar to the decision
makers, the labels given the cues are assumed to be an integral factor
in decision making. In these situations would the absence of any cue
labels still result in nonlinear combinations? Cue labels would seem to
be a vital aspect of Brunswik's (1955) concept of representative design.
In an MCPL study, Miller (1971) found that judges used information less
consistently when cue labels were absent than when the cue labels were
present. The lowest consistency indices occurred where the cues were
unlabelled and the values were incongruous (i.e., unlikely negative
intercorrelations). Miller did not examine the effects of labels on
nonlinear processing.

A number of situational characteristics have been advanced as
encouraging the use of nonlinear information combination models. Just
as certain features of the decision task can bias information use in
favor of nonlinear models, some task characteristics can induce a linear
response bias. The Goldberg (1968, 1971) and Dawes and Corrigan (1973)
studies, which provide major support to the linear superiority argument,
primarily utilized two tasks: a rating based on MMPI profiles and a
prediction of performance in graduate school. These two tasks have been

used in a number of studies, with many using the same overworked Meehl

26

(1954) data set. These tasks may contain an inherent linear bias which
precludes nonlinearity. A wider variety of tasks should be employed in
studying models of decision making to minimize task specific results.

In review, evidence from a number of sources has indicated that
the linear model is not the universal model (or perversely pervasive one
as Green (1968) termed it) for all human information processing. It may
perform well in a number of general situations, but evidence also indi-
cates that nonlinear models may be more appropriate in a number of speci-
fic situations.

Therefore, the present research was designed to examine decision
making in situations which have been hypothesized to favor the use of
nonlinear models, especially the conjunctive model. The factors to be
manipulated to create the desired circumstances are (l) the type of judge
(novice and expert), (2) cue labels (present and absent), (3) the cost of
false positive decisions (high and low cost instructions), and (4) the
payoff to judges (monetary payment and nonpayment). Factors 1-3 were
systematically varied in one experiment, and factors 2-4 were varied in
a second study. It was hypothesized that nonlinear models would be used
more frequently by expert judges than novices and that the nonlinear use
of cues would be more frequent when labels are present. Likewise, high
cost instructions and monetary payment would also result in greater use

of nonlinear models.

METHOD

Experiment I

 

Experimental Design

The experimental design consisted of a 2 (judge type: novice,
expert) by 2 (cue labels: present, absent) by 2 (cost of false positive
decision instructions: high, low) by 2 (decision models: linear, con-
junctive) fixed effects design with repeated measures on the last

factor.

Subjects

Ninety-two persons served as subjects in this experiment with
at least 10 in each cell. Forty-eight of the subjects, the novice
condition, consisted of students in Introductory Psychology classes at
Michigan State University. For their participation, they received
extra credit towards their final grade.

The additional 44 subjects, those in the expert condition, con-
sisted of recruitment interviewers at the Placement Center at Michigan
State University. Interviewers' participation was voluntary. All
subjects were randomly assigned to experimental conditions with the

exception of the subject variable. type of judge.

Decision Task

 

Subjects were asked to rate the expected job performance level

of 55 job applicants on a ten-point scale, ranging from performs "most

27

28

duties inadequately" to performs "at the very best in this occupation."
The first 15 judgments served as practice trials. The job to be filled
was described as "one requiring a reasonable amount of responsibility
and contact with people." It was also stated that a bachelor's degree
was "typically required" for job entry.

Subjects were presented with six types of numerical information
about the applicants. Each source of information was on a scale from
one to ten. The values of the cues were assigned by the experimenter so
as to suggest nonlinear combination of cues. This was done by assigning
one or more cues a low value of one or two. In many cue sets there were
also one or more high values, i.e., values of seven or larger. Attempts
were made to evenly distribute the high and low values across all infor-
mation sources, so not to concentrate the low values in specific cues
or cue combinations. Even distribution of these values did result in
some negative cue intercorrelations as can be seen in Table l, where the
cue intercorrelations are presented.

As seen in Table 1, however, only one negative cue intercorre-
lation was significant, that between experience and dependability.

While in most job situations those two variables are generally unrelated,
that particular negative correlation did not prove problematic for the
subjects. Although there are many negative intercorrelations, the
magnitudes of interrelationship were slight enough so that no subject
reported noticing any irregular aspects in the cue sets. The intended
positive (but non-significant) correlation between class standing and
intelligence was obtained. Data also presented in Table 1 revealed

that the means and standard deviations were approximately equal for all

29

mo. v a»;

.mcowu_u:ou amp—mamp cw vow: «cm:
mew—a Pmugo~_co; mgu co m—mnm_ m_w;z covuwucou um_mnmp:: mg» com ummz mum: mpmnm_ _mupucm>a .muoz

 

 

 

 

m_.m m~.N m_.~ m_.~ mo.~ N—.N meowumP>mo

vcmucmum

No.m mo.m om.m oo.m ¢~.m o—.m mama:

mucmm_—pwu:_ m:_u=mum :o_umu_::asou mmocmcmmmm mu:m_cmaxu Auw__nmccmamo mpmnmo mzu
mmm_u Paco

o muczom m moczom a mocaom m moczom N mucsom _ muczom

oo._ m—N. mmo.- em_.1 mm_.- mmP.- o woczom

oo._ oo_.- mm_.- mop.1 N—P.- m mucaom

oo.~ _m_.- wkmmm.- 0pm.- e mucaom

oo._ mm_.- mo_.- m mucaom

oo.p “Pp. N muczom

oo.F a~ «venom

 

111.. .I'IIITI...II.I‘II.
1) C]... III 1 -11: 'IIIII‘II‘I.‘ 1.-.-.l ‘.l..| ‘I.11I.I.III1II.IIIIIO I-

 

 

.mm:_m> mzu acoe< mcowumpmccoucmucH can meowum_>mo ecmucmum .mcmwz .p m_amp

3O

cue types. All cue values and the job performance scale are presented

in Appendix A.

Procedure

A booklet containing instructions, 15 practice applicants, 40
job applicants, and a feedback sheet with manipulation check was given
to the subjects. Subjects were asked to read the instructions and then
to evaluate the practice candidates first to clarify any questions they
might have had on the task. Evaluations were based on the level of job
performance the applicants could be expected to attain if they were
hired. Subjects were also told that an average applicant would generally
have a level of between five and six on each source. After they had an
opportunity to ask questions, subjects were then directed to evaluate
the 40 applicants. All subjects were instructed to use their judgment,
avoid mathematical calculations, and be as accurate as possible.

Instructions were used to manipulate the cue labels and cost of
a false positive decision factor. The instructions were shuffled
together so distribution of the booklets to subjects would result in
random assignment.

For the cue labels factor, half of the subjects were given a

 

description of each of the six cue labels (e.g., dependability, oral
communication), and an explanation of how the values were arrived at.
These cue labels were printed above the columns in which those cue

values appeared. The other half of the subjects received no source
description, and cue headings were not labelled. Unlabelled instructions
explained that researchers were interested in how information was used,

independently of the source.

31

The two levels of the cost of a false positive decision factor
were similarly manipulated through the instructions. The introduction
of the low cost instructions indicated that the information did not
describe actual college students and that their responses would be used
only to study their use of information. This sentence was not present
in the high cost instructions. High cost instructions contained an
additional paragraph explaining the problems organizations are having
with false positive decisions and warning subjects that their "primary
focus" should be to avoid making those kinds of decisions.

After evaluating all applicants, subjects were asked to describe
how they used the information to reach their decisions. They were also
asked to respond to a series of questions on a five point scale. These
questions asked for perceived confidence in decisions, ability at making
these decisions, degree of concern about those errors, and overall guality
of applicants. When finished, subjects were debriefed and dismissed.

Full instructions for each condition are reproduced in Appendix B.

Dependent Variables

Multiple correlations describing the relationship between the
six informationsources and subjects' judgments served as indices of
model performance. Multiple correlations obtained from the 40 job
applicant trials represented the degree to which a linear model
”captures" the judge's rating policy. More specifically, the correla-
tion indicates how consistently the information was weighted and summed
in making judgments. This measure is analogous to the consistency
index in the usual MCPL study (Dudycha & Naylor, 1966; Schmitt, Coyle &
King, 1976).

32

To assess the degree to which subjects' judgments fit the con-
junctive model, nonlinear transformations of the cues and criterion were
performed (Einhorn, 1970, 1971). Logarithmic transformations of the cue
values and subjects' responses were computed. Then the transformed cue
values were regressed on the subjects' responses. Mathematically, the
conjunctive model is operationalized by Equation 1. The resulting
multiple correlations reflect how consistently subjects used the con-
junctive model or multiple cut-off strategy in making their job per-

formance estimates.

Data Analysis

The multiple correlations from each model were then transformed
to Fisher 25 to become the primary dependent measure. The Fisher 2
values were submitted to a 2 (Judge type - J) by 2 (Labels -L) by 2
(Cost of false positive decision -C) by 2 (Decision Models -DM) fixed
effects univariate analysis of variance with repeated measures on the
last factor.

The responses to the five questions on the feedback page were
submitted to a multivariate analysis of variance to assess the perceived
effects of experimental manipulations on the subjects.

To further explore the possibility that subjects may have used
alternative decision models, all possible two-way and three-way inter-
action configural terms were added to each judge's regression equation
to see if they would improve predictability. Configural models were
developed by first multiplying all possible information cue combinations,
taking two and three cues at a time. This procedure resulted in 15

two-cue products and 20 three-cue products. A hierarchical regresssion

33

procedure was then used to test if configural terms were being used by
judges in making decisions. The regression procedure employed forced
inclusion to enter the six linear terms into the regression equation
first. A stepwise inclusion procedure brought in significant two-way
interaction terms and then any significant three-way interaction terms.
In this manner significant terms represented increases in the
predictability of a subject's decisions beyond that afforded by the
simple linear model. Two-term products were included first to keep
explanations as parsimonious as possible. Any three cue products had to
make unique increments beyond those of any significant two-way terms.
Criteria for significant predictors were set at fairly rigorous levels
to try to avoid the inclusion of statistically significant predictors
which had no practical significance. Configural predictors were required
to have significant (p_ < .05) F-ratio to enter the regression equation
and in addition had to increase the squared multiple correlation by at
least .02. The latter criterion is equal to eta squared, representing
at least a 2 percent incremental contribution to the predictable vari-

ance in the judges' decision policies.

Experiment II

 

Experimental Design

The experimental design consisted of a 2 (labels: present,
absent) by 2 (cost of false positive decision: high, low) by 2 (payment:
$2.00, $0.00) by 2 (decision models: linear, conjunctive) fixed effects

design with repeated measures on the last factor.

34

Subjects

Eighty-nine persons originally served as subjects with at least
10 in every cell. Thirty-two subjects were males (four per cell) and the
remainder females. All subjects come from Introductory Psychology
classes at Michigan State University and received extra credit for their
participation. Forty-seven of these subjects also received $2.00 as a

feature of the payment condition.

Decision Task

 

Subjects performed the same expected job performance ratings of
55 job applicants as did the subjects in Experiment I. The same cue

values were also utilized.

Procedure

As in Experiment I, subjects were given a booklet of materials
and asked to read the instructions. They were told to evaluate the 15
practice applicants, ask questions, and then rate the remaining 40 appli-
cants.

All experimental factors were manipulated via instructions with
all instruction conditions shuffled for random assignment. The lepel§_
factor was manipulated as in Experiment I. The cost of a false positive
decision factor was manipulated as in Experiment I for the low cost
condition, but the following two sentences were added to the high cost
instructions:

Your decisions will be evaluated against those of expert inter-
viewers and personnel administrators. Past experience on this
task has demonstrated that that those who perform well on the

task are good evaluators of human behavior and would make good
interviewers.

35

The addition was made to further increase the involvement and perceived
cost of false positive decisions since the manipulation in Experiment I
was felt to be somewhat weak. In the nonpayment condition of the payment
factor, subjects received no additional instructions. The subjects in
the paid condition were told that they had an initial sum of $2.00 but
would lose St for each incorrect decision made in order to increase the
cost of a false positive decision.

After completing all forms, the subjects were debriefed and dis-
missed. In addition, subjects in the payment condition were given two
dollars, regardless of their performance on the task. The debriefing
procedure for paid subjects consisted of an explanation that there were
no right or wrong answers, but that the experimenters were interested in
how they used information under conditions in which their decisions might

result in financial loss.

Dependent Variables

 

As in Experiment I, the dependent variables in this study were
the multiple correlations between the subjects' judgments and the infor-
mation cues from both linear and conjunctive models. Subject responses

to feedback questions were also treated as dependent measures.

Data Analysis

 

The multiple correlations from each model were first transformed
to Fisher Zs. The transformed values were then submitted to a 2 (Payment
- P) by 2 (Labels - L) by 2 (Cost of false positive decision - C) by 2
(Decision Models - 0M) fixed effects univariate analysis of variance with

repeated measures on the last factor.

36

The responses to the five questions on the feedback page were
submitted to a multivariate analysis of variance to assess the perceived
effects of experimental manipulations on subjects.

Configural models were also developed for each decision maker to
examine the possibilities that alternative decision strategies were used
by judges. As in Experiment I, configural models were developed by
adding two-way and three-way interaction terms to the linear model by
means of step-wise regression procedures. Similar criteria of a signi-
ficant [:ratio and a 2 percent incremental contribution to the variance

were established for configural terms.

RESULTS AND DISCUSSION

Results: Experiment I

Model Consistency Analyses

 

The means and variances of the Fisher Zs from each cell of the
design are presented in Table 2. One subject was deleted from the high
cost instruction, unlabelled cues, expert judge condition because he had
indicated in his self-report that he had summed the cue values (in
violation of the instructions) to reach his judgments. A test for
homogeneity of variance on the resulting data demonstrated that the cell

variances were relatively homogeneous, F = 3.55, n.s.

-max
The results of the analysis of variance, presented in Table 3,
indicated that the linear model, 2 = 1.1466, more consistently accounted
for the subjects' use of information than did the conjunctive model,
2D= 1.0623. However, the main effect for models was moderated by the
interaction between cost of a false positive decision and models, which
is graphically described in Figure 1. Simple effects tests of this
interaction revealed that the multiple correlations for the linear model
were significantly greater than those for the conjunctive model only
when experimental instructions described the cost of a false positive
decision as being low, 5 (1,83) = 19.798, p_< .001. With high cost
instructions, (HC), the models did not differ, [_(l,83) = 1.761, n.s.

Conversely, the significant interaction also demonstrated that the

37

38

Table 2. Experiment I: Means, Variances and Cell Sizes.

 

Low Cost Instructions

 

 

 

 

 

 

 

 

 

 

 

Unlabelled Labelled
Linear Conjunctive Linear Conjunctive

n* 12 12 11 ll
Novice X l.0817** .9213 1.2677 1.1612

52 .0498** .0429 .0595 .0600

n 10 10 11 11
Expert 1 1.1405 .9858 1.3418 1.2369

52 .0848 .0575 .0554 .0481

High Cost Instructions
Unlabelled Labelled
Linear Conjunctive Linear Conjunctive

n 11 ll l4 l4
Novice X .9079 .8649 1.1255 1.0899

s2 .0832 .0450 .0783 .0575

n 11 11 ll 11
Expert R 1.1479 1.1619 1.1598 1.1765

s2 .1411 .0917 .0398 .0519

*n = number of subjects per cell.

é = cell mean
s = cell variance

**Fisher Z values

39

Table 3. Experiment 1: Analysis of Variance Summary.

 

C.

 

Source df SS F
_Judge.(J) 1 .4871 4.237*
Label (L) 1 1.4763 12.841**
Decision Cost (C) l .2563 2.229
J x L 1 .0827 .719
J x C 1 .0571 .497
L x C l .0674 .586
J x L x C 1 .0827 .719
Subjects 83 9.5427

(within groups)
Model (M) 1 .3208 16.657**
J x M l .0002 .010
L x M 1 .0322 1.673
C x M 1 .1011 5.248*
J x L x M 1 .0059 .305
J x C x M 1 .0000 .000
L x C x M 1 .0000 .001
J x L x C x M
M X Subjects 83 1.5987

(within groups)

Total 181 14.0947

 

Note. *p <.05

**p <. 001

40

 

 

1.3 J
1.2 4
:2
h!
c
:‘U
o
:z
1.1 -
Linear
1.0 a
I”.
T 1 .
Low Cost High Cost
Decision Cost Factor
Figure 1.

Experiment I: Decision Cost by Model Interaction.

41

linear model accounted for information use less consistently in the
high cost condition than the low cost (LC) condition, E_(l,83) = 5.097,
.p < .05. Instructions did not effect the use of the conjunctive model,
.5 (1,83) = .255.

In addition to cost of a false positive decision, other situa-
tional variables were found to influence information processing. Signi-
ficant main effects were obtained for both label and judge factors.
Subjects given labelled cues, Zr = 1.1949, were significantly more con-
sistent in using the cue information to make job predictions than were
subjects given-unlabelled information, Zr = 1.0140, and expert judges,
Zr = 1.1564, made decisions more consistent with cue information than

did novice judges, Zr = 1.0525. These factors did not interact with

each other or with the model factor as predicted.

Perceived Manipulation Effects

To evaluate subject perceptions' of the experimental manipula-
tions, their responses to the questions on the feedback sheet (see
Appendix A) were analyzed. The significant manipulation effects are
presented in Table 4.

The most perceived differences occurred between types of judges.
Expert judges felt they were significantly better at making job selection
decisions than novices, and also felt more concerned about making false
positive decision errors than did introductory psychology students.
Novice judges rated the overall quality of the applicants higher than
their experienced counter parts, but an interaction between judges and
decision cost moderated that effect with novices rating applicant

quality higher only in the low cost condition. Within the interaction,

42

Table 4. Experiment I: Summary of Perceived Manipulation Effects.

-.

 

Factor n Variable Means Univariate F
Judges (J) How Good* 21.465
96 Novice 3.17
86 Expert 3.81
Concern 49.965
96 Novice 3.37
86 Expert 4.19
Quality 24.560
96 Novice 3.16
86 Expert 4.19
Labels CL) Confidence 6.005**
88 Unlabelled 3.22
94 Labelled 3.54
Decision Cost (C) Concern 14.204
88 Low 3.97
94 High 3.54
J x C Quality 10.101
461 Novice,Low 3.35
50 Novice,High 2.98
42 Expert,Low 2.57
44 Expert,High 2.80

 

Note. * How good are you at making selection decisions?

** p< .05, all other E's p< .Ol.

43

there were no differences in mean ratings for higher decision costs or
expert judges. Thus, high decision cost instructions significantly
reduced the novice judges' perception of the quality of the applicants
they reviewed, but instructions had no effect on experts' quality
ratings.

Differences in perceptions were found between subjects given
labelled and unlabelled cues. Those judges making decisions with
labelled information cues expressed more confidence in their decisions
than those deciding on the basis of unlabelled data only.

The expected differences between levels of the decision cost
factbr were not obtained on perceptions of how costly false positive
decision errors would be for the organization in the study. Thus, the
decision cost instructions did not appear to have any measurable effects
on perceptions of error cost. Unexpectedly, significant differences
were obtained on the concern variable with low decision cost subjects
being more concerned than high decision cost judges. This outcome was

the opposite of what was expected and was somewhat counter-intuitive.

Configural Models

 

As an alternative to linear and conjunctive models, configural
models were developed for each judge. Configural models included the
six cues (as in the linear model) plus any two-way and three-way inter-
action products which had significant §_~ ratios to enter the regression
equation and added 2 percent additional variance to the prediction of
job success estimates.

A summary of the configural models along with a comparison of

linear and conjunctive models for each judge is listed in Table 5.

44

Table 5. Experiment I: Linear, Conjunctive and Configural Model
Multiple Correlations for all Judges.

4

 

Subject Linear Conjunctive Configural AR2*
Novice, Low Cost, Unlabelled
l .870 .846 .890 .034
2 .894 ' .811 .944 .092
3 .834 .737 .879 .077
4 .865 .797 .883 .031
5 .709 .600 .752 .062
6 .728 .783 .764 .055
7 .783 .691 .820 .060
8 .777 .728 .824 .074
9 .804 .622 - .833 .048
10 .584 .522 .646 .076
11 .703 .622 .789 .128
12 .813 .797 .848 .058
Novice, High Cost, Unlabelled
l .690 .730 .735 .065
2 .850 .776 .926 .135
3 .857 . .769 **
4 .846 .817 .888 .073
5 .480 .554 **
6 .703 .703 .781 .116
7 .394 .445 .598 .203
8 .635 .685 .709 .110
9 .663 .537 **
10 .732 .730 .828 .150
11 .788 .776 **
Novice, Low Cost, Labelled
1 .904 .785 .935 .057
2 .805 .784 .832 .045
3 .866 .848 **
4 .823 .742 .871 .082
5 .915 .854 .934 .035
6 .762 .702 **
7 .849 .890 .872 .039
8 .895 .902 .934 .071
9 .854 .861 .885 .054
10 .924 .879 .937 .024
11 .617 .602 **
Novice, High Cost, Labelled
l .774 .729 .818 .069
2 .743 .674 .779 .056
3 .791 .627 ' .848 .093

45

Table 5. (Cont'd.).

 

 

'5-”

 

Subject Linear Conjunctive Configural AR2*
Novice, High Cost, Labelled (cont'd.)
4 .813 .796 .846 .056
5 .886 .830 .953 .123
6 .858 .822 .895 .065
7 .807 .795 .844 .061
8 .945 .029 **
9 .905 .831 .920 .027
10 .709 .593 .780 .105
11 .912 .890 **
12 .865 .847 .888 .040
13 .808 .762 .843 .059
14 .720 .784 .858 .217
Expert, Low Cost,Unlabelled
1 .746 .660 .782 .056
2 .914 .901 **
3 .850 .779 .888 .066
4 .681 .702 **
5 .553 .545 .667 .140
6 .888 .817 .922 .062
7 .860 .819 .893 .059
8 .729 .589 .776 .071
9 .871 .745 .913 .074
10 .840 .808 .948 .194
Expert, High Cost, Unlabelled
l .860 .840 .940 .144
2 .856 .827 **
3 .785 .712 .845 .098
4 .777 .789 .878 .168
5 .414 .436 .630 .226
6 .753 .700 **
7 .870 .785 .892 .039
8 .665 .659 .727 .086
9 .763 .826 **
10 .950 .919 **
11 .918 .881 .932 .027
12 .980 .895 **
Expert, Low Cost, Labelled
1 .785 .678 .894 .182
2 .946 .922 **
3 .841 .799 .863 .038
4 .813 .832 .865 .087
5 .926 .911 .941 .028
6 .824 .831 .854 .050

46

Table 5. (Con’t.d).

—. *o—na— -- _ - ---_—_-.-- s

 

 

._—.—.—.—__. a— -. ...._
-_..-_————— ..—. -_ —._n.... - -. .— ------

 

 

Subject Linear Conjunctive Configural AR *
Expert, Low Cost, Labelled (cont'd.)
7 .888 .850 **
8 .837 .792 .860 .039
9 .917 .891 **
10 .877 .836 .913 .066
11 .830 .834 .857 .046
Expert, High Cost, Labelled
1 .913 .849 **
2 .812 .804 .840 .046
3 .857 .889 .910 .094
4 .768 .786 .844 .124
5 .770 .788 .820 .080
6 .753 .776 **
7 .779 .757 .818 .063
8 .859 .937 .957 .179
9 .808 .837 .843 .058
10 .890 .832 **
11 .723 .713 .763 .060

 

Note. * 0R2 2 Increment in R2 obtained by adding configural terms to
the Linear model.

** No configural terms meeting specified criteria.

47

Abundant use of configural models was evidenced in all cells of the
experimental design. Overall, configural terms meeting the above
criteria were found for 74 percent of the judges (68 of 92). With a
more rigorous criterion of a 5 percent increment in the variance in
judgment policy accounted for, 56.5 percent (52 of 92) of the judges
were still found to utilize configural terms. The mean increment in R2
for all 68 configural judges were .08363. Configural terms accounted
for as much as 22 percent additional variance for one judge.

Thus, considerable use of configurality was evidenced by judges
in all categories. The significant linear and configural terms used by
each judge are presented in Appendix C. An examination of the use of
specific configural terms revealed no clear pattern of cue combination.
The most frequently used term was a product of oral communication and
intelligence. Five of 21 experts (24%), given labelled dues, relied on

that configural term, to make their success estimates.

Discussion: Experiment I

The results of this study did not support the hypotheses that
situational factors would lead to increased use of the conjunctive model
of information processing. The interaction of decision cost with models
was due to the decreased ability of the linear model to account for
information use in the high cost condition, and not the improved per-
formance of the conjunctive. Thus, the theoretical and empirical
statements of Einhorn (1971) were not supported. The reduced ability
of the linear model in the high cost condition was similar to the effect
found by Ogilvie and Schmitt (1978) for the amount of information

variable. Rather than increasing the use of the conjunctive model, the

48

situational variables amount of information and false positive decision
cost have resulted in less consistent use of an averaging or additive
model.

A challenge of Einhorn's research is appropriate to the extent
that similar variables were tested. Einhorn (1971) used a decision task
presumed to have a high false positive cost f0r decision makers. The
present study attempted to vary levels of decision cost via experimental
instructions to determine if information use varied across levels. The
analyses of perceived decision cost effects yielded inconsistent support
for the instruction manipulations. The decision cost variable did not
affect cost perceptions, yet information processing was influenced by
that variable. In reviewing a number of studies, Nisbett and Wilson
(1977) have concluded that subjects have difficulty reporting those
factors which affect their responses on tasks involving cognitive
activity. Therefore, the decision cost manipulation may have achieved
its intended effect despite the failure of subjects to report different
levels of perceived cost.

Further support for the manipulation can be obtained from the
interaction between judges and cost on the quality variable. In the HC
condition, both novice and expert judges rated the quality of applicants
at similar levels. All cues were constructed to have one or more low
cue values; therefore, the information presented about applicants was
somewhat below that of the typical college graduate applicant. Based
on their experience, expert judges were evidently more aware of the low
values since they rated the overall quality of the applicants lower.

In support of the manipulations, the high cost instructions may have

49

made the unexperienced judges more cognizant of the low values since
they rated the applicants similarly to the experts in this condition.

The higher concern perceptions in the LC condition are more
puzzling. That result may be artifactual and unrelated to any experi-
mental manipulations. However, the results for concern about making
decision errors may have a rational basis. Concern self-reports were
obtained after the task was completed. HC judges were asked to specifi-
cally avoid making decision errors. Having concentrated on that task
and followed instructions, they may have been less concerned that they
had made those errors. LC judges, who were asked to report their con-
cern for making decision errors after completing a task may then have
become more concerned since they may have been casual while making
judgments. The reversal could thus represent a type of "Hawthorne
effect."

While the type of judge and label variables did not differentially
affect model performance as hypothesized, those variables did influence
overall information processing in a predictable manner. The type of
judge effect confirmed and extended the research of Miller (1971), who
found that statisticians and clinicians used infbrmation differently.
The present study attempted to vary the background and experience of
judges on the same continuum. The experienced Placement Center inter-
viewers used the cue information more consistently than the relatively
inexperienced introductory psychology students on a decision task
relevant to the experience differential.

Several explanations can account for the greater consistency by
expert judges. Greater familiarity with the task may have increased

attentiveness to numerical cues. Perhaps, specific training or practice

50

making selection decisions enabled experienced interviewers to use
information more consistently than novices. On the job they may receive
some form of feedback about the outcome of their decisions which helps
them to attend to background data carefully. The interviewers also may
have viewed the task as more realistic and relevant than introductory
psychology students would view them.

The cue label finding confirmed another aspect of Miller's (1971)
research, i.e., that the removal of cue labels reduces subjects' ability
to consistently use information. This study did not find an interaction
between judges and labels as Miller had, but the diversity and orienta-
tion of his subjects was greater. The subjects in this study could not
be simply dichotomized into verbal and mathematical strategists as could
Miller's clinicians and statisticians.

The reported low confidence ratings for unlabelled conditions
may provide a theoretical explanation of the cue label effect when
viewed relative to Brehmer's (1974) work with ecological hierarchies of
functional relationships. He found that in uncertain environments,
i.e., low cue-criterion correlations, subjects tried a variety of
infbrmation utilization rules. The uncertainty generated by unlabelled
cues may cause judges to try a variety of decision rules, resulting in
lowered consistency values for any one model. The labels may also lead
to the development of heuristics or encoding processes as described by
Miller (1965), which enabled decision makers to use cue information in
a more consistent fashion.

The addition of configural terms resulted in significant
improvements in ability to account for information use. Even with

strict criteria for the inclusion of configural terms, over half of the

51

judges could be classified as configural. This finding challenges the
assertion of Goldberg (1971) that linear models (or perhaps methods) will
generally provide as good a fit as any model in most decision situations.
Dawes and Corrigan (1974) have also argued that the characteristics

of many decision making situations favor linearity. At least this study
has provided one context where linearity appears not to be the heavy
favorite.

To summarize, the results of Experiment I have shown that
situational factors do influence the way in which information is used
to make judgments although those factors did not result in greater use
of the conjunctive model. Furthermore, the abundance of configurality
demonstrated that alternative models can exceed the ability of the
linear model to "capture" judges' policies. However, the absence of any
pattern in use of specific configural terms limits interpretability.
While the specific configural terms may be somewhat spurious, evidence
for some type of nonlinear processing was f0und.

In attempts to provide greater insight into the use of nonlinear
models by decision makers in a somewhat different situational context,
attention will now focus on the results of Experiment II, which used
pay to increase the cost of false positive decision errors and studied

novice judges in all conditions.
Results: Experiment II

Model Consistency Analyses

 

Initial computation of cell means and variances revealed some-

what large discrepancies in cell variances, Emax (16.13) = 11.273, p.<

52

.05. The departure from the homogeneity of variance assumption plus
unequal cell frequencies would bias resulting E_tests (Winer, 1971).
Since equal cell frequencies make this test relatively robust against
homogeneity violations, nine judges were randomly deleted from cells
resulting in ten observations per cell. Descriptive statistics on the
resulting data are presented in Table 6. The cell variances were not
significantly different within this data set, Emax (16,9) = 7.205.

An analysis of variance treating sex of subject as an independent
variable found no differences between males and females on either the
dependent variable or any of the manipulation checks. Therefore, the
data were combined for subsequent analyses.

The analysis of variance results, combined across sex of subject,
are presented in Table 7. The highly significant model effect was
moderated by two-way and three-way interactions. The multiple correla-
tions from the linear model (2 = 1.1934) were significantly larger than
the multiple correlations from the conjunctive model (2 = 1.0793).
However, the interaction of models with labels and decision costs
revealed that the linear model was not always superior.

Simple effects tests of the interaction between labels and
models at decision cost levels yielded a significant two-way interaction
for low decision costs, §_(l,72) = 9.593, p_< .01, and a nonsignificant
interaction in the HC condition, E_(1,72) = .014. Simple main effects
tests of the model factor at both levels of the label factor within LC
further revealed that the linear model multiple correlations were
significantly greater than the conjunctive only in the unlabelled con-
dition, §_(l,72) = 15.361, p < .001. Models did not differ signifi-

cantly in the labelled, low cost condition, f_(l,72) = .676. In

53

Table 6. Experiment 11: Means, Variances and Cell Sizes.

 

Low Cost Instructions

 

 

 

 

 

 

 

 

 

Unlabelled Labelled
Linear Conjunctive Linear Conjunctive

n* 10 10 10 10
Unpaid X: l.279** .9979 1.1301 1.1297

52 .0521 .0151 .0448 .0558

n 10 10 10 10
Paid Y" 1.0360 1.1546 1.2165 1.1420

s2 .0583 .0555 .0931 .0787

High Cost Instructions
Unlabelled Labelled
Linear Conjunctive Linear Conjunctive

n 10 10 10 10
Unpaid Y. 1.0270 .8982 1.3182 1.1875

s2 .0971 .0455 .1295 .1150

n 10 10 10 10
Paid 1' 1.2212 1.1147 1.2434 1.1283

52 .0481 .0291 _.0298 .1059

*p_= number of subjects per cell

X = cell means

52 = cell variances

**Fisher Z values

54

 

 

 

 

'Table 7. Experiment II: Analysis of Variance Summary.

Source df 55 F
Pay (P) 1 .1879 1.486
Label (L) 1 .4101 3.243
Decision Cost (C) 1 .0057 .045
P x L 1 .2395 1.894
P x C I .0000 .000
L x C 1 .1116 .883
P x L x C 1 .1383 1.093
Subjects 72 9.1054

(within groups)
Model (M) 1 .5208 50.258**
P x M l .0111 1.067
L x M 1 .0460 4.441*
C x M 1 .0015 .147
P x L x M 1 .0002 .016
P X C x M l .0273 2.633
L x C x M 1 .0535 5.165*
P X L x C x M 1‘ .0005 .052
M X Subjects 72 .7461
(within groups)
Total 159 11.6053
Ngtg. *p < .05

**p < .001

55

addition, the changes in model performance with labelled and unlabelled
cues were nonsignificant for both linear and conjunctive models.
Figure 2 graphically describes these statistical findings.

Analysis of the simple main effects for labels and models in the
HC condition indicated that the linear model yielded higher multiple
correlations for both unlabelled, E_(l,72) = l3.35l, p_< .Ol, and
labelled cues, f_(l,72) = l4.594, p_< .Ol. Differences between labels
at both levels of model were nonSignificant. As depicted in Figure 2,
the between model values in HC are different from each other, but the
apparent rise in slope from unlabelled to labelled was nonsignificant
for both models.

Analysis of the two-way interaction of labels with models yielded
an outcome similar to the three-way interaction (see Figure 3). Simple
effects tests for differences between models demonstrated that the
linear model was significantly better than the conjunctive model at
"capturing" subjects' decision policies in both unlabelled, E (1.72) =
42.284, p_< .OOl, and labelled conditions, E_(l,72) = 12.4ll, p_< .00l.
The combination of significant interaction and significant simple
effects at both model levels then meant that the difference between
models was significantly greater in the unlabelled condition than the
difference in the labelled one. Labelling cues resulted in significant
improvements for the conjunctive model, E_(l,72) = 5.34l, E.< .025, but
not the linear model, F (l,72) = 1.326.

Perceived Manipulation Effects

 

Analyses of the subjects' self-reports of experimental manipula-

tion effects yielded no differences on the decision cost variable. 0n '

56

 

 

Decision Cost Factor
Low Decision Cost High Decision Cost
1.3 4
Linear
m 1.2 s
N
10:: Conjunc-
. tive
Conjunc-
tive
l. 1 fi
1.0 1
,L
Unlabelled Labelled Unlabelled Labelled
Cue Labels Factor
Figure 2.

Experiment II: Label by Decision Cost by Model Interaction.

57

 

 

1.3 .1
Linear
1.2 4
:x
N
§ Conjunctive
2:
1.1 A
1.0 1
Ila-fl
’r"
r 1
Unlabelled Labelled

Cue Labels Factor

Figure 3. Experiment II: Label by Model Interaction.

58

the label variable, subjects using labelled cues (Y'= 3.58) were again
more confident than those using unlabelled cues (X'= 3.05). Subjects in
labelled conditions (X'= 3.60) expressed more concern about making false
positive decision errors than did subjects in unlabelled conditions
(Y’= 2.90). The pay variable yielded the expected difference in per-
ceived cost with paid subjects (Y'= 3.80) viewing decision errors as
more costly than unpaid subjects (X'= 3.28). However, paid judges (Y'=
3.l0) were less confident than their unpaid counterparts (Y'= 3.53).
Thus, telling subjects that they would lose 5¢ for each incorrect
decision did serve to increase the cost of false positive decision
errors, but unexpectedly lowered their confidence in their judgments.
Decision cost instructions did not yield any measured differences in

perceptions for this study.

Configural Models

 

As in Experiment I, configural models were developed for each
judge by adding two and three-way interaction terms to the regression
equation. The same criteria of a significant F to enter the equation
and a 2 percent increment in variance accounted for were employed.

The multiple correlations for the linear, conjunctive and con-
figural models are presented in Table 8. Once again a large number of
judges' models met the statistical criteria for configurality. Seventy-
four percent (66 of 89) of the judges regression equations included
configural terms. With a more rigorous statistical criterion of 5 per-
cent incremental variance, 64 percent (57 of 89) of the judges could
still be classified as configural. The average increment which con-

figural terms made for all 66 configural judges was a substantial

59

Table 8. Experiment II: Linear, Conjunctive and Configural Model
Multiple Correlations for all Judges.

“_

 

 

Subject Linear Conjunctive Configural AR2*
Unpaid, Low Cost, Unlabelled
1 .845 .809 .872 .046
2 .859 .796 .928 .123
3 .870 .827 .942 .130
4 .744 .743 .814 .109
5 .797 .716 .851 .089
6 .813 .755 .926 .198
7 .828 .786 **
8 .705 .673 .812 .162
9 .609 .683 .704 .126
10 .863 .762 **
Unpaid, High Cost, Unlabelled
1 .842 .786 **
2 .562 .513 .642 .096
3 .652 .645 **
4 .825 .734 **
S .890 .837 .925 .063
6 .409 .542 .624 .223
7 .782 .655 **
8 .809 .709 .886 .131
9 .859 .790 **
10 .813 .798
Unpaid, Low Cost, Labelled
l .736 .791 .813 .120
2 .697 .751 **
3 .852 .869 .910 .104
4 .910 .918 **
5 .706 .648 **
6 .829 .733 .882 .090
7 .831 .780 .894 .109
8 .768 .800 .799 .049
9 .842 .784 .869 .047
10 .794 .735 .857 .104
11 .881 .904 .937. .102
Unpaid, High Cost, Labelled
1 .892 .859 .927 .063
2 .927 .868 .947 .036
3 .700 .656 .795 .142
4 .850 .794 **
5 .845 .795 .927 .146
6 .758 .743 .877 .196
7 .527 .565 .862 .465
8 .875 .771 .938 .114

60

 

 

Table 8. (Cont'd.).
Subject Linear Conjunctive Configural AR2*
Unpaid, High Cost, Labelled (cont'd.)
9 .920 .920 .949 .055
10 .892 .859 .923 .056
11 .945 .043 .956 .022
Paid, Low Cost, Unlabelled
1 .639 .595 **
2 .689 .661 .853 .253
3 .929 .896 .942 .024
4 .839 .766 .866 .045
5 .922 .851 **
6 .860 .625 .884 .042
7 .924 .836 .936 .023
8 .879 .821 .895 .029
9 .813 .774 .889 .129
10 .886 .800 .929 .078
11 .874 .815 .889 .028
Paid, High Cost, Unlabelled
1 .918 .880 **
2 .864 .802 .934 .127
3 .883 .864 .901 .032
4 .879 .772 .923 .080
5 .774 .829 .879 .175
6 .801 .808 **
7 .782 .727 .814 .052
8 .877 .841 .914 .066
9 .685 .673 .745 .085
10 .899 .848 .923 .043
11 .829 .786 .903 .128
Paid, Low Cost, Labelled
l .817 .753 .929 .196
2 .850 .820 .913 .112
3 .950 .946 **
4 .813 .768 .845 .053
5 .779 .738 .818 .063
6 .767 .756 .915 .249
7 .644 .670 .874 .349
8 .780 .764 .880 .166
9 .858 .866 .939 .145
10 .834 .783 .910 .133
11 .981 .978 **
12 .920 .869

61

Table 8. (Cont'd.).

 

- .- “..
_fla-‘—-._-.__ -..——.- ‘-,-. .—-._...-——c_.—- ‘fi. -._-..—-.—-..-

 

 

 

Subject Linear Conjunctive Configural AR2*
Paid, High Cost, Paid
l .891 .882 .917 .047
2 .873 .841 .907 .060
3 .771 .785 .835 .103
4 .877 .920 .921 .079
5 .832 .752 .912 .145
6 .882 .884 .943 .112
7 .895 .872 **
8 .717 .644 .823 .163
9 .798 .724 .862 .107
10 .857 .756 **
11 .804 .690 .845 .068
12 .849 .744 .899 .088
13 .896 .934 .930 .058 l

 

7
Note. * AR2 3 Increment in R‘ obtained by adding configural terms to

the Linear Model.
** No configural terms meeting specified criteria.

62

configural terms made for all 66 configural judges was a substantial
ll percent with the greatest increment for any one judge being 47 percent.

The least number of configural judges per cell, four of ten,
were found in the unpaid, high cost, unlabelled condition. When the
inclusion criterion was raised from .02 percent to .05 percent incre-
ments, six of nine judges in the paid, low cost, unlabelled condition
lost their configural designation, indicating that the effects of those
configural terms were slight. With the more rigorous criterion, l4 of
32 overall linear judges were in these two unlabelled cue conditions and
2l of 32 linear judges were in all the unlabelled conditions. A chi—
square test of the distribution of linear judges (using a .05 criterion)
across labelled conditions was marginally significant (x2 = 3.l25, p.<
.l0) with more linear judges in the unlabelled conditions. Other chi-
square tests of the distribution of linear judges across labelled and
paid conditions were non-significant (p > .lO) for incremental criteria
of .02 and .05.

An examination of the use of specific configural terms in
Appendix C did not indicate any dominant pattern of usage. A wide range
of cue combinations were used in all cells, both labelled and unlabelled.
A slight trend was present in the paid, labelled conditions among three-
way interaction terms. Four of the l6 judges with higher interactive
terms evaluated the combination of experience, references and oral
communication simultaneously when making decisions. Greater use of
higher order interactive tenns was found in this study with 45 instances
of three-way term use as opposed to 24 in Experiment I. As such, there

is some evidence that complex decision processes were being used.

63

Discussion: Experiment 11

Overall, the linear model more consistently accounted for the
subjects' use of information in making judgments than did the conjunctive
model. The superiority was not evident in the labelled, low decision
cost condition. Thus, in the LC condition the policy of judges using
labelled cues was captured with equal proficiency by both linear and
conjunctive models. Although the rise in the performance of the con-
junctive model with labelled cues was not statistically significant, it
did account for enough additional variance so that it did not differ
from the linear. In the two-way interaction involving labels and models,
the effect was attributable to a significant rise in conjunctive model
performance with labelled cues while performance was constant for the
linear. Thus, the hypothesis that cue labels would result in greater
use of the conjunctive model was supported.

Cue labels added sufficient information to the decision context
so that information was used more consistently with the conjunctive
model in making judgments. Miller (l97l) had suggested that cue labels
represented additional amounts of information in decision making studies.
The nature of the information provided by the cues, i.e., specific cue
labels, may have facilitated the use of a conjunctive strategy. An
alternative explanation is that the conjunctive model was used to reduce
the amount of information which was increased by adding cue labels.

However in the HC condition, the superiority of the linear model
was maintained in both labelled and unlabelled conditions. HC instruc—
tions had been hypothesized to lead to greater nonlinear use of infor-
mation. As before, the manipulation checks did not reveal any differ-

ences on the five feedback items between LC and HC. This outcome was

64

even more unexpected since the cost manipulation was further strengthened
in this study to increase the ego involvement of subjects. As in
Experiment I, the absence of any perceived effects can be explained by
the Nisbett and Wilson (1977) argument that subjects are unaware of the
factors influencing their use of information on cognitive tasks. This
explanation becomes more plausible here since relevant model effects
were found in both studies while subjects were unable to report any
perceived cost differences.

The consistent model differences in the HC condition could be
attributed to the instructions which told subjects that their score
would be compared against experts and that those performing the task
well would make good selection interviewers. Presumed to increase ego
involvement, these instructions served to uniformly increase performance
for both models. The intended ego involvement most likely also created
greater task involvement and attentiveness to the cues which resulted
in constant differences between models.

In this study the threat of financial loss for making decision
errors had no effect on the overall use of information and did not lead
to increased use of the conjunctive model as predicted. The analyses
of the feedback items helped explain this finding. Subjects receiving
pay, Y'= 3.80, did feel that the cost of making false positive decisions
was greater than unpaid subjects, X’= 3.275, indicating the pay factor
achieved the intended manipulation result. However, unpaid subjects,
Y'= 3.525, were more confident in their decisions than those paid, Y'=
3.l0. Interpreting the lack of confidence as uncertainty about which
strategy to use then leads to similar explanations as were employed for

labels in Experiment I. As Brehmer (l974) suggested, it may be that

65

uncertainty leads to the application of a variety of decision strategies
thoughout the task, lowering the consistency of each individual model.
Any effect that pay had on increasing conjunctive use may have been
offset by the uncertainty. Thus, pay may have had some unintended
affects on subjects and may not have provided the best test of the effect
of decision cost on information processing.

As in Experiment I, the development of configural models revealed
that the majority of decision makers were using information in a complex,
configural manner. The somewhat greater use of configural terms in
labelled cue conditions in this study implies that configural use may
have been meaningful. However, the absence of any consistent use of
specific terms, especially with labelled cues, muddles interpretation of
the configural effects. The empirical nature of configural models often
results in a wide range of model differences, some of which are meaning-
ful and some which are merely statistical artifacts. Without patterns
of specific term use, one can not state definitely that the configural
terms represent actual decision processes. Nonetheless, these models
have provided a clear indication that something more than mere summing
or averaging of cue values occurred in this decision context. Goldberg
(l97l) criticized Einhorn's (l97l) work with the conjunctive model on
the basis that he did not employ alternative models as a control. If
configural models are viewed as a control, then neither the linear nor
the conjunctive model may have been used by decision makers in their
pure form. More complex decision policies may have been employed by

judges to estimate job success.

SUMMARY AND CONCLUSION

This investigation was designed to study the effects of aspects
of the decision context on nonlinear information processing. Previous
research (Goldberg, l97l; Slovic & Lichtenstein, l97l; Dawes & Corrigan,
l974) had suggested that a linear model would provide a good approxi-
mation of how information was used to make decisions in many situations.
Empirical support for nonlinear and curvilinear models had been infre-
quent and limited. Einhorn (1970, l97l) proposed a nonlinear alterna-
tive, the conjunctive model, which he hypothesized would represent a
more appropriate decision strategy where the cost of false positive
decision errors were high.

In two separate experiments, this investigation systematically
varied aspects of the decision context which were hypothesized to affect
the cost of false positive decision errors and the use of the conjunctive
model. The factors predicted to encourage nonlinear information use
were experienced decision makers, cue labels, instructions noting the
cost of decision errors, and the loss of money for incorrect decisions.
The first three factors were varied in the one study while the last
three were varied in a second. The effect of these factors on the
ability of the linear and conjunctive models to account for the sub-
jects' use of information in making job success estimates of job appli-

cants was studied. As an alternative strategy. configural models were

66

67

also developed for each judge. The effects of these variables and their
implications for information processing are discussed below.

The type and experience of the decision maker affected the use
of information. Placement Center interviewers used the information more
consistently in reaching their decisions than did introductory psychology
students. This result was supportive of previous research and was
attributed to the greater familiarity, involvement and training of the
experienced judges on the decision task. Future researchers should be
aware of the background and experiences of judges when selecting a
decision tasks and in generalizing from their results.

4 Experienced judges did not evidence greater use of nonlinear
models; linear models were used more consistently. The decision making
experience of the expert judges may have been primarily linear in
nature, and so they followed their predominant response tendencies.
Thus, they were not as likely to respond to situational manipulations
designed to facilitate nonlinear use. Further study of the different
types of decision makers could benefit by measuring and attempting to
control the type and amount of prior experience that judges possessed.

Although the threat of financial loss served to increase per-
ceptions of the cost of decision errors, the pay variable had no effect
on information processing. Paid subjects also felt less confident with
their decisions. The uncertainty associated with low confidence was
believed to have minimized any potential pay effect. The anxiety and
involvement which can accompany conditions of financial loss, especially
with college subjects, may render pay a poor means of operationalizing

decision cost.

68

Cue labels were found to have an overall effect of increasing
the consistency of information use and were also found to result in
greater use of the conjunctive model, supporting one hypothesis and
previous research. By providing heuristics or a basis for encoding cue
information, the labels facilitated more consistent use of information.
The labels may have either provided a specific context for conjunctive
use or may have generated enough additional information so that the
conjunctive model was needed to reduce the amount. Further work should
be done with specific labels and the order of labels utilized.

The final situational factor manipulated was cost of decision
errors. In one study high decision costs reduced the performance of the
linear model so it did not differ from the conjunctive. In the second
study the high cost manipulation was made more ego involving. Here, high
costs resulted in linear superiority while in the low cost condition the
conjunctive and linear models accounted for information use similarly
when cues were labelled. The greater ego involvement may have accounted
for the difference between studies.

However, the subjects perceived few, if any, differences between
high and low cost situations. While other research has shown that
subjects are often unaware of factors influencing their cognitive judg-
ment, the investigator did not feel that decision cost was adequately
tested with the instructions employed. This conclusion is especially
disconcerting since Einhorn (l97l) found the most convincing support
for the conjunctive model on a task presumed to have high cost outcomes
for decision makers. A more desirable research design is to vary levels
of cost within a single experiment as was done here, but the means of

operationalizing that variable were inadequate. Thus, the most promising

69

variable for future research with the conjunctive model remains false
positive decision costs. In those situations, searches for negative
infonnation would occur, which should be best “captured" by the con-
junctive model. A selection interview task may provide the needed
support for the conjunctive model since reviews have indicated that
interviewers search for negative information (Schmitt, l976).

The poorer overall performance of the conjunctive model leads
one to conclude that few people use a conjunctive strategy in combining
information to make judgments. The low usage of nonlinear models has
been attributed to their complex nature in the past (Brehmer, I973).
Yet the conjunctive model is purported to represent a more simple
cognitive strategy. Furthermore, Brehmer (l974) has demonstrated that
linear models were the predominant means of describing cue-criterion
relationships. Judges may not have used the conjunctive strategy
because they were not aware of the ability of a multiple cut-off approach
to minimize decision errors. Therefore before disgarding the conjunctive
model as useless, further research should be done using learning
paradigms, comparing rates of learning and conjunctive relationships
with other nonlinear relationships and linear ones. In this manner,
both acquisition of knowledge of the conjunctive rules as well as the
application of that rule can be studied.

While the conjunctive model performed less well than expected,
considerable nonlinear use of information was indicated from the con-
figural models. A great many judges were found to use configural terms
(including novice judges). If the configural models are only used as
controls, they nonetheless indicate that complex information processing

strategies were being used by judges. In fact, neither the conjunctive

70

nor the linear model were probably being used in their pure forms.
Instead, more complex models perhaps sequential or combination models,
may have been used. The use of configural models in this study supports
Goldberg (l97l) in urging that control models be employed to provide
checks that alternative models are not being used.

The configural abundance in both studies contradicts Green's
(l968) comments on the "perverse pervasiveness" of linearity. Con-
figural terms provided a better representation of judgment policies and
at times substantially better than the linear terms alone. Therefore,
it hoped that this study will begin to dispel some of the notions,
fostered by previous research (Goldberg, l97l; Dawes & Corrigan, l974),
that all human judgment is basically linear. The findings of this study
may have limited generality due to nature of the decision task utilized.
Nonetheless, future research should begin to concentrate on developing

more complex, multi-stage models of information processing.

L I ST 0F REFERENC ES

LIST OF REFERENCES

Angell, J. R. The providence of functional psychology. Psychological
Review, l907, 14, 6l-9l.

 

Brehmer, B. Cognitive dependence of additive and configural cue-
criterion relations. American Journal of Psychology, 1969, 22,
490-503.

Brehmer, B. Subjects' ability to use functional rules. Psychonomic
Science, l97l, 24, 259-260.

Brehmer, B. Effect of cue validity upon interpersonal learning of
inference tasks with linear and nonlinear cues. American Journal
of Psychology, l973, §§, 29-48.

Brehmer, B. Hypotheses about relations between scaled variables in the
learning of probabilistic inference tasks. Organizational
Behavior and Human Performance, l974, ll, l-27.

Brunswik, E. Representative design and probabilistic theory in a
functional psychology. Psychological Review, l955, g2, l93-2l7.

Dawes, R. M. A case study of graduate admissions. Application of three
principles of human decision making. American Psychologist,
l97l, 26, l80-188.

Dawes, R. M., & Corrigan, 8. Linear models in decision making. Egyf
chological Bulletin, l974, 81, 95-l06.

Deane, D. H., Hammond, K. R., & Summers, D. A. Acquisition and appli-
cation of knowledge in complex inference tasks. Journal of
Experimental Psychology, I972, 22, 20-26.

Dobmeyer, T. W. Modes of information utilization by employment inter-
viewers in suitability ratings of hypothetical job applicants.
Paper presented at Midwestern Psychological Association,
Cincinnati, l97l.

Dudyca, L. W., & Naylor, J. C. Characteristics of the human inference
process in complex choice behavior situations. Organizational
Behavior and Human Performance, l966, l, llO-l28.

71

72

Einhorn, H. J. The use of nonlinear, noncompensatory models in decision
making. Psychological Bulletin, l970, 26, 22l-230.

Einhorn, H. J. Use of nonlinear, noncompensatory models as a function
of task and amount of information. Organizational Behavior and
Human Performance, l970, 26, 22l-230.

 

Estes, W. K. State of the field: General problems and issues of theory
and meta theory. In W. K. Estes (ed.) Handbook of Learning and
Cognitive Processes. Vol. l. Hillsdale, N.J.: Lawerence
Erlbaum Assoc., T975.

 

Goldberg, L. R. Simple model or simple processes? Some research on
clinical judgments. American Psychologist, l968, 26, 483-496.

 

Goldberg, L. R. Five models of clinical judgment: An empirical com-
parison between linear and nonlinear representations of the
human inference process. Organizational Behavior and Human
Performance, 197l, 6, 458-479.

 

Green, B. F. Descriptions and explanations: A comment on papers by
Hoffman and Edwards. In B. Kleinmuntz, (ed.), Formal repre-
sentation of human judgment. New York: Wiley, l968.

 

 

Hammond, K. R. (ed.) The psychology of Egon Brunswik. New York: Holt,
Rinehart and Winston,"l966.

Hammond, K. R., & Summers, D. A. Cognitive dependence on linear and
nonlinear cues. Psychological Review, l965, 22, 2l5-234.

Hammond, K. R., & Summers, D. A. Cognitive Control. Psychological
Review, l972, 26, 58-67.

 

Heider, F. The psychology of interpersonal relations. New York: Wiley,
l958.

 

Hoffman, P. J. The paramorphic representation of clinical judgment.
Psychological Review, l960, 62, ll6-l3l.

 

Hoffman, P. J., Slovic, P., & Rorer, L. G. An analysis of variance
model for the assessment of configural cue utilization in
clinical judgment. PsycholOQical Bulletin, l968, 66, 338-349.

 

Hursch, C., Hammond, K. R., & Hursch, J. Some methodological consider-
ations in multiple-cue probability studies. Psychological
Review, 1964, 21, 42-60.

 

Kort, F. A nonlinear model for the analysis of judicial decisions.
The American Political Science Review, l968, 62, 546-555.

 

Libby, R. Man versus model of man: Some conflicting evidence. Organi-
zational Behavior and Human Performance, l976, 16, l-l2.

73

Meehl, P. E. Clinical versus statistical prediction: A theoretical
analysis and a review of the evidence. Minneapolis: University
of Minnesota, 1954.

Miller, G. A. The magical number seven, plus or minus two: Some limits
on our capacity for processing information. Psychological
Review, 1956, 62, 81-97.

 

Miller, P. McC. Do labels mislead? A multiple cue study within the
framework of Brunswik's probabilistic functionalism. Organi-
zational Behavior and Human Performance, 1971, 6, 480-500.

Nisbett, R. E., & Wilson, T. D. Telling more than we can know: Verbal
reports on mental processes. Psychological Review, 1977, 62,
81-97.

 

Ogilvie, J. R., & Schmitt, N. Situational influences on linear and non-
linear use of information. Proceedings of the Tenth Annual
Meeting of the American Institute of Decision Sciences. St.
Louis, October, 1978.

 

Popper, K. A. Conjecture and refutations: Theggrowth of scientific
knowledge. New York: Basic Books, 1962.

Rappoport, L., & Summers, D. A. Human judgment and social interaction.
New York: Holt Rinehart & Winston, 1973.

Schmitt, N. Social and situational determinants of interview decisions:
Implications for the employment interview. Personnel Psychology,
1976’ £29 79-10].

 

Schmitt, N. Comparison of subjective and objective weighting strategies
in changing task situations. Organizational Behavior and Human '
Performance, 1978, 21, 171—188.

 

Schmitt, N., Coyle, B. W., & King, L. Feedback and task predictability
as determinants of performance in multiple cue probability

learning tasks. Organizational Behavior and Human Performance,
1976, 16, 388-402.

Schultz, D. A. History of modern psychology. New York: Academic Press,
1969.

Simon, H. Behavioral model of rational choice. Quarterly Journal of
Economics, 1955, 62, 99-118.

 

Slovic, P., & Lichtenstein, 5. Comparison of Bayesian and regression
approaches to the study of information processing in judgment.
Organizational Behavior and Human Performance, 1971, 6, 649-744.

Toffler, A. Future shock. New York: Random House, 1970.

 

74

Tucker, L. R. A suggested alternative formulation in the developments
by Hursch, Hammond and Hursch, and by Hammond, Hursch and Todd.
Psychological Review, 1964, 21, 528-530.

Valenzi, E., & Andrews, 1. R. Individual differences in the decision
process of employment interviewers. Journal of Applied Psy-
chology, 1973, 62, 49-55.

 

Wiggins, N., & Hoffman, P. J. Three models of clinical judgment.
Journal of Abnormal Psychology, 1968, 22, 70-77.

Winer, B. J. Statistical principles in experimental design (2nd ed.).
New York: McGraw-Hill, 1971.

APPENDICES

APPENDIX A

APPENDIX A

Practice Applicants

To indicate your rating of each applicant, choose the appropriate number
from the expected job performance scale below, and write that number
in the blank space in the far right column.

The applicant can be expected to perform

1. Most duties inadequately.

2. Many duties at a substandard level.

3. Some duties at a less than satisfactory level.

4. Duties at minimal levels of satisfactory performance, i.e., just

get by.

5. All duties at satisfactory levels, but none beyond.

6. All duties acceptably and some above standard.

7. Many duties above standard.

8. Consistently beyond expected standards.

9. At high levels of performance; one of the best in the department.
lO. At levels of the very best in this occupation; far exceeding

standards.
Depend- Experi- Refer- Oral Class Intelli-

Person ability ence ences Com. Standing gence Rating

A 2 6 7 8 8 4

B 7 9 2 5 7 4

C l 4 8 6 5 7

D 7 2 3 6 8 10

E 6 7 6 4 2 6

F 6 5 5 2 6 4

G 6 5 4 7 7 2

H 6 2 9 4 6 7

I 2 7 3 5 9 6

J 2 7 3 5 9 6

K 9 5 4 6 l 3

L 6 6 l 6 5 7

M 3 5 5 7 4 2

N 7 4 2 6 5 4

O 5 4 7 7 2 5

75

76

Job Applicants

Indicate your evaluation of the following candidates using the scale
below.

The applicant can be expected to perform

1. Most duties inadequately.

2. Many duties at a substandard level.

3. Some duties at a less than satisfactory level.

4. Duties at minimal levels of satisfactory performance, i.e., just

get by. ,

5. All duties at satisfactory levels, but none beyond.

6. All duties acceptably and some above standard.

7. Many duties above standard.

8. Consistently beyond expected standards.

9. At high levels of performance; one of the best in the department.
10. At levels of the very best in this occupation; far exceeding

standards.
Depend- Experi- Refer- Oral Class Intelli-

Person ability ence ences Com. Standing gence Rating

1 l 4 3 7 7 8

2 5 6 5 2 4 3

3 7 5 3 8 6 3

4 8 l 6 4 9 7

5 6 4 7 4 2 6

6 6 3 2 9 5 8

7 6 8 7 2 7 5

8 7 9 2 5 3 5

9 5 8 2 6 6 3

10 5 7 6 6 2 2

11 6 8 7 5 4 3

12 9 5 8 2 6

13 8 6 7 l 6 4

14 4 3 6 6 5 l

15 2 7 4 7 5 6

77

Indicate your evaluation of the following candidates using the scale
below.

The applicant can be expected to perform

1. Most duties inadequately.
2. Many duties at a substandard level.
3. Some duties at a less than satisfactory level.
4. Duties at minimal levels of satisfactory performance, i.e., just
get by.
5. All duties at satisfactory levels, but none beyond.
6. All duties acceptably and some above standard.
7. Many duties above standard.
8. Consistently beyond expected standards.
9. At high levels of performance; one of the best in the department.
10. At levels of the very best in this occupation; far exceeding
standards.
Depend- Experi- Refer- Oral Class Intelli-
Person ability ence ences Com. Standing gence
16 6 2 5 4 6 4
l7 5 6 l 4 3 7
18 6 9 4 1 5 8
l9 5 2 5 8 8 4
20 2 5 7 9 3 6
21 4 4 2 5 5 4
22 5 4 6 6 2 7
23 l 6 7 5 3 6
24 ' 5 4. 6 2 5 6
25 4 5 5 5 2 5
26 4 l 6 6 5 4
27 7 6 2 5 4 3
28 2 5 4 3 6 8
29 8 8 6 7 2 3
30 7 6 8 2 6 5
31 6 8 2 9 8 10

Rating

78

Indicate your evaluation of the following candidates using the scale
below.

The applicant can be expected to perform

1. Most duties inadequately.
2. Many duties at a substandard level.
3. Some duties at a less than satisfactory level.
4. Duties at minimal levels of satisfactory performence, i.e., just
get by.
5. All duties at satisfactory levels, but none beyond.
6. All duties acceptably and some above standard.
7. Many duties above standard.
8. Consistently beyond expected standards.
9. At high levels of performance; one of the best in the department.
10. At levels of the very best in this occupation; far exceeding
standards.
Oepend- Experi- Refer- Oral Class Intelli-
Person ability ence ences Com. Standing gence Rating
32 5 2 5 7 4 4
33 2 4 4 6 10 7
34 9 6 7 5 5 _ 2
35 5 6 8 l 7 8
36 3 6 9 7 5 2
37 8 6 2 3 8 5 _____
38 4 7 8 5 6 l
39 3 2 5 6 5 8
4O 4 5 5 5 2 4

79

Before leaving, we would like you to verbally, graphically, or alge-
braically describe how you used the information to reach your decision.
If possible, make it as clear as possible so another person could use
your rule with the information and obtain your judgments. (Continue
your answer on the back.of this sheet if necessary.)

Please respond to the following questions by circling the appropriate
response number under each question.

1. How confident were you in your decisions?

1 2 3 4 5
Not Somewhat Very
Confident Confident Confident

2. In general, how good are you at making these kinds of decisions?

1 2 3 4 5
Poor Average Above Good Very
Average Good

3. How costly would wrong decisions (i.e., people hired who were
inadequate for the job) be for the organization in this research

project?
1 2 3 4 5
No Nominal Somewhat Costly Very
Costs Costs Costly Costly

4. How concerned were you about making those kinds of wrong decisions?

1 2 3 4 5
Very Slightly Somewhat Concerned Very
Unconcerned Concerned Concerned . Concerned

5. Overall, how would you rate the quality of the applicants?

1 2 3 4 5
Far Somewhat Average Somewhat Far
Below Below Above Above
Average Average Average Average

Thank you for your assistance.

APPENDIX B

82

High Cost, Labelled
General Instructions

You are to review the information given you about the applicants and
then decide how likely each individual is to succeed on the job if hired.
The position to be filled is one requiring a reasonable amount of respon-
sibility and contact with people. A bachelor's degree is typically
required for this position, and all applicants are scheduled to graduate
this year. (The position is described in general terms so the results
can be applied to many occupations.)

In a number of occupations today, the number of applicants is in-
creasing relative to the number of available positions. With this trend,
organizations are still making many less than desirable selections. A
number of candidates appraised as likely to succeed have not worked out.
These false positive mistakes are rather expensive, due to high training
costs and the costly errors made on the job. Therefore, in reviewing the
following applicants, please carefully evaluate each to insure that no
"wrong" decisions are made. Your primary focus should be to avoid the
selection of candidates who might not work out.

'Based on previous studies, six pieces of infbrmation have been
identified as being valuable in making job selection decisions. Each
source of information will be placed on a ten-point scale. A value of 10
represents a large quantity or very high perfonnance level while a value
of 1 represents very low amounts or levels. The average person would
generally possess a level of between five and six for each source. The
sources of information are listed below. Each value was determined by
at least two interviewers or reviewers.

(l) Dependability: a rating based on inquiries into previous work experi-
ences that the applicant had.

(2) Career-related experience: a rating based on the previous jobs and
other experience that the applicant had.

(3) Letters of reference: a rating based on three letters of reference
supplied by the applicant.

(4) Oral communication: a rating based on the applicant's ability to
communicate during an interview.

(5) Scholastic standing: a number representing the decile standing of the
applicant in her/his class (e.g., a 7 means the 70th percentile level).

(6) Intelligence: a rating of the mental ability of the applicant based
on an interview.

On the following pages will be presented the information for 40
applicants. Before evaluating the 40 applicants, you will be given the
opportunity to practice on 15 additional applicants. You are to examine
the information for each applicant individually, and then to rate the per-
son's expected level of job performance on a ten-point scale, ranging
from "inadequate" to "one of the very best in this occupation." Assume
equal gradations of performance between each interval on the scale. The
scale will be presented on each page for your convenience. In making
your ratings, please refrain from mathematical calculations. Use your
judgment to decide how likely each applicant is to succeed. Please be as
accurate as possible.

83

Please turn the page and make your decisions for the practice
applicants. Use the trials to see if you have any questions about the
task. If you have no questions, proceed to evaluate the 40 actual appli-
cants.

84

Low Cost, Labelled
General Instructions

You are to review the information given you about the applicants and
then decide how likely each individual is to succeed on the job if hired.
The information does not represent material from actual college students,
and your responses will only be used to study your use of the information.
The position to be filled is one requiring a reasonable amount of respon-
sibility and contact with people. A bachelor's degree is typically
required for this position, and all applicants are scheduled to graduate
this year. (The position is described in general terms so the results
can be applied to many occupations.)

Based on previous studies, six pieces of information have been identi-
fied as being valuable in making job selection decisions. Each source of
information will be placed on a ten-point scale. A value of 10 repre-
sents a large quantity or very high performance level while a value of 1
represents very low amounts or levels. The average person would generally
possess a level of between five and six for each source. The sources of
information are listed below. Each value was determined by at least two
interviewers or reviewers.

(l) Dependability: a rating based on inquiries into previous work experi-
ences that the applicant had.

(2) Career-related experience: a rating based on the previous jobs and
other experiences that the applicant had.

(3) Letters of Reference: a rating based on three letters of reference
supplied by the applicant.

(4) Oral communication: a rating based on the applicant's ability to
communicate orally during an interview.

(5) Scholastic standing: a number representing the decile standing of the
applicant in her/his class (e.g., 7 means 70th percentile level).

(6) Intelligence: a rating of the mental ability of the applicant based
on an interview.

On the following pages will be presented the information for 40
applicants. Before evaluating the 40 applicants, you will be given the
opportunity to practice on 15 additional applicants. You are to examine
the information for each applicant individually, and then to rate the
person's expected level of job performance on a ten-point scale, ranging
from "inadequate" to "one of the very best in this occupation." Assume
equal gradations of performance between each interval on the scale. The
scale will be presented on each page for your convenience. In making
your ratings, please refrain from mathematical calculations. Use your
judgment to decide how likely each applicant is to succeed. Please be
as accurate as possible.

Please turn the page and make your decisions for the practice appli-
cants. Use the trials to see if you have any questions about the task.
If you have no questions, proceed to evaluate the 40 applicants.

APPENDIX C

85

I
~ I I I
I I I I
n I I I I
w I I I I
n I I
I I I I
_ I I I I
1...:514
2 I I
a. I I I I
a I
I I I I
~ I I I
o I I I I
n I
v I I I I
a I I I
I I I I I I I
q I I I
«5:21-33 .
2 I I I I
-— I ‘ I I
e. I I
a I I I I
I I I I I
~ I I I I
o I I I
a I I I I
c I I I I
n I I I I I
~ I I I I I
_ I I I I
33:94.43.
as: on- e:« nqq IAN oe~ new saw aq~ qa~ on. o¢_ no- oI- an. In— o~_ na— cm— a~. ca II no 0« an Ia o~ n~ «N n~ 3. w— I. n. - a a Q
8:3. 13:13 2.3-93.3 '25 1:323:30 b.3133. III-G.

— Fit—:uaxs

Isa—n.s.»: 13.43! :— Ianqz: x: 30.5 33:... 2.10:1.a—m

U l.3§.~.~<

‘

i

I
NHQAON‘

. :3 2.... . 3:3.

I I I 2

I I I a.
1 fl 0

I i I
a

I I I O
a

I I I v
I I I q.
I I I ~
I I -

Iago III: .au_>::

I I I ~—
I I I _—
I =.

I I I a
I I I a
I I s

I I I I
I I I a
I I I I
I I I n
I I I ~
I I I I

II; III- .23»...—

n ~ . .ua.asa
I133:

86

I I I I I I I I
s I I I I I I I ~
4 I I I I I a
a I I I a
4 I I I I I I I I I v
I I I I I I A
~ . I I I I I I n
. I I I I I I I I I I a
332.5125 .250 :1... .IIu..Iu
:— I I I I I I I I I a.
a I I I I I I I I a
I I I I I I I
~ I I I I I I I ~
I I I I I I I I I a
a I I n
4 I I o
I I I I I I I I a
n I I I I I I ~
I I I I I I I I _
3:33.... .33 :3 .22.:
e. I I I I I I I I 4—
q- I I I I I I.
N— I I I I I I I u.—
.. I I I _.
a. I I I a.
o I I I I o
u I I I 9
~ I I I s
a I I I I a
n I I I I I I I I a
w I I I e
a I I I I I I I n
~ I I I I ~
_ I I I I .
2.2214 :25 II... .32....
__ I I .-
3. I I I I o.
a I I I I a

73:5... 3:31. .33 .5. :32...

III an. o«« AI. aAu o:~ n¢~ o.~ a.~ «an as. be. n¢~ c.— aq. 4.. an. n~. «~. —~_ on II nq a. AI 49 as aw q~ _~ a- n— o- n. I. o A e a I — Iguassu
33.3... 1.25:...5 x23....u..:. 3...... 1.3%....33 x3735. III—8h. .323.-

87

~ I I I I I I ~
0 I I I I I I I I I I I o
I. I I I I I I I I n
I I I . I I I I I I
q I I I f I I I I I I.
N I I I I I I I I I I ~

. I IIIIII—
as..uaa.a= ..Iau 3:. .I.I.==
.. haa:_ag.nn

: I I I I .—
0. I I I I a—
n I I I I I a
I I I I I I I I a
I I I I I u
a I I I I I I a
a I I I I I I n
q I I . I I I I
I I I I I I I n
u I I I I I I ~
. I I I I I .

99:33.3 2.2.”. .32. 3.3:..-

: I I I I _—
o. I I I I o.
a I I a
I I I I I I I
~ I I I I n
o I I I I o
n I I I I n
I I I I I I
q I I I q
u I I I ~
. . I I I I I I I I —

99:931.. .225 .5; .39.:8

.n. I I I I I I ~.
2 I I I I .-
3. I I I I I I :-
a I I I I I a

..1..::,. so..u:a_=: _.nau :a.= ...:z.a

Inc an. a: a: cn~ sou n34 is 29 «mm on. a: a: on. 2— II. a: a: I: «w. In 64 AI 3 an IQ n~ w~ :N «a I: n‘ .2 n- V— o a 4 n ~ — 95.33.."
3...!— _:I:I:.IJ >13 14.3.2. alt-h. _I.:I:::Q >372; 11.3. :31...

88

GNHQAONQOQ

—
d-

ficao~naa ‘NRQ-flONOSS
‘

Fl

-

t

as. _a.1.._
I

.5- _ 0.1..—

‘ I

l

.

~fufl€~l°

3"!

. 4'30 :33. 4:32.:

4

‘-
.1

q‘fio“

N

. and .54 3:19.:

4

I

3
no

NH-Dflfl‘.’

‘

3:32.... :3: .3... .31....
I I I l I i

:-
a
o

3:33. 3:211... .33 .5. .31....

One On. a: a: cox a¢~ awn 3N 2m in an. a: a: 2- na— 3— o- a: c: a: on 04 at on an 4... 8.. mu .1. am .6— n— v- n— Na 0 a Q n ~

In: .4.— qnu:a — 2:5 51:13.15.

‘53—. 1.2.3235 x..3|3)—.

ll. .5. uni—3

-

ave—9.5

89

-I N q C 1‘ J N D $
I
i

3 I

a
I
I I I

I I l I I

dNHQfiON‘
I
I
‘
‘

I

‘

i

i

‘

fl

3232.... .38

Sud-933a .u039 3%;

I
l
as...
l

3:315. .33 I...

__ I I I I I
a. I I I I I I I
a I I I I I I
a I I I I I
~ I I I I I
a I I I I I
a I I I I
e I I I I I
q I I I I I
s I I I I I I
g

__ I I
a. I I I I I

i

¢~HQ6‘~.’

.33.
2
a.

NHQRON‘

‘

.1.i&
«—
:—

.a‘sa
.—
:—

sn_.aasa ..-:u :I.= .s.aa::

a? a2 3. a: :2 3‘ 3a a: a: q: 3. 3. 3. a... n2 3. a: a: q: 3. an 3 3 a. an x 3 2 .R 2 2 2 3 2 2 a a I n

15.2? aa.:i_-=au xlzxua..:. alga? _susa...zxa x13132h

N

Il.ub uvvaqé

— «HI—4:.n

90

~n¢fl°~fia
fl

an
I

-
.-
=—

13- «383 .933 i::.

I

l

I

a.-
N-
_—
:-

-‘ ~ q ,' a a N ‘

.e_.;
2
Z
a.

334n9.‘¢— .013 )3.— Iced...—

ane on. .q. n;. aaa aqu “an a.» aq~ ¢.n on. sq. A}. 0.. “I. «I. on. n~. «~_ .I. an ae a. o. a. e. a~ n~ «m .a a. a. c. 5. ~. I a c n a _ .29.;sm

‘3... .325 . 2.33 s13- 93....

3.9... —a.:a::3 5475.5.

1:3. 32....

"‘filfifflffllfiifimfiflflflflwfiflmflfifl“