THESIS

=1
QCX to

m _.

i\iiiiiiiiiWW

1812 9456

r

 

\
LIBRARY

quiuiuiifliii
Michigan State
University

 

 

 

This is to certify that the

thesis entitled

"The Relationships Between the Perceived Job
Relatedness of Selection Tests, Motivational
Components of Test-Taking and Race"

presented by

Joshua M. Sacco

has been accepted towards fulfillment
of the requirements for

MA Psychology

degree in

 

 

) , ’ (I
V , ‘ , Ii . \_____f__~
, . ' / I ‘
. f. ' 71/ f1' /’
l -- / c [c /1’ j], /,'.- l] It-Z//

Major professor

Date May 13, 1999

 

0-7639 MS U is an Affirmative Action/Equal Opportunity Institution

THE RELATIONSHIPS BETWEEN THE PERCEIVED JOB RELATEDNESS OF
SELECTION TESTS, MOTIVATIONAL COMPONENTS OF TEST-TAKING AND
RACE
By

Joshua M. Sacco

A THESIS
Submitted to
Michigan State University
In partial fulfillment of the requirements
For the degree of
MASTER OF ARTS
Department of Psychology

1 999

ABSTRACT
THE RELATIONSHIPS BETWEEN THE PERCElVED JOB RELATEDNESS OF
SELECTION TESTS, MOTIVATIONAL COMPONENTS OF TEST-TAKING AND
RACE
By

Joshua M. Sacco

An important determinant of reactions to a selection test is a test’s perceived job
relatedness (PJ R), the extent to which the test ‘looks valid' or appears to predict job
performance. Recent empirical and theoretical work suggests that a test’s PJR is related
to justice perceptions and motivational components of test taking. This study sought to
Show experimentally that these relationships do indeed exist and to demonstrate how
these processes should differ across Blacks and Whites. Thus, two versions of a
cognitive test were developed to differ only in terms of their PJR by writing items in
either a job-related or non-job-related context. Data were gathered from a sample of 267
undergraduates. There was no evidence that the different test forms influenced reactions
or test scores either within or across race. The hypothesis that PJR affects test-taking
motivation through these fairness reactions, self-efficacy and internal attributions of test
performance was not supported. As predicted, there was a significant relationship

between race and belief in tests, and these beliefs were significantly related to PJR.

However, Race was not related to the other motivation variables as was hypothesized.

 

 

ACKNOWLEDGEMENTS

I would like to thank my thesis committee, which provided me with invaluable
guidance, support and constructive criticism that was instrumental in helping me
complete this project to the best of my abilities.

First, I would like to thank Rick DeShon for getting me down this road early on in my
graduate career by saying "Hey Josh, why dont you turn that project into a thesis?". I
wasn‘t thinking ahead at the time, but Rick’s matter-of-fact question certainly got me
oriented in the right direction. Many thanks to Ann Marie Ryan for discussing the test
reactions literature with me and for assisting with data collection towards the end of this
project when things were looking bleak. As you may recall, a certain ’research assistant’
could not navigate the library, but she sure could recruit subjects!

Extra Special thanks to Neal Schmitt for providing me with guidance and assistance
from before this project was even conceived of as a thesis. From the start, you provided
me with an enormous amount of assistance and feedback on NSF proposal which became
the basis for this project. Although I did not fully appreciate it at the time, this allowed
me to do more than I otherwise would have been able to, and to make the most out of my
first few years here. Your contributions to this project cannot be overstated, but I
especially appreciated your lighting-quick and incisive feedback, your seemingly endless
availability (and patience!) to discuss issues with me, and by providing structure when I

needed it.

iii

Last, but not least, I would like to thank Christine Scheu, for providing far more
assistance than can be conveyed here. From discussing the initial proposal to collecting

data and giving me helpful feedback, you have been there for me all the way. Thank you!

-Josh

iv

TABLE OF CONTENTS

CHAPTER 1 ........................................................................................................................ 1
INTRODUCTION
Organizational Justice ...................................................................................................... 5
Face Validity .................................................................................................................. 10
Hypothesis 1 ............................................................................................................... l4
Hypothesis 2 ............................................................................................................... 14
Hypothesis 3 ............................................................................................................... 17
Psychometric Perspectives on Cognitive Ability Testing .............................................. 18
Motivational Components Of Test Taking ..................................................................... 20
P] R and Motivational Components of Test-Taking ...................................................... 22
Hypothesis 4 ............................................................................................................... 23
Hypothesis 5 ............................................................................................................... 24

Mechanisms That May Underlie the Relationship Between PJR and Test-Taking

Motivation ..................................................................................................................... 24
Attributional Antecedents of Motivation. .................................................................. 24
Hypothesis 6 ............................................................................................................... 26
Hypothesis 7 ............................................................................................................... 26
Self-Efficacy. ............................................................................................................. 26
Hypothesis 8 ............................................................................................................... 27

PJ R, Motivation and Race ............................................................................................. 28

Hypothesis 9 ............................................................................................................... 29

Hypothesis 10 ............................................................................................................. 30
Hypothesis 11 ............................................................................................................. 30
Hypothesis 12 ............................................................................................................. 3O
Hypothesis 13 ............................................................................................................. 3O
Hypothesis 14 ............................................................................................................. 31
Hypothesis 15 ............................................................................................................. 31
Hypothesis l6 ............................................................................................................. 32
Hypothesis 17 ............................................................................................................. 32
Hypothesis 18 ............................................................................................................. 32
Hypothesis 19 ............................................................................................................. 32
Hypothesis 20 ............................................................................................................. 32
Summary ........................................................................................................................ 33
CHAPTER 2 ...................................................................................................................... 35
METHOD

Overview ........................................................................................................................ 35
Subjects .......................................................................................................................... 35
Measures ........................................................................................................................ 36
Cognitive Ability Test ................................................................................................ 36
Pilot Study #1 ................................................................................................................ 37
Pilot Study #2 ................................................................................................................ 38

vi

Test Reactions and Motivational Measures ............................................................... 39

Procedure ....................................................................................................................... 41
CHAPTER 3 ...................................................................................................................... 43
RESULTS

Item Context and Test Perceptions ................................................................................ 50

Item Context and Test-Taking Motivation .................................................................... 50

Race and Test Perceptions & Test-Taking Motivation .................................................. 52

Belief in Tests, Race, Test-Taking Motivation, and Test Reactions ............................. 57

Summary of Independent Hypothesis Tests ................................................................... 59

Tests of the overall model .............................................................................................. 59
CHAPTER 4 ...................................................................................................................... 74
DISCUSSION

Test perceptions & test-taking motivation and item context ......................................... 74

Perceived Job Relatedness, Test-Taking Motivation, and Test Performance ................ 76

Internal Attributions, Self-Efficacy, Test-Taking Motivation, and Test Performance .. 77

Race and BIT, Test Reactions, Test-Taking Motivation, Self-Efficacy, and Test

Performance ................................................................................................................... 78
Implications for Theory, Research, and Practice ........................................................... 81
APPENDICES ............................................................................................. ..................... 83
Appendix A. - Face Valid Test Form ............................................................................ 84
Appendix B - Non Face-Valid Test Form ................................................................... 118

vii

Appendix C - Test Reactions Items ............................................................................. 152
Appendix D - Factor loading matrix of post-test items (principal axis analysis; varimax
rotation) ....................................................................................................................... 159
Appendix E - Factor loading matrix Of pre-test items (principal axis analysis; varimax
rotation) ....................................................................................................................... 162
Appendix F - Means, SDS, and Intercorrelations of study variables broken down by
Race ............................................................................................................................. 164

REFERENCES ................................................................................................................ 167

viii

LIST OF TABLES
Table 1 - Means, SDs, Intercorrelations and Internal Consistency Estimates ((1) of study
variables (N = 267) ............................................................................................................ 45
Table 2 - Fit indices associated with multiple-group confirmatory analytic models testing
measurement invariance across test forms ........................................................................ 48
Table 3 - Hierarchical Regressions for Test Form, Perceived Job Relatedness and Pre-
Test Motivation as Predictors of Post-Test Motivation (N=267) ...................................... 51
Table 4 - Hierarchical Regressions for Internal Attributions and Pre-Test Motivation as
Predictors Of Post-Test Motivation (N=267) ..................................................................... 52
Table 5 - Hierarchical Regressions for Race, Test Form, and Race x Test Form as
Predictors Of Perceived Job Relatedness (N=267) ............................................................ 53
Table 6 - Hierarchical Regressions for Race, Pre-Test Motivation, and Race x Pre-Test
Motivation as Predictors of Post—Test Motivation (N=267) .............................................. 54
Table 7 - Hierarchical Regressions for Race, Pre-Test Motivation, Perceived Job
Relatedness, and Race x Perceived Job Relatedness as Predictors of Post-Test Motivation
(N=267) ............................................................................................................................. 55
Table 8 - Hierarchical Regressions for Race, Pre-Test Motivation and Post-Test
Motivation as Predictors Of Total Cognitive Test Score (N=267) .................................... 56
Table 9 - Hierarchical Regressions for Race, Pre-Test Self-Efficacy, and Race x Pre-Test
Self-Efficacy as Predictors of Post-Test Self-Efficacy (N=267) ....................................... 57

Table 10 - Summary of Hypotheses and Results ............................................................... 6O

ix

Table 11 - Completely standardized factor loadings and error variances of indicator
variables on endogenous latent variables (AY and Ge, respectively) ................................. 65
Table 12 - Summary of Standardized total effects, broken down by direct, and indirect

effects between all variable pairs in the path model .......................................................... 67

LIST OF FIGURES
Figure 1 - Relationships to be tested in current study ....................................................... 11
Figure 2 - Revised model of relationships to be tested in current study ........................... 63

Figure 3 - Completely standardized LISREL estimates for hypothesized relationships 64

xi

Chapter 1

INTRODUCTION

In recent years, research by personnel psychologists has shown that applicant
reactions to selection systems are important constructs worthy of empirical study (e. g.,
Chan, Schmitt, DeShon, Clause, & Delbridge, 1997; Macan, Avedon, Paese, & Smith,
1994; Schuler, 1993; Schuler, Farr & Smith, 1993; Srnither, Reilly, Millsap, Pearlman, &
Stoffey, 1993). This growing body of literature recognizes that personnel selection
should strive to satisfy both parties in the selection decision: The organization making
the decision and the applicants who experience the selection process. This perspective
has been termed the social side of selection (Herriot, 1989) and it is informative to
examine some of the arguments supporting this position. After all, on the face of it, it
seems as if a practitioner hired by an organization should be primarily concerned with the
selection system from the organization’s perspective; a focus on applicant reactions
would seem misguided from the perspective of selection system efficiency (even given
that this notion sometimes diverges from predictive efficiency due to adverse impact
concerns).

Perhaps of primary importance from an organization’s perspective is the possibility
that applicant reactions to selection systems affect organizationally relevant behaviors
such as applicant withdrawal from the selection process and the rejection of job offers
(e.g., Macan et al., 1994; Murphy, 1986; Ryan, Sacco, McFarland, & Kriska, 1998;

Rynes, Bertz & Gerhart, 1991; Rynes, Leek & Saunders, 1995; Schmit, 1994). These

outcomes have Si gnificant consequences for the organization because selecting and
recruiting applicants is oftentimes very expensive. Perhaps most troublesome is when an
applicant declines a job Offer because of his or her displeasure with the selection process
(Rynes, 1993). If minorities, in particular, withdraw or decide not to apply at all, an
organization may encounter substantial difficulties in achieving racial diversity among
new hires (Schmit, 1994). One can easily imagine this scenario resulting in a
snowballing effect, as applicants ‘spread the word’ about a company’s selection
procedures.

It is perhaps more realistic, however, to argue that applicant reactions to selection
systems affect the organization in subtle ways. This is because it seems unlikely that
many highly qualified applicants will turn down offers or do not apply for otherwise
desirable jobs My because of a company’s selection procedures (cf. Rynes, 1993).
This argument is bolstered by the self-serving biases candidates may hold after
successfully navigating a selection system (e.g., Lounsbury, Bobrow & Jensen, 1989).
Alternatively, it may be more reasonable to argue that selection systems shape applicants’
general perceptions of the organization and/or the particular job for which they apply.
These perceptions, in turn, may subtly affect how applicants behave towards the
organization (Schuler et al., 1993).

Recent research seems to support the argument that applicant reactions are
organizationally relevant in the more subtle sense as argued above. For instance, Powell
(1991) found that characteristics of the recruiter affected applicants’ perceptions of the

job, the company’s work environment and applicants’ intentions of accepting a job offer.

In a similar Study, Harris and Fink (1987) reported that the likableness Of the recruiter
was significantly related to applicants’ regard for the job and intentions of accepting a job
offer. This research suggests that recruiting interviews are important determinants of
organizationally relevant attitudes. However, there is no research of which I am aware

that has strongly linked these attitudes to behaviors.

 

Other research has focused on reactions to a wide range of selection situations. For
example, Konovsky and Cropanzano (1991) found that justice perceptions of selection
procedures were Significant predictors of turnover intentions and affective commitment.
Some researchers have compared applicant reactions across test types, reporting that
cognitive ability tests were perceived more positively than biographical inventories
(Daum & Ryan, 1995; Kluger & Rothstein, 1993; cf. Ryan & McFarland, 1997),
assessment centers (Macan etal., 1994) and work samples (Steiner & Gilliland, 1996), as
were work samples relative to paper-and-pencil measures designed to measure the same
constructs (Cascio & Phillips, 1979; Schmidt, Greenthal, Hunter, Bemer & Seaton,
1977). Further, Lounsbury et a1. (1989) report findings consistent with the notion that
selection decisions (i.e., being rejected or hired for a job) affect attitudes towards testing.

One of the more Obvious yet relatively unexplored outcomes of selection procedures
is the likely effect on the emotional well being of rejected applicants (Schuler etal.,
1993). Specifically, the self-efficacy and self-esteem of rejected applicants may suffer as
a result of being rejected in a selection situation (e.g., Gilliland, 1994). From the
practitioner’s instrumental perspective, organizations oftentimes face costly litigation

regarding their selection systems and those perceived in a better light are probably less

likely to be the focus of such legal action (e. g., Bible, 1990; Cascio, 1991). A broader
perspective might posit that society’s acceptance of selection systems is desirable for a
number of reasons and this acceptance may be at least partly contingent upon how
applicants react to selection systems (e.g., Anastasi, 1988; Nevo, 1987). We must not
forget, after all, that virtually every American encounters many selection systems over the
course Of a lifetime and the rejection of these systems would be devastating to the
practice and science of many fields of psychology. Additionally, personnel psychologists
have a professional (and perhaps moral) Obligation to treat applicants with dignity and
respect, and part of this imperative seems to involve the consideration of applicant
reactions to selection systems (Schuler et al., 1993). In sum, applicant reactions are
important for a number of reasons; however, this topic has only recently been the focus of
sustained empirical efforts.

Nonetheless, the growing body of literature provides a solid foundation for guiding
research and practice. Indeed, the present study focuses on what is hypothesized to be an
important determinant of applicant reactions, the perceived job relatedness (PJR) of
selection instruments. In particular, the present Study seeks to Show that not only is PJR
an important determinant of applicant reactions, but that it also may affect test-taking
motivation and thus test performance, a relationship that Should be moderated by race.
The empirical and theoretical foundations for these assertions are outlined below.
Applicant reactions research is examined within the framework of organizational justice
theories; the literature on motivational components Of test-taking will be reviewed in a

later section.

Organizatiorfl Justice

Theories of organizational justice are generally centered around two principal
constructs: Procedural and distributive justice. Distributive justice is concerned with
outcomes whereas procedural justice focuses on how these outcomes are determined
(Greenberg, 1990). These two constructs will be briefly discussed and will then be used
as a framework for understanding the recent wave of applicant reactions research and as a
basis for a number of hypotheses in the present study.

Distributive justice focuses on the equity of reward distribution. In other words,
people compare the ratio of their own inputs to outcomes and compare it with those of
others (Greenberg, 1990). Thus, in a selection context an equity rule stipulates that
people compare the proportion of inputs (e.g., qualifications) and outcomes (e.g., hiring
decisions) in making fairness judgments; a perceived inequity results when there is a
disparity between the ratio of inputs to outcomes. Thus, distributive unfairness results
when someone who is seemingly unqualified is hired or someone who seems qualified is
not hired.

Procedural justice, on the other hand, is concerned with the perceived fairness of the
rules used to make decisions rather than the decisions themselves. Gilliland’s (1993)
model classifies these procedural rules into three categories: Formal, explanation, and
interpersonal treatment rules. Formal characteristics are determined by the type and
content of the specific selection devices used in hiring decisions and are comprised of job
relatedness, opportunity to perform, reconsideration opportunity and consistency of

administration. Explanation refers to the extent to which applicants are informed about

the information derived from the selection process and consists of feedback, selection
information and honesty. These factors are heavily influenced by the prevailing human
resources policy at the time of selection. Lastly, interpersonal treatment is determined by
the actions of human resources personnel and is the extent to which people perceive
themselves as being treated fairly. This category is comprised of the following three
dimensions: Interpersonal effectiveness, two-way communication and propriety of
selection questions.

Recent research provides evidence to support the argument that some of these
procedural rules are antecedents of organizationally relevant attitudes. Rynes and
Connerley (1993) studied reactions in a sample of 390 undergraduates, a significant
portion of whom were actively seeking employment. Subjects were given 13 selection
scenarios, yielding three factors that were significant predictors of perceptions of
organizational attractiveness: The applicants’ faith in the employer’s ability to accurately
interpret results, the extent to which applicants think employers need to know the
information gleaned from the selection device, and their beliefs about their own ability to
perform well on the selection measure. The first two factors are consistent with
Gilliland’s formal procedural justice rules.

Research in field settings has yielded similar results. In particular, Macan et a1.
(1994) collected reactions data from over 3,000 manufacturing applicants who took a
cognitive ability selection instrument. Results indicated that overall perceptions of
fairness were significant predictors of intentions to accept a job offer, even after

controlling for test reactions, although attractiveness was a much stronger predictor than

fairness perceptions. Similarly, Smither et al. (1993) found that perceptions of procedural
justice were significantly (though weakly) related to organizational attractiveness. In
interpreting these findings, however, it is informative to note the heterogeneity of the jobs
for which applicants were applying; justice perceptions may interact with job type or
other unmeasured variables in predicting perceptions of organizational attractiveness.
Lastly, in a sample of current employees, Konovsky and Cropanzano (1991) found that
explanation was a statistically and practically Significant predictor of a wide array of
attitudes such as affective commitment and trust of management, another finding
consistent with Gilliland’s ( 1993) model.

Gilliland (1993) argues that the PJR of a selection system exerts the greatest influence
on procedural justice perceptions, although there is little direct evidence supporting this
assertion. In particular, when selection devices are seen as more job related, applicants
are expected to judge the selection procedure as more procedurally fair. From the
applicant’s point of view, job relatedness may be articulated as the extent to which a
particular selection device accurately assesses traits or abilities underlying job
performance. This perception, in turn, has been further divided into two components,
namely, perceived predictive validity (PPV) and face validity (FV; e.g., Gilliland, 1994;
Smither et al., 1993; cf. Gilliland, 1993). PPV is the extent to which the test appears to
accurately predict job performance, whereas FV, on the other hand, has been described as
perceived content validity. For valid tests, FV roughly corresponds to the notion of

transparence proposed in Schuler’s (1993) model of social validity.

Schuler’s (1993) model argues that selection systems should not only meet traditional
psychometric standards of validity but Should also be acceptable to the applicants who
experience these systems. Although there is not a one-to-one mapping of constructs, this
model significantly overlaps and is broadly consistent with Gilliland’s (1993)
conceptualization reviewed earlier. According to Schuler’s model, there are four factors
that contribute to a more socially acceptable selection system: Information, participation,
feedback, and transparency. Receiving relevant information regarding the characteristics
of the job and organization has been shown to be related to a number of positive
outcomes (Schuler, 1993). Participation and control in the selection setting refer to the
extent that the applicant feels involved in the selection situation. For instance, Schuler
(1993) reports that nondirective interviewers enhance applicants’ perceptions of the
interviewer’s ability to accurately assess their qualifications. Receiving feedback is
thought to engender positive applicant attitudes as well, though there is little direct
evidence in support of this notion; one may have to make the more general argument that
people prefer to receive feedback in a wide range of situations and that these effects
generalize to selection settings. Lastly, the transparency of a selection instrument is the
degree to which applicants perceive that they understand what a particular test assesses.
Task relevance is a key component of transparency and is closely related to a test’s FV.

An instrument's FV, or the extent to which a test appears valid (Anastasi, 1988) is
probably the factor most directly under the control of personnel psychologists, yet it has
received surprisingly little attention in the empirical literature. Additionally, the literature

reviewed thus far makes clear why FV should be of central concern to applicant reactions

researchers and practitioners alike. To reiterate, the PJR of a selection instrument is an
important determinant of organizationally relevant applicant reactions in which FV and
PPV play a central role (e.g., Steiner & Gilliland, 1996); thus, personnel psychologists are
well advised to focus on FV and PPV as important properties of selection instruments.
Although some researchers argue that FV and PPV are conceptually distinct, empirical
work reports contradictory findings. For example, although Chan (1997) convincingly
argues for a conceptual distinction between FV and PPV, researchers have found factor
structures for which neither one or two factors provides a very good fit to the data (e. g.,
Brutus, 1995; Ryan & McFarland, 1997) and internal consistency estimates that are
substantially higher than intercorrelations in some Studies (e. g., Chan, 1996; Smither et
al., 1993) but not in others (e. g., Ryan & McFarland, 1997); further, the one study
(Smither et al., 1993) reporting only moderate correlations between FV and PPV (:37)
is accompanied by the interpretational difficulties mentioned earlier (e.g., Chan et al.,
1997; Macan et al., 1994) and was also the sample on which the factors were originally
derived. It is possible that FV and PPV are conceptually distinct in the minds of
researchers but not in the minds of examinees. Accordingly, the present work will simply
refer to P] R as a composite of FV and PPV, although some empirical and theoretical
work has explicitly referred to one or the other. For the sake of accuracy, in these
instances the terminology adopted by the original work will be used; however, the reader
should keep in mind that the present work focuses on both FV and PPV as components of
P] R and will not attempt to disentangle conceptually these constructs unless the data

gathered in the present Study support this distinction.

The following section outlines the empirical and theoretical literature on FV (PPV has
received far less attention; cf. Chan, 1997). This review will then serve as a foundation
for generating hypotheses for the present study, the underpinnings of which are depicted
in heuristic form by Figure l (the relevant links will be mentioned after each hypothesis is
presented). In particular, the following sections will indicate how perceptions of
procedural justice should relate to the PJR of selection instruments. I will then consider
the implications of these reactions for motivational components of test taking and will
demonstrate how these findings may be able to further our understanding of Black-White
differences in performance on cognitive ability tests.

Face Validity

Mosier (1947) conducted one of the earliest conceptual analyses of FV, the primary
objective of which was to explicate the best definition of FV and then to Show why “the
term should be banished to outer darkness” (p. 191). Mosier argued that there was
significant confusion over the true definition of the term which subsequently hampered
practical efforts to construct psychometrically valid tests; psychologists were confusing
FV with construct and content validity. Alternatively, Mosier (1947) argued that a FV
test is one in which the test items appear as if they measure the relevant abilities or traits,
that is, that the test appears construct valid; however, Mosier argued that the concept
Should be abandoned altogether because of the definitional confusion it engendered.

Despite this recommendation, Mosier's work includes some insights that have gone

10

      
       

ZO_._.<>_._.OS_
wZO;DmEhF<
4<mekz.
a ’

o—

2

32» E256 E 339 on 8 aEmcoca—om ._ Semi

wmeOMijm
qu.

I I E
w

  

whww...
Z_ uwfiwm

N
b o _

 

wOZSZmOnEmE >O<O_umm-ur_ww c.

w0<m

 

11

relatively unnoticed in the 50 years Since his paper was published. In particular, Mosier
argued that FV is a critical component of employment tests because applicants who feel
their competencies are not assessed accurately will not bother applying. Though this
argument may overstate the importance of FV, this notion bears directly on the present
study. Second, Mosier argued that tests lower in FV will engender external rather than
ability-related attributions about test performance.

Adams (1950) conducted what was perhaps the first empirical study of FV, the
primary goal of which was to determine whether FV could be reliably measured. Despite
espousing Mosier’s definition, Adams (1950) proceeded to tell a sample of 39 personnel
department employees to disregard FV and instead consider which of a battery of tests
would be best for a specific job. Although the results indicated that this perception could
indeed be somewhat reliably measured, this study may have confounded PPV and FV.
Interestingly, the results indicated that these perceptions were positively related to the test
battery’s true validity as assessed by its correlation with peer ratings of job performance,
a finding that contradicts recent research (e. g., Rynes & Connerley, 1993).

In the 35 years following Adams’ (1950) work, all studies focusing on FV of which I
am aware have centered on personality assessment. Specifically, there has been a stream
of research focusing on the relationship between the face and predictive validity of MMPI
items (e.g., Gynther, Burkhart & Hovanitz, 1979; Holden & Jackson, 1979; Spence &
Helmreich, 1983). However, a detailed analysis of this issue is beyond the scope of this

paper. Suffice it to say there has been little recent research on FV in general and even

12

less in regard to cognitive ability testing in particular. One area in which the literature is
particularly sparse is the EV of cognitive ability tests in personnel selection.

Though Mosier’s (1947) work is not often cited in contemporary accounts of FV, the
definitional and conceptual clarity he advocates has been realized by the recent literature.
Indeed, the widely accepted definition of FV is the extent to which a test "looks valid"
(Anastasi, 1988, p. 144). In stark contrast to Mosier’s (1947) shrill calls to abandon the
term, Nevo (1987) noted the importance of FV by stressing that highly FV tests will
improve relations with the lay public, enhance fairness reactions among poor performers
and could convince policymakers to implement larger scale testing programs.
Additionally, Nevo (1987) argued that FV is a potential antecedent of important test-
taking attitudes, an idea research has only recently begun to address. Nevo’s research
also showed that examinees' perceptions of FV could be reliably measured, although it is
somewhat puzzling as to why he did not attempt to relate FV to any of the salient
outcomes he identified. If researchers had been more responsive to Nevo's claim that FV
was an important component of psychological tests, perhaps research on the effects of FV
would be further along than it is today.

One thing we do know is that the FV of a cognitive ability test used in personnel
selection is the extent to which the test is perceived to measure cognitive abilities that are
important for job success. Anastasi (1988) argues that “face validity can often be
improved by merely reforrnulating test items in terms that appear relevant and plausible

in the particular setting in which they will be used” (p. 145). Smither et a1. (1993) noted

13

that no research has actually tested this hypothesis, an endeavor that is one aim of the
present study. Hence, it is predicted that
Hypothesis 1: PJ R can be enhanced by framing cognitive ability test items in a
business context if the test is used for selecting applicants for business-related
positions (Link 5 in Figure 1).

This hypothesis implicitly assumes that presenting items in a job related context does
not significantly alter the underlying constructs being measured. Like the above
hypothesis, this issue has not been addressed by the empirical literature. Hence, to test
this assumption, it is predicted

Hypothesis 2: Framing cognitive ability test items in a job related context will not
alter the underlying constructs measured by the test.

Smither et al. (1993) attempted to test the first hypothesis by presenting one or two
sample items in a job related context. As the authors noted, this attempt failed to produce
differences in PJR probably because of the weakness of their manipulation. Additionally,
this effort did not Simulate real selection situations because applicants’ perceptions of job
relatedness are normally formed during the course of taking an entire battery rather than
examining one or two sample items (Smither et al., 1993). The present study overcomes
this limitation by reframing an entire test in a job related context. This will be
accomplished by rewriting a test recently developed to select entry level managers at a
telecommunications firm. Specifically, two parallel forms were created, one of which
Should be perceived as related to the job for which it is used and another that Should be

perceived as less job related.

14

Consistent with Gilliland’s (1993) model, Smither et al. (1993) found significant
relationships between FV and PPV and procedural and distributive justice perceptions
and willingness to recommend the employer to others. Also consistent with Gilliland’s
(1993) position, PPV and FV were Significantly and positively correlated with the former
three variables (rs=.22, .21 & .23 and .16, .09 & .14, for PPV and FV, with procedural
and distributive justice and recommendation, respectively). Although these correlations
are relatively low, these findings are striking in that justice and recommendation attitudes
were measured an entire month after test administration, implying that these relationships
may endure over time. These results are consistent with the existence of a causal
relationship between PJR and justice perceptions, though there have been no strong tests
of this hypothesis.

Organizational justice researchers provide evidence broadly consistent with the notion
that PJR is causally related to justice perceptions. For example, research by Tyler and
colleagues (e.g., Tyler & Caine, 1984; Tyler & Folger, 1980; Tyler, Rasinski, & McGraw,
1985) showed that perceived fairness is related to the perceived opportunity to contribute
process information (i.e., facts thought to be helpful in reaching an informed decision)
across a variety of situations. In a selection context, this would amount to the perception
of being given the opportunity to provide evidence that one can perform well on the job
(e.g., Steiner & Gilliland, 1996); by definition, this is more likely with selection
instruments that are perceived as job related. Similarly, Greenberg (1987) found that the

accuracy of information in performance appraisals is positively related to procedural

15

fairness perceptions, another finding consistent with the notion that tests high in PJR will
enhance fairness reactions.

There are a number of studies that have examined this hypothesized relationship in
selection contexts. For instance, Schmidt et al. (1977) found evidence consistent with
this relationship in reporting that a job sample was perceived to be significantly fairer
than a paper and pencil test designed to measure the same abilities. Cascio and Phillips
(1979) noted that no complaints were filed during a 17-month study of work sample tests
whereas prior complaint rates ranged from 5% to 20%. Further, Steiner and Gilliland
(1996) found that FV was the strongest of seven predictors of procedural justice
perceptions (1:.81). Similarly, Ryan and colleagues (Daum & Ryan, 1995; Ryan,
Greguras, & Ployhart, 1996) found that PJR was the Single best predictor of overall
fairness reactions. These findings strongly suggest that PJR may be causally related to
fairness perceptions, though this interpretation is problematic because some of these
studies (e.g., Cascio and Philips, 1979; Schmidt et al., 1977) confounded difficulty and
test content with the presumed FV difference between the two test types and because it is
possible that one’s prior attitudes towards an organization spuriously inflate the observed
correlations. Gilliland (1994) concluded that his study showed that PJR is an important
determinant of fairness perceptions because he found different fairness reactions to a
work sample, cognitive ability and overt integrity test. Not only did this design confound
test type with test content (also see Kluger and Rothstein, 1993 and Rynes and Connerley,
1993; cf. Chan and Schmitt, 1997), but perceived difficulty may have been inadequately

controlled as well (see Gilliland, 1994, and Kluger and Rothstein, 1993). Thus, few

16

studies have directly focused on whether PJR is causally related to fairness perceptions
and some of those that have suffer from methodological limitations that fail to rule out
plausible alternative interpretations. The design of the current study will permit a
stronger test of this hypothesis by attempting to manipulate PJR while eliminating the
potentially confounding effects of test method, type and content, and perceived difficulty.
Hence, it is predicted that

Hypothesis 3: A test framed in a job related context will enhance justice

perceptions, an effect that should be mediated by PJR (Links 5 & 11).

This hypothesis is important for a number of reasons. For instance, these effects
should be related to the likelihood of the litigation of a selection system (e.g., Smither et
al., 1993). Applicants who perceive particular selection instruments as more procedurally
fair may be less inclined to pursue legal action when denied a job. Additionally, a test
high on PJR should better withstand legal scrutiny, as juries may be more inclined to
decide cases based on information they can readily understand (e.g., PJR) rather than
technical evidence that may contradict their intuitions (e.g., validation studies). Thus,
potential litigants may have an advantage in legal proceedings regarding tests low in PJR
and Should thus be more inclined to pursue legal action (e.g., Cascio, 1991; Smither et al.,
1993). Unfortunately, it would be extremely difficult to disentangle empirically the
relationship between PJ R and examinees’ intentions to take legal action: A laboratory
setting would probably induce strong demand characteristics whereas a field setting
would encounter the problem of trying to predict a very low base-rate phenomenon on the

basis of a specific aspect of what is oftentimes an enormously complex situation.

17

Perhaps the most severe limitation of the applicant reactions literature reviewed thus
far is that it fails to relate applicant reactions to specific, salient and concrete outcomes
(i.e., something other than mere intentions; Smither et al., 1993; cf. Chan et al., 1997).
Additionally, much of the research has been correlational in nature and is thus open to
plausible competing interpretations. A less frequently stated (though not necessarily less
valid) criticism is that the bulk Of the research has narrowly focused on applicant
reactions from the organization’s perspective, though some of the organizational justice
literature is a notable exception. These limitations have given rise to test reactions
studies showing that applicant reactions to specific tests may influence test-taking
motivation and thus test performance. This nascent literature is better understood when
the applicant reactions literature is discussed in light of psychometric perspectives on
cognitive ability measurement.

Psychometric Persppctives on Cogpitive Ability Testing

Classical test theory maintains that the observed score on a psychological test is the
sum of the true score and measurement error. For instance, in cognitive ability testing,
one of the most common forms of employment testing, the true score represents a
person's standing on the underlying cognitive ability construct and the remainder is
measurement error. Measurement error may be further partitioned into unsystematic and
systematic components. Unsystematic error is essentially random, and may be conceived
of as an anomaly in a particular testing situation, such as a noise that causes an examinee
to select an unintended response on an answer sheet. Systematic measurement error, on

the other hand, occurs when a test consistently measures something other than what the

18

test developer intends. An example is a cognitive ability test that induces anxiety,
perhaps contaminating true ability scores with the ability to perform well in stressful
situations.

Systematic error may be described as having three sources: The instrument itself, the
person taking the test, and the instrument x person interaction. In cognitive ability
testing, systematic error resulting from the test instrument may be exemplified by test
items that only measure exposure to a particular book, for instance. The elimination of
this type of measurement error ideally takes place during test construction and
development. An example Of systematic error stemming from the examinee may be an
unmotivated individual who responds randomly to all tests he or She takes. By definition,
this type of error cannot be directly controlled by test developers; it is strictly a function
of personal dispositions and experiences, and it cuts across all testing situations.
Alternatively, an instrument x person interaction occurs when a test measures different
traits or abilities for different people. This type of systematic measurement error is the
focus of the most controversial topic in employment testing: Black-White differences in
cognitive ability test performance. Relating this to our example, these interactions occur
when a test measures different things for Blacks than for Whites. The bulk of the
research in this area has focused on whether cognitive ability tests measure academic
intelligence for Whites and exposure to White culture for Blacks (e.g., Helms, 1992). An
area that has received less attention is the possibility that cognitive ability tests may
differentially motivate Black and White examinees (cf. Arvey et al., 1990; Chan et al.,

1997). This scenario implies that performance differences are at least partly due to

19

differences in motivation. Assuming one does not include motivation as a component of
cognitive ability, it would be desirable to eliminate these influences, as they contaminate
the construct of primary interest. Thus, test reactions research tries to parse what
psychometricians have traditionally conceived Of as measurement error into a line of
research that has the potential to make substantial theoretical and practical impact by
broadening our understanding of Black-White differences in cognitive ability testing.

Despite this potential, motivational components of test taking have received scant
attention in the empirical literature; in this regard, potential Black-White differences have
received even less attention (cf. Arvey, Strickland, Drauden, & Martin, 1990; Chan et al.,
1997; Chan & Schmitt, 1997). The small corpus of literature that does exist, however,
can be used as a basis for understanding how P] R and race should interact to affect test-
taking motivation and in turn, test performance. This literature is reviewed in the
following section.
Motivational Components of Test Ta_1_tipg

Arvey et al.’s (1990) work represented a major contribution to research on
motivational components of test taking by developing a reasonable instrument to assess
test reactions and by calling attention to salient issues that have not been adequately
addressed by previous research.

In developing the Test Attitude Survey (T AS), Arvey et al. (1990) sought to
determine whether motivational and attitudinal dispositions towards employment testing
were related to test performance. In a sample of 263 applicants for a financial worker

position, the researchers found small to moderate correlations between TAS scores and

20

performance on a battery of selection measures; however, a separate sample of highway
maintenance workers yielded no evidence of the scale’s incremental validity, a null result
since reported by Others using Similar items (e.g., Ryan & McFarland, 1997; Schmit &
Ryan, 1992).

A more specific goal of Arvey et al.’S (1990) research was to explore the possibility
that subgroup differences in test-taking motivation accounted for a Significant portion of
Black-White differences in test performance. The researchers found that the 10-item
motivation scale of the TAS was Significantly, albeit weakly related to race (r=.129), with
Whites reporting they were more motivated than Blacks, a finding that has since been
replicated by others (e.g., Barbera, Ryan, Burris Desmaris, & Dyer, 1995; Chan et al.,
1997). Additionally, Whites performed significantly better than Blacks on all three tests.
On the basis of these findings, the researchers hypothesized that statistically controlling
for TAS scores would reduce the correlation between race and test performance (also see
Barbera et al., 1995). Indeed, for the arithmetic and work sample tests, the correlation
between race and test performance was significantly reduced when controlling for these
factors, a result that was not found on the comparison test. It is informative to observe
that this effect was evident only for the two tests presumably related to cognitive ability
(the work sample had arithmetic and verbal components), a finding that is consistent with
the argument that Black-White differences in cognitive ability tests in particular are
partially due to differences in motivation.

Although Arvey et al.’s (1990) findings seem to provide strong support for this

interpretation, there are a number of limitations with their design that weaken the force of

21

this assertion. Specifically, the researchers administered the TAS only after applicants
took the selection battery. Thus, their findings are also consistent with the possibility that
test performance may have affected reported motivation rather than vice-versa. To test
this alternative interpretation, Arvey et al. (1990) used a separate sample to compare pre-
and post-test TAS scores. There were no significant differences in reported motivation,
and a regression analysis indicated that performance was not a significant predictor of
post-test motivation after pre-test motivation was statistically controlled. As the
researchers noted, however, difficulties of interpretation persisted because reported post-
test motivation was not significantly correlated with test performance ($.07).

Chan et al. (1997) took this research one step further by showing that test-taking
motivation predicted cognitive ability test performance even after controlling for race and
previous performance on a parallel test. Additionally, this research replicated the finding
that race was Si gnificantly related to test-taking motivation, indicating that the
relationship between race and cognitive ability test performance may be partially
mediated by test-taking motivation. Further, Chan et al. (1997) reasoned and provided
some evidence indicating that a construct proximal to test-taking motivation is a test’s
FV .

PJRaLnd Motivational Components of Test-Tak_ing

It may not be immediately apparent why PJR should be related to test-taking
motivation. Though few studies besides Chan et al. (1997) have examined this putative
relationship, one can find much theoretical support for this position. For instance, Mosier

(1947) and Nevo (1987) note that measures low in PJ R may discourage applicants from

22

applying for jobs presumably because they feel their ability to perform the job will not be
accurately assessed. It does not seem unreasonable to extend this hypothesis to
motivation during the testing situation itself: Tests lower in PJR may be demotivating as
compared to those higher in PJR. Others (e.g., Cascio, 1991; Chan & Schmitt, 1997;
Jensen, 1980; Robertson & Kandola, 1982; Srrrither et al., 1993) explicitly mention this
possibility by stating that FV probably affects examinee motivation. Similarly, Gordon
and Kleinman (1976) observed that the high PJR of work sample tests may enhance
interest and motivation to learn a particular job, a notion that can be extrapolated to
particular tests as well. Thus, it is predicted

Hypothesis 4: A test framed in a job related context will enhance post-test

motivation (even after controlling for pre-test motivation), an effect that Should be

mediated by PJR (Links 5 & 9).

Additionally, it is plausible that this motivational effect is partially mediated by
justice perceptions. In other words, examinees may be reluctant to put forth effort for a
selection procedure perceived as unfair (e.g., Thorsteinson & Ryan, 1997). This may be
because by definition, selection devices perceived as procedurally fair are more likely to
reward good performance, thus fostering motivation; those that are unfair will not have
this motivational advantage. No research of which I am aware has tested this hypothesis;
however, this proposition is consistent with the recent finding that procedural fairness
perceptions moderate the validity of cognitive tests, perhaps indicating that examinees are

more motivated on tests perceived as fairer (Thorsteinson & Ryan, 1997; the putative

23

relationship between test-taking motivation and the validity of cognitive ability tests is
discussed in more detail later and in Schmit & Ryan, 1992). Thus, it is predicted
Hypothesis 5: The effects of PJ R on motivation will be partially mediated by the
perceived procedural justice of a selection instrument (Links 9, 11 & 14).

Another explanation regarding the role of PJR in test-taking motivation is available in
the empirical literature on motivation in occupational and educational settings. This
literature is helpful in that it discusses specific mechanisms that may underlie
motivational effects relevant to the present study. These mechanisms are the primary
focus of the next few sections of this paper.
Mechanisms That May Underlie the Relationship Between PJR and Test-Tamg
Motivation

In reviewing cognitive evaluation theories of motivation, Kanfer (1992) notes that
“...motivation is optimized when events are perceived as providing information about
one’s competencies...” (p. 89). Tests that are high on PJR should have precisely this
effect by supposedly providing more information to examinees regarding their ability to
perform well on the job relative to measures lower in PJR. This notion is closely related
to a stream of research in achievement motivation that focuses on attributions as
determinants of future motivation (e.g., Arnold, 1985; Graham, 1988; 1991; Weiner
1979; 1985).

Attributiorfi Antecedents of Motivation. The research on attributional components
of motivation has identified three causal dimensions believed to be especially salient in

determining motivational outcomes across a wide range of settings. Specifically, the

24

locus, stability and controllability of attributions are important in this regard (Graham,
1988; Weiner, 1985), though only the first is directly related to the aims of the present
Study. The locus of a causal attribution refers to whether the cause of a specific outcome
is perceived as internal or external to the individual making the attribution. Ability and
effort are often cited as internal causes, representing stable and unstable attributions,
respectively (e.g., Platt, 1988; cf. Dweck & Leggett, 1988). Alternatively, external
attributions are generally conceptualized as focusing on aspects of the task or luck, for
stable and unstable attributions, respectively. Internal attributions have been shown to
lead to future performance in a number of settings, a relationship believed to be mediated
by motivation, SE and a number of other affective variables (e.g., Platt, 1988). For
example, Platt (1988) used covariance structure modeling to test the effects of attributions
of first year college success. Attributions of success to internal causes (e.g., ability) were
positively related to success expectancies and predicted effort; these outcomes, in turn,
mediated a positive relationship between ability attributions and GPA. Similarly,
external attributions (e. g., task ease and luck) were negatively related to GPA. Now
recall that Mosier (1947) argued that FV tests will lead to internal rather than external
attributions, as a FV test appears to be indicative of one’s ability to perform well on the
job. This suggests that one way in which tests high in PJR may enhance test-taking
motivation is through the mediating effects of internal attributions. On the other hand,
the fear of failure is an important motivational factor in achievement settings (e.g.,
Graham, 1991; Weiner & Kukla, 1970), an effect also associated with attributions to

internal sources (e.g., Platt, 1988). For example, research conducted by Dweck and

25

Reppucci (1973) found that internal attributions for failure were associated with task
persistence, a construct that is construed as proximal to test-taking motivation in the
present context. This leads one to predict that examinees will be more motivated to
succeed on a highly PJ R test because failing on such an instrument (i.e., one supposedly
measuring something salient) would be an unattractive alternative. Thus, there exists
both conceptual and empirical work supporting the hypothesis that PJR affects
examinees’ attributions and that this effect will be related to test-taking motivation,
although it should be noted that any relationship between internal attributions may be
bidirectional due to self—serving biases (e.g., Chan, Schmitt, Sacco, & DeShon, 1998).
Thus, it is predicted

Hymthesis 6: PJ R will be related to internal attributions of test performance

(Link 10).

Hypothesis 7: The intemality of these attributions will be positively related to test-

taking motivation even after controlling for pre-test motivation (Link 20).

Self-Efficacy. It should be evident from this discussion that another relevant variable

is an examinee’s self-efficacy (SE). Individuals high in SE are confident that they have
the necessary knowledge, Skills or abilities to perform well on a particular task (Bandura,
1977). If an examinee makes an internal attribution about his or her test performance,
one would expect this to have a greater impact on his or her SE regarding test
performance (e.g., Dweck & Leggett, 1988; Gist & Mitchell, 1992; Platt, 1988; Weiner,
1985). Hence, one would expect that an examinee taking a relatively difficult test high on

PJR would suffer a loss in SE because the test is perceived as measuring something of

26

relevance (i.e., it is job related) and is thus more closely related to people’s ability self-
evaluation; on the other hand, examinees taking a parallel test low in PJR should see a
lesser decrement in SE. Thus, it is predicted
Hypothesis 8: Examinees taking a test lower in PJR will have higher SE (after
pre-test SE is controlled; Ryan et al., 1996) regarding their test performance than
examinees taking a parallel test high in PJR, an effect that should be mediated by
internal attributions (Links 10 & 15).

AS was mentioned earlier, the literature (and hypotheses) reviewed thus far suffer
from the limitation that plagues the applicant reaction literature in that it fails to answer
the ‘so what?’ question (Chan et al., 1997; Smither et al., 1993). Put another way, the
bulk of the research has failed to demonstrate the relationship between examinee attitudes
and important behavioral outcomes such as applicant withdrawal (Smither et al., 1993; cf.
Schmit, 1994). Indeed, there has been little research that has Shown that applicant
reactions are in any way directly and strongly related to organizationally relevant
outcomes. Accordingly, it is a major aim of the present study to draw clear connections
between applicant reactions to the PJR of selection instruments and outcomes critical
from the applicant’s and the organization’s perspective.

In light of the issues discussed earlier, perhaps the most logical salient outcome of
applicant reactions to P] R is test performance. The hypothesized relationship between
P] R and test-taking motivation was outlined earlier, and one would expect these
motivational influences to influence test performance. Recall that a test’s PJR is thought

to enhance motivation, the importance of which is twofold. Primarily, motivated

27

examinees have been shown to yield higher validities on cognitive ability tests (Barbera
et al., 1995; Schmit & Ryan, 1992; also see Thorsteinson & Ryan, 1997; cf. Sackett,
Gruys, & Ellingson, 1998). This finding should not be surprising in the least, as one
would expect test performance to most accurately assess cognitive ability when subjects
put forth the effort required to demonstrate their ability. Hence, uniformly increasing
motivation by enhancing a test’s PJR may enhance its validity; however, it is far from
obvious what benefits this confers when one extends these possible motivational effects
to test performance (i.e., why should we care if everyone performs better on tests high on
PJR?) The importance of these hypothesized effects becomes far more salient if one
examines the relationship between PJR, motivation and race.

PJR, Motivation grid Race

 

It is informative to revisit the earlier discussion of Arvey et al.’s (1990) research on
motivational components of test-taking and how these results related to race. To
reiterate, the researchers found that Blacks reported less test-taking motivation than
Whites and that controlling for these differences significantly reduced the magnitude of
Black-White differences in cognitive ability test performance. This finding supports the
thesis that factors minimizing Black-White differences in test-taking motivation will
reduce Black-White differences in cognitive ability test performance.

There is some evidence to suggest that PJR may be one such motivational factor. In
particular, FV has been shown to be related to test-taking motivation and recent work has
found that Blacks rate cognitive tests as less job related than Whites (Chan, 1997; Chan et

a1, 1997; also see Steele & Aaronson, 1995). For instance, Chan and Schmitt (1997)

28

found that controlling for FV on a situational judgment test (SJT) significantly reduced
Black-White performance differences. This study also found moderate Black-White
differences in FV and PPV perceptions on the paper —and—pencil test but no differences on
the video-based version of a SJT, a finding that suggests that race may moderate the
relationship between job relatedness and PJR, though there have been no strong tests of
this hypothesis. Another study that examined Black-White differences in PJR was
Smither et al. (1993), who failed to find significant differences, although this is probably
due to the previously mentioned heterogeneity of the jobs in their sample and the
concomitant diversity of tests (e.g., both ability and knowledge tests) used as a referent.
It is informative to note that these Black-White differences in perceived validity seem to
be particularly salient on cognitive ability tests (e.g., Chan, 1997; Chan & Schmitt, 1997;
Chan et al., 1997; Ryan & McFarland, 1997), as this effect is not found (or is
significantly smaller) on biodata (Ryan & McFarland, 1997; also see Brutus & Ryan,
1997), personality (Chan, 1997) or video-based SJTs (Chan, 1996). This is consistent
with my earlier observation that there may be Black-White differences in motivation for
cognitive ability tests in particular rather than selection tests in general. Specifically, PJR
is supposedly related to motivation and also to race but more strongly for cognitive ability
tests than those of other types. Additionally, for cognitive ability tests, if PJR is related to
motivation and race and if motivation, in turn, is related to race and test performance, we
can expect the following outcomes:

Hypothesis 9: There will be significant Black-White differences in PJR; Blacks

will rate tests as less job related than Whites (Link 6). Additionally, there should

29

be smaller Black-White differences on a test framed in a job related context
(Links 3 & 5).

Hypothesis 10: There will be a significant decrease in Black-White differences in
post-test motivation after pre-test motivation is statistically controlled. (It is not
depicted in Figure 1 because that figure only includes the post-test motivation
variable.)

Hypothesis 1 1: P] R will interact with race in affecting test-taking motivation
such that Black-White differences are smaller for a cognitive ability test higher in
PJ R (even after controlling for pre-test motivation; Links 7 & 9).

Hypothesis l2: Motivation will mediate the relationship between the interaction
presented in Hypothesis 1] and test performance (Links 7, 9, & 21).

The degree to which race affects performance on cognitive ability tests beyond this
motivational effect (e.g., Chan et al., 1997) will be tested by assessing the impact of race
after controlling for test-taking motivation. Specifically, it is predicted:

Hypothesis 13: Race will affect test performance even after controlling for pre-
and post-test motivation (Link 18).

Additionally, recall that SE is a factor expected to be related to PJR and motivation.
In considering recent work that suggests that Black-White differences in SE regarding
cognitive tests should be pervasive (e.g. Steele & Aaronson, 1995; cf. Jennings, 1997),

one can make the following predictions:

30

Hypothesis 14: There will be a significant decrease in Black-White differences in
post-test SE after pre-test SE is statistically controlled. (It is not depicted in
Figure 1 because that figure only includes the post-test SE variable.)

Iivpothesis 15: These differences in SE, in turn, will contribute to Black-White
differences in test-taking motivation and indirectly, test performance (even after
controlling for pre-test SE and motivation; Links 16, 19 & 21).

In addition to the evidence cited above supporting the notion that there are Black-
White differences in PJR, there are a number of other studies reporting results highly
consistent with this notion. Reviewing these studies is not only informative from this
perspective, but it also helps to identify variables that may account for the PJR X Race
interaction hypothesized above. Specifically, a dispositional variable that relates to race,
PJR and test-taking motivation is subjects’ generalized belief in employment testing
(BIT). Recently researchers (e.g., Arvey et al., 1990; Barbera et al., 1995; Schmit, 1994)
have found that subjects with a greater BIT reported higher test-taking motivation, a
finding that makes intuitive sense: Subjects having less of a BIT would indeed be
expected to exhibit less test-taking motivation because from this perspective the test
provides little relevant information about one’s true abilities (see the earlier discussion of
Kanfer’s ( 1992) point). Additionally, and consistent with my earlier predictions
regarding PJR, subjects with a greater BIT were found to make more internal attributions
about their test performance (Arvey et al., 1990; Schmit, 1994) and perceive tests as fairer
(Schmit, 1994). Lastly, researchers have found significant Black-White differences in

BIT (Barbera et al., 1995; Schmit, 1994; cf. Arvey et al., 1990) and perceived fairness

31

(Schmit, 1994). Further, on the basis of theoretical arguments (e.g., Helms, 1992) and
everyday practical experience, it seems that one would expect there to be significant
Black-White differences in BIT and fairness. This may be due, perhaps, to a generalized
mistrust of Whites on the part of Blacks, or perhaps more simply, cultural differences in
which Blacks are less likely to believe that there is only one right answer to a question
(e.g., Helms, 1992). This discussion suggests the following hypotheses:

Hypothesis 16: BIT will be related to internal attributions of test performance

(Link 12).

Hypothesis 17: BIT will be related to procedural justice perceptions (Link 4).

Hypothesis 18: There will be significant Black-White differences in BIT that will

partially account for Black-White differences in procedural justice perceptions

(Links l & 4).

Hypothesis 19: BIT will be positively related to motivation both directly (Link 13)

and as mediated by (a) the PJR of the test (Links 2 & 9) and (b) internal

attributions examinees make regarding their test performance (Links 12 & 20).

In considering the implications of the above hypotheses, it Should be obvious that the
hypothesized P] R X Race interactions affecting test-taking motivation and performance
should be diminished by controlling for subjects’ BIT. Specifically, it is hypothesized:

Hypothesis 20: Controlling for BIT will (a) reduce Black-White differences in
PJR (Links l, 2 & 6) and (b) significantly decrease the interactive effects of race
and PJR on motivation (Links 2, 7 & 9) and (c) test performance (Links 2, 7, 9 &

21).

32

These hypotheses are important for a number of reasons. Specifically, one must
consider that a priori Black-White differences in BIT may contribute to Spurious Black-
White differences in PJR that affect test-taking motivation and performance. The
implication is that if the Race x P] R interaction predicting motivation and performance
persists even after considering examinees’ prior BIT then Black-White differences in
motivation and performance can be reduced by enhancing the PJR of a Specific test.
Alternatively, if these interactions are eliminated by controlling for BIT, this indicates
that a far more pervasive effort must be undertaken if one seeks to influence these
motivational factors.

Summagy

The goal of the present study is to systematically manipulate PJR by creating two
parallel forms of a cognitive ability test designed to differ only in terms of their PJR with
regard to a specific job. By manipulating job relatedness while preserving the substantive
content of the test, this research seeks to address limitations of earlier research that
confounded job relatedness with test type, method and content, and job type. Further, this
research addresses the shortcomings of past research by recognizing that “applicants may
respond more favorably to an entire test presented in a business context as opposed to one
or two sample items” (Smither et al., 1993, p. 62) an approach that is also superior to
merely reading descriptions of selection procedures (Rynes & Connerley, 1993). This
research also heeds the calls of researchers (e.g., Chan, 1997; Chan & Schmitt, 1997;
Chan et al., 1997; Rynes & Connerley, 1993) who stress the need for experimental

manipulations of applicant reactions while controlling for the potentially confounding

33

effects of test type, content and method. The present study also seeks to unambiguously
Show that applicant reactions are strongly related to PJR by controlling for perceived
difficulty across parallel forms. This precaution provides evidence against the argument
that there are any meaningful differences in the two test forms other than job relatedness.
In addition, the present work is directed to an evaluation of the effects of PJR on
perceptions of procedural justice and the mechanisms underlying the hypothesized effect
of PI R on test—taking motivation and test performance. A conceptual model representing
the hypotheses presented above is depicted in Figure 1. The fit of this model to data in
the present study will be tested using covariance structure modeling in LISREL 8
(JOreSkog & SOrbom, 1993). This analysis will allow one to assess the relative
importance of many of the direct and indirect effects hypothesized individually

throughout this paper, as it will take into account the nonindependence of the hypotheses.

Chapter 2

METHOD

Overview

Examinees were given a job description for an entry-level managerial job at a large
telecommunications firm and completed either a job related (JR) or non-job related (NJR)
fOrm of a cognitive ability test geared towards these jobs. Pre-test attitudes, dispositions
and demographics were collected before examinees took the test and test reactions and
attitudes were collected after each of the three test sections. Examinees were given
sample items before the pre-test motivational measures so as to establish a concrete
referent for these items.
was

Subjects were 267 undergraduates from Michigan State University. 63.7% were
female, 49.8% were Black, 70.7% were freshmen or sophomores, and 18.7% indicated
some sort of full-time permanent employment. Initially, subjects were recruited through
the introductory psychology pool. However, since enough Black subjects could not be
obtained from the subject pool, Blacks were recruited via a variety of means including
announcements at Black caucus meetings and via fliers placed in mailboxes. The
subjects recruited through the subject pool received course credit while those recruited
externally received $20 for participating in the present study along with a few other

Studies (the present study was completed first). In addition, to enhance motivation to do

35

well on the tests, all subjects were told they would receive $15 if they scored in the top-
third of the examinees. All responses were completely anonymous.
Measures

Cognitive Ability Test. The cognitive ability measure used in the present study was
adapted from Chan et al. (1997) and is used to select entry-level managers at a large
telecommunications firm. Based on the job descriptions for these managers, items were
inspected to determine whether they would probably be perceived as job related (JR) or
non-job related (NJ R). Items that seemed to fall at either end of the JR-NJR continuum
were added to the appropriate test form (and revised as necessary) and a parallel item was
generated for the other form; items that fell somewhere in the middle were revised for
each test form.

Thus, two parallel forms were constructed in an effort to manipulate PJR while
controlling for the test’s substantive content. In writing the parallel items, attention was
paid to maintaining equal length, concreteness and difficulty (see Appendices A and B for
the JR and NJR versions of the test, respectively). A number of items and scales (e.g.,
analogies) from Chan et al. (1997) were deleted because it was apparent that strong
parallelism could not be achieved.

Each parallel form consisted of verbal, quantitative and logical reasoning scales (41,
14 & 16 items, respectively). The verbal scale consisted of verbal comprehension and
grammar questions (27 and 14 items, respectively). The quantitative scale consisted of 14
word problems. The logical reasoning section contained deductive, syllogistic and

inferential reasoning questions (8, 4 & 3 items, respectively).

36

Pilot Study #1

A pilot study was conducted to determine whether it was possible to manipulate the
PJR of the cognitive tests described above. Specifically, in a between subjects design
composed primarily of Whites, 20 subjects took the JR and 17 took the NJR test form and
completed test reactions and motivational measures (these scales are described in more
detail below) after each of the three test sections. The test form manipulation yielded
mean differences in the predicted direction for PJR (g=.28). Also consistent with the
hypotheses outlined above, the JR form yielded mean differences in the predicted
direction for test performance, test—taking motivation, and SE (as = .48, .37 & —.5 l ,
respectively). These latter two motivational effects are more clearly assessed by
controlling for the pre-test measures of motivation and SE, respectively. These pre-test
measures can be conceptualized as SE and motivation regarding logical reasoning, verbal
and quantitative reasoning questions (the tests were described as such; examinees did not
see any sample items). After partialling the corresponding pre-test measure, the effect
sizes for the correlations between test form and motivation and SE rose to .49 and -.68,
respectively. Though these fall within the range of moderate effects (Cohen, 1988), their
magnitude is quite striking given the subtlety of the manipulation in this design.

This pilot study also provided data that helped in refining the measures. In particular,
parallel items with large mean performance differences (in either direction) were
scrutinized especially carefully to insure that these differences were not due to a
detectable difference in item length, complexity, etc. After examining these item

statistics, some items were deleted because it was evident that they were not parallel;

37

however, these problems did not significantly influence the performance difference
between the two test forms reported above, as further analyses omitting these items
yielded nearly identical results. The data gathered in the pilot study were also used in
refining the test reactions and motivational measures described below. One item was
deleted because of a low item-total correlation; excluding this item did not significantly
influence the results reported above.

Pilot Study #2

One concern about previous research in this area was the lack of control for test

difficulty in comparisons of hi gh-PJR and low-PJR measures (e.g., Cascio & Philips,
1979; Gilliland, 1994; Kluger & Rothstein, 1993; Rynes & Connerley, 1993; Schmidt et
al., 197 7). Further, a number of the hypotheses in the present study can be more clearly
tested if perceived difficulty is controlled across the two test forms. This serves to
increase the likelihood that main effects for test form are due to job relatedness
differences rather than unintended differences in the difficulty of items across the two test
forms. For instance, Abraham and Kluger (1993) argue that the favorable applicant
reactions may be negatively related to cognitive demand. Additionally, recall that Rynes
and Connerley (1993) found that examinees’ perceptions of how they would fare on
selection instruments was a significant predictor of their reactions to the instruments. This
suggests that one should hold constant perceived difficulty across test forms to rule out
this alternative interpretation. A number of the hypotheses focusing on the effects of the
job relatedness manipulation seemed sufficiently important to warrant a pilot study that

would significantly weaken the force of this alternative interpretation. Further, the results

38

of the first pilot study can be explained on the basis of possible differences in perceived
difficulty, so it was desirable to rule out this interpretation.

Thus, a second pilot study was conducted to Show that the two test forms do not differ
in perceived difficulty (or that they differ in ways that are inconsistent with the alternative
interpretation mentioned above). Specifically, 15 subjects took the JR version of the test
and 18 subjects took the NJR version. In this study, these tests were framed as measures
of ‘logical, verbal and quantitative problem solving.’ In other words, they were not
associated with selection applications in any way. To obtain measures of perceived
difficulty, examinees Simply rated the perceived difficulty of each question on a 5-point
Likert scale after answering each question (or = .97). Interestingly, results indicated that
the JR form of the test was perceived as more difficult (d = -.46), less motivating (d = -
.74) and was associated with poorer performance (d = -.24). Thus, taken together, these
two pilot studies indicate that framing measures in job-related contexts may be a strong
enough manipulation to result in large reversals of motivational and performance effects
(differences in effect sizes of 1.11 and .72, respectively).

Test Reagtions and Motivational Measures. Responses to all items described in this
section (Appendix C) were made on a 5-point Likert scale with responses ranging from
“strongly agree” to “strongly disagree.” Test-taking motivation (MOT; 7 items) was
assessed with items adapted from Arvey et al. (1990). An example item is “I want to be
among the top scorers on this test.” Perceived Fairness (PF; 4 items) items were adapted
from those described in Gilliland (1994); an example is “Whether or not I get the job,

using this test to select people is fair.” Self-efficacy (SE; 5 items) were adapted from

39

Pintrich and DeGroot (1990) and are typified by “I’m certain Icould solve the problems
on this test.” Face validity (FV) and perceived predictive validity (PPV; 4 and 5 items,
respectively) were based on and adapted from Smither et al. (1993). Representative items
are “I did not understand what this test had to do with the job” and “I am confident that
the examination can predict how well an applicant will perform on the job” for FV and
PPV, respectively. Examinees’ generalized belief in employment tests (BIT; 5 items)
were assessed with items adapted from Arvey et al. (1990) such as “Employment tests are
a good way of selecting people into jobs.” Internal attribution items (IA; 10 items) were
based on the descriptions of Platt (1988) and were written by the author. An example
item is "my score on this test was a function of a lack of practice on similar items" (some
of the items on this scale are written with external attributions in mind and were reverse
scored SO as better assess the construct of interest). The second pilot study described
above provided some data regarding the reliability and validity of these items. In
particular, after two of the six items used in the pilot study were deleted due to low item-
total correlations, the internal consistency estimate was adequate (or = .75) and a principal
components analysis extracted one factor which accounted for 58% of the common
variance. While factor analyses on such small sample sizes are generally unstable, these
results nonetheless provide preliminary evidence for the unidimensionality of the items
written for the present study. In an effort to better assess the construct, 4 additional items
were generated to yield a total of 10 items used for the final measure.

Evidence regarding the reliability and (factor analytic, convergent and discriminant)

validity of the scales used in the present Study is reported by a number of researchers

40

(e.g., Arvey et al., 1990; Chan, 1996; Chan et al., 1997; Schmit, 1994; Schmit & Ryan,
1992; Smither et al., 1993) and is supported by the results of the two pilot studies
described earlier.
Procedure

Upon arrival at the testing session, examinees were given a brief overview of the
project stating that the experimenter was developing a measure of verbal, quantitative and
logical reasoning used to select entry-level managerial positions for a large
telecommunications firm. After completing the demographics questionnaire and BIT
items, examinees were presented with a recruiting brochure the author constructed based
on materials used by telecommunications firms to recruit college graduates. These
content of these materials were obtained from the University’s placement office. This
brochure includes a detailed job description based in part on a job analysis (see Chan et
al., 1997) of entry-level managerial jobs at a large telecommunications firm. College
graduates are the typical focus of recruiting efforts for these jobs. After reviewing the
recruiting brochure, examinees were presented with parallel sample items framed in a
context appropriate for the test form (i.e., subjects in the JR condition will have job
related sample items, etc.). After reviewing these items as a referent, examinees
responded to the two pre-test motivational measures (PRE-MOT, PRE-SE). This was
done to yield a relevant covariate for the motivational analyses, as these items are
designed to assess PRE-SE and PRE-MOT for the specific types of questions found on
the tests used in the present study. These items are of moderate difficulty so as not to

induce floor or ceiling effects on the pre-test measures.

41

Subjects were randomly assigned to take one of the two test forms and completed
post-test measures (MOT, SE, PPV, PJ, FV, IA) after each of the three sections (i.e.,
verbal, math, and logical reasoning sections). The job description was reiterated to
refresh examinees’ memories and the examinees were instructed to respond to the
reactions measures using the specific test section they have just taken as a referent. The
pre- and post-test measures were rephrased as necessary and the JR and NJR sections

were not mixed within each subject's test.

Chapter 3

RESULTS

Before any of the substantive hypotheses were tested, principal axis factor analyses of
the perceptions items were performed to examine and refine the scales’ measurement
properties. In addition, reliabilities and intercorrelations among scales were examined.
For each of the posttest scales, the mean of the item after each of the three sections was
used as the index of a person’s responses to that item. These averages were then treated
as items in the overall scales. The principal axis factor analysis with a varimax rotation
of the post-test reactions items yielded a solution consisting of six interpretable rotated
factors which accounted for 67% of the total variance (see Appendix D). The patterns of
factor loadings suggested that items measuring perceived fairness, face and perceived
predictive validity all measured the same underlying construct, and so these items were
averaged to form a global reactions variable (hereinafter referred to as PJR; see also,
Chan et al., 1998). This factor analysis strongly supported the a priori scale assignments
of the motivation and self-efficacy items. Three of the ten internal attribution items
loaded on a single factor that was interpreted as ability-related internal attributions. A
fourth item (“my performance on this section of the test was an accurate assessment of
my ability”) loaded highly on the first overall reactions factor. Based on the content and

its high factor loading, this item was added to the perceived job relatedness scale. The

43

analysis with the pre-test items fully supported the a priori assignment of items to scales
(see Appendix E).

Total test score was measured by the average of each examinee’s percent correct on
the verbal, quantitative, and logical reasoning sections of the ability measure. Two items
from the verbal scale were dropped because after examining item statistics and item
content, it became apparent that these items were ambiguous and may have had more than
one correct answer.

Means, standard deviations, internal consistency estimates and intercorrelations of the
Study variables are reported in Table 1. The same table broken down by race is presented
in Appendix F. Inspection of this table suggests that there is little bivariate support for
the hypotheses in this study. Although race was strongly related to test performance,
neither test form, reactions, nor motivation was related to test performance. Further, test
form was not related to P] R or test-taking motivation. This is inconsistent with previous
research that has consistently found relationships between test reactions and test
performance (e.g., Chan et al., 1997, Chan, 1997, Chan & Schmitt, 1997, Chan et al.,
1998; Smither et al., 1993). Tests of the individual hypotheses are reported next,
followed by an overall test of the conceptual model presented in Figure 1.

H2 stated that re-framing the cognitive test in IR versus NJR terms will not alter the

underlying constructs measured by these tests. Multiple groups covariance structure

modeling as implemented in LISREL 8 (Joreskog & Sorbom, 1993) was used to test this

.Ecomfle 65 :0 93 33:58 SEER—.8 HESS
.zoaoEm—éom mob-“mom H mm aeossntzxx RES:— u <— EonwéoE amok-«mom u HOE nauseous—om pea 33830
n ME ”monofim-bom “mob-80 u mmm E33382 sch-o5 n HOSE ”88,—- E 03.5 n Em ”ASE? u v £62m u

C 83— " comm ”806m mob 32¢on :33. u woo MAE—.3 comm n _ ”23> com-,— 82 u 00 Econ mop. n "E. 5802 .m0.va...

 

mm. *hmd ivd 3N0 .1010 *mmd no.0- *N_.0- 00.0 _0.0 mm.0 mod mm
N0. _ _ .0 *w _ .0 M00 8.0- *2 .0 00.0 *de m0.0 5.0 m0.m 5

Va. «0.0 0nd 30.0 000 N00- 0_.0 00.0 mod :5 H02

N0. 20. 00.0- .800 3.0 00.0 50.0 N00 bad man—

50. 0nd N00- *Vm.0- 00.0- _0.0 05.0 $2“ mam

vw. No.0 L. _ .0- 8.0- 00.0 5.0 00d H020

0w. *mmd *m: .0 _0.0 50.0 SN .Em

- med 0— .0- 0m._ EN mU<m

- _0.0 _ _ .0 Sud UOU

- 0V0 $0 "3.

mm 5 H02 ma Mmm HOSE .Em mU<m 000 "E- Qm :32

 

 

gm u 20 mus—flwqmflweam we 08 moan-80mm mocoaficoflmEBE cam macaw—EOBBE «mam flawed-4-

.H 2an

45

hypothesis. In particular, a series of nested models were compared such that the relevant
parameters were progressively fixed across the two test forms. Each of the individual

models was evaluated using the traditional x2 fit index; comparisons between the nested
models were made using the x2 difference test. This allows one to test the significance of
the difference between nested models because the difference between the model x2 is

itself distributed as x2 with degrees of freedom equal to the difference between the two
models’ degrees of freedom. By progressively constraining model parameters, one thereby
increases the parsimony (and degrees of freedom) of the successive models. Due to the

sensitivity of x2 to sample Size, however, other less sample-size dependent fit indices

were also used. These practical fit indices include the Root Mean Square Error of
Approximation (RMSEA), Goodness of Fit Index (GFI), Comparative Fix Index (CFI),
Non-Normed Fit Index (NNFI), and the Standardized Root Mean Residual (SRMR).
These five fit indices are commonly used in psychological research as indicators of
convergent evidence of a conceptual model’s fit to observed data. For the RMSEA and
SRMR, a perfect fit yields a value of zero whereas perfect fits for the other indices equal
one.

Each model consisted of three latent factors, corresponding to the three a priori
dimensions of the test, i.e., verbal, quantitative and logical reasoning. The least
parsimonious (i.e., baseline) model, Ml, consisted of the three latent factors with factor
covariances, error variances, and factor loadings freely estimated across groups. Nested
within the baseline model is M2, which is equivalent to M1 except that the factor

loadings are constrained across the two groups. M3, the next model considered,

46

constrained the indicator error variances to be equal across the two groups. Finally, M4,
the most parsimonious model tested, adds the further constraint of equal factor
covariances across groups.

The measurement models described above were formulated using three indicators for
each of the three a priori constructs. These indicators were created by assigning every
third item within the scale measuring each of the constructs to one of the three indicators
for that scale. This was done to get a good mix of items of different types (e.g., analogies
versus vocabulary items on the verbal reasoning section) contributing to each of the three
indicators for each construct. This also served to ensure that the slightly speeded nature
of the tests did not differentially affect any of the indicators, as items toward the end of
the test were distributed equally among the indicators (4.0%, 24.5%, and 31.6% of the
examinees failed to complete the logical, quantitative, and verbal sections, respectively).

Table 2 presents these series of nested models, along with the associated fit indices

and x2 difference tests. The baseline model, M l , provided a good fit to the data as
illustrated by the non-significant x2 test (x2 (51)=54.17, p>.05) and the practical fit

indices. Nested within M1 is M2, in which factor loadings were constrained to be equal
across the two groups. This more parsimonious model, continued to provide a good fit to
the data as indicated by the non-significant x2 difference test (1338 (6)=4.65, p>.05).
Additionally, as is reported in the table, three of the six practical fit indices are within .01
of what would be expected from a perfect fit, indicating that this model fits the data very

well.

47

8.x. ..

 

M: N0.mm v2 .m> NE
N0. 50. wo. wo. mm. m 50. v2 .m> m2

m0. 50. 8. 3. m0. 0 .002 m2 .m> NE

00. 00. 00. 00. _ cm. 0 36 N2 .m> :2

mo

00

mm

_m

oft.

N28

Nwwm

Cam

soccer—«>8
.290 98 828:?»

hobo .wwcfiao— .868 Escm
.anoam 3.22.50 69::- - 32
.moocmtgoo

8.20 8.0 can accent?

cote new $598. 868 Ram
.mcoaomm Bugaboo 025. - m2
accent?

coho ecu $25566 8620
out amp—=03— couofl 1.50m
.muouomm noun—2.60 8E0. - N2
accent? note 98 .3598.
c220 £85566 .820 68”—
.806£ team—oboo 85,—. - :2

 

<mm§ diam EZZ EU ED :5 ~x< :OmcwanU

to

x

8:80 mmoco< 02-2625 3022

 

 

 

HES $8 3212-“ 8:“th EoEoEmmoEquaB £262: 25% boamccceoo macaw-202:5 5:5 eoammoomwmflofifi .E

.N 2.3

48

Next, a third model, M3, was evaluated in which the factor loadings and indicator
error variances both were constrained to be equal across the two groups. The fit of M3

was significantly worse than that of M2 as indicated by the Significant x2 difference test
(Ax2 (9)=18.30, p<.05), although the practical fit indices still indicate a reasonable fit to

the data. Based on the facts that a fourth, more constrained model (described below)

would gain additional degrees of freedom, the sz reported above only slightly exceeded
the x2 critical value, and the practical fit indices still indicated a reasonable fit, it seemed
reasonable to continue to test additional models despite the significant A12 reported

above. Inspection of the common metric completely standardized parameter estimates of
the error variances in M2 did not yield any discemable pattern of differences. The mean
difference in the parameter estimates was .09, with the absolute values of the differences
ranging from .33 to .11. The lack of a patterned set of differences and the relatively small
absolute value of most of these differences indicated that their practical impact was
questionable, providing additional evidence that estimating an final, more constrained
model was reasonable.

The last and most parsimonious model to be tested constrained the indicators’ factor
loadings and error variances and factor covariances to be equal across the two groups.
This model fit the data well, as indicated by the practical fit indices as well as the
nonsignificant x2 difference test as compared to M3 (sz (3):.07, p>.05). Similarly, as
compared to our original baseline model, M1, the decrease in model fit was not
significant (Ax2 (18)=23.02, p>.05). Thus, this more parsimonious model was accepted

as the final model providing the most parsimonious explanation of the data along with

49

good fit indices. This model supports the notion that there was measurement invariance
across the two test forms (Hypothesis 2) and allows for the appropriate interpretation of
tests of mean differences in test scores and test perceptions. The practical importance of
the significant sz when moving from M2 to M3 was discounted for the reasons
discussed above.

Item Context and Test Perceptions

 

m stated that PJR will be enhanced when a cognitive test is framed in a face-valid
context. As can be seen in Table l, the correlation between test form and PJR is small
and nonsignificant, r=.07, p>.05. Thus, _I_I_1 was not supported.

fl; stated that framing a test in a job-related context will enhance justice perceptions
and that PJR will mediate this effect. However, the principal axis factor analysis
described above suggested that perceived fairness and face validity and perceived
predictive validity were not distinct constructs as measured by the items used in this
study. Thus, this hypothesis cannot be meaningfully tested because the present data do
not support the conceptual distinctions made between these types of test reactions.

Item Context and Test-Taking Motivation

33 stated that framing a test in a job-related context will enhance post-test motivation
after controlling for pre-test motivation, and that this effect will be mediated by PJR.
This hypothesis was tested with the hierarchical regressions reported in Table 3. These
regressions indicate that test form had no discernible effect on post-test motivation either

when entered after pre-test motivation (AR2=.003, p>.05) or after both pre-test motivation

and PJR (AR2=.003, p>.05). Thus, Hg was not supported.

50

__5 stated that the effects of PJR on motivation will be partially mediated by justice
perceptions. However, as was previously noted, the data presented here do not support
the conceptual distinction between PJR and fairness perceptions. Thus, it is not possible
to meaningfully test this hypothesis.

Table 3

Hierarchical Reggessions for Test Form, Perceived Job Relatedness and Pre-Test

Motivation as Predictors of Post-Test Motigtion (N=267).

 

Criteria and Predictors R2 (If AR2 Adf Af B

 

Post-Test Motivation

 

 

Step 1. Pre-Test Motivation .405 1 180.60* .637*

Step 2. Test Form .408 2 .003 l 1.29 .054

Post-Test Motiyation

Step 1. Pre-Test Motivation .410 2 91.886* .641*
Perceived Job Relatedness .072

Step 2. Test Form .413 3 .003 1 1.062 .049

* p < .05

__6_ stated that PJR will be related to internal attributions of test performance. As is
indicated by Table 1, there was a small, yet significant correlation between these variables
(r=.15, p<.05). Thus, H_6 was supported.

m stated that internal attributions will be related to post-test motivation even after
controlling for pre-test motivation. This hypothesis was tested with the hierarchical
regressions reported in Table 4. The correlation between internal attributions and post-

test motivation was practically small, yet statistically significant (see Table 1; r==.l l,

51

p<.05), and the results of the hierarchical regression do indeed support m, (AR2=.017,

p<.05). Thus, m was supported, though the size of the effect was very small.

Table 4

Hierarchical Reggessions for Internal Attributionsfi and Pre-Test Motivation as Predictors

of Post-Test Motivation (N=267).

 

Criteria and Predictors R2 df AR2 Adf Af B

 

Post-Test Motivation
Step 1. Pre-Test Motivation .405 1 180.60* .637*

Step 2. Internal Attributions .422 2 .017 l 7.67* .130*

 

*p<.05

_8_ stated that the relationship between self efficacy and test form would be mediated
by internal attributions. However, the bivariate relationship between test form and self-
efficacy was not Significant (r=.01). Thus, fl was not supported.

Md Test Perceptions & Test-Taking Motivation

H_9 stated that there will be significant race differences in P] R and that this difference
will be greater for a test that is framed in a job-related context. This hypothesis was
tested with the moderated regression analyses reported in Table 5. The results of these

analyses does not support this hypothesis. In particular, race was not significantly related

52

to PJR (AR2=.001, p>.05), and the race x test form interaction was non significant after
test form was also entered into the equation (AR2=.001, p>.05).
Table 5

Hierarchical Regressions for Rape. Test Form. and Race x Test Form as Predictors of

Perceived Job Relagedness (N=267).

 

Criteria and Predictors R2 (If AR2 Adf Af B

 

Perceived Job Relatedness

 

Step 1. Race .001 1 .340 .036

Step 2. Test Form .006 2 .005 1 1.357 .072

Step 3. Race x Test Form .007 3 .001 l .199 -.061
* p < .05

flQ stated that there will be a significant decrease in racial differences in post-test
motivation after pre-test motivation has been Statistically controlled. This interaction
between pre-test motivation and race as a predictor of post-test motivation was tested via
moderated regression, the results of which are reported in Table 6. As can be seen in the
table, the first step accounted for a significant amount of variance in post-test motivation
(R2=.4l , p<.05). The beta weights indicate that only pre-test motivation makes a

Si gnificant contribution to the prediction of post-test motivation. However, the

53

hypothesized interaction between race and pre-test motivation was not significant
(AR2=.003, p>.05). Thus, H10 was not supported.

Table 6

 

Hierarchical Reggessions for RaceaPre-Test Motivation and Race x Pre-Test Motivation

as Predictors of Post-Test Motivation (N=267).

 

Criteria and Predictors R2 df AR2 Adf Af B

 

Post-Test Motivation
Step 1. Race .413 2 92.75 .087
Pre—Test Motivation .652*

Step 2. RacexPre-Test Motivation .416 3 .003 1 1.282 .354*

 

*p<.05.

__1_1 stated that race and PJ R will interact to predict post—test motivation even after
controlling for pre-test motivation. This hypothesis was tested via the hierarchical
regression reported in Table 7. As can be seen in the table, only pre-test motivation
makes a significant contribution to post-test motivation. The role of the interaction term
was not significant (AR2=.000) after controlling for pretest motivation as well as the main
effects of race and PJR. Thus, flu is not supported.

_Ifl; Stated that the interaction hypothesized in _Hl_l would subsequently affect test
performance. Because no evidence of such an interaction was found, _H_lg is not

supported.

54

Table 7
Hierarchipal Regressions for Race, Pre-Test Motivation. Perceived Job Relatedness. a_n_c_1

Race x Perceived Job RelatednessJas Predictors of Post-Test Motivation (N=267).

 

 

 

 

Criteria and Predictors R2 df AR2 Ad Af B

Post-Test Motivation

Step 1. Pre-Test Motivation .405 1 180.60 .637*

Step 2. Perceived Job Relatedness .417 3 .012 2 2.75 .069
Race .085

Step 3. Race x Perceived Job .417 4 .000 j 1 .003 -.013
Relatedness

* p < .05.

_l_3 stated that race would affect test performance even after controlling for pre-test
and post-test motivation. In particular, it was thought that Black-White performance
differences would decrease once controlling for motivation, but that not all of this
difference would be accounted for by motivation. This hypothesis was tested via the
hierarchical regression analysis reported in Table 8, which indicates that race contributed
significantly to the prediction of test performance after controlling for the effects of

motivation (AR2=.19, p<.05). Thus, _I-Ll; was supported. It should also be noted that the

effect of both motivation measures on total test performance was relatively small

(AR2=.03, p<.05), and that the size of the effect described above is no different from the

zero-order R2 due to race alone (see Table 1; R2 = .21), suggesting that the effects of race

on test performance operate independently from motivation.

55

Table 8

Hierarchical Regpessions for Race, Pre-Test Motivation and Post-Test Motivation as

 

Predictors of Total Cognitive Test Score (N=267).

 

Criteria and Predictors R2 df AR2 Adi Ai B

 

Cognitive Test Score

 

Step 1. Pre-Test Motivation .031 2 4.191 -.187*

Post-Test Motivation .219*
Step 3. Race .222 3 .191 1 64.556 .446*
* p < .05.

14 stated that there would a Si gnificant decrease in Black-White differences in post-
test self-efficacy after pre-test self-efficacy is statistically controlled. This hypothesized
interaction was tested via the moderated regression analysis reported in Table 9. As can

be seen in the table, this interaction term is statistically significant (AR2=.026, p<.05).

Inspection of the means and standard deviations broken down by race (Appendix F),
however, indicates that the form of these differences was inconsistent with the predicted
form of this interaction in that Blacks had higher self-efficacy than Whites on both the
pre-test and post-test measures and that controlling for pre-test self-efficacy merely
served to reduce these differences. Thus, M was not supported because the direction

was Opposite that of the predicted direction.

56

Table 9
Hierarchical Regressions for Race. Pre-Test Self-Efficacy, and Race x Pre-Test Self-

Efficacy as Predictors of Post-Test Self-Efficacy (N=267).

 

Criteria and Predictors R2 df AR2 Adf Af B

 

Post-Test Self-Efficm

Step 1. Pre—Test Self-Efficacy .216 2 36.312 .477*
Race .041

Step 3. Race x Pre-Test Self-Efficacy .242 3 .026 1 8.935 .805*

 

*p<.05.

_l_5 stated that Black-White differences in self-efficacy will contribute to Black-
White differences in test-takin g motivation and thus test performance. Since LIE was not
supported, H_l_5 is also not supported.

Belief in Tests. Race, Tesflaking Moti_vation, and Test Reactions

_H_16 stated that people with a greater belief in tests 031T) will also make more internal
attributions about their test performance. As can be seen in Table 1, this hypothesis was
supported, as the bivariate correlation between these two variables was significant,
though small (r=. 15, p>.05). Thus, H_1_6_ was supported.

H_ll Stated that BIT would be significantly and positively related to justice
perceptions. Because justice perceptions were combined with the face validity and
perceived predictive validity perceptions, this hypothesis was tested by examining the

zero-order correlation between this composite PJR variable and BIT. As can be seen in

57

the table, these variables are indeed Significantly related (r=.32, p<.05). Thus, 1111 was
supported.

H_18_ stated that Black-White differences in BIT would partially account for Black-
White differences in fairness perceptions, a hypothesis that should be tested using the
composite PJ R variable given the factor analysis reported earlier. AS can be seen in Table
1, however, there were no Black-White differences in PJR. Thus, I_-I_1_8_ was not
supported.

_H_19 stated that BIT will be related to test-taking motivation both directly and via
examinees’ internal attributions about their test performance. As can be seen in Table 1,
however, BIT was not significantly related to motivation. Thus, mg was not supported.

HE stated that BIT should be an important factor mediating racial differences in test-
taking motivation, reactions, and test-performance. However, because there is no
evidence in the present study for Black-White differences in test-taking motivation or
test-reactions, the only hypothesis that can be tested here is whether controlling for BIT
reduces the relationship between race and test performance. The zero-order correlation
between BIT and total test score is small yet Statistically Significant (r=.16, p<.05).
However, controlling for BIT did not substantially reduce the correlation between race
and test performance (r=.46 and r=.43, respectively). These results indicate that although
BIT is indeed a significant predictor of test performance, controlling for Black-White
differences on this variable does not substantially reduce Black-White differences in test

performance. Thus, _Ifl) was not supported.

58

Summg of Indep_endent Hypothesis Tests

Based on the results reported above, Table 10 lists each hypothesis, briefly describes
how each was tested, and indicates whether each was supported. The results in the table
suggest that item context had no effects on the dependent variables under study. The
relationship between internal attributions and motivation was supported, though there was
no support for the importance of race as a determinant of PJR and motivation, nor for the
hypothesis that PJR and motivation would be significantly related to test performance.
However, race was identified as a significant determinant of belief in tests, which in turn,
significantly determined PJR. The hypotheses and results described above were also
tested more comprehensively using structural equation modeling. The results of these
analyses are reported below.

Tests of the overall model

Based on the results of the hypothesis test reported above, it is very likely that the
conceptual model as presented in Figure 1 will not be supported by these data.
Nonetheless, this model was tested, although two modifications were made to the original
model before proceeding: (1) the two interaction terms were removed due to the lack of
support for these hypotheses and the difficulty of modeling interaction terms with
structural equation modeling, and (2) perceived job relatedness and procedural justice
perceptions were combined to form an overall reactions measure based on the factor

analysis described above.

59

Table 10.

Summm of Hypotheses and Results.

 

 

 

 

 

 

 

 

 

 

 

 

 

H # Hypothesis Tested Via Support

1 P] R can be enhanced by framing cognitive Correlational No
ability test items in a business context if the analysis.
test is used for selecting applicants for
business-related positions

2 Framing cognitive ability test items in a job Multiple groups Yes
related context will not alter the underlying structural equation
constructs measured by the test. modeling

3 A test framed in a job related context will N/A1 N/A1
enhance justice perceptions, an effect that
should be mediated by PJR

4 Framing a test in a job-related context will Hierarchical No
enhance post-test motivation after controlling Regression
for pre-test motivation, an effect that will be
mediated by P] R.

5 The effects oi PJR on motivation will be N/A1 N/A1
partially mediated by the perceived procedural
justice of a selection instrument

6 P] R will be related to internal attributions of Correlational Yes
test performance Analysis

7 Internal attributions will be positively related to Hierarchical Yes
test-taking motivation even after controlling for Regression
pre-test motivation

8 Examinees taking a test lower in PJR will have Correlational No
higher SE (after pre-test SE is controlled) Analysis
regarding their test performance than
examinees taking a parallel test high in PJR, an
effect that should be mediated by internal
attributions

9 Blacks will rate tests as less job related than Hierarchical No
Whites. There should be smaller Black-White Regression
differences on a test framed in a job related
context

10 There will be a significant decrease in Black- Hierarchical No
White differences in post-test motivation after Regression
pre-test motivation is statistically controlled

1 l PJR will interact with race in affecting test- Hierarchical No
taking motivation such that Black-White Regression

 

differences are smaller for a cognitive ability
test higher in PJ R (even after controlling for

 

 

 

60

 

 

 

 

 

 

 

 

 

 

 

 

 

H # Hypothesis Tested Via Support
pre-test motivation)

12 Motivation will mediate the relationship N/A2 N/A7
between the interaction presented in
Hypothesis 11 and test performance

13 Race will affect test performance even after Hierarchical Yes
controlling for pre- and post-test motivation Regression

14 There will be a significant decrease in Black- Hierarchical No
White differences in post-test SE after pre-test Regression
SE is statistically controlled.

15 Controlling for SE, which Should be higher for MA1 N/AI
Whites than for Blacks, Should reduce the
Black-White difference in motivation and test
performance (even after controlling for pre-test
SE and motivation)

16 BIT will be related to internal attributions of Correlational Yes
test performance Analysis

17 BIT will be related to procedural justice Correlational Yes
perceptions (tested with P] R because justice Analysis
perceptions were not distinct from PJ R)

18 There will be significant Black-White N/AT N/A‘
differences in BIT that will partially account
for Black-White differences in procedural
jUSIICCJJCI'CCPIIODS

19 BIT will be positively related to motivation Correlational No
both directly and as mediated by (a) the PJR of Analysis
the test and (b) internal attributions examinees
make regarding their test performance

20 Controlling for BIT will (a) reduce Black- Correlational NO
White differences in PJR; (b) significantly Analysis

 

decrease the interactive effects of race and PJ R
on motivation; and (c) test performance.

 

 

 

 

r Not applicable because face and predictive validity perceptions were not distinct in the

present dataset. 2 This hypothesis was not supported because an earlier hypothesis upon

which it was based was not supported.

Notes: PJ R = Perceived Job Relatedness; BIT = Belief in Tests; SE = Self-Efficacy.

61

Three indicators were created for each latent construct by assigning every third item
of each scale to an indicator. The results from the initial model test revealed that the path
from test form to test reactions was near-zero, and therefore this path was deleted (this is
also consistent with the univariate results presented above). Thus, the next model tested
is the one presented in Figure 2. Note that this model is identical to the one in Figure 1
except for the changes noted above. As was discussed above, multiple fit indices were
used to evaluate the fit of the model. The model depicted in Figure 2 fit the data quite
well as indicated by the practical fit indices (28038) = 276.21, p<.01; RMSEA = .06; GFI
= .90; CFI = .95; NNFI = .94; SRMR = .07).

Direct Effects. Figure 3 depicts the model presented in Figure 2 with standardized
path coefficients for each of the hypothesized effects (other relevant model parameters
can be found in Table 11). Significant paths are indicated by an asterisk. Tests of
significance were performed on the unstandardized parameters estimated from the
covariance matrix. AS would be expected given the results reported above, these
coefficients reveal mixed support for the hypotheses presented in this study. Consistent
with expectations, Race was significantly related to BIT, Performance, and Self-Efficacy
(although opposite to the predicted direction; Blacks had slightly higher Self-Efficacy
than Whites). Contrary to expectations, Race was related to neither Motivation nor PJR.

As expected, BIT was significantly related to PJR, Internal Attributions and Motivation.

62

 

Toz<szammfl

 

Scam accuse E 353 on 9 3320028 0o _oeoE 336% .N Semi

 

 

 

 

 

 

 

20.2252
”520.595?
a<zmmez_
mhmme
m2 2. amsmm
_>o<o_damdme mos".

 

/!

63

.3. V a * 29:25:22 3255093 5.« 335:8 Amy—w: wonfiaeefim beau—0:50 .m 2:?”—

 

\ll

ZO.._.<>_.PO_2

 

 

w20_._.Dm_m._.-_.< aw _.
4<me._.z.

 

*2.
. memme
an 2. dmsmm
Ed

50.-

 

awn.

$95.29“.me L _ .- mo<m

 

 

 

 

 

Fozszmodmmb

J/I L-m.

 

Table 1 1.

Completely standardized factor loadings and error variances of indicator variables on

endogenous latent vapiables (AYJand 98, respectively).

 

Latent Endogenous Variables

 

 

 

 

 

 

Error
Indicator IA SE PJ R MOT BIT PERF Variances
SE1 -- 0.83 -- -- -- -- .31
SE2 -- 0.91 -- -- -- -- .18
SE3 - - 0.86 - - - - - - - - .27
PJR] -- -- 0.89 -- -- -- .21
PJR2 - - - - 0.93 - - - - — - .14
PJR3 -- -- 0.91 -- -- -- .17
MOT] -- -- -- 0.93 -- -- .13
MOT2 -- -- -- 0.91 -- -- .18
MOT3 - — - - - - 0.89 - - - - .20
BITl -- -- -- -- 0.83 -- .31
BIT2 - - - - - - - - 0.79 - - .37
BIT3 -- -- -- -- 0.57 —- .67
PERFl - - - - - - - - - - 0.72 .48
PERF2 - - - - - - - - - - 0.83 .31
PERF3 - - - - - - - - - — 0.74 .45
1A1 0.20 -- -- -- -- -- .96
1A2 0.79 - - - - - - - - - - .38
1A3 0.88 - - - - - - - - - - .23
Notes: All factor loadings and error variances are significant at p<.05. - - = constrained

to zero. IA: Internal Attributions; SE = Self-Efficacy; P] R = Perceived Job Relatedness;

MOT = Motivation; BIT = Belief in Tests; PERF = Test Performance.

65

PJ R was moderately related to Internal Attributions, which in turn was moderately related
to Self-Efficacy, though internal attributions were not related to Motivation as
hypothesized. Self-Efficacy was strongly related to Motivation, which in turn was only
weakly related to test performance.

Indirect Effects. The indirect effects that are most strongly relevant to the ideas
discussed in the present study are those that account for the effects of Race on
Performance, especially as compared to the direct effects. Not surprisingly, the direct
effects of Race on Performance were substantial. However, the indirect effects were
trivial (.01; see Table 12 for the parameter estimates for all the direct and indirect effects).
This occurred for a number of reasons. Primarily, Motivation is the variable that
mediates the effects of Race and other variables on Test Performance, though it was only
very weakly related to performance. Second, of the paths that one might expect to yield
among the highest coefficients (Race to Self-Efficacy and Race to PJR), one yielded a
Si gnificant relationship opposite the predicted direction, and the other a non-Significant
relationship, respectively. Third, there were several mediating variables between race and
performance presented in this conceptual model. Adding additional mediated steps
should generally decrease total indirect effects because these effects are the result of the
products of all relationships mediating the relationship between the focal variables.
Nonetheless, these variables were included because such relationships are of theoretical

interest.

66

 

 

 

 

 

 

$5. 525 52R.
*wm. 525 55..-
ME 9 Em
000a awn. x 33 m.. Em a;
m.. 3625 52:5. .30...
50.- 525 525
A. to... a 00. 525 .«5...
ME 2 83.
an. x mm. x 33 .0. ME .55 «S
RN. x 50.-. 8.. ME a.>
8.. x 3.. 00. Em a.>
wo. $025 52:5. 55.5
-- 525 52.n—
w0. 525 55...
S 2 33.
. .. $625 52:5. 55...
*w . . 52.5 525
c. on. 525 .36...
S o. Em
i: 8625 52:5. 55...
5m. 525 525
0 ram. 525 .85...
S 2 ME
.25... E 55505 555553855555 .8625 55585 can 5.5.22,

 

 

 

 

 

 

5.55

 

 

 

:5 2: c. 2.x 0.5.29 .5 .2852. 3625 52:5. .28 .525 E :38 .255 £525 .98 52.2555 .5 325:6

.N. 0.25...

67

 

Z.-

525 525
525 :5...
PCS. 2 ME

 

Cm. x Bram. x 3.. co.
Coffin. x 33 co.
A.o.-*w.. x v0. 8.

3.. x 3... mo.
Cm. x 2:. ac.-
mo.-
8.-
3.-

mm .s rm?— a.>
5 J3... as
s vim a.>
hm a.>
Mm a.>

892.5 52.2.. 55%

525 525

52.5 .50....

HOS. 2 8mm

 

in.
in.

38% 62.2. .35
“8:0 825
525 .83.
.52 9 mm

 

A.m.th.va..
2.
.o.-
2.

mm m.>
£025 526:. .55...
525 525
525 .80..-
HOS. o. <.

 

no.

no.

a.

cor-w... 8.
.3. x :..&$ 5.-
2 7&3 3.-

5.

E.

S.

S «S
S .mE 2>
ma «3
Book—D .02:va _QHOH.
«00:0 “UP—mn—
820 .28.
.82 2 :m

 

38% 52.2. .58

 

 

8:“; :

 

558.50 55.22% 55.23.85

 

$025 55583 can 5.5.5.»

 

 

68

 

*VM.
*3”.

385 82...... .39.
.85 .85
525 .55...

.5. o. 83.

 

Ch. x am. x mm. 3.
Ann. x m... we.
Z.

Z.

5 .5. a...
S «S
385 .85... as...
.85 .85
.85 .9...
mm o. .5

 

*hm.
*NM.

385 .85... .50...
.85 825
.85 :8...

mm o. <.

 

.2483 ...
...

I.

S ~.>
985 .85... .9...
.85 .85
.85 .3...

mm 2 ma

 

.2. x .2. x 3.. 8.
.R. x a. x S.-. 8.
.52. x mm. x 3... .o.

8.

L. .-

E .-

<. H... a.>
5 J5. a.>
5 J5. .....m. m.>
38.5 52...... 55...
525 525
525 55...
Ma 2 3mm

 

:m. x R. x a. no.
scram. 8.
8.

mm .<. a.>
<. a.>
3025 52...... 55...

 

 

5.8.. :

 

mean—.230 .oaoemhdnw GONm—uaflcflam

 

.525 55589. .25 5.5....3

 

 

69

 

+5.. 8 MP.

 

 

 

.2. x .m. x R. x a. x .5. 8. .82 .mm .5 .m... «S
.2... .m. in. S... 8. 52.5.3 «5
.2. x 5.-.»... 8. .82 .<. «S
.2. x ...-_....m.. 8. 8.). .5. «S
.2. x 2.. 8. .52 «S
S. 385 82...... .38.
.85 .825
8. .85 Bob
5.. o. .5
.2. x .m. x 3.. 8. .52 .mm «5
.2. x 8.-. 8. 5.). «S
m... 385 32...... .2....
.85 825
8. .85 .38.
5“. o. <.
.2. x .m. x R. x am. x S.-. 8. .82 .mm .5 .5. 8S
.2. x Scam. x 8.-. 8. .52 .<. .5. «S
.2. x ...-...S.-. 8. 5:. .5. <5
.2. x .n. x R. x a. x mm. x 3... 8. Hos. .% .<. .5. Em «S
.2. x 5.8%. x mm. x 3... 8. .52 .<. a... Em «S
.2. x .o......... x 3... 8. .52 .S ....m «S
.2. x 2. x .5. 8. 5.). H5 .5
.2. x .m. x t.-. 5.- .82 .mm «5
.2. x 2.. 8. .52 «S
.o.- 285 62...... .soe
Ln. .8...» .85
cm. .85 .909
run. 2 83.

 

 

.88.: ..

 

558.50 55:55.. 3n...5..=3m

 

35...» 85.8%.. .28 332:3

 

 

7O

5 5.2.5:. 2550...... 55 555 555.. 55.5 £5.55 55 .3 5.5. .252... 5: 25 3.25 5.5. 05 :5... .5550
5552...... .mo.v.. u * 5.550%... n I 55.5553. 5:55. n S ”55550:... .5... n .5. Soweto-bum H mm 5.5..-

:. 555 u H... 55:55.3. 5.. .5>.o2on. u M... 50.5353. u HOS. 50.55:... ..:5 w:...::2 o. 2... 55.5 .55. 55 5:5
3.5.55: 5: 55 55.5 52... .55 55.5 525:. .55. 55 5 E... 55 .55 55.5 525:. .55. 55 53.5 3.5555: 5:

55 50.5 525:. 5.5 5 2:52.58 55 .5 58 2.... 5.5:. 1.55.1. 55 :. 55:55 5: 53 55.5 5555:. ----..5..oZ

 

*2.
*2.

25.5 525:. .55..-
55.5 525
55.5 .55...

.5. o. ..O.>.

 

.2. x .m. mo.
no.

no.

..-O.>. 5.>
25.5 525:. .55..-
525 52.0
55.5 .55.-
.5. 0. mm

 

.2. x .m. x R. x 3.. .o.
.2. x 5.55.. 8.
.2. x .7. m...-

5..

5.-

H02 .mm .5 5.>
..-O.>. .<. 5.>
.-O.>. 5.>

25.5 525:. .55..-

55..5 52.0

55.5 .55...

 

5.8-. :

 

 

55555 55555.. 5555555

 

25.5 55.555 ..5 535.55

 

 

71

5.55:5 1.55.. 55 52. .5552 553 .55 555.5

55.55:. 55:5 .2555 £35... 55 :. 55.55:. 553 55555 .535. .._:o. 2:555 .5.-2. 5 .5255. 55.5555.

72

Since there were many indirect effects, and because most of them are very small (see
Table 12), only a select few will be mentioned here. One of the primary expectations of
this study was that there would be Black-White differences in PJR. However, as can be
seen in Figure 3, the direct effect of race on PJR was very small and non-significant. As
can be seen in the table, however, race effects BIT which in turn is related to PJR; this
indirect effect is moderate (. 13). Another primary expectation was that there would be
Black-White differences in Test-Taking Motivation, although the direct effect depicted in
the figure indicated that this was not the case. The indirect effect of Race on Motivation
via Self-Efficacy was actually negative, though small (-.09), indicating that Blacks
actually had higher Motivation than Whites. Lastly, the indirect effect of Internal
Attributions on Motivation via Self-Efficacy was significant (.19), resulting from the fact
that Internal Attributions were weakly related to Self-Efficacy, and Self-Efficacy was
strongly related to Motivation. However, because of the weak relationship between
Motivation and Performance, this effect is minimal if one uses Test Performance as the
dependent variable of interest. The results of the LISREL analysis, as well as those of the
univariate and multivariate analyses reported above, are discussed in the following

section.

73

Chapter 4

DISCUSSION

This study sought to address a number of issues relating to the relationships between
test perceptions, test-taking motivation, race, and test performance. The context of
cognitive test items was manipulated while controlling for substantive item content in an
effort to manipulate test perceptions. The results of this study suggest that item context
does not affect test-perceptions, test-taking motivation or test performance on cognitive
test items. Additionally, there was no evidence of race differences in test perceptions, nor
was there evidence that test-taking motivation or perceived job relatedness are related to
test performance. These null results are discussed in light of previous research and theory
in the following sections.

Test perceptions & test—taking motivation and item context

This study sought to manipulate test perceptions by framing cognitive test items in a
job—related context. In particular, these perceptions were conceptualized as being
composed of perceived face and predictive validity and procedural fairness. Upon
examining the factorial structure of the data, it became apparent that these three
perceptions were empirically indistinguishable, which should not be altogether
unexpected given their nature and the test content. Face and perceived predictive validity
may be distinct conceptually though not in the minds of examinees. Procedural justice
perceptions, on the other hand, probably cannot be formed without reference to the

specific application of a particular test. In other words, determining whether using a test

74

is fair should be strongly related to the purpose for which it is intended. For instance, one
might have difficulty answering the question of whether the SAT is fair unless one is told
that it is designed to predict college grades rather than automobile driving competency,
for instance. Given the test used here was designed to predict employee performance, and
that the examinees were explicitly told that this was the intended purpose, it is not
surprising that these scales exhibited little discriminant validity.

The results described above indicate that although the substantive content of the test
was indeed preserved, there were no meaningful differences in test perceptions across the
two test forms. Subsequent inspection of the differences across individual item means
indicated that there were more positive reactions to the job-related test form, but these
differences were generally very small (i.e., less than one-fifth of a standard deviation).
There are a few possible explanations for why this occurred. First, it is possible that the
manipulation employed was too subtle. However, researchers with training in [/0
psychology upon seeing the two test forms remarked that these manipulations were not
subtle at all - they felt that it was in fact a very strong manipulation. Second, it is
possible that college students have been presented with cognitive ability items lacking
face-valid contexts for so long that they have become accustomed to such features and do
not react negatively to additional items presented as such. Alternatively, the students may
have sensed that the relevance of cognitive ability for these jobs was high, thus fostering
positive impressions. Thirdly, it is possible that because cognitive items are so familiar to

most college students that ’surface’ features do not really mask what is being measured by

75

the items. However, neither of these explanations are consistent with the results of the
pilot studies conducted for this study, so it remains unclear as to why this happened.

Given that the manipulation failed to affect test perceptions, it is not surprising that
there were also no observed results regarding test-taking motivation or self-efficacy. This
is because the underlying rationale for motivational differences across the two test forms
was that if one or the other is perceived as less job-related then examinees will also be
less motivated to try their best to do well. However, not only were there no motivational
differences across the two test forms, but there was also a non-significant relationship
between perceived job relatedness and test-taking motivation. This is inconsistent with
the existing body of literature on test reactions (e.g., Chan 1997; Chan & Schmitt 1997;
Chan et al., 1997; Chan et al., 1998) and there is no clear explanation as to why this might
have occurred in the present study. This is especially puzzling given that two of the
studies cited above had measures, samples, and testing situations that were highly similar
to those in the present study. Further, generally speaking, when one measures perceptions
in roughly the same domain via self—report, one would expect to find significant positive
relationships among the measured variables due to common method bias, if nothing else.
Perceived 1 ob Relatedness. Test-Taking Motivation, and Test Performance

The present study also sought to investigate the relationships between perceived job
relatedness (PJ R), test-taking motivation, and test performance. Previous research has
indicated that these variables should all be moderately and positively related, although the
direction of causality is still as of yet unresolved. These relationships were not borne out

in the data presented here. In particular, P] R, test-taking motivation, and test

76

performance were only very weakly related to each other according to the LISREL model.
This is somewhat puzzling considering at the very least, that test performance may cause
both post-test reactions and test-taking motivation via self-serving bias mechanisms
(Chan et al., 1998). Perhaps one explanation for this may be that the test was too
difficult, and that accordingly, neither motivation nor test reactions were related to test
performance. However, if the hypothesis that self-serving mechanisms of post-test
reactions is true, one would still expect a difficult test to yield performance scores that are
related to reactions; specifically, if one does poorly on a test one may report very negative
reactions to it due to ego-enhancing mechanisms. Aside from this explanation, however,
it is difficult to come up with a reasonable theory about why this might have happened
given that these relationships have been reported in a number of empirical studies (e.g.,
Chan, 1997; Chan et al., 1997; Chan & Schmitt, 1997; Chan et al., 1998; Smither et al.,
1993, Schmit, 1996).

Internal Attributions, Self-Efficacy, Test-Taking Motivationhand Test Performing;

As was expected, internal attributions were indeed positively correlated with test-
taking motivation, test performance, and self-efficacy, which were all positively related to
each other. These findings are consistent with the literature that indicates that the extent
to which one has a more internal belief about the determinants of success or failure will
be positively related to both higher performance and higher feelings about one’s efficacy
about doing well. This is likely because people who believe they do not control the
outcome of a particular task might be less likely to exert effort because it will be

perceived as a waste. Similarly, people with more internal attributions also feel more

77

efficacious about their ability to do well probably because their performance is seen as
being at least partly dependent on their own effort and abilities. This is consistent with
the empirical and theoretical literature on short-term task performance as well as longer-
terrn outcomes such as academic achievement. However, it should be noted that based on
the LISREL analysis reported above, the relationship between internal attributions and
motivation was fully mediated by self-efficacy, and the magnitude of this effect was not
very large (.19).
Race and BIT. Test Reactions. Test- aking Motivation, Self-Efficacy, and Test
Performance

The primary goals of this study were to investigate the extent to which differences in
test reactions and test-taking motivation related to test performance according to one’s
race, and to investigate the extent to which these differences might be influenced by item
context. Contrary to previous research, there was little evidence for Black-White
differences in test perceptions (Chan, 1997; Chan et al., 1997) or test-taking motivation
(Arvey et al., 1990; Chan et al., 1997). This is also inconsistent with work by Steele that
suggests African Americans should react to cognitive tests more negatively than Whites
because of the risk of fulfilling stereotype-consistent performance patterns. In addition,
there were no Black-White differences in post-test-taking motivation, although Blacks
actually had slightly higher pretest self-efficacy and motivation as compared to Whites.
Although this is certainly unintuitive given Steele’s recent work, it is possible that
specifically recruiting Blacks and testing them separately from Whites makes them feel

better about their chances of doing well. This may be because if no other White

78

examinees are present during the testing session they may feel they have a better chance
of doing well, at least relative to the rest of the examinees. This is consistent with a
recent study that recruited and tested Blacks under similar circumstances which found
that Blacks actually had higher self-efficacy than Whites (Jennings, 1998).

Another goal of this study was to determine whether framing cognitive test items in a
face-valid context would reduce Black-White differences in test-taking perceptions, and
hence motivation and performance. Because no main effect for the test form
manipulation was detected, any interaction one might find would necessarily have to be
disordinal in nature, something very unlikely to occur when dealing with Black-White
differences in cognitive test performance. Thus, given that the manipulation of the item-
context did not work in the present study, detecting an interaction type that might be more
reasonable (i.e., an ordinal interaction) becomes impossible. Reasons for the lack of such
a main effect were discussed above; however, even had the manipulation worked, test
reactions and test-taking motivation were still unrelated to test performance, making it
unlikely that such an interaction effect would be detected. These pattern of results are
certainly inconsistent with those of Chan and Schmitt (1997), who found that controlling
for face validity perceptions on a situational judgment test substantially reduced Black-
White differences in test performance. It may be that given high levels of motivation, test
reactions are not important determinants of test performance. However, the present study
found that motivation was reasonably high, but certainly not high enough to yield a non-

zero relationship between test reactions and performance.

79

Another set of hypotheses in the present study were aimed at determining whether
controlling for pre-test motivation and self-efficacy would substantially reduce
differences in reported post-test motivation, self-efficacy and performance. The
reasoning underlying these research questions was that Blacks probably have more
negative motivational sets going into these tests (e.g., Steele & Aronson, 1995), and that
controlling for these pre-test differences should thus reduce post-test differences.
However, these expected relationships did not emerge because Blacks were found to have
higher pre-test motivation and self-efficacy.

This study also sought to examine how prior attitudes about employment testing
might mediate Black-White differences in PJR. Although there was no evidence for the
direct effects of Race on PJR, there was a small effect of Race on PJR (.13) when one
considers these effects as mediated via BIT. Though this effect was certainly smaller than
would be expected given past research, it points to the importance of one’s prior beliefs as
a potential key determinant of P] R rather than simply examining the direct effects of race.
Given research by Steele and Aaronson (1995) and Helms (1992), it is likely that a much
larger mediated effect might be found if one asked about the use of cognitive tests for
employment rather than employment tests in general. This is because it seems likely that
Blacks may have a particular dislike of cognitive tests due to their well-documented
performance differences as compared to Whites. Future research should investigate this

possibility.

80

Implications for Theory. Research, 3M Practice

Despite the null results reported in the present study, there are perhaps some lessons
to be learned that relate to theoretical, empirical, and practical matters in personnel
selection. Conceptually, the present study found that fairness perceptions were not
distinct from perceived predictive validity and face validity perceptions. However, the
conceptual literature generally construes these perceptions as determinants of fairness
perceptions rather than equivalent constructs. When evaluating an entire selection
system, it may well be the case that these perceptions are only one of many determinants
of justice perceptions, but that when evaluating a particular test these perceptions are
equivalent to face and perceived predictive validity perceptions. It would be useful for
future conceptual literature on this topic to examine antecedents of fairness perceptions
for specific tests rather than simply selection systems as a whole. This is especially true
given that oftentimes the first exposure one has to an employer consists of a selection
measure of one sort or another.

Another implication of this research relates to the lack of racial differences in test
reactions. Although some research indicates that these differences should be pervasive
(e.g., Arvey et al., 1990; Chan et al., 1997; Chan 1997;, Chan & Schmitt, 1997; Steele &
Aaronson, 1995), other research (e.g., Ryan, Sacco, McFarland, & Kriska, 1998; Schmitt,
Chan, Sacco, McFarland, and Jennings, 1998), as well as that reported here, suggests that
this may not be the case. One possibility is that recruiting and testing Blacks separately

may lead them to have more positive reactions than they otherwise might have. An

81

analogous theory may also explain why they reported higher self—efficacy and motivation.
Future research should not only investigate the practical impact of these effects, if
replicated, but also the conceptual implications as well. For instance, findings such as
these have implications for the current interpretation of Steele’s recent work. Future
research, both in the field and the laboratory, should explore these avenues of inquiry.

These lack of racial differences also have implications for practitioners. In particular,
they suggest that one need not be overly concerned with negative reactions to cognitive
testing among African Americans in particular. However, given the contradictory results
reported above, this issue is certainly not settled and deserves additional attention.

Lastly, the lack of differences in perceptions across the two test forms suggests that a
lack of face validity in and of itself (i.e., aside from item content) may not have the
negative impact on test reactions that one would traditionally expect. This suggests that
there may be less reason to take the trouble of reframing cognitive ability tests into job-
related contexts depending on the particular application — a generic instrument should

yield similar reactions among examinees.

82

APPENDICES

83

APPENDD( A

84

APPENDIX A

Face valid test form

LOGICAL REASONING - 15 ITEMS - 10 MINUTES

 

This is a test of your ability to tell whether the conclusion drawn from certain
premises is correct or incorrect. For each problem, the various conclusions
drawn from the premises may or may not be correct. You are to assume that
the premises in each problem are true. Without assuming the truth or falsity of
anything in addition to the information given in the premises. decide which
conclusion is correct. There is only one correct conclusion. Bubble in the
circle on your answer sheet corresponding to the answer you have chosen.

There is no penalty for guessing, so please mark an answer for each item.

Conditional Reasoning
1. Premises
If the president sells his stock tonight then he will make a profit.
The president did not make a profit.
Conclusion
a. The president sold his stock tonight.
b. The president did not sell his stock tonight.

c. The president may or may not have sold his stock tonight.

85

APPENDIX A

2. Premises
If Linda joins the board of directors then she will improve her financial position.

Linda improved her financial position.

Conclusion
a. Linda joined the board of directors.
b. Linda did not join the board of directors.

c. Linda may or may not have joined the board of directors.

3. Premises

If the wire is removed then the telephone switches will be damaged.
The wire is not removed.

Conclusion

a. The telephone switches were damaged.

b. The telephone switches were not damaged.

c. The telephone switches may or may not have been damaged.

4. Premises

If the client is an intemet service provider, then the client will receive a discount.

The client is an intemet service provider.

86

APPENDIX A

Conclusion
a. The client will receive a discount.
b. The client will not receive a discount.

c. The client may or may not receive a discount.

5. Premises

If the company does not offer a paid vacation, then the employees will reject the
contract.

The company offered a paid vacation.

Conclusion

a. The employees will reject the contract.

b. The employees will not reject the contract.

c. The employees may or may not reject the contract.

6. Premises
If the firm does not secure the contract, then the employees will not receive a bonus.
The firm secured the contract.

Conclusion

21. The employees will receive a bonus.

b. The employees will not receive a bonus.

C . The employees may or may not receive a bonus.

87

APPENDIX A

7. Premises
If the loan is not approved, then work on the communications project cannot begin.
Work on the communications project has begun.

Conclusion

a. The loan was approved.

b. The loan was not approved.

c. The loan may or may not have been approved.

8. Premises

If the two intemet service providers merge, then the high speed wiring will not be
installed in December.

The high speed wiring was not installed in December.

Conclusion

a. The intemet service providers merged.

b. The intemet service providers did not merge.

c. The intemet service providers may or may not have merged.

88

APPENDIX A

Syllogistic Reasoning

9. Premises

All employees who received a pay raise are spending more time on vacation.
All employees who are promoted received a pay raise.

Conclusion

a. All employees who received a pay raise are promoted.

b. All employees who are promoted are spending more time on vacation.
c. All employees who are spending more time on vacation are promoted.

d. All employees who are spending more time on vacation received a pay raise.

10. Premises

Some employees who completed the training are promoted.

Every employee who completed the training performs the job well.
mm

a. Every employee who is promoted did not complete the training.
b. Every employee who is promoted completed the training.

c. Some employees who perform the job well are promoted.

(1. Every employee who performs the job well completed the training.

89

APPENDIX A

l 1. Premises

None of the employees who attended the picnic are from the accounting department.
All employees who had their salaries increased attended the picnic.

Conclusion

a. All employees whose salaries were not increased are from the accounting
department.

b. All employees from the accounting department did not have their salaries’
increased.

c. All employees who participated in the picnic had their salaries increased.

(1. Some of the employees who participated in the picnic did not have their salary

increased.

12. Premises

Some applicants who passed the selection test are selected for the job.
None of the MBA graduates are selected for the job.

mm

a. Every MBA graduate failed the selection test.

b. Some of the MBA graduates passed the selection test.

c. Some applicants who passed the selection test are not MBA graduates.

(1. Every applicant who is selected for the job passed the selection test.

90

APPENDD( A

Inferential Reasoning
This is a test of your ability to tell whether the conclusion drawn from certain
premises is correct or incorrect. For each problem, the various conclusions drawn

from the premises may or may not be correct. You are to assume that the premises

in each problem are true. Without assuming the truth or falsity of anjahing in

 

 

addition to the information given in the premises, decide which conclusion is

correct. There is only one correct conclusion.

13. More employee theft occurs after dark than during daylight hours.

a. A decrease in work-related activities tends to increase criminal activities.

b. Employee theft after dark is generally more profitable than during daylight
hours.

c. After dark, most employees engage in theft as their primary form of criminal
behavior.

d. The chance of an item being stolen at work is lower during daylight hours than
it is after dark.

e. More employee theft occurs after dark since there are fewer people at work to

see them steal.

91

APPENDIX A

14. Today it generally costs less to buy a modem than it cost to buy one three years
ago.

a. Modems are made using less expensive materials today.

b. The purchasing power of modern buyers has risen during the past three years.
c. The cost of modems has fallen during the past three years.

(1. Companies are buying more modems today than three years ago.

e. Modem buyers three years ago are generally richer than modern buyers today.

15. The combined presence of the president, secretary, and the presence of at least
50 members at the annual shareholders meeting is necessary for the approval of the
budget estimate.

a. The budget estimate cannot be approved without the members’ support.

b. The president and the secretary are the two most important persons in the
budget estimate approval process.

c. If the budget estimate was approved, then the secretary was present.

d. Budget estimates are only approved at annual meetings.

e. If both the president and the secretary support the budget estimate, then
approval is most likely as long as the number of members present at the annual

meeting is at least 50.

92

APPENDIX A

QUANTITATIVE REASONING - 14 ITEMS - 16 MINUTES

 

The next 14 items assess your ability to use quantitative information to make
decisions. Read the description of each set of information and choose the best
answer to each question. Bubble in the circle on your answer sheet corresponding
to the answer you have chosen. There is only one answer to each item. There is no
penalty for guessing, so please mark an answer for each item.

The table below contains a breakdown of the way in which three levels of

workers in Company A spend their 40-hour work week.

 

Level 1 Level 2 Level 3
(No.=100) (N0. = 80) (N0. = 20)
Management 10 5 20
Training 20 5 2
Clerical 6 10 6
Customer Service 4 20 12

16. For these three groups of employees, what percentage of work time is spent with

customer service?

a. 30
b. 28
c. 26
d. 24
e. 20

93

APPENDIX A

The table below is foguestions l7 & 18.

The table below is a description of the speed at which three different telephone
circuits can process phone calls in specific locations along with the cost of each of
these circuits.

Location 1 Location 2 Location 3 Cost

 

Circuit A 10 sec. 180 sec. 200 sec. $3500
Circuit B 12 sec. 200 sec. 300 sec. $3000
Circuit C 20 sec. 400 sec. 320 sec. $2000
Number of calls per 2000 1500 1000

Week

17. How many Model C circuits can be purchased for the same price as 10

Model B circuits?

a. 10
b. 15
c. 20
d. 24
e. 30

94

APPENDIX A

18. How much time would be saved in calls at Location 3 each week by
purchasing Model A circuits over Model C circuits?

a. 20,000 minutes

b. 2000 minutes

c. 100,000 seconds

(1. 12,000 seconds

e. 200 minutes

19. The company vice president’s office, which is three times as large as the
junior accountant’s office, is 18 feet by 20 feet. The length of one side of the junior

accountant’s office is 10 feet. How long is the other side of the junior accountant’s

office?
a. 36 feet
b. 18 feet
c. 12 feet
(1. 10 feet
e. 8 feet

95

APPENDIX A

20. The company borrowed $2,250,000 to create a new product line. They paid
10 percent interest on money borrowed from a bank and 8 percent from a group of
investors. Yearly interest paid to both the bank and the private investors was the
same. How much was borrowed from the bank?

a. $1,500,000

b. $ 1 ,250,000

c. $1,000,000
(1. $750,000
e. $500,000

21. Corded telephones comprise 20 percent of revenues; wireless
communications products, 75 percent of revenues; and intemet hardware, 5 percent
of revenues. If 1000 units of these three products are sold each month, how many

corded telephones are sold in a year?

a. 2400

b. 600

c. 200

d. 50

e. The number of corded telephones sold cannot be determined.

96

APPENDIX A

22. A company has an inventory of over 4,800 phones it wants to reduce to
3,000. At the current price, the company sells 500 of these phones a month while
producing 600 phones per month. Assume the company decided to reduce
production as a means of reducing inventory, by how much would they have to
reduce production each month to reach the target of 3000 inventory items in a year?

a. 320 phone units

b. 267 phone units

c. 250 phone units

d. 200 phone units

e. 100 phone units

97

APPENDD( A

23. Assume that you have to acquire a building with work space for a minimum
of 100 technical staff. Each person requires a minimum of 120 square feet of work
space and a computer hookup. You’ve located the following five possible buildings
described below in the immediate area and must decide which would be most

economical over a three year period. Which would you choose?

 

Year Square Feet Cost: Computer Estimated cost
Built Square Feet Stations in per new
/Y ear place hookup
a. 1968 15,000 $20.00 20 $5000.00
b. 1988 12,000 $30.00 80 $2000.00
c. 1972 18,000 $20.00 50 $4000.00
d. 1991 14,000 $25.00 90 $1500.00
e. 1985 10,000 $35.00 20 $3000.00

24. Of the five office buildings above, are any of them clearly unacceptable?

a. All are acceptable.

b. a and b are unacceptable.
c. c is unacceptable.

d. d is unacceptable.

e. e is unacceptable.

98

APPENDD( A

A new client wants to know whether AT & T’s long distance rates are such that
the client should adopt a per call charge basis ($3.00 per call regardless of length) or
length of call pay plan ($.50 per minute for calls less than five minutes; $.30 per
minute for calls between 5 and fifteen minutes; and $.20 per minute for calls over
fifteen minutes). The client gave you the following information regarding their

calling behavior over the last year.

 

Calls < 5 minutes Calls 5 - 15 minutes Calls over 15 minutes
(average 3 min.) (average 8 min.) (average 20 min.)
Quarter 1 300 500 700
Quarter 2 50 600 800
Quarter 3 250 300 p 800
Quarter 4 100 700 800

25. What billing procedure (i.e., per call charge vs. charge per minute) would be
most advantageous to the client?

a. I would advise the client to use the per call charge basis.

b. I would advise the client to use the length of call (per minute) charge
basis.

c. I would advise the client that the two charge plans are equal.

99

APPENDD( A

26. In a certain company, the ratio of the number of managers to the number of
production-line workers is 5 to 72. If 8 additional production-line workers were to
be hired, the ratio of the number of managers to the number of production-line

workers would be 5 to 74. How many managers does the company have?

a. 5

b. 10
c. 15
d. 20
e. 25

27. Thirty percent of the members of a sales force have passed the product
knowledge test. Among the members who have not passed the test, 12 have taken
the training course and 30 have not taken the course. How many members are there

in the sales force?

a. 60
b. 80
c. 100
d. 120
e. 140

100

APPENDD( A

28. For telephone calls between two particular cities, a telephone company

charges $.40 per minute if the calls are placed between 5:00 am. and 9:00 pm. and

$.25 per minute if the calls are placed between 9:00 pm. and 5:00 am. If the charge

for a call between the two cities placed at 1:00 pm. was $10.00, how much would a

call of the same duration have cost if it had been placed at 11:00 pm?

a.

b.

$3.75
$6.25
$9.85
10.00

$16.00

29. Of 30 applicants for a job, 14 had at least 4 years experience, 18 had degrees,

and 3 had less than 4 years experience and did not have a degree. How many of the

applicants had at least 4 years experience and a degree?

a.

b.

l4

13

101

APPENDD( A

VERBAL REASONING - 41 ITEMS - 16 MINUTES

 

Grammar
The next 14 items are designed to measure your knowledge of English grammar
rules. Read each sentence and select the word or phrase that best completes the
sentence from the alternatives provided. Bubble in the circle on your answer sheet
corresponding to the answer you have chosen. There is only one answer to each

item. There is no penalty for guessing, so please mark an answer for each question.

30. Yesterday, the workers _ wearing the appropriate safety equipment.
a. was
b. were
c. are
d. rs
31. Both lunch break occurred at the same time.

a. employees
b. employee’s

c. employees’

102

APPENDIX A

32. The firm’s financial data correct, but the manager’s conclusions were
wrong.
a. are
b. is
c. were
(1. was
33. The cellular division improved productivity over the past month.
a. its
b. their
c. it's
34. Name the subordinate is most qualified.
a. whom
b. which
c. that
(1. who

103

APPENDIX A

 

35. The firm had much to do with being the winners of the contract.

a. them

b. their

0. those
36. The consultant timely but also provided competent financial
advice.

a. not only was

b. was not only

 

c. is not only

d. not only is
37. Today the regulatory environment is totally different in the
past.

a. from what we have seen

b. from we have seen

c. than what we have seen

(1. than we have seen
38. This technical innovation will provide service to those in
remote locations.

a. more superior

b. most superior

c. superior

104

APPENDD( A
d. best

38. Although many of the computers were no longer needed, they still worked

a. good
b. better
c. well

d. adequate
38. No action can be taken a new communications convention
until we have a full cost accounting for the last one.

a. on planning

b. to inaugurate

c. to plan

(1. on inaugurating
38. The supplier afford to expand its manufacturing capacity
at this time.

a. can hardly

b. cannot hardly

c. are unable to

d. will hardly

105

APPENDIX A

39. If the economy had not declined, the supplier recovered from the
labor strike.

a. might of

b. might have

c. may be

(1. could of

40. The supervisor was happy to report that employees had filed

grievances than any prior year in the company’s history.

a. less

b. many
c. fewer
d. lesser

Verbal Comprehension

The next 27 items are designed to measure your verbal comprehension. Read each
item and select the word or phrase among the set of alternatives that best completes
the sentence. Bubble in the circle on your answer sheet corresponding to the answer
you have chosen. There is only one answer to each item. There is no penalty for

guessing, so please mark an answer for each item.

106

APPENDIX A

41. The secretary reminded the chairperson to discuss the matter pertaining to the

owner’s claim of exclusive rights to the manufacture and sale of the product. The

secretary was reminding the chairperson to discuss the issue of

a.

b.

C.

propaganda

agenda

product distribution
competitive advantage

proprietorship

42. Mr. Harding asked his subordinate managers to assess the cost savings

associated with the purchase of Macintoshes versus IBMs. This request is an

example of

a. investment

b. accountability
c. empowerment
d. cost-accountin g

6.

agenda-setting

43. A business unit is asked to develop budgets for possibilities that various levels

of sales and taxation will result from a change in federal taxation legislation. These

alternate budgets are best referred to as

a.

negotiation points

107

d.

6.

APPENDIX A

cost analysis briefings
forecasting
contingency planning

effects analyses

44. Dynamic organizations are interested in employing people who are capable of

switching jobs easily. That is, these organizations are interested in people who are

a.

b.

d.

C.

schizophrenic
motivated
versatile
innovative

mobile

45. As a catalyst to improve sales, the corporation offered an increase in sales

commissions of five percent. Catalyst means

a.

b.

incentive

index

method

process

investment

108

APPENDIX A

46. The goods are transferred to the company for the purpose of being sold. The
process here is best described as

a. consignment

b. loan

c. transaction

(1. communication

e. negotiation

47. Jim proposed a new way to organize the flow of paperwork through the office
and his supervisor complimented him for his entemrising effort. Enter-prising effort
in this context means

a. provocation

b. induction

c. extroversion

d. initiative

e. delegation

48. The survey only addressed issues pertaining to the interactions between two
managers. In other words, the relationship of interest here was
a. microscopic

b. informal

109

APPENDIX A

c. dyadic
d. supervision

e. cooperation

49. A supervisor indicated that she would approve a day off with pay to all
employees who became proficient in a new software package. The day off with pay
in this context serves as

a. support

b. a commitment

c. a negotiating factor

(1. an additional responsibility

e. an inducement 50. To say that an employee is lacking in frankness or sincerity
is to say that he or she is

a. incoherent

b. unconvincing

c. disingenuous

d. ambiguous

e. fallacious

110

APPENDIX A

51. A company whose controlling interest is owned by another company is best
described as a

a. partner

b. headquarters

c. retail

(1. subsidiary

e. division

52. John was praised for his formulation of a set of heuristics designed to help
people develop their annual budgets. Heuristics in this context means

a. strategies

b. assumptions

c. limits

(1. improvements

e. incentives

111

APPENDD( A

53. The integration of office activities under the supervision of Jeremy was a major

mistake. Integration here means

a.

b.

combination
segregation
bureaucratization
labeling

verification

54. The board members felt that the new project was peripheral and that it would

burden them with obligations and hinder the achievement of the business objective.

From the perspective of the board members, the new project is best described as an

d.

C.

encroachment

encumbrance

estrangement

evocation

exuberance

55. The executive committee was surprised at the manager’s decision which was

tantamount to a resignation. Tantamount here means

a.

b.

C.

accumulated
resulted

equivalent

112

d.

C.

APPENDD( A
implicit

tangential

56. The telephone company’s tactical objective was to achieve medium-term

performance targets defined in terms of service and production. Tactical means

a.

b.

priman/
sole
legitimate
contractual

strategic

57. Relevant and timely information helps managers make decisions that do not

create overloads for subordinates. Relevant information is

unambiguous
selective
scientific
pertinent

situational

58. A posthumous managerial excellence award means that the recipient is

a.

honorable

113

6.

APPENDIX A
dead
courageous

deserving

predetermined

59. After the recent downturn in the demand for its fiber optic wiring offerings, the

company adopted an aggressive approach to the diversification of its product line.

Diversification in this context means

a.

b.

narrowing
consolidation
divesting
broadening

development

60. The workgroup leader put together the separate ideas from the staff members

and produced a unified proposal. What the leader did is best described as

a.

b.

analysis

supervision

synthesis

strategic management

evaluation

114

APPENDIX A

61.In many new jobs, there are non-routine Situations demanding prompt or urgent

action. These situations or demands are called

e.

exigencies

roles

job descriptions
occupational hazards

duties

62. The frugality with which the company’s administrative component conducted its

affairs set an example for the rest of the company during the downturn in business.

Frugality in this sentence means

a.

b.

e.

emptiness
cleanliness
thriftiness
efficiency

orderliness

63. The large scale manufacture of a new product is almost always preceded by the

development of a first or experimental working model of the product. This working

model is usually called

a.

b.

C.

an innovation
a prototype

a surrogate

115

d.

6.

APPENDIX A

an implementation plan

an isomer

64. It appears that the only way to complete our production today is to expe_dite the

paperwork in the quality control department. Exp_edite here means

a.

b.

C.

d.

6.

do away with
speed up
postpone
reduce

prioritize

65. Because of the incredible family demands and her efforts to continue her

 

education after work hours, the employee’s performance had been erratic. Erratic in

this sentence means

(1.

C.

untrue

lewd

terrible

monitored

inconsistent

66. The primary consideration in the formulation of the proposal was that the

proposal must be tenable. Tenable here means

a.

b.

capable of being maintained or defended

capable of adapting to changing circumstances

116

APPENDD( A

c. capable of attracting attention
(1. capable of exerting a wide influence

e. comprehensive

67. In spite of management’s statements to the contrary, most employees felt that
bonuses were imminent because of the closing of a major deal. Imminent in this
sentence means

a. determined

b. impending

c. arbitrary
d. random
e. inevitable

117

APPENDIX B

118

 

Appendix B

Non - Face Valid Test Form

LOGICAL REASONING - 15 ITEMS - 10 MINUTES

 

This is a test of your ability to tell whether the conclusion drawn from certain
premises is correct or incorrect. For each problem, the various conclusions drawn

from the premises may or may not be correct. You are to assume that the premises

in each problem are true. Without assuming the truth or falsity of anming in

 

addition to the information given in the premises. decide which conclusion is
correct. There is only one correct conclusion. Bubble in the circle on your answer
sheet corresponding to the answer you have chosen. There is no penalty for

guessing, so please mark an answer for each item.

Conditional Reasoning
1. Premises
If John sells his car tonight then he will throw a party.
John did not throw a party.
Conclusion
a. John sold his car tonight.
b. John did not sell his car tonight.

c. John may or may not have sold his car tonight.

119

Appendix B

2. Premises

If Linda joins a health club then she will improve her physical condition
Linda improved her physical conditioning

Conclusion

a. Linda joined a health club.

b. Linda did not join a health club.

c. Linda may or may not have joined a health club.

3. Premises

If the wire is removed then the automobile’s transmission will be damaged.
The wire is not removed.

Conclusion

a. The automobile’s transmission was damaged.

b. The automobile’s transmission was not damaged.

c. The automobile’s transmission may or may not have been damaged.

4. Premises

If the customer is a senior citizen, then the customer will receive a discount.
The customer is a senior citizen.

M5492

a. The customer will receive a discount.

b. The customer will not receive a discount.

120

Appendix B

c. The customer may or may not receive a discount.

5. Premises

If the supermarket does not offer double coupons, then the shoppers will Shop
elsewhere.

The supermarket offered double coupons.

Conclusion

a. The shoppers will shop elsewhere.

b. The shoppers will not shop elsewhere.

c. The shoppers may or may not shop elsewhere.

6. Premises

If the mayor does not improve her health, then she will not be reelected.
The mayor improved her health.

Conclusion

a. The mayor will be reelected.

b. The mayor will not be reelected.

c. The mayor may or may not be reelected.

121

7.

Appendix B

W

If the topic is not approved, then work on the term project cannot begin.

Work on the term project has begun.

Conclusion

a. The topic was approved.

b. The topic was not approved.

c. The topic may or may not have been approved.
8. Premises

If the two second-grade classes merge, then the students will not go on a field trip in

December.

The students did not go on a field trip in December.

Conclusion

a. The second-grade classes merged.

b. The second-grade classes did not merge.

c. The second-grade classes may or may not have merged.

122

Appendix B

S yllogistic Reasoning
9. Premises
All children who received a high grade are spending more time in gym.
All children who did their homework received a high grade.
Conclusion
a. All children who received a high grade did their homework.
b. All children who did poorly on the exam are Spending more time in tutoring.
c. All children who are spending more time in tutoring did their homework.

d. All children who are spending more time in tutoring are spending more time in

10. Premises

Some pets who auditioned for the pet Show are given a ribbon.

Every pet who was selected for the pet show performs well.

Conclusion

a. Every pet who was given a ribbon did not audition for the pet show.
b. Every pet who was given a ribbon auditioned for the pet show.

c. Some pets who perform well are given a ribbon.

(1. Every pet who performs well auditioned for the pet Show.

123

Appendix B

1 l. Premises

None of the students who attended the lecture are from the biology department.

All students who had their financial aid increased attended the lecture.

Conclusion

a. All students whose financial aid was not increased are from the biology
department.

b. All students from the biology department did not have their financial aid
reduced.

c. All students who participated in the picnic had their financial aid increased.

(1. Some of the students who participated in the picnic did not have their financial

aid increased.

12. Premises

Some products that passed the safety test are selected for the award.
None of the allergy pills are selected for the award.

Conclusion

a. Every allergy pill failed the safety test.

b. Some of the allergy pills passed the safety test.

c. Some products who passed the safety test are not allergy pills.

(1. Every product who is selected for the award passed the safety test.

124

Appendix B

Inferential Reasoning
This is a test of your ability to tell whether the conclusion drawn from certain
premises is correct or incorrect. For each problem, the various conclusions drawn

from the premises may or may not be correct. You are to assume that the premises

in each problem are true. Without assuming the truth or falsity of anfling in

 

addition to the information given in the premises, decide which conclusion is

 

correct. There is only one correct conclusion.

13. More people watch television after dark than during daylight hours.

a. A decrease in work-related activities tends to increase entertainment activities.
b. Watching television after dark is generally more enjoyable than during daylight
hours.

c. After dark, most people watch television as their primary form of
entertainment.

(1. The chance of a television show being watched is lower during daylight hours
than it is after dark.

e. There are more people watching television after dark Since most people are not

at work.

125

Appendix B

14. Today it generally costs less to buy a home than it cost to buy one three years
ago.

a. Homes are made using less expensive materials today.

b. The purchasing power of home buyers has risen during the past three years.
c. The cost of homes has fallen during the past three years.

(1. Home buyers are buying more homes today than three years ago.

e. Home owners of three years ago are generally richer than home owners today.

15. The combined presence of the class president, secretary, and the presence of at
least 50 students at the annual senior class meeting is necessary for the approval of
the prom location.

a. The prom location cannot be approved without the students’ support.

b. The president and the secretary are the two most important persons in the prom
location approval process.

c. If the prom location was approved, then the secretary was present.

(1. Prom locations are only approved at senior class annual meetings.

e. If both the president and the secretary support the prom location, then approval
is most likely as long as the number of students present at the annual meeting is at

least 50.

126

Appendix B

QUANTITATIVE REASONING - 14 ITEMS - 16 MINUTES

 

The next 14 items assess your ability to use quantitative information to make
decisions. Read the description of each set of information and choose the best
answer to each question. Bubble in the circle on your answer sheet corresponding
to the answer you have chosen. There is only one answer to each item. There is no
penalty for guessing, so please mark an answer for each item.

The table below contains a breakdown of the classes in which three grade levels of

students in School A spend their 40-hour school week.

 

2nd Grade 3rd Grade 4th Grade

(No.=100) (N0. = 80) (N0. = 20)
Mathematics 10 5 . 20
English 20 5 2
History 6 10 6
Science 4 20 12

16. For these three groups of students, what percentage of school time is spent in

science classes?

a. 30
b. 28
c. 26
d. 24
e. 20

127

Appendix B

The table below is for questions 17 & 18.
The table below is a description of the speed at which three different oven models

can cook pizzas for specific stores along with the cost of each of these ovens.

 

Store 1 Store 2 Store 3 Cost
Oven A 10 sec. 180 sec. 200 sec $3500
Oven B 12 sec. 200 sec. 300 sec $3000
Oven C 20 sec 400 sec. 320 sec $2000
Number of Pizzas per 2000 1500 1000

Week

17. How many Model C ovens can be purchased for the same price as 10 Model B

ovens?

a. 10
b. 15
c. 20
d. 24
e. 30

128

Appendix B

18. How much time would be saved in making pizzas at Location 3 each week by

purchasing Model A ovens over Model C ovens?

a.

b.

20,000 minutes
2000 minutes
100,000 seconds
12,000 seconds

200 minutes

19. The traditional Tudor-style house on the comer, which is three times as large as

the Victorian house in the middle of the block, is 18 feet by 20 feet. The length of

one side of the Victorian house is 10 feet. How long is the other side of the

Victorian house?

36 feet

18 feet

12 feet

10 feet

8 feet

129

Appendix B

20. A couple borrowed $2,250,000 to build their vacation home in the mountains.

They paid 10 percent interest on money borrowed from their parents and 8 percent

from a group of friends. Yearly interest paid to both their parents and the friends

was the same. How much was borrowed from their parents?

a.

b.

21.

$1,500,000
5 1 ,250,000
$1 ,000,000
$750,000

$500,000

Chocolate brownies comprise 20 percent of the desserts baked; chocolate chip

cookies, 75 percent of the desserts; and apple pies, 5 percent of the desserts. If 1000

of these three desserts are baked each month, how many chocolate brownies are

baked in a year?

a.

b.

2400

600

200

50

The number of chocolate brownies baked cannot be determined.

130

Appendix B

22. Mary has a collection of over 4,800 coins and She wants to reduce it to 3,000.
At the current pace, Mary trades away 500 of these coins a month while acquiring
600 coins per month. Assume Mary decided to reduce coin acquisitions as a means
of reducing her collection, by how much would she have to reduce acquisitions each
month to reach the target of 3000 coins in a year?

a. 320 coins

b. 267 coins

c. 250 coins

d. 200 coins

e. 100 coins

131

Appendix B

23. Assume that you have to acquire a school building with work space for a
minimum of 100 students. Each student requires a minimum of 120 square feet of
classroom Space and a computer hookup. You’ve located the following five possible
buildings described below in the immediate area and must decide which would be

most economical over a three year period. Which would you choose?

 

Year Square Feet Cost: Computer Estimated cost
Built Square Feet Stations in per new
/Y ear place hookup
a. 1968 15,000 $20.00 20 $5000.00
b. 1988 12,000 $30.00 80 $2000.00
c. 1972 18,000 $20.00 50 $4000.00
d. 1991 14,000 $25.00 90 $1500.00
e. 1985 10,000 $35.00 20 $3000.00

24. Of the five buildings above, are any of them clearly unacceptable?
a. All are acceptable.

b. a and b are unacceptable.

c. c is unacceptable.

d. d is unacceptable.

e. e is unacceptable.

132

Appendix B

A New Yorker wants to know whether taxicab rates are such that She should opt for
a per ride charge ($3.00 per ride regardless of length) or length of ride pay plan ($.50
per mile for rides less than five miles; $.30 per mile for rides between 5 and fifteen
miles; and $.20 per mile for rides over fifteen miles). The New Yorker supplied the

following information regarding her taxi riding behavior over the last year.

Rides < 5 miles Rides 5 - 15 miles Rides over 15 miles

 

(average 3 mi.) (average 8 mi.) (average 20 mi.)
J an-Mar 300 500 700
Apr-Jun 50 600 800
J ul-Sep 250 300 800
Oct-Dec 100 700 800

25 . What payment plan (i.e., per ride vs. charge per mile) would be most
advantageous to this New Yorker?

a. I would advise the New Yorker to use the per ride charge basis.

b. I would advise the New Yorker to use the length of ride (per mile) payment
plan.

c. I would advise the New Yorker that the two payment plans are equal.

133

Appendix B

26. In a certain school, the ratio of the number of teachers to the number of special
education students is 5 to 72. If 8 additional special education students were to be
enrolled, the ratio of the number of teachers to the number of special education

students would be 5 to 74. How many teachers does the school have?

a. 5

b. 10
c. 15
d. 20
e. 25

27. Thirty percent of the members of a swim club have passed the lifesaving test.
Among the members who have not passed the test, 12 have taken the preparatory

course and 30 have not taken the course. How many members are there in the Swim

club?

a. 60
b. 80
c. 100
d. 120
e. 140

134

Appendix B

28. For taxi rides between two particular cities, a taxi service charges $.40 per mile

if the ride is taken between 5:00 am. and 9:00 pm. and $.25 per mile if the ride is

taken between 9:00 pm. and 5:00 am. If the charge for a ride between the two

cities taken at 1:00 pm. was $10.00, how much would a ride of the same distance

have cost if it had been taken at 11:00 pm?

a.

b.

$3.75
$6.25
$9.85
$10.00

$16.00

29. Of 30 students in a class, 14 had at least 4 years experience, 18 had a masters,

and 3 had less than 4 years experience and did not have a masters. How many of the

students had at least 4 years experience and a masters?

a.

b.

l4

13

135

Appendix B
VERBAL REASONING - 41 ITEMS - 16 MINUTES

 

Grammar
The next 14 items are designed to measure your knowledge of English grammar
rules. Read each sentence and select the word or phrase that best completes the
sentence from the alternatives provided. Bubble in the circle on your answer sheet
corresponding to the answer you have chosen. There is only one answer to each

item. There is no penalty for guessing, so please mark an answer for each item.

30. Yesterday, the players wearing the appropriate safety equipment.
a. was
b. were
c. are
(1. IS
31. Both newscasts occurred at the same time.

a. anchors

b. anchor’s

c. anchors'

136

Appendix B

32. The FBI’s crime data correct, but the sociologist’s conclusions were
wrong.
a. are
b. is
c. were
(1 was
33. The cleaning crew improved cleanliness over the past month.
a. its
b. their
c. it's
34. Name the person is most agreeable.
a. whom
b. which
c. that
(1. who
35. The weather had much to do with being the winners of the
tournament.
a. them

137

Appendix B

b. their
c. those

36. The housekeeper

 

cleaning help.
a. not only was
b. was not only
c. is not only

d. not only is

37. Today the environmental legislation is totally different

the past.
a. from what we have seen
b. from we have seen
c. than what we have seen
d. than we have seen
38. This new bus route will provide
remote locations.
a. more superior
b. most superior
c. superior

d. best

timely but also provided competent

 

 

138

service to those in

39.

40.

Appendix B

Although many of the hammers were no longer needed, they still worked

a. good .

b. better

c. well

(1. adequate

No action can be taken

 

a full ingredient analysis for the last one.

41.

a. on planning
b. to inaugurate
c. to plan

d. on inaugurating

The high school

a new apple pie contest until we have

afford to expand its classroom

 

capacity at this time.

a. can hardly
b. cannot hardly
c. are unable to

d. will hardly

139

Appendix B
42. If attendance had not declined, the school

overcrowding problem.

a. might of
b. might have
c. may be

(1. could of

43. The professor was happy to report that

 

appeals than any prior year in the professor’s career.

a. less

b. many
c. fewer
d. lesser

Verlgl Comprehension

recovered from the

students had filed grade

The next 27 items are designed to measure your verbal comprehension. Read each

item and select the word or phrase among the set of alternatives that best completes

the sentence. Bubble in the circle on your answer sheet corresponding to the answer

you have chosen. There is only one answer to each item. There is no penalty for

guessing, so please mark an answer for each item.

140

Appendix B

44. The mother reminded the father to discuss the matter pertaining to the child’s
claim of exclusive rights to the television and video game system. The mother was
reminding the father to discuss the issue of

a. chores

b. punishment

c. allowance

d. gifts

e. sharing

45. Mrs. Harding asked her husband to assess the potential cost savings associated
with the use of gas versus electric heat. This request is an example of

a. relegation

b. trust

c. empowerment

d. cost-analysis

e. nagging

46. A married couple has decided to develop strategies for possibilities that various
levels of property taxes will result from a change in local zoning regulations. These
alternate strategies are best referred to as

a. forecasts

b. confusions

141

Appendix B

c. obsessive compulsions

(1. contingency models

e. negotiations

47. Theatrical organizations are interested in attracting actors who are capable of
switching roles easily. That is, these organizations are interested in people who are
a. schizophrenic

b. motivated

c. versatile

d. innovative

e. mobile

48. As a catalyst to the class discussion, the new teacher resulted in an increase in
participation of five percent. Catalyst means

a. accelerant

b. effect

c. method

d. process

e. retardant

142

Appendix B

49. The watch was transferred to the broker for the purpose of being sold. The
process here is best described as

a. consignment

b. loan

c. transaction

d. communication

e. negotiation

50. Susan proposed a new way to organize the toys in her son’s playroom and her
mother complimented her for her enterprising effort. Entegprising effort in this
context means

a. provocation

b. induction

c. extroversion

d. initiative

e. delegation

51. The survey only addressed issues pertaining to the interactions between two
children. In other words, the relationship of interest here was

a. microsc0pic

b. informal

c. dyadic

(1. sharing

143

Appendix B
e. cooperation
52. A father indicated that he would grant a day off of chores with allowance to his
children who became proficient in programming the VCR. The day off with
allowance in this context serves as
a. support
b. a commitment
c. a negotiating factor
(1. an additional responsibility
e. an inducement
53. To say that someone is lacking in frankness or sincerity is to say that he or she
is
a. incoherent
b. unconvincing
c. disingenuous
d. ambiguous

e. fallacious

54. A candy store that is largely owned by another candy store is best described as

a. partner

b. headquarters

144

Appendix B

c. retail
(1. subsidiary

e. division

55. John was praised for his formulation of a set of heuristics designed to help
people develop a proper golf swing. Heuristics in this context means

a. strategies

b. assumptions

c. limits

(1. improvements

e. incentives

56. The integration of baking activities under the supervision of Jeremy was a major
mistake. Integration here means

a. combination

b. segregation

c. bureaucratization

d. labeling

e. verification

57. The thesis committee felt the suggestion was peripheral and that it would

burden the research with constraints and hinder the achievement of the research

145

Appendix B
objective. From the perspective of the thesis committee, the suggestions are best
described as an '
a. encroachment
b. encumbrance
c. estrangement
d. evocation
e. exuberance
58. The other players were surprised at her reaction which was tantamount to an
admission of guilt. Tantamount here means
a. accumulated
b. resulted
c. equivalent
(1. implicit
e. tangential

59. The indoor track team’s tactical objective was to achieve long-distance

 

performance targets defined in terms of speed and endurance. Igjaal means
a. primary

b. sole

c. legitimate

d. contractual

e. strategic

146

Appendix B

60. Relevant and timely information helps doctors make decisions that do not
create overloads for nurses. Relevant information is

a. unambiguous

b. selective

c. scientific

d. pertinent

e. situational

61. A posthumous volunteer of America award means that the recipient is
a. honorable

b. dead

c. courageous

d. deserving

e. predetermined

147

Appendix B
62. After the recent downturn in the demand for admissions to the medical school,
the university adopted an aggressive approach to the diversification of its courses.
Diversification in this context means
a. narrowing
b. consolidation
c. divesting
d. broadening

e. development

63. The debate leader put together the separate ideas from the debaters and
delivered a unified argument. What the leader did is best described as

a. analysis

b. overseeing

c. synthesis

d. strategic debating

e. evaluation

64. For many new parents, there are non-routine situations demanding prompt or
urgent action. These situations or demands are called

a. exigencies

b. roles

c. parenting demands

148

d.

C.

65.

Appendix B

household hazards
duties

The frugality with which the family conducted its household affairs set an

example for the rest of the community during the downturn in household income.

Frugality in this sentence means

a.

b.

C.

66.

emptiness
cleanliness
thriftiness
efficiency
orderliness

The full scale launching of a new NASA rocket is almost always preceded by

the development of a first or experimental working model of the rocket. This

working model is usually called

a.

b.

e.

67.

an innovation

a prototype

a surrogate

an implementation plan
an isomer

It appears that the only way to complete our chores today is to exmdite the

process of assorting recyclables. Exp_edite here means

a.

do away with

149

Appendix B
b. speed up
c. postpone
(I. reduce
e. prioritize
68. Because of the incredible family demands and her efforts to continue her post-

doctoral training, Jane’s dinner schedule has been erratic. Erratic in this sentence

 

 

means

a. untrue
b. lewd

c. terrible

d. monitored

e. inconsistent

69. The primary consideration in the grading of the exam questions was that the
grading must be tenable. Tenatala here means

a. capable of being maintained or defended

b. capable of adapting to changing circumstances

c. capable of attracting attention

(1. capable of exerting a wide influence

e. comprehensive

150

Appendix B

70. In spite of administration’s statements to the contrary, most students felt that
financial aid increases were imminent because of the increase in state aid. Imminent
in this sentence means

a. determined

b. impending

c. arbitrary
(1. random
e. Inevitable

151

APPENDIX C

152

Appendix C

Test Reactions Items.
Belief In Tests

Employment selection tests can predict how well someone will perform a job.
Employment selection tests are apt valid.*

Employment testing accurately assesses the skills and abilities necessary for good job

performance.
Employment selection tests are good ways of selecting people into jobs.

Employment selection tests should my be used.*

Pre-Test Motivation

1 am extremely motivated to do well on this test.

I just d_ora’_t care how I do on this test.*

I will try my best on this test.

While taking this test, I will try to concentrate and do well.
Doing well on this test is important to me.

I want to be among the top scorers on this test.

I will pat put much effort into this test.*

153

 

 

Appendix C
Pre-Test Self-Efficacy

I believe I will have n_o problems on this test.*

I think I will do very well on this test.

Compared with other applicants taking this test, I expect to do well.
I am confident that I will receive a high score on this test.

I’m confident I can solve the problems presented in this test.

Face Validity

The actual content of this section of the test was clearly similar to the job tasks listed
above.

I can n_ot see any relationship between the section of the test and what I think is
required by the job tasks.*

It would be obvious to anyone that this section of the test is related to the job tasks.

I did n_ot understand what this section of the test had to do with this job.*

Perceived Predictive Validity

The employer can tell a lot about the applicant’s ability to do this job based on the
results of this section of the test.

Failing to pass this section of the examination clearly indicates that you cant do the

job.

154

Appendix C

Applicants who perform well on this section of the examination are more likely to
perform well on the job than those who perform poorly.

I am confident that this section of the examination can predict how well an applicant
will perform on the job.

One’s performance on this section of the examination is a good indicator of one’s ability

to do the job.

Perceived Ffimess
Whether or not I got the job, I would be satisfied if this section of the test were used to

assess my qualifications to perform this job.

Whether or not I got the job, using this section of the test to select people for this job is

fair.

Whether or not I got the job, I would be satisfied if this section of the test were used to

make hiring decisions.

Overall, I believe this section of the examination was fair.

Internal Attributions
Success on this section of the test is primarily a function of one’s knowledge of specific

aspects of the job rather than one’S ability.*

My performance on this section of the test is an accurate assessment of my ability.

155

L-

 

 

Appendix C

My score on this section of the test was a function of a lack of practice on similar

items.*
My score on this section of the test was a function of the item wording.*

My score on this section of the test was a function of a lack of familiarity with the

subject matter of the items.*

Success on this section of the test is dependent upon one’s knowledge of specific

aspects of the job an applicant should not be expected to not know.*
My score on this section of the test was a function of luck.*

My score on this section of the test was a function of the extreme difficulty of the

questions.*

Success on this section of the test is primarily a function of one’s ability rather than

one’s knowledge of specific aspects of the job.

Success on this section of the test is dependent upon one's ability.

Post-Test Motivation

1 was extremely motivated to do well on this section of the test.
I just didn’t care how I did on this section of the test.*
I tried my best on this section of the test.

While taking this section of the test, I tried to concentrate and to do well.

156

Appendix C

Doing well on this section of the test was important to me.
I wanted to be among the top scorers on this section of the test.

I did p_o_t put much effort into this section of the test.*

 

 

157

 

Appendix C

Post-Test Self-Efficacy

I believe I had pg problems on this section of the test. *

I think I did very well on this section of the test.

Compared with other applicants taking this section of the test, I think I did well.
I am confident that I will receive a high score on this section of the test.

I’m confident I could solve the problems presented in this section of the test.

Notes: * indicates that an item is reverse scored. The sequential order of each item

within each question category corresponds to the numbers in Appendices D & E.

158

APPENDIX D

159

Appendix D

Factor loading matrix of post-test items (principal—axis analysis; varimax rotation)

 

Factor
1 Z 3 5 5 6 Z
PVl 0.84 -0.01 0.02 -0.04 0.08 0.06 0.1 1
PV4 0.82 -0. 15 0.01 0.1 1 0.07 0.04 0.23
PF2 0.80 -0.07 0.04 -0.09 0.14 0.02 -0. l 3
PV5 0.76 -O. 13 0.10 -0.10 0.05 0.02 0.18
1A2 0.73 0.06 0.13 -0.01 -0.02 0.24 -0.20
PF 3 0.72 -0.03 0.10 -0.06 0.12 0.01 -0.14
PF] 0.72 0.13 0.19 -0.01 0.18 -0.07 -0.35
PV2 0.64 -O.20 0.06 0.04 -0.03 0.01 0.34
FV3 0.63 -0.01 0.06 -0. 18 0.34 -0. 12 0.00
PV3 0.58 0.05 0.06 -0.1 l 0.24 0.10 0.44
PF4 0.55 0.17 0.16 0.05 0.32 0.09 0.00
WI 0.49 0.21 0.13 -0. 14 0.43 -0. 14 -0.21
MOT3 -0.03 0.84 0.1 1 -0.05 -0.01 0.12 0.01
MOT4 -0.08 0.84 0.10 -0. l l 0.00 0.13 0.03
MOTS 0.08 0.84 0.20 -0.08 0.01 —0.02 -0.08
MOT7 -0.09 0.84 0.07 -0.02 -0.02 0.01 0.03
MOT2 -0.01 0.83 0. 10 0.07 0.08 -0.06 -0.06
MOT6 -0.08 0.76 0.19 -0.05 0.09 0.08 0.08

160

MOT]

IA7

SE4

SE3

SE2

SE5

SE1

1A8

1A5

1A3

1A4

1A6

FV4

IAlO

1A9

1A1

0.11

-0.14

0.13

0.08

0.20

-0.02

0.23

0.01

-0.02

-0.04

-0.13

-0.21

0.39

0.38

0.29

0.20

-0.20

0.75

0.53

0.26

0.27

0.32

0.29

0.03

-0.04

-0.02

-0.05

-0.03

0.01

0.05

0.03

0.08

0.11

-0.02

0.23

0.26

0.86

0.82

0.79

0.63

0.55

0.12

0.19

0.15

0.02

-0.09

0.01

0.00

0.14

0.24

-0.02

Appendix D

-0.02

0.36

0.09

0.03

0.16

0.18

0.20

0.73

0.71

0.61

0.50

0.40

0.07

0.06

-0.05

0.04

0.31

0.00

0.18

-0.01

—0.05

—0.01

0.17

0.03

0.10

0.07

-0.05

-0.08

0.14

0.75

0.75

0.14

-0.02

-0.15

-0.12

0.13

0.09

0.09

0.09

0.11

-0.01

-0.17

0.07

0.17

0.02

0.38

0.05

0.04

0.78

0.65

0.44

-0.11

0.02

—0.02

0.07

-0.02

0.07

-0.10

0.03

0.01

-0.05

0.02

-0.07

0.07

0.01

0.08

0.04

-0.06

 

Notes: 62% of variance accounted for by 7 extracted factors. MOT = Motivation; IA

= Internal Attributions; SE = Self-Efficacy; FV = Face Validity; PV = Perceived

Predictive Validity; PF = Perceived Fairness.

161

APPENDIX E

162

Appendix E

Factor loading matrix of pre-test items (principal axis analysis; varimax rotation)

 

 

Factor
1 2 3

MOT7 0.71 0.01 -0.03
MOT3 0.66 0.19 0.02
MOT4 0.66 0.22 0.04
MOT2 0.63 0.15 -0.01
MOT6 0.59 0.26 0.00
MOT] 0.57 0.35 0.01

MOTS 0.56 0.16 0.04
SE4 0.35 0.86 0.00
SE2 0.23 0.81 0.07
SE5 0.33 0.72 -0.05
SE3 0.26 0.68 0.02
SE] 0.06 0.61 -0.04
BITS 0.02 0.05 0.78
BIT 2 0.01 -0.04 0.70
BIT] -0.05 —0.09 0.70
BIT4 0.03 0.03 0.61

BIT3 0.03 0.03 0.58

 

Notes: 50% of variance accounted for by 3 extracted factors. MOT = Motivation;

SE = Self-Efficacy; BIT = Belief in Tests.

163

Appendix F

164

 

*Omd

*Hmd

13.6
*2 .c

HOE

“*de
Nod

HOE

*de
mod.
Ed.

ME

* _ Nd
“and
mod

ME

 

*de
8.0
s. _ Md
w— d.

mmm

_.. _ ed
vod
* _ md
cod

mmE

*vmd 56.
mod- cod
30.: mod
_ _d. and
*mmd E .o
8.0
HOSE H5
3
*w _ d 86.
Ed *mmd
$06 3 .o
56 $36
5:. 36.
2 .o
HOSE HE

 

Gum _HZV BE?

3 .o
mod
n _ .o
8.0..
N_ .o
3.0
Nod.

0N6
_md
A: .O
*NNd
3.0
mod
m _ .0

COD

_od.
86
Had
A: .o
mod-
cod
Ed
_od

56
mod
_ _d
3.9
cod
2 .o
cod
2 .o

EH

nmd
cod
2.6
and
:5
cod
Inc
3 .o
ovd
Om

36
mod
cod
$6
$6
mmd
wmd
o_ .o
cmd
Om

no.m
wad
mum
mnd
wed
aqm
GEN
:6
Nod
:32

mad
wo.m
ohm
and
_N.m
3mm
8%
de
mmd
=32

Mm

HOE
ME
mmE

HOSE
Hum

mm

HOSH
ME
MmE

HOSE
HE

 

n. 5225.4,

gm ,3 5526 .5pr 85mg; 525 .8 meets—MEOBBE calmlwflm £582

165

 

.55me 2: :0 8m 835:8 3:036:00 BEBE .moaoEmafim “onamoE n Mm "mp—2:532, 3585 u 5 623282

“moHLmoE u HOE ”mmocvoEoM new coZoeoE .I. ME ”zomQEm—Eom .moHoE u MmE 828382 .moHeE u HOSE ”memo H

 

E Mozom n HE ”28m “mo H 33.—woo .80 H n woO x21; BE .1. _ ”23> 8mm 82 u 9 Eon— so H n E H ”3qu .3. v a...

m 5283

166

References

Adams, S. (1950). Does face validity exist? Educational and Psychological
Measurement, 10. 320-328.

Anastasi, A. (1988). Psychological Testing (6th Ed). New York: MacMillan.

Arnold, H. (1985). Task performance, perceived competence, and attributed
causes of performance as determinants of intrinsic motivation. Academy of
Management Journal, 28, 876-888.

Arvey, R. D., Strickland, W., Drauden, G., & Martin, C. (1990). Motivational
components of test-taking. Personnel Psychology, 43, 695-716.

Bandura, A. (1977). Self-efficacy mechanisms in human agency. Ameg'can
Psychologist, 37, 122-147.

Barbera, K. M., Ryan, A. M., Burris Desmaris, L., & Dyer, P. J. (1995).
Multimedia employment tests: Effects of attitudes and exmriences on validity.
Paper presented at the Tenth Annual Conference of the Society for Industrial and
Organizational Psychology Inc., Orlando, Florida.

Bible, J. D. (1990). When employers look for things other than drugs: The
legality of AIDS, genetic, intelligence and honesty testing in the workplace. Labor
_Law JournaL41, 195-213.

Brutus, S. & Ryan, A. M. (1997). Determinants of mrceived job relatedness.
Paper to be presented at the Twelfth Annual Conference of the Society for Industrial
and Organizational Psychology Inc., St. Louis, MO.

Cascio, W. F. (1991). Applied Psychology in Personnel Mmagepaapt (4th ed.).
Englewood Cliffs, NJ: Prentice-Hall.

Cascio, W. F. & Phillips, N. F. (1979). Performance Testing: A rose among
thorns. Personnel Psychology, 32, 751-766.

Chan, D. (1997). Racial subgroup differences in predictive validity perceptions
on personality and cognitive ability tests. Journal of Applied Psychology.

Chan, D. & Schmitt, N. (1997). Video-based versus paper-and-pencil method
of assessment in situational judgment tests: Subgroup differences in test
performance and face validity perceptions. Journal of Applied Psychology.

167

 

Chan, D. (1996). Video-based versus pamr-and-pancil method of assessment in

Situational judgment tests: Subgroup differences in parformance and examinee
reactions. Masters Thesis, Michigan State University, East Lansing, MI.

Chan, D., Schmitt, N., DeShon, R. P., Clause, C. S., & Delbridge, K. (1997).
Reactions to cognitive ability tests: The relationship between race, test
performance, face validity and test-taking motivation. Journal of Applied
Psychology.

Cohen, J. (1992). A Power Primer. Psychological Bulletin, 112, 155-159.

Cohen, J. (1977). Statistical mwer analysis for the behavioral sciences.
Hillsdale, NJ: Erlbaum.

Cohen, J. (1988). Statistical mwer analysis for the behavioral sciences. (2nd
Ed.). Hillsdale, NJ: Erlbaum.

Daum, D. L. & Ryan, A. M. (1995). Examinee attitudes toward pagr and
pencil and computerized selection procedures. Paper presented at the Tenth Annual
Conference of the Society for Industrial and Organizational Psychology Inc.,
Orlando, Florida.

Dweck, C. S. & Leggett, E. L. (1988). A social-cognitive approach to
motivation and personality. Psychological Review. 95. 256—273.

Dweck, C. S. & Reppucci, N. D. (1973). Learned helplessness as a
reinforcement responsibility in children. Journal of Personality and Social
Psychology. 25. 109-116.

Gilliland, S. W. (1993). The perceived fairness of selection systems: An
organizational justice perspective. Academy of Management Review, 18, 694-734.

Gilliland, S. W. (1994). Effects of procedural and distributive justice on
reactions to a selection system. Journal of Applied Psychology, 79, 691-701.

Gist, M. E. & Mitchell, T. R. (1992). Self-efficacy: A theoretical analysis of its
determinants and malleability. Academy of Management Review, 17, 183-211.

Gordon, M. E. & Kleinman, L. S. (1976). The prediction of trainability using a

work sample test and an aptitude test: A direct comparison. Personnel Psychology,
_2_9, 243-253.

Graham, S. (1988). Can attribution theory tell us something about motivation in
Blacks? Educational Psychologist, 23, 3-21.

168

Graham, S. (1991). A Review of attribution theory in achievement contexts.
Educational Psychology Review. 3. 5-39.

Greenberg, J. (1987). Using diaries to promote procedural justice in
performance appraisals. Social Justice Research 1 219-234.

 

Greenberg, J. (1990). Organizational justice: Yesterday, today, and tomorrow.
Journal of Management, 16, 399-432.

Greguras, G. J. (1997). Test ta_ker reactions, negative affectivig, and test
parformance. Paper to be presented at the Twelfth Annual Conference of the

Society for Industrial and Organizational Psychology Inc., St. Louis, MO.

Gynther, M. D., Burkhart, B. R., & Hovanitz, C. (1979). Do face-valid items
have more validity than subtle items? The case of the MMPI Pd scale. Journal of

Consulting and Clinical Psychology, 47, 295-300.

Harris, M. M. & Fink, L. S. (1987). A field study of applicant reactions to
employment opportunities: Does the recruiter make a difference? Personnel

Psychology, 40, 765-784.

Helms, J. E. (1992). Why is there no study of cultural equivalence in
standardized cognitive ability testing? American Psychologist, 47, 1083-1101.

Herriot, P. (1989). Selection as a social process. In M. Smith & I. Robertson,
Advances in selection and assessment. (EdS.). USA: Wiley.

Holden, R. R. & Jackson, D. N. (1979). Item subtlety and face validity in

personality assessment. Journal of Consulting and Clinical Psychology, 47, 459-
468.

Jennings, D. A. (1997). Subgroup differences in cognitive ability test
performance: An alternative explanation.

Jensen, A. R. (1980). Bias in Mental Testing. New York: Free Press.

Joreskog, K. & Sorbom, D. (1993). LISREL 8: Structural Quation modeling
with the SIMPLIS command langaage. Hillsdale, NJ: Lawrence Erlbaum
Associates, Inc.

Kanfer, R. (1992). Motivation theory in industrial/organizational psychology.

In M. D. Dunnette (Ed.), Handbook of industrial and organizational psychology:
Vol. 1: Theory in industriaL and organizational psychology. Palo Alto, CA:

Consulting Psychologists Press.

169

Kluger, A. N. & Rothstein, H. R. (1993). The influence to selection test type on

applicant reactions to employment testing. Journal of Business and Psychology, 8,
3-25.

Konovsky, M. A. & Cropanzano, R. (1991). Perceived fairness of employee
drug testing as a predictor of employee attitudes and job performance. Joumal pf

Applied Psychology, 76, 698-707.

Lounsbury, J. W., Bobrow, W., & Jensen, J. B. (1989). Attitudes toward
employment testing: Scale development, correlates, and “known group” validation.

Professional psychology: Research and Practice, 20, 340-349.

Macan, T. H., Avedon, M. J ., Paese, M., & Smith, D. E. (1994). The effects of
applicants’ reactions to cognitive ability tests and an assessment center. Personnel

Psychology, 47, 715-738.

Mosier, C. I. (1947). A critical examination of the concepts of face validity.
Educational and Psychological Measurement, 7, 191-205.

Murphy, K. R. (1986). When your top choice turns you down: Effects of

rejected job offers on the utility of selection tests. Psychological Bulletin, 99, 133-
138.

Nevo, B. (1985). Face validity revisited. Journal of Educational Measurement
2_2_, 287-293.

 

Pintrich, P. R., & DeGroot, E. V. (1990). Motivational and self-regulated
learning components of classroom academic performance. Journal of Educational

Psychology, 82, 33-40.

Platt, C. (1988). Effects of causal attributions for success on first-term college
performance: A covariance structure model. Journal of Educational Psycholpgy,
Q, 569-578.

Powell, G. N. (1991). Applicant reactions to the initial employment interview:
exploring theoretical and methodological issues. Personnel Psychology, 44, 67-83.

Robertson, 1. T. & Kandola, R. S. (1982). Work sample tests: Validity, adverse
impact and applicant reaction. Journal of Occupational Psychology, 55, 171-183.

Ryan, A. M. & McFarland, L. A. (1997). Perceived job relatedness as a
moderator of the validity of cogritive ability and biodata instruments. Paper
submitted to the Academy of Management Conference, Boston, MA.

170

 

Ryan, A. M., Greguras, G. J ., & Ployhart, R. E. (1996). Perceived job
relatedness of physical ability testing for firefighters: Exploring variations in
reactions. Human Performance 9 219-240.

 

Rynes, S. L. & Connerley, M. L. (1993). Applicant reactions to alternative
selection procedures. Journal of Business and Psychology, :2, 261-277.

Rynes, S. L. (1993). Who’s selecting whom? Effects of selection practices on
applicant attitudes and behaviors. In N. Schmitt & W. Borman (Eds.), Eempnel
selection in organizations. San Francisco: Jossey-Bass.

Rynes, S. L., Bertz, R., & Gerhart, B. (1991). The importance of recruitment in
job choice: A different way of looking. Personnel Psycholpgy, 32, 729-758.

Schmidt, F. L., Greenthal, A. L., Hunter, J. E., Bemer, J. G., & Seaton, F. W.
(1977). Job sample versus paper-and-pencil trades and technical tests: Adverse
impact and examinee attitudes. Personnel Psychology, 30, 187-197.

Schmit, M. J. (1994). Pre-employapent processes and outcomes, applicant
belief systems and minorigl-majorig group differences. Unpublished doctoral

dissertation, Bowling Green State University, Bowling Green, OH.

Schmit, M. J. & Ryan, A. M. (1992). Test-taking dispositions: A missing link?
Journal of Applied Psychology, 77, 629-637.

Schuler, H. (1993). Social validity of selection Situations: A concept and some
empirical results. In H. Schuler, J. L. Farr, & M. Smith (Eds.), Personnel Selga'op
and Assessment: Individual and Organizational Perthives. Hillsdale, NJ:
Erlbaum.

Schuler, H., Farr, J. L., & Smith, M. (1993). The individual and organizational
sides of personnel selection and assessment. In H. Schuler, J. L. Farr, & M. Smith

(Eds.), Personnel Selection and Assessment: Individual and Organizational
Perspectives. Hillsdale, NJ: Erlbaum.

Smither, J. W., Reilly, R. R., Millsap, R. E., Pearlman, K., & Stoffey, R. W.
(1993). Applicant reactions to selection procedures. Personnel Psychology, 46, 49-
76.

Spence, J. T. & Helmreich, R. L. (1983). Beyond face validity: a comment on
N icholls, Licht, and Pearl. Psychological Bulletin, 94, 181-184.

171

 

Steele, C. S. & Aaronson, J. (1995). Stereotype threat and the intellectual test

performance of African Americans. Journal of Personality and Social Psychology,
62, 797-811. -

Steiner, D. D. & Gilliland, S. W. (1996). Fairness reactions to personnel
selection techniques in France and the United States. Journal of Applied

Psychology, 81, 134-141.

Thorsteinson, T. J. & Ryan, A. M. (1997). The effect of selection ratio on
perceptions of the fairness of a selection test battery. International Journal pf
Selection and Assessment. Manuscript in submission.

Turban, D. B., Sanders, P. A., Francis, D. J., & Osbum, H. G. (1989).
Construct equivalence as an approach to replacing validated cognitive ability

selection tests. Journal of Applied Psychology, 74, 62-71.

Tyler, T. & Caine, A. (198 l ). The role of distributive and procedural fairness in
the endorsement of formal leaders. Journal of Social and Personalig Psychology,
4_1, 643-655.

Tyler, T. & Folger, R. (1980). Distributive and procedural aspects of

satisfaction with citizenship encounters. Basic and Applied Social Psychology, 1,
282-292.

Tyler, T., Rasinski, K., & McGraw, K. (1985). The influence of perceived
injustice on support for political authorities. Journal of Applied Social Psychology,
1_5_, 700-725.

Weiner B. & Kukla (1970). An attributional analysis of achievement motivation.
Journal of Personality and Social Psychology, 15, 1-20.

Weiner, B. (1979). A theory of motivation for some classroom experiences.
Journal of Educational Psychology, 71, 3-25.

Weiner, B. (1985). An attributional theory of motivation and emotion.
Psychological Review. 92. 548-573.

172

 

MICHIGAN STATE UNIV. LIBRRRIES
11111 1111111111 ”11111111111111“!Ill] 111111 ”111111
31293018129456