7:1 .
. 2 u .. no... .to‘wcn .
. .. .u . 3—. . 7 :31. .....—..
.0. o. no . .o. .u. ‘0 ..- .14. o. o
. . A . ... .... P. .— .0.
0.. . ,
... , l..........
. . 1:33. g...
.o .
Q "$.13-
A... 31;...
, . u

v. ;
a Q... .
.73... .u. .o . .
u c o I . .
. . . . . . . . U. .. .9. 1.“ .. . .. ..
. . .. 3.1:.» , .
. A . a‘ca ...
., . . y ‘ . . . , ‘ 0.3 . . . . . . . . ‘0 .
. _ . . a . _ _ 5...“... .m.............. u .
. . _ . . . . . . .u . . . tum“. L2.- 0.33".
. . . 55...“: 0 . . . . . . . . .«.o “Exhi‘ Y
_ :. , .. . . ., .23.»...
on. u .. . . . .. .. ..
. . .. .. . .«Afltmfiurht fl“ .
.n 69.39.7m3 a3 3:»:
.. . . a. ¢ . . . s.
.uwuflmi . ‘39.. Hot-Stuéq . .mrmhfi..‘ int:
99...... .n.3s‘31?3. J p a .
.53! SKVUMPchQvJ-ivms. .
.. n , «R n}.
1 . I.
. I . Jug mg no
. a.

.‘ .o‘. 9. . ..
.3. a... .
3....

. . .
o. .90.... .
. .50....

o
1 o . v
.u .3. o
. . a... n .
Q..-
.. 0.30... c...
. 0.0.1.3.. o

.0.... .
. v..vv7
Q
0‘ fi
. . . I ‘ u
Vi... . . . . .
7....1-3. . .
Lao . . . .
in? “I 1.. . .
‘ . n. . . . a .
”‘12.... T. a. . ..
. . . {3%,
.II; .
OJ, , .
9...: ,
Eu...“ 4.. .-
auc a . . .o. 5
. . 33. a. $53”...
ft..83$€£.w
3!:...‘.‘.:.
J.

o . no.
.9503”. u}:

.n\ .35.! ..
$I$§§ir.'§: “a
~12“... .“

. . t .
0|...3 S‘i .. . .
a 05... . . . .
. . . .. . . .0.“ taut“ " a a:
l. .
. . . . . , PnSLuoLVlnti‘llg
:31: . . .. .«sa. .. k a: in?“
o. .13.. .3: .dflJo ‘mnfimw -
. .. . .
. 5H0;
~05‘
.u.‘ to
01.1.
.00r

.¢-<o.. .
.
‘I. I I
5"}
2.1.3

2' 5.0...
. ‘3‘. 9.. VO“
.0 . .b.-;l
1...... _ . It
.- 5. ti‘: O.
o‘.‘.‘
9.... 1.
.ni

.‘uo

n‘nt

. ‘3.\. cl. -
l... K 0!

.“.-.h~tnl¢ onfii

3s Isn‘t-It

“

1 I o.
'1‘... o.
I. o...

a .

~..

5 o

.. u...

.0“.

fit:

f‘ O} o...

. 00. 1.01, A.
.nO-o‘ot;

10“

p.03 0.. .
.1. . ..
. .. o
. .‘s =3..- e:
. a

. 3
{Cara-“k.
l..1..|1‘.&v\o...
uooc‘..nlt Co‘.‘
.\. ‘13- 32-05.
.1. YK‘.1-4..v.0.
..a$.i53..$..1
$1!...nc’.uwu§av$
. . .cou‘.c\,-!.
.1‘1‘210

»o
. .- 2.2
.. n n.

.I- . a.
. .‘bh...

'c .o- .I.

 

b
. . . .
. .
.. .
o .. . l
. _ ..
o. . .4.
. n . 3:: t... ...
. ... ..:..t: A...
. . '0 i t... .06; .3.
0. .13....2.‘ .31‘3...,
8.....2735... .oa.fl.b.s.
... . en‘s. 11.9%....1. .-
. . .7. 22.6.... .ulgétsgcoo Q
. 7-. . 03‘; 0 ... ‘93..
. . .. . . . . ..
s
.-

 

 

m2.
W;
M/
77715

 

LIBRARY
Mic""e5§:. " State

U: IIVCIasiy

 

 

 

This is to certify that the
dissertation entitled

EMPLOYEE REACTIONS TO PERFORMANCE APPRAISAL:
DEVELOPMENT OF AN INTEGRATIVE FRAMEWORK AND
META-ANALYSIS

presented by

SHAUN MICHAEL PICHLER

has been accepted towards fulfillment
of the requirements for the

PhD. degree in Labor & Industrial Relations

 

 

   

 

' I \
1/ ‘
' Major Professor’s Signature 2/

fill/1400‘?

Date

MSU is an Affinnative Action/Equal Opportunity Employer

- .Iu-IO-II‘I-l-l-l-I_I- -‘-I"-'-"‘-I-'-T-" -'-I‘A-.I-

a.-.-..—--.------.—----o-~-n------o--C-c-o-:---n------'-»—

PLACE IN RETURN Box to remove this checkout from your record.
TO AVOID FINES return on or before date due.
MAY BE RECALLED with earlier due date if requested.

 

DATE DUE DATE DUE DATE DUE

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

5/08 K1/ProVAocaPres/CIRC/DateDueJndd

EMPLOYEE REACTIONS TO PERFORMANCE APPRAISAL: DEVELOPMENT OF
AN INTEGRATIVE FRAMEWORK AND META-ANALYSIS

By

Shaun Michael Pichler

A DISSERTATION

Submitted to
Michigan State University
in partial fulfillment of the requirements
for the degree of

DOCTOR OF PHILOSOPHY
Labor & Industrial Relations

2009

ABSTRACT

EMPLOYEE REACTIONS TO PERFORMANCE APPRAISAL: DEVELOPMENT OF
AN INTEGRATIVE FRAMEWORK AND META-ANALYSIS

By
Shaun Michael Pichler

Performance appraisal is one of the most heavily researched topics in human
resource management, and employee reactions to appraisals are an important outcome of
the appraisal process (some scholars and practitioners say the most important outcome),
yet there has been a critical science-practice gap in this literature in that research has not
produced a coherent understanding of why employees react in different ways to
appraisals. In response to criticisms from the scholarly and practitioner communities
both, research has shifted its focus to contextual aspects of the appraisal, and to employee
reactions to appraisals. The context of performance appraisal and employee reactions to
appraisals are highly interrelated, yet there has been no comprehensive review of the
employee reaction literature, and no integrative framework exists with propositions about
when and why contextual antecedents should be related to employee reactions.

The purpose of this dissertation is to provide such an integrative framework in
order to organize the literature on employee reactions to performance appraisal, including
mediating mechanisms whereby contextual antecedents are related to reactions, as well as
potential moderators of the relationships between contextual antecedents and reactions.
Organizational justice theory is used as an over-arching theoretical lens to develop
hypotheses about these relationships. A related purpose is to test, using meta-analytic

correlations and multivariate analyses with these meta-analytic correlations, some of the

relationships articulated by this framework, as to guide future theory, research and
practice related to performance management and appraisal.

Results indicated that social contextual variables (supervisor support, supervisor
trust, supervisor satisfaction and supervisor-subordinate relationship quality) were all
similarly related to appraisal reactions (at or around Mr = .60); were most strongly
related to perceptions of interactional justice (as compared to distributive and procedural
justice); that relationships between social contextual variables and appraisal reactions
were partially mediated by perceptions of organizational justice. Results also indicated
that relationships between due process performance appraisal characteristics (adequate
notice, fair hearing and judgment based on evidence) and appraisal reactions were
mediated (fully or partially, depending on the antecedent) by perceptions of
organizational justice. Performance appraisal rating favorability moderated the
relationship between feedback frequency (an aspect of adequate notice) and appraisal
reactions, but did not moderate the relationship between voice (instrumental and value-
expressive) and appraisal reactions.

These results support key propositions of organizational justice and social
exchange theory, as well as the due process model of performance appraisal developed by
Folger Konovsky, & Cropanzano (1992). They indicate that employees react
differentially to performance appraisals based on their perceptions of the fairness of the
appraisal, in terms of different substantive aspects of organizational justice, and that it is
important to consider rating favorability as a moderator of relationships between

contextual antecedents and appraisal reactions.

To my parents and grandparents -- who have always been my inspiration in work and life.

iv

ACKNOWLEGMENTS

Completing a Ph.D. is a time-consuming, challenging and self-revelatory
experience that is best done with the support and encouragement from advisors, friends
and family. I would like to express my deep and sincere thanks to those persons who
have helped me throughout this humbling process.

My advisor and committee chair, Mark Roehling, was truly the best chair a
doctoral student could ask for, and for that I owe him a debt of gratitude. He challenged
me to write and develop a proposal and, ultimately, a dissertation that was truly reflective
of my full range of academic skills, and would not accept anything less. I would not be as
proud of the final result of my academic training, my dissertation, if it weren’t for his
very astute guidance. I am honored to call him a colleague.

I would also like to thank each of my committee members, Ellen Kossek, Fred
Oswald and Mike Moore for providing critical feedback in a supportive and
developmental way. My dissertation benefitted tremendously from their knowledge and
insight. My committee members were always constructive and helpful, and made the
dissertation process more productive and, yes, more enjoyable for me than I could have
expected.

Finally, I would like to thank my friends and family. Without their patience,

support and consistent encouragement, this dissertation would not be possible.

TABLE OF CONTENTS

LIST OF TABLES .......................................................................... v
LIST OF FIGURES ........................................................................ vi
CHAPTER 1 - INTRODUCTION ......................................................... 1

CHAPTER 2 - EMPLOYEE REACTIONS AS A CRITERION OF
PERFORMANCE APPRAISAL EFFFECTIVENESS.................... .....6

2.1 The Psychometric Approach to Performance Appraisal .................. 6
2.2 Performance Appraisal Context and Employee Reactions to

Performance Appraisal ............................................................. 7
2.3 Existing Research on Employee Reactions ................................. 8
2.4 Limitations of Extant Research on Employee Reactions ................ 10

CHAPTER 3 - OVERVIEW OF ORGANIZATIONAL JUSTICE THEORY....12

3.1 Distributive Justice ............................................................ 13
3.2 Procedural Justice ............................................................ 14
3.3 Interactional Justice .......................................................... 16
3.4 Research Supporting the Theoretical Distinctions Between Justice
Dimensions ..................................................................... 16

CHAPTER 4 - A REVIEW AND INTEGRATION OF CONTEXTUAL
ANTECEDENTS OF EMPLOYEE REACTIONS PART I: THE SOCIAL

CONTEXT OF APPRAISAL ................................................... 21
4.1 An Overview of The Social Context of Performance Appraisal ....... 23
4.2 Relationship Quality ......................................................... 24
4.3 Supervisor Trust ............................................................... 27
4.4 Supervisor Support ............................................................ 28
4.5 Supervisor Satisfaction ...................................................... 29

CHAPTER 5 - A REVIEW AND INTEGRATION OF CONTEXTUAL
ANTECEDENTS OF EMPLOYEE REACTIONS PART II: DUE

PROCESS PERFORMANCE APPRAISAL .................................. 37
5.1 The Psychometric and Due Process Models of Performance
Appraisal ............................................................................ 37
5.2 Adequate Notice and Employee Reactions ................................ 39
5.3 F air Hearing and Employee Reactions ..................................... 48
5.4 Judgment Based on Evidence and Employee Reactions ................. 61
CHAPTER 6 - METHODS ............................................................... 67
6.1 Meta-Analytic Method ....................................................... 67
6.2 Literature Search and Inclusion Criteria ................................... 69

vi

CHAPTER 7 - RESULTS ................................................................. 74

7.1 Meta-Analytic Correlations .................................................. 74
7.2 Tests of Mediation ............................................................ 79
7.3 Tests of Moderation .......................................................... 84
CHAPTER 8 - DISCUSSION ............................................................ 86
8.1 Social Context of Performance Appraisal ................................. 99
8.2 Due Process Performance Appraisal ......................................... 93
8.3 Additional Limitations and Future Research Directions ............... 105
8.4 Conclusion ................................................................... 108
APPENDICES ........................................................................... 1 10
APPENDIX A
Definitions of Due Process Performance Appraisal Dimensions and Sub-
Dimensions and Corresponding Chapter Sections ................................... 1 10
REFERENCES ........................................................................... 1 11

vii

LIST OF TABLES
Table 1: Meta-analytic Results for Employee Reactions to Performance Appraisal ...... 75

Table 2: Meta-analytic Results for Employee Reactions to Performance Appraisal: Meta-
Analytic Correlations for Hypothesis Tests and Path Analyses .............................. 77

Table 3: Results of Continuous Variable Moderator Analysis in Regression for Employee
Reactions ............................................................................................. 79

Table 4: Meta-Analytic Correlation Matrix for Justice Variables (Used as Input Data for
Path Analysis) ....................................................................................... 84

viii

LIST OF FIGURES
Figure 1: An Integrative Framework of Employee Reactions to Performance Appraisal...4

Figure 2: Model of the Relationship Between Social Context, Organizational Justice
Variables and Employee Reactions ............................................................... 80

Figure 3: Model of the Relationship Between Knowledge of Performance Standards,
Distributive and Procedural Justice and Employee Reactions ................................. 81

Figure 4: Model of the Relationship Between Supervisor Job Knowledge, Distributive
Justice and Employee Reactions .................................................................. 82

Figure 5: Model of the Relationship Between Supervisor Lack of Bias, Distributive and
Interactional Justice and Employee Reactions ................................................... 83

Figure 6: Model of the Relationship Between Job Relevant Factors, Procedural Justice
and Employee Reactions ........................................................................... 84

ix

CHAPTER 1
INTRODUCTION

Performance appraisal (PA) is perhaps the most ubiquitous human resource
management tool used in organizations (Rynes, Gerhart & Parks, 2005). Scholars
(Folger, Konovsky and Cropanzano, 1992) and practitioners alike (Thomas & Bretz,
1994) contend that employee reactions to performance appraisal are key criteria by which
performance appraisals should be judged, given inconsistencies across raters in
performance evaluations and the importance of buy-in to the process among both raters
and ratees (Murphy & Cleveland, 1995). Since performance appraisals have such
significant implications for organizations in terms of performance management and the
allocation of scarce resources, persons administering and being evaluated by a
performance appraisal must evaluate and react to it positively in order for the appraisal
process to be effective (Bernardin & Beatty, 1984; Cardy & Dobbins, 1994; Folger et al.,
1992)

Scholars and practitioners have noted, however, that managers and employees
alike are ofien dissatisfied with the performance appraisal process (e.g. Taylor et al.,
1995), and that extant research has not adequately addressed how to increase
performance appraisal acceptance. This has lead to a critical science-practice gap (e. g.
Balzer & Sulsky, 1990). In fact, practitioners are more interested in employee reactions to
performance appraisalI than their psychometric properties (Balzer & Sulsky, 1990). Since
the key purpose of performance appraisal is to provide employees with formal

performance feedback as to improve subsequent performance (Tziner, Murphy, &

 

' Employee reactions to performance appraisal will be referred to simply as “employee reactions” for the
remainder of the dissertation for purposes of brevity.

Cleveland, 2005), and since acceptance of appraisal feedback is a necessary precursor to
changing behavior, i.e. performance (Carver & Scheier, 1981; Kluger & DeNisi, 1996), it
seems paramount to better understand the factors that predict employee reactions to
performance appraisal.

In response to calls for research on contextual and qualitative aspects of
performance appraisal (e. g. Murphy & Cleveland, 1991; 1995), recent research has
focused on due process characteristics of performance appraisal (F olger et al., 1992), the
social context of appraisals (Levy & Williams, 2004), perceptions of justice in appraisals
(Erdogan, 2002), as well as overall reactions to the appraisal process (Cawley, Keeping &
Levy, 1998). Each of these research streams is related in important ways to ratee (and
rater) evaluations of the performance appraisal process. While qualitative reviews have
highlighted the importance of each aspect of performance appraisal, no review has
integrated these topics—despite their conceptual and theoretical overlap. Moreover, no
published research has systematically and comprehensively reviewed the extant research
on employee reactions to performance appraisal.

This is a problem because the literature on employee reactions is a) lacking a
cohesive framework that organizes relevant antecedents into conceptually meaningful
groups, and b) is lacking an explication of the mediating mechanisms whereby
antecedents are related to employee reactions. The lack of such an overarching
framework has lead to an apparent lack of consistency in the literature in terms of
substantive relationships studied, a lack of clarity in terms of the theoretical contributions
of this literature, and a lack of a meaningful assessment of cumulative findings in the

literature. Thus not only is the employee reaction literature limited in its ability to

contribute to managerial understanding of why certain antecedents are important as
related to employee reactions, the employee reaction literature has probably not made as
big an impact as possible on OB/HR theory.

The purpose of the current dissertation, therefore, is to provide an integrative
framework or process model2 that organizes the existing literature by grouping
theoretically related antecedents, i.e. social contextual variables and due process
variables; developing hypotheses about why these antecedents should be related to
employee reactions, i.e. mediated by perceptions of organizational justice; and by 1
developing hypotheses about the boundary conditions of these relationships, i.e.
moderation by rating favorability. Meta-analysis will be used to estimate some of the key
relationships implied by the model. The integrative framework proposed in this
dissertation will not only organize exiting research conceptually, it will also provide the
basis for testing cumulative relationships, and will identify areas for future research and

theoretical development. See Figure 1 for an illustration of the process model3

 

2 The terms integrative framework and process model will be used synonymously.
3 Similar to other conceptual models used to guide meta-analysis (e.g. Cohen-Carash & Spector. 2001). this
is model is for illustrative purposes and is not intended to be a structural model.

 

 

Appraisal Context

 

 

 

 

Justice Perceptions

Figure 1. An Integrative Framework of Employee Reactions to Performance Appraisal

 

 

 

 

 

Social Context
Relationship Quality
Supervisor Trust
Supervisor Support

Supervisor Satisfaction

 

 

 

 

Due Process Characteristics
AdMNotice
Knowledge of Standards
Frequent Feedback
Fair Hearing
ParticipationNoice
Supervisor Job Knowledge
Judgment Based on Evidence
Supervisor Neutrality
Rating Instrument
Job-Relevant Factors

Appeals Process

 

 

 

Appraisal Reactions

 

 

Distributive
Justice

Interactional
Justice

Procedural
Justice

 

 

 

 

 

V

 

Accuracy
Fairness
Procedural Justice

Motivation to
Improve

Satisfaction

Utility

 

 

 

This study will examine the interconnectedness between appraisal context,
perceptions of justice and employee reactions to performance appraisal, and will also test
some primary tenets of organizational justice theory. While employee reactions have
received increased research attention in recent years, performance appraisal researchers
have been interested in employee reactions as an important outcome of the appraisal
process for decades (e.g. Zander & Gyr, 1955).

The following chapter introduces the concept of employee reactions as a criterion
of appraisal effectiveness, and describes those primary constructs typically measured as
employee reactions. Chapter 3 will then provide a review of organizational justice theory
and research in order to provide the theoretical foundation for explaining relationships
between performance appraisal context, justice perceptions and employee reactions.
Chapters 4 and 5 will review the literature on relationships between performance
appraisal context and employee reactions. These chapters will develop the integrative
framework that organizes the existing literature, and will develop hypotheses based on
justice theory about how and why appraisal context should be related to employee
reactions. Chapter 6 will then explain the meta-analytic methods that will be used to test

parts of this model.

CHAPTER 2
EMPLOYEE REACTIONS AS A CRITERION OF PERFORMANCE APPRAISAL
EFF FECTIVENESS
2.1 The Psychometric Approach to Performance Appraisal

Although researchers have expressed interest in employee reactions for some time
now (e. g. Zander & Gyr, 1955), until recently most performance appraisal research
focused on what has been called the psychometric or test approach (F olger et al., 1992) to
performance appraisal effectiveness (Murphy & Cleveland, 1995). The psychometric
approach focused on increasing rating accuracy and reducing rating errors. The
underlying assumption of this model was the accuracy is a key dimension of appraisal
effectiveness. The psychometric model of performance appraisal assumes that ratings that
are free from halo, leniency and range restriction are accurate (c.f. Saal, Downey, &
Lahey, 1980), and that developing rating instruments that reduce biases (the “rating
format” perspective; Landy & Farr, 1980) or training raters to avoid these biases (the
“cognitive” perspective; Feldman, 1981) will increase rating accuracy,

Scholars have noted that the psychometric model is limited in its capacity to
consider contextual factors in performance appraisal (Arvey & Murphy, 1998; Murphy &
Cleveland, 1995), and that rating accuracy is also a function of these factors (Cardy
Dobbins, 1994; Murphy & Cleveland, 1995). Indeed, Ilgen, Bames-Farrell and McKellin
(1993) argued that performance appraisal research needed to move beyond both the rating
format and cognitive perspectives of performance appraisal, as to better inform
performance appraisal in theory and, especially, in practice. Building on the psychometric

approach, Cardy and Dobbins (1994) identified three primary criteria of performance

appraisal effectiveness: rater errors, rating accuracy and qualitative criteria. Employee
reactions to performance appraisal are a major component of qualitative criteria (Murphy
& Cleveland, 1995). Indeed, Levy and Williams (2004) later replaced qualitative criteria
with employee reactions in their model of appraisal effectiveness.
2. 2 Performance Appraisal Context and Employee Reactions to Performance Appraisal

Partly in response to critical reviews (e.g. Ilgen et al., 1993), research on
performance appraisal shifted away from the psychometric approach, and increasingly
focused on qualitative aspects of appraisals, such as employee reactions. Researchers
(e. g. Cawley et al., 1998; Levy & Williams, 2004) have not proposed an operational
definition of employee reactions, but employee reactions can be defined as individual-
level attitudinal evaluations of and responses to the performance appraisal process.
which include perceptions of appraisal accuracy, fairness (overall fairness, as well as
perceptions of performance appraisal distributive, interactional and procedural justice)
and utility, satisfaction with the appraisal, and motivation to improve performance.

In order to understand how contextual variables are related to employee reactions,
it is important to briefly explain each of these reactions (also see Cawley et al., 1998;
Keeping & Levy, 2000). Accuracy is the perception that one’s performance rating
accurately reflects one’s actual performance. Fairness is an overall evaluation of the
perceived fairness of a performance appraisal. Motivation to improve is the perception
that one is motivated to increase his/her performance based on the appraisal. Satisfaction
is an overall evaluation of how satisfied one is with the appraisal. Utility is a perception
that the appraisal was worthwhile and effective in terms of evaluating work performance.

Finally, performance appraisal procedural and distributive justice are adaptations of

justice variables to specifically reflect perceptions of the fairness of policies/procedures
and outcomes of the appraisal process, respectively. Performance appraisal procedural
and distributive justice differ from fairness perceptions in the in that the former two
measure specific aspects of appraisal fairness, and the latter is an overall evaluation of
fairness. While not included in the Keeping and Levy (2000) study, recent research has
also begun to consider perceptions of interactional justice as related to performance
appraisal (e.g. Buehler, 2006; Elicker, 2000; Johnson, 2003).

2.3 Existing Research an Employee Reactions

Keeping and Levy (2000) found that the variables used to measure these
constructs generally performed well, i.e. represented distinct constructs, and that
appraisal satisfaction could be effectively separated into session satisfaction and system
satisfaction. Session satisfaction reflects how satisfied one is with the appraisal session;
system satisfaction reflects how satisfied is with the performance appraisal system. It is
important to distinguish between session and system satisfaction given their conceptual
differences, and differential relationships with correlates (Cawley et al., 1998; Keeping &
Levy,2000)

Cawley, Keeping and Levy (1998) conducted a meta-analysis of this literature,
which included 27 field studies and demonstrated that employee participation in the
appraisal is an important contextual antecedent of employee reactions. Since research
throughout the 1990’s continued to focus on employee reactions as a criteria of appraisal
effectiveness, Keeping and Levy (2000) evaluated the measurement properties of
employee reaction variables, namely perceived accuracy, perceived fairness, motivation

to improve, appraisal satisfaction and perceived utility, as well performance appraisal

procedural justice and distributive justice. The first five reaction variables were included
in the Cawley et a1. (1998) study, whereas the last two were not. This is important to note
since, while Keeping and Levy (2000) consider justice perceptions to be appraisal
reactions much like perceptions of overall fairness and satisfaction, justice perceptions
are not necessarily treated as such in the literature, and should theoretically precede these
reactions, as will be explained in the following chapter.

Scholars and practitioners have been critical of the psychometric approach to
performance appraisal effectiveness, suggesting that it is important to take a more
“qualitative” or context-based approach to appraisal effectiveness. While the conceptual
and empirical research has helped to establish employee reactions as important
qualitative criteria of appraisal effectiveness (Cawley et al., 1998; Keeping & Levy,
2000; Levy & Williams, 2004), the literature on employee reactions has grown
considerably since the time of the Cawley et al. (1998) review. Indeed, Levy & Williams
(2004) identified 20 new studies on employee reactions in their review of the social
context of performance appraisal. The present literature review identified 84 studies with
94 unique samples that met inclusion criteria for analysis (see Chapter 6), or 57 studies
on employee reactions that were published since or were otherwise not included in the
Cawley et al. (1998) meta-analysis.

Therefore, it seems important, not to mention timely, to reconsider the
relationships tested in the Cawley et al. (1998) study, as the authors themselves
recommended future researchers do, as well as other theoretically meaningful
relationships between performance appraisal context and employee reactions. Indeed,

while there have been a number of studies related to employee reactions since the Cawley

et a1. (1998) study, they do not necessarily build upon or complement one another in
terms of theory testing or development, most likely because this research has not been
organized into an integrative framework.

2. 4 Limitations of the Extant Literature on Employee Reactions

While it employee reactions are no longer “neglected criteria” (Murphy &
Cleveland, 1995, pg. 310) in terms of the volume of studies related to these criteria, this
literature is still neglected in the sense that it lacks cohesion and real developmental
progress in terms of the types of antecedents studied in the literature and the theoretical
contributions of this literature. While researchers are still interested in relationships
between employee participation and employee reactions (e.g., Jawahar, 2006; Kavanagh,
Benson, Brown, 2008), some are critical of the finding by Cawley et al. (1998) that
value-expressive voice is more strongly related to employee reactions than is
instrumental voice (Bonness & Macan, 2006; Sub, 1992). This will be considered in more
detail in chapters to follow.

There have also been a number of studies in recent years that test relationships
between social contextual variables and employee reactions (Williams & Levy, 2004). A
variety of variables have been used to represent the social context of appraisal, including
rater-ratee relationship quality, supervisor support, supervisor satisfaction, and supervisor
trust (see chapter 4). This was not articulated in the review by Levy and Williams (2004),
however, and it is therefore important to consider how these antecedents are interrelated,
and how they are related in similar ways to employee reactions.

While no existing review has identified this trend, my review of the literature also

uncovered that a number of studies have focused on relationships between due process

10

appraisal characteristics (Folger et al., 1992) and employee reactions (Buehler, 2006;
Taylor et al., 1995). Almost none of these studies does so using the due process model
proposed by F olger et al. (1992), however, thus limiting the contribution these studies
make to the understanding of due process and employee reactions. It accordingly seems
important to integrate this literature in order to utilize and build upon the due process
model of performance appraisal as it relates to appraisal effectiveness in terms of
employee reactions.

While most of the existing research has focused on the antecedents of employee
reactions, given that scholars assume they are important criteria in and of themselves
(Bernardin & Beatty, 1984; Cardy & Dobbins, 1994; Folger et al., 1992; Williams &
Levy, 2004), research has found that employee reactions are related to important
outcomes measured after the performance appraisal (e. g. Blau, 1999; Jawahar, 2006;
Kinicki et al., 2004). As would be expected based on theory, recent research by Pettijohn
and colleagues (Pettijohn, Pettijohn Taylor, & Keillor, 2001; Pettijohn, Pettijohn, &
d’Amico, 2001) found that employee reactions can and do influence job attitudes, namely
job satisfaction and organizational commitment, and Jawahar (2006) found that employee
reactions influence future performance. While the current study focuses on the
antecedents of employee reactions, existing scholarship suggests that employee reactions
are important criteria, and that these reactions are related positively to subsequent job-

related attitudes and behaviors.

11

CHAPTER 3
OVERVIEW OF ORGANIZATIONAL JUSTICE THEORY

The purpose of this chapter is to provide an overview of organizational justice
theory. Chapters to follow will build upon this overview, and will explicate relationships
between appraisal context and employee reactions. Research on organizational justice has
focused largely on two types of justice perceptions, distributive and procedural justice.
The concept of distributive justice is based on several theories about the content, i.e.
outcomes, of decisions, whereas procedural justice is based on theories about the process
by which decisions are made (Greenberg, 1990). More recently, research has addressed
the concept of interactional justice, which is comprised of interpersonal justice, i.e. the
fairness of interpersonal treatment one receives from authorities in terms of dignity and
respect, and informational justice, i.e. the adequacy of explanations or justifications for
decisions that one receives from authorities (Bies & Moag, 1986; Bies, 1987).
Cumulatively, the application of distributive, procedural and interactional justice theories
to organizational behavior represents the domain of organizational justice (Greenberg,
1990).

Greenberg’s (1987) taxonomy of organizational justice differentiated theories
according to two dimensions, resulting in four distinct types of justice theories. The first
dimension, reactive-proactive, classifies theories according to whether the emphasis is on
reactions to unfair situations or on preventing unfair situations. The second dimension,
process-content, classifies theories according to whether the emphasis is on the fairness
of procedures (procedural justice) or outcomes (distributive justice). Most of the research

in the performance appraisal domain has been influenced by reactive justice theories, and

12

has been both process- and outcome-oriented, as will be seen below. Interactional justice
research might also be thought of as reactive-oriented since it focuses on employee
evaluations of interpersonal treatment from authority figures. Of course, this was not
included in Greenberg’s (1987) taxonomy since the concept of interactional justice was
not yet introduced.

The following sections will review primary theories of distributive, procedural
and interactional justice in that order, given that this is the order in which the concepts
were introduced chronologically in the OB/HR literature (Greenberg, 1990).

3.1 Distributive Justice

As Greenberg (1990) explains in his review, two leading theories of distributive
justice are Adam’s (1963) equity theory (a reactive theory) and Leventhal’s (1980) justice
judgment model (a proactive theory). As applied to the workplace, equity theory
postulates that workers compare their perceived inputs (work performance) and perceived
outputs (rewards) to similar others, and modify their behavior and/or cognitions when
their ratios are different. That is, when one’s ratio is higher (lower) than a referent, he or
she is inequitably overpaid (underpaid), which will lead to changes in behavior, i.e.
increased (reduced) work performance, or cognitions, i.e. re-evaluation of one’s work
performance. This proposition can and has been applied to the performance appraisal
context, such that one’s job attitudes and work-performance should be negatively affected
when one receives a performance rating that one believes does not accurately reflect
one’s work performance (e.g. Ilgen et al., 1979; Kinicki et al., 2004).

Leventhal’s (1980) justice judgment model stipulates that social harmony and

optimal work performance are maintained by the equal distribution of rewards. Both

13

types of content theories would suggest that when an employee feels he or she has been
given an unfair performance rating, i.e. one that is lower than deserved, perceptions of
distributive justice will decrease, and the employee will react negatively, for instance by
becoming dissatisfied and demotivated (Greenberg, 1987). Conversely, perceptions of
outcome fairness are related positively to reactions to decisions (Greenberg, 1987).

3. 2 Procedural Justice

While distributive justice theory is relevant to employee reactions to performance
appraisal, procedural justice theory has had the greatest impact on the performance
appraisal literature (e.g. Folger et al., 1992). Reactive process theories, i.e. procedural
justice theory, stipulate that to the extent processes and procedures used to arrive at a
decision are fair, perceptions of procedural justice increase, as will the acceptance of and
reactions to the outcome of the process (Greenberg, 1987; 1990).

There are several theories of procedural justice, all of which assume that
perceptions of the fairness of processes by which decisions are made are related to
reactions to those processes and their decision outcomes, but that differ in the way
procedural justice is conceptualized. Seminal work by Thibaut and Walker (1975)
indicated that disputants in a dispute-resolution context desired both decision control, i.e.
control over the ultimate outcome of the process, and process control, i.e. control over the
procedures used to arrive at an outcome. While the focus of Thibaut and Walker (1975)
was on legal contexts, the theory was effectively applied to a variety of human resource
systems, foremost among them being performance appraisal. Performance appraisal

research has consistently documented that the opportunity to express oneself and one’s

14

viewpoints during an appraisal is related to employee reactions to the appraisal (Cawley
et al., 1998).

While the control perspective developed by Thibaut and Walker (1975) has been
influential in organizational research, other conceptualizations of procedural justice have
been proposed, which have been applied to the performance appraisal context. Notably,
Lind and Tyler (1988) proposed the self-interest and group-value models of procedural
justice, providing rationale for why fair processes are related to procedural justice
perceptions. The self-interest or instrumental model of procedural justice proposes that
people want voice in processes because they can more effectively influence the outcome
of the processes, which is similar to the decision control perspective above. The group-
value or value-expressive model, on the other hand, contends that the opportunity to
express one’s viewpoint in a process has justice-enhancing properties, regardless of the
extent to which outcomes are actually affected, because this opportunity reflects group
status. As applied to organizational research, the group value model suggests that
employees value voice and participation in decision-making processes that affect them,
such as performance appraisal, because it demonstrates that they are a valued party to a
long-term social exchange, and that authorities value their input (e.g. Tyler, 1989).

The process control, self-interest and group-value models of procedural justice
have greatly informed and influenced performance appraisal research, especially in terms
of the fairness of performance appraisals. That said, Leventhal’s (1980) model of
procedural justice elements is also relevant to employee reactions. The author proposed
that there are specific criteria by which processes such as performance appraisal are

evaluated in terms of fairness, namely: bias suppression, consistent allocation of rewards,

15

reliance on accurate information, the correctability of allocation decisions, the
representativeness of decisions as related to each party’s concerns, and the extent to
which allocation decisions are based on moral and ethical standards. As will be explained
in more detail later, this model of procedural justice is highly consistent with the due
process model of performance appraisal (Folger et al., 1992).
3. 3 Interactional Justice

Interactional justice represents the fairness of interactions through which
decision-making processes occur, e. g. interactions with one’s supervisor during a
performance appraisal (Bies & Moag, 1986; Bies, 1987). Some scholars have proposed
that interactional justice is an aspect of procedural justice, whereas others have suggested
that interactional justice is a distinct concept which can be further decomposed into
interpersonal and informational justice factors (Greenberg, 1993). Empirical research
generally supports the contention that interactional justice is distinct from procedural
justice (Colquitt, 2001; Colquitt et al., 2001). This theory and research proposes that
when employees feel that they have been treated fairly interpersonally, perceptions of
ineractional justice will increase, which will affect reactions to those persons with whom
interactions occur, as well as the organizational systems that constrain and determine
these reactions (Bies & Moag, 1986; Colquitt et al., 2001; Greenberg, 1983)
3. 4 Research Supporting the Theoretical Distinctions Between Justice Dimensions

Since most of the organizational justice research has focused on distributive and
procedural justice, one of the primary ways in which researchers have established that
these two dimensions are distinct, i.e. a two-factor model or differential effects approach

(e. g. Ambrose, Hess, & Ganesan, 2007) is to demonstrate their differential relationships

16

with correlates. Theoretically, procedural justice should be related to variables that reflect
long-term relationships, e.g. between employees and their supervisor or their
organization, whereas distributive justice should be related to specific outcomes (Lind &
Tyler, 1988; Greenberg, 1990). Indeed, organizational justice research has indicated that
procedural justice is related to variables that reflect relationship-oriented attitudes,
whereas distributive justice is related to variables that reflect outcome satisfaction (Folger
& Konovsky, 1989; McFarlin & Sweeney, 1992).

Colquitt (2001) also demonstrated that interactional justice is distinct from both
distributive and procedural justice. The author found support for the construct validity of
a four-factor model of organizational justice (i.e. distributive, procedural, interpersonal
and informational) in two separate studies. Not only were the factors empirically distinct,
they were differentially related to respective correlates. In study 1, for instance, the
author proposed that distributive justice should be uniquely related to outcome
satisfaction; that interactional justice should predict evaluations of the agents that enact
processes, whereas procedural justice should predict evaluations of the processes
themselves; and that informational justice should be uniquely related to collective self-
esteem, since it is related to reduced secrecy and dishonesty. The proposed model was
supported, except that a path from procedural justice to leader evaluation was added. This
added path is also consistent with procedural justice theory which in that procedural
justice perceptions reflect not only evaluations of systems, but also of the agents involved
in administering those systems, as will be seen in Chapter 4.

Meta-analytic studies have also supported the distinctiveness of the

aforementioned justice dimensions. Cohen-Carashand Spector (2001) conducted a meta-

17

analysis of the organizational justice literature, focusing on differential relationships
between justice constructs and various correlates. They found that the “big three”
(Colquitt et al., 2001 ) justice constructs were moderately correlated, but distinct. The
authors did not separate interpersonal from informational justice because of the debate
about the theoretical meaningfulness of separating the constructs, and because too few
studies reported differential correlations. Also as expected based on theory, the three
types of justice, i..e. distributive, procedural and interactional, were differentially related
to constructs representing outcomes, organizational procedures and interpersonal
treatment, respectively.

Colquitt and colleagues (2001) also conducted a meta-analysis of the
organizational justice literature. Their study included meta-analytic estimates of
relationships between procedural justice, interactional justice, interpersonal justice
distributive justice, overall procedural fairness perceptions and correlates such as trust,
outcome satisfaction, job satisfaction, organizational commitment and evaluation of
authority. Like Cohen-Carashand Spector (2001), the authors found that justice
dimensions were positively correlated, but represented distinct constructs, and were
related differentially to correlates in expected ways.

It should be noted that some of the relationships reported in the Colquitt et al.
(2001) study are relevant theoretically to the present study, such as relationships between
justice and trust and outcome satisfaction. Indeed, several studies in the present database
were included in the Colquitt et al. (2001) study. The focus of the present study is on
estimating relationships between performance appraisal context and employee reactions,

as opposed to substantive relationships with justice constructs and various correlates.

18

Moreover, the constructs included in the Colquitt et al. (2001) study were more general
than those in the present study. For instance, while the authors estimated relationships
between justice dimensions and outcome satisfaction, outcome satisfaction was not
restricted to performance appraisal satisfaction. Moreover, perceptions of justice were not
necessarily related specifically to the performance appraisal context. Thus, previous
meta-analytic research is supportive of the hypotheses developed below, but does not
necessarily mean that relationships will be of the same form or magnitude in the present
study.

In the sections below, hypotheses will be developed based on organizational
justice theory that involve differential relationships between contextual antecedents and
justice constructs. I hypothesize that social contextual variables will be more strongly
related to interactional justice than procedural and distributive justice. Differential
predictions between justice dimensions and employee reactions will not be made since
these reactions do not neatly map onto the types of attitudes typically studied as outcomes
of j ustice perceptions in the organizational behavior/human resource management
literature (see Colquitt et al., 2001; Greenberg, 1990).

Consistent with existing theory and research, a three-factor model of
organizational justice will be adopted for the current study. Similar to other meta-
analyses (i.e. Cohen-Carash & Spector, 2001), the present study does not differentiate
between informational and interactional justice since there is some debate about whether
or not informational and interpersonal justice should be treated separately (Ambrose,

Hess, & Ganesan, 2007; Cohen-Carash& Spector, 2001; Greenberg, 1993), and since too

19

few studies in the current database measured these constructs separately as to be able to
make consistent, meaningful distinctions.
Summary

The preceding chapter explained the concept of organizational justice, introduced
theories related to the different types of organizational justice, and reviewed literature
which has demonstrated the construct and differential validity of these justice
dimensions. Cumulatively, construct validity and differential effects approaches have
demonstrated that the aforementioned justice dimensions, i.e. distributive, interactional
and procedural, and related in unique ways to important organizational attitudes and
behavior (Cohen-Carash& Spector, 2001; Colquitt, 2001; Colquitt et al., 2001;
Greenberg, 1990). In the following two chapters, theory and research in the
organizational justice literature will be used to develop hypotheses about relationships

between performance appraisal contextual variables and employee reactions.

20

CHAPTER 4
A REVIEW AND INTEGRATION OF CONTEXTUAL ANTECEDENTS OF
EMPLOYEE REACTIONS PART I: THE SOCIAL CONTEXT OF APPRAISAL

A significant limitation of existing models of the appraisal process is that they do
not clearly represent how contextual variables are related to the employee reactions
typically studied in the literature. Indeed, existing models of the appraisal process are not
models of employee reactions per se. Yet there is a need for such a model given the
importance of employee reactions to appraisal effectiveness. Thus, a primary purpose of
this dissertation is to develop such a model.

The purpose of the following two chapters is to review and organize the existing
literature related to employee reactions by creating an integrative framework that more
effectively organizes and models the contextual antecedents of employee reactions.
Murphy and Cleveland (1995) defined performance appraisal context as “a heterogeneous
mix of factors, ranging from the social and legal system in which the organization exists
to the climate and culture within the organization” (pg. 31). While the appraisal context
in general may be heterogeneous, a thorough review of the employee reaction literature
identified two major categories of contextual variables as antecedents of employee
reactions: 1) the social context of appraisal (Levy & Williams, 2004), and 2) due process
performance appraisal characteristics (Folger et al., 1992), which includes participation in
the appraisal (Cawley et al., 1998).

A graphic illustration of this framework is provided in Figure 1. Like other
conceptual models involving multiple meta-analytic estimates (e.g. Cohen-Crash &

Spector, 2001), this illustration is not meant to identify all of the hypotheses or

21

relationships that will be proposed in this study, or all of the potential relationships
between constructs. Instead, the figure is meant to a) group the constructs that are
included in the two major categories of contextual antecedents of employee reactions and
b) illustrate that perceptions of organizational justice are hypothesized to mediate
relationships between contextual antecedents and employee reactions.

Scholars have proposed various models of contextual aspects of performance
appraisal (Erdogan, 2002; Levy & Williams, 2004; Murphy & Cleveland, 1995), but they
do not always deliberately consider employee reactions as criterion of the appraisal. For
instance, the model of antecedents and consequences of justice perceptions in
performance appraisal proposed by Erdogan (2002) includes justice perceptions as a
mediator of the relationship between contextual antecedents and job-attitudes and
performance, but does not include other commonly studied employee reactions as
outcome variables, besides motivation to improve. The model of the social context of
appraisal developed by Levy and Williams (2004) considers employee reactions and
justice perceptions as outcome variables, but they also include performance ratings and
“rater/ratee behavioral reactions” as “rater and ratee behavior” outcomes. Models of the
performance appraisal context also regularly include distal factors which may be
relatively unrelated to employee reactions, such as legal (6. g. Murphy & Cleveland) and
economic (Levy & Williams, 2004) conditions.

As a group, existing models also suffer from inconsistent categorizations of
antecedent constructs. More specifically, existing models of contextual antecedents
sometimes consider the same variables, but categorize them using different labels,

depending on the specificity of the particular model (as will be demonstrated in more

22

detail below). An additional limitation of exiting models is that they do not represent the
types of constructs that are typically measured in research reports. Since these constructs
are consistently measured as antecedents, and they are related in important ways
theoretically to employee reactions, it is concerning that existing models do not take into
consideration the types of relationships being modeled in individual studies. A related
problem with the employee reaction literature is that it is lacking a cohesive framework,
as noted in detail above.

The sections immediately below review and integrate the research literature on
the social context of appraisal. Chapter 5 will review and integrate the literature on due
process characteristics of performance appraisal as related to employee reactions.

4.] An Overview of The Social Context of Performance Appraisal

Relatively recent models of performance appraisal have proposed that the social
context of performance appraisal is an important area of research inquiry (i.e. Erdogan,
2002; Levy & Williams, 2004; Murphy & Cleveland, 1995). Indeed, some scholars have
argued that the social context of performance appraisal is the key context to study as
related to employee reactions (e. g. Russel & Goode, 1988). The model developed by
Erdogan (2002), for instance, included leader-member exchange as a “contextual factor”,
and impression management as a “rater behavior” (pg. 559). Similarly, the model
developed by Levy & Williams (2004) includes leader-member relationship quality,
supervisor trust and impression management as “proximal process variables”; this
category of variables also includes system-related variables, such as previous

performance ratings, however.

23

While these models have highlighted the importance of social context, they have
done so using different taxonomies. These constructs can more efficiently and
meaningfully be categorized as the social context of performance appraisal. Being
conceptually related to rater-ratee relationships (Pichler, Varma, & Petty, 2008), these
constructs should be related in similar ways to employee reactions, and should thus be
treated as a group of similar antecedents.

My review of the employee reaction literature identified four constructs related to
the social context of performance appraisal as correlates of employee reactions: i) rater-
ratee4 relationship quality, ii) supervisor trust, iii) social support and iv) supervisor
satisfactions. Studies involving these constructs often employ theories of social exchange,
such as organizational justice, to develop hypotheses about relationships between these
constructs and employee reactions. Indeed, organizational justice theory serves as a
useful framework for linking social contextual variables to employee reactions, as will be
articulated later in this study. For now, it is important to understand what each of these
constructs represents and how they are being operationalized in the literature,.

4.2 Relationship Quality

Relationship quality can be described as the extent to which a rater and a ratee
have a working relationship characterized by a high-quality social exchange, which can
include characteristics such as mutual trust and social support (Dansereau, Graen, &

Haga, 1975; Pichler et al., 1998). Seven studies measured the relationship between

 

4 The term “rater” is used to refer to the person conducting the appraisal, and is used interchangeably with
terms such as “supervisor” and “manager” in the literature and in this study. The term “ratee” is used to
refer to the person being evaluated in the appraisal, and is used interchangeably with terms like “employee"
and subordinate in the literature, and in this study.

5 While this review is not meant to be exhaustive, i.e. to describe all of the constructs that have been
measured as correlates of employee reactions, this review is representative of the literature in the sense that
any relevant construct that was measured across multiple studies is included.

24

supervisor-subordinate relationship quality and employee reactions to performance
appraisal, and researchers have most often used measures of leader-member exchange
quality to operationalize relationship quality.

For instance, Dulebohn and Ferris (1999), Dobbins and colleagues (1990), Elicker
(2000), Klein and Snell (1994), Orpen (1995), and Roberson and colleagues (1993)
measured relationship quality in terms of exchange quality, but they used different
published scales. Dulebohn & Ferris (1999) used a measure validated by Wayne and
Ferris (1990), Elicker used a measure validated by Scandura and Schriescheim (1994),
and Klein and Snell, Orpen (1995), and Robeson and colleagues (1993) used a measure
validated by Dansereau, Graen and Haga (1975). Finally, Nathan and colleagues (1991.)
used a measure of the quality of rater-ratee interpersonal relations using semantic
differential scales.

The reactions measured as criteria also varied across studies. Reactions that were
measured in multiple studies include procedural justice (Dulebohn & Ferris, 1999;
Elicker, 2000), session satisfaction (Nathan et al., 1991; Roberson et al., 1993),
motivation to improve (Elicker, 2000; Orpen, 1995), and a composite of reactions
(Dobbins et al., 1990; Klein & Snell, 1994; Orpen, 1995). Researchers have also
measured the relationship between relationship quality and accuracy, distributive justice,
interpersonal justice, utility (Elicker, 2000), and appraisal quality (Nathan et al., 1991).

While the way in which relationship quality was operationalized across studies
varies somewhat, it is clear that each measure represents the quality of the rater-ratee
relationship (Pichler, Varma, & Petty, 2008). Given that all but one of the studies

mentioned above measured relationship quality in terms of exchange quality, the leader-

25

member exchange literature provides a particularly useful framework for understanding
this construct. Moreover, leader-member exchange theory also illuminates how each of
the constructs identified as aspects of the social context of appraisal are interrelated. Each
of the constructs below, i.e. trust, social support and satisfaction, represent aspects of
relationship quality, or a high-quality leader-member exchange.

Unlike more traditional leadership theories, leader-member exchange theory
contends that characteristics of the dyadic relationship, as compared to characteristics of
the leader, are predictive of outcomes at multiple levels of analysis, i.e. the employee,
dyadic, group and organizational levels of analysis (Gerstner & Day, 1997). According to
leader-member exchange theory, leaders (supervisors) do not treat all members
(subordinates) equally, but form different exchange relationships, i.e. of high and low
quality, with members (Dansereau, Graen, & Haga, 1975). High quality relationships are
characterized by increased levels of mutual trust, social support and member satisfaction,
whereas low-quality relationships are characterized by lower levels of these
characteristics (Dansereau, Graen, & Haga, 1975; Liden, Sparrowe, & Wayne, 1997).
Thus, each of the constructs below can be thought of as aspects of relationship quality.

Research also indicates that members in high quality relationships receive
preferential treatment, such as increased participation in the appraisal context (Wexley &
Klimoski 1984). Indeed, Cleveland and Murphy (1992) posited that exchange quality
should affect how supervisors treat employees in the appraisal process because theory
and research suggest that members in high-quality dyadic relationships receive greater
amounts of information and are allowed greater upward influence in decisions

(Dansereau et al., 1975). Perhaps because of this, leader-member exchange research has

26

shown that in-group members perceive higher levels of trust in leaders than out-group
members (Dienesch, & Liden, 1986), and that in-group members even receive higher
performance ratings (Wayne, & Liden, 1995) than do out-group members.

4.3 Supervisor Trust

The preceding section documented that high quality supervisor-subordinate
relationships are characterized by increased trust (Dansereau et al., 1975; Dienesch, &
Liden, 1986). Trust involves an expectation that the parties involved in a relationship will
behave ethically, i.e. treat each other with fairness (Hosmer, 1994). Supervisor trust, i.e.
the perception among ratees that one’s supervisor is trustworthy, was measured in five
studies in the employee reactions literature. The measures used to capture this construct
differ across studies, but still reflect the same underlying construct.

For instance, Hubbell and Chory-Assad (2005) used a measure of managerial
trustworthy behaviors based on social exchange theory, which included items related, for
instance, to truth-telling and behavioral consistency. Folger and Konovsky used a
measure of supervisor trust developed by Roberts and O’Reilly (1974). Kinicki, Prussia,
Wu, and McKee-Ryan (2004) used a single item, which measured the extent to which the
ratee believes he or she can trust what the rater says. Korsgaard and Roberson (1995)
used a measure of supervisor trust developed by Cook and Wall (1980), which included
items about supervisor honesty and trustworthiness. Lee and Akhtar (1996) used what
appears, based on the report, to be a single-item measure of supervisory trust, which
reflected the extent to which the ratee trusts the rater in the appraisal context.

Researchers have measured relationships between supervisor trust and several

employee reaction criteria, i.e. accuracy (Kinicki, Prussia, Wu, & McKee-Ryan, 2004),

27

distributive justice (Folger & Konovsky, Korsgaard & Roberson, 1995), session
satisfaction (Korsgaard & Roberson, 1995), interactional justice, procedural justice
(Hubbell & Chory-Assad, 2005), and motivation to improve (Kinicki, Prussia, Wu, &
McKee-Ryan, 2004; Lee & Akhtar, 1996).
4. 4 Supervisor Support

As explained above, supervisor support is also an important aspect of relationship
quality. Social support is the perception that one’s partner in a social exchange provides
supportive behavior in the workplace, and the social support literature (House, 1981;
Caplan et al., 1975, 1980) emphasizes three types of support: emotional, informational
and instrumental (House, 1980).The workplace-related social support literature typically
focuses on support from supervisors (LaRocco, House and French, 1980), which, in the
performance appraisal literature, measures the extent to which one’s supervisor exhibits
supportive appraisal-related behavior. Four studies measured relationships between
supervisor support and employee reactions.

For instance, Burke, Weitzel, and Weir, (1978) and Burke and Wilcox (1969)
measured the extent to which one’s supervisor was “constructive and helpful” in the
appraisal using a one-item scale. Nemeroff and Wexley (1979) developed and used a
multi-item scale, “supportive appraisal behavior”, which similarly measured constructive
supervisory behaviors, such as praising ratees for good performance, and other helpful
behaviors, such as scheduling follow-ups (pg. 27). Giles and colleagues (1997) developed
a measure of supervisor support, which included four items related to helpful supervisor

behaviors.

28

While researchers have used somewhat different operationalizations of supervisor
support, there has been some consistency in the reaction criteria measured as correlates of
support. Burke, Weitzel, and Weir, (1978), Burke & Wilcox (1969) and Nemeroff and
Wexley (1979) measured session satisfaction as well as motivation to improve, and Giles
and colleagues and Burke et al. (1978) measured overall fairness. Researchers have also
measured reaction criteria such as interpersonal (Burke et al., 1978) and informational
justice (Giles et al., 1997), as well as utility (Burke et al., 1978).

4. 5 Supervisor Satisfaction

Closely related to rater-ratee relationship quality (as perceived by the ratee) is
supervisor satisfaction, i.e. the extent to which a ratee is satisfied with the supervision he
or she receives. Indeed, research has found that members in high-quality relationships are
more satisfied with their supervisors (e. g. Liden et al., 1997). Four studies in the
employee reaction literature included this construct. Since the aforementioned constructs,
i.e. relationship quality, supervisor trust, supervisor support should all predict supervisor
satisfaction, one might think of this construct as an omnibus representation of the social
context of appraisal, at least in terms of its potential to represent, in an overall sense, the
extent to which ratees feel satisfied with their supervisors.

Supervisor satisfaction has been measured in the employee reaction literature
using the scale developed by Hackman and Oldham (1975) (i.e. Elicker, 2000), scales
developed by authors for the purposes of their study (Giles & Mossholder, 1990), and
single items (Gaby, 2004; Nathan et al., 1991). Despite this variability in measurement,
each of these measures reflects the extent to which a ratee is satisfied overall with his or

her supervisor.

29

Researchers have found relationships between supervisor satisfaction and session
satisfaction (Gaby, 2004; Giles & Mossholder, 1990; Nathan et al., 1991), motivation to
improve (Elicker, 2000; Gaby, 2004), utility (Elicker, 2000; Nathan et al., 1991)
interpersonal justice (Elicker, 2000; Gaby, 2004) and distributive justice (Elicker, 2000;
Gaby, 2004), as well as accuracy, procedural justice, utility (Elicker, 2000), feedback
satisfaction, informational justice (Gaby, 2004), system satisfaction (Giles & Mossholder,
1990), and quality (Nathan et al., 1991).

4. 6 Relationships Between Social Contextual Variables and Employee Reactions

While relationship quality, supervisor trust, supervisor support and supervisor
satisfaction represent distinct constructs in the employee reaction literature, and the
organizational behavior literature more generally, the preceding section noted that these
constructs are overlapping conceptually in that they all represent the social context of
performance appraisal. Moreover, since leader-member exchange theory has proposed
and documented that supervisor trust and support are aspects of exchange quality
(Dansereau et al., 1975), one would expect that these constructs would be related in
similar ways to reaction criteria. In fact, Johnson (2003) measured leader-member
exchange and supervisor trust and found, through factor analysis, that the scale items best
represented a single, unidimensional factor, not two separate factors. And while the
studies outlined above have measured a variety of reaction criteria, which would appear
problematic in that researchers are apparently not measuring similar substantive
relationships across studies, there are some notable consistencies in terms of measured
relationships between social contextual variables and employee reactions, which are

consistent with theory.

30

For instance, across studies which measured both social contextual and employee
reaction variables, the following constructs were included as reactions in at least three
studiesbz distributive justice (Elicker, 2000; Folger & Konovsky, 1989; Gaby, 2004;
Korsgaard & Roberson, 1995), interactional justice7 (Burke et al., 1978; Elicker, 2000;
Gaby, 2004; Giles et al., 1997; Hubbell, & Chory-Assad, 2005) motivation to improve
(Burke, Weitzel, & Weir, 1978; Burke & Wilcox, 1969; Nemeroff & Wexley; 1979;
Elicker, 2000; Gaby, 2004; Kinicki, Prussia, Wu, & McKee-Ryan, 2004; Lee & Akhtar,
1996; Orpen, 1995), procedural justice (Dulebohn & Ferris, 1999; Elicker, 2000; Hubbell
& Chory-Assad, 2005), session satisfaction (Burke, Weitzel, & Weir, 1978; Burke &
Wilcox, 1969; Gaby, 2004; Giles & Mossholder, 1990; Korsgaard & Roberson, 1995;
Nathan et al., 1991; Nemeroff & Wexley, 1979; Roberson et al., 1993), and utility (Burke
et al., 1978; Elicker, 2000; Nathan et al., 1991).

While these reaction criteria were not invariably measured specifically as
outcome variables (some were used as covariates or control variables), the preceding
paragraph illustrates the consistency by which certain reaction criteria were considered as
correlates of social contextual variables. What is perhaps more notable here is that all
three primary types of organizational justice (distributive, procedural and interational)
were measured as correlates of social context. This may be surprising since one would
expect that social context should be uniquely related to employee perceptions of

interactional justice. In fact, scholars have assumed that supervisor treatment is related

 

6 The decision to use at least three studies as a cutoff is not only for illustrative purposes, but for analytical
purposes related to meta-analysis, as will be described in more detail in the Methods section.

Too few studies measured interpersonal and informational justice as related to social contextual variables
as to treat them separately.

31

distinctly to interactional justice perceptions as compared to procedural justice (e. g.
Moorrnan, 1991), which is related distinctly to the fairness of organizational procedures.

Including all three types of justice as correlates of social contextual variables
makes sense, however, when considering the social exchange literature. That is, the
leader-member exchange literature would suggest that higher quality relationships,
characterized by exchange quality, supervisor trust, support and satisfaction, should be
related to better treatment in the appraisal context (Cleveland and Murphy, 1992; Wexley
& Klimoski 1984), not only in terms of interpersonal treatment (interactional justice), but
also in terms of decision influence and voice. Research has also indicated that exchange
quality is positively related to performance ratings (Wayne & Liden, 1995).

Extending this argument, the decision control (Thibuat & Walker, 1975) and self-
interest (Lind & Tyler, 1988) models of procedural justice would also suggest that with
increased decision control in the appraisal process, perceptions of procedural justice will
increase. The group value model of procedural justice would suggest that with increased
voice in the appraisal, perceptions of procedural justice will increase. Of course, if one
perceives the one has influenced the decision of the appraisal, i.e. one’s performance
rating, perceptions of distributive justice should also increase (Greenberg, 1987;
Greenberg, 1990). If social context is indeed related to rating favorability, this is also an
explanation for why social context should be related to perceptions of distributive justice
(Greenberg, 1987; Greenberg, 1990).

Hypothesis 1: Social contextual variables will be positively related to a)

interactional justice perceptions, b) procedural justice perceptions, and c)

distributive justice perceptions.

32

While the sections above argue that social contextual variables should
theoretically be related to interactional, procedural and distributive justice perceptions, it
is clear that organizational justice theory would argue that social contextual variables
should be most strongly related to interactional justice. That is, interational justice theory
proposes that fair interpersonal treatment should increase perceptions of interactional
justice. Meta-analytic evidence too is consistent with the notion that social contextual
variables should be most strongly related to perceptions of interactional justice since they
should be related to more positive interpersonal treatment (Cohen-Carash& Spector,
2001; Colquitt et al., 2001). This is important to test in order to be consistent with and
further test the robustness of organizational justice theory.

Hypothesis 2: Social contextual variables will be most strongly related to

interactional justice perceptions.

What is also notable here is that motivation to improve, session satisfaction and
utility were measured most often as correlates of social contextual variables. System
satisfaction was not measured as a reaction criterion, which is intuitive since system
satisfaction should be uniquely related to characteristics of the performance appraisal
system (Keeping & Levy, 2000), as opposed to social contextual constructs.
Organizational justice theory is also informative as to why these criteria should be related
to social contextual variables.

Since the fairness of the treatment one receives in one’s work group signifies the
extent to which one is a valued member of that group (Lind & Tyler, 1988; Tyler, 1989),
and being a valued group member should predict to one’s fixture inputs into a long-term

social exchange, one would expect that social contextual variables would be related to

33

motivation to improve, through their effects on interactional and procedural justice. Of
course, being involved in a high-quality relationship should be directly related to
motivation to improve based on the norm of reciprocity (Gouldner, 1960), but this should
be enhanced due to perceptions of fairness, since fair treatment is what actually signifies
one is a valued group member (Lind & Tyler, 1988).

Interactional and procedural justice should similarly explain the relationship
between social context and session satisfaction and utility. That is, if ratees desire to be a
valued group member, which is supported by perceptions of interactional and procedural
justice, then one’s satisfaction with the appraisal session should also increase, as should
perceptions of the overall utility of the appraisal. Being involved in a high-quality
relationship should be directly related to session satisfaction and utility, since so-called
in-group members likely receive preferential treatment in the appraisal (Wexley &
Klimoski 1984; Cleveland & Murphy, 1992). This preferential treatment should increase
the perception that one is a valued group member by increasing perceptions of
interactional and procedural justice, which should increase one’s satisfaction with the
appraisal, as well as one’s perception that the appraisal was valuable.

Since distributive justice is theoretically a key antecedent of employee
motivation, satisfaction with organizational systems such as performance appraisal
(session satisfaction), and perceptions related to the value or utility of these systems (e.g.
Greenberg, 1987), one would also expect distributive justice perceptions to mediate the
relationship between social context and these employee reactions, since social contextual

variables are positively related to distributive justice perceptions.

34

Hypothesis 3: Perceptions of organizational justice (distributive, procedural and
interactional) will partially mediate relationships between social contextual
variables and employee reactions.

Previous models of the performance appraisal process have positioned social
exchange constructs such as leader-member exchange as outcomes of the appraisal
(Masterson et al., 2000). The present model instead proposes that high quality supervisor-
subordinate relationships should affect ratee perceptions of justice in the appraisal
process, and ultimately their reactions to the appraisal, which is congruent with
organizational justice theory (Greenberg, 1993) and social exchange theory (Liden,
Sparrowe, & Wayne, 1997). Since the social context literature has been lacking an
explication of the processes whereby social context is related to employee reactions, the
hypotheses developed above explicate these processes, and are consistent with existing
models of the appraisal process.

For instance, Inderrieden, Keaveny, and Allen (1988) posited that performance
appraisal should be treated as a process, and not a single event. Likewise, they suggested
that while the actual performance evaluation is itself a process, it is important to
understand how rater-ratee interactions that precede the evaluation impact reactions to the
evaluation. Likewise, Klein, Snell and Wexley (1987) developed a model of the appraisal
process, which proposed that the appraisal is comprised of three sets of variables: inputs
(the appraisal context, e. g. relationship quality), throughputs (the appraisal session, e. g.
participation), and outputs (reactions). Their process model posits that the organizational
context within which the appraisal takes place affects the interaction that occurs during

the actual appraisal session, which then determines how ratees react to the session. The

35

hypotheses developed above build upon this process model by developing specific
relationships between inputs, throughputs and outputs, and by articulating the role of
organizational justice perceptions in the process.

Summary

Scholars have duly noted the importance of social context in the performance
appraisal process. Models of the appraisal process have grouped social contextual
variables in categories such as “process proximal variables” (Levy & Williams, 2004)
along with unrelated constructs, such as group/task characteristics and organizational
policies (pg. 884). Other models consider social contextual variables, but include these as
“contextual factors” along with other broader contextual variables such as organizational
culture (Erdogan, 2002, pg. 559). Grouping relatively unrelated variables together into
similar conceptual groups can lead to confusion in terms of how the performance
appraisal context should actually be characterized. Moreover, no existing model or
review has systematically identified the social contextual constructs that are commonly
studied in the employee reaction literature.

The preceding section, therefore, reviewed and described the social contextual
constructs which are actually being included in studies of employee reactions, described
how these constructs are being measured, outlined how these constructs are interrelated,
described the types of reaction criteria these constructs should be related to, and
identified the explanatory mechanisms whereby social contextual constructs are related to
employee reactions. This chapter proposed that it is best to categorize these constructs as

part of the social context of performance appraisal, or social contextual constructs.

36

CHAPTER 5
A REVIEW AND INTEGRATION OF CONTEXTUAL ANTECEDENTS OF
EMPLOYEE REACTIONS PART II: THE DUE PROCESS MODEL OF
PERFORMANCE APPRAISAL

The preceding chapter reviewed and integrated the literature on the social context
of performance appraisal as related to employee reactions. The purpose of this chapter is
to similarly review and integrate the literature on due process performance appraisal with
literature on employee reactions. In support of using a due process approach to
performance appraisal, Taylor and colleagues (1995) described performance appraisal as
“one of the great paradoxes of effective human resource management” in that appraisals
have the potential to provide valuable feedback to employees and enhance the
effectiveness of other human resource management systems, but this potential is often
unrealized because of negative employee reactions, due in part to a lack of due process.
Folger and colleagues (1992) developed a due process model of performance appraisal
based on due process of the law, with the purpose of increasing fairness in appraisal
systems, and increasing appraisal effectiveness through positive reactions to these
systems. See Appendix A for a description of due process performance appraisal
dimensions and sub-dimensions.
5.1 The Psychometric and Due Process Models Performance Appraisal

Folger and colleagues (1992) developed the due process model of performance
appraisal in response to what they call the “test metaphor” of performance appraisal,
which parallels the psychometric model of performance appraisal as described above in

Chapter 2. Both the test metaphor and psychometric model of performance appraisal

37

assume that rating accuracy is the key criterion of appraisal effectiveness. As F olger et al.
(1992) point out, the test metaphor of performance appraisal relies on several tentatively
held assumptions: that work performance can be measured validly and reliably; that raters
are able to rate performance accurately; and that an ultimate criterion of performance
exists, which perforrnance can be evaluated against. These assumptions, and the test
metaphor which they comprise, have been criticized given research which indicates that
raters are susceptible to a variety of rating errors, and that raters are biased by political
processes and motivational constraints (Folger et al., 1992). Thus, scholars have begun to
redefine appraisal effectiveness according to employee reactions, as opposed to test
accuracy (see also Chapter 2).

The due process model of performance appraisal (F olger et al., 1992), on the other
hand, is based on principles of the due process of law, which is proscribed by the Fifth
and Fourteenth amendment to the US. Constitution. Collectively, these amendments
enhance fairness in legal proceedings by requiring adequate notice, i.e. the publication
and communication of laws, a fair hearing (meaning all relevant information is presented
and the accused has the right to present evidence and have voice in legal proceedings)
and judgment based on evidence (meaning decisions are free from bias) (F olger et al.,
1992)

While researchers have not used the due process performance appraisal model
explicitly to study appraisal employee reactions such as fairness, much of the extant
research can be organized using this rubric. For instance, the seminal study by Greenberg
(1986) identified factors that represented fairness in appraisals, such as employee input

before the appraisal (adequate notice), rater familiarity with employee work performance

38

(fair hearing), and consistent application of standards (judgment based on evidence). In
fact, while less than a handful of studies on employee reactions have explicitly used the
due process performance appraisal model to study relationships between the appraisal
context and employee reactions (e.g. Taylor, Masterson, Renard, & Tracy, 1998), in part
due to the timing of the publication of the due process appraisal model (Folger et al.,
1992), many of the studies on employee reactions can be organized using this framework,
as will be seen below. Please see Table 2 for a visual display of due process performance
appraisal dimensions, sub-dimensions and associated chapter sections.

5.2 Adequate Notice

As applied to performance appraisal, the adequate notice dimension of due
process holds that employees should be accountable for performance standards that they
are aware of and understand. Allowing employee input into the standard-setting process
and disseminating performance standards before the appraisal session can increase
employee awareness and understanding. The adequate notice dimension also holds that
employees should receive frequent feedback regarding their performance throughout the
appraisal process.

Scholars have noted that managers have difficulty in controlling their natural
tendencies toward involving subordinates in the appraisal process (French, Kay & Meyer,
1966), and that they have a great deal of discretion when it comes to conducting
performance appraisals (Murphy & Cleveland, 1991). This is why organizations
implement due process appraisal systems—to address variability between raters in

appraisal-related behavior (Folger et al., 1992). Researchers have also noted that some of

39

these features are more system-related, whereas others are relatively supervisor-initiated
(Erdogan et al., 2001; Murphy & Cleveland, 1991).

Adequate notice in the form of knowledge of performance standards is
transparently more system-oriented than is its counterpart, feedback frequency.
Knowledge of performance standards is related to the publication and dissemination of
performance objectives, which is ofien an organizational — not supervisory - function, and
is oftentimes gained from employee initiative (e.g. Levy & Williams, 2000). Feedback
frequency, on the other hand, reflects how often raters provide ratees with feedback,
which is generally discretionary (Murphy & Cleveland, 2001). Considering these
differences, hypotheses related to the two types of adequate notice will be developed
separately.

Knowledge of performance standards. Ten studies (twelve unique samples) in the
employee reaction literature measured the construct of adequate notice in the form of
knowledge of performance standards, although using different operationalizations.
Buehler (2006) deliberately measured multiple aspects of adequate notice as aspects of
due process performance appraisal, including discussion of expectations, employee
training on evaluation, employee input on evaluation, and employee input on goals,
which formed a composite variable, adequate notice. Erdogan, Kraimer and Liden (2001)
measured knowledge of appraisal criteria using a single item, “Before the performance
appraisal, I was familiar with the criteria by which my performance was appraised” (pg.
213). Evans and McShane (1998) measured goal establishment, or the “establishment of
specific and relevant job goals” (pg. 177), which is different from goal setting in the

appraisal interview, which the authors also measured. Inderriden, Allen and Keavey

40

(2004) measured employee knowledge of performance standards using a six-item scale
developed by the authors, which captured “the extent to which the respondent was
informed of performance standards” (pg. 470). Inderrieden, Keaveny and Allen (1988)
measured participation in setting standards using a single item. Levy and Williams (1998)
and Williams and Levy (2000) measured perceived system knowledge based on the due
process model, i.e. “the extent to which individuals perceive that they understand the
objectives of the appraisal system, how the appraisal process works, and the overall goal
of the appraisal process” (pg. 54). Tang and Sarsfield-Baldwin (1996) measured
performance appraisal clarity using three items, which measured, for instance, “how
much information was given... about the performance appraisal criteria used for your
evaluation” (pg. 23). Thurston (2001) measured setting criteria using several items, such
as “my organization requires supervisors to set, publish and distribute standards and
criteria before evaluating their employees” (pg. 49).

While the specific measures used vary, these variables all reflect the construct
adequate notice, and specifically refer to employee knowledge of performance standards
prior to a formal appraisal. Extant findings are consistent across measures. For instance,
researchers have found positive relationships between adequate notice and justice
perceptions, including interactional justice (Buehler, 2000; Thurston, 2001); procedural
justice (Buehler, 2000; Levy & Williams, 1998, Study 2; Williams & Levy, 2000);
specific measures of procedural justice, i.e. rater procedural justice (Buehler, 2000),
system procedural justice (Erdogan et al., 2001); and distributive justice (Buehler, 2006;
Inderriden et al., 1988; Inderriden et al., 2004; Tang, & Sarsfield-Baldwin, 1996;

Thurston, 2001). Researchers have also found positive relationships between adequate

41

notice and employee reactions, including overall fairness (Evans & McShane, 1988, both
samples; Tang, & Sarsfield-Baldwin, 1996); motivation to improve (Buehler, 2006); and
system satisfaction (Inderriden et al., 1988; Williams & Levy, 2000).

In fact, the knowledge of performance standards dimension of due process is
consistent with multiple theories of procedural justice. For instance, the criterion that
processes are based on prevailing moral and ethical standards (Leventhal, 1980) is
fulfilled to the extent that it is unethical for employees to be kept unaware of the type of
performance desired by one’s employer. Knowledge of performance standards is also
congruent with the self-interest model of procedural justice (Lind & Tyler, 1988), which
posits that employees desire fairness in decision-making processes for self-serving
interests. As applied to the current discussion, knowledge of performance standards
allows an employee to match his or her work performance to a level required for optimal
rewards. In this connection, knowledge of performance standards is also consistent with
distributive justice theory (Greenberg, 1987; 1990) in that this knowledge should be
related to an employee’s ability to achieve a positive performance rating and, hence,
desired rewards. Since knowledge of performance standards is relatively system-oriented,
theory related to interactional justice does not lend itself well to understanding why this
dimension might lead to increased perceptions of justice, which is perhaps why this type
of justice was not regularly measured as a correlate. Based on this discussion, I expect
that knowledge of performance standards will be positively related to a) procedural and
b) distributive justice perceptions.

Organizational justice theory provides rationale for why knowledge of

performance standards should be related to employee reactions. Procedural justice should

42

explain the relationship between knowledge of performance standards and motivation to
improve, since this reflects one’s inputs into a long-term social exchange relationship,
whereas distributive justice should explain the relationship between knowledge of
performance standards and appraisal (system) satisfaction, since this is an outcome-
oriented reaction. Given that knowledge of performance standards is related to procedural
and distributive justice, as well as fairness perceptions, both types of justice perceptions
should explain why knowledge of standards is related to perceptions of overall appraisal
fairness. The preceding discussion leads to the following hypothesis:

Hypothesis 4: Procedural and distributive justice will partially mediate the

relationship between knowledge of performance standards and employee

reactions.

Frequency of feedback. Knowledge of performance standards prior to the
performance appraisal session is one important aspect of adequate notice. The other
aspect is frequent feedback. Frequent feedback is important because raters can
communicate to ratees areas of performance that need improvement prior to the appraisal,
allowing the ratee adequate notice as to how to receive a positive rating.

Feedback frequency has received relatively more attention than knowledge of
performance standards in the broader performance appraisal literature. Indeed, empirical
studies have shown that more frequent appraisals are related to employee reactions for
some time now (e.g. Zander & Gyr, 1955). For instance, research has found that
perceived accuracy of performance appraisal feedback is related to the frequency,
specificity and sign of the feedback (e.g. Ilgen et al., 1979; Landy et al., 1978). In support

of Larson’s (1984) model of performance feedback, Kinicki, Prussia, Wu, and Mckee-

43

Ryan (2004) found that the feedback environment within which a performance appraisal
occurs is best represented by multiple indicators of feedback quality, i.e. frequency,
specificity and sign of feedback. The authors also found that cognitive variables, e. g.
perceived accuracy and motivation to improve, mediated the relationship between
appraisal context (feedback environment) and future job performance. Klein and Snell
(1994) also noted that feedback frequency was important to consider in relation to
employee participation in the appraisal to reactions.

The frequency of performance feedback clearly seems an important antecedent of
employee reactions. Twelve studies (including fourteen unique samples) in the employee
reaction literature included feedback frequency as a contextual variable; some of these
studies (6 in total) were also studies that measured the knowledge of performance
standards dimension of adequate notice.

Buehler (2006) measured the frequency and timelines of feedback one receives, to
form a composite feedback variable. Dobbins, Cardy, & Platz-Vieno (1990) measured
frequency of evaluation by asking the number of times one’s performance was evaluated
during the past year. Erdogan, Kraimer, & Liden (2001) measured performance feedback
using items such as “I received continuous feedback about my performance during the
appraisal period” (pg. 214). Evans & McShane (1988) (2 samples) measured appraisal
frequency and follow-up using three items, which were based on scales used by previous
authors in this group (e. g. Landy et al, 1978). Gaby (2004) measured appraisal frequency
using three items, e.g. “My supervisor and I engage in frequent performance discussions”
(pg. 52). Kinicki, Prussia, Wu, & Mckee-Ryan (2004) measured frequency of feedback

by asking respondents “How often does your supervisor tell you how you are doing

44

overall” using a Likert-type scale (pg. 1061). Klein & Snell (1994) measured
performance appraisal frequency in terms of the time since the employee’s last
performance review. Inderrieden, Allen, Keavey, (2004) used nine items to measure
feedback through the evaluation period, e.g. “During the year, if my supervisor thought I
could perform part of my job better, he/she would talk to me about it” (pg. 470).
Inderrieden, Keaveny, & Allen (1988) measured “the degree of performance feedback
(that) was received throughout the appraisal period “ (pg. 308) using a single item.
Landy, Barnes, & Murphy (1978) measured frequency by asking whether or not
performance was evaluated at least once per year. Secunda (1984) (2 samples) measured
frequency of evaluation using a single item. Thurston (2001) included a variable,
providing feedback, using several items, such as “my rater frequently lets me know how I
am doing” (pg. 53).

While the measures used to operationalize feedback frequency vary, the preceding
paragraph indicates that they all represent the same construct, i.e. are intended to measure
the extent to which performance feedback is regularly communicated. Researchers have
found positive relationships between feedback frequency and perceptions of justice,
including interactional justice8 (Buehler, 2000; Gaby, 2004; Thurston, 2001); procedural
justice (Buehler, 2000; Erdogan et al., 2001; both also measured rater procedural justice);
and distributive justice (Buehler, 2006; Inderrieden et al., 1988; Inderrieden et al., 2004;
Thurston, 2001).

For reasons similar to the knowledge of performance standards dimension of due

process performance appraisal, feedback frequency should be related to perceptions of

 

8 Too few studies separately measured interpersonal and informational justice to make meaningful
distinctions.

45

procedural justice. That is, frequent feedback discussions increase the extent to which the
appraisal can actually be based on valid and accurate information (Leventhal, 1980).
Moreover, frequent feedback should also increase the perception that the performance
appraisal process is fair, and that one is a valued group member (Lind & Tyler, 1988). If
adequate notice should increase perceptions of distributive justice due to an increased
awareness of how to link one’s performance to valuable rewards (Greenberg, 1987;
1990), so too should frequent feedback. Frequent feedback should also be related to
perceptions of interactional justice, since it is generally a discretionary appraisal-related
behavior (Cleveland & Murphy, 1991). Frequent feedback should also increase
perceptions that one is being treated fairly, and that information is being communicated
sincerely and adequately, all of which define interactional justice (Greenberg, 1993).

Hypothesis 5: Frequent feedback will be positively related to perceptions of a)

interactional justice, b) procedural justice and c) distributive justice.

Researchers have also found positive relationships between feedback frequency
and employee reactions, such as motivation to improve (Buehler, 2006; Gaby, 2004;
Kinicki et al., 2004) and system satisfaction (Inderrieden et al., 1988). Mixed results have
been found for perceptions of accuracy (Kinicki et al., 2004; Secunda, 1984), composite
measures of reactions (Dobbins et al., 1990; Klein & Snell, 1994; Landy et al., 1978), as
well as measures of overall fairness (Evans & McShane, 1988; Secunda, 1984).

The relationship between feedback frequency and employee reactions is
inconsistent, despite the long held notion that feedback frequency should predict
reactions such as fairness and accuracy (Kinicki et al., 2004; Zander & Gyr, 1955), and

supporting theory from the due process model of performance appraisal (Folger et al.,

46

1992). While theory would suggest that knowledge of performance standards and
feedback frequency should be similarly related to employee reactions, given that they are
assumed to represent the same over-arching construct, i.e. adequate notice, the
inconsistent relationships between feedback frequency and employee reactions suggest
that feedback frequency may be operating differently than knowledge of performance
standards.

One theoretical explanation for this inconsistency is that reactions to feedback are
determined not only by feedback frequency, but feedback sign (Ilgen et al., 1979; Landy
et al., 1978; Kinicki et al., 2004; Kluger & DeNisi, 1996). Thus, it seems particularly
important theoretically and empirically to simultaneously consider rating favorability
when measuring the relationship between feedback frequency and employee reactions,
both at the individual study level, and in meta-analysis. Thus, the current study is able to
potentially clarify why there are inconsistent results across studies in terms of the
feedback frequency-employee reaction relationship.

More specifically, the relationship between feedback frequency and employee
reactions may be non-significant, or negative perhaps, when rating favorability is low
because ratees may feel that they have received a rating that is not reflective of their true
performance or their responsiveness to regular feedback. Thus, employee reactions may
in fact be not be positive when feedback is high and when rating favorability is low—or,
in other words, less positive as compared to when feedback is low. This potential
interactive relationship is consistent with research that suggests communication

frequency may increase the expectation of favorable outcomes (Kinicki et al., 2004).

47

This would suggest that feedback frequency and rating favorability interact such
that the relationship between feedback frequency and reactions is positive when rating
favorability is high, and non-significant (or perhaps negative) when rating favorability is
low. This thesis has not been investigated in the employee reaction literature, which
assumes that feedback frequency and sign mutually determine employee reactions—but
in an additive, not multiplicative sense. Finding that feedback frequency and sign interact
to determine employee reactions would have important implications for performance
appraisal theory and practice.

Hypothesis 6: Feedback sign will moderate the relationship between feedback

frequency and employee reactions such that the relationship between feedback

frequency and reactions will be stronger (weaker) when feedback sign is positive

(negative).

5.3 Fair Hearing

The preceding sections reviewed and integrated the employee reaction literature
as related to the role of adequate notice of performance standards, an important aspect of
due process performance appraisal. Another important feature of due process appraisals is
a fair hearing, which involves a formal performance review and employee voice (i.e.
participation) in the review, as well as rater familiarity with ratee job performance. More
studies in the employee reaction domain have focused on employee participation in the
appraisal session than any other contextual variable.

Participation in the appraisal. As is described above, Cawley and colleagues
(1998) conducted a meta-analysis of the relationship between participation in the

appraisal evaluation and employee reactions in 27 field studies. Participation included

48

instrumental voice, value-expressive voice, self—appraisal, the proportion of time talked
by the employee, as well as an “other” category, which included goal setting and
variables that represented multiple types of participation. Since several studies besides
those included in the previous meta-analysis have measured goal setting in the appraisal
interview (totaling 13 studies), this will be treated separately from the “other” category,
which represents mixed types of participation.

The main finding of the Cawley et al. (1998) study was that the overall
relationship between participation and employee reactions, corrected for unreliability,
was strong (p = .61 ). This is to be expected since participation is a key antecedent of
reactions to decision-making processes and systems (e.g. Locke & Schweiger, 1979).
Overall, I expect a similar relationship in the current study between participation and
employee reactions.

Cawley and colleagues (1998) provided a relatively thorough review of the
literature on different types of employee participation in the appraisal session, including
explanations of the various aforementioned constructs. Thus, it would be redundant here
to fully describe how measures in the studies that will be included in the present database
represent each construct. It is important though to at least describe these constructs, and
also to note here how many studies are included in the present database that represent
each type of participation.

Goal setting. Goal setting in the appraisal session represents a particular type of
participation in the appraisal session, i.e. the opportunity for a ratee to set goals for his or
herself for the next appraisal period (N emeroff & Wexley, 1979). This type of

participation differs from voice, i.e. value-expressive and instrumental voice, in the sense

49

that the ratee is not necessarily voicing his or her opinion about his or her performance,
but it does allow the ratee an opportunity to participate in the process and to voice
opinions and concerns about future performance and how it will be evaluated.

Goal setting theory would suggest that setting goals should be related to employee
reactions such as motivation to improve (e. g. Locke & Latham, 1990) because, for
instance, performance standards are clearer to the ratee. Goal setting should also be
related to reactions such as appraisal satisfaction and fairness because of the opportunity
for increased participation in the process; indeed, Cawley et al. (1998) found a positive
relationship between the “other” category of participation, which included goal setting,
and employee reactions (p = .70.). Goal setting is important to study because the
relationship between goal setting and employee performance (i.e. behavioral change) has
also been documented for some time (e.g. Kay, Meyer & French, 1965). Cawley and
colleagues (1998) identified five studies that measured “other” types of participation,
including goal setting. The current review identified thirteen studies that specifically
included a measure of goal setting.

Self-appraisal. Self-appraisal is the opportunity for an employee to rate his or her
own performance, which is intended to increase discussion in the appraisal session, and
increase a ratee’s voice as related to describing his or her performance. This construct is
related to the fair hearing dimension of performance appraisal (Folger et al., 1992) in that
it increases the extent to which the ratee is able to present evidence on his or her behalf.
Indeed, scholars have argued that self-appraisal should increase participation in the

appraisal session (Burke, Weitzel, & Weir, 1978; Farh, Werbel, & Bedeian, 1988), which

50

should lead to more positive reactions to the appraisal process (Burke et al., 1978; F arh et
aL,1988)

Cawley and colleagues (1998) found that the relationship between self-appraisal
and overall reactions was small (p = .25), but this estimate was based on only three
studies, which ranged in magnitude from r = .06 to r = .31 (pgs. 632-633). Indeed,
scholars have previously noted the inconsistent relationship between self-appraisal and
employee reactions (e. g. Korsgaard et al., 1998). The current literature review identified
five additional studies that measured self-appraisal and one or more employee reactions,
which will allow for a more robust test of this relationship.

One reason for this variability may be due to the interaction that occurs during the
PA session. For instance, if self-appraisals are related to increased rater-ratee
disagreement about ratee performance or ratings, self-appraisals may not be related to
positive reactions, or may even be negatively related to reactions. For instance,
Korsgaard, Roberson, & Rymph (1998) found ratees who were trained to self-appraise
and to be assertive during the appraisal were more satisfied with the appraisal and were
more trusting of their managers as compared to ratees who only self-appraised. An
additional, and just as relevant, explanation for this inconsistency is that self-appraisal
may be positively related to employee reactions only when it is coupled with decision
influence. For instance, Basset and Meyer (1968) found that self-appraisals were
positively related to reactions (when self-appraisals and not supervisor appraisals were
the basis of the PA discussion), yet these same results were not found when formal self-
appraisals were used alongside supervisor ratings (Robserson et al., 1993). Indeerieden

and colleagues (2004) actually found that self-appraisal was negatively related to

51

perceptions of rating fairness, i.e. distributive justice, which could be due to the fact that
self-appraisals were not allowed to influence supervisor ratings in this study. DeGregorio,
M., & Fisher, CD. (1988) similarly found that when self-appraisals were used in the
performance rating decision process, this resulted in higher perceptions of accuracy and
satisfaction than a top-down appraisal, but simply self-appraising (without decision
influence) did not.

The existing evidence of the relationship between self-appraisal and employee
reactions is mixed, which may be due to the potential moderating role of decision
influence and rater-ratee agreement. This is an important area for future research. These
propositions can not be tested in the current study, however, since too few studies
simultaneously measured both self-appraisal and decision influence or rater-ratee
agreement. I expect to find a relationship between self-appraisal and employee reactions
that is similar in magnitude to that found by Cawley and colleauges (1998).

Time talked. The proportion of time talked reflects the amount of time talked by
the ratee in relation to the amount of time talked by the rater. Cawley and colleagues
(1998) noted that the proportion of time talked is an aspect of participation in the
appraisal setting, but scholars have established that it is distinct from value-expressive
voice (e. g. Greller, 1975). As applied to the current study, proportion of time talked is
related to fair hearing since it reflects that the ratee had an opportunity to explain his or
her side views of his or her performance. Cawley and colleagues (1998) identified five
studies that measured the proportion of time talked by the ratee. The current literature

review did not identify any additional studies that measured the proportion of time talked.

52

As such, this relationship will not be re-analyzed, but effect sizes will be included on the
overall participation-reaction relationship, as they were in the Cawley et al. (1998) study.

Participation-other. Cawley et al. (1998) noted that some studies confounded
different types of participation in the same measure. The current review identified a total
of five such studies, three of which were not included as measures of “other”
participation in the previous meta-analysis. As such, this relationship will be meta-
analyzed, with the expectation that the mean effect size will be similar to the one
previously reported.

Instrumental and value-expressive participation. Another key finding of the
Cawley et al. (1998) meta-analysis was that value-expressive voice was more strongly
related to reaction criteria than was instrumental voice. The majority of the research on
participation in the appraisal has focused on value-expressive and instrumental voice,
perhaps because of their obvious overlap with procedural justice theory. The former
measures the extent to which an employee has opportunity to express his or her feelings
in the appraisal, and the latter measures the extent to which an employee has opportunity
to influence the decision outcome (e.g. Korsgaard & Roberson, 1995). These results
suggest that simply being able express oneself during an appraisal is related to positive
employee reactions, and that decision control is not necessary to produce favorable
reactions, which is consistent with procedural justice theory (Lind & Tyler, 1988), as the
authors noted.

Revisiting the Cawley, Keeping, & Levy (1998) meta-analysis. Cawley et al.
(1998) also noted that the differential relationships between value-expressive voice,

instrumental voice and employee reactions were tentative given the relatively small

53

number of studies in their review, and that this relationship should be re-tested meta-
analytically in the future. Given the important theoretical and practical implications of
their review, the robustness of this relationship will be meta-analyzed again by including
15 additional effect sizes for value-expressive participation, and 9 additional effect sizes
for instrumental participation.

Sub (1992) tested whether or not voice simply for voice’s sake was related to
increased perceptions of procedural justice, i.e. as compared to a no voice condition, in a
performance appraisal context. While the instrumental voice condition resulted in higher
procedural justice perceptions than considered voice (i.e. value-expressive voice that was
acknowledged), and while there was a marginal effect for considered voice as compared
to non-considered voice, non-considered voice (voice simply for the sake of voice) was
not related to procedural justice, i.e. as compared to being unable to express voice. Thus,
the author concluded that the theoretical assumption of the process control perspective of
voice was unsupported, whereas the decision control perspective was supported (Lindy &
Tyler, 1988).

A study by Bonness and Macan (2006) also provides evidence that value-
expressive voice may not be more strongly related to employee reactions than
instrumental voice. The authors found that when appraisals were used in the appraisal
session, as compared to when employees were given a chance to self-appraise but the
self-appraisal was not used, employees reported more positive attitudes to their appraisal.
The authors, therefore, suggest their results indicate instrumental voice was more
important to ratees than was value-expressive voice, i.e. the opportunity to self-appraise

without influence.

54

There is reason, therefore, to doubt the finding by Cawley et al. (1998) that voice
alone, i.e. without decision influence, is more strongly related to employee reactions than
is instrumental voice. Cawley and colleagues (1998) did not include appraisal-related
justice perceptions in their study. Voice in and of itself may be more strongly related to
more general reactions, such as satisfaction, than to evaluations of specific aspects of
appraisal fairness.

Given the tentative nature of their finding, however, the following paragraphs
explain how the relationship between participation and reactions will be meta-analyzed,
and the conceptual and analytical differences between the Cawley et al. (1998) study and
the present study. As will be described, there is also empirical reason to doubt their
finding. The present study accordingly seeks to determine more conclusively a) if there is
a significant difference between instrumental and value-expressive participation as
related to employee reactions, and b) if instrumental and value-expressive voice
contribute unique variance to employee reactions, as would be expected based on
procedural justice theory (Lind & Tyler, 1988).

While Cawley and colleagues (1998) compared corrected effect sizes between
instrumental and value-expressive participation, effect sizes for each type of participation
were generally from different studies. Even though meta-analytic effect sizes were
corrected for error and unreliability, this involves a comparison of effect sizes not only
according to type of participation, but across studies with different samples, and
methodologies. A more direct comparison - ruling out study-level or idiosyncratic
effects - would be to compare relationships between each type of participation and

employee reactions within studies, and then meta-analyze the differences in effect size (if

55

any) between value-expressive and instrumental participation across studies. In this way,
one can more effectively answer the question: Is value-expressive voice more strongly
related to employee reactions? Given the additional number of effect sizes documented in
the literature review of the employee reaction literature, this type of analysis will be
conducted as to determine if there is indeed a significant difference between value-
expressive and instrumental voice as related to employee reactions. Based on the existing
literature, I predict that:

Hypothesis 7: Value-expressive voice will be more strongly related to employee

reactions than instrumental voice.

An additional limitation of the Cawley et al. (1998) meta-analysis was that they
did not investigate the role of outcome favorability as related to employee reactions. One
explanation as to why value-expressive voice was more strongly related to employee
reactions than was instrumental voice is that outcome (i.e. rating) favorability was
already high in the studies analyzed, thus minimizing the need for decision influence.
This is concerning since scholars such as Ilgen, Fisher, and Taylor (1979) have
consistently noted that the perceived favorability of one’s performance rating is a key
determinant of reactions to a performance appraisal.

In support of this, Klein and Snell (1994) noted that the relationship between
participation and employee reactions is inconsistent across studies, and suggested that
contextual aspects of the appraisal should moderate the participation-reaction
relationship. The authors found that performance ratings moderated relationships between
participation and reactions such that the relationship between participation and employee

reactions is stronger (weaker) favorability was low (high). This is consistent with existing

56

research on organizational justice, which has documented an interactive relationship
between procedural fairness and outcome favorability, such that they act as substitutes
(Brockner & Wiesenfeld, 1996).

Extending this finding as applied to performance appraisal, it may be that value-
expressive participation is related to reactions, but only when one’s performance rating is
high; when one’s performance rating is low, instrumental voice may become more
important. For instance, while the review by Brockner and Wiesenfeld (1996) indicates
that outcome favorability is less strongly related to a variety of dependent variables when
procedural justice is high, and vice versa, this interactive effect is inconsistent in that
some studies have only found main effects for each type of justice (Lind, 2001).

While heretofore unacknowledged, this may be due, at least in part, to the way in
which procedural justice is operationalized, or the type of procedural justice that is
evident in the appraisal session. That is, decision control may be more important when
outcome favorability is low, since it involves the ability to actively influence the outcome
of a decision, whereas value-expressive voice may not mitigate the effect of an
unfavorable outcome, since this aspect of procedural justice is relatively unrelated to
decision influence (Lind & Tyler, 1988). Thus, it is important to not only determine if
outcome favorability and procedural justice interact, but also to establish when this is the
case.

Indeed, this proposition is supported by Brockner’s (2002) theoretical review,
which argues that the judgment of responsibility, i.e. an attribution that the decision
maker is responsible for an unfavorable outcome, determines the interactive relationship

between outcome favorability and procedural justice. The author posited, based on

referent cognitions theory, that blame for an unfavorable outcome will be attributed to the
decision maker to the extent that the decision maker is perceived to be responsible for the
decision.

Extending this argument, in a situation where decision control is low, blame
should be attributed to the decision maker, as compared to a situation where decision
control is high, where blame is less likely to be directed only at the decision maker. That
is, since value-expressive voice does not necessarily imply that one’s voice had an impact
on a decision, it seems less likely that attribution of blame would be placed on the
decision maker. Thus, the type of procedural justice, i.e. participation, involved in a
performance appraisal may serve as a boundary condition for the interactive relationship
between outcome favorability and procedural justice. If this is indeed the case, some of
the basic theoretical assumptions regarding procedural justice, and the interactive
relationship between procedural justice and outcome favorability, would need to be
revisited.

Hypothesis 8a: The relationship between value-expressive voice and employee

reactions will be stronger (weaker) when outcome favorability is high (low).

Hypothesis 8b: The relationship between instrumental voice and employee

reactions will be stronger (weaker) when outcome favorability is low (high).

Supervisor job-related knowledge. While more research in the employee reaction
literature has focused on participation in the appraisal than any other contextual
antecedent of employee reactions, an additional dimension of a fair hearing is supervisor
knowledge of employee job performance. This can encompass both observation of ratee

performance and knowledge of the ratee’s job—both of which should enable'the

58

supervisor or rater to provide a more valid assessment of performance. Twelve studies
(fourteen unique samples) measured supervisor job-related knowledge, or the extent to
which a supervisor is knowledgeable about the ratee’s job functions and performance.
Those studies that investigated how rater job-related knowledge is related to
employee reactions measured ratee’s perceptions that his or her rater is knowledgeable
about the ratees job duties and job performance. Operationalizations varied, but all reflect
the same construct. For instance, Evans and McShane measured appraiser’s knowledge
using five items, based on previous research, such as “my appraiser has an excellent
personal knowledge of my performance level in my current position” and “my appraiser
has a good understanding of the skills required to perform my job” (pg. 183). Landy et al.
(1978) used a single item, “supervisor’s knowledge of performance”, using a Likert-type
scale. Buehler (2006) included an item which measured rater knowledge of ratee job
performance. Folger and Konovsky (1989) asked participants if their supervisor “Became
thoroughly familiar with your performance” (pg. 120) using a Likert-type response
format. Giles et al. (1997) measured supervisor observation using six items such as “My
supervisor has adequately observed my performance dining the past year” (pg. 498).
Inderrieden et al. (2004) measured opportunity for supervisor to observe work using three
items, such as “my supervisor seldom sees my wor ” (pg. 470). Jawahar (2006) measured
what the author reported to be supervisor satisfaction, but actually measured the extent to
which one’s rater “knows how well I am doing my job”, using the scale developed by
Russell and Goode (1988), who also measured employee reaction as related to this
construct. Kinicki et a1. (2004) measured supervisor job knowledge and observation using

the source credibility scale developed by Vandaveer (1982). Lee and Akhtar (1996)

5‘)

measured supervisor knowledge of subordinate’s job using the item “My supervisor is
knowledgeable about the kind of work I am doing” (pg. 883). Thurston (2001) included a
scale called assigning raters, which included five items such as “My organization makes
sure that I am assigned a rater who understands that requirements and constraints of my
work” (pg. 50).

While a fair hearing is theoretically related to procedural justice (Folger et al.,
1992), research in the employee reaction literature has instead measured and found
positive relationships between supervisor job knowledge and distributive justice
(Buehler, 2006; Folger & Konovsky, 1989; Thurston, 2001) as well as perceptions of
overall appraisal fairness (Evans & McShane, 1988, 2 samples; Giles, Findley, & Field,
1997; Secunda, MD. 1984, 2 samples). Researchers have also found positive
relationships between supervisor job knowledge and other employee reactions, such as
motivation to improve (Kinicki, Prussia, Wu,& Mckee-Ryan, 2004; Lee, & Akhtar, 1996)
and composite measures of reactions (Landy, Barnes, & Murphy, 1978; Lee, Akhtar,
1996)

In fact, this makes sense given that the perception that one’s rater is familiar with
one’s job and work performance should be related to the perception that one’s rater is
able to assign a valid and fair performance rating, i.e. should increase perceptions of
outcome fairness (Greenberg, 1987; 1990). Theory would also suggest that perceptions of
outcome fairness should predict reactions to the performance appraisal system
(Greenberg, 1987). I, therefore, predict that:

Hypothesis 9: Distributive justice will partially mediate the relationship between

supervisor job-related knowledge and employee reactions.

60

The preceding section developed rationale as to why employee participation in
performance appraisal and supervisor job-related knowledge are related to the concept of
fair hearing, which is an important aspect of due process performance appraisal. Theory
related to organizational justice was used and critically examined in order to develop
novel and potentially revolutionary hypotheses about the relationships between employee
reactions and value-expressive and instrumental voice. Organizational justice theory was
also used to develop hypotheses about the mediating role of justice perceptions in the
relationships between employee reactions and participation and supervisor job-related
knowledge.

The following section will further organize the existing literature on employee
reactions by explaining how certain variables in the literature directly reflect the
judgment based on evidence principle of due process performance appraisal (Folger et al.,
1992). Further, organizational justice theory will be used to develop hypotheses about the
relationship between judgment based on evidence and employee reactions.

5. 4 Judgment Based on Evidence

The judgment based on evidence dimension of due process performance appraisal
(Folger et al., 1992) holds that performance standards should be consistently applied and
free from bias, and that ratees have an opportunity to appeal the performance appraisal
decision. The judgment based on evidence dimension of due process performance
appraisal can be fulfilled, at least in part, through supervisor neutrality or lack of bias, the
use of valid appraisal instruments, judging performance based on job-relevant factors,
and an appeals process. As will be explained in more detail below, research in the

employee reaction literature has investigated how each of these factors is related to

61

employee reactions. While all of these appraisal features represent an aspect of judgment
based on evidence, they also represent substantively different constructs, and may be
related to reactions in different ways. As such, hypotheses will be developed as related to
each type or aspect of judgment based on evidence.

Supervisor neutrality and lack of bias. Research in the employee reaction
literature has investigated how supervisor neutrality or lack of bias are related to
employee reactions. While these are slightly different concepts, they are all related to the
extent to the perception that one’s supervisor is unbiased and is able to provide a valid
performance rating. Eight studies measured supervisor neutrality or lack of bias.

Research on supervisor lack of bias has found positive relationships with
perceptions of distributive justice (Buehler, 2006;Tang, & Sarsfield-Baldwin, 1996;
Thurston, 2001) and interactional justice (Buehler, 2006; Gaby, 2004; Thurston, 2001).
This research has also found positive relationships between supervisor lack of bias and
employee reactions, including perceptions of fairness (Kavanagh, Benson, Brown, 2008;
Tang, & Sarsfield-Baldwin, 1996), motivation to improve (Gaby, 2004; Kinicki, Prussia,
Wu, & Mckee-Ryan, 2004), and composite reactions (Kleiman, Bidennan, & Faley,
1987; Tang, & Sarsfield-Baldwin, 1996).

If one perceives that one’s rater is credible, neutral or lacking bias, this should
increase the likelihood that one has been treated fairly interpersonally in the appraisal
because bias is less likely to be introduced into the session. As such, perceptions of
interactional justice should increase (Greenberg, 1983). This being the case, one’s

perception that one’s rating is fair should also increase, thus increasing perceptions of

62

outcome fairness (Greenberg, 1987). Perceptions of justice should be related to more
favorable reactions to the appraisal. I, therefore, predict that:
Hypothesis 10: Perceptions of distributive and interactional justice will partially
mediate the relationship between supervisor lack of bias and employee reactions.
Valid rating instruments. While the employee reaction literature has not
investigated the relationship between the use of valid rating instruments and employee
reactions per se, that is in the traditional psychometric sense, the research on rating
format has been integrated with the employee reaction literature. Scholars have proposed
that behavioral observation scales should be perceived as fairer by employees than other
formats, i.e. graphic rating scales, because they require raters to observe and document
employee work behavior (Tharenou, 1995). Indeed, research indicates that BOS scales
(as compared to graphic rating scales) are related to higher levels of performance
appraisal satisfaction and subsequent job performance (Tziner, Kopelman, & Joanis,
1997; Tziner, Kopelman, & Livneh, 1993). Thus, as opposed to rating formats where
employees are assigned a numeric rating based on some adjective (graphic rating) or a
pre-determined behavioral prototype (behaviorally-anchored rating scale, BARS),
behavioral observation scales emphasize actual observations of work behavior. Behavior
observation scales are, therefore, expected to be positively related to employee reactions
since they utilize more “valid”, i.e. behavior-based, information than other formats.
Only three studies were located that measured relationships between rating format and
employee reactions. Based on the positive results in these studies, I predict that:
Hypothesis 11: Rating format (i.e. BOS scales as compared to other graphic and

BARS scales) will be positively related to employee reactions.

63

Job-relevant factors. To the extent that a rater judges a ratee on what the ratee
perceives to be job-relevant factors, reactions to the appraisal should be more positive
(F olger et al., 1992). Nine studies in were located that measured relationships between
the ratee’s perception that his or her performance was judged based on job-relevant
factors and employee reactions. For instance, Burke et al. (1978) asked respondents
“How much of a balance would you say there was between your job performance and
your personality and mannerisms in the last evaluation of your performance” (pg. 908).
Dipboye and DePontbriand (1981) asked ratees the extent to which there was “Relevance
of factors on which you were evaluated” (pg. 250). Nathan et al. (1991) developed a
“criteria scale” which measured “the extent to which the actual evaluation was based on
results achieved, job-related behaviors, skills and abilities and predetermined goals” (pg.
358).

This research has found that the perception that performance was judged based on
job-relevant factors is related to perceptions of interactional justice (Buehler, 2006; Burke
et al., 1978; Gaby, 2004) and procedural justice (Buehler, 2006; Cobb et al., 1998;
Erdogan et al., 2001). These findings suggest that judging performance based on job-
relevant factors increases the perception that one has been treated fairly interpersonal] y,
and that the performance appraisal process was fair, which is consistent with theory
(Folger et al., 1992; Greenberg, 1987; 1993). That is, when performance is judged based
on relevant, objective factors as opposed to more subjective factors which are relatively
unrelated to job duties or performance, the perception that the appraisal process is fair
should increase (Greenberg, 1987). Since it is often up to the rater to determine what

types of factors he or she will use to rate his or her subordinates’ performance,

64

perceptions of interactional justice should also increase because ratees should feel that
their supervisor has treated them fairly.

Research has also found that judging performance based on job-relevant factors
is related to employee reactions, particularly appraisal satisfaction (Burke, Weitzel,
&Weir, 1978, Dipboye & de Pontbriand, 1981, Gaby, 2004; Igoumenopolous, 2001;
Mount, 1984 ; Nathan, Mohrman, & Milliman, 1991). Since judgment based on relevant
standards increases the perception that one has been treated fairly, and that the appraisal
process was fair, one’s reactions to the process should be more positive.

Hypothesis 12: Perceptions of interactional and procedural justice will partially

mediate the relationship between judgment based on job-relevant factors and

employee reactions.

Appeals process. When decisions related to the performance appraisal, e. g. one’s
performance rating, can be appealed, ratees should react more favorably to the appraisal.
Procedural justice theory is particularly relevant here, which would suggest that having
recourse about the appraisal decision should increase the perception that the process is
fair and just, which should be related to positive employee reactions (e. g. F olger etal.,
1992; Greenberg, 1987). Only five studies were found which measured relationships
between whether or not an appeals process is available to ratees, and employee
perceptions of fairness and appraisal reactions, but results have been positive (Buehler,
2006; Cobb, Vest, Hills, 1998; Giles, Findley, & Field, 1997; Thurston, 2001). Based on
theory and existing research, I predict that:

Hypothesis 13: The availability of an appeals process will be positively related to

employee reactions.

65

Sununary

This chapter organized the extant research on employee reactions using the due
process model of performance appraisal. It was demonstrated that variables that represent
each aspect of due process have been used in the employee reaction literature, and that
organizing these variables according to the due process model facilitated the explication
of relationships with fairness perceptions and, ultimately, employee reactions to the
appraisal process. Basic tenets of organizational justice theory were utilized to develop
hypotheses about relationships between due process contextual aspects of performance
appraisal and employee reactions. Organizational justice theory was also extended in the
sense that the interactive relationship between procedural justice, i.e. participation in the
appraisal, and outcome favorability, i.e. performance ratings, was proposed to be
qualified by type of participation, i.e. instrumental vs. value-expressive. The following
chapter will explain how meta-analysis will be used to test the hypotheses developed

above.

66

CHAPTER 6
METHODS
6.] Meta-Analytic Method

The effect size metric chosen was the correlation coefficient, given that effect
sizes reported in the employee reaction literature are almost invariably reported in this
metric. Correlations were first transformed into the Fisher’s z for analysis, and were then
back-transforrned in the correlation metric for presentation of results. Each effect size
was weighted by the inverse of its sampling error variance. No other effect size
corrections were employed since this can lead to inaccurate conclusions about the
population mean and variability (e. g. Deshon, 2002). Random effects models were
employed when the effect size was significantly heterogeneous; otherwise, a fixed-effects
model was employed (Hedges & Vevea, 1998).

Tests of significance for effect size centrality were conducted using Z-tests
(Lipsey & Wilson, 2001). Significance tests for moderation (with continuous
moderators) were conducted using procedures developed for meta-analytic regression in
SPSS by Lipsey ad Wilson (2001). Procedures for testing effect size centrality,
homogeneity and moderation were consistent with convention (i.e. Hedges & Olkin,
1985).

Models of meta—analytic correlation matrices (Viswesvaran & Ones, 1995) were
used to test hypotheses involving mediation (Hypotheses 3, 4, 9, 10 and 12). These
hypotheses posited that one or more organizational justice variables would mediate the
relationship between the antecedent of interest and employee reactions. Meta-analytic

correlations between justice variables and employee reactions from Table 3 were used as

67

input data (all correlations were significant), as were meta-analytic correlations between
relevant antecedents, justice variables and employee reactions from Table 49. Also, meta-
analytic correlations between each type of justice variable were computed for the
correlation matrices (See Table 6).

All latent factors were treated as single item indicators, and models were tested
using maximum likelihood estimation, with the harmonic mean of the samples sizes, as is
consistent with published research in applied psychology (e. g. Chen, Casper & Cortina,
2001; Ford et al., 2007). A variety of fit indicates are reported for each model in order to
evaluate model fit, as is consistent with recommendations from the structural equation
modeling literature (Hu & Bentler, 1999; Joreskog & Sorbom, 1993).

Consistent with Baron and Kenny (1986), firll mediation was established if and
when a fully mediated model fit the data well and, when adding a direct path from the
antecedent to employee reactions, the path was non-significant. Partial mediation was
established when the standardized path from the antecedent to employee reactions was
smaller than its respective meta-analytic correlation (i.e. from Tables 3 and 4). That is,
mediation is established when 1) the independent variable is correlated with the outcome
variable, 2) the independent variable is correlated with the mediator, 3) the mediator is
correlated with the outcome variable, and 4) when controlling for the relationship
between the mediator and the outcome variable, the relationship between the independent
variable and the outcome variable is non-significant. Partial mediation is established

when the path from the independent variable to the outcome variable “is reduced in

 

9 For correlations used as input data for path-analytic models, employee reactions did not include any
measure of organizational justice, since this would naturally confound tests of mediation (i.e. since justice
variables were posited as mediators).

68

absolute size, but is still different from zero when the mediator is controlled”
(http://davidakenny.net/cm/mediate.htrn) .
6.2 Literature Search and Inclusion Criteria

Literature review. Relevant databases, i.e. Psyclnfo, Business Source Elite, Jstor,
Proquest, Sage Journals Online, and Social Science Abstracts were searched using sets of
keywords such as “employee reactions and performance appraisal”, “organizational
justice and performance appraisal”. While employee reactions to performance appraisal is
a topic of growing but relatively recent interest, no specific starting date was defined, in
order to ensure that all potentially relevant studies were included. Reference lists of major
reviews related to contextual or qualitative aspects of performance appraisal (i.e. Cawley,
Keeping & Levy, 1998; Erdogan, 2002; Levy & Williams, 2004) were searched for
unidentified articles. In order to minimize a file drawer effect, theses and dissertations
were identified using the Dissertation Abstracts International database, and were included
in this study. The online database for the Society of Industrial-Organizational Psychology
conference presentations was also searched, and relevant papers were requested from
authors. The online database for the Academy of Management publications, e.g.
Academy of Management Journal, conference proceedings and presentations was also
searched to identify relevant published and unpublished manuscripts. Unpublished
manuscripts were also solicited from the Organizational Behavior and Human Resource
Management listservs of the Academy of Management.

Inclusion criteria. First, given the central interest in employee reactions to the
current model, only studies that measured a relationship between one or more employee

reaction criteria and one or more contextual variables were included. Second, since the

69

focus of the present study is on performance appraisal, each study had to reflect an actual
performance review session, as opposed to simply giving feedback related to a specific
task, for instance. While laboratory studies were included in the database, participants
must have participated in an actual performance review process (e. g. Kacmar, Wayne, &
Wright, 1996). Studies that involved a participant reading a fictitious performance
evaluation of a job incumbent (e.g. Holbrook, 1999), for example, were excluded. Both
laboratory and field studies were included in the database since there is no a priori reason
to expect differences across study type in substantive relationships between appraisal
context and reactions (Levy, Cawley, & Foti, 1998). Indeed, researchers have tested the
same relationships between appraisal context and subsequent reactions in laboratory and
field settings, and have found convergent results (of. Levy, Cawley, & Foti, 1998). That
said, the number of lab studies in the current database is small relative to the number of
field studies. Finally, the study must have reported an effect size, i.e. r or d, or
information that was sufficient to calculate an effect size to be included. Based on these
inclusion criteria, a total of 83 studies with 89 unique samples were included in the
database. Whenever a study reported multiple correlations between a contextual
antecedent(s) and an employee reaction(s), non-independent effect sizes were
transformed into a correlation of composite variablesl0 (Hunter & Schmidt, 1990).
Study Coding

Of the 89 independent samples included in this study, 48 were independently
coded by a second coder (a Ph.D. student in human resource management). Levels of

agreement between coders are reported below for each set of variables.

 

'0 This procedure was used whenever possible. When insufficient information was available, i.e. when
correlations between relevant variables were not reported, non-independent effect sizes were averaged
arithmetically.

70

Antecedent variables. The antecedent variables coded in this study were treated as
antecedents if they were measured before or currently with the measurement of the
outcome (reaction) variablesl '.

Social contextual variables were coded as to whether they measured relationship
quality, supervisor trust, supervisor support or supervisor satisfaction. Relationship
quality was measured using a leader-member exchange variable (e. g. Graen, Novak &
Sommerkarnp, 1982) in six studies; in another study it was measured using a semantic
differential scale; and in one additional study, it was measured using a scale developed by
the authors. Supervisor trust was measured using a validated measure of supervisor trust
(e. g. Roberts & O’Reilly, 1974) in all but two studies, where items were developed for
those particular studies. While there is an extensive literature on supervisor support,
which includes validated scales (e.g. Caplan et al., 1975), the studies that measured
supervisor support did so using self-developed items. Some studies that measured
supervisor satisfaction did so using validated scales (e. g. Smith, Kendall & Hulin, 1969),
whereas others developed items for the purposes of their study (e. g. Russell & Goode,
1988), which were then used by other researchers in the performance appraisal context.
Agreement between raters ranged from 83% to 100% for social contextual variables.

Due process variables were coded according to which aspect of due process was
being measured. In terms of adequate notice, several studies measured perceived system
knowledge, a scale developed by Williams and Levy (1992); several others used self-
developed items. Most of the studies that measured feedback frequency did so using

original items, which typically asked employees in a straightforward manner to rate the

 

" Some studies measured rating favorability, for example, after the appraisal session.

71

extent to which they felt they received frequent feedback. Agreement ranged from 70% to
82%.

For fair hearing, the employee participation variables were coded consistent with
Cawley and colleagues (1998). Whenever a variable included an instrumental component,
it was coded as instrumental voice. Variables that only measured voice for the sake of
voice were coded as value-expressive. No additional studies were located which
measured proportion of time talked, so this construct was not included in this study.
Cawley et al. (1998) coded studies that “did not fit neatly into these categories” as other,
which was also done in this study, except goal setting was included as distinct from this
category since several studies that were not included in the previous meta-analysis
specifically measured goal setting. Also consistent with Cawley et al. (1998), if a study
measure self-ratings, this was coded as self-appraisal. Supervisor job knowledge typically
was measured with original items developed by authors for the purposes of their study,
and measured the extent to which one’s supervisor was familiar with the employee’s
performance and/or had opportunity to observe their performance. Agreement ranged
from 75% to 100%.

For judgment based on evidence, studies that measured the extent to which a
decision was appealable were coded for an appeals process. Studies that measured the
extent to which one’s evaluation criteria were based on objective and job-relevant
standards were coded for job-relevant factors. Studies that measured the extent to which
performance standards were consistently applied or the extent to which one’s supervisor
was perceived as neutral or lacking bias were coded for supervisor lack of bias. Studies

that compared reactions to a rating instrument using a behavioral observation scale

72

(compared to a graphic rating scale) were coded for valid rating instrument. Agreement
ranged from 80% to 100%.

Given the interest in rating favorability in this study, studies that measured the
relationship between reactions and the favorability of the employee’s performance rating
(in terms of perceived favorability, self-reported performance rating or supervisor
reported performance rating) were coded for rating favorability. Agreement for this
variable was 93%.

Employee reactions. Employee reaction variables, i.e. accuracy, fairness, (session
and system) satisfaction, utility and composite variables were coded consistent with
Cawley and colleagues (1998). When studies measured performance appraisal
distrbutive, interactional (interpersonal or informational), or procedural justice, studies
were coded for the type of justice construct measured (consistent with Colquitt et al.,

2001 and Keeping & Levy, 2000). Agreement ranged fi'om 75% to 100%.

73

CHAPTER 7

RESULTS

7.1 Meta-Analytic Correlations
Meta-analytic correlations between each of the independent variables in this study

2 are reported in Table 1. The correlations under fair hearing that

and employee reactionsl
represent employee participation in the appraisal are noticeably similar in magnitude to
those reported by Cawley and colleagues (1998), although with larger numbers of
independent effect sizes. When comparing meta-analytic correlations (Mr) between the
Cawley study and this study (in parentheses), the relationships between overall employee
reactions and various forms of participation are remarkably similar: instrumental voice
Mr =.42, k =16 (Mr =.40, k =27), composite measures of participation Mr =.47, k =5
(Mr =.53, k =9), self-appraisal Mr =.22, k =3 (Mr =.23, k =8), and value-expressive
voice Mr =.47, k =18 (Mr =.52, k =35). It is important to note here that, in the current
study, research reports were also coded for goal setting in the appraisal (another form of
participation), which was moderately correlated with employee reactions (Mr =.49, k

=15). Another aspect of fair hearing, supervisor knowledge of job performance, was also

correlated with overall reactions (Mr =.54, k =12).

 

'2 For these results, employee reactions are the aforementioned reactions (Chapter 2), i.e. accuracy fairness,
satisfaction, utility, composite (other) measures, including measures of performance appraisal distributive,
interactional and procedural justice.

74

 

Table 1
Meta-analytic Results

for Employee Reactions to Performance Appraisal

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

N K SDr Mr Lower Upper Q
95%Cl 95%Cl
Social Context
Relationship Quality 1,356 8 .14 .60 .55 .62 27.65“
Supervisor 2,235 1 1 .28 .60 .47 .70 169.40*
Satisfaction
Supervisor Support 633 3 .1 1 .58 .47 .68 7.4 I *
Supervisor Trust 1,103 8 .24 .60 .48 .71 63.21 "'
Adequate Notice
Frequent Feedback 3,899 15 .25 .38 .26 .49 234.60"
Knowledge of 4,661 18 .18 .50 .42 .58 154.10*
Performance
Standards
Fair Hearing
Goal Setting 3,992 15 .21 .49 .40 .57 168.85*
Instrumental Voice 5,104 27 .16 .40 .35 .46 126.51 "
Other (composite 1,397 9 .21 .53 .40 .63 58.34“
participation)
Self-appraisal 1,3 12 8 .17 .23 .10 .35 15.06“
Value-expressive 9,528 35 .20 .52 .46 .57 369.03"I
Voice
Supervisor 3,51 1 12 .28 .54 .41 .65 277.82"
Knowledge
Judgment Based on Evidence
Appeals Process 966 4 .20 .60 .43 .73 37.3 7*
Job-Relevant 3,767 1 1 .26 .51 .38 .62 244.50“
Factors
Supervisor Lack of 3,898 8 .18 .55 .42 .65 l 18.10“
Bias
Valid Instrument 361 3 .05 .23 .13 .33 .96
(BOS vs. Other)
Rating F avorability and Justice Variables
Rating Favorability 10,880 36 .17 .39 .33 .44 300.07"I
Distributive Justice 2,441 16 .37 .59 .45 .70 32800"
Interactional Justice 1,664 9 .22 .64 .48 .8 1 78.3 8"
Procedural Justice 1,550 12 .13 .73 .69 .76 25.1 1"

 

 

 

 

 

 

 

 

 

Note. N= Cumulative sample size; k =Cumulative number of effect sizes; SDr = weighted standard

deviation of observed effect sizes; Mr = Population mean correlation.

 

75

 

While the above findings indicate that participation in the appraisal is important,
in that participation is correlated with employee reactions, they also indicate that the
social context within which the appraisal occurs and other aspects of due process are
important (sometimes potentially more important). Each of the correlations between
overall reactions and social contextual variables (relationship quality, supervisor
satisfaction, supervisor support and supervisor trust) are at or around Mr = .60 (Table 1).
This is consistent with a primary thesis of the integrative review above (Chapter 4): that
social contextual variables should be similarly related to employee reactions.

Meta-analytic correlations for hypothesis tests and input data for path analyses are

presented in Table 2.

76

 

Table 2

Meta-analytic Results for Employee Reactions to Performance Appraisal: Meta-A nalytic Correlations
or Hypothesis Tests and Path Analyses

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

N K SDr Mr Lower Upper Q
95%Cl 95%CI
Social Contextual Variables (Overall) (Hypotheses [-3)
Distributive Justice 1,821 12 .22 .51 .37 .65 88.32“
Interactional Justice 1,841 9 .22 .79 .62 .95 88.27"
Procedural Justice 949 7 .I l .63 .57 .70 12.00ns
Employee Reactions 4,042 23 .16 .66 .59 .73 105.09“
Knowledge of Performance Standards (Hypothesis 4)
Distributive Justice 645 3 .26 .58 .21 .96 42.82“
Procedural Justice 391 5 . 16 .50 .34 .67 10.01 "‘
Employee Reactions 4,123 14 .16 .53 .42 .64 98.43“
Frequent Feedback (Hypothesis 5)
Distributive Justice 827 3 .28 .51 .12 .90 38.07“
Interactional Justice 1,023 3 . 15 .69 .44 .94 23. 15 "
Procedural Justice 433 3 .1 1 .47 .38 .58 4.69
Supervisor Job Knowledge (Hypothesis 9)

Distributive Justice 1,162 6 .18 .42 .26 .59 3868*
Employee Reactions 3,057 10 .23 .53 .37 .69 156.23"
Supervisor Lack of Bias (Hypothesis [0)

Distributive Justice 565 4 .35 .55 .47 .64 68.04“
Interactional Justice 1,023 3 .1 1 .92 .85 .98 1 1.65“
Employee Reactions 3,814 7 .14 .57 .42 .71 72.90"
Job-relevant Factors (Hypothesis I 2)

Procedural Justice 645 3 .21 .64 .31 .98 26.67“
Employee Reactions 3,683 10 .25 .57 .41 .73 219.03“

 

 

77

Note. N= Cumulative sample size; k =Cumulative number of effect sizes; SD, = weighted standard
deviation of observed effect sizes; Mr = Population mean correlation; Q = heterogeneity of Mr, * =
significant at p < .05.

 

While meta-analytic correlations between social contextual variables and each
type of justice variable are also large (Table 2), which supports Hypothesis 1, the largest
correlation is with interactional justice (Mr = .79, k =9), which supports Hypothesis 2,
and is consistent with organizational justice theory and research (e. g. Colquitt et al.,
2001; see Chapter 2).

As was expected based on the review of due process performance appraisal,
adequate notice and judgment based on evidence were also related to employee reactions.
Both aspects of adequate notice, i.e. frequent feedback (Mr= 38, k =15) and knowledge
of performance standards (Mr = .50, k =18), were correlated (moderate to large) with
overall reactions. Hypothesis 5 predicted that feedback frequency would also be related
to distributive justice (Mr = .51, k =3), interactional justice (Mr.= 69, k =3) and
procedural justice (Mr = .47, k = 3), which was supported. What is interesting here is
that frequent feedback is an aspect of due process and, hence, an aspect of procedural
fairness, but it is most strongly related to interactional justice—indicating that frequent
contact regarding performance between supervisor and subordinate increases perceptions
of interpersonal fairness most.

Each of the aspects of judgment based on evidence, i.e. instrument validity (M,.=
23, k =3, which supports Hypothesis 11), an appeals process (Mr = .60, k =4, which
supports Hypothesis 13), judgment based on job-relevant factors (Mr = .51, k =11), and
supervisor lack of bias (Mr = 55, k =8) were correlated (moderate to large) with overall

reactions.

78

7. 2 Tests of Mediation

Meta-analytic correlations between organizational justice variables used as input

data for path analyses can be found in Table 3.

 

 

 

 

 

 

 

 

 

 

Table 3
Meta-A nalytic Correlation Matrix for Justice Variables (Used as Input Data for Path
Analysis)
DJ IJ PJ

DJ 1
[J .69* 1

k =8

N = 1,057
P] .92“ .80"

k =8 k =5

N = 1,104 N = 630
Note. N= Cumulative sample size; k =Cumulative number of effect sizes; M, = Population
mean correlation, * p < .05.

 

 

Hypothesis 3 predicted that organizational justice variables (distributive,
interactional and procedural justice) would partially mediate the relationship between
social context variables13 and employee reactions. As can be seen from Table 4, social
context was significantly related to each measure of organizational justice, and to
employee reactions. A firlly mediated model fit the data reasonably well [(1, N = 149) =
14.56 p < .01, GF I = .95, CFI = .98, RMSEA = .34, SRMR = .03 R2 = .62, but the path
from interactional justice to employee reactions was non-significant, perhaps because it is
so strongly related to the social context. When adding a direct path from social context to

employee reactions”, the path ([3 = .36, p < .05) was noticeably smaller than the bivariate

 

'3 Since all of the social context variables are conceptually related (Chpater 4) and were all related to
employee reactions at about Mr = .60, these were combined into a single antecedent variable.
” This is ajust-identified model; fit statistics will not be reported.

79

correlation (i.e. M, = .66), R2 = .51; thus, partial mediation was established. See Figure 2

for an illustration of the model with standardized paths.

Figure 2

Model of the Relationship Between Social C ontext, Organizational Justice Variables and
Employee Reactions

          
        

   
  

Social
Context

   
  

   
  

Distributive
Justice

Appraisal
Reactions

Interactional
Justice

Hypothesis 4 predicted that organizational justice variables (distributive and and
procedural justice) would partially mediate the relationship between knowledge of
performance standards and employee reactions. As can be seen from Table 4, knowledge
of performance standards was related to distributive and procedural justice, and employee
reactions; there were too few (k =2) effect sizes to calculate a mean correlation between
knowledge of performance standards and interactional justice. A fully mediated model

did not fit the data well f(1,N = 149) = 19.95 p < .01, GFI = .91, CFI = .92, RMSEA =

80

.42, SRMR = .07, R2 = .58. When a direct path was added from knowledge of
performance standards to employee reactions”, the path ([3 = .35, p < .05) was noticeably
smaller than the bivariate correlation (i.e. M, = .53), R2 = .67; thus, partial mediation was

established. See Figure 3 for an illustration of the model with standardized paths.

Figure 3

Model of the Relationship Between Knowledge of Performance Standards, Distributive
and Procedural Justice and Employee Reactions

       
  

     
    
   
   

   
  

   
  

Knowledge Distributive Appraisal
of Justice Reactions
Standards

Procedural
Justice

Hypothesis 9 predicted that distributive justice would partially mediate the
relationship between supervisor job knowledge and employee reactions. As can be seen
from Table 4, supervisor job knowledge was significantly related to both distributive
justice and employee reactions. A fully mediated model did not fit the data well [(1, N =
149) 14.56p < .01, GFI = .91, CFI = .79, RMSEA = .38, SRMR = .03, R2 = .31. The
partially-mediated modell6 indicated that the magnitude of the direct effect from

supervisor job knowledge to employee reactions (B = .3 6, p < .05) was smaller than the

 

'5 This is a just-identified model; fit statistics will not be reported.
'6 This is a just-identified model; fit statistics will not be reported.

81

bivariate correlation (i.e. M, = .53), R2 = .42; thus, partial mediation was established. See
Figure 4 for an illustration of the model with standardized paths.
Figure 4

Model of the Relationship Between Supervisor Job Knowledge, Distributive Justice and
Employee Reactions

Supervisor Distributive ‘ j __ q , _ Appraisal
Job . Justice . Reactions
Knowledge

 

Hypothesis 10 predicted that distributive justice and interactional justice would
partially mediate the relationship between supervisor lack of bias and employee reactions.
As can be seen from Table 4, supervisor lack of bias was correlated with each of these
variables. A fully mediated model fit the data well [(1, N = 167), .06 p =.81 , GFI =
1.00, CFI = 1.00, RMSEA = .00, SRMR = .002, R2 = .45. When a direct path was added
from supervisor lack of bias to employee reactions, the path was non-significant (B = .23,
ns), R2 = .45. Distributive and interactional justice fully mediated the relationship
between supervisor lack of bias and employee reactions. See Figure 5 for an illustration

of the model with standardized paths.

82

Figure 5
Model of the Relationship Between Supervisor Lack of Bias, Distributive and
Interactional Justice and Employee Reactions

 
   

   
    
  

 
  
   

Distributive
Justice

Supervisor
Lack of
Bias

  
   
 

Appraisal
Reactions

Interactional
Justice

 
   
 

Hypothesis 12 predicted that interactional and procedural justice would partially
mediate the relationship between job-relevant factors and employee reactions. While
there were too few effect sizes to calculate a mean correlation between knowledge of
performance standards and interactional justice, the former was correlated with
procedural justice and employee reactions (Table 2). A fully mediated model fit the data
reasonably well [(1, N = 167), .5.02 p = .03 , GFI = .98, CFI = .98, RMSEA = .18,
SRMR = .04, R2 = .53. When a direct path was added from knowledge of performance
standards to employee reactions”, the path ([3 = .36, p < .05) was noticeably smaller than
the bivariate correlation (i.e. M, = .57), R2 = .55. See Figure 6 for an illustration of the

model with standardized paths.

 

'7 This is a just-identified model; fit statistics will not be reported.

83

Figure 6
Model of the Relationship Between Job Relevant Factors, Procedural Justice and
Employee Reactions

Job
Relevant
Factors J

Procedural
Justice

Appraisal
Reactions

 

7.3 Tests of Moderation
A modified weighted least squares regression approach was used to test
hypotheses involving moderation (Hypotheses 6, 8a, 8b) with continuous moderators

(Lipsey & Wilson, 2001, pg. 141). See Table 4 for a summary of the results.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Table 4
Results of Continuous Variable Moderator Analysis in Regression for Employee Reactions
Independent Moderator B Se Lower Upper Mr R2 Q
Variable Variable 95%Cl 95%Cl
Hypothesis 6

Feedback Rating 11 .56 .18 .21 .91 .44 .05 9.58“
Frequengy Favorability

Hypotheses 8a and 8b
Instrumental Rating 14 - .12 -.30 .16 .48 .00 .40
Voice Favorability .07
Value- Rating 9 .00 .14 -.27 .28 .38 .00 .00
expressive Favorability
Voice
Note. K =number of effect sizes; B = regression coefficient; se = standard error; Mr =
Population mean correlation; Q = heterogeneity of Mr; " = significant at p < .05.

 

84

 

Hypothesis 6 predicted that the relationship between feedback frequency and
employee reactions would vary according to rating favorability, i.e. that the relationship
would become stronger as the relationship between rating favorability and employee
reactions increased. The relationship between feedback frequency and employee
reactions was significantly heterogeneous (Q = 234.60, p < .05), and varied according to
the strength of the relationship between rating favorability and employee reactions (B =
.56, R2 = .05, k =11). Hypothesis 6 was supported.

Hypothesis 8a predicted that the relationship between value-expressive voice and
employee reactions would vary according to the relationship between rating favorability
and employee reactions. The former relationship was significantly heterogeneous (Q =
369.03, p < .05; Table 5), but did not vary according to the relationship between rating
favorability and reactions (B = -.07, ns, k =9). Hypothesis 8b predicted that the
relationship between instrumental voice and employee reactions would vary according to
the relationship between rating favorability and employee reactions. The former
relationship was significantly heterogeneous (Q = 126.51, p < .05), but did not vary
according to the relationship between rating favorability and reactions (B = .00, ns, k

=14).

85

CHAPTER 8

DISCUSSION

Performance appraisal is one of the most heavily researched topics in OB/HR, and
employee reactions to appraisals are an important outcome of the appraisal process (some
scholars and practitioners say the most important outcome), yet there has been a critical
science-practice gap in this literature in that research has not produced a coherent
understanding of why employees react in different ways to appraisals. Scholarly literature
has traditionally focused on the psychometric properties of appraisals, with little
consideration of the context within which appraisals occur. In response to criticisms from
the scholarly and practitioner communities both, research has shifted its focus to
contextual aspects of the appraisal, and to employee reactions to appraisals. The context
of performance appraisal and employee reactions to appraisals are highly interrelated, yet
there has been no comprehensive review of the employee reaction literature, and no
integrative framework exists with propositions about when and why contextual
antecedents should be related to employee reactions.

The purpose of this dissertation was to provide such an integrative framework in
order to organize the literature on employee reactions to performance appraisal, including
mediating mechanisms whereby contextual antecedents are related to reactions, as well as
potential moderators of the relationships between contextual antecedents and reactions.
Organizational justice theory was used as an over—arching theoretical lens to develop
hypotheses about these relationships. A related purpose was to test, using meta—analytic

correlations and multivariate analyses with these meta-analytic correlations, some of the

86

relationships articulated by this framework, as to guide future theory, research and
practice related to performance management and appraisal.

Based on social exchange theory and research, the integrative framework
proposed that social contextual variables would be similarly related to employee
reactions. Researchers have measured relationships between employee reactions and
supervisor-subordinate relationship, quality, supervisor trust, supervisor and support. An
additional proposition, based on organizational justice theory, was that justice
perceptions, and especially interactional justice, should explain (mediate) the relationship
between social contextual variables and employee reactions.

Researchers have also measured relationships between employee reactions and all
aspects of due process outlined in Folger and colleagues’ (1992) due process model of
performance appraisal, but consider only bivariate relationships. The due process model
proposes that employees should react more positively to performance appraisals that are
procedurally fair. Integrating this thesis with propositions from organizational justice
theory, the integrative framework developed in this dissertation posited that perceptions
of justice should explain (mediate) relationships between due process performance
appraisal and employee reactions. Differential predictions were developed and tested
based on justice theory as to which types of justice perceptions should mediate different
elements of due process. An additional proposition developed and tested in this
dissertation, also based on justice theory, was that the relationship between rating
favorability and reactions should bound (moderate) the relationship between voice and
reactions, and between feedback frequency and reactions. The theoretical, managerial and

future research implications of the results of these analyses are described below.

87

8.1 The Social Context of Appraisal

Recent models of the performance appraisal process have proposed that the social
context of performance appraisal is an important area of research inquiry (i.e. Erdogan,
2002; Levy & Williams, 2004; Murphy & Cleveland, 1995), if not the key context to
study as related to employee reactions (e.g. Russel & Goode, 1988). A number of studies
in recent years have tested relationships between social contextual variables and
employee reactions (Williams & Levy, 2004). Being conceptually related to rater-ratee
relationships (Pichler, Varma, & Petty, 2008), these constructs should be related in
similar ways to employee reactions, and should thus be treated as a group of similar
antecedents. Leader-member exchange theory has proposed and documented that
supervisor trust and support are aspects of exchange quality (Dansereau et al., 1975), and
empirical research supports this multidimensional perspective (e.g. Johnson, 2003). This
was supported: all of the social contextual variables were related to reactions at or around
M, = .60 with employee reactions. These results suggest that future researchers should
continue to investigate the role of social context as related to employee reactions, and that
the variables included in the extant literature can perhaps be subsumed and considered as
aspects of relationship quality.

Theory and research have also indicated that more positive affect (Varma, Pichler,
& Srinivas, 2005) and better interpersonal relationships (Vanna, Pichler, Srinivas, &
Abarillo, 2007) between supervisors and subordinates are related to better interpersonal
treatment (interactional justice), increased participation (procedural justice) and better
ratings and rewards (distributive justice) (Cleveland and Murphy, 1992; Wayne & Liden,

1995; Wexley & Klimoski 1984). It was, therefore, proposed that social contextual

88

variables should be positively related to each type of justice (Hypothesis 1), which was
also supported.

Organizational justice theory and research have proposed and found that socially-
relevant variables (e.g. relationship quality) are most strongly related to interactional
justice, as compared to procedural and distributive justice. Results indicated that social
contextual variables were most strongly related to interactional justice (Hypothesis 2).
Thus, this supports a primary theoretical tenet of justice theory: perceptions of
interactional, procedural and distributive justice are independent, and each type of j ustice
should be differentially predicted by content-similar correlates. Consistent with this,
although not predicted a priori, frequent feedback was most strongly related to
interactional justice perceptions, as well. Frequent feedback fiom one’s supervisor
regarding performance probably indicates to the subordinate that s/he is valued (which is
consistent with leader-member exchange theory), and it is logical that this increased
social interaction would increase perceptions that one is being treated more favorably
interpersonally.

An additional proposition from organizational justice theory that was tested is that
perceptions of justice help to explain why individuals react the way that they do to
organizational systems. Thus, if perceptions of interactional, procedural and distributive
justice are positively related to social contextual variables, perhaps because these
variables are related to better interpersonal and instrumental treatment, then overall
reactions to the appraisal systems should be more favorable because of more favorable

justice perceptions (Greenberg, 1990). This was supported: perceptions of organizational

89

justice partially mediated the relationship between the social context of appraisal and
reactions to the appraisal.

These results suggest that the way employees are treated interpersonally before, in
terms of frequent feedback, and during the appraisal, in terms of dignity and respect, can
have an important impact upon the way in which they respond to their appraisal. If a
major goal of performance management and appraisal is to provide employees with
feedback to remedy performance problems and identify areas for improvement so that
performance can be maximized, then this has to be done in an environment where
employees feel supported by their supervisor, and where they are treated with dignity and
respect. In fact, social contextual variables were more strongly correlated with reactions
than was rating favorability and, in most cases, more strongly correlated with employee
reactions than were due process variables.

Organizational research has shown, however, that more positive supervisor-
subordinate relationships are related to better treatment, and leader-member exchange
theory and research have shown that that supervisors treat employees differently based on
in- and out-groups. Results outlined above, therefore, suggest that training programs
related to performance appraisal for supervisors should address ways for supervisors to
attend to not just performance-related information before and during the appraisal, but to
the way in which they deliver their feedback, both before and during the appraisal. That
is, supervisors should be trained in how to deliver feedback in a way that is courteous and
respectful of employees, and that provides adequate explanations regarding how and why

performance ratings and appraisal-related decisions were made.

90

In fact, this may be an effective way to produce more favorable reactions to the
appraisal, and thus potentially increase employee performance, even when feedback is
negative. Likewise, better interpersonal treatment during the appraisal, i.e. interactional
justice, might be a way to prevent employees from responding negatively to the appraisal,
which is exceedingly common, even when the system itself does not exemplify a due
process model. Future research should test whether or not perceptions of interactional
justice interact with perceptions of procedural and distributive justice to predict employee
reactions. There is supportive evidence in the broader justice literature, but again, most of
the employee reaction literature is limited in the extent to which complex relationships
such as these are considered or tested.

The importance of supervisor-subordinate relationships and the various aspects of
relationship quality studied in this dissertation to performance appraisals have been
highlighted in the recent scholarly literature (Pichler et al., 2008), but are not salient in
popular human resource management texts (e. g. Noe, Hollenbeck, Gerhart, & Wright,
2000), government handbooks on performance appraisal
(httpz/lwwwthgOV/hmi/guidancefl 70dm430hndbk.pdf), or discussions in the
practitioner literature (e. g. Fox, 2009). For instance, Noe and colleagues (2000) propose
that there are nine (9) characteristics of effective performance feedback processes,
including characteristics studied herein, such as frequent feedback, employee
participation, focusing feedback on objective results or behaviors, setting specific goals,
as well as characteristics that are related to interactional justice, namely recognizing

performance through praise and minimizing criticism.

91

What is missing here is that trust in one’s supervisor, supportive supervisory
behaviors, and a quality supervisor-subordinate relationship both during and before the
appraisal session seem to be related in important ways to employee reactions to the
appraisal. While most of the studies in the current database are cross-sectional, results
suggest that not only is interpersonal treatment during the appraisal strongly related to
positive appraisal reactions, so too are aspects of relationship quality. This makes sense:
To the extent that an employee feels supported by his or her supervisor, i.e. is provided
with the resources needed to effectively perform his or her job, then perceptions of fair
treatment should increase, thereby leading to more positive appraisal reactions. The
implication here is that organizations should focus on developing managers who are
socially supportive of their employees, and who are able to develop high-commitment
relationships with them.

The practitioner literature is saturated with survey statistics indicating that
employees often disagree with their supervisors about performance feedback and are,
therefore, dissatisfied with the appraisal process (e.g. Fox, 2009). The results of this
dissertation suggest that supervisor-subordinate relationships that are characterized by
trust and mutual satisfaction are less likely to result in perceptions of unfairness in the
appraisal, and are more likely to lead to positive employee responses to the appraisal.
This is an important way in which this study helps to remedy the science practice gap in
performance appraisal: employees react differentially to appraisals not just because of
their feedback or rating — but because of the way in which they are treated interpersonally

by their supervisors both before and during (and probably after) the appraisal.

92

Of course, employees themselves can take leadership in developing high quality
relationships with their supervisors, and seek out feedback from and develop trust with
their supervisors (Dansereau et al., 1975). Since social relations with one’s supervisor are
so strongly related to perceptions of fairness in the appraisal, it seems advantageous for
employees to communicate regularly with supervisors, respond actively to feedback from
supervisors related to performance, as well as to the delegation of tasks and
responsibilities—all of which should increase relationship quality and trust, according to
the leader-member exchange literature (e. g. Dansereau et al., 1975). Organizations might
consider training programs for employees that develop interpersonal communication
competence as related to receiving and responding to performance feedback. In this way,
employees themselves can more actively engage in the performance management process
through more effective two-way communications with supervisors.

While the practical implications here are relatively straightforward, there is much
empirical evidence — and common knowledge - to suggest that supervisors do not always
treat employees with respect, perhaps especially when they are delivering critical
feedback. Since performance feedback is not always positive, and since supervisors are
have a great deal of discretion when it comes to determining the way in which the
appraisal session will be lead (Murphy & Cleveland, 1991), it is important to consider
how to increase procedural fairness in performance appraisal, which will be considered
next
8.2 Due Process Performance Appraisal

The review above identified a number of studies that measured relationships

between due process performance appraisal and employee reactions. That said, almost

93

none of these studies did so using the due process model of appraisal developed by

F olger and colleauges (1992), with a few exceptions (e.g. Taylor et al., 1995). This limits
the contribution these studies make to the understanding of due process and employee
reactions. As such, studies were organized according to which aspects of due process
were measured, and how these aspects were related to reactions, in order to utilize and
build upon the due process model of performance appraisal as it relates to appraisal
effectiveness. The meta-analytic correlations between due process and employee
reactions suggest that due process is indeed an effective way to produce more favorable
employee reactions to performance appraisal; all of the correlations were moderate to
large. Each aspect of due process will be considered in turn below, as will the
implications of the results.

Adequate Notice. Adequate notice requires that the person being judged or
evaluated by a system be aware of the standard against which s/he is being judged before
being evaluated by the system. As applied to performance appraisal, it requires that
employees are aware of the performance standards by which they will be evaluated, and
are given frequent feedback about their performance relative to those standards.

Knowledge of performance standards is a system-oriented (Erdogan et al., 2001)
aspect of due process. It involves, for instance, knowledge of standards through job
descriptions and published documents related to organizational performance goals, and is
often gained by employee initiative (Levy & Williams, 2000). It was, therefore, proposed
that procedural and distributive justice would mediate the relationship between
knowledge of standards and reactions. That is, knowledge of standards should make the

performance management system and the appraisal process seem fairer, and the employee

94

should be better able to perform to set standards and, thus, receive a more favorable
review. This (Hypothesis 4) was supported, at least in part: procedural and distributive
justice partially mediated the relationship between knowledge of standards and reactions.

Feedback frequency has received a good amount of attention in the performance
appraisal literature for decades, and has been included as a central variable to the
feedback environment of performance appraisal (e.g. Kinicki et al., 2001) and to the
overall performance management process (Ilgen et al., 1979; Klein & Snell, 1994). It was
proposed that feedback frequency should also be related to perceptions of procedural and
distributive justice, for reasons similar to those for knowledge of standards, but that
feedback frequency should also be related to interactional justice, since it is a relatively
discretionary appraisal-related behavior (Cleveland & Murphy, 1991). This (Hypothesis
5) was supported and results seem to indicate that feedback frequency is indeed
discretionary since it was most strongly related to interactional justice.

While these results suggest that the relationship between feedback frequency and
reactions is positive overall, there are inconsistent results across studies. It was
accordingly proposed (Hypothesis 6) that the relationship between feedback frequency
and reactions would be stronger as the relationship between rating favorability and
reactions increased. That is, reactions to feedback are determined not only by feedback
frequency, but feedback sign as well (Ilgen et al., 1979; Landy et al., 1978; Kinicki et al.,
2004; Kluger & DeNisi, 1996), such that reactions to appraisals will be most positive
when both feedback is frequent and the subsequent rating is more positive. Thus
hypothesis was supported, indicating that some of the variability in the relationship

between feedback frequency and reactions is due to the relationship between rating

95

favorability and reactions. Future research should measure both aspects of the feedback
environment, and test this moderating effect across different types of samples and
organizational settings.

These results suggest that employee awareness of performance standards, and
feedback about how their performance compares to these standards prior to the appraisal,
can result in more positive employee reactions to performance appraisal, perhaps
especially when ratings are more favorable. While the importance of regular feedback for
employees has been touted by practitioners and by models of the appraisal, these results
provide some evidence as to the importance of regular feedback. They also suggest that
knowledge of standards is important, perhaps even more important than feedback
frequency. Of course, performance feedback is meaningless without knowledge of
performance standards, so it would seem that the two go hand-in-hand and are
complementary. Future research should, therefore, consider that extent to which
knowledge of performance standards and frequent feedback are complimentary, i.e. the
extent to which frequent feedback enhances the positive relationship between knowledge
of performance standards and employee reactions.

These results suggest that organizations should implement processes that increase
employee knowledge of performance standards, both through formal and informal
channels. For instance, publication of performance standards for various job families in
organizational handbooks would seem to be helpful. Increasing supervisor-subordinate
discussions of performance standards and goals throughout the year (and not just during
the appraisal session) would also seem to be helpful. Employee knowledge of

performance standards should be beneficial not just to employees — but to organizations

96

as well. Research indicates that most performance management systems do not align
strategic organizational goals with employee behaviors or the assessment of these
behaviors. For the performance management system to have an effect on an
organization’s bottom line and thus contribute to a high performance work system, it is
important that organizations and their managers make these goals and standards salient to
employees, and a salient aspect of their performance assessment.

Fair Hearing. A fair hearing involves the opportunity for the person being judged
by a system, in this case a performance appraisal system, to present evidence on one’s
behalf, as well as knowledge on the part of the person doing the judging about the facts of
the situation, in this case knowledge about the employee’s job and his or her performance
as related to the requirements of that job. A previous meta-analysis (Cawley et al., 1998)
established, using data from 27 studies, that the former, i.e. employee participation, was
positively related to employee reactions to appraisals.

Since bivariate meta-analytic relationships between participation and employee
reactions were the focus of the Cawley et al. (1998) study, it is reassuring that, even when
adding to (sometimes doubling) the number of studies reported for a particular
correlation, results are quite consistent. What is also important here is one of the more
prominent findings of the previous meta-analysis, that value-expressive voice was more
strongly related to employee reactions than instrumental participation, is buttressed
(Hypothesis 7). Scholars have been critical of the proposition that value-expressive voice
is more strongly related to reactions to performance appraisal than instrumental voice
(Bonness & Macan, 2006; Suh, 1992). It seems logical that instrumental voice would be

as important or more important than value-expressive voice, since this involves not only

97

voice, but additionally the sense that one has controlled the outcome of the appraisal,
which should result in favorable results for the employee.

In order to test the voracity of the proposition that value-expressive voice is more
strongly related to employee reactions than is instrumental voice, a proposition was
developed, based on justice theory, that rating favorability might moderate the way in
which voice is related to reactions. More specifically, it was proposed that value-
expressive voice would be more important when the relationship between rating
favorability and reactions was strong (Hypothesis 8a); if the employee is already satisfied
because of his or her rating, then instrumental control is less important. Conversely, it
was proposed that instrumental voice would be more important when the relationship
between rating favorability and reactions was weak (Hypothesis 8b); if one is dissatisfied
because of one’s rating, then instrumental control would be more important. Results were
unsupportive of either hypothesis.

One might alternatively propose that since the data upon which this study is based
are cross-sectional and reflective, the relationship between instrumental voice and
reactions might actually be made weaker when the relationship between rating
favorability and reactions is high. That is, if one is dissatisfied with one’s appraisal
because of a poor rating, then the extent to which instrumental control was exercised
might exacerbate this dissatisfaction because the instrumentality was unsuccessful. Of
course, this perspective would not be supported either since the results of the hypothesis
tests were non-significant.

While the finding that the relationship between rating favorability and employee

reactions did not moderate relationships between instrumental- and value-expressive

98

voice and employee reactions is inconsistent with the notion in the organizational justice
literature that procedural justice and outcome favorability should interact to predict
reactions towards organizational decisions, it is consistent with the proposition that voice
for the sake of voice, i.e. regardless of outcome favorability, is important (Cawely et al.,
1998; Lind, 2001). Moreover, the two-way interaction between procedural justice and
outcome favorability is also inconsistently demonstrated empirically in the justice
literature, suggesting that the latter does not attenuate the former under all circumstances.
While it was suggested here that this may be due to the differences between instrumental
and value-expressive voice, it seems that there are other explanations for this
inconsistency that should be explored.

Results related to participation in the appraisal support both the self-interest and
group-value models of procedural justice (Lind & Tyler, 1988). The former posits that
persons desire control over processes that affect them so that they can affect more
positive outcomes from the process for themselves (Thibaut & Walker, 1975). As applied
to performance appraisal, this would suggest that instrumental voice is important to
employees because they can influence the process and the outcome. The group-value
model posits that persons want to express voice in processes that affect them not for
control, but so that their perspective is heard. Since value-expressive voice was more
strongly related to employee reactions than was instrumental voice, and since the
relationship between voice (both forms) and reactions was not related to relationship
between rating favorability and reactions, this gives further credence to the thesis that
employees desire voice for voice’s sake—regardless of control for self-interested reasons

(Tyler, 1989).

99

In fact, of all the different forms of participation in the appraisal, the correlation
between reactions and instrumental voice was the weakest, excepting self-appraisal.
Since Cawley and colleagues (1998) did not measure the relationship between goal
setting and reactions, it is important to note here that goal setting is strongly related to
reactions. Employee participation in the appraisal can take various forms, and since goal
setting has been found to be related to a variety of positive outcomes for employees and
organizations, e. g. increased motivation and performance, implementing goal setting into
the appraisal seems to be an effective way to not only manage employee performance,
but also to keep employees interested in and satisfied with the appraisal process.

It is also interesting to note that the correlation between self-appraisal and
reactions is about half the size of most of the correlations between reactions and other
forms of participation (which is also consistent with Cawley et al., 1998). This is
consistent with the observation made above that some studies find positive results for
self-appraisal while others do not. Reasons for this should be articulated an explored,
since many organizations encourage self-appraisal. One reason could be the potential for
disagreement that this creates, which could create both psychological dissonance, as well
as actual conflict in the appraisal session and afterwards.

These results suggest that, regardless of the favorability of one’s performance
rating, it is important for organizations and supervisors to allow employee participation in
the appraisal process. To the extent that employees are given opportunity to set goals
prior to the appraisal, and participate in the appraisal by voicing their opinion about their
performance, perceptions of justice increase, thereby producing more favorable reactions

to the appraisal. This is consistent with textbook descriptions of effective performance

100

appraisal systems (e.g. Noe et al., 2000), but managers vary in the extent to which they
are likely to allow employee input into the process.

Organizations should, therefore, consider interventions designed to enhance
employee participation, such as training programs for employees and managers. One way
to increase employee participation is through self-appraisal. That said, the qualitative
review above highlighted the inconsistent relations between self-appraisal and reactions,
and the quantitative review found that self-appraisal was only modestly related to
reactions. Perhaps the best way to increase employee participation is to adopt a mutual
problem-solving approach to the appraisal, wherein managers and employees share in the
responsibility of diagnosing and remedying performance problems (e.g. Wexley, Singh &
Yukl, 1973). In this way, employee participation is encouraged, and future goals are set
based on mutual and open discussion between managers and employees.

In addition to participation in the appraisal, supervisor job knowledge of
employee performance is a key aspect of a fair hearing. It was proposed (Hypothesis 9)
that distributive justice would mediate the relationship between supervisor job knowledge
and employee reactions. If an employee believes that his or her supervisor is aware of and
knowledgeable about his or her performance, this should increase the perception that the
supervisor is able to provide a valid and fair rating, thus increasing perceptions of
distributive justice.

Distributive justice partially mediated the relationship between supervisor job
knowledge and employee reactions, which supports the notion that perceptions of
outcome fairness increase due to perceived supervisor job knowledge and also explain

why employee reaction are more favorable when this knowledge is relatively high. Future

101

research should attempt to uncover the circumstances under which perceptions of
supervisor job knowledge are increased. This has yet to be investigated in the employee
reaction literature.

Judgment Based on Evidence. The judgment based on evidence component of due
process requires that decisions are made that are consistent and free from bias, and that
decisions can be appealed. Supervisor lack of bias and the availability of an appeals
process have both been measured in the employee reaction literature as correlates of
employee reactions.

While supervisor lack of bias is an aspect of due process and, hence, procedural
fairness, it was proposed (Hypothesis 10) that perceived lack of bias would be related to
distributive and interactional justice (perceptions of procedural justice have not been
measured as a correlate in the extant literature), and that these justice perceptions would
mediate the relationship between lack of bias and reactions. If an employee perceives his
or her supervisor to be unbiased, then the supervisor should be perceived as providing
better interpersonal treatment; the extent to which a supervisor is biased is likely to be
perceived as discretionary, at least in part, and not determined by the appraisal system
itself. Moreover, if there is a lack of bias, perceptions of outcome fairness should increase
since the outcome was determined by a fair decision maker.

This hypothesis was supported: perceptions of distributive and interactional
justice completely mediated the relationship between supervisor lack of bias and
reactions, indicating that employees associate lack of bias with better interpersonal
treatment and outcome fairness, and that this results in more positive reactions to the

appraisal. Future research should investigate procedural justice as a correlate of lack of

102

bias, in addition to interactional and distributive justice, in order to determine if
perceptions of procedural fairness explain additional variance in reactions. It seems
interesting that an aspect of due process is completely mediated by justice perceptions—
but not procedural justice perceptions.

Since supervisors are not always unbiased, different forms of performance rating
instruments have been developed in an attempt to increase the fairness of the appraisal.
This “psychometric” approach to performance appraisal effectiveness has been criticized,
but results indicate that behavior observation scales are viewed more favorably by
employees than other types of rating formats (i.e. graphic rating scales, Hypothesis 11).
While only based on three studies, the correlation between rating instrument type and
reactions was the smallest of all due process variables. Rating instrument type is
somewhat different from instrument validity, which has not been investigated as a
correlate of employee reactions in the extant literature, but should be.

For decisions to be free from bias, they should be based on relevant information.
As applied to performance appraisal, a performance rating should be based on job-
relevant information. Employee perceptions of judgment based on evidence have been
measured as a correlate of appraisal reactions. It was proposed (Hypothesis 12) that
perceptions of interactional and procedural justice would mediate the relationship
between judgment based on evidence and reactions, since this should increase the
perception that the process by which performance was evaluated was fair, as well as the

perception that interpersonal treatment was fair, given that it is often somewhat

103

discretionary as to which factors will be used to evaluate performance. Procedural
justicel8 partially mediated the relationship between job-relevant factors and reactions. '

Finally, judgment based on evidence requires that the outcome of a decision
process should be appealable. It was proposed that the opportunity to appeal one’s
performance rating would be positively related to employee reactions, which was
supported; the meta-analytic correlation was large, .60. This correlation was based on
only four studies, so future research should include appeals processes as an antecedent of
employee reactions, and should investigate under what circumstances an appeals process
is most effective. For instance, it seems logical that appeals processes are most desirable
to employees when they feel that they have been treated unfairly interpersonally, which
would increase the perception of bias in the appraisal system, or when they feel their
rating is inaccurate.

This set of results supports the proposition that each aspect of judgment based on
evidence, i.e. supervisor lack of bias, judgment based on job-relevant factors and an
appeals process, should be related to perceptions of justice in the performance appraisal
and, hence, more positive appraisal reactions. Since bias in the appraisal can be due to a
variety of factors, such as political considerations, interpersonal biases (e.g. aversive
sexism or racism), etc., organizations should consider using ratings formats that are
relatively more objective, namely behavioral observation scales, as well as providing
employees with decision recourse.

Behavioral observation scales are meant to maximize the extent to which ratings
are based on job-relevant factors, and appeals processes are meant to scrutinize the extent

to which ratings — and related decisions — were based on job-relevant factors, and not

 

'8 There were too few effect sizes to include interactional justice in the model.

104

extraneous influences, such as bias. Interestingly, judgment based on job-relevant factors
and decision recourse are not included as “characteristics of effective performance
feedback processes” in major human resource management textbooks (e. g. Noe et al.,
2000), but results of this dissertation suggest that are related in important ways to
employee reactions to appraisals, and should thus be given adequate attention from
organizations.
8. 3 Additional Study Limitations and Future Research Directions

As is the problem with much of the empirical research in the organizational
sciences, and with meta-analyses of this literature (e.g. Cohen-Carash & Spector, 2001;
Colquitt et al., 2001), the effect sizes included in the database for this meta-analysis were
almost exclusively based on cross-sectional, percept-percept data. This is a concern not
only because of the potential for effect sizes to be inflated, but also because this limits the
extent to which research sheds light on the process by which employee reactions occur.
That said, meta-analysis allows for 1) more stable estimates of bivariate relationships,
which is important to the current study given discrepancies in the extant literature in
terms of magnitude and sometimes direction of effects, 2) tests of moderators of bivariate
relationships, which is also highly relevant to the current study in terms of including
rating favorability as a potential moderator of relationships between voice, feedback
frequency and employee reactions, and 3) meta-analytic path analysis is well-suited for
clarifying empirical models and informing theory (e.g. F assina, Jones, & Uggerslev,
2008). Some researchers (e.g. Pettijohn et al., 2001) have begun to test some aspects of
the performance appraisal process over time, and future researchers should continue to

employ more longitudinal research designs.

105

The model developed in this paper contends that social interactions and
procedural elements of performance appraisals — which occur or are revealed both before
and during the actual appraisal session — affect employee perceptions of appraisal justice,
which then affect employee reactions. Results are largely supportive of this process
model, but are not as yet conclusive given the cross—sectional nature of this research
domain. Future research should test the robustness of this model with longitudinal
research -- which measures contextual antecedents before the appraisal, and reactions
after the appraisal.

For instance, researchers could measure relationship quality and feedback
frequency before the appraisal, then measure perceptions of justice shortly after the
appraisal, and follow-up with subsequent measures of employee reactions, such as
motivation to improve, as well as actual performance measures. In this way, one could
more fully understand how relationship quality and pre-appraisal feedback interact and
affect employee perceptions of fairness of the appraisal, and how perceptions of fairness
affect reactions and, ultimately, performance. This would be a more robust test of the
process model developed in this dissertation.

In fact, it is likely that the process model may act as a sort of feedback loop such
that appraisal processes deemed to be fair and satisfactory lead to increased performance,
which then leads to more positive supervisor-subordinate interactions, which then leads
to a more favorable session during the next appraisal period. Likewise, appraisal
processes that are deemed to be unfair and unsatisfactory may lead to performance
decreases and, thus, a less positive experience during the next appraisal session and,

perhaps ultimately, reduced commitment and increased burnout. Researchers should,

106

therefore, measure perceptions of appraisal processes, reactions to appraisals and job
performance at multiple points in time over more than one appraisal period.

It is also important to include employee individual difference variables in future
research on appraisal reactions. Very few studies in the current database included
employee individual differences as correlates of appraisal reactions. It is possible —
perhaps likely — that reactions to appraisals may differ according to individual
differences, such as personality. For instance, individuals who are higher on trait negative
affectivity may respond less favorably to negative feedback, all else equal. Integrating
measures of individual differences into models of appraisal reactions, therefore, could
potentially explain more variance in reactions, and provide more stable estimates of
substantive relationships between social contextual and process-oriented variables as they
relate to reactions.

Differences in employee performance prior to the appraisal should also be
measured and included in models of appraisal reactions. While some studies did so in the
current database, future research should more thoroughly investigate how employees with
different performance levels respond differentially to appraisal feedback and to different
aspects of the performance appraisal process. For instance, it may be the case that for
high performers, better interpersonal treatment is most likely to lead to more positive
reactions and subsequent positive outcomes, such as increased organizational
commitment and performance increases. For low performers, using job-relevant factors in
the appraisal and mutual goal setting may be more likely to lead to positive appraisal

reactions and performance increases. This has important implications for the way in

107

which performance appraisals are tailored to different types of employees, and for the
way in which performance feedback is delivered.
8. 4 Summary of Study Contributions and Conclusion

Employee reactions to performance appraisal have long been an interest to
researchers and to managers, but have only recently received extensive research attention.
The purpose of this dissertation was to organize this disconnected literature by
developing an integrative framework and hypotheses about when and why contextual
antecedents are related to employee reactions. In this way, the critical science-practice
gap, “Why do employees react differentially to performance appraisals” — or, perhaps
more accurately “Why do employees react so negatively to performance appraisals” —
was addressed by identifying and organizing the antecedents of employee reactions
studied in the extant literature into conceptually meaningful groups, identifying and
testing the boundary conditions of these antecedents, and identifying and testing the
mediators of the relationships between various antecedents and employee reactions.

Two sets of antecedents were shown to be related in important ways to employee
reactions: social contextual variables and due process performance appraisal
characteristics. Heretofore, these antecedents were typically studied in a bivariate fashion
with little consideration of their interrelatedness, and when and why they should be
related to reactions. Results suggest that reactions are the result of a somewhat complex
process whereby perceptions of justice mediate relationships between appraisal context
and reactions - and rating favorability moderates relationships between appraisal context

(i.e. feedback frequency) and reactions.

108

Future research should continue to address the reasons why contextual
antecedents predict employee reactions (mediation) and under what circumstances
(moderation) as to come to a more thorough understanding of the employee reactions
nomological net, and to guide practical applications of this research. The framework

developed in this study can serve as a guide for this research, and a preliminary

nomological net.

109

Appendices

 

APPENDIX A:

Definitions of Due Process Performance Appraisal Dimensions and Sub-Dimensions and

Corresponding Chapter Sections

 

 

Adeqtfle Notice (Ch. 5.
Set—etiom
Employees are held
accountable for performance
standards they understand.

Knowledge of Standards
Employees are made aware of

performance standards, e.g.
through organizational
publication and dissemination
of standards.

Freguent Feedback
Supervisors provide frequent

performance feedback to
employees so that they may
perform to expectations.

Due Process Dimensions

Fair Healing (Ch. 5. Section 3)
Employees are given a
review/hearing regarding their
performance rating.

Sub-Dimensions

Formal Review
Formal performance reviews are
held in order to provide
feedback regarding performance
ratings.

Employee Participation
Employees have an opportunity
to participate in the review and
express their opinions regarding
their performance.

Mwisor J ob-Relafid

Knowledge
The supervisor (rater) is
knowledgeable about a ratee’s

job and job-related performance.

Judgment Based on Evidence
(Ch. 5, Section 4)
Performance ratings are based on
objective performance-related

factors.

Sypgrvisor Neufialitv/Larflf
Big
The supervisor (rater) judges
performance in a neutral way,
i.e. without bias (either positive
or negative) towards any
particular employee.

Valid Rating Instrument
The appraisal instrument is
based on valid performance
dimensions.

Job-Relevant Factors
Performance is rated based on
objective, job-relevant
performance dimensions.

Appeals Process
Employees (ratees) have

opportunity to appeal their
performance rating.

 

110

 

References

Adams, S. (1963). Toward an understanding of inequity. Journal of A bnormal and Social
Psychology, 67, 422-436.

Ambrose, M. L., Hess, R.L., & Ganesan, S. (2007). The relationship between justice and
attitudes: An examination of justice effects on event and system—related attitudes.
Organizational Behavior and Human Decision Processes, 103, 2 1 -3 6.

Arvey, D., & Murphy, K. (1998). Performance evaluation in work settings. Annual
Review of Psychology. 49, 141-168.

Balzer, W.K., Sulsky, L.M. (1990). Performance appraisal effectiveness. In Murphy,
K.R., Saal, F.E. (Eds), Psychology in Organizations: Integrating Science and Practice.
Erlbaum, Hillsdale, NJ, pp.133-56.

*Bartol, K.M., Durham, C.C., & Poon, J .M.L. (2001). Influence of performance
evaluation rating segmentation on motivation and fairness perceptions. Journal of
Applied Psychology, 86 (6), 1106-1119.

Bassett, G.A., & Meyer, H.H. (1968). Performance appraisal based on self-review.
Personnel Psychology, 21, 421 -430.

Bemardin H.J., & Beatty, R.W. (1984). Performance appraisal: Assessing human
behavior at work. Boston, MA: Kent Publishing Co.

Bies, R.J. (1987). The predicament of injustice: The management of moral outrage. In
Cummings, L.L, Staw, B.M. (Eds), Research in Organizational Behavior. JAI Press,
Greenwich, CT, Vol. 9 pp.289-319.

Bies, R.J., & Moag, J .S. (1986). Interactional justice: Communication criteria of fairness.
Research on Negotiation in Organizations. JAI Press.

*Bonness, B., & Macan, T. (2006). Reactions to the performance appraisal process:
Effects of self-appraisals. Paper Presented at the Society for Industrial-Organizational
Psychology conference.

*Boswell, W.R., & Boudreau, J.W. (2000). Employee satisfaction with performance
appraisals and appraisers: The role of perceived appraisal use. Human Resource
Development Quarterly, 11 (3), 283-299.

Brockner, J. (2002). Making sense of procedural fairness: How high procedural fairness

can reduce or heighten the influence of outcome favorability. Academy of Management
Review, 27(2), 58-76.

111

Brockner, J ., & Wiesenfeld, BM. (1996). An integrative framework for explaining

reactions to decisions: Interactive effects of outcomes and procedures. Psychological
Bulletin, 120, 189-208.

*Brown, M., & Benson, J. (2003). Rated to exhaustion? Reactions to performance
appraisal processes. Industrial Relations Journal, 34 (1), 67-81.

*Buehler, L.M. (2006). Due process dimensions of performance appraisal, perceptions of
organizational justice. and some outcomes. Unpublished Doctoral Dissertation, DePaul
University.

”Burke, R.J. (1970). Characteristics of effect performance appraisal interviews: Open
communication and acceptance of subordinate disagreements. Training and Development
Journal, 24, 9-12.

"Burke, R.J., Weitzel, W., & Weir, T. (1978). Characteristics of effective employee
performance review and development interviews: Replication and extension. Personnel
Psychology, 31, 903-919.

"Burke, R.J., & Wilcox, D.S. (1969). Characteristics of effective employee performance
review and development interviews. Personnel Psychology, 22, 291-305.

"Bustamante, C., & Dickinson, TL. (1996). Some determinants of employee attitudes
about the performance appraisal process. Paper presented at the 11th annual conference
of the Society for Industrial and Organizational Psychology, San Diego.

Caplan, R., Cobb, S., French, J ., Harrison, R., & Pinneau (1975). Jod demands and
worker health. Ann Arbor, MI, Institute for Social Research.

Caplan, R., Harrison, R., Wellons, R., & French. J. (1980). Social support and patient
adherence: Experimental and survey findings. Ann Arbor, MI, Institute for Social
Research.

Cardy, R.L., & Dobbins, G.H. (1994). Performance appraisal: Alternative perspectives.
Cincinnati, OH: Southwestern Publishing.

Carver, C. S., & Scheier, M. F. (1981). Attention and self-regulation: A control-theory
approach to human behavior. New York: Springer.

Cawley, B.D., Keeping, L.M., & Levy, PE. (1998). Participation in the performance
appraisal process and employee reactions: A meta-analytic review of field investigations.
Journal of Applied Psychology, 83 (4), 615-633.

Cleveland, N. J ., & Murphy, K. R. (1992). Analysis of performance appraisal as goal-

directed behavior. In G. Ferris & K. Rowland (Eds.), Research in personnel and human
resource management (Vol. 10, pp. 121-185). Greenwich, CT: JAI Press.

112

 

*Cobb, A.T., Vest, M., & Hills, F. (1997). Who delivers justice? Source perceptions of
procedural fairness. Journal of Applied Social Psychology, 27 (12), 1021-1040.

Cohen-Charash, Y., & Spector, P. E. (2001). The role of justice in organizations: A meta-
analysis. Organizational Behavior and Human Decision Processes, 86, 278-321.

Colquitt, J .A. (2001). On the dimensionality of organizational justice: A construct
validation of a measure. Journal of Applied Psychology, Vol. 86, 386-401.

Colquitt, J .A., Wesson, M.J., Porter, C.O.L.H., Conlon, D.E., Ng, KY. (2001). Justice at
the millennium: A meta-analytic review of 25 years of organizational justice research.
Journal of Applied Psychology, 86, 425-46.

Cook, J., & Crossman, A. (2004). Satisfaction with performance appraisal systems.
Journal of Managerial Psychology, 1 9 (5), 526-541. I

Cook, J ., & Wall, T. (1980). New work attitude measures of trust, organizational

commitment and personal need non-fulfillment. Journal of Occupational Psychology 53,
39—52.

Dansereau, F., Graen, G., & Haga, W.J. (1975). A vertical dyad linkage approach to
leadership within formal organizations: A longitudinal investigation of the role making
process. Organizational Behavior and Human Performance, 1 3, 46—78.

*DeGregorio, M., & Fisher, CD. (1988). Providing performance feedback: Reactions to
alternate methods. Journal of Management, I 4 (4), 605-616.

DeShon, R. (2002). A generalizaiblity theory perspective on measurement error
corrections in validity generalization. In K. R. Murphy (Ed.). Validity Generalization: A
critical review. Mahwah, NJ: Erlbaum.

Dienesch, R.M., & Liden, RC, 1986. Leader-member exchange model of leadership: A
critique and further development. Academy of Management Review, I I , 618—634.

"Dipboye, R.L., & de Pontbriand, R. (1981). Correlates of employee reactions to
performance appraisals and appraisal systems. Journal of Applied Psychology, 66 (2),
248-251.

“Dobbins, G.H., Cardy, R.L., & Platz-Vieno (1990). A contingency approach to
appraisal satisfaction: An initial investigation of the joint effects of organizational
variables and appraisal characteristics. Journal of Management, 3, 619-632.

*Dobbins, G.H., Platz, S.J., Houston, J. (1993). Relationship between trust in appraisal
and appraisal effectiveness: A field study. Journal of Business and Psychology, 7 (3),
309-322. Database 3-3-2009.

113

*Dorfman, P.W., Stephan, W.G., & Loveland, J. (1986). Performance appraisal
behaviors: Supervisor perceptions and subordinate reactions. Personnel Psychology, 39,
579-597.

*Dulebohn, J .H., & Ferris. GR. (1999). The role of influence tactics in perceptions of
performance evaluation’s fainress. Academy of Management Journal, 42 (3), 288-303.

*Elicker, JD. (2000). An organizational justice explanation of feedback reactions in
different leader-member exchange relationships. Unpublished Doctoral Dissertation,
University of Akron.

Elicker, J .D., Levy,. P.E., & Hall, R.J. (2006). The role of leader-member exchange in the
performance appraisal process. Journal of Management, 32 (4), 531-551.

Erdogan, B. (2002). Antecedents and consequences of justice perceptions in performance
appraisals. Human Resource Management Review, 12, 555-578.

*Erdogan, B., Kraimer, M.L., & Liden, RC. (2001). Procedural justice as a two-
dimensional construct: An examination in the performance appraisal context. Journal of
Applied Behavioral Science, 3 7, 205-222.

"Evans, E.M., & McShane, S.L. (1988). Employee perceptions of performance appraisal
fairness in two organizations Canadian Journal of Behavioral Science, 20(2), 177-191.

Fahr, J .L., Werbel, J .D., & Bedeian, AG. (1988). An empirical investigation of self-
appraisal-based performance evaluation. Personnel Psychology, 41, 141-156.

F assina, N. E., Jones, D. A., & Uggerslev, K. L. (2008). Relationship clean-up time:
Using meta-analysis and path analysis to clarify relationships among job satisfaction,
perceived fairness, and citizenship behaviors. Journal of Management, 34, 161—188.

F eldman, J .M. (1981). Beyond attribution theory: Cognitive processes in performance
appraisal. Journal of Applied Psychology, 66, 127-148.

*Folger & Konovsky (1989). Effects of procedural and distributive justice on reactions to
pay raise decisions. Academy of Management Journal, 32 (1), 115-130.

Folger, R., Konovsky, M., & Cropanzano, R. (1992). A due process model of
performance appraisal. In B. M. Staw & L. L. Cummings (Eds.), Research in
organizational behavior (Vol. 14, pp. 129-177). Greenwich, CT: JAI Press.

Fox, A. (2009). Curing what ails performance reviews. HR Magazine, 54 (1), 52-56.

“French, J.P., Kay, E., & Meyer, H.H. (1966). Participation and the appraisal system.
Human Relations, 19(3), 3-20.

114

*Gabris, G.T., & Ihrke, D.M. (2000). Improving employee acceptance toward

performance appraisal and merit pay systems: The role of leadership credibility. Review
of Public Personnel Administration, 20, 41 -53.

*Gaby, SH. (2004). Summarizing and measuring participants ’ perceptions related to
performance appraisal effectiveness. Unpublished Doctoral Dissertation, University of
Tennessee, Knoxville.

Gerstner, C.R., & Day, D.V. (1997). Meta-analytic review of leader—member exchange
theory: Correlates and construct issues. Journal of Applied Psychology, 82, 827—844.

*Giles, W.F., Findley, H.M., & Field, HS (1997). Procedural fairness in performance
appraisal: Beyond the review session. Journal of Business and Psychology, 1 I (4), 493-
506.

**Giles, W.F., & Mossholder, K.W. (1990). Employee reactions to contextual and
session components of performance appraisal. Journal of Applied Psychology, 75 (4),
371-377.

Gill, H. (2005). Reactions to injustice: Development and validation of a measure.
Unpublished Doctoral Dissertation, The University of Western Ontario.

Gouldner, AW. (1960). The norm of reciprocity: A preliminary statement. American
Sociological Review, 25, 161-177.

Greenberg, J. (1987). A taxonomy of organizational justice theories. Academy of
Management Review, 12, 9-22.

Greenberg, J. (1990). Organizational justice: Yesterday, today, and tomorrow. Journal of
Management, 16, 399-432.

Greenberg, J. (1993). The social side of fairness: Interpersonal and information classes of
organizational justice. In R. Cropanzano (Ed.), Justice in the Workplace: Approaching
fairness in human resource management (pp. 79-103). Hillsdale, NJ: Erlbaum.

”Greller, M.M. (1975). Subordinate participation and reactions to the appraisal
interview. Journal of Applied Psychology, 60 (5), 544-549.

“Greller, M.M. (1978). The nature of subordinate participation in the appraisal
interview. Academy of Management Journal, 21 (4), 646-658.

Hackman, J .R., & Oldham, GR. (1974). Development of the job diagnostic survey.
Journal of Applied Psychology, 60, 159-170.

115

Hedges, L.V. & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando:
Academic Press.

Hedges, L.V., & Vevea, J .L. (1998). Fixed and random-effects models in meta-analysis.
Psychological Methods, 3, 486-504.

*Hillery, J .M., & Wexley, K.N. (1974). Participation effects in appraisal interviews
conducted in a training session. Journal of Applied Psychology, 59(2), 168-171.

*Homer, M.T., Cook, A.L., & Payne, SC. (2008). When does affect relate to
performance appraisal reactions? The moderating role of situational constraints. Paper
presented at the annual meeting of the Society for Industrial Organizational Psychology,
San Francisco, CA.

Horvath, M., & Andrews, SB. (2007). The role of fairness perceptions and accountability

attributions in predicting reactions to organizational events. Journal of Psychology, 14
(2), 203-222.

House, J. (1981). Work stress and social support. Addison-Wesley, Reading.

*Hubbel, A.P., & Chory-Assad (2005). Motivating factors: Perceptions of justice and
their relationship with managerial and organizational trust. Communication Studies, 56
(1), 47-70.

*Hurd, B., Byme, Z.S., & Gross, MA. (2009). Moderation of communication medium
richness on reactions to task feedback. Paper Presented at the annual conference of the
Society of Industrial and Organizational Psychology, New Orleans, LA.

Ilgen, D.R., Barnes-Farrell, J .L., & McKellin, DB. (1993). Performance appraisal
process research in the 1980’s: What has it contributed to appraisals in use?
Organizational Behavior and Human Decision Processes, 54, 321-368.

*Igoumenopolous, K. (2001). Factors relating to performance appraisal satisfaction and
acceptance of lower-level employees. Unpublished Master’s thesis, San Jose State
University.

Ilgen, D.R., Fisher, C.D., & Taylor, MS. (1979). Consequences of individual feedback
on behavior in organizations. Journal of Applied Psychology, 64, 349-371.

MIlgen, D.R., Peterson, R.B., Martin, B.A., & Boeschen, DA. (1981). Supervisor and
subordinate reactions to performance appraisal sessions. Organizational Behavior and
Human Performance, 28, 311-330.

”Inderrieden, E.J., Allen, R.E., & Keaveny, T.J. (1992). An investigation of the
antecedents and consequences of voluntary self-ratings in a performance appraisal

116

system. Paper presented at the Annual Conference of the Academy of Management, Las
Vegas.

*Indeniden, E.J., Allen, R.E., Keavey, T.J. (2004). Managerial discretion in the use of
self-ratings in an appraisal system: The antecedents and consequences. Journal of
Managerial Issues, 16 (4), 460-482.

”Inderrieden, E.J., Keaveny, T.J., & Allen, RE. (1988). Predictors of employee
satisfaction with the performance appraisal process. Journal of Business and Psychology,
2 (4), 306-310.

*Jawahar, I.M. (2006). An investigation of potential consequences of satisfaction with
appraisal feedback Journal of Leadership and Organizational Studies, 13 (2), 14-28.

*Jawahar, I.M. (2006). Correlates of satisfaction with performance appraisal feedback.
Journal of Labor Research, 2, 213-235.

*Jawahar, I.M. (2007 ). The influences of perceptions of fairness on performance
appraisal reactions. Journal of Labor Research, 28, 735-754.

*Johnson, J .S. (2003). Employees ’ justice perceptions of performance appraisal systems:
Attitudinal, behavioral, and performance consequences. Unpublished Doctoral
Dissertation, Portland State University.

*Kacmar, K.M., Wayne, S.J., & Wright, PM. (1996). Subordinate reactions to the use of
impression management tactics and feedback by the supervisor. Journal of Managerial
Issues, 8(1), 35-53.

*Kamer, B. (2006). Key factors contributing to effectiveness in the performance
appraisals of military cadres. International Military Testing Association Conference.

*Kavanagh, P., Benson, J ., & Brown, M. (2007). Understanding performance appraisal
fairness Asia Pacific Journal of Human Resources, 45 (2), 132 - 150.

Kay, E.. Meyer, H. H., & French, 1. R. P. (1965). Effects of threat in a performance
appraisal interview. Journal of Applied Psychology, 4 9, 31 1-317.

*Keeping, L.M. (2000). The frustration effect and self-ratings: Do explanations and
individual differences matter? Unpublished Doctoral Dissertation, University of Akron.

*Keeping, L.M., & Levy, PE. (2000). Performance appraisal reactions: Measurement,
modeling, and method bias. Journal of Applied Psychology, 85 (5), 708-723.

*Kinicki, A.J., Prussia, G.E., Wu, 3., & Mckee-Ryan, FM. (2004). A covariance

structure analysis of employee’s response to performance feedback. Journal of Applied
Psychology, 89(6), 1057-1069.

117

 

Kleiman, L.S., Biderman, M.D., & Faley, RH. (1987). An examination of employee

perceptions of a subjective performance appraisal system. Journal of Business and
Psychology, 2(2), 112-121.

MKlein, H.J., & Snell, S. (1994). The impact of interview process and context on

performance appraisal interview effectiveness. Journal of Managerial Issues, 6 (2), 160-
175.

Klein, H.J., Snell, S.A., & Wexley, K.N. (1987). Systems model of the performance
appraisal interview process. Industrial Relations, 26 (3), 267—280.

Kluger, A.N. & DeNisi, A. (1996). The effects of feedback interventions on performance:
A historical review, a meta-analysis, and a preliminary feedback intervention theory.
Psychological Bulletin, 119, 254-284.

MKorsgaard, M.A., & Roberson, L. (1995). Procedural justice in performance
evaluation: The role of instrumental and non-instrurnental voice in performance appraisal
discussions. Journal of Management, 21 (4), 657-669.

“Korsgaard, M.A., Robserson, L., & Rymph, RD. (1996). Promoting fairness through
subordinate training: The impact a subordinate communication style of manager ’s
fairness. Paper presented at the 11 Annual Conference of the Society for Industrial and
Organizational Psychology, San Diego.

*Korsgaard, M.A., Roberson, L., & Rymph, RD. (1998). What motivates fairness? The
role of subordinate assertive behavior on manager’s interactional fairness. Journal of
Applied Psychology, 83 (5), 731-744.

"Landy, F.J., Barnes, J .L., & Murphy (1978). Correlates of perceived fairness and
accuracy of evaluation. Journal of Applied Psychology, 63 (6), 751-754.

Landy, F. J ., & Farr, J. L. (1980). Performance rating. Psychological Bulletin, 87, 72-107.
LaRacco, J. M., House, J. R. & French, J. R. P. (1980). Social support, occupational
stress, and health. Journal of Health and Social Behavior, 21, 202-218.

Larson, J .R. (1984). The performance feedback process: A preliminary model.
Organizational Behavior and Human Performance, 33, 42-76.

Lee, J .S.Y., & Akhtar, S. (1996). Determinants of employee willingness to use feedback
for performance improvement: cultural and organizational interpretations. International
Journal of Human Resource Management, 7(4), 879-890.

*Lee, M., & Son, B. (1998). The effects of appraisal review content on employee’s

reactions and performance. International Journal of Human Resource Management, 9
(1), 203-214.

118

Leventhal, GS. (1980). What should be done with equity theory? In: K.J. Gergen, M.S.
Greenberg, & R.H. Willis, (Eds), Social exchange: Advances in theory and research.
Plenum: New York. (pp. 27—55).

*Levy, P.E., Cawley, B.D., & Foti, R. (1998). Reactions to appraisal discrepancies:
Performance ratings and attributions. Journal of Business and Psychology. 12 (4), 43 7-
455.

*Levy, P.E. & Williams, J .R. (1998). The role of perceived system knowledge in
predicting appraisal reactions, job satisfaction, and organizational commitment. Journal
of Organizational Behavior, 1 9, 53-65.

Levy, P.E. & Williams, JR. (2004). The social context of performance appraisal.
Journal of Management, 30(6), 881-905.

Liden, R.C., Sparrow, R.T., & Wayne, SJ. (1997). Leader member exchange theory: The
past, present and the potential future. Research in Personnel and Human Resources
Management, 15, 47-119.

Lind, E.A., & Tyler , TR. (1988). The social psychology of procedural justice. N.Y.:
Plenum.

Lipsey, M.W., & Wilson, DB. (2001). Practical meta analysis. Applied Social Research
Methods Series, vol. 49. Thousand Oaks: Sage.

Locke, E. A., & Latham, G. P. (1990). A theory of goal setting and task performance.
Englewood Cliffs, NJ : Prentice Hall

Locke, E. A., & Schweiger, D. M. (1979). Participation in decision-making: One more
look. In B. M. Staw (Ed.), Research in organizational behavior (Vol. 1) (pp. 265—340).
Greenwich, CT: JAI Press.

*Lundquist, C., Kudisch, J .D., & Fortunato, V. (2004). Influence of perceptions of
fairness on performance appraisal effectiveness. Presented at the Society for Industrial-
Organizational Psychology conference, Chicago, IL.

Masterson, S.S., Lewis, K., Goldman, B.M. & Taylor, MS. (2000). Integrating justice
and social exchange: the differing effects of fair procedures and treatment on work
relationships. Academy of Management Journal, 43, 73 8—748.

McFarlin, D.B, & Sweeney, PD. (1992). Distributive and procedural justice as predictors

of satisfaction with personal and organizational outcomes. Academy of Management
Journal, 35, 626—637.

119

 

*Miller, J. (2001). Self-monitoring and performance appraisal satisfaction: An
exploratory field study. Human Resource Management, 40(4), 321-332.

Moonnan, RH. (1991). Relationship between organizational justice and organizational

citizenship behaviors: Do fairness perceptions influence employee citizenship? Journal of
Applied Psychology, 76, 845-855.

*Mount, MK. (1984). Satisfaction with performance appraisal system and appraisal
discussion. Journal of Occupational Behavior, 5 (4), 271-279.

Murphy, K.R., & Cleveland, J .N. (1991). Performance appraisal: An organizational
perspective. Boston: Allyn & Bacon.

Murphy, K.R., & Cleveland, J .N. (1995). Understanding performance appraisal: Social,
organizational, and goal-based perspectives. Thousand Oaks: Sage.

MNathan, B.R., Mohrman, A.M., & Milliman, J. (1991). Interpersonal relations as a
context for the effects of appraisal interviews on performance and satisfaction: A
longitudinal study. Academy of Management Journal, 34 (2), 352-369.

"Nemeroff, W.F., & Wexley, K.N. (1979). An exploration of the relationships between
performance feedback interview characteristics and interview outcomes as perceived by
managers and subordinates. Journal of Occupational Psychology, 25-34.

Noe, R., Hollenbeck, J .R.H., Gerhart, B., & Wright, P.M. (2000). Human resource
management: Gaining a competitive advantage. McGraw-Hill: Boston, MA.

*Orpen, C. (1995). Employee job performance and relations with superior as moderators
of the effect of appraisal goal setting on employee work attitudes. International Journal
of Career Management, 7 (2), 3-6.

*Pangabean (2001). Impact of perceived justice in performance appraisal on work
attitudes and performance. Unpublished Doctoral Dissertation, Nova Southeastern
University.

*Payne, S.C., Homer, M.T., Boswell, W.R., Wolf, A.N., & Stine-Cheyen, K.J. (2008).
Comparison of online and traditional performance appraisal systems. Paper Presented at
the Society for Industrial-Organizational Psychology conference, San Francisco, CA.

Pettijohn, C.E., Pettijohn, L.S., & d’Amico (2001). Characteristics of performance
appraisals and their impact on sales force satisfaction. Human Resource Development
Quarterly, 12(2), 127-146.

Pettijohn, C.E., Pettijohn, L.S.,Taylor, A.J. & Keillor, B.D. (2000). Adaptive selling and

sales performance: An empirical examination. Journal of Applied Business Research, I 6,
91—1 1 1.

120

 

Pichler, S., Varma, A., & Petty, R. (2008). Rater-ratee relationships. In. A. Vanna, A., P.
Budhwar, & A. DeNisi (Eds.), Performance Management Systems: A global perspective
(pp. 55-66). New York: Routledge.

**Pooyan, A., & Eberhardt, B.J. (1989). Correlates of performance appraisal satisfaction
among supervisory and non-supervisor employees. Journal of Business Research, 19,
21 5-226.

a""‘Prince, J.B. & Lawler, BE. (1986). Does salary discussion hurtthe developmental

performance appraisal? Organizational Behavior and Human Decision Processes, 3 7,
357-3 74.

MRoberson, L., Torkel, S., Korsgaard, A., Klein, D., & Diddams, M. (1993). Self-
appraisal and perceptions of the appraisal discussion: A field experiment. Journal of
Organizational Behavior, 14, 129-142.

Roberts, K.H., & O'Reilly, CA. (1974). Measuring organizational communication.
Journal of Applied Psychology, 59, 321—326.

*Roberts, G.E., & Reed, T. (1996). Performance appraisal participation, goal setting and

feedback: The influence of supervisory style. Review of Public Personnel Administration,
16, 29-59.

Rynes, S.L., Gerhart, B., & Parks, L. (2005). Personnel Psychology: Performance
evaluation and pay for performance. Annual Review of Psychology, 56, 571-600.

*Russell, J .S. & Goode, BL. (1988). An analysis of manager’s reactions to their own
performance appraisal feedback. Journal of Applied Psychology, 73 (1), 63-67.

*Sabeen, Z., Mehboob, S.A., Jinnah, M.A. (2008). Perceived fairness of and satisfaction
with employee performance appraisal and its impact on overall job satisfaction. The
Business Review, 10 (2), 185-191.

Sall, F.E., Downey, R.G. & Lahey, ME. (1980). Rating the ratings: Assessing the
psychometric quality of rating data. Psychological Bulletin, 88 (2), 413-428.

Scandura, T.A., & Schriesheim, GA. (1994). Leader-member exchange and supervisor
career mentoring as complementary constructs in leadership research. Academy of
Management Journal, 3 7, 1588—1602.

*Secunda, MD. (1984). Employee perceptions of performance appraisal systems: Causal

determinants of fairness, accuracy, and acceptability. Unpublished doctoral dissertation,
Old Dominion University.

121

 

**Silvennan, S.B., & Wexley, K.N. (1984). Reaction of employees to performance
appraisal interviews as a function of their participation in rating scale development.
Personnel Psychology, 3 7, 703-710.

* *Simmons, RR. (1986). Organizational effects of several appraisal approaches.
Unpublished Doctoral Dissertation, University of South Florida.

*Steensma, H., & Otto, L. (2000). Perception of performance appraisal by employees and
supervisors: Self-serving bias and procedural justice. Journal of Collective Negotiations,
29(4), 307-319.

Suh, Y. (1992). Instrumental and noninstrumental voice effects on perceptions of
procedural justice in a performance appraisal. Unpublished Doctoral Dissertation, The
University of Nebraska — Lincoln.

*Tang, T.L.P., & Sarsfield-Baldwin, L.J. (1996). Distributive and procedural justice as
related to satisfaction and commitment. Paper Presented at the Annual Meeting of the
Southwestern Psychological Association, Houston, TX.

MTaylor, M.S., Tracy, K.B., Renard, M.K., Hanison, J.K., & Carroll, SJ. (1995). Due
process in performance appraisal: A quasi-experiment in procedural justice.
Administrative Science Quarterly, 40, 495-523.

*Taylor, M.S., Masterson, S.S., Renard, M.K., & Tracy, K. (1998). Manager’s reactions
to procedurally just performance management systems. Academy of Management
Journal, 41 (5), 568-579.

"Taylor, R.L., & Zawacki, RA. (1978). Collaborative goal setting in performance
appraisal: A field experiment. Public Personnel Management, 62, 170.

Tharenou, P. (1995). The impact of a developmental performance appraisal program on
employee perception in an Australian federal agency. Group & Organization
Management, 20, 245-271.

Thibaut, J ., & Walker, L. (1975). Procedural justice: A psychological analysis. Erlbaum,
Hillsdale, NJ.

Thomas, S.L.,& Bretz, RD. (1994). Research and practice in performance appraisal:
Evaluating employee performance in America’s largest companies. SAM Advanced
Management Journal, 59, 28-34.

*Thurston, PW. (2001). C Iarijying the structure of justice using fairness perceptions of

performance appraisal practices. Unpublished Doctoral Dissertation, University at
Albany, State University of New York.

122

 

Tyler, TR. (1989). The psychology of procedural justice: a test of the group values
model. Journal of Personality and Social Psychology, 5 7 (5), 830—838.

*Tziner, A., Joanis, C., & Murphy, KR. (2000). A comparison of three methods of
performance appraisal with regard to goal properties, goal perception, and ratee
satisfaction. Group & Organization Management, 25 (2), 175-190.

*Tziner, A., Kopelman, R.E., & Joanis, C. (1997). Investigation of raters’ and ratees’
reactions to three methods of performance appraisal: BOS, BARS, and GRS. Canadian
Journal of Administrative Sciences, 14 (4), 396-404.

*Tziner, A., Kopelman, R.E., & Livneh, N (1993). Effects of performance appraisal
format on perceived goal characteristics, appraisal process satisfaction, and changes in
rated job performance: A field experiment. The Journal of Psychology, 127(3), 281-291.

Tziner, A., Murphy, K.R., Cleveland, J .N. (2005). Contextual and rater factors affecting
rating behavior. Group and Organization Behavior, 30 (1), 89-98.

US. Department of the Interior. Performance appraisal handbook: A guide for
managers/ supervisors and employees.
http://www.doi.gov/hnn/guidance/3 70dm430hndbk.pdf . Retrieved June 22, 2009.

Vandaveer, V.V. (1982). The dynamics of the performance feedback process in
organizations. (Doctoral Dissertation, University of Houston, 1982). Dissertation
Abstracts International, 42, 4609.

Wayne, S.J., & Ferris, GR. (1990). Influence tactics, affect, and exchange quality in

supervisor-subordinate interactions: a laboratory experiment and field study. Journal of
Applied Psychology, 75, 487-599.

Wayne, S.J., & Liden, RC. (1995). Effects of impression management on performance
ratings: a longitudinal study. Academy of Management Journal, 38, 232-60..

Wexley, K.N., & Klimoski, R. (1984). Performance appraisal: an update. In Rowland,
K.M., Ferris, G.R. (Eds), Research in Personnel and Human Resources Management
(V 01. 2). JAI Press, Greenwich, CT. (pp. 23-79).

*Wexley, K., Singh, J .P., & Yukl, GA. (1973). Subordinate personality as a moderator
of the effects of participation in three types of appraisal interviews. Journal of Applied
Psychology, 58(1), 54-59.

*Williams, J .R., & Levy, PE. (2000). Investigating some neglected criteria: The

influence of organizational level and perceived system knowledge on appraisal reactions.
Journal of Business and Psychology, 14 (3), 501-513.

123

 

 

Wood, R.E., & Marshall, V. (2008). Accuracy and effectiveness in appraisal outcomes:
The influence of self-efficacy, personal factors and organizational variables. Human
Resource Management Journal, 18 (3), 295-313.

*Wright, RP. (2004). Mapping cognitions to better understand attitudinal and behavioral
responses in appraisal research. Journal of Organizational Behavior, 25 , 339-3 74.

*Wuthnow, DP. (1992). Procedural and interactional justice variables in performance
appraisals. Unpublished Doctoral Dissertation, University of Kansas.

Zander, A., & Gyr, J ., (1955). Changing attitudes toward a merit rating system. Personnel
Psychology, 8, (1955).

Note: * Indicates study was included in meta-analysis; ** Indicates study was also
included in Cawley et al. (1998) study.

124

 

 

 

 

I

ll
1

,1
9

Illllll
293 03221

I

3 1

S
W.
R
A
R
m
L
V-
H
S
R
E
w
N
U
E
T
A
T
s
N
A
m
H
m
M