. LIBRARY 2003 Michigan State University This is to certify that the dissertation entitled CHOOSING PERFORMANCE MEASURES FOR INCENTIVE COMPENSATION presented by FREDDY CORONADO has been accepted towards furfillment of the requirements for the PhD. degree in Accounting and Information Systems W/M flame Major Professor’s Signature May 19, 2008 Date MSU is an afi'innative-action, equal-opportunity employer PLACE IN RETURN BOX to remove this checkout from your record. TO AVOID FINES return on or before date due. MAY BE RECALLED with earlier due date if requested. DATE DUE DATE DUE DATE DUE SIOB K:lProj/Acc&Pres/CIRC/DateDue.indd CHOOSING PERFORMANCE MEASURES FOR INCENTIVE COMPENSATION By Freddy Coronado A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Accounting and Information Systems 2008 ABSTRACT CHOOSING PERFORMANCE MEASURES FOR INCENTIVE COMPENSATION By Freddy Coronado Choosing performance measures (PMs) for incentive compensation is frequently performed subjectively by individuals who are not compensation experts (e.g., first-line supervisors, branch managers, small-business owners). Prior research finds that choosing PMs can be a complex decision. In response to this decision complexity, psychology literature indicates that boundedly rational individuals will simplify the choice of PMs by using heuristics which vary in their cognitive demands and can influence decision performance. Consequently, identifying characteristics of decision tasks that influence individuals’ use of heuristics can help to explain individuals’ decision performance when choosing PMs for incentive compensation. Based on contingent-decision-behavior literature in psychology, I identify two task characteristics that are expected to influence which heuristics individuals use to choose PMs, and as a result, explain their decision performance: PM attribute conflict and difference between PM attribute differences. The experiment’s results provide support for the hypothesized two-way interaction between these two task characteristics affecting individuals’ decision performance. Overall, the experimental evidence suggests that individuals who are not compensation experts, while using high-complexity heuristics (i.e., require high cognitive effort), can chose an incorrect PM when PM attribute conflict is present and the difference between PM attribute differences is small. When PM attribute conflict is absent or PM attribute conflict is present and the difference between PM attribute differences is large, however, even these unsophisticated individuals can choose the best PM because the available heuristics, while requiring low cognitive effort, lead them to the same solution as the solution derived from the agency-based model in Feltham and Xie (1994). I also study goal commitment and empathic concern as individual differences expected to explain individuals’ decision in the low-performance condition. The experimental evidence supports the expected effect of goal commitment, but not the expected effect of empathic concern, on decision performance when individuals use these high-complexity heuristics. Individual with high goal commitment are more likely to make the correct choice than individuals with low goal commitment, not because they follow Feltham and Xie’s (1994) optimal analysis, but because they focus more on the PMs’ goal congruence than on the PMs’ noise when making tradeoffs between the conflicting PMs’ attributes. Dedicated to my inspiring wife and best friend, Paula, to my wonderful children, Felipe and Diego, and to my loving and supportive parents, Pedro y Lula. This dissertation is also dedicated to the fi'iendship and memory of my grandfather, Sergio Coronado. iv ACKNOWLEDGEMENTS Though the following dissertation is an individual work, I could never have completed this journey without the help, support, guidance and efforts of a lot of people. I would like to thank my advisor and chair, Michael Shields, for his continuous support in the Ph.D. program. He was always there to talk about my ideas, to proofi'ead and mark up my papers, and to help me think through my problems. I would also like to thank the rest of my dissertation committee Ranjani Krishnan, Joan Lufi, and Gerry McNamara, for their guidance over the years. Their critical - but always insightful - comments and suggestions had a substantial impact on both the clarity and quality of my dissertation. Many other faculty members helped and encouraged me in various ways during my stay at Michigan State University. I am especially grateful to Susan Haka, John J iang, Thomas Linsmeier, Kathy Petroni, and Isabel Wang for their support during difficult times. My very special thanks to the persons whom I owe everything I am today, my wife, Paula, my children, Felipe and Diego, my parents, Pedro and Lula, my sister Anita, and all of my in-laws. My thanks also go out to my late grandfather, Sergio Coronado. Their love and their faith in me is what have shaped me to be the person I am today. Thank you for everything. Finally, I have made many friends along the way. They have helped me, one way or another, in my struggle to complete a Ph.D. Many thanks to Marcela Carvallo, Gaston Etchegoyen, Francisco Herrera, Edward Li, Maria Alejandra Manzan, Christian Mastilak, Fabienne Miller, Roger Stace, Franciso Uribe, Eduardo Vaz, Setsuko Wakao, Alex Woods, and Han Yi. TABLE OF CONTENTS LIST OF TABLES ................................................................................... vii LIST OF FIGURES ................................................................................ viii CHAPTER 1: INTRODUCTION .................................................................. 1 CHAPTER 2: LITERATURE REVIEW AND HYPOTHESIS DEVELOPMENT... ......5 Choosing and Weighting Performance Measures ............................................. 5 Theoretical Economic Model for Performance Measure Choice ......................... 6 Performance Measures’ Attributes ........................................................ 6 Performance Measures’ Attributes and Optimal Performance Measure Choice ............................................................................... 8 Contingent-Decision Behavior and Performance Measure Choice ...................... 11 Determinants of Individuals’ Use of Heuristics ........................................ 12 Available Heuristics ......................................................................... 18 Hypotheses ................................................................................... 19 CHAPTER 3: EXPERIMENTAL METHOD ................................................... 26 Participants ....................................................................................... 26 Experimental Materials and Procedure ....................................................... 26 Variables ......................................................................................... 28 Independent and Control Variables ...................................................... 28 Dependent Variable ........................................................................ 29 CHAPTER 4: RESULTS ........................................................................... 3O Descriptive Statistics .............................................................................. 31 Hypothesis Testing .............................................................................. 37 Supplementary Evidence ....................................................................... 40 Sample Size ................................................................................. 40 Performance Measure Attributes’ Relative Importance .............................. 42 CHAPTER 5: DISCUSSION ..................................................................... 46 APPENDICES ....................................................................................... 50 Appendix A: F eltham and Xie (1994) Model ............................................... 51 Appendix B: Compensatory Heuristics ...................................................... 54 Appendix C: Experimental Materials ........................................................ 57 BIBLIOGRAPHY ................................................................................... 73 vi LIST OF TABLES TABLE 1: PERFORMANCE MEASURES’ ATTRIBUTES AND OPTIMAL SOLUTION .......................................................................................... 10 TABLE 2: PERFORMANCE MEASURES’ ATTRIBUTES AND OPTIMAL SOLUTION-- ATTRIBUTE CONFLICT ABSENT VS. PRESENT ........................ 13 TABLE 3: PERFORMANCE MEASURES’ ATTRIBUTES AND OPTIMAL SOLUTION WHEN PERFORMANCE MEASURE ATTRIBUTE CONFLICT IS PRESENT ............................................................................................ 16 TABLE 4: FACTOR ANALYSIS -- MEASURED INDEPENDENT VARIABLES... ..31 TABLE 5: DESCRIPTIVE STATISTICS -- MEASURED INDEPENDENT VARIABLES ........................................................................................ 33 TABLE 6: PEARSON CORRELATION MATRIX: PARTICIPANTS’ CHARACTERISTICS ............................................................................. 36 TABLE 7: PARTICIPANTS’ DECISION PERFORMANCE AND HYPOTHESIS TESTING RESULTS ............................................................................... 38 TABLE C1: DESCRIPTION OF PERFORMANCE MEASURES’ ATTRIBUTES ....................................................................................... 63 TABLE C2: BELIEFS ABOUT EACH ATTRIBUTE-DECISION RELATION .......................................................................................... 66 TABLE C3: SAMPLE OF PM ATRRIBUTES PROVIDED TO PARTICPANTS ..................................................................................... 67 TABLE C4: POST-EXPERIMENTAL QUESTIONNAIRE ................................. 68 TABLE C5: PM ATTRIBUTES’ VALUES PROVIDED FOR PERFORMANCE MEASURE CHOICE ........................................................ 72 TABLE C6: OPTIMAL SOLUTION ............................................................ 73 vii LIST OF FIGURES FIGURE 1: ASYMMETRICAL EFFECT OF PRECISION AND ACTION CONGRUITY ON PRINCIPAL’S EXPECTED GROSS PAYOFF ........................ 17 FIGURE 2: HYPOTHESIS 1 ..................................................................... 21 FIGURE 3: MEDIATOR ROLE OF PERFORMANCE MEASURE ATTRIBUTES’ RELATIVE IMPORTANCE WHEN PERFORMANCE MEASURE ATTRIBUTE CONFLICT IS PRESENT AND THE DIFFERENCE BETWEEN PERFORMANCE MEASURE ATTRIBUTE DIFFERENCES IS SMALL ....................................... 44 viii CHAPTER 1: INTRODUCTION Agency theory predicts that the choice of performance measures (PMs) for incentive compensation depends on how informative the PMs are about agents’ actions and how congruent the PMs are with the principal’s welfare (Banker and Datar 1989; Feltham and Xie 1994; Datar et al. 2001). However, archival studies that test associations between PM choices and proxies for informativeness and congruity have reported mixed results (Bushman et al. 1995; Ittner et al. 1997; Ittner et al. 2003). The purpose of my dissertation is to provide a partial explanation for the mixed archival results by developing and experimentally testing psychology theory-based predictions about how individuals’ choice of PMs for incentive compensation can diverge from agency-theory based predictions of the choice of PMs depending on the PMs’ attributes. Empirical research shows that the choice of a PM can be a complex decision (Davila and Sirnons 1997; Ittner et al. 2003), which is often subjectively made by individuals who are not compensation experts, such as first-line supervisors, branch managers, or small-business owners (Baker et al. 1994; Ittner and Larcker 1998; Banker et al. 2000; Ittner et al. 2003; Gibbs et al. 2004). When a choice task is complex (e. g., high demand of cognitive resources) or when knowledge is limited, boundedly rational individuals use heuristics which, while simplifying their decision process, can sometimes decrease their decision performance (Payne et al. 1993; Gigerenzer and Todd 1999). Psychology literature suggests that individuals’ use of heuristics is highly contingent on the task characteristics (Einhom and Hogarth 1981; Payne et al. 1993). Consequently, identifying characteristics of decision tasks that influence individuals’ use of heuristics can help to explain their decision performance when choosing PMs for incentive compensation. Based on contingent-decision-behavior literature in psychology, I identify two task characteristics that are expected to explain individuals’ use of heuristics, and as a result, their decision performance. The PM choice requires individuals to process PMs’ attributes (informativeness and congruity), therefore, the task characteristics of interest are the attributes of the PMs being considered. Conflict between PMs’ attributes is expected to influence individuals’ use of decision heuristics. For example, suppose a fum is designing incentive compensation for a plant manager. Using market value of the firm as a PM in the plant manager’s incentive compensation captures the principal’s expected gross payoff more directly than does a measure of manufacturing cost. However, while market value is a more congruent (or relevant) measure, it is less controllable by the plant manager (i.e., less precise (or more noisy)) compared to manufacturing cost. Individuals’ use of heuristics is also expected to be influenced by the difference between PM attribute differences. Under the agency- based optimal incentive compensation, in order for the more precise PM (e.g., manufacturing cost) to be preferred over a more congruent PM (e.g., market value), the difference in precision must be of a much larger magnitude than is the difference in congruity. This is because the economic effect of congruity and precision are asymmetrical (i.e., changes in congruity usually have a larger effect on a principal’s expected gross payoff than do changes in precision).1 1 Based on Felthman and Xie (1994), mathematical examples of this asymmetry are presented in the literature review section. This dissertation develops psychology theory-based predictions and provides experimental evidence on the effect of these two task characteristic on individuals’ decision performance. When PM attribute conflict is absent or when PM attribute conflict is present and the differences between PM attribute differences is large, even individuals who are not compensation expert are likely to use heuristics which, while requiring low cognitive effort (e. g., choosing the dominant PM or the PM that has a decisive advantage) lead them to the same PM choice as the solution derived from the agency-based model in Feltham and Xie (1994). However, when PM attribute conflict is present and the difference between PM attribute differences is small, the heuristics available require high cognitive effort but do not necessarily lead individuals to an optimal PM choice, and thus individuals’ decision performance decreases. Specifically, high-complexity heuristics require individuals to make judgments about the PM attributes’ relative importance in order to tradeoff PMs’ attributes (i.e., compensatory heuristics) and thus, individual differences are expected to explain individuals’ decision performance. This dissertation identifies two individual differences that are expected to influence individuals’ judgment about PM attributes’ relative importance when using compensatory heuristics. Specifically, individuals’ goal commitment is expected to increase the importance of congruity to their PM choice and, as a result, it is expected to increase the likelihood of individuals choosing the most congruent PM. Individuals’ empathic concern is expected to increase the importance of precision to their PM choice, and consequently it is expected to increase the likelihood of individuals choosing the most precise PM. The experimental evidence supports the predicted effect of goal commitment, but not the predicted effect of empathic concern, on decision performance. This dissertation is intended to make three contributions to the accounting literature. First, it identifies heuristics that individuals are likely to use when choosing PMs for incentive compensation. Second, it develops predictions and provides experimental evidence on two task characteristics that influence individuals’ use of heuristics, and as a result, explain their decision performance. Third, it provides evidence that individual differences can affect individuals’ PM choice performance when tradeoffs between PMs’ congruity and precision are required. The remainder of this dissertation is structured as follows. Chapter 2 reviews literature from accounting, economics, and psychology as the basis to develop three hypotheses. Chapter 3 describes the experimental method and Chapter 4 presets results of hypothesis testing. Finally, Chapter 5 provides a discussion of the results, limitations, and implications of this dissertation. CHAPTER 2: LITERATURE REVIEW AND HYPOTHESIS DEVELOPMENT This section has three parts which review accounting and psychology literature as the basis for developing three hypotheses about individuals’ choice of PMs for incentive compensation. In the first part, choosing and weighting PMs are represented as two separate decisions. The second part introduces Feltham and Xie’s (1994) agency-based model to provide an optimal solution to the choice of PMs for incentive compensation. In the third part, based on psychology literature, three hypotheses are developed to explain when and how individuals’ PM choices are expected to deviate from economically optimal decisions. Choosing and Weighting Performance Measures Agency models represent choosing and weighting PMs as a single decision in which, after analyzing the properties of the available PMs, all informative PMs are given nonzero weight, i.e., they are chosen (Hohnstrom 1979; Feltham and Xie 1994). In practice, however, this decision can be cognitively too complex for individuals. Thus, in order to reduce this complexity, the decision is likely to be split into two decisions: first, choosing a smaller set of PMs from a set of available PMs (e.g., a few PMs for each dimension of a Balanced Scorecard) and then weighting the chosen PMs. Practice and research literature provides evidence consistent with the single decision being split into two decisions. For example, Lyons and Ben-Ora (2002), compensation experts at what was Arthur Andersen’s Human Capital Practices, explicitly propose first selecting PMs and then deciding how much they should be weighted. Ittner et al. (2003) provide archival evidence that choosing and weighting PMs are temporally separated. Banker et al. (2000) and B01 and Moers (2006) provide anecdotal evidence on how these decisions are separated and made by different individuals. Based on this evidence, to understand individuals’ decision performance when designing incentive compensation, we must study how individuals make each decision separately. This dissertation focuses on how and how well individuals make the first decision when designing incentive compensation: the choice of PMs. Theoretical Economic Model for Performance Measure Choice Feltham and Xie’s (1994) model is a mathematical representation of a multiple- action multiple-PM agency that explains and predicts the optimal weights on PMs. In this section, Feltham and Xie’s (1994) model is used to derive the optimal choice of PMs. I first introduce the PMs’ attributes to be considered in designing incentive compensation and then explain how these attributes should be combined to make an economically optimal choice. Performance Measures ’ Attributes Agency-based analytical research indicates that any PM that provides incremental information about actions that a principal wants to motivate should be used for incentive compensation (Holmstrom 1979). In F eltham and Xie’s (1994) model, PM informativeness and, as a result, its optimal weight depends on two PM attributes: congruity and precision. PM congruity is the degree of congruence between the impact of an agent’s (e. g., manager) action on the PM and on the principal’s expected gross payoff. PM congruity can be decomposed into PM sensitivity and action congruity.2 While PM sensitivity is the 2 F eltham and Xie (1994) do not explicitly decompose PM congruity, but their congruity formula has two parts which I label action congruity and PM sensitivity. expected effect of an agent’s action on the PM, action congruity is the expected effect of an agent’s action on the principal’s expected gross payoff (Banker and Datar 1989; Feltham and Xie 1994; Datar et al. 2001). To illustrate, using Feltham and Xie’s (1994) model (Appendix A provides a description of Feltham and Xie’s [1994] model),3 suppose there are two actions (a1 and a2) that an agent can implement. If a; and a; are not publicly observable, then incentive compensation is assumed to be based on publicly reported PMs. If there are two PMs (PM1 and PM2), then they can be represented as the following linear functions of the agent’s actions: PM1= P1131 + #1232 + 81 (1) PMz 2 #2131 + “2232 + 82 (2) where uij is the sensitivity of a PM, (i.e., the change in PM, for an incremental change in the agent’s actions a, and aj).4 Suppose the principal’s expected gross payoff (X) can also be represented as a linear function of the two actions: X = b1a1+ bzaz + 8,, (3) In Equation (3), b, is the action congruity of a PM (i.e., the change in the principal’s expected gross payoff for an incremental change in the agent’s action a). PM congruity can be expressed as the alignment between p. (PM sensitivity) and b (PM action congruity). All else being equal (e. g., cost of agent’s actions, PMs’ sensitivities, PMs’ precision), PMI is more congruent than PMz when PM] has higher 3 Appendix A provides a full description of the model and its optimal solution. In this section, the equations are only introduced to facilitate understanding of the PMs’ attributes. 4 Note that PM sensitivity is the slope of the relation between an agent’s actions and the PM, not the correlation between these variables. Consequently, PM sensitivity does not capture the strength of the relation between these variables. action congruity than PM; (i.e., when the use of PMI motivates the agent to implement actions that increases the principal’s expected gross payoff more than does the use of PM;). Holding action congruity constant, if PM1 has higher sensitivity than PM;, then PMI is less congruent than PM; because a change of one unit of PM] is associated with a lower effect on the principal’s expected gross payoff than does a change of one unit of 1)sz PM precision refers to the lack of noise in the PM (i.e., the lack of variation in the PM due to factors the agent can not control). In Feltham and Xie (1994), PM’s noise is captured by the variance of the error term (8;) in Equations (1) and (2). Hence, PM; is more precise than PM; when the variance of e; is lower than the variance of 3;. Because action congruity and precision are more directly related to characteristics of accounting information (i.e., relevance and reliability, respectively), I hold sensitivity constant and examine the decision-performance effects of differences in action congruity and precision.6 Performance Measures ’ Attributes and Optimal Performance Measure Choice Feltham and Xie’s (1994) model assumes that individuals are able to identify the relevant PM attributes and combine them in a complex system of linear equations to 5 If PM1 has higher sensitivity than PM;, PM, presents a higher expected response to the agent’s actions than PM2. However, since action congruity is held constant, agent’s actions that are intended to affect PM1 have the same impact on the principal’s expected gross payoff than agent’s action intended to affect PMz. Therefore, the higher expected response in PM1, relative to PM;, is not associated with a higher expected effect on principal’s gross payoff. 6 Action congruity is a more important attribute than sensitivity when choosing PMs for incentive compensation. In general, when the PMs are affected by the agent’s action (i.e., the PMs are sensitive), it is more important to choose a PM that motivates the agent to perform activities that have a larger effect on the principal’s expected gross payoff than to choose a PM that motivates an agent to perform activities that have a larger effect on the PM. determine the optimal weights (i.e., monetary incentive per unit of PM) on PMs (Equation (4) in Appendix A). By identifying the optimal weights on each PM, the model determines the principal’s expected gross payoff. Thus, in order for individuals to choose the optimal PMs, they should choose the PMs that maximize the principal’s expected gross payoff. For example, suppose an individual needs to choose a PM from a set of two available PMs, and relies on information about the PMs’ attributes to make his or her decision (Panel A of Table 1). To simplify the decision, assume that the agent’s absolute risk aversion is equal to one7 and that both the agent’s actions and the uncontrollable factors that affect one PM do not affect the other PM.8 Following Feltham and Xie’s (1994) model, Panel B of Table 1 presents the optimal solution. If PM] is chosen and optimally weighted ($0.50), then the principal’s expected gross payoff will be $1,800,000. Instead, if PM; is chosen and optimally weighted ($0.56), then the principal’s expected gross payoff will be $2,500,000. Consequently, an individual should choose PM; because it results in the highest expected gross payoff to the principal. 7 Feltham and Xie (1994) represented the agent’s risk preference as a negative exponential utility fimction u(w)= -e"”’, where r is the agent’s absolute risk aversion and w is the agent’s compensation minus personal cost. The coefficient of absolute risk aversion (Pratt 1964) is positive if the individual is averse to risk, zero if the individuals is risk neutral, and negative if the individual prefers risk. Consistent with a risk-averse agent and the numerical example provide in Feltham and Xie (1994), I assume r = l. The r coefficient is defined as r = -u" (w)/u' (w) and can be interpreted as a measure of the curvature of the agent’s utility function (i.e., a measure of the amount of wealth an individual is willing to risk losing as a function of changes in wealth). For example, when r = 1 (high risk aversion), an agent will be indifferent between obtaining 0.7 dollars at no risk or a ticket for a lottery in which he or she gains 10 dollars with probability 0.5 or 0 dollars with probability 0.5. In the case of an agent with r = 0.2 (low risk aversion), he or she will need a payment of 2.80 dollars at no risk to be indifferent to the same lottery ticket. Empirical studies of individuals’ risk attitudes have provided evidence that individuals’ absolute risk aversion coefficients are positive but lower than one (Gertner 1993; Metrick 1995; Brunello 2002; Kirkwood 2004; Cohen and Einav 2007). 8 Consequently, the optimal weight on one PM does not depend on the other PM’s characteristics. TABLE 1 PERFORMANCE MEASURES’ ATTRIBUTES AND OPTIMAL SOLUTION PANEL A: Performance Measures’ Attributes PMs’ Attributes 1 PM1 PM; Action Congruity 2 $2,000 $2,500 Precision 3 $1,200 $1,300 Sensitivity 4 $3,600 $3,600 PANEL B: Optimal Solution Optimal Solution 5’ 6 PM1 PM; Weights $0.50 $0.56 Principal’s Expected Gross Payoff $1,800,000 $2,500,000 10 TABLE 1 (continued) NOTES TO TABLE 1: 1 I generated the PM attributes’ values for expositional convenience as opposed to reference to a specific setting in which these values are in effect. 2 Expected effect on a firm’s economic (market) value when the agent implements one unit of action that is intended to affect PM;. 3 Standard deviation of PM, due to factors the agent can not control. Although Feltham and Xie (1994) use variance as a measure of precision, consistent with Krishnan et al. (2005), I use standard deviation. While both measures provide economically the same information it may be easier for individuals to understand values in dollars than in dollars squared. The use of standard deviation also helps to experimentally control for differences in the scale among PMs’ attributes. 4 Expected effect on PMi when the agent implements one unit of action that is intended to affect PM;. 5 In order to simplify the choice, assume that both the agent’s actions and the uncontrollable factors that affect a PM do not affect the other PM. 6 The optimal solution assumes a agent’s absolute risk aversion, r, is equal to one. Contingent-Decision Behavior and Performance Measure Choice Although agency-based models can be used to guide the choice of PMs, boundedly rational individuals are not likely to use the complex systems of linear equations proposed by Feltham and Xie’s (1994) model to subjectively combine PMs’ attributes when choosing PMs. Psychology literature indicates that the complexity of a decision can exceed the cognitive capacity of boundedly rational individuals (Payne at al. 1993; Bonner 1994). Contingent-decision-behavior literature provides evidence that 11 when making decisions, individuals use heuristics to reduce their cognitive effort (Payne et al. 1993). Individuals have available a set of heuristics for making decisions, with each heuristic having benefits and costs (e.g., expected decision performance and cognitive effort), and individuals use these heuristics contingent on task characteristics (Payne et al. 1993). In the case of the choice between two PMs for incentive compensation, individuals are expected to combine PM attributes’ values to make a decision (e. g., numerical values presented in Panel A of Table 1). Consequently, they are expected to use heuristics contingent on the attributes’ values for the PMs being considered. The subjective complexity of the choice of a PM and decision performance in choosing a PM depend on the PMs’ attributes (e. g., precision and action congruity) and the heuristic used to make the choice. In this section, two task characteristics, which represent PM attributes’ values, are identified as influencing individuals’ use of heuristics: PM attribute conflict and difference between PM attribute differences. Then, the set of available heuristics is identified. Finally, based on contingent-decision—behavior literature, three hypotheses are developed to explain individuals’ decision performance. Determinants of Individuals ’ Use of Heuristics Performance Measure Attribute Conflict PMs usually differ on more than one attribute and these differences can result in conflicts between the attributes and thus cognitive conflict in choosing a PM. PM attribute conflict occurs when one PM is superior on one attribute and another PM is superior on another attribute. In the case of two PMs that differ only with respect to 12 action congruity and precision (i.e., sensitivity is held constant), PM attribute conflict is present when one of the two PMS is superior on action congruity and the other PM is superior on precision. Table 2 presents two examples of a two-PM setting with their optimal solutions as modeled by F eltharn and Xie (1994). TABLE 2 PERFORMANCE MEASURES’ ATTRIBUTES AND OPTIMAL SOLUTION: ATTRIBUTE CONFLICT ABSENT VS. PRESENT PANEL A: PM Attribute Conflict Absent PMs’ Attributes PM; PM; Optimal Solution PM; PM; Action Congruity $2,500 $2,000 Weights $0.63 $0.44 Precision $1,200 $1 ,800 Principal’s Expected Gross Payoff when $2,812,500 $1,600,000 Sensitivity $3,600 $3,600 Only PM; is Used PANEL B: PM Attribute Conflict Present PMS’ Attributes PM; PM; Optimal Solution PM; PM; Action Congruity $2,000 $2,500 Weights $0.50 $0.56 Precision $1,200 $1,800 Principal’s Expected Gross Par/91f when $1,300,000 $2,500,000 Sensitivity $3,600 $3,600 Only PMi '5 Used 13 In Panel A, PM attribute conflict is absent because the values of PM action congruity and PM precision indicate that PM; is superior on both attributes and thus is economically preferred to PM;. In Panel B, PM attribute conflict is present because PM; has higher PM precision and PM; has higher PM action congruity. As indicated by the optimal solution, perfectly rational individuals would choose PM; for the example in Panel B. While PM attribute conflict should not affect the decision performance of perfectly rational individuals, it is expected to influence boundedly rational individuals’ use of heuristics and thus their decision performance (Tversky et al. 1988; Bonner 1994). Diflerence Between Performance Measure Attribute Differences The economic effects of PM action congruity and PM precision on the principal’s expected gross payoff are asymmetrical, in ways that also can affect individuals’ use of heuristics. Table 3 presents two examples of PM attribute conflict and their optimal solutions. In Panel A, the agency-based optimal solution supports the choice of the PM; because it has higher action congruity. The superiority of PM; on action congruity (difference in action congruity between PM; and PM; is $500) is enough to compensate for the superiority of PM; on precision (difference in precision between PM; and PM; is $600).9 In Panel B, in contrast, the more precise PM is the economically preferred PM: PM; has a large superiority on precision (difference in precision is $2,400) which compensates for the superiority of PM; on action congruity (difference in action congruity is $500). This is because a unit of improvement in congruity usually has a 9 As explained in Table 1 (first example), precision has been described as the standard deviation of PM; due to factors the agent can not control. Thus, PMS’ attributes values are provided in a common scale (i.e., dollars). l4 larger effect on a principal’s expected gross payoff than does a unit of improvement in precision. Figure 1 presents the principal’s expected gross payoff as a function of the change on action congruity and precision respectively. The example assumes the following initial values for a PM’S attributes: (1) action congruity = $2,500; (2) precision = $3,600; and (3) sensitivity = $3,600. The Slope of each line represents the impact on the principal’s expected gross payoff of a unit of improvement of each PM’S attribute. The contrast between slopes indicates that the effect on the principal’s expected gross payoff of a unit of improvement in action congruity is almost three times larger than the effect of a unit of improvement in precision.10 10 The slope contrast is not qualitatively different when the other PMS in Tables 1, 2, and 3 were used as the baseline. In addition, the asymmetric effects of PM action congruity and PM precision on the principal’s expected gross payoff is valid for a wide range of absolute risk aversion values. For example, when r = 2, the effect on the principal’s expected gross payoff of a unit of improvement in action congruity is more than two times larger than the effect of a unit of improvement in precision. As r get closer to zero (i.e., an individual’s absolute risk aversion decreases), the asymmetric impact of action congruity and precision on the principal’s expected gross payoff increases (i.e., the lower the agent’s absolute risk aversion, the lower the cost of motivating him or her by using a low-precision PM and thus, the lower the impact of changes in PM precision on the principal’s expected gross payoff relative to changes on action congruity). For example, when r = 0.5, the effect on the principal’s expected gross payoff of a unit of increase in action congruity is almost five times larger than is the effect of a unit of increase in precision. 15 TABLE 3 PERFORMANCE MEASURES’ ATTRIBUTES AND OPTIMAL SOLUTION WHEN PERFORMANCE MEASURE ATTRIBUTE CONFLICT IS PRESENT PANEL A: Small Difference Between the Difference for Precision and the Difference for Action Congruity PMS, PM PM Difference Optimal Solution PM PM Attributes ' 2 l 2 Action . Congruity $2,000 $2,500 $500 Weights $0.50 $0.56 Precision $1,200 $1,800 $600 Principal’s Expected Gross Payoff when $1,800,000 $2,500,000 Sensitivity $3,600 $3,600 $0 Only PM; is Used Difference Between PM $100 Attribute Differences PANEL B: Large Difference Between the Difference for Precision and the Difference for Action Congruity PMS, PM PM Difference Optimal Solution PM PM Attributes 1 2 l 2 Action . Congruity $2,000 $2,500 $500 Weights $0.50 $0.35 Precision $1,200 $3,600 $2,400 Principal’s Expected Gross Payoff when $1,800,000 $1,562,500 Sensitivity $3,600 $3,600 $0 Only PM.- is Used Difference Between PM $1,900 Attribute Differences l6 FIGURE 1 ASYMMETRICAL EFFECT OF PRECISION AND ACTION CONGRUITY ON PRINCIPAL’S EXPECTED GROSS PAYOFF I Principal’s Expected A Gross Payoff PM Attributes — PM Action $2,500,000 — Congruity $2,000,000 1 "" PM Precision $1,500,000 - $1,000,000 a $500,000 — $0 . T i . , e $0 $100 $200 $300 $400 $500 Change in a PM’S Attribute NOTE TO FIGURE 1: 1 This graph presents changes in the principal’s expected gross payoff due to a unit of change in a PM’S attribute. The example assumes the following initial values for a PM’S attributes: (1) action congruity = $2,500; (2) precision = $3,600; and (3) sensitivity = $3,600. 17 Available Heuristics Three heuristics are expected to be used by individuals when choosing a PM from a set of two available PMS.ll Although these heuristics represent simplifications of the choice of a PM as modeled by Feltham and Xie (1994), they also vary in terms of their complexity (e.g., cognitive demands required to process the PMS’ attributes). AS a first step in choosing a PM, individuals look for a dominant PM (Tversky et al. 1988). If one PM measure is superior on both action congruity and precision, then this measure is chosen. This is a relatively low-complexity heuristic, because individuals only need to know whether a higher or lower PM attribute is preferred but they are not required to make a judgment about the PM attributes’ relative importance. However, this heuristic can only be used when PM attribute conflict is absent. If no dominant PM is present due to PM attribute conflict, then individuals may examine whether a PM has a decisive advantage -- that is, whether the difference in one attribute far outweighs the difference in the other attribute for most plausible values of the PM attributes’ relative importance (Tversky et al. 1988). Consequently, this is also a relatively low-complexity heuristic because individuals have to compare the magnitudes of the PM attribute differences, but they do not necessarily require making assessments of the PM attributes’ relative importance. 11 Although the contingent-deciSion-behavior literature identifies more heuristics, I present the heuristics most likely to be used in this decision. Some heuristics, such as lexicographic (e.g., choose the PM that is superior on the most important attribute), are more likely to be used when there are either a larger number of alternatives PMS or a large number of PM attributes. Other heuristics either are simplifications of the heuristics presented in this section or require that individuals develop acceptable standards for at least one attribute (e.g., sets a cutoff value for an attribute; Payne et al. 1993). Non-compensation-expert individuals though are unlikely to develop these standards due to their lack of knowledge and/or expertise. 18 If neither PM is dominant nor has a decisive advantage, then individuals are likely to use a compensatory heuristic. These decision strategies require individuals to make judgments about the PM attributes’ relative importance in order to resolve the conflict by making tradeoffs between PMS’ attributes (Appendix B provides a mathematical representation of these heuristics).12 Research in psychology provides evidence that making such tradeoffs increases subjective decision complexity (Shepard 1964; Payne et al. 1993; Bonner 1994; Shah and Oppenheimer 2008). Consequently, although compensatory heuristics are simplifications of the decision as modeled by Feltham and Xie (1994), they are more complex than the dominant and decisive-advantage heuristics because they require individuals to make judgments about the PM attributes’ relative importance. Hypotheses Three hypotheses are developed in this section. The first hypothesis provides theoretical analysis on the expected frequency of correct decisions resulting from the interaction between PM attribute conflict and difference between PM attribute differences. The next two hypotheses provide theoretical analysis on the role of individual differences when individuals are expected to use compensatory heuristics. Based on contingent-deciSion-behavior literature, I predict a two-way interaction between PM attribute conflict and difference between PM attribute differences on individuals’ decision performance. Figure 2 presents the form of the expected interaction and predictions (Panels A and B respectively). When PM attribute conflict is absent (1 [2 Previous research indicates that compensatory heuristics provide a very good approximation to ratings of evaluative alternatives under the presence of attribute conflict, especially in situations where alternatives are described by only a two or three attributes (e.g., Fisher 1976). 19 and 3 in Figure 2), individuals are expected to use the dominant heuristic which will lead them to choose the PM that is superior on both attributes, thus individuals are expected to make optimal PM choices. Consequently, the frequency of correct decisions is expected to be high and not affected by the difference between PM attribute differences (1 = 3). When PM attribute conflict is present, however, the difference between PM attribute differences is expected to affect individuals’ decision performance. When this difference is large (2 in Figure 2), individuals are likely to use the decisive-advantage heuristic and make correct choices (e.g., choose the more precise PM in panel B of Table 3). AS a result, the frequency of correct decisions Should not differ from the conflict-absent conditions (1 = 2, 2 = 3). When this difference is small (4 in Figure 2), however, individuals are expected to use compensatory heuristics and thus their PM choice is expected to be a ftmction of their judgment about the PM attributes’ relative importance. In this condition, due to individual differences, some individuals are expected to make correct decisions but others are expected to make incorrect decisions; as a result, the frequency of correct decisions is expected to be lower than in the other three conditions (1, 2, 3 > 4). H1: Frequency of correct decisions is a two-way ordinal interactive function of PM attribute conflict and difference between PM attribute differences. 20 FIGURE 2 HYPOTHESIS l PANEL A: Form of the Expected Interaction Between Performance Measure Attribute Conflict and Difference Between Performance Measure Attribute Differences on Individuals’ Decision Performance Frequency of Correct Decisions Difference Between PM Attribute Differences — Large ' ' ' ‘ Small (1) (2) (3) (4) 4 . Absent Present Pgrformance Measure Attribute Conflict PANEL B: Summary of Predictions 1) 1=3 2) 1=2 3) 2:3 4) 1>4 n 2>4 6) 3>4 21 Individual differences are only expected to affect individuals’ decision performance when they use compensatory heuristics (4 in Figure 2). If Hypothesis 1 is supported, then the test for no effect of individual differences on individuals’ decision performance in the other three conditions (1, 2 and 3 in Figure 2) is trivial due to no (or no difference in) incorrect choices. Consequently, the following two hypotheses address the role of individual differences when individuals face PM attribute conflict and the difference between PM attribute differences is small. Individual differences are assumed to affect individuals’ judgments about the PM attributes’ relative importance. Specifically, individuals’ goal commitment is expected to increase the importance of action congruity relative to precision, and their empathic concern is expected to increase the importance of precision relative to action congruity. As a result, individual differences are expected to explain the decision performance of individuals using compensatory heuristics. Psychology theory suggests that a task goal is expected to affect individuals’ behavior through the psychological process of directing attention and effort to their decision (Locke and Latham 1990). The effect of the task goal on effort, however, is not likely to lead boundedly rational individuals to solve the system of equations described by Feltham and Xie (1994) because is too cognitively complex. Task goal is expected to affect individuals’ attention to PMS’ attributes and their judgment about the PM attributes’ relative importance. This judgment is not expected to be free of bias and, individuals are expected to simplify the agency problem by focusing on (i.e., considering relatively more important) the PM attribute that is more directly related to the task goal. The following hypotheses explain how individuals’ goal commitment (HZ) and empathic 22 concern (H3) are expected to affect this judgment and consequently, individuals’ decision performance. Goal Commitment and Individuals ’ Decision Performance Assume that individuals designing incentive compensation have the specific goal of choosing the PM that maximizes their firm’s economic value (e. g., principal’s expected gross payoff). The effect of the firm’s goal on individuals’ choices is expected to depend on their commitment to the firm’s goal (i.e., the extent to which individuals try to maximize their firms’ goal; Locke and Latham 1990). I argue that individuals who are more committed to the firm’s goal will pay more attention to action congruity than to precision because action congruity seems to be more directly related to the firm’s goal than precision. Action congruity is a direct measure of how congruent a PM is with the goal of maximizing the firm’s economic value. In contrast, the link between PM precision and the economic value of the firm is expected to be indirect because it requires individuals to think about how uncontrollable factors affect a risk-averse agent, how the firm Should pay this agent for the risk imposed by the incentive compensation on him or her, and how the agent will act based on this risk-adjusted incentive. In summary, goal commitment is expected to positively affect the importance of action congruity relative to precision on individuals’ PM choice. AS a result, individuals’ goal commitment is expected to be positively associated with the choice of the most action congruent PM and thus positively associated with individuals’ decision performance. H2: When PM attribute conflict is present and the difference between PM attribute differences is small, individuals’ goal commitment is expected to be positively associated with their decision performance. 23 Empathic Concern and Individuals ’ Decision Performance The choice of PM is not free from a social context. Individuals need to understand the agent’s strategy to make correct decisions in an agency context (first-order reasoning in Wilks and Zimbelman 2004). Consequently, they are expected to make attributions about the mental states (desires, beliefs, intentions) of the agent by playing the agent’s role and sharing the agent’s feelings and emotions (i.e., by empathizing with the agent). Literature in social psychology and social neuroscience suggests that empathy is a major determinant of voluntary behavior intended to benefit others (i.e., prosocial behavior in Eisenberg and Miller 1987; other-regarding behavior in Singer and Fehr 2005). This process is automatic and individuals are likely to represent the goals of others in terms of their own goals without even being aware of it (Singer and Fehr 2005). Research indicates that there are individual differences in empathy (Singer et al. 2004). Therefore, individuals with high-empathic concern are more likely to have prosocial behavior than are individuals with low-empathic concern (Singer and Fehr 2005). I argue that individuals with high-empathic concern will pay more attention to PM precision than to action congruity because precision seems to be more directly related to the agent’s goal than action congruity (prosocial behavior). PM precision directly captures the effect of uncontrollable factors on the risk-averse agent’s compensation. The link between action congruity and agent’s goal is expected to be indirect because it requires individuals to think about how much the firm’s economic value will increase and how much the firms is expected to compensate the agent for a unit of increases on the PM (i.e., the PM weight) considering the risk imposed by the PM precision. In summary, empathic concern is expected to positively affect the importance of PM precision relative 24 to action congruity on individuals’ PM choices. AS a result, individuals’ empathic concern is expected to be positively associated to their choice of the most precise PM and thus negatively associated to their expected decision performance. H3: When PM attribute conflict is present and the difference between PM attribute differences is small, individuals’ empathic concern is expected to be negatively associated with their decision performance. 25 CHAPTER II]: EXPERIMENTAL METHOD Participants Seventy-Six MS Accounting students volunteered to participate in the experiment.13 The participants were paid performance-contingent compensation, as described below. Experimental Materials and Procedure Participants received introductory materials explaining the task in terms consistent with Feltham and Xie’S (1994) model (e. g., multiple-action, multiple-PMS, risk-averse and effort-averse managers, risk-neutral firm owners, one-period decision setting). Appendix C presents the content of the experimental materials. Participants assumed the role of a manager at the headquarters of a large firm, and their job was to choose a PM that will be used for incentive compensation to motivate a manager. The objective of the owner of the firm, which their choice should support, was to maximize the economic (market) value of the firm, net of the cost of the manager’s total compensation. The manager had a one-year non-renewable employment contract to implement a performance-improvement program that would maximize the economic (market) value of the firm. Participants were told that for each available PM, they will receive information on its three attributes that would help them in making their PM choice. In order to ensure that participants understood the setting, definitions and examples were provided (Table C1 of Appendix C). ‘3 In a Similar task, decision performance in Krishnan et al. (2005) did not differ significantly between MBA and MS students. Their findings provide support for the use of only one type of student in this study. 26 Participants were asked to choose one of two available PMS and their perforrnance-contingent compensation was explained. “You will receive fixed pay of $10 for completing this accounting Simulation, regardless of your performance in choosing a performance measure. You will also receive variable pay up to $10 in addition, depending on how well you choose a performance measure. That is, the closer your choice is to the choice that would maximize the expected economic (market) value of your firm, the higher your total pay will be, up to a maximum of $20 (= $10 +$10)”. ’4 Before participants received information on the PM attributes’ values to make their choices, and in order to capture their beliefs about each attribute-decision relation, they were asked to provide their beliefs about how each attribute Should directionally affect their decision (Table C2 of Appendix C). Next, participants were asked to return their answers to the administrator and told they could keep the introductory materials until the end of the Simulation. Participants then received information about the attributes’ values for the two available PMS (PM; and PM ;) and were asked to choose one of them (Table C3 of Appendix C). Because labels affect cognition (Muchinsky and Dudycha 1975; Sniezek 1986; Broniarczyk and Alba 1994), generic names were used for both PMS so that participants’ decisions were influenced only by the PMS’ attributes and not by their labels (e. g., percent of defects units, cost per unit). In order to deter possible concerns that a lower expected incentive compensation would result in compensation below the plant manager’s reservation utility, participants 14 Performance-contingent compensation is intended to align participant’s goal with their firm’s goal. In natural settings, individuals designing incentive compensation are supposed to make decisions that maximize their firm’s expected wealth. 27 were told that their PM choice only determined how dependent the manager’s annual bonus was on these two PMs: “Your firm’s human resources department will design the managers’ total compensation packages to be competitive in the managerial labor market”. Participants self-paced their way through the experimental materials. After they finished the task, they turned in the materials to the administrator and received post- experiment questions (Table C4 of Appendix C). Variables Independent and Control Variables A 2x2 (independent variables) x 2x2 (control variables) between-subjects experimental design was used to test the hypotheses. The manipulated independent variables were PM attribute conflict (absent or present) and the difference between PM attribute differences (small or large). The PM attributes’ values for the four cells resulting from the interaction between these independent variables are in Table C5 of Appendix C. To control for the order in which the PMS were provided, participants were randomly assigned to one of two possible column orders in which the two PMS were presented. Finally, to control for the row order in which PMS’ attributes were presented, participants were randomly assigned to one of the following two orders of presentation: (1) action congruity, sensitivity, and precision; or (2) sensitivity, precision, and action congruity. Two additional independent variables were measured in post post-experimental questions. Consistent with prior literature in accounting (Kadous et al. 2003), Klein et al.’s (2001) five-item instrument was used to measure goal commitment (Panel A of Table C4). Empathic concern was measured by using Davis’ (1980, 1994) seven-item 28 instrument (Panel B of Table C4). Post-experirnental questions were also intended to measure control variables for participants’ understanding of the materials (Panel C of Table C4), their experience, and their knowledge of accounting, calculus, finance, incentive compensation plans, microeconomics, and statistics (Panel D of Table C4). Finally, four post-experimental questions were included to measure participants’ judgments about the PM attributes’ relative importance (Panel E of Table C4). Dependent Variable The dependent variable was decision performance. Decision performance was measured as a dichotomous variable coded 0 or 1 depending on whether a participant’s choice was inconsistent or consistent, respectively, with the agency-based prediction. The optimal choices were obtained by plugging the PM attributes’ values into the F eltham and Xie’s (1994) model (Appendix A). The optimal decisions for the four cells of the manipulated independent variables are in Table C6 of Appendix C. 29 CHAPTER IV: RESULTS Descriptive Statistics Prior research indicates that directional errors in the use of PMS’ attributes are relatively frequent (Krishnan et. a1 2005).15 To ensure that individuals responded to the experimental manipulation of PM attribute conflict, they were required to have at least a minimum understanding of how each attribute is directionally related to their goal. Consequently, eighteen participants were excluded because they reported agency- inconsistent beliefs about how each PM attribute Should directionally affect their choice.16 In addition, to control for a minimum understanding of the decision task, 15 participants were excluded because they answered incorrectly one or more questions related to the information provided about the task and their firm, such as their firm’s goal and the risk preferences of both the manager and principal. Thus, 43 of the 76 participants (57%) were retained for hypotheses testing.17 The participants had completed a mean of 8.39 three-credit semester accounting courses (SD. = 1.66, range 4-12), 1.78 finance courses (SD. = 1.33, range 1-6), 1.41 microeconomics courses (SD. = 0.71, range 1-4), 1.90 statistics courses (SD. = 0.49, range 1-4), and 1.14 calculus courses (SD. = 0.69, range 0-3). Their work experience as a manager or an accountant ranged fiom zero to 25 months, with a mean of 4.53 months (SD. = 6.21). None of them had any experience designing incentive compensation. 15 In their study, only 15 participants out of 32 (47%) made correct decisions about how a PM’S precision should affect the PM’S weight on the incentive compensation. '6 For example, if there are two available PMs which differ only with respect to action congruity (precision), then they should choose the more action congruent erecise) PM. 17 None of the variables intended to capture participants’ characteristics (GPA, experience, age, goal commitment, empathic concern) explained this sample selection process. 30 TABLE 4 FACTOR ANALYSIS: MEASURED INDEPENDENT VARIABLES PANEL A: Goal Commitment Factor Item 1. 2 Standardized Eigenvalue Variance Cronbach’s oadrng Extracted Alpha 1 (R) 0.757 3.095 62% 0.837 2 (R) 0.902 0.969 3 0.892 0.51 1 4 (R) 0.794 0.232 5 0.530 0.144 PANEL B: Others Concern Behavior Factor Item 2’ 3 Standardized Eigenvalue Variance Cronbach’s oadrng Extracted Alpha 1 0.800 3.557 51% 0.824 2 (R) 0.689 0.904 3 0.596 0.770 4 (R) 0.734 0.663 5 (R) 0.664 0.497 6 0.727 0.335 7 0.710 0.273 31 TABLE 4 (continued) NOTES TO TABLE 4: 1 Panel A of Table C4 presents the five-item instrument used to measure goal commitment. Denotes that item was reverse-scored before analysis. Panel B of Table C4 presents the seven-item instrument used to measure empathic concern. With respect to the participants’ goal commitment and empathic concern, the results of a confirmatory factor analysis for each variable are presented in Table 4 (Panels A and B respectively). The results indicated that, for each variable, all of its items loaded in a common factor. Participants’ goal commitment and empathic concern scores were calculated by using the standardized factor loading as weights to aggregate the respective standardized items.18 The two scales had acceptable reliabilities with coefficients greater than .70 (Cronbach 1951; Nunnaly 1978). Descriptive statistics for these two variables are presented in Table 5. 18 The results of hypotheses testing when the independent variables (goal commitment and empathic concern) were calculated by using an equal (unit) weights aggregation of items were not qualitatively different than those when the items were aggregated by using standardized factor loading as weights. Consistent with prior literature (Klein et al. 2001; Kadous 2003), the reported results are based on the weighted aggregation of items for each independent variable. 32 TABLE 5 DESCRIPTIVE STATISTICS: MEASURED INDEPENDENT VARIABLES PANEL A: Mean Goal Commitment Factor (Standard Deviation)[n] PM Attribute Conflict Absent Present Overall Lar e 0.128 -0.419 -0.102 _ g (1.133) (1.137) (1.163) Drfference 8 19 B [11] [ 1 [ ] etween PM Attribute Differences Small -0.242 0.620 0.081 (0.847) (0.599) (0.862) [15] [91 [24] -0.086% 0.131 0.000 Overall (1.000) (1.015) (1.000) [261 [171 [43] PANEL B: Mean Empathic Concern Factor (Standard Deviation) [11] PM Attribute Conflict Absent Present Overall Lar e 0.129 0.596 0.326 , g (1.155) (0.934) (1.066) Difference [1 1] [8] [19] Between PM Attribute Differences Small -0.235 -0.296 -0.258 (0.999) (0.705) (0.883) [15] [9] [24] -0.081 0.123 0.000 Overall (1.061) (0.916) (1.000) [26] [17] [43] 33 Prior literature indicates that goal commitment and empathic concern are likely to be affected by task characteristics (Eisenberg and Miller 1987; Klein at al. 2001). Thus, the experimental manipulation of task characteristics (action congruity and precision) may have had an effect on participant’s answers to post-experimental questions intended to measure their goal commitment and empathic concern. The hypotheses are intended to test the effect of ex ante individual differences and not those individual differences resulting from the experimental manipulations. Therefore I eliminated the effect of the manipulated independent variables on the measured independent variables by using as measures of goal commitment and empathic concern for hypothesis testing the standardized residuals of the linear regressions of each individual difference variable on the two manipulated independent variables and their interaction term.19 To test whether differences across participants other than the measured independent variables may have driven results, measures of participants’ knowledge (such as undergraduate GPA and GPA in courses related to accounting, calculus, finance, microeconomics, and statistics) were included as both the dependent variable in a 2 (PM attribute conflict) x 2 (difference between PM attribute difference) AN OVA and independent variables in a logistic regression on decision performance. The results of the AN OVAS indicated that (1) participants’ GPAS in calculus, microeconomics, and statistics did not Significantly differ (p > .05) across the manipulated experimental 19 Consistent with results documented by Klein at al. (2001), the interaction term had a statistically Significant effect (p < .05) on the goal commitment factor score such that participants on the most complex task (cell 4) reported higher goal cormnitment than participants in the other tasks (cells 1, 2 and 3). With respect to the regression on empathic concern, the results indicted a Statistically Significant effect (p < .05) of the difference between PM attribute differences such that the larger the difference the higher the reported empathic concern. Although no prior research has reported such an effect, this finding is consistent with individuals being more concerned about others when the effect on others is expected to be higher. The tests of H2 and H3 using the original factor scores provided qualitatively the same results as those reported. 34 conditions; and (2) a main effect of PM attribute conflict on undergraduate GPA, accounting GPA and finance GPA such that participants in the conflict-present condition reported Significantly (p < .05) higher GPAS than participants in the conflict-absent condition. However, results of the logistic regressions indicated that none of these variables was significantly associated (p > .05) with the likelihood of participants making a correct decision. Thus, although random assignment may not have been completely successful in equalizing all of the participants’ demographic characteristics across experimental conditions, these differences across experimental conditions did not explain the participants’ decision performance. The Pearson correlations between measures of the participant’s characteristics and two measured independent variables are presented in Table 6. Most of the participants’ knowledge measures were significantly correlated (p < .05). More importantly, neither goal commitment nor empathic concern was significantly (p < .05) correlated with any of the participants’ knowledge measures. Thus, differences across participants on the two measured independent variables were not likely to be driven by participants’ prior knowledge. 35 TABLE 6 PEARSON CORRELATION MATRIX: PARTICIPANTS’ CHARACTERISTICS (TWO TAILED P-VALUES) [N = 43] Cfnfriiit- Empathic Overall Actirfgm- Calculus Finance exh‘brrfiibs Statistics merit Concern GPA GPA GPA GPA GPA GPA ggzimit- 1.00 0.34 0.03 0.151 -0.08 -0.00 -0.1 1 0.12 ment (0.02) (0.84) (0.333) (0.62) (0.96) (0.47) (0.45) Empathic 1.00 -0.00 0.18 -0.01 0.18 0.17 0.260 Concern (0.98) (0.26) (0.97) (0.26) (0.28) (0.09) Overall 1.00 0.64 0.39 0.61 0.37 0.60 GPA (0.00) (0.02) (0.00) (0.02) (0.00) Accoun- 1.00 0.43 0.52 0.36 0.53 ting GPA (0.01) (0.00) (0.02) (0.00) Calculus 1 .00 0.1 7 0.49 0.40 GPA (0.32) (0.00) (0.01) Finance 1.00 0.12 0.36 GPA (0.29) (0.02) xgfiggics 1.00 0.33 GPA (0.03) Statistics GPA 1.00 36 Hypothesis Testing Hypothesis I predicted a two-way interaction between PM attribute conflict and difference between PM attribute differences, such that decision performance will be relatively low when PM attribute conflict is present and the difference between PM attribute differences iS small (point 4 in Figure 2); otherwise predicted decision performance is higher and equal for the other three conditions (points 1, 2 and 3 in Figure 2). Participants’ decision performance is presented in Panel A of Table 7. Overall, 86 percent of the participants made correct decisions (37 of 43 participants). The pattern of the frequency of correct decisions was consistent with the form of the hypothesized interaction such that the frequency of correct decisions was 100% in cells 1, 2 and 3, but only 33% in cell 4. Hypothesis 1 was tested by a set of six planned contrasts based on the predicted pattern of cell frequencies in Figure 2. Since small sample Size and lack of incorrect decisions in cells 1, 2 and 3 precluded the use of either a chi-square test or logistic regression for testing differences in frequencies across cells, Fisher’s exact test was used to determine the Significance of each planned contrast. A Bonferroni adjustment was used to control the family-wise error at p < .05 for the overall test of the interaction hypothesis; thus, the critical p-value level for each contrast was .008. Results for each planned contrast are presented in Panel B of Table 7.20 20 Each contrast refers to predictions presented in Panel B of Figure 2 and the respective difference between cells in Panel A of Table 7. 37 TABLE 7 PARTICIPANTS’ DECISION PERFORMANCE AND HYPOTHESIS TESTING RESULTS PANEL A: Frequency of Correct Decisions (Number of Participants) PM Attribute Conflict Absent Present Overall 1 2 . Large 100% 100% 100% Difference (1 1) (8) (19) Between 0 PM Attribute 75 4 Differences Small 100% 33% (24) (15) (9) 86% 100% 65% Overall (26) (17) (43) PANEL B: Fisher’s Exact Test (H1) Planned Contrastl Fisher’s p—value Contrast Supported? (one tailed) (yes/no) 1) 1 = 3 1.00 yes 2) 1 = 2 1.00 yes 3) 2 = 3 1.00 yes 4) l > 4 0.001 yes 5) 2 > 4 0.007 yes 6) 3 > 4 0.002 yes 38 TABLE 7 (continued) PANEL C: Kendall's Rank Correlation (H2 and H3) [11 = 9] Goal Commitment Empathic Concern . . 0.707 0236 Correct Decrsron p = 0.01 p = 0.24 NOTE TO TABLE 7: 1 Each contrast refers to predictions presented in Panel B of Figure 2 and the respective difference between cells in Panel A of Table 7. When PM attribute conflict was absent, results of contrast 1 indicated that participants’ decision performance was not significantly affected (p = 1.00) by the difference between PM attribute differences. However, when PM attribute conflict was present, the difference between PM attribute differences affected participants’ decision performance. When this difference was large (cell 2), results of contrasts 2 and 3 indicated that the frequency of correct decisions did not significantly (p = 1.00) differ from the conflict-absent conditions (cells 1 and 3). However, when this difference was small (cell 4), results of contrasts 4, 5 and 6 indicated that the frequency of correct decisions was Significantly (p < .008) lower than in the other three conditions (cells 1, 2 and 3). Overall, each of the six contrast tests provided support for H1. 39 Hypotheses 2 and 3 predicted that when there is PM attribute conflict and a small difference between PM attribute differences (cell 4), the likelihood of a correct decision would be positively associated with participants’ goal commitment and negatively associated with their empathic concern, respectively. Since perfect separation of the data 2‘ or a point-biserial and small sample size precluded the use of a logistic regression correlation,22 the nonparametric Kendall's rank correlation was used to test the association between correct decision and each of the two individual difference variables.23 The Kendall's rank correlations are presented on Panel C of Table 7. While H2 was supported by a positive and statistically Significant correlation between correct decision and goal commitment (p < .05), H3 was rejected due to the lack of a significant correlation (p > .05) between correct decision and empathic concern.24 Supplementary Evidence Sample size In order to address concerns about small sample Size I performed two analyses. I first conducted ex post power calculations for each hypothesis, and then, based on 2‘ Perfect separation occurs if all of the observations having unique covariate profiles have the same response outcome. For example, in cell 4, all of the participants that made correct decisions reported higher goal commitment than did all of the participants that made incorrect decisions. If perfect separation is present, then the use of logistic regression results in infinite parameter estimates (Allison 1999). 22 The point-biserial correlation is a special case of the Pearson correlation in which one variable is dichotomous and the other iS continuous. 23 While Spearman rank correlation is more frequently used in research, Kendall’s rank correlation includes an adjustment to account forties in the data (Kendall 1976). The use of dichotomous variable as dependent variable guarantees the presence of several ties when testing H2 and H3, and thus, Kendall’s rank correlation is the most appropriate test. 24 The results of hypotheses testing when point-biserial correlation was used to test the association between individual differences and decision performance were not qualitatively different than when Kendall’s rank correlation was used (point—biserial correlation for H1 was 0.77 and p = .005; point-biserial correlation for H2 was -0.05 and p = .88). 40 Hollenbeck et al. (2006), I analyzed parameter stability for the Significant Kendall's rank correlation between individuals’ goal commitment and their decision performance (i.e., test of H2). H1 was tested by using Fisher’s exact test to test for differences in proportion. Based on the estimated proportions (100% vs. 33%), for a conventional power of 80% (Cohen 1988) and the probability of rejecting the null hypothesis at a =.008, nine participants per cell were required. While cell 2 had only 8 participants, the other cells had nine or more participants. Thus, power for the test of H1 appeared to be satisfactory.25 HZ and H3 were tested using Kendall's rank correlation. Based on a conventional power of 80% and the probability of rejecting the null hypothesis at 0. =05 (one tailed), 34 and 14 participants were required to detect a medium and a large effect Size respectively (i.e., correlations of 0.3 and 0.5 respectively; Cohen 1988). While the test of H2 and H3 only included nine participants, the actual Kendall's rank correlations were 0.707 and -0.236 for the test of H2 and H3 respectively. As a result, the power for the test of H2 appeared to be satisfactory but the power for the test of H3 was extremely low.26 Since only nine participants were included in the test of H2 (participants of cell 4), the estimated Kendall's rank correlation (rk = 0.707; p < .05) may have been impacted Significantly by sampling error, and as a result it may be unstable. In order to address this concern, I analyzed the parameter stability of the test of H2 by using the “exclusion/inclusion of a single data point” procedure suggested by Hollenbeck et al. 25 The observed power of contrasts 4, 5, and 6 (Panel B of Table 7) was 84%, 73% and 91% respectively. 26 The observed power for the test of H2 and H3 was 85% and 22% respectively. 41 (2006). I calculated nine Kendall's rank correlations between individuals’ goal commitment and their decision performance. Each of these correlations was calculated by excluding one participant from the sample (i.e., sample size = 8 for each correlation). The results were robust to the removal of any of the nine participants. Each of the nine Kendall's rank correlations was significantly positive (p < .05) and not Significantly different than 0.707 (p > .05). Considering the results of power analysis and stability test, the small sample Size used in testing H1 and H2 did not seriously impair the statistical validity of the conclusions. In the case of H3, however, the lack of Significant correlation and the low power of the test provided two competing conclusions: 1) the null hypothesis of no effect of individuals’ empathic concern on decision performance is actually true; and (2) the null hypothesis of no effect is actually false, but a combination of some or all of small effect Size, measurement error, and small sample sizes prevent the test from being able to detect this effect. Performance Measure Attributes ’ Relative Importance Hypothesis 2 predicted that, because goal commitment was expected to increase individuals’ attention to action congruence, individuals with higher goal commitment were more likely to chose the more action congruent PM, and as a result were more likely to make correct decisions. Therefore, H2 assumed that PM attributes’ relative importance mediated the relation between individuals’ goal commitment and decision performance. To provide evidence on the validity of the mediating role of PM attributes’ relative importance, I used Holmbeck’s (1997) four criteria to establish a mediating relationship. The links tested under the Holmbeck (1997) approach are presented in Panel 42 A of Figure 3. Four questions in the post-experimental questionnaire were used for measuring PM attributes’ relative importance (Panel B in Table C4 of Appendix C). While, questions 1 and 4 were intended to directly measure the PM attributes’ relative importance, a third measure of their relative importance was constructed by dividing the answer to question 3 by the answer to question 2. Correlation results for the four criteria (a, b, c and d) for each of the three measures of the mediator variable (question 1, question 4, and question 3 /question 2) are presented in Panel B of Figure 3. While Pearson correlation was used to test criteria a, b, and c, partial correlation was used to test criterion (1. The last row of Panel B indicated whether the test of the four criteria supported the mediating role of the PM attributes’ relative importance. A complete mediation model Should result in Significant correlations for criteria a, b and c and a non-Significant correlation for criterion d. The test of Holmbeck’s (1997) four criteria supported a complete mediation model for two of three measures used (question 4, and question 3/question 2). For question 1 (first column on Panel B of Figure 3), the results for criterion b indicated that the independent variable was not Significantly (p > .05) associated with the mediator variable, and thus, it did not provide support for the mediation model. Because question 1 was more Specific than the other questions, it required individuals to describe a higher-order mental operation relative to the other questions. Consistent with prior literature in psychology (N isbett and Wilson 1977), question 1 may be an unreliable construct due to the individuals’ limited ability to ex post describe their higher-order mental Operations. Overall, the results provided some support for the complete mediation of goal commitment on decision performance by PM attributes’ relative importance. 43 FIGURE 3 MEDIATOR ROLE OF PERFORMANCE MEASURE ATTRIBUTES’ RELATIVE IMPORTANCE WHEN PERFORMANCE MEASURE ATTRIBUTE CONFLICT IS PRESENT AND THE DIFFERENCE BETWEEN PERFORMANCE MEASURE ATTRIBUTE DIFFERENCES IS SMALL PANEL A: Examining How Goal Commitment Affects Individual Decision Performance Dep_endent Variable Independent Variable a) Direct Effect Goal Commitment d) Decrease in Direct Effect Performance Due to Mediator (b) (C) Mediator Importance of Action Congruity Relative to Precision 4' Individual Decision PANEL B: Test to Establish the Mediating Role of Performance Measure Attributes’ Relative Importance. q1 q4 q3/q2 a) the effect of independent variable 0.87 0.87 0.87 on dependent variable (0.002) (0.002) (0.002) b) the independent variable is associated 0.64 0.92 0.79 with the mediator (0.085) (0.001) (0.020) c) the mediator is associated with 0.791 0.88 0.85 the dependent variable (0.020) (0.004) (0.007) d) the mediator causes the direct effect (a) 0.80 0.41 0.66 to decrease (0.029) (0.365) (0.103) Mediating Role Supported (yes/no) no yes yes Correlation (two tailed p-value) [n= 9] 44 Since hypothesis 3 was rejected, empathic concern was not likely to explain the behavior of individuals choosing the more precise PM when PM attribute conflict was present and the difference between PM attribute differences was small. In order to investigate an alternative explanation, I studied judgment about PM attributes’ relative importance of those individuals who made incorrect decisions. T-test results indicated that their answers to questions 1 and 4 (on scale 1 to 9) were not significantly (p >.05) different than five and that question 3/question 2 is not Significantly (p >.05) different than one. These results suggested that, when PM attribute conflict was present and the difference between PM attribute differences was small, the incorrect choices may be explained by the use of a compensatory heuristics with equal weights. 45 CHAPTER V: DISCUSSION Many individuals who are not compensation experts choose PMS for incentive compensation. Boundedly rational individuals are likely to, contingent on the characteristics of the task, use heuristics to simplify the choice of PMS. This dissertation identifies PM attribute conflict and difference between PM attribute differences as two task characteristics that influence individuals’ use of heuristics, and as a result, explain their decision performance. The experimental results provide support for two of the three hypotheses. First, decision performance was explained by a two-way ordinal interactive function between PM attribute conflict and difference between PM attribute differences such that that decision performance was relatively low when PM attribute conflict was present and the difference between PM attribute differences was small; otherwise decision performance was 100% and equal for the other three experimental conditions (H1). Second, goal commitment explained differences in decision performance when PM attribute conflict was present and the difference between PM attribute differences was small. Supplementary evidence supported the assumption that this relation was fully mediated by the PM attributes’ relative importance such that individuals with higher goal commitment were more likely to consider action congruity more important to their decision relative to PM precision; as a result, they were more likely to choose the better PM (H2). Finally, the expected negative effect of empathic concern on individuals’ decision performance when individuals’ use compensatory heuristics was not supported (H3). While power analysis indicated that the sample size for the test of H3 was insufficient, there are other two potential reasons for the lack of Significance of this 46 result. First, although participants reported different levels of empathic concern, the social context may not have been strong enough to activate prosocial behavior. Second, empathic concern does not explain individuals’ choice of PMS. Supplementary evidence, consistent with the second reason, suggested that the incorrect PM choices of individuals using compensatory heuristics may have been explained by the use of equal weights. Although This dissertation has three limitations arising from choices made in the design of the experiment. The first and most important limitation is small sample size; the second is characteristics of the participants; and the third is the information about PM attributes provided to the participants to make their PM choice. First, significant effort was made to include a larger sample Size in this dissertation, but only 76 individuals participated. In addition, 43% of the participants (33 of 76) were not retained for hypotheses testing due to lack of understanding of the task. While the sample Size across cells was sufficient to test H1, the small sample in cell 4 (n = 9) forced the use of a nonparametric test for testing H2 and H3. Although the observed power for the test of H2 appears to be satisfactory, the power for the test of H3 was extremely low. As a result, the small sample affected the validity of the evidence regarding H3. The participants were a relatively homogenous group of students from an MS. in Accounting program, and the information provided to them was exactly what they needed for making their PM choices. In natural settings, decision makers’ characteristics and the information provided to make PM choices may differ considerably. Individuals’ prior experience in designing incentive compensation may allow them to make appropriate judgments about the PM attributes’ relative importance. Consequently, the conclusions of 47 this dissertation are restricted to situations where individuals choosing a PM do not have prior experience in deciding PM-based incentive compensation with these particular tradeoffs. With respect to the information provided, although PM-based reports are common managerial practices, individuals are not likely to receive detailed information about the numerical value of each PM attribute. Instead, they are likely to make subjective assessments about these attributes’ values. For example, in the case of PM precision, they may try to determine the extent to which a PM reflects factors outside the manager’s control. After individuals complete the subjective assessment of each attribute’s value for the available PMS, however, they are likely to face PM attribute conflict and difference between PM attribute differences. Therefore, results of the effect of these two task characteristics on individuals’ decision performance presented in this dissertation are not experimental artifacts. There are two important lessons learned from this dissertation. First, when the number of participants is limited and the sample selection process is expected to decrease the number of participants retained for hypotheses testing, then the participants Should be strategically allocated to cells in order to ensure enough sample Size for the most interesting hypotheses. Particularly in the case of this dissertation, I Should have allocated more participants to cell 4 where H2 and H3 were tested instead of trying to ensure equal sample Size across cells. Second, the agency situation described in the experimental materials did not necessarily provide a strong manipulation of the social context. If social context is expected to drive individuals’ behavior, then it is recommended to ensure a strong manipulation of the social context by, for example, designing an initial activity in which two participants interact with each other, and then 48 they are told that their counterpart is the agent that they need to motivate by designing the incentive compensation. This dissertation provides several Opportunities for future research. First, identifying omitted variables can lead to new and interesting testable predictions. For example, a stronger social context can stimulate individuals’ empathic concern and/or goal commitment and thus explain individuals’ performance when PM attribute conflict is present and the difference between PM attribute differences is small. On the one hand, knowing the manager can increase individuals’ empathic concerns and, as a result, increase the likelihood of individuals choosing the more precise PM. On the other hand, accountability can increase the effect of goal commitment and, as a result, increase the likelihood of individuals choosing the more action congruent PM. Second, future research may address how individuals change their PM choices in response to feedback over time. Although feedback in natural settings can be difficult to understand due to a incorrect initial PM choice (Krishnan et al. 2002) or changes in an environment (e.g., uncontrollable factors, delegation), feedback may still provide learning opportunities. Finally, this dissertation provides a starting point for understanding how individuals put weights on PMs (second step on PM-base incentive compensation design). The variables identified in this dissertation are likely to affect individuals’ use of heuristics when weighting PMS, and as a result, they are expected to explain individuals’ performance on the PM weighting decisions. 49 APPENDICES 50 APPENDIX A FELTHAM AND XIE (1994) MODEL 51 Table 1 Shows a baseline version of the information participants received. The parameter values in this table are linked to the Feltham and Xie’S (1994) model as follows. Performance measures (here PM; and PM;, y; and y;, respectively) are modeled as linear functions of the agent's actions (one action of dimension A and one action of dimension B, a; and a;, respectively). Yr =l~l 1131 +H1232+8b and (1) Y2 = H2131 + 112232 + 82 (2) Each action is weighted by a sensitivity parameter (uij) that indicates the effect of a agent’s action j on performance measure i. An error term (8;) represents effects on the performance measure i other than the plant agent’s actions. The variances and covariances of these si's are represented in a matrix 2, with the diagonal terms indicating the variance of the error term in each individual measure and the off-diagonal terms indicating error covariance. The principal's expected gross payoff (x) is also affected by the two types of agent’s actions: X = b141 + bzaz + 8x: (3) where b;- = effect of a agent’s action j on the principal's expected gross payoff (a component of congruity) and ex = error term representing effects of other influences on the principal’s expected gross payoff. Optimal performance-measure weights are calculated using the parameters given in Panel A of Table land Equation (7) in Feltham and Xie (1994): v’ = [uu’+r£1" 11b. (4) 52 where: + . . . v = a vector of Incentive weights; u = matrix of sensitivity coefficients (expected change in PMS associated with a one-unit effort allocated to actions associated to those PMS); r = agent’s absolute risk aversion (value assumed to be 1, as in numerical example presented in Table 1 by Feltham and Xie 1994); 2 = variance-covariance matrix representing error variance and error covariance of the PMs; b = vector of action congruity coefficients (expected effects of the agent’s actions associated to each PM on the principal’s gross payoff). The agent’s actions resulting from the optimal weight is calculated using the results of Equation (4) on Equation (6) in Feltham and Xie (1994): a= n'v (5) Finally, the principal's expected gross payoff resulting from this Optimization is calculated using the result Of Equation (5) on Equation (3). 53 APPENDIX B COMPENSATORY HEURISTICS 54 Two compensatory heuristics available for choosing PMS are weighted additive (WADD) and additive differences (ADDIF) (Payne at al. 1992). When individuals use the WADD heuristic, they make an overall evaluation of one PM by considering both the values of all of its attributes and the attributes’ importance. The same procedure is repeated for the other PM, and the PM choice is made by comparing the overall evaluations of the two PMS and then choosing the PM with higher overall evaluation. For example, Equations (1) and (2) represent the use of WADD to make evaluations of PM; and PM;: Evaluation of PM; = Wp P; + WAC AC; (1) Evaluation of PM; = Wp P; + WAC AC; (2) where P; and AC; represents the values of precision and action congruity for the PM;, and Wp and WAC represents how subjectively important precision and action congruity are to the decision. When individuals use the ADDIF heuristic, they first determine the difference between PMS on each attribute. Then a subjective weight, based on the attributes’ importance, is applied to each difference and the weighted differences are summed over all of the attributes to obtain an overall evaluation of the difference between the two PMS, which will be the basis for the decision. Using the same parameters of equation (1) and (2), Equation (3) represents the use Of ADDIF heuristic: Evaluation of PM; vs. PM; = Wp (P;- P;) + WAC (AC;- AC;) (3) AS modeled in Equations (1), (2), and (3), when individuals use compensatory heuristics, their PM choice depends on their judgments about the importance of each attribute (Wp and WAC). Thus, consistent with physiology literature (Shah and 55 Oppenheimer 2008), WADD and ADDIF are expected be equivalent in terms of cognitive demand (i.e., complexity). In addition, Since the Equation (3) is mathematically equivalent to the difference between Equation (1) and Equation (2), the use of ADDIF is expected to lead individuals to the same choice as the use of WADD. Consequently, this dissertation refers to compensatory heuristics, and not to the two Specific compensatory heuristics. 56 APPENDIX C EXPERIMENTAL MATERIALS 57 BACKGROUND INFORMATION Assume you are a manager working at the headquarters Of a large, diversified, publicly owned firm with many stockholders. Your firm wants to improve its overall economic performance, and it is planning to hire a manager on a one-year employment contract to implement a performance-improvement program. Your job is to design an incentive bonus plan for this manager. Below is information that can help you in designing this manager’s incentive bonus plan: 1. The one-year contract starts on January 1, 2007 and ends on December 31, 2007. This manager has no chance of future employment with your firm. The manager will be rewarded at the end of 2007 for his or her performance in 2007. The manager does not have any incentive to “game” the incentive bonus plan in order to tradeoff rewards in one year against rewards in future years, because he or she will not be employed by your firm after 2007. 2. Your firm’s policy about incentive contract design for this type of managers has two steps: 0 Choice of a performance measure: select a performance measure from a set of two available performance measures to be included in the manager’s incentive bonus plan. 0 Weighting performance measures: put a weight on the previously selected measure in the manager’s incentive bonus plan. For example, if a performance measure is weighted as $5, it means that for every unit of that measure, the manager will receive $5 as an incentive bonus. Thus, if measured performance at the end of 2007 is 100, then the manager will receive an annual 58 incentive bonus of $500 (= $5/unit of measure x 100 units of measure). . Your job is to choose one of the two available performance measures to be included in this manager’s incentive bonus plan. Another manager at your firm’s headquarters will make the best possible weighting decision based on the performance measure that you have selected, in order to complete the design of this manager’s incentive bonus plan. . Your firm’s human resource department will design the manager’s total compensation package to be competitive in the managerial labor market. If you choose a performance measure, then this only means that the manager’s annual incentive bonus will be more or less dependent on that particular measure. You do not need to be concerned about whether your choice of a performance measure will make the manager’s total compensation package uncompetitive in the labor market. . The objective of the owners of your firm, which your choice Should support, is to maximize the economic (market) value of your firm, net of the cost of the manager’s total compensation. . The owners of your firm are risk neutral. They are well-diversified and therefore they care only about the expected economic (market) value of your firm and not about unpredictable uncontrollable variation around this expected economic (market) value. . The manager for whom you are designing an incentive bonus plan can implement a wide variety of actions that are intended to maximize the economic (market) value of your firm by affecting one dimension of your firm’s performance (e.g., reducing costs, increasing quality, increasing innovation, increasing market Share). These actions require time and effort from the manager, however. The more actions the 59 10. manager is already implementing to affect this dimension of your frrm’s performance, the more difficult (personally costly) it is for him or her to identify and implement additional actions to affect this dimension of your firm’s performance. The manager expects to receive a larger annual incentive bonus when he or She implements more actions to affect this dimension of your firrn’s performance. The smaller the annual incentive bonus that he or She expects to receive for his or her actions that affect this dimension of your firm’s performance, the less motivated he or She will be to implement actions that affect this dimension. The manager for whom you are designing the incentive bonus plan is risk averse. That is, the more uncertainty he or She has about his or her expected incentive bonus, the less motivating the expected incentive bonus will be to him or her. The cost of collecting data and reporting a performance measure is the same for both available performance measures. No performance measure is a perfect indicator of how the manager’s actions can affect your firm’s economic (market) value, and therefore no performance measure is a perfect basis for an incentive bonus that is intended to motivate the manager to maximize your firm’s economic (market) value. However, one performance measure may be better than the other for this purpose. Three attributes of a performance measure, common to both available performance measures, can help you to design the manager’s incentive bonus plan. These attributes are defined by relating the performance measure to the actions that the manager can implement as a part of his or her job activity during 2007. In order to 60 describe a variety of possible managerial actions in common terms, we use the term “a unit of action”. (A) How much does a manager’s unit of action that is intended to change a (B) (C) particular performance measure also increase the economic (market) value of your firm, on average? For example, implementing a unit of action that is intended to increase measured market share could be expected to add more to the economic (market) value of your firm than implementing a unit of action that is intended to increase a measure of employee Skill, or vice versa. How much does a manager’s unit of action change a performance measure, on average (that is, in a year without unusually good or bad uncontrollable events)? For example, in an average year, a unit of action intended to increase market Share might be expected to increase it by 2%, or a unit of action intended to increase customer satisfaction might be expected to increase it by one point on an 11-point scale. Given the manager’s choice of actions, how much unpredictable variation in the measured performance is caused by factors the manager can not control (e.g., competitors’ actions)? For example, if the manager chooses to implement actions that are expected to increase market Share by 2% in an average year, then the measured market Share increase might be somewhere between 1.5% and 2.5% if this unpredictable 61 uncontrollable variation in the measured performance is small and somewhere between 0% and 4% if this unpredictable uncontrollable variation is large. The larger the unpredictable uncontrollable variation in the performance measure used for the incentive bonus plan, the more uncertain the manager will be about the expected amount of the incentive bonus that he or She will receive for implementing a unit of action. General description of your job Your job is to design an incentive bonus plan for a manager who will work under a one- year employment contract to implement a performance-improvement program during 2007. In order to meet your firm’s policy about incentive bonus plans, your job is to choose one of two available performance measures for the dimension of your firm’s economic performance that is expected to be affected by this manager. Another manager at your firm’S headquarters will make the best possible weighting decision based on the performance measure that you have selected in order to complete the design of this manager’s incentive bonus plan. For each performance measure, your firm has provided you with estimates that are intended to capture the values of the three attributes (A, B and C) of the performance measures described in point 10 on the prior page. The following table provides a description of what each of these estimates means. 62 TABLE C1 DESCRIPTION OF PERFORMANCE MEASURES’ ATTRIBUTESl Performance Definition Description of Attribute Meaning Measures’ Attributes Expected effect on your The higher this attribute value, the firm’s economic (market) larger the expected effect of a unit of Agibn value when the agent action on the economic (market) value C . rrnplements one umt Of of your firm. ongrurty action that is intended to affect PM;. Expected effect on PM; The higher this attribute value, the (B) when the agent implements larger the expected effect of a unit of one unit of action that is action on the performance measure. Sensitivity intended to affect PM;. Standard deviation of PM; The higher this attribute value, the C due to factors the agent can higher the unpredictable variation in this ( ) not control. performance measure that is due to Precision factors the manager can not control. NOTE TO TABLE C1: 1 Participants received the descriptions Shown below, but without the names which are in italics that identify attributes in terms of Feltham and Xie (1994) model. 63 PM3 and PM7 refer to the two available performance measures for the dimension of your firm’s performance that is expected to be affected by the manager. The specific names of the two performance measure have been replaced by generic names so that you make your choice of a performance measure based on their attribute values only and not on their names. The numbers used to identify the two measures were randomly chosen and do not represent any order of importance. The estimates of the performance measures’ attribute values provided by your firm are based on experience with Similar one-year employment contracts. Your firm’s accounting and human resources departments agree that all of the estimates you will receive for the manager are accurate. In addition, the manager also is aware of these estimates and agrees with them. Your pay You will receive fixed pay Of $10 for completing all of this accounting simulation, regardless of your performance in choosing a performance measure. You will also receive variable pay up to $10 in addition, depending on how well you chose a performance measure. That is, the closer your choice is to the choice that would maximize the expected economic (market) value of your firm, the higher your total pay will be, up to a maximum of $20 (= $10 + $10). If you do not complete the accounting simulation, you will not be penalized. You will not be paid, however, for an uncompleted accounting Simulation. 64 Task Your firm will provide you with two performance measures, PM3 and PM;, for the dimension of your firm’s performance that is expected to be affected by this manager. Based on the information provided, your job is to choose one of these two performance measures to be included in this manager’s incentive bonus plan. 65 TABLE C2 BELIEFS ABOUT EACH ATTRIBUTE-DECISION RELATION 1. If there are two available performance measures which differ only with respect to attribute (A): a. I would choose the performance measure with a higher value on attribute (A). b. I would choose the performance measure with a lower value on attribute (A). c. I would be indifferent between choosing one or the other performance measure. 2. If there are two available performance measures which differ only with respect to attribute (B): a. I would choose the performance measure with a higher value on attribute (B). b. I would choose the performance measure with a lower value on attribute (B). c. I would be indifferent between choosing one or the other performance measure. 3. If there are two available performance measures which differ only with respect to attribute (C): a. I would choose the performance measure with a higher value on attribute (C). b. I would choose the performance measure with a lower value on attribute (C). c. I would be indifferent between choosing one or the other performance measure. 66 TABLE C3 SAMPLE OF PM ATRRIBUTES PROVIDED TO PARTICPAN TS Performance-Measure Attributes PM3 PM7 (A) Expected effect on your firm’S economic (market) value . . . . 2,500 2,000 when the manager Implements one umt of action that IS intended to affect PM;. (B) Expected effect on PM; when the manager implements one 3 600 3 600 unit of action that is intended to affect PM;. ’ ’ (C) Standard deviation of PM; due to the effect of factors the 1 200 1 800 manager can not control. ’ ’ 67 TABLE C4 POST-EXPERIMENTAL QUESTIONNAIRE 1 PANEL A: Goal Commitment 2 9:59P!“ It was hard to take my firm’s goal seriously. (R) Quite frankly, I didn’t care ifI achieved my frrrn’s goal or not. (R) I was strongly committed to pursuing my frrm’s goal. It wouldn’t take much to make me abandon my firm’s goal. (R) I think my firm’s goal was a good goal to Shoot for. PANEL B: Empathic Concern 2 . I often have tender, concerned feelings for people less fortunate than me. Sometimes I don't feel very sorry for other people when they are having problems. (R) 3. When I see someone being taken advantage of, I feel kind of protective towards them. Other people's misfortunes do not usually disturb me a great deal. (R) 5. When I see someone being treated unfairly, I sometimes don't feel very much pity for them.(R) I am Often quite touched by firings that I see happen. 7. I would describe myself as a pretty soft-hearted person. 68 TABLE C4 (continued) PANEL C: Understanding of the Materials 1. The objective of the owner of your firm, which your performance-measure choice Should support, was described in the background information as: a) Maximizing sales b) Minimizing cost c) Maximizing profit (1) Maximizing the economic (market) value of your firm, net of the cost of the manager’s total compensation. e) None of the above The owner of your firm is: a) Risk averse b) Risk seeking 0) Risk neutral d) I don’t know The manager for whom you are choosing PM for an incentive bonus plan is: a) Risk averse b) Risk seeking c) Risk neutral (1) I don’t know 69 TABLE C4 (continued) PANEL D: Experience and Knowledge 1. How many months of full-time work experience do you have as a manager or an accountant? Have you have any experience choosing performance measures for incentive bonus plans? (yes or no) . Please complete the following table about your undergraduate and/or graduate coursework you have completed before today. Please be sure to provide the total number of credits for each alternative and indicate whether your credits are from a quarter 01' SCIDCSICI' ICI'ITI. Number of Course Length Average GPA in Credits these courses Accounting Quarter Semester Calculus Quarter Semester Finance Quarter Semester Microeconomics Quarter Semester Statistics Quarter Semester 70 TABLE C4 (continued) PANEL E: PM Attributes’ Relative Importance 1. How important to your choice was $100 of attribute A relative to $100 of attribute C? 2. How important to your choice did you think the difference on attribute C between PM3 and PM7 3. How important to your choice did you think the difference on attribute A between PM3 and PM7 was? 4. In comparing PM3 and PM7, how important to your choice did you think the difference on attribute A was relative to the difference on attribute C? NOTES TO TABLE C4 Responses to Panels A and B were on a scale 1 (strongly agree) to 9 (strongly disagree). Responses to Panel B were on scale 1 (extremely less important) to 9 (extremely more important). Items followed by (R) indicate that the item was reverse-scored before analysis 71 TABLE C5 PM ATTRIBUTES’ VALUES PROVIDED FOR PERFORMANCE MEASURE CHOICE Large Difference Between PM Attribute Differences Small PM Attribute Conflict Absent Present :f'ItrSibutes PMI PMZ imbutes PMI PMZ 231'ng $2,500 $2,000 égggity 3;,000 3;,500 Precision $1,200 $3,600 Precision $1,200 $3,600 Sensitivity $3,600 $3,600 Sensitivity $3,600 $3,600 Eligibmes PMI PM2 :fibmes PMI PMZ 231';th $2,500 $2,000 Sign 3;,000 3;,500 Precision $1,200 $1,800 Precision $1,200 $1,800 Sensitivity $3,600 $3,600 Sensitivity $3,600 $3,600 72 TABLE C6 OPTIMAL SOLUTIONS PM Attribute Conflict Absent Present PMS’ * PMs’ * Attributes PMI PM2 Attributes PMl PM2 Large Weights $0.63 $0.28 Weights $0.50 $0.35 Principal’s Principal’s Difference Expected iii: Ellis Expected gigs gills Between Gross Payoff ' ' Gross Payoff ' ‘ PM Ambme PMs’ * PMS’ * Differences Attributes PM] PMZ Attributes P M1 PMZ Small Weights $0.63 $0.44 Weights $0.50 $0.56 Principal’s Principal’s Expected iii. Elf. Expected Elf iii. Gross Payoff ' ' Gross Payoff ' ' NOTE TO TABLE C6 * Economically preferred PM. 73 BIBLIOGRAPHY Allison, PD. 1999. Logistic regression using the SAS system: theory and application. Cary, NC: SAS Institute. Baker, G., R. Gibbons, and K. J. Murphy. 1994. Subjective performance measures in optimal incentive contracts. Quarterly Journal of Economics 109: 1124-1156. Banker, R. D. and S. M. Datar. 1989. Sensitivity, precision, and linear aggregation of signals for performance evaluation. Journal of Accounting Research 27: 21-39. Banker, R. D., Potter, G., and D. Srinivasan. 2000. An empirical investigation of an incentive plan that includes nonfinancial performance measures. The Accounting Review 75: 65-92. Bimberg, J ., J. Luft, and M. Shields. 2006. Psychology Theory in Management Accounting Research. In Chapman, C., A. Hopwood, and M. Shields. (Eds). 2006. Handbook of Management Accounting Research. Volume 1. Oxford: Elsevier. Bol J. and F. Moers. 2006. The Cost-Benefit Trade-Off in Adopting Performance-Based Incentives. Working Paper, Available at SSRN: http://Ssm.com/abstract=772085 Bonner, S. 1994. A model of the effect of audit task complexity. Accounting, Organization and Society 19 (3): 213-234. Broniarczyk, S. M., and J. W. Alba. 1994. Theory vs. data in prediction and correlation tasks. Organizational Behavior and Human Decision Processes 57 (1):] 17-139. Brunello, G. 2002. Absolute risk aversion and the returns to education. Economics of Education Review 21: 635-540. Bushman, R. M., R. J. Indjejikian, and A. Smith. 1995. Aggregate performance measures in business unit manager compensation: The role of intrafirm interdependencies. Journal of Accounting Research 33 (Supplement): 101-134. Cohen. 1988. Statistical Power Analysis for the Behavioral Sciences (2nd Edition). Hillsdale, NJ: Lawrence Erlbaurn Associates. Cohen, A. and L. Einav. 2007. Estimating Risk Preferences from Deductible Choice. American Economic Review 97 (3): 745-788. Cronbach, L. J. 1951. Coefficient alpha and the internal structure of tests. Psychometrika 16: 297-334. 74 Datar, S., S. C. Kulp, and R. Lambert. 2001. Balancing performance measures. Journal of Accounting Research 39 (1): 75-92. Davila, T., and R. Sirnons. 1997. Citibank: Performance Evaluation. Harvard Business School Case #9-198-048. Boston, MA: Harvard Business School Publishing. Davis, M. 1980. A multidimensional approach to individual differences in empathy. JSAS Catalogue of Selected Documents in Psychology 10 (85): Davis, M. 1994. Empathy: A social Psychological Approach. Wisconsin: Brown & Benchmark. Einhom H. J., and R M Hogarth. 1981. Behavioral Decision Theory: Processes of Judgment and Choice. Annual Review of Psychology 32: 53-88. Eisenberg, N. and P. Miller. 1987. The relation of empathy to prosocial and related behaviors. Psychological Bulletin 101 (1): 91-119. Feltham, J. and J. Xie. 1994. Performance measure congruity and diversity in multi-task principal/agent relations. The Accounting Review 69 (3): 429-453. Fisher, G. 1976. Multi-dimensional utility models for risky and riskless choice. Organizational Behavior and Human Performance 17: 127-146. Gertner, R. 1993. Game Shows and Economic Behavior: Risk-Taking on ‘Card Sharks’. Quarterly Journal of Economics 108 (2): 507-521. Gibbs, M., K. A. Merchant, W. A. Van der Stede, and M. E. Vargus. 2004. Performance Measure Properties and Incentives. IZA Discussion Paper No. 135 6. Available at SSRN: hltpsflssrn. com/abstract=51 6 4 82 Gigerenzer, G., Todd, P.M., and The ABC Research Group. 1999. Simple Heuristics That Make Us Smart. Oxford University Press, New York. Goldsteirr, W. 1990. Judgments of relative importance in decision making: Global vs. local interpretations of subjective weight. Organizational Behavior and Human Decision Processes 47: 313-336. Hollenbeck, J. DeRue D. and M. Mannor. 2006. Statistical power and parameter stability when subjects are few and test are many: comment on Peterson, Smith, Martorana, and Owens (2003). Journal of Applied Psychology 91 (1): 1-5. Holmbeck, G. N. 1997. Toward terminological, conceptual, and statistical clarity in the study of moderators: Example from the child-clinical and pediatric psychology literatures. Journal of Consulting and Clinical Psychology 65: 599-610. 75 Hohnstrdm, B. 1979. Moral hazard and observability. The Bell Journal of Economics 10: 74-91. ’ Ittner, C. D., D. F. Larcker, and M. V. Rajan. 1997. The choice of performance measures in annual bonus contracts. The Accounting Review 72: 231-256. Ittner, C., and D. Larcker. 1998. Innovations in performance measurement: trends and research implications. Journal of Management Accounting Research 10: 205-238. Ittner, C. D., D. F. Larcker, and M. Meyer. 2003. Subjectivity and the weighting of performance measures: Evidence from a balanced scorecard The Accounting Review 78 (3): 725-758. Klein, H., J. Hollenbeck, P. Wright, and R. DeShon. 2001. The Assessment of Goa] Commitment: A Measurement Model Meta-Analysis. Organizational Behavior and Human Decision Processes 85 (1): 32—55. Kadous, K., J. Kennedy, and M. Peecher. 2003. The effect of quality assessment and directional goal commitment on auditors’ acceptance Of client-preferred accounting methods. The Accounting Review 78 (3): 759-778. Kendall, MG. 1976. Rank Correlation Methods. 4th Ed. Griffin. Kirkwood, C. W. 2004. Approximating Risk Aversion in Decision Analysis Applications. Decision Analysis 1 (1): 55-72. Krishnan, R., Luft, J. and M. Shields. 2002. Competition and cost accounting: adapting to changing markets. Contemporary Accounting Research 19 (2): 271-302. Krishnan, R., Lufi, J. and M. Shields. 2005. Effects of accounting-method choices on subjective perforrnance-measure weighting decisions: experimental evidence on precision and error covariance. The Accounting Review 80 (4): 1163-1192. Locke, E. and G. Latham. 1990. A Theory of Goal Setting and Task Performance. Englewood Cliffs, NJ: Prentice Hall. Lyons F. and Dan Ben-Ora. 2002. Total Rewards Strategy: The Best Foundation of Pay for Performance. Compensation Benefits Review 34: 34-40. Metrick, A. 1995. A Natural Experiment in ‘Jeopardy!’. American Economic Review 85 (1): 240-253. Muchinsky, P. and Dudycha, A. 1975. Human inference behavior in abstract and meaningfirl environments. Organizational Behavior and Human Performance 13: 377—391. 76 Newell A. and H. Simon. 1972. Human Problem Solving. Englewood Cliffs, NJ: Prentice-Hall. Nisbett R. and T Wilson. 1977. Telling more than we can know: Verbal reports on mental processes. Psychological Review 84 (3): 231-259. Nunnaly, J. 1978. Psychometric theory. New York: McGraw-Hill Payne, J., J. Bettman, and E. Johnson. 1992. Behavioral decision research: a constructive process perspective. Annual Review of Psychology 43: 87-131. Payne, J ., J. Bettman, and E. Johnson. 1993. The adaptive decision maker. Cambridge: Cambridge University Press. Pratt, J. 1964. Risk Aversion in the small and in the Large. Econometrica 32: 122-136. Shah A. and D. Oppenheimer. 2008. Heuristics Made Easy: An Effort-Reduction Framework. Psychological Bulletin 134 (2): 207—222 Shepard, R. 1964. On subjectively optimum selection among multi-attribute alternatives. In Shelley, M. and G. Bryan (Eds). Human Judgments and Optimality. New York: Wiley. Singer, T., B. Seymour, J. O’Doherty, H. Kaube, R. Dolan, and C. Frith. 2004. Empathy for pain involves the affective but not sensory components of pain. Science 303 (5661): 1157-1162. Singer, T. and E. Fehr. 2005. The neuroeconomics of mind reading and empathy. American Economic Review 95 (2): 346-352. Sniezek, J. 1986. The role of variable labels in cue probability learning tasks. Organizational Behavior and Human Decision Processes 38: 141—161. Tversky A., Sattath S. and P. Slovic. 1988. Contingent Weighting in Judgment and Choice. Psychological Review 95: 371-84. Wilks, T. and M. Zimbelman. 2004. Using game theory and strategic reasoning concepts to prevent and detect fraud. Accounting Horizons 18 (3): 173-184. 77 utingling);iii:1