\ ,1 :11; . .a VIEFISITY LIB BRARI IE IIIIIIIIIIIIIIIIIIIIIIIIIIIIII IIIIIIIIII II 3 1293 01029 III This is to certify that the dissertation entitled SYSTEMS DEVELOPMENT FRICTION: A CAUSAL MODEL BASED ON EQUITY CONSIDERATIONS presented by Patricia Ann Essex has been accepted towards fulfillment of the requirements for Ph. D. degree in Accounting ’- Major professor Date November 11, 1993 MS U is an Affirmative Action/Equal Opportunity Institution 0- 12771 LIBRARY Michigan State University PLACE IN RETURN BOX to rem ave this checkout from your record. TO AVOID FINES return on or before date due. ‘ DATE DUE DATE DUE DATE DUE l II + II__ Ifi=lil MSU Is An Affirmative Action/Equal Opportunity Institution anmmH-I SYSTEMS DEVELOPMENT FRICTION: A CAUSAL MODEL BASED ON EQUITY CONSIDERATIONS BY Patricia Ann Essex A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Accounting 1993 ABSTRACT SYSTEMS DEVELOPMENT FRICTION: A CAUSAL MODEL BASED ON EQUITY CONSIDERATIONS BY Patricia Ann Essex In an effort to move information systems (IS) research towards a more sound theoretical base, an M18 Law was proposed by Cushing.* The proposed law states that systems success is inversely related to the amount of friction between user and analyst during systems development and implementation. This dissertation builds and tests a causal model of friction in the IS development environment with an emphasis on the allocation of costs and resources. Friction is seen here as a resistance to IS interaction. Causes of friction are the result of management policies and actions. Friction is hypothesized to decrease with the provision of incentives and increase with higher levels of resource conflict and organizational pressure. Equity theory provides the basis for operational definitions of the three exogenous variables. In the proposed model, the level of friction impacts the quality of user-developer interaction. Interaction quality can be enhanced when top management or other parties utilize countermeasures to deal with friction's adverse effects. The higher the quality of interaction, the more likely the user will have a positive attitude toward the system being developed and will actually employ it when it is implemented. An experiment was conducted to test the developed model. The results of that experiment provide some support for the model, but suggest that the model is not fully specified. A potentially omitted variable, affected by perceived resource conflict, impacts interaction quality without affecting friction. Furthermore, the organizational pressure effect upon the participant's attitude towards the new system is unexpectedly positive, which is opposite the prediction of the model. The friction model and its test supply a framework to aggregate a large amount of prior systems research. Further, it provides a springboard for additional research into the consequences and costs of allowing or mitigating friction. * B. E. Cushing, "Frameworks, Paradigms, and Scientific Research in Management Information Systems," The Journal of Information systems (Spring 1990) pp. 38-59. Copyright by PATRICIA ANN ESSEX 1993 ACKNOWLEDGMENTS I would like to express my appreciation to the members of my dissertation committee: Severin Grabski, Chairperson; Franklin Boster; Susan Haka; and William McCarthy. Their comments and help were very valuable. Special thanks are due Sev Grabski for keeping me actively working on the project and guiding me constantly toward completion. Without the help of the following persons as research confederates, this project would have been impossible: Mark Asman, Antonio Carter, Julie David, Glen Holland, Lambert Lam, Julie MacBeth, Daisy Ng, and Jeff Robinson. Their diligent efforts on my behalf and in the face of their own busy schedules is extremely appreciated. Especial thanks are due my husband, Mark Asman, for driving 750 miles a week during the course of the experiment to provide invaluable support. Joe Anthony, Mark Asman, and Julie David provided insightful comments on the proposal for this dissertation. Their suggestions related to experimental procedures proved to be extremely successful. I am also very grateful to the Department of Accounting for its generous financial support of this dissertation. TABLE OF LIST OF TABLES . . . . . LIST OF FIGURES . . . . LIST OF ABBREVIATIONS . . . CHAPTER 1 - INTRODUCTION . CHAPTER 2.1 Cushing's Proposed MIS Law 0 2 - A MODEL OF IS-DEVELOPMENT The Physical Concept of Friction 2.2 2.2.1 Friction . . . 2.2.2 Adhesion . . 2.2.3 Normal force 2.2.4 lubricant . . 2.2.5 Energy Loss . 2.2.6 Coolant . . . 2.3 The Proposed Model CHAPTER 3 - THEORETICAL DEVELOPMENT 3.1 Roots of Friction 3.1.1 Inputs . . . . . 3.1.2 Outcomes . . 3.1.3 Equity Change Evaluation 3.1.4 Interventions of Managers 3.2 Consequences of Friction vi CONTENTS FRICTION OF CONSTRUCTS xii xiii 10 11 12 13 14 18 18 20 22 24 26 27 CHAPTER 4 - EXPERIMENTAL DESIGN . . . 4.1 Research Design . . . . . . . 4.2 Experimental Setting . . . . . 4.2.1. Participants and Their Task 4.2.1.1 Recruiting Procedures and Results . . . . . . . 4.2.1.2 Early Session Procedures 4.2.2 Confederates and Their Task 4.3 Operational Variables . . . . 4.3.1. Resource Conflict . . . . 4.3.2 Organizational Pressure . 4.3.3 Perceived Treatments . . 4.3.4 Friction . . . . . . . . 4 . 3 . 5 Quality of Interaction . 4.3.5.1 Development of Instrument 4.3.5.2 Analysis of the Interaction Quality scale 0 O O O O O O 0 4.3.6 Attitude as a Measure of System Success . . . . . . . . . 4.3.6.1 Attitude Scale Development 4.3.6.2 Analysis of the Attitude Scale 4.4 Summary Measures of Interaction Quality Attitude . . . . . . . . . . . 4.5 Additional Manipulation Checks CHAPTER 5 - RESULTS . . . . . . . . . 5.1 Path Coefficients . . . . . . 5.2 Goodness of Fit . . . . . . . 5.2.1. Tests using the Basic Variable Definitions . . . . . . . vii 31 31 34 35 35 38 40 43 45 47 48 53 59 60 65 68 70 72 73 77 82 85 89 91 5.2.2 Tests with Alternative Variable Definitions . . . . . . . 5.3 Measurement Error . . . . . . 5.4 Possible Explanations for the Findings . 5.4.1. Experimental or Statistical Artifacts . . . . . . . . 5.4.1.1 Restriction in Range 5.4.1.2 Measurement Error in 5.4.2 IMisspecified Model . . . CHAPTER 6 - DISCUSSION . . . . . . . . 6.1 Summary of Project . . . . . . 6.1.1. Model Construction . . . 6.1.2 lModel Testing . . . . . . 6.1.3 Construct Development . . 6.2 Major Conclusions . . . . . . 6.3 Other Notable Observations . . 6.3.1 Individual Differences . 6.3.2 Equity Adjustments . . . 6.4 Implications . . . . . . . . . 6.4.1. Research Implications . . 6.4.2 Practice Implications . . 6.5 Research Extensions . . . . . LIST OF REFERENCES . . . . . . . . . . Friction .APPENDIX A; "THE MATERIALS WORKSHOP" RELEASES: RECRUITING AND DEBRIEFING MATERIALS Exhibit 13 Announcement . . . . . Exhibit 2: Application . . . . . . Exhibit 3: New Employee Letter . . viii 95 97 100 101 101 102 104 109 109 109 110 111 114 116 117 119 120 120 121 122 124 127 127 128 130 Exhibit 4: .Accounting Lab Assignment Form . . . 132 Exhibit 5: Compensation Agreement . . . . . . . 133 Exhibit 6: Debriefing Letter . . . . . . . . . . 134 APPENDIX B: EXPERIMENTER'S SCRIPT . . . . . . . . . 136 APPENDIX C: SYSTEMS LAB INSTRUCTIONS AND MATERIALS . . . . . . . . . . . . . . . 138 Exhibit 1: Orientation Meeting Memo . . . . . . 138 Exhibit 2: Systems Lab Materials . . . . . . . . 139 Exhibit 3: Systems Interaction History . . . . . 152 APPENDIX D: DATA COLLECTION INSTRUMENTS . . . . . . 155 EXhibit 1: Systems Lab Assignment Form-- case A O O O O O O O O O O O O O O 0 155 Exhibit 2: Systems Lab Assignment Form-- case B O O O O O O O O O O 0 O O O O 157 Exhibit 3: Systems Lab Assignment Form-- Case C . . . . . . . . . . . . . . . 159 Exhibit 4: Systems Lab Assignment Form-- case D O O O O C O O O O O O O O O O 161 EXhibit 5 : EXit Form 0 O O O O O O O O O O O O O 163 ix TABLE TABLE TABLE TABLE TABLE TABLE TABLE TABLE TABLE TABLE TABLE TABLE TABLE TABLE TABLE TABLE 1: 2: 3: 4: 5: 6: 7: H I-' 15: 16: LIST OF TABLES MODEL CORRESPONDENCE AND VARIABLE DEFINITIONS . . . . . . . . . . . . RESEARCH DESIGN . . . . . . . . . . . PARTICIPANT DEMOGRAPHICS BY ASSIGNED CASE . . . . . . . . . . . . . . . . . PARTICIPANT DEMOGRAPHICS BY ASSIGNED CONFEDERATE . . . . . . SUMMARY OF EXPERIMENTAL TREATMENTS . . RESOURCE CONFLICT BY PERCEIVED RESOURCE CONFLICT . . . . . . . . . . . . . . . ORGANIZATIONAL PRESSURE BY PERCEIVED ORGANIZATIONAL PRESSURE . . . . . . . ASSIGNED TREATMENT BY PERCEIVED TREATMENT . . . . . . . . . . . . . PARTICIPANT DEMOGRAPHICS BY PERCEIVED CASE . . . . . . . . . . . . . . . . . FRICTION BY ASSIGNED CASE . . . . . . FRICTION BY PERCEIVED CASE . . . . . . INTERACTION QUALITY FACTORS . . . . . INTERACTION QUALITY CONSTRUCT VALIDATION . . . . . . . . . . . . INTERACTION QUALITY INTERCORRELATION AND LOADING MATRIX . . . . . . . . . MOORE- BENBASAT FACTORS AFFECTING IS-ATTITUDE . . . . . . . . . . . . IS-ATTITUDE INTERCORRELATION AND LOADING MATRIX . . . . . . . . . . . . . . . . 15 32 37 44 49 50 52 52 54 57 58 61 64 67 71 74 TABLE TABLE TABLE TABLE TABLE TABLE TABLE TABLE TABLE TABLE TABLE TABLE 25: 26: 27: 28: INTERCORRELATIONS AMONG FACTOR SCORES AND OVERALL INTERACTION QUALITY MEASURES . . . . INTERCORRELATIONS AMONG FACTOR SCORES AND OVERALL IS-ATTITUDE MEASURES . . . . . . . . POST-EXPERIMENTAL MANIPULATION CHECKS . . . DESCRIPTIVE STATISTICS FOR EXPERIMENTAL MEASURES O O O O O O C O O O O O O O O O O 0 THE MODEL'S MEASURED VARIABLES CORRELATION MATRI x O O O O O O O O O O O O O O O O O O DIFFERENCES BETWEEN PREDICTED AND OBSERVED CORRELATIONS . . . . . . . . . . . . . . . . CONFIDENCE INTERVALS AROUND OBSERVED CORRELATIONS . . . . . . . . . . . . . . . . SUMMARY OF STATISTICAL TESTS OF MODEL USING ALTERNATIVE DEFINITIONS OF INTERACTION QUALITY AND IS-ATTITUDE . . . . . . . . . . THE MODEL'S MEASURED VARIABLES CORRELATION MATRIX--CORRECTED FOR MEASUREMENT ERROR IN INTERACTION QUALITY AND IN IS-ATTITUDE . . . SENSITIVITY ANALYSIS OF THE RESULTS TO MEASUREMENT ERROR IN FRICTION . . . . . . . ANALYSIS OF INTERACTION QUALITY MEASURES ON THE PERCEIVED RESOURCE CONFLICT SUB-MODEL . . . . . . . . . . . . . . . . ATTITUDES OF PARTICIPANTS ABOUT PROSPECTIVE SYSTEMS DEVELOPMENT ACTIVITY . . . . . . . . xi 75 76 78 83 84 93 94 96 99 103 106 120 FIGURE FIGURE FIGURE FIGURE FIGURE 1 A CAUSAL MODEL OF CUSHING'S PROPOSED MIS 2: 3: 4: 5: LIST OF FIGURES LAW 0 O O O O O 0 PROPOSED MODEL OF IS-DEVELOPMENT FRICTION . . . . . . MODEL AS TESTED . . OBTAINED RESULTS . . SUB-MODEL FOR PERCEIVED RESOURCE CONFLICT TO INTERACTION QUALITY xii 16 32 90 105 CFA IQ IS OI OP RI RC LIST OF ABBREVIATIONS Attitude toward the information system Confirmatory factor analysis Residual (error) from subtracting observed Eorrelation from predicted correlation for variable Friction Interaction Quality Information System (As a subscript) Other entity Organizational Pressure Perceived Organizational Pressure Organizational Pressure (As a subscript) Participant Path coefficient between variables j and i Perceived Organizational Pressure Perceived Resource Conflict Interaction Quality Observed correlation between variables i and j Predicted correlation Correlation corrected for measurement error Resource Conflict Perceived Resource Conflict Resource Conflict xiii SLA Systems Lab Assignment [form] TMW' The Materials Workshop US User satisfaction xiv CHAPTER 1 - INTRODUCTION The systems research literature is replete with studies investigating the reasons why an information system (IS) is or is not successful. Much of the work has focused upon the involvement of the user in IS design and provides mixed evidence as to whether increased involvement helps or hinders systems success (Wong-On-Wing, 1988; Ives and Olson, 1984). Cushing (1990) reviews the progress of the IS field with respect to the standards of scientific research. He concludes his review by proposing that friction between systems personnel and users, rather than lack of user involvement, is associated with systems that are unsuccessful. In this study, the physical concept of friction was used to develop a behavioral model of how discord between users and systems personnel (hereafter, developers) relates to IS success during the system development cycle. An experiment was conducted to test the model, and path analysis provided the means of evaluating the data. Equity theory supplied the basis for defining variables that produce friction and for predicting the reactions of an IS user to a situation characterized by friction. Friction was examined as a situation-specific concept. The level of friction may be mitigated by individual factors, such as past experience with group tasks or affinity for computerized work environments. However, these individual 2 factors were beyond the scope of this research. This model of friction can be utilized by managers and developers to anticipate the occurrence of friction, to develop plans to modify its level, or to manage any adverse consequences of realized friction. Friction is interpreted in this model as a resistance by potential system users or developers to interaction with one another on an IS development project. Friction is predicted to arise out of situations where users and developers have resource conflicts or where organizational pressure exists. Resource conflicts occur when the parties disagree about the use of personally controlled resources, such as time and knowledge. Organizational pressure exists when the costs of development are disproportionately borne by one party as compared to some other party who receives benefits from the new system. Management plays an important role in the model through the assignment of resources, the provision of incentives, and the structuring of performance contracts. Factors contributing to system success are important both to users and to developers. Accountants are one group of users particularly concerned with IS success. The general objective of accounting is the provision of timely, relevant information for decision making. In many firms, accountants were the first user group to participate in IS development. In most firms, accountants are still heavily involved. 3 Accounting systems are sometimes changed to provide for the information needs of non-accountants. Such situations arise when the data needed by non-accountants can be collected simultaneously with accounting data. Thus, IS changes can force the accountant to bear substantial costs related to the transfer of specific knowledge about the current system while another entity reaps substantially all the benefit accruing to the new system. When such inequities are perceived, the accountant is subjected to friction as a user. Alternatively, the accountant acting as an IS developer can generate friction felt by others. A proposed accounting information system that collects new performance data may produce a perception of threat to non-accountants. Perceived threat has been shown to affect the way a user behaves during system development (Newman and Sabherwal, 1991). User resistance to a project bearing a perceived threat is an understandable response. The research proposed, therefore, is important to accountants who are involved with information systems either as users or as developers. The model developed herein supplies a theoretically-based foundation for future IS research. In addition, the results of this study carry implications for wage contracts, incentives, and actions of top management. The remainder of this dissertation is organized as follows. Chapter 2 of the paper derives a model of friction 4 from Cushing's (1990) discussion and from physics. In Chapter 3, the causes and consequences of friction are theoretically extended. Chapter 4 describes the research design for testing the model, and the results of the experiment are discussed in Chapter 5. Finally, conclusions, limitations, and future research are addressed in Chapter 6. CHAPTER 2 - A MODEL OF IS-DEVELOPMENT FRICTION This chapter develops a model of friction as it applies to the information systems development environment. The model was inspired by Cushing's (1990) proposed MIS Law and was refined using concepts derived from two arenas: the physical concept of friction and the psychological concern of comparative equity. 2.1 Cushing's Proposed MIS Law IS research has been criticized for its lack of theoretical grounding (Sethi and King, 1991; Cushing, 1990; Wong-On-Wing, 1988). Cushing (1990) reviews the state of management information systems (MIS) research in an effort to assess the development of MIS as a scientific research discipline. He concludes that, although a consensus with respect to the appropriate domain of MIS research has been achieved, no particular paradigm has yet emerged from the MIS literature. In an effort to drive the MIS research discipline toward a higher evolutionary state, Cushing proposes the following MIS Law: The success of any MIS will tend to be inversely related to the degree of friction that exists between MIS users and MIS developers during the processes of development and use of the MIS (p. 51). The appropriate measures of MIS success are not specified by Cushing. Furthermore, neither a definition nor a measure of friction currently exists in the MIS literature. 6 Consequently, the proposed law has yet to be submitted to a direct, empirical test. Without attaching a specific meaning to the term friction, Cushing asserts that friction between user and developer is inherent in the systems environment and is present during both the development and the use stages of the system life cycle. Friction is viewed as a function of user and developer characteristics, MIS characteristics, and factors found in the organizational environment. According to Cushing, friction is caused primarily by the fact that developers and users have different goals. From an organizational perspective, developers have a technical goal while users have a functional goal. The prevalence of this basic goal incongruence is described astutely in a recent article on improving user-analyst communication (Christensen, 1991): Usually, the technology experts focus on speed and power, which is pretty interesting, considering the primary concern of most users is usability and efficiency (p. 72). Cushing suggests that friction affects system success indirectly through its impact upon the quality of interaction between the user and the analyst. He states that management policies can reduce friction, acting as lubricants, and thus smooth the user-analyst interactions. However, friction is not expected to be entirely eliminated, even with the use of such policies. 7 From Cushing's description of the proposed MIS Law, a causal model can be drawn. It is shown in Figure 1, where the signs beside each path indicate the predicted sign of the correlation between the connected variables. User- developer differences are expected to cause friction although it can be mitigated if management policies are in force to make interactions go more smoothly. Cushing's proposed law states that friction negatively affects the quality of user-developer interaction. Finally, the higher the quality of user-developer interaction, the more likely that the system will be successful. 2.2 The Physical Concept of Friction Friction in the IS environment has not been defined by prior research. However, a parallel to the IS environment can be found in the physical concept of friction. Reviewing the physical concept of friction is helpful in the development of the friction model. 2.2.1 Friction Friction is defined as "the resistance to sliding that originates at the boundary between two contacting solids." (Encyclopedia Americana, 1991, p. 90.) This definition requires that two entities be touching and that movement be involved. In a systems development project, the user and the developer are the two contacting entities. Since change to an IS captures the notion of movement, resistance to >33 92 QmmOmOmn— $025030 ".0 .5002 .2916 < up mmDOE mwmoozm Ego $530.. 556 A 3 20.55.32. A 3 20:03“. A 3 pzwsmom0-¢wwD 9 interaction on a proposed IS development project seems comparable to the resistance to sliding displayed by physical objects. Friction is inherently neither good nor bad. Just as the physical world relies on friction for such events as stopping a car, the firm may benefit from friction's power to slow IS development. Friction, the resistance to interaction on IS development projects, can help ensure that IS projects are not undertaken when no user in the firm values the project. Similarly, friction may aid developers in determining which projects to undertake when their resources are limited. However, if an IS project with high levels of friction proceeds into the development stage, negative repercussions during user and developer interaction are a very likely result. 2.2.2 Adhesion The major cause of friction, as explained by adhesion theory, is the presence of highly stressed spots on the contacting surfaces. Adhesion, the interaction at these spots, usually explains over 80% of the frictional force. At these spots, the surface atoms merge and create very strong bonds. The bonding of the surface atoms can be considered a result of two solids striving to capture space, with each solid taking what space it can, an atom at a time. In other words, bonding arises out of the contention for resources (space) demanded by two entities (the solids). 10 In the IS development environment, friction can also be caused by disagreements over resources. The resources required to be supplied by people interacting on an IS project include time, effort, and knowledge. The developer working on a new IS wants the user to spend time clarifying system requirements using specific knowledge of the domain area, something a user may be uniquely qualified to do. The user, on the other hand, likely wants the developer to contribute a great deal in terms of knowledge about systems and to spend adequate time on the development effort so as to bring it to implementation in a timely manner. Friction arises, therefore, whenever the resources desired by one party are not willingly provided by the other. In the proposed model, this contribution to friction is called Resource Cbnflict. 2.2.3 Normal force A second factor affecting friction in the physical sense is the load, called normal force, that presses the two surfaces together when they are stationary. This load usually is measured by gravity or weight. Friction is proportional to the normal force. In other words, heavier loads pressing two surfaces together require more force to initiate and sustain movement. The load or external pressure that bears upon the contacting surface is independent of the surface area. Thus, in a sense, load is 11 an autonomous agent. It does not enter directly into the bonding process but does affect it. Similar events occur in the IS environment. Consider the user and the developer as the contacting surfaces. External pressures--particularly the need for the user to supply knowledge about the task environment--often push these two agents together, and the amount of external pressure varies from one situation to another. Examples of such pressures include demands by a non-accounting department for accounting information in a new format or a request by the external auditor for a higher level of accounting control. In the proposed model, this concept of load or pressure is labeled Organizational Pressure, and it is a representation of the amount of pressure placed by other parties upon the user or the developer to become involved in IS development. 2.2.4 Lubricant Physical friction can be reduced by the application of a lubricant between the contacting surfaces. The lubricant reduces the bonding action between the solids and increases the ease with which sliding can be initiated. Common examples of lubricants are water and oil. Lubricants are especially beneficial when friction is likely to cause an unwanted loss of energy, such as in a machine that runs continuously. 12 In the IS environment, lubricant can be described as the provision of incremental resources in anticipation of development interaction. These resources might be supplied to either the user, the developer, or both. Commonly provided lubricants, described as Incentives in the model, include extended deadlines for functional tasks and budgetary slack to allow the hiring of temporary help. 2.2.5 Energy Loss The by-product of friction in sliding surfaces is a loss of energy, often appearing as heat. The heat is absorbed by the surfaces and affects the way in which they interact. In some cases, this energy loss is considered to be a desirable product, such as when a person rubs his hands together to warm them. In many situations, however, this loss of energy is considered a serious problem which can cause damage to the solids' surfaces as well as to the mechanical interaction process. As the user and developer work together to create an IS, friction between them can be revealed in a number of ways--perhaps through heated argument or wasted effort. Symptoms of friction, such as conflict, can be positive occurrences if they lead to more discussion and better IS design. On the other hand, conflict can also result in negative outcomes, particularly when one of the parties refuses to seek mutually acceptable terms. As Cushing (1990) noted, conflict is just one manifestation of 13 friction. Others might include passive resistance (such as presenting the appearance of participation in the development effort but without participating in fact) or even sabotage of the IS. Friction, as defined above, is a resistance to interaction and accordingly has a negative impact on IS development interaction. This impacted variable is called Interaction Quality in the model. 2.2.6 Coolant When an excessive loss of energy is imminent in a physical system, the system is often supplied with a coolant to reduce the amount of heat absorbed by the solids. Oil and water, listed earlier as examples of lubricants, also are common coolants. Coolants can be distinguished from lubricants by their timing. A lubricant is the provision of additional resource in anticipation of interaction; its presence keeps some friction from arising. A coolant may or may not increase the supply of resource available to the entities, and it is not applied in order to avoid friction. Rather, a coolant is applied after friction has generated heat or other by-products and is aimed at reducing the negative effect of the by-product. When interactions between user and developer in the IS environment become negatively heated, members of the organization can employ a variety of techniques to calm the parties and reduce undesired expenditure of resources. These techniques are called Cbuntermeasures in the model. l4 Probably the most common countermeasure used is top management involvement in the IS project; user sign-off on important decisions is another. Table 1 summarizes the terms used in the physical explanation of friction. The corresponding terms used in the systems development model are listed along with the definitions used in this study. 2.3 The Proposed Model The proposed model of IS development friction is shown in Figure 2. It has been developed using Cushing's framework (Figure 1) as a base and refining the terms and linkages in light of physical friction concepts. Note that all three variables having an impact upon friction (resource conflict, organizational pressure, and incentives) are controlled by managers. They approve the terms of compensation packages and control the basis of performance evaluations. These evaluations promote certain user—developer differences and can foster disagreements about the use of personally controlled resources. The presence of resource conflict increases the level offriction. Also generating increased friction is the presence of organizational pressure. This pressure occurs when management allocates resources (or circumstances bestow the benefits of a new project) disproportionately from the distribution of the project's costs. To alleviate friction by altering resource allocations, incentives can be 15 TABLE 1 MODEL CORRESPONDENCE AND VARIABLE DEFINITIONS PHYSICAL MODEL TERM SYSTEMS MODEL TERM DEFINITION USED IN THIS STUDY Friction Friction Resistance to interaction on the IS project. Adhesion Resource Conflict Contention between user and developer over the use of resources. Such conflicts arise because one of the participants is facing a potential drop in net equity. Normal Force Organizational Pressure Pressure placed on the participant by others who stand to benefit from the new system (disproportionately as compared to the participant). Lubricant Incentives Provision of incremental resources by an external party to encourage participation in the proposed project. The timing of incentives is after the initial proposal of the new IS. Examples include budgetary slack, bonuses, temporary help, and the promise of job promotion. Energy Loss Interaction Quality Quality of interaction between user and developer during IS development. This assessment deals with the process of development and is a retrospective or concurrent evaluation. Coolant Countermeasures Practices that reduce the negativity of interactions. Can be provided by management, developers, or users. Timing is after onset of the system development interaction. Examples include top management involvement, user sign-off, and the avoidance of computer jargon. 16 ZOFOEH... Hzm2a04w>mo w. “.0 swan—OS. omwOaOmn. ”N meOE mmewwm—Q S ..__.rzm02_ ._.03n_ZOO momDmem 17 provided. Friction has a negative impact upon interaction quality, but this impact may be offset by the use of countermeasures. Finally, the model indicates that the success of the system is a function of the quality of user- developer interaction. CHAPTER 3 - THEORETICAL DEVELOPMENT OF CONSTRUCTS The physical model of friction was applied to the information systems environment in the previous chapter. In this chapter, the IS-development friction model is interpreted so as to be applicable to human interactions. The interpretation is facilitated by the psychological concept of comparative equity. 3.1 Roots of Friction Equity theory suggests that a person considers a proposed change, such as a new IS, in two different lights (Adams, 1965; Walster et al., 1976; Joshi, 1991). First, a person decides whether the change is likely to increase or decrease current personal equity by comparing the potential outcomes from the change to the required inputs. Outcomes are defined as the positive and negative consequences of taking part in a relationship with another. Inputs are a person's contributions to the interaction that entitle that person to be the recipient of the outcomes. Equity theory's Proposition I (Walster et al., 1976) states that people try to maximize personal net outcomes or equity. If this is so, each individual asked to interact on IS development projects considers the expected future change in personal equity when determining the quantity of resources to commit to the proposed project. If the anticipated change in equity is negative, the potential participant declines to commit 18 19 personal resources and a resource conflict is likely to arise. The second type of equity analysis is concerned with comparative changes in equity. The expected change in personal outcomes, considering personal inputs, is compared to that of others who participate in or are affected by the proposed IS project. Two likely comparison groups when equity theory is applied to employee-employer situations are the organization itself (or its owners) and other employees of the organization. The participant determines the situation to be inequitable if the following relationship is perceived to hold (Adams, 1965)1: Outcomesp Outcomeso (1) Inputsp .Inputso where p represents the potential participant, and 0 represents some other party. Inequity resulting from an equity ratio that is lower than that of others is considered to be organizational pressure in the model. A person faced with such an inequity feels higher levels of friction. In other words, the person is more resistant to interaction. Consequently, the party who benefits disproportionately exerts pressure (directly or indirectly through others) on the potential participant to 1Walster et a1. (1976) present a more complex formula to represent equity in cases where inputs of one or more of the parties do not represent positive costs, such as cruelty in social relationships. This dissertation assumes that inputs are positive in the IS development environment, as does Joshi (1991). 20 donate personally controlled resources to the development project. Project interaction is not avoided, however, when the direction of the inequity favors the potential participant. Instead, the participant attempts to adjust the equation components in some way so as to make the equality hold. Such actions are discussed in Part 3.2. 3.1.1 Inputs Several different groups of people are classified as users of an IS--those who handle input to the IS, those who perform processing tasks, and those who utilize the IS output. An IS can be proposed that requires the involvement of one, several, or even all of these people. Likewise, one or more of them may receive the benefits from IS use. The proposed model considers the target user to be a non- developer who would hear input costs in the proposed IS development regardless of the group to which the user belongs. The major input cost incurred by this user is assumed to be time, which is a function of both knowledge and effort. IS development requires knowledge about IS technology as well as knowledge about the tasks of users being affected by (or affecting) the IS. The amount of knowledge required about users' tasks can vary. Some tasks utilize a great deal of knowledge that only the user holds. This knowledge is called specific knowledge. Other tasks are more routine; 21 many people perform the task in about the same way. Task knowledge in such situations is likely to be considered general knowledge. In a systems development project, many decisions about the IS are delegated to the developer. If the task environment utilizes general knowledge, the developer may be able to develop the IS with little input from a particular user. Often, however, specific knowledge about the user's task is necessary for the success of the system. The developer has two choices with respect to specific ”knowledge: (1) learn the task well enough to acquire the needed level of specific knowledge directly or (2) ask that the user reveal specific knowledge by participating in the IS development project. Jensen and Meckling (1992, p. 255) note that "specific knowledge . . . is often acquired jointly with the production of other goods." Accordingly, the user's cost to acquire specific knowledge of the task environment is likely to be very low. The cost for the developer to acquire it is almost certainly much higher. Thus, if the IS development project is judged to require specific knowledge about the user's task, the developer will urge the user to participate in the IS development project. The user, however, bears a cost in addition to the acquisition cost (which is assumed to be minimal.) This second cost is the cost of transferring the knowledge. The 22 transfer cost can seem high relative to benefit from the transfer. Jensen and Meckling (1992) explain: After the fact, it is often obvious that a specific piece of knowledge critical to a decision could have been transferred at low cost . . . . But transferring this specific piece of knowledge in advance requires knowing in advance that it will be critical. Without such clairvoyance, transfer of the fact must occur as part of a larger and more costly-to-transfer body of knowledge, most of which will never be used. The expected cost of transferring that larger body of data, not the particular fact, is the relevant transfer cost. (p. 255) The developer is assumed to be able to estimate the quantity of resource the user needs to commit to the project with a reasonable degree of accuracy. The estimate is keyed to the amount of specific knowledge needed to develop the IS successfully. The developer communicates the input estimate to the user at the start of the project.2 3.1.2 Outcomes For participants, IS project outcomes can include positive outcomes such as increased salary or lighter workloads as well as negative outcomes like unwanted changes or increased task complexity. In the proposed model, the most important outcome for participants in an IS project is 2Baiman and Lewis (1989) demonstrated that agents will communicate nontruthfully if doing so is of benefit to them. In the IS development environment, the estimate is assumed to be communicated truthfully. The developer has little incentive to overestimate the user's resource expenditure. If the estimate is high relative to the user's outcomes, the user would refuse to interact on the project. Similarly, an underestimation of the needed resource commitment can be counterproductive. The user is likely to consider the input estimate as an upper limit and may view a later request for additional resources with dismay, if not outright rebellion. 23 assumed to be the impact of the new system on the participant's expected future compensation. Firms typically pay most developers and many users a salary set at the beginning of the contract period. The contract (whether oral or written) may contain a provision for salary revision in the next period, provided that the worker continues in the firm's employment. Salary changes often depend upon both the firm's overall performance and the employee's performance evaluation. Such a pattern is typical of situations where annual raises in contract salaries include an "across-the-board" portion and a "merit" amount. This salary structure is presumed to motivate workers to select actions that will enhance the value of the firm (Baker, Jensen, and Murphy, 1988). The basis for performance evaluation could be a major factor creating differences between developers and users. Generally, the performance evaluation considers the worker's resource expenditure upon both functional and non-functional tasks. The task of IS development is a functional task of the developer. However, IS development falls in the category of non-functional tasks of the user. Using Antle and Demski's (1988) criterion for controllability, the variable in the salary revision function that is probably considered more highly controllable by an employee is the performance evaluation (Baker, Jensen, and Murphy, 1988). Further, an employee is likely to be more skilled in the functional task than in the 24 non-functional task. Thus, functional performance reasonably is perceived as weighing more heavily in that evaluation than non-functional activity (Dye, 1992). A user anticipates whether the present IS can handle functional tasks in the future as well as it does currently. If a new information system is perceived as necessary to the achievement of the desired level of functional performance, a user can be expected to actively promote its development. However, the user may be less than cooperative on IS development projects that are not expected to contribute to (may even be perceived as detrimental to) the accomplishment of assigned functional duties. 3.1.3 Equity Change Evaluation The potential participant's beliefs about expected outcomes and inputs, along with other information gathered from a variety of sources, constitute private information that is used to determine the level of personal resource to commit as inputs to the proposed IS project. A high level of input is chosen only if doing so is expected to shift the probability distribution of the firm's earnings or the employee's own performance evaluation in such a way that the outcomes are greater than the resource cost. Otherwise, a lower input level is chosen by the participant.3 3Potential participants are expected to act as if they evaluate their potential inputs and outcomes in terms of present values. Thus, if a user is close to retirement and the IS is not expected to be operable for several years, the resource cost may substantially outweigh the benefit. If the user expects to be in the same functional capacity 25 If the amount committed by the user differs from the amount requested by the developer, a resource conflict arises. Similarly, the user may perceive that the developer needs to devote resources to a proposed project in a quantity that exceeds the amount the developer intends to commit.4 If the developer does not commit the incremental resources, a resource conflict also exists. Resource conflict in the model encompasses conflicts over both user- or developer-controlled resources. Not only does a potential participant examine personal outcomes and inputs, but those of others are evaluated, too. Thus, even when a person desires a new IS due to an increase in net equity, friction can arise because of perceived inequitable distributions of outcomes compared to inputs. The participant's expectation that benefits of the proposed IS will be disproportionately garnered by someone else is captured as organizational pressure in the model. for a long time, the benefits likely are given a higher value. Because IS development costs must be borne in the near future and IS benefits could be years away, the net present value approach automatically assigns equal utils of cost more weight than benefit. In addition, since input costs can often be estimated much more accurately than outcomes, the probability associated with future costs is likely to be higher than the probability associated with future outcomes. Thus, both the time differential and the certainty calculations cause development costs to be more highly weighed in the equity analysis than if the approach were not done on a present value basis. ‘In firms with MIS steering committees, the developer may have less control over the choice of projects. However, the developer does consider personal changes in equity when setting his or her own task priorities. 26 3.1.4 Interventions of Managers The bases for the organizational pressure and resource conflict variables are determined by management when they define performance evaluation criteria, approve IS development efforts that cut across departmental lines, and distribute rewards to members of the organization. If management realizes that IS activity is important to the firm and wishes to minimize resistance of participants, one or more incentives can be offered to them. Incentives operate by increasing the participant's outcomes or by supplementing the participant's resource supply (effectively reducing input costs.) Since incentives typically are costly to the firm, they are supplied only when friction between the user and the developer must be reduced. The incentives variable in the model measures the presence of resources (supplied by some other party) that are available only if the development project is undertaken. The participant does not know about the availability of these resources at the time of initial evaluation of the IS project; otherwise, the participant would incorporate the value of the incentive into the above outcomes to inputs analysis. Earlier studies (Sen and Yardley, 1989) that have examined budgetary slack or allocation schemes have, in effect, studied the resources provided via incentives in order to reduce or avoid friction. Such mechanisms also have been used to increase friction (by decreasing user 27 resource allotments) in order to moderate and control demands made upon IS resources (Henry, 1990). 3.2 Consequences of Friction As discussed earlier, friction can ensure that unnecessary IS development is not undertaken and can improve the allocation of scarce resources. Accordingly, the full elimination of friction is not a goal for most firms. High friction levels in ongoing projects, however, are likely to affect interaction quality adversely because such projects force participants into perceived inequitable relationships. When a person is involved in an inequitable relationship, distress is the predicted result (Walster et al., 1976). This reaction may take the form of anger if the person feels less rewarded than some reference group, or it may take the form of guilt if the person is over-rewarded relative to others. Propositions III and IV of equity theory (Walster et al., 1976) state that the higher the level of inequity, the more distress is felt and the more vigorously a person seeks to restore equity. This process is known as dissonance reduction. It can be accomplished by changing either actual or perceived outcomes and inputs. In the case of a resource conflict, dissonance reduction must address the participant's outcomes and inputs. However, in the case of organizational pressure, the outcomes and inputs of the firm or the reference group can also be addressed. 28 A number of practices can be utilized for dissonance reduction. Countermeasures can be undertaken by management, by other parties to the relationship, or even by the potential participant. For example, the user may have determined that personal outcomes will be less than inputs and accordingly is disinclined to participate on the IS project. If pressured to interact, the user can reduce distress by decreasing personal inputs or increasing personal outcomes. Increasing actual outcomes at will may be difficult, but reducing actual inputs is often relatively easy. Participating in appearance only, shirking duties, or being more efficient with resources all diminish the cost of input. Management in organizations with successful IS typically provide some form of support during IS development. This support can be characterized as involvement or participation (Jarvenpaa and Ives, 1991), and such activities are costly to the manager in differing degrees. Participation occurs when managers personally intervene in the IS project. Involvement is less concerned with IS-related actions than with the manager's psychological state with respect to IS activity. When management support takes the form of a countermeasure (as both participation and involvement generally do), the inequity felt by a user as a result of comparing personal equity changes to those of the firm or its owners will be reduced, all else being equal. 29 Although participation is probably the more costly action for a manager, it likely changes only one component of the equity equation (Eq. 1): inputs of others. On the other hand, manager involvement can operate by changing the user's perceptions of personal outcomes, personal inputs, other's outcomes, or other's inputs. For example, the manager might emphasize in discussions with users the firm's provision of special training classes (other's input) that will increase the level of IS-related skill of the user (participant's outcome) and the potential for promotion due to the new skill (participant's outcome). For a given level of management resource expenditure, top management potentially can alter the perceived levels of up to four variables in the equity equation through involvement and to encourage positive interaction results quite efficiently. Although distress from being under-rewarded is expected to be greater than that from being over-rewarded, both types of inequity have been shown to influence attitudes and to produce actions to alleviate the distress (Adams and Freedman, 1976.) While participants are expected to experience friction due to inequity that favors others, they are not expected to display friction (that is, to resist interaction) in the opposite case. However, actions may be taken by participants during development that are aimed at alleviating perceived inequity. Examples of dissonance reduction actions are offering side payments or accepting 30 more input cost than originally deemed necessary. Such actions also are countermeasures. Whether inequity favors the participant or favors others, adjustments to outcomes or inputs affect the model construct called interaction quality. This construct is an appraisal of the user's or the developer's attitudes with respect to the process of systems development. It is a retrospective or concurrent evaluation that is expected to be affected by levels of friction inherent in the project's environment and by the presence of countermeasures. As previously discussed, interaction quality is expected to have a direct and positive impact upon system success. CHAPTER 4 - EXPERIMENTAL DESIGN This chapter describes an experiment that tested part of the conceptual model developed in the previous chapters. In the process of designing the experiment, several changes were made to the model. These adaptations are described below, and the tested model is shown in Figure 3. (a) Two variables were included to aid in testing the validity of the experimental manipulations. These variables are the perceived levels of resource conflict and organizational pressure. (b) The countermeasures construct was omitted in order to reduce the number of cells in the experiment. This was necessary in order to acquire an adequate number of subjects per cell. (0) System success was measured by examining the user's attitude toward the IS being developed. The rationale for using this measure for systems success is discussed in Part 4.3.6. (d) The incentives construct was omitted. Pilot testing of a related experiment indicated that the impact of reasonable incentives upon friction was less significant than the effects of resource conflict and organizational pressure. 4.1 Research Design To test the adapted model, a laboratory experiment of actual IS development interaction was conducted. A laboratory experiment was chosen over a field study because of a need for increased control over many extraneous variables that affect the IS development process. In particular, potential effects of pre-existing attitudes 31 32 om...th w< JmDOE ”m mmDOE works? A 3 >._.3_momma E 3 20.5.”... 3 531200 momaowmm AI anbWwwOmw om>_mommn_ E 33 related to prior experiences between the user and developer could be minimized. The presence of such experiences could seriously obscure conclusions about friction during IS development by introducing the complications of a friction- in-use model. A between-subjects design was utilized due to the length of the experiment (approximately three hours.) The model was tested with an orientation toward the viewpoint of the user because it is the user's favorable attitude and satisfied IS use that developers must promote. Table 2 illustrates the research design. Two levels of resource conflict (low and high) and organizational pressure (low and high) were combined to create four experimental treatments. Resource conflict was manipulated by varying the cost to the user of spending time on IS development. Organizational pressure was varied by changing the reward given to a comparable other person for the same task. Both manipulations are described fully in Part 4.3. TABLE 2 RESEARCH DESIGN ORGANIZATIONAL PRESSURE RESOURCE CONFLICT Low . High Low Case A I Case B High Case C Case D 34 Measures of the five endogenous variables--perceived resource conflict, perceived organizational pressure, friction, quality of interaction, and attitude toward IS-- were gathered for each of the four cells in the design. These measures are also described in Part 4.3. 4.2 Experimental Setting The experiment took place during workshop sessions for accounting students. These sessions were conducted by the researcher as if she were the retained representative of a fictional firm, called The Materials Workshop (TMW). TMW's stated function was to develop and test educational materials for accounting students and to run research projects "for hire." TMW had two departments: an accounting lab and a systems lab. Testing of accounting educational materials occurred in the accounting lab under the supervision of the researcher. Computerized accounting educational materials were developed in the systems lab. The sessions were held in the evenings over the course of four weeks; each session lasted three hours. Two distinct groups of people were involved in the experiment. Participants were exposed to the experimental manipulations and assigned as users to the accounting lab. Confederates to the researcher were assigned to the systems lab to act as developers. In order to create a realistic laboratory setting, the fact that an experiment was taking place was hidden as far 35 as possible while still complying with the requirements of the university's human subjects review board. Participants were not told the primary purpose of their involvement until the debriefing, which occurred after all experimental sessions were completed. Since undergraduate accounting students served as participants, the accounting faculty and graduate students who had knowledge of the experiment were cautioned to keep it confidential. Recruiting and debriefing materials are contained in Appendix A. 4.2.1 Participants and Their Task 4.2.1.1 Recruiting Procedures and Results Ninety-six university students were recruited from upper-level accounting classes to serve as participants in this study. Recruiting was accomplished by posting announcements (Appendix A, Exhibit 1) on bulletin boards and distributing them in accounting classes. Students were told at the time of recruitment that most students who participated would earn the average pay of $15 for a three- hour session. In order to volunteer, each student completed a TMW application (Appendix A, Exhibit 2), returned it to the researcher, and was subsequently contacted to schedule a convenient session date. Demographic information about each participant was collected on the application for TMW employment. The average age of participants was 21.5 years. Fifty were female, and 46 were male. The number of accounting courses 36 that each participant had completed ranged from 2 to 10; the average amount of accounting education was 5.4 courses. The number of computer courses that each participant had completed ranged from 0 to 6, but 79 (82%) of the participants had taken only one computer course. Two of the. participants had previously earned a college degree (both in non-business fields); the rest had not. Seventy-one of the participants had part-time or limited full-time work experience. Eighteen had worked full-time for more than one year but less than three years. The remaining seven participants were divided nearly equally across the categories of no work experience and extensive work experience. Sixty-nine of the participants (72%) had no accounting work experience, and only five had ever worked full-time at an accounting-related job. Five participants had some part-time experience that involved computer skills; the remainder had no significant computer-related work experience. Prior to the start of each session, participants were randomly assigned to one of the four treatments. As shown in Table 3, participants did not differ significantly across assigned treatments with respect to gender, age, work experience, accounting coursework, computer coursework, or degree status. PARTICIPANT 37 TABLE 3 DEMOGRAPHICS BY ASSIGNED CASE ASSIGNED CASE CHI- P- OVER- SQ.‘ 17m ALL V3: A B c D (dfl PANEL A Average Age 22.00‘ 21.08 21.13 21.96 21.50 22.42 0.554 (4.31) (1.02) (0.85) (3 .48) (2.84) (24) Average Number of 5.08 5.21 5.46 6.04 5 .45 19.96 0.699 Accounting Courses (1.77) (1.72) (1.91) (1.63) (1.77) (‘24) Average Number of 1.33 1.08 1.17 1.00 1.15 9.55 0.656 Computer Courses (1.09) (0.28) (0.58) (0.42) (0.67) (12) Average Level of Work 1.33 1.26 1.21 1.21 1.25 4.44 0.880 Experience‘ (0.64) (0.62) (0.41) (0.59) (0.56) (9) Average Level of 0.42 0.26 0.38 0.33 0.35 6.61 0.678 Accounting Work (0.58) (0.54) (0.58) (0.76) (0.61) (9) Experience Average Level of Computer 0.08 0.04 0.04 0.04 0.05 0.61 0.895 Work Experience (0.28) (0.21) (0.20) (0.20) (0.22) (3) PANEL 8 No. with No Prior 22 24 24 24 94 6.13 0.106 Bachelor’s Degree (3) No. of each Gender: Female 12 13 12 13 50 0.17 0.983 Male 12 1 1 12 11 46 (3) Total Number 24 24 24 24 96 —- -— 38 TABLE 3 (continued) 8The Chi-Square Test Statistic is related to the distribution by case of frequency data. bCell entries in Panel A are means and standard deviations. Cell entries in Panel B are counts. cAll work experience data has been coded as: 0 No work experience 1 Some work experience, but less than one year of full-time work. 2 One to three years of full-time work. 3 More than three years of full-time work. 4.2.1.2 Early Session Procedures At the start of each session, participants read a letter from the TMW Director of Accounting Projects (Appendix A, Exhibit 3.) This letter continued to promote TMW as a real company and introduced participants to their major assignment. Participants also learned that some data would be collected to aid research into productivity and other issues. The letter, therefore, also served as the means to collect written permission to use data about participants for research. Also at the beginning of the session, the need for each person to meet a performance-related goal (Appendix A, Exhibit 4) was stressed. Participants read and signed a compensation agreement (Appendix A, Exhibit 5) that allowed for reducing their pay by half if their progress in the accounting lab was poor. The performance goal was said to lie within range of most people in the allotted time, provided they were attentive to and not diverted from their work. 39 The main task of participants was the testing of a business simulation involving the manual accounting system and related paperwork for a small company. The format, procedures, and content of the simulation borrowed extensively (with the permission of the authors) from the Systems Understanding Aid for Financial Accounting by Kieso, Arens, and Ward (1989). A new setting for the simulation was written by the researcher and the scope of the simulation was limited to the revenue cycle. Participants were told that other cycles of the simulation were being tested at other locations, due to the length of the project. Any errors found in the simulation by the participants were to be noted on a special form, and the simulation was seeded with several errors so that the task did not seem artificial. All materials required by the participants, including calculators, were supplied by the researcher. During the course of the workshop session, participants interacted only with the assigned confederate and the accounting lab supervisor. The next half hour was spent getting started on the accounting lab task with the researcher helping participants as needed. A pilot session (discussed more fully in Section 4.2.2) had indicated that half an hour was sufficient for participants to assess the scope and difficulty of their assigned task. After half an hour, the researcher distributed the case-specific Systems Lab Assignment (SLA) forms (Appendix 40 D, Exhibits 1 through 4). Participants received only the version of the SLA form that corresponded to their pre- assigned treatment. Before the participants were allowed to read the SLA form, the researcher announced that a team of systems developers was in the building that evening hoping to conduct interviews with the participants. Participants were told that the systems lab was working on software that would be useful to most accounting students past the principles stage and were asked to help in the system design. They also were told that this system would not be available in time for their own use. Their involvement was simply to meet with a systems lab employee--a confederate-- and discuss the participant's experiences with computers in accounting courses at the university. Thus, participants emulated real-world users who are often asked to contribute specific knowledge to a development project that will be (1) used on a regular basis by someone else or (2) used only on a voluntary basis by themselves. The script and session schedule followed by the researcher appears as Appendix B. 4.2.2 Confederates and Their Task The duty of the confederates was to pose as system developers working for TMW on the design of a computerized product that would be used by accounting students after implementation. (The system proposal for this product is in Appendix C, Exhibit 2.) Seven confederates were recruited from graduate-level systems classes. An additional 41 confederate was recruited from the accounting faculty at another university.5 All confederates were compensated. An orientation meeting was held to familiarize confederates with their experimental duties and props. During orientation, a memorandum of significant details was distributed and potential questions that participants might ask a confederate were discussed (Appendix C, Exhibit 1.) A review of session procedures was conducted, and the materials each confederate would use during the experimental sessions were explained. Session materials included interview outlines for each meeting, a systems proposal, documentation to support the system design, and a form for reporting on each meeting (Appendix C, Exhibit 2.) Confederates were instructed to use the system documentation in any way they felt would support their role. They were, however, expected to be as consistent as possible across participants. In addition to the orientation meeting, a pilot session, which is not included in the data analysis, was held the week before the experimental sessions started. This pilot was designed to give the confederates practice playing their role and to work out the logistics of the experimental procedures. During the pilot, confederates met with participants who were recruited in the same manner as 5This latter confederate was present at all sessions and lent credibility to the existence of TMW by presenting to the participants a more mature, unknown person who seemed to be in charge of the systems lab effort. 42 the experimental participants and who were not different from the experimental participants in any way. The single distinction of the pilot from the confederates' viewpoint was that each confederate was assigned only one participant for the evening, rather than the normal two. A review meeting was held after the pilot session to discuss problems and solutions; no significant changes in confederate procedures were made. For the actual experiment, confederates met twice with each participant, on a one-to-one basis, during the course of each session. Deviations from the interview outlines were taken when necessary to preserve an aura of authenticity. Meetings were held in private rooms and were planned to last approximately ten minutes each. Unless the participant requested otherwise, the meetings between participant and confederate were audio-taped. Tapes were spot checked by the researcher to verify that confederate meetings were being conducted as expected. At the end of each meeting, confederates recorded on a Systems Interaction History form (Appendix C, Exhibit 3) the length of the meeting, their assessment of the participant's attitude, and notes on any significant occurrences. This form served as an inducement to the confederates to attend seriously to the length and content of each meeting. Due to their other personal commitments, confederates did not all work the same number of sessions. Although the participant treatments had no impact upon the confederate's 43 assignment, confederates were rotated across cases (participant treatments) as uniformly as possible in order to identify possible confederate effects on the interaction quality and IS-attitude measures. Confederate effects on these measures are addressed in Part 4.3. Table 4 reports participant demographic information for each confederate. Participants did not differ significantly across confederates on any of these demographic characteristics. 4.3 Operational Variables Testing the theoretical model of friction required manipulating two variables: resource conflict and organizational pressure. The manipulation intervention occurred at the half hour mark as the researcher distributed the SLA form (Appendix D, Exhibits 1-4) to each participant. The manipulations were pretested on students in a graduate-level accounting class. At the time of the pretest, the manipulation of organizational pressure had been defined as a comparison of participant equity to that of TMW owners. Following the pretest, minor revision of the wording of the SLA form was done. Then the pilot session was held. Results of the pilot session indicated that participants seemed always to believe that TMW's owners would be better off than they. Therefore, after the pilot session, the comparison group was changed to be similar students rather than TMW's owners. Neither the pretest nor 44 TABLE 4 PARTICIPANT DEMOGRAPHICS BY ASSIGNED CONFEDERATE ASSIGNED CONFEDERATE CHI- P- ITEM -‘ VAL 1 2 3 4 s 6 7 8 “0 UE PANEL A Average Age 21.8‘ 20.9 21.2 21.7 21.7 24.7‘ 20.7 20.8 65.1 0.19 (3 .4) (0.8) (0.6) (0.8) (1 .8) (8.0) (l .0) (1.5) (56) Average Number of 5.50 5.15 6.23 5.33 5.54 5.00 5 .00 5 .50 60.7 0.31 Accounting Couraea (1.5) (2.2) (1.7) (2.3) (1.6) (1.7) (2.1) (2.0) (56) Average Number of 1.12 1.23 1.08 1.00 1.08 2.0‘ 1.00 1.00 28.5 0.44 Computer Courses (0.6) (0.4) (0.3) (0.0) (0.5) (2.0) (0.0) (0.6) (28) Average Level of 1.31 1.15 1.08 1.00 1.15 1.67 1.54 1.00 25.4 0.23 Work Experience“ (0.6) (0.4) (0.3) (0.0) (0.6) (0.8) (0.8) (0.0) (21) Average Level of 0.42 0.15 0.38 0.00 0.46 0.17 0.38 0.50 18.5 0.62 Accounting Work (0.8) (0.4) (0.7) (0.0) (0.5) (0.4) (0.8) (0.6) (21) Experience Average Level of 0.04 0.00 0.08 0.00 0.15 0.17 0.00 0.00 6.55 0.48 Computer Work (0.2) (0.0) (0.3) (0.0) (0.4) (0.4) (0.0) (0.0) (7) Experience PANEL B No. with No Prior 9.90 Bachelor’s Degree 24 13 13 6 12 5 l3 6 (7) 0.19 No. of each Gender: Female 15 6 6 2 8 3 7 3 2.04 0.96 Male 1 1 7 7 4 5 3 6 3 ('7) Total Number 26 13 13 6 l3 6 13 6 —— —— 45 TABLE 4 (Continued) aThe Chi-Square Test Statistic is related to the distribution by confederate of frequency data. bCell entries in Panel A are means and standard deviations. Cell entries in Panel B are counts. cAll work experience data has been coded as: No work experience Some work experience, but less than one year of full-time work. One to three years of full-time work. More than three years of full-time work. (AMI-'0 dThis average is driven by one person who was 41 years old and the low number of persons assigned to Developer 6. eThis average is driven by one person with six courses. the pilot session led to changes in the definition of resource conflict. Besides manipulations of exogenous variables, measures for the constructs of perceived resource conflict, perceived organizational pressure, friction, interaction quality and systems success were required for the experiment. The operational forms of both the exogenous and endogenous variables are described below. 4.3.1. Resource Conflict The manipulation of resource conflict (RC) required conditions such that the participants in high-RC cases would anticipate a less positive change in their own equity status than would those participants assigned to low-RC cases. The research setting was assumed to offer little inherent benefit to participants for involvement in the systems lab project. They would be out of school before being able to 46 utilize the finished systems product.6 Therefore, manipulation of resource conflict required variation in the participant's cost of systems lab involvement. This manipulation was accomplished by adapting the terms of the performance evaluation. At the start of each session, participants had signed compensation agreements (Appendix A, Exhibit 5) stating that half of the $15 compensation would be forfeited for under- achievement with respect to the accounting lab task. Along with this agreement, a goal with respect to expected progress in the accounting lab task had been stated. When the systems lab project was announced and before the SLA forms were read, participants were told that meetings with confederates would take approximately thirty minutes away from the remaining two hours available for accounting lab tasks and that meeting the stated goal would be very difficult for most people. (See Appendix B for the researcher's session schedule and script.) The SLA form explained to participants how their performance evaluations would be handled, considering the unexpected systems lab activity. The experimental treatment 6A participant might have perceived inherent benefit from the TMW session--particularly if encouraged by a professor to attend. These inherent benefits were expected to be attached to the accounting lab task, and systems interaction interfered with the attainment of such benefits. The impact of such a perception should have strengthened the manipulation for participants assigned to high-RC cases. For those persons assigned to low-RC cases, perception of inherent benefit might have led them to over-estimate the cost of systems work. If they inflated the cost enough, they may have perceived themselves to be in a high perceived resource conflict situation. (At most, eight participants might have been affected in such a manner.) 47 for the low-RC condition (Cases A and B) utilized a promise that the participant's performance evaluation would ignore the announced goal and would reward participants fully who seemed diligent at the accounting and systems tasks. Alternatively, for the high-RC condition (Cases C and D), participants were reminded that their performance evaluations were to be based solely upon meeting the accounting lab performance goal, which would be difficult to do. Participants were asked to mark on a scale on the SLA form how many minutes they were willing to spend on systems lab activity, given their compensation terms. In Cases A and B, where their time away from the accounting lab task would not cause a penalty, participants were expected to be very cooperative on the systems lab project. However, for those persons in Cases C and D who were likely to pay a penalty for the time spent on the systems lab project, responses were expected to indicate more conflict with respect to the requested time allocations. 4.3.2 Organizational Pressure Organizational pressure (OP) was expected to increase friction when the reference group's equity ratio differed from the participant's ratio, as stated by Equation 1 in Chapter 3. The experiment addressed organizational pressure by providing the participant with a reference group of students from a similar university. Participants were told that the reference group had tested a different accounting 48 cycle of the same practice set for TMW and also had been asked to participate in a systems development project. The SLA form described the compensation provided to these comparison students. For the low-OP condition (Cases A and C), the form noted that the comparison students had been compensated in the same manner and at the same level as the participants in the experimental session. Participants assigned to these cases were expected to feel that their equity ratio was very equal to that of other students. For the high-OP condition (Cases B and D), the SLA form stated that the comparison students had received $7.50 (in addition to their compensation for the accounting lab project) for participation in the systems lab project. In Cases B and D, therefore, participants were expected to feel that they were being treated inequitably when compared to the reference group of the other students. Table 5 summarizes the operational definitions for each level of the exogenous variables. 4.3.3 Perceived Treatments Participants were asked to respond to questions on the SLA form that were designed to gauge their perception of the resource conflict and organizational pressure treatments. First, participants were asked how many minutes they were willing to devote to the systems lab project. The measure ranged from zero to thirty minutes, which was the total time requested. A continuous measure of perceived resource 49 THUELE 55 SUMMARY OF EXPERIMENTAL TREATMENTS The participant's performance evaluation equally weighs accounting lab and systems Absent lab tasks. (Goal does not count.) RESOURCE CONFLICT . . . The partiCipant's performance evaluation weighs only accounting lab task Present performance. (Goal does count.) Similar students are compensated in the same way as the participant for the same Absent accounting lab and systems lab duties. (Participants receive same pay.) ORGANIZATIONAL PRESSURE Similar students receive additional money for participation in the systems lab Present project and the participant does not. (Participants receive less pay.) conflict was computed by subtracting from thirty the amount of time the participant would be willing to provide. By graphing the continuous measure, a binary measure of perceived resource conflict was derived based upon a "natural" split of the participants' responses. Most participants assigned to the low-RC condition indicated a willingness to spend from twenty to thirty minutes on the systems lab project (i.e., a conflict of zero to ten minutes.) In contrast, most of those persons assigned to the high-RC condition reported that they wished to spend fewer than twenty minutes involved in the systems lab project (i.e., a conflict of eleven to thirty minutes.) Thus, participants whose resource conflict measure was ten 50 minutes or less were considered to perceive low resource conflict. Those participants with resource conflict measures exceeding ten minutes were placed in the high perceived resource conflict group. As reported in Table 6, the assigned treatments were correlated with the participants' perceptions (r=0.585, p— value = 0.0001). Of the participants assigned to the low-RC condition, 83.3% reported little disagreement about the way in which they were being asked to use their time. In contrast, 75.0% of the participants assigned to the high-RC treatment reported that they were being asked to spend considerably more time than they wished on the systems lab project. TABLE 6 RESOURCE CONFLICT BY PERCEIVED RESOURCE CONFLICT PERCE IVED RESOURCE CON FL I CT Low High T°TAL Low 40 8 48 (41.67%) (8.33%) (50.00%) Rssouncs CONFLICT at h 12 36 48 9 (12.50%) (37.50%) (50.00%) 52 44 96 T°TAL (54.17%) (45.83%) (100.00%) Chi-Square = 32.895, 1 d.f. p = 0.000 r = .585 (p=.0001) Average (continuous measure) perceived resource conflict = 10.89 Average (binary measure) perceived resource conflict = .458 51 The second question on the SLA form asked participants whether they felt about equal to, better off than, or worse off than students of the comparison group. Participants who felt either equal to or better off than the others were presumed to feel little perceived organizational pressure. Participants who indicated that they were worse off than the comparison students were deemed to act as if under high perceived organizational pressure. All persons assigned to the low-OP cases reported feeling no worse off than the comparison group. However, only 56.3% of participants reported feeling treated unfairly when told that other students had received more money than they would receive. While this number is lower than would be ideal, the correlation between assigned OP treatment and the participants' perceptions of organizational pressure was strong (r=0.626, p-value = 0.0001). Table 7 summarizes the results of the organizational pressure manipulation. Table 8 presents a breakdown of participants by assigned case and perceived case. Successful treatment manipulation occurred for those participants who fall in the diagonal cells. The most notable discrepancy is for assigned Case B (low RC, high OP) where thirteen out of twenty-four participants did not believe that they were being treated inequitably.7 7Exit interviews with several of these participants indicated that they would have answered differently had they had more time to think about their situation. For data analysis purposes, their original answers are reported. Knowing when during the experimental session they changed their minds is impossible to ascertain. 52 TATHHS 7 ORGANIZATIONAL PRESSURE BY PERCEIVED ORGANIZATIONAL PRESSURE PERCEIVED ORGANIZATIONAL PRESSURE TOTAL Low High Low 48 O 48 ORGANIZATIONAL (50.00%) (0.00%) (50.00%) passsunz at h 21 27 48 9 (21.88%) (28.12%) (50.00%) 69 27 96 Toan (71.88%) (28.12%) (100.00%) Chi-Square = 37.565, 1 d.f. p = 0.000 r = .626 (p=.0001) Average perceived resource conflict = .281 IEABIJS 8 ASSIGNED TREATMENT BY PERCEIVED TREATMENT PERCEIVED TREATMENT a B c TOTAL a 20 o 4 o 24 (20.8) (0.0) (4.2) (0.0) (25.0) a 11 9 2 2 24 (11.5) (9.4) (2.1) (2.1) (25.0) ASSIGNED 6 0 18 0 24 TREATMENT c (6.3) (0.0) (18.8) (0.0) (25.0) n 3 3 5 13 24 (3.1) (3.1) (5.2) (13.5) (25.0) 40 12 29 15 96 TOTAL (41.7) (12.5) (30.2) (15.6) (100.0%) 53 In order to examine whether any personal characteristics of the participants contributed to their perceptions of either resource conflict or organizational pressure, Table 9 reports the demographic information of participants according to perceived case. Participants did not differ significantly across perceived treatments on the bases of age, accounting coursework, computer coursework, accounting work experience, degree status, or gender. They did differ across perceived treatments on the basis of work experience (p<.05), primarily attributable to persons who perceived themselves to be in Case C. Participants who perceived that they had a resource conflict but who did not perceive organizational pressure tended to have less work experience than did other participants. 4.3v4 Friction Friction between developers and users cannot occur without the expectation of a common project. In a static physical model, the measure of friction is the amount of force that must be applied in order to initiate movement. A potential measure of friction in the IS development environment, therefore, is the minimum amount of money to be paid in a lump sum that would persuade the user and developer to agree to cooperate willingly on an IS development project. To assess friction, participants were asked if they were happy about being expected to participate on the 54 TABLE 9 PARTICIPANT DEMOGRAPHICS BY PERCEIVED CASE PERCEIVED CASE Cfll' p- a . ITEM OVER-ALL ' VILU A B C 0 (df) E PTUIEI..A Average Age 21.731’ 21.42 20.90 22.40 21.50 26.50 0.33 (3.34) (1.16) (0.86) (4.40) (2.84) (24) Average Number of 5.33 5.50 5.17 6.27 5.45 27.74 0.27 Accounting Courses (1.61) (2.02) (1.83) (1.79) (1.77) (24) Average Number of 1.25 1.00 1.14 1.00 1.15 8.26 0.77 Computer Courses (0.90) (0.00) (0.52) (0.38) (0.67) (12) Average Level of work 1.35 1.27 1.10 1.27 1.25 17.440 0.04 Experience (0.62) (0.47) (0.31) (0.80) (0.56) (9) Average Level of 0.40 0.09 0.28 0.53 0.35 8.94 0.44 Accounting work (0.59) (0.30) (0.53) (0.92) (0.61) (9) Experience Average Level of 0.08 0.00 0.03 0.07 0.05 1.26 0.74 Computer Hark (0.27) (0.00) (0.19) (0.26) (0.22) (3) Experience PEHIEI..B Number with No Prior 38 12 29 15 94 2.86 0.41 Bachelor's Degree (3) Number of each Gender: 22 5 17 6 50 2.03 0.57 Female 18 7 12 9 46 (3) Male Total Number 40 12 29 15 96 ---- ---- 55 TABLE 9 (Continued) aThe Chi-Square Test Statistic is related to the distribution by perceived case of frequency data. bCell entries in Panel A are means and standard deviations. Cell entries in Panel B are counts. cAll work experience data has been coded as: O No work experience 1 Some work experience, but less than one year of full-time work. 2 One to three years of full-time work. 3 More than three years of full-time work. systems lab project. (See the SLA form, Appendix D, Exhibits 1-4, page 2.) Participants who indicated that they were not happy (or indifferent) about the systems lab project were asked to name the minimum amount of money that would make them willing to interact. (Note that they were not promised any money for doing so.) Alternatively, participants who were happy about becoming involved with the systems development project were asked how much of their own money they would be willing to pay in order to increase their participation level. These monetary amounts were used as measures of friction, with positive signs associated with the amounts desired by the participant and negative signs on the amounts participants would be willing to pay to someone else. Inspection of the data revealed two outliers. One was a participant from the first session who indicated that he would be willing to pay a maximum of $30 to increase his participation level. Willingness to pay to participate more was a relatively rare response; only two other participants replied that they would pay, and both used $5 as the amount. 56 The $30 response was explained based upon the participant's perception that he would learn a lot about systems development from participating in the systems lab project. In succeeding sessions, participants were told that they would not experience any significant learning from participation (as noted in Appendix B.) The second outlier was a response that a minimum of $425 would be required to make the participant happy about systems lab project involvement. When questioned about the size of the amount, this participant insisted that $425 was the minimum since his apartment rent was due the next day. However, he then willingly participated in the systems lab project without additional payment and without further encouragement to do so. In Tables 10 and 11, both of these outliers are omitted. Table 10 presents statistics dealing with the friction measure by assigned case, and Table 11 repeats the analysis by perceived case. As expected, the friction amount is uniformly larger for those persons who were not happy about participation as compared to persons who were happy to participate.8 The actual amounts of friction in each case were not predicted. However, prior expectations on these amounts, assuming participants have utility for nothing but money, 88y design, these amounts cannot overlap. However, the spreads need not be as large as those observed. All assigned and perceived cases have significantly different friction levels for those persons reportedly "happy" and those who are "unhappy or indifferent" (p < .01.) 57 TABLE 10 FRICTION BY ASSIGNED CASE HAPPY TO PARTICIPATE? ASSIGNED CASE N 0 or 2°. Indifferent A ll 6 18 24 Range $0.00 to $0.00 $1.00 to $15.00 $0.00 to $15.00 Mean $0.00 $5.42 $4.06 St.Dev. 0.00 3.55 3.88 3 n 6 17 23 Range $0.00 to $0.00 $0.00 to $15.00 $0.00 to $15.00 Mean $0.00 $5.74 $4.24 St.Dev. 0.00 3.28 3.80 c n 4 19 23 h Range -ss.oo to $0.00 $1.00 to $15.00 -ss.oo 10 $15.00 Mean -$l.25 $8.29 $6.63 St.Dev. 2.50 4.45 5.54 D n 3 21 24 Range -$5.00 to $0.00 $2.50 to $20.00 -$5.00 to $20.00 Mean -$l.67 $7.55 $6.40 St.Dev. 2.89 3.98 4.92 OVERALL n 19 75 94 Range -$5.00 to $0.00 $0.01to $20.00 -$5.00 to $20.00 Mean -$0.53 $6.82 $5.33 ll St.Dev. 1.58 3.97 4.67 58 TABLE 11 FRICTION BY PERCEIVED CASE HAPPY TO PARTICIPATE? PERCEIVED CASE N OVERALL 0 or Y" Indifferent A n 12 27 39 Range -$5.00 to $0.00 $0.00 to $15.00 -$5.00 to $15.00 Mean -$0.42 $5 .46 $3.65 St.Dev. 1.44 4.24 4.52 B n l 10 11 Range $0.00 to $0.00 $3.00 to $7.50 $0.00 to $7.50 Mean $0.00 $6.05 $5.50 St.Dev. 0.00 1.64 2.40 c n 6 23 29 Range -$5.00 to $0.00 $2.50 to $20.00 -$5.00 to $20.00 Mean $0.84 $8.37 $6.47 St.Dev. 2.04 4.44 5.53 D n 0 15 15 Range N/A $2.50 to $15.00 $2.50 to $15.00 Mean N/A $7.38 $7.38 St.Dev. NIA 3.03 3.03 OVERAIJ. ll 19 75 94 Range -$5.00 to $0.00 $0.00 to $20.00 -$5.00 to $20.00 Mean -$0.53 $6.82 $5.33 St.Dev. 1.58 3.97 4.67 59 would be around $0.00 for Case A, $7.50 for Cases B and C, and $15.00 for Case D. The friction amounts reported for both assigned and perceived Cases B and C are larger than for Case A, as expected. An unexpected result is that amounts for Case D are so low and are generally smaller than for Case C. This result, however, is not inconsistent with the model. Persons perceiving themselves to be in Case D may have planned from the very beginning of the systems lab project to shirk (or utilize other input-reducing practices.) Such an attitude would alter their input-to- output ratio and place them in a more equitable position relative to other persons who expended more effort and received higher pay. 4.3.5 Quality of Interaction After the SLA forms were collected, participants returned to the accounting lab task. Within a few minutes, confederates appeared in the accounting lab, requesting to speak with their assigned participant. Confederate arrivals were staggered so that usually only two participants were out of the accounting lab at any one time. During each experimental session, participants interacted repeatedly with confederates (two meetings with a target minimum of ten minutes each meeting.) The quality of these interactions was measured by the participant's responses to a data collection instrument administered at the end of the experimental session. 60 Although several prior studies have addressed the construct of interaction quality, none of them evaluated interaction solely as a process, without reference to the product under development. Since this paper draws a distinction between attitudes toward the process and attitudes about the product, a new instrument was developed based on those of prior works. 4.3.5.1 Development of Instrument Kaiser and Srinivasan (1982) specifically identified five factors related to interaction quality: communication quality, needs awareness, flexibility, power, and competence. Later research also investigated these factors (e.g., Gregson, 1990; Cronan and Means, 1984.) Two other factors, compatibility (Gregson, 1990; Bailey and Pearson, 1983) and user involvement (Doll and Torkzadeh, 1991; Cronan and Means, 1984), were also reported to be important to successful interaction. Of these seven factors, the ones deemed potentially important and appropriate to this experimental environment were communication, compatibility, involvement, awareness of user needs, and user power. Flexibility and competence were not used because the experimental environment did not allow users a good opportunity to judge these characteristics. Definitions of the interaction quality factors appear in Table 12. Development of the interaction quality (IQ) instrument started by assembling from these prior works a set of 61 EEABIJB 12 INTERACTION QUALITY FACTORS FACTOR DEFINITION REPRESENTATIVE STUDIES Communication Measures the users' Gregson (1990) Quality feelings about how Cronan and Means (1984) successfully the Kaiser and Srinivasan (1982) developer listens or talks to them. Compatibility Measures how well the Gregson (1990) users feel they work Bailey and Pearson (1983) with the developers. User Measures whether the Doll and Torkzadeh (1991) Involvement users' level of Cronan and Means (1984) involvement matches the level desired. Needs Measures whether the Cronan and Means (1984) Awareness users feel that their Kaiser and Srinivasan (1982) own needs are being considered by the developers of the system. Power Measures whether the Bailey and Pearson (1983) users feel they have power to sway the development process or to influence the product. 62 fifteen items, three items for each of the five factors, and formulating them in a consistent style. Only items relating to the process of systems development were selected for the IQ measure. To validate the IQ instrument, procedures similar to those followed by Moore and Benbasat (1991) were utilized. The items were numbered randomly, and each was placed on an index card. Twenty-four students in a graduate-level systems class were asked to serve as judges of content. They were each given a set of item cards and told to sort the cards into piles according to their similarity. No categories were suggested to these judges, nor were they given numbers of categories or cards per category. An analysis of the sorting results indicated that the judges tended to place particular items together in the expected groupings although the grouping may have carried a name different from the factor name. Three items were often misplaced so they were changed to be more indicative of their intended constructs. A second pretest used the revised set of items with a new set of judges, thirty-five undergraduate accounting information systems students from another university. In this pretest, the judges were asked to sort the cards into five categories, which were named. Judges were not told how many items belonged to any category, and items which were not perceived as belonging to one of the categories were listed separately. Results from this pretest indicated that 63 four items (none of which were the three new ones) were ambiguous. These four were revised, and a final validation sort was run. The third pretest was done by the original twenty-four judges, using the procedures from the second pretest. (The items sorted on this round are listed as Section A of the Exit Form, Appendix D, Exhibit 5.) No unclassifiable items remained, and most misclassifications seemed random. Following Moore and Benbasat (1991), construct validity was judged by examining the number of items that were placed by the judges in the expected target categories. Table 13 shows these results from the third pretest. All factor categories indicate placement ratios above 85%, and the overall placement ratio is 91.8%. Items 8 and 13 were the only systematic misclassifications. Item 8 theoretically dealt with involvement and was placed in the power category six times. Similarly Item 13, dealing with power, was placed in involvement seven times. The coefficient of agreement, Cohen's Kappa (Cohen, 1960), was calculated for each possible pair of judges. The average coefficient of agreement was 0.828. Moore and Benbasat (1991) report that a coefficient above 0.65 is generally considered acceptable. Thus, pretesting of this instrument was concluded. The fifteen items from the third pretest were used to collect data regarding the participant's assessment of the quality of interaction between the participant and the confederate during the experiment. Each item was 64 1EABIJ3 13 INTERACTION QUALITY CONSTRUCT VALIDATION TARGET ACTUAL CATEGORY CATEGORY Communi- Compati- Involve- Uaer Target cation bility ment Needs Power Total % Communication 5 (1,5,9). 71 3 2 2 O 78 91.0 Compatibility (3,12,15) 1 76 1 0 O 78 97.4 Involvement (4’ 8,11) 0 3 67 1 7 78 85.9 User Needs (7. 10,14) 1 2 O 75 0 78 96.2 Power (2, 6, 13) O O 7 2 69 78 88.5 Total Item Total Target Overall Placement Placements: 390 Placements: 358 Ratio: 91.8 % ‘These numbers refer to the items from Section A of the Exit Form (Appendix D, Exhibit 5). hCell entries refer to the number of items placed in that category by the judges. Each cell on the upper-left to lower-right diagonal is a target cell; placement numbers in these cells could be as high as 78 (3 items in the category times the 26 judges.) Entries in off-diagonal cells are misclassifications, based on the theoretical target. Average Coefficient of Agreement (Cohen's Kappa) across all possible pairs of judges is 0.828. 65 accompanied by a seven-point Likert-type scale, anchored with the words "Disagree" and "Agree". Coding of the data assigned the scale a -3.00 to +3.00 range, where the "Disagree" end was assigned a score of -3.00. Participants' responses were measured to the nearest eighth of a unit. Items 2, 3, 6, 8, 11, and 12 were reverse-coded so that higher scores always indicate more positive opinions. 4.3.5.2 Analysis of the Interaction Quality Scale Before evaluating the IQ scale, an analysis of variance was conducted to determine whether any of the items had been systematically affected by the assignment of particular confederates. Item 8 and 11 were found to be significantly (p50.05)affected by assigned confederate. Accordingly, these two items were eliminated from further analysis. The responses to the remaining IQ items were subjected to a confirmatory factor analysis (CFA) in order to assess the scale's validity and reliability. Confirmatory factor analysis was used instead of exploratory factor analysis because of the extensive pretesting of this instrument. Since theoretical constructs had been identified and used to determine the IQ items, this information was used to make the scale itself more grounded in theory rather than driven completely by statistics. During the CFA process both the internal consistency and the parallelism of the scale had to be assessed. If a factor is to be internally consistent, similar correlations 66 among its items will be exhibited. Parallelism of a scale implies that same-factor items will have similar relationships to items outside their factor. The parallelism tests indicated that the three items composing the Power factor were not parallel, and the items were dropped from the IQ scale. The Communication and Compatibility factors were so highly correlated (r=1.16, corrected for measurement error) that they were deemed to be the same construct, which is hereafter referred to as Compatibility. Item 4 also was placed into this factor. It was the only item remaining from the Involvement factor, and it was highly correlated with Compatibility. The internal consistency and the parallelism of the two factors remaining after these changes, Compatibility and Needs Awareness, was satisfactory. Table 14 presents the intercorrelation and factor loading matrix for the ten items in the interaction quality scale. The observed inter-item correlations are presented in the lower triangle, and predicted inter-item correlations are shown in the upper triangle. Factor loadings are presented in the side panel. An examination of the factor to factor relationships indicated that the factors were highly correlated and, thus, could be summed to create a scale for measuring Interaction Quality. Overall reliability, measured with Spearman Brown's Standard Score Coefficient Alpha, for the ten-item scale taken as a single-item construct was 0.846. 67 EEABIJE 14 INTERACTION QUALITY INTERCORRELATION AND LOADING MATRIX‘ ITEM FACTOR LOADINGb 1 3 4 5 9 12 15 7 10 14 c N 1 .54 .42 .46 .48 .45 .42 .18 .18 .49 .69 .56 3 .60 .48 .52 .54 .51 .48 .20 .20 .55 .79 .39 4 .33 .47 .41 .42 .40 .37 .16 .16 .43 .61 .37 s .45 .51 .38 .46 .43 .41 .17 .17 .47 .67 .42 9 .46 .58 .48 .37 .45 .42 .18 .18 .49 .69 .61 12 .39 .52 .57 .54 .33 .40 .17 .17 .46 .65 .39 15 .53 .40 .27 .43 .56 .31 .16 .16 .43 .61 .72 7 .41 .36 .27 .18 .47 .16 .40 .12 .33 .47 .35 10 .20 .09 .13 .30 .20 .26 .29 .05 .33 .31 .35 14 .31 .19 .20 .22 .34 .22 .48 .38 .38 .42 .96 3Numbers in the Item columns are correlations. Observed correlations are shown in the lower triangle, and predicted correlations are shown in the upper (shaded) triangle. Numbers in the Factor Loading columns are factor loadings using a two-factor model. bThe two factors are Compatibility (C) and Needs Awareness (N). The Spearman Brown Standard Score Coefficient Alpha for the ten-item scale, taken as one factor, is 0.846. 68 4.3.6 Attitude as a Measure of System Success The ultimate measure of information system success is the system's contribution to the economic well-being of the organization, but direct contributions are rarely observable. Many items influence a firm's value, and the length of time between implementation of a system and the reaping of its associated benefits can be quite long. In lieu of measuring the impact of an IS upon the organization's value, IS researchers have had to rely upon surrogate measures of system success. The most common surrogates have been measures of user satisfaction (US). An underlying assumption of the US measure is that the system must be successful at promoting the organization's goals if users report being satisfied with the system. In a review of the user satisfaction literature, Kim (1989) suggested that the reason for inconsistent findings across studies examining IS success is that the surrogate measures for success have not been used appropriately. He examined measures used in prior research and determined that three basic concepts were involved: 1. US-Attitudes: A measure of the user's satisfaction with situational factors such as vendor support, user involvement, communication with systems staff, determination of priorities, the schedule of products and services, and the organizational levels of the systems and user departments. 2. US-Information Quality: A measure of the user's satisfaction with information attributes such as reliability, format, timeliness, level of detail, and content. 69 3. US-Effectiveness: A measure of the perceived contribution of the IS to the achievement of organizational objectives. The perceived contribution could manifest in such measures as user decision-making effectiveness, job productivity, job satisfaction, and interpersonal relations. Kim noted that the measures of US-Information Quality and US-Effectiveness have shortcomings. US-Information Quality does not capture all IS factors contributing to organizational effectiveness. On the other hand, US- Effectiveness is difficult to measure accurately due to the common intervention of other factors in the time between the use of the IS and the receipt of associated benefits. Both of these US measures have been investigated by IS researchers, and no consistent results have been found with respect to their relationships with user attitudes or to system usage (e.g., Srinivasan, 1985, and O'Reilly, 1982). Further, neither of these measures was appropriate for this study because the model proposed deals with the IS development process. Until a system is implemented and used, actual US-Information Quality and US-Effectiveness cannot be judged. The US-Attitudes measure was considered to be appropriate because attitudes do form during IS development and are likely to affect the user's willingness to utilize the system once it is in place. Unlike the link between information quality and IS usage, the relationship between attitude and usage has received relatively consistent support in the IS literature (e.g., Baroudi, Olson, and Ives, 1986; Maish, 1979; Lucas, 1978). 70 Therefore, this experiment utilized the participant's attitude toward the proposed IS, called IS-Attitude, as a predictor of system success. 4.3.6.1 Attitude Scale Development In contrast to the interaction quality variable, IS- attitude assesses a potential user's feelings about the product under development. In the development environment, IS-attitude is a prospective assessment by non-users (that is, by potential future users) because the product has not yet been delivered. Moore and Benbasat (1991) developed a scale to measure the adoption of information technology innovations. (The Systems Lab project in the experiment, if actually developed, would be an innovation in the field of accounting education.) The Moore-Benbasat scale is a measure of people's perceptions about using the product rather than about the features of the product under development. Thus, it was particularly suitable for this experiment. The Moore-Benbasat scale has been subjected to rigorous validation procedures. It was designed so that it could be adapted easily to specific information technology projects and to either current or future users. Moore and Benbasat suggest a short form of the scale and report factor reliabilities (Chronbach's Alpha) ranging from .71 to .90. Eight factors were used by Moore and Benbasat to explain the adoption of information technology innovations: 71 voluntariness, image, relative advantage, system compatibility, ease of use, result demonstrability, trialability, and visibility. However, in order to reduce the length of the Exit Form, only factors with more than two items on the short form were used. This procedure eliminated the factors of visibility, trialability, and voluntariness. Result demonstrability was also omitted because judging it would require fairly deep knowledge of the proposed system; participants were unlikely to have such knowledge. Table 15 describes the Moore-Benbasat factors that were used in this study. TABLE 15 MOORE-BENBASAT FACTORS AFFECTING IS-ATTITUDE FACTOR DEFINITION Relative Advantage Perceived incremental usefulness of the new IS over methods used previously. System Compatibility Perceived degree to which use of the IS requires changes to the user's job. Image Perceived degree to which the IS enhances its user's status Ease of Use Perceived degree to which the IS is simple to learn and use. Items from the Moore-Benbasat short form pertaining to the four remaining factors were used as the IS-attitude instrument. It was incorporated as Section B of the Exit Form (Appendix D, Exhibit 5.) Coding of the experimental 72 data followed the same procedure as used for the interaction quality scale. 4.3.6.2 Analysis of the Attitude Scale An analysis of variance on the IS-Attitude instrument indicated that the assigned confederate did not affect any of the fifteen items (p s .05). Confirmatory factor analysis procedures used on the IS- attitude data were similar to those reported in Part 4.3.5.2 for the analysis of interaction quality. The experimental data yielded CFA results very like those reported by Moore and Benbasat (1991). Item 4 was dropped from the scale because it caused many problems with parallelism. Moore and Benbasat also dropped this item from their long scale although they recommended it for the short one. Factor-to-factor correlations suggested that the concepts of relative advantage and system compatibility were highly related. Again, this result echoes that of Moore and Benbasat. They report a 0.99 correlation between the first two factors on the long form of their instrument; this study finds a 0.97 correlation (corrected for measurement error) between the two factors when using the shorter instrument. Accordingly, these two factors were combined and will be called Relative Advantage in the remainder of this dissertation. The intercorrelation and factor loading matrix for IS- Attitude's three factors is shown in Table 16. Inspection 73 of the factor-to-factor correlations determined that these remaining factors could be summed to create a single IS- Attitude scale. The Spearman Brown Standard Score Coefficient Alpha for the fourteen-item scale, taken as one factor, is 0.878. 4.4 Summary Measures of Interaction Quality and Attitude Summary measures have been reported to be more reliable with respect to test-retest results than some of the detailed scales of user satisfaction that have been previously used (Galletta and Lederer, 1989) although these results have been questioned (Hawk and Raju, 1991). Accordingly, the Exit Form included an overall evaluation item for interaction quality and another for IS-attitude in addition to the detailed items described above. Table 17 presents correlations among several potential IQ measures. These include the two individual factor scores (compatibility and work-related needs) and the two overall IQ measures (an averaged score and a one-item summary score.) The factor scores are the unweighted averages of the items included in their respective factors. The Average IQ score is the unweighted average of the ten items included in the IQ scale. Summary IQ is the one-item summary measure. Because Summary IQ was an overall assessment of the participant's interaction experience, interpretation is difficult. Average IQ and Summary IQ were strongly 74 {HABIJB 16 IS-ATTITUDE INTERCORRELATION AND LOADING MATRIX“ JHEESI lUUlflBEfll l 2 5 6 9 10 13 15 3 7 11 8 12 14 l .49 .53 .64 .54 .59 .51 .55 .32 .28 .29 .23 .24 .17 2 .49 .43 .53 .44 .49 .42 .45 .26 .23 .24 .19 .20 .14 5 .45 .50 .57 .48 .52 .45 .49 .28 .25 .25 .20 .21 .15 6 .70 .45 .54 .58 .64 .55 .59 .34 .30 .31 .24 .26 .18 9 .56 .50 .47 .51 .54 .46 .50 .29 .25 .26 .20 .22 .16 10 .46 .52 .60 .72 .51 .50 .55 .31 .28 .29 .22 .24 .17 13 .55 .44 .52 .53 .38 .51 .47 .27 .24 .25 .19 .21 .15 15 .64 .36 .39 .65 .61 .51 .43 .29 .26 .27 .21 .22 .16 3 .38 .19 .36 .25 .33 .20 .18 .34 .57 .59 .20 .22 .16 7 .39 .30 .48 .36 .36 .39 .21 .40 .58 .52 .18 .19 .14 11 .24 .13 .26 .11 .21 .22 .10 .25 .60 .52 .19 .20 .14 8 .13 .07 .32 .21 .11 .33 .25 .20 .15 .28 .12 .68 .48 12 .18 .16 .33 .19 .08 .29 .38 .08 .13 .25 .17 .70 .52 14 .20 ‘39 .16 .33 .13 .20 .18 .29 .22 .20 .11 .47 .52 ll .77 ~63 .68 .83 .70 .77 .66 .71 .39 .50 .26 .28 .29 .26 I .45 in .48 .32 .40 .35 .22 .44 .81 .71 .73 .25 .24 .24 E1 .23 I” .36 .32 .14 .36 .36 .25 .22 .32 .18 .80 .86 .61 ‘Numbers in the Item columns are correlations. Observed correlations are shown in the lower triangle, and predicted correlations are shown in the upper (shaded) triangle. Numbers in the last three rows are factor loadings using a three-factor model. The three factors are Relative Advantage (R), Image (I), and Ease of Use (E). The Spearman Brown Standard Score Coefficient Alpha for the 14-item scale, taken as one factor, is 0.878. 75 TABLE 17 INTERCORRELATIONS AMONG FACTOR SCORES AND OVERALL INTERACTION QUALITY MEASURES IQ Pactora' E 1 Average 1 s I Compat- work—Related : IQ b “:zzzzc Q ibility Needs . Score 1 Compatibility 1,000 . Work-Related Average IQ 1 Score 0.950 0.651 : 1.000 1 Summary IQ : Score 0:719 0-445 : 0.725 1.000 aIQ Factor scores are the unweighted averages of the scores on each item included in the factor. That is, Compatibility = Average (IQ Items 1, 3, 4, 5, 9, 12, 15) Work-Related Needs = Average (IQ Items 7, 10, 14) pAverage IQ is the unweighted average of the ten items left in the scale after factor analysis. cSummary IQ is a one-item overall assessment measure reported by the user participant on the Exit Form. correlated (r=0.725) but obviously did not measure exactly the same constructs. Of the two identified factors, Summary IQ relates more to the factor of Compatibility (r=0.719) and less to Work-related Needs (r=0.445). These relationships may be driven by the experimental environment. Confederate compatibility probably was more salient to the participant than was the confederate's awareness of the participant's work-related needs. Table 18 presents parallel computations for the individual factor and overall measures of IS-attitude. The 76 1EABIJ§ 18 INTERCORRELATIONS AMONG FACTOR SCORES AND OVERALL IS-ATTITUDE MEASURES IS-Attitude Factors“ 3 Average hry Relative Ease 18- b 18- c va::;9e 1.39e J2: Attitude Attitude Relative Advantage 1.000 : Image 0.427 1.000 I Ease of Use 0.307 0.251 1.000 Average Is-Attitude 0.902 0.695 0.563 : 1.000 Summary IS-Attitude 0.681 0.396 0.427 E 0.709 1.000 aFactor scores are the unweighted averages of the scores on each item included in the factor. That is, Relative Advantage = Average (IS-Attitude Items 1, 2, 5, 6, 9, 10, 13, 15) Image = Average (IS-Attitude Items 3, 7, 11) Ease of Use = Average (IS-Attitude Items 8, 12, 14) pAverage IS-Attitude is the unweighted average of the fourteen items remaining on the scale after factor analysis. cSummary IS-Attitude is a one-item overall assessment measure reported by the user participant on the Exit Form. 77 Average IS-Attitude and Summary IS-Attitude scores were strongly correlated (r=0.709), but as above they did not seem to be measuring exactly the same constructs. The factor of Relative Advantage (inclusive of System Compatibility) was more highly related to Summary IS- Attitude than were either the Image or Ease of Use factors. Since the system being developed in the experiment was at a very preliminary stage, confederates may have been able to promote easily the general advantages of the system. Increasing the participant's awareness of the ease of use or image enhancement features may have been more difficult at such an early stage. 4.5 Additional Manipulation Checks Section C of the Exit Form (Appendix D, Exhibit 5) was administered to assess the participants' feelings with respect to a number of issues and to serve as a check on the manipulations. These ten items were scored similarly to other Exit Form items; the range of response was from -3.00 (Disagree) to +3.00 (Agree). However, no items were reverse coded, and higher scores do not necessarily imply more positive attitudes. Table 19 contains the means and standard deviations of the responses to each item. The table was constructed so that the effects of perceived resource conflict (PRC) could be examined while perceived organizational pressure (POP) 78 {HABIJE 19 POST-EXPERIMENTAL MANIPULATION CHECKS 1 PERCEIVED casa' P-VALUES" T a A a c 1) A11 . , , , u n=40 n=12 n=29 n=15 =96 “‘3 c'” A“ 3’” 1 1.436 1.001 0.790 1.085 1.132 0.254 0.482 0.043 0.836 (1.219) (0.838) (1.369) (1.167) (1.233) 2 .2142 -1519 -1.602 -1.916 —1.866 0.068 0.449 0.049 0.424 (0.849) (1.448) (1.381) (1.089) (1.160) 3 -1.582 0.749 0.929 -1.691 -l.298 0.084 0.102 0.069 0.090 (1.393) (1.567) (1.528) (1.211) (1.457) 4 -1 .920 -1.790 -1.554 -1.999 -1.806 0.754 0.231 0.260 0.519 (1.336) (0.873) (1.294) (0.782) (1.198) 5 0.671 0.241 0.239 0.401 0.443 0.412 0.687 0.258 0.707 (1.663) (1.227) (1.384) (0.972) (1.433) 6 1.590 0.720 1.437 1.063 1.355 0.084 0.397 0.662 0.555 (1.419) (1.761) (1.430) (1.134) (1.440) 7 ~2.048 -1.812 -1.516 -1.599 -1.788 0.475 0.828 0.051 0.612 (0.951) (1.147) (1.276) (1.008) (1.100) 8 -1.661 0.982 -1.019 0.593 -0.784 0.000 0.003 .052 0.608 (1.422) (1.551) (1.182) (2.181) (1.804) 9 2.077 1.357 1.516 0.985 1.647 0.089 0.215 0.055 0.544 (1.110) (1.688) (1.259) (1.447) (1.332) 10 -l.614 -1.768 0.916 -1.165 -1352 0.729 0.598 0.061 0.173 (1.461) (0.838) (1.556) (1.286) (1.425) ‘Cell entries for this section are the means and standard deviations of The scale ranged from -3.00 (Disagree) to +3.00 (Agree), Refer to Part C of the Exit Form the responses. and no items were reverse coded. (Exhibit 5 of Appendix D) for the text of the items. bP-values are those associated with the two-tailed t-statistic for testing the difference between two sample means. tested are labeled by perceived case. The sample means being For example, the column titled A:B compares the mean for Case A to that of Case B and reports the associated probability. A:B C:D Tests POP when PRC is low. Tests POP when PRC is high. In summary, Tests PRC when POP is low. Tests PRC when POP is high. Milli-n1 79 was held constant or, alternatively, the effects of POP could be considered while PRC was held constant. Items 9 and 10 were designed to address specifically the change in equity that underlies PRC by focusing on the personal outcome-input relationship. Item 9 stated "Considering the terms of my compensation agreement, I was happy to be able to cooperate on the Systems Lab project." As expected, the happiest group consisted of those participants who perceived themselves to be in Case A (low PRC, low POP) and the least happy were those persons perceived to be in Case D (high PRC, high POP). When POP was low, the high-PRC group (Case C) was significantly (p=.055) less happy than was the low-PRC group (Case A). In contrast, the high-POP participants (Cases B and D) reported no significant difference across PRC levels on responses to Item 9 (although the trend echoes the low-POP cases.) Overall the responses seem to indicate that, as expected, the presence of PRC or POP (or both) decreased the motivation to cooperate on the systems lab project. Item 10 stated, "The cost to me of participation on the Systems Lab project was too high, considering the benefits I derived from it." At the end of the session, all groups reported that the cost was not too high, but the responses showed a significant difference in opinion (p=.061) between the low and high PRC groups when POP is low (that is, between Cases A and C.) As with Item 9, the pattern shown by the low-POP groups was repeated by the high-POP groups, 80 but not at a statistically significant level. These results suggest that the cost of Systems Lab cooperation was perceived throughout the experimental session to be higher to high-PRC participants than to low-PRC participants. Items 6 and 8 were used to examine the manipulation of perceived organizational pressure by looking at outcomes and inputs of both the participants and the comparison group of students. Item 6 said, "Considering my efforts, I got about the same benefits out of working on the Systems Lab project as the Ohio State University students probably received from doing a similar project." All groups agreed with the statement, but when POP was low, agreement was higher than when POP was high. Thus, the levels of POP seem to have persisted throughout the experiment. Note that the difference between low and high POP groups was significant if PRC was also low (Cases A and B), but not if PRC was high (Cases C and D); the implications of this pattern are addressed in Chapter 6. Item 8 also addressed POP by stating, "The Ohio State University students received unfair compensation on their Systems Lab project, considering my own efforts and compensation tonight." As expected, if original perceptions were maintained throughout the session, the low POP participants (Cases A and C) disagreed with Item 8 and the high POP participants (cases B and D) agreed. When PRC was controlled, differences between low and high POP groups are highly significant (p < .01). 81 Besides learning that the manipulations perceived at the start of the session seemed to persist to the end of the session, Section C of the Exit Form yielded information about the participants' views of their own and others' inputs and outcomes. Responses indicated that participants felt they had relevant knowledge to contribute to the Systems Lab project (Item 3), but they did not feel that they learned much from the project (Item 5). Compared to the participants, students at other colleges were not seen as more qualified for (Item 2) nor more interested in (Item 7) helping with the Systems Lab project. Further, the Systems Lab assignment was not considered to be harder than the Accounting Lab task (Item 4). Differences in responses to certain of these items across perceived cases are evident and are discussed in Chapter 6. CHAPTER 5 - RESULTS Chapter 4 presented an experiment designed to test a variation of the theoretical model of IS development friction presented in Figure 3. Operational definitions of variables and the results related to these variables were discussed. A descriptive statistical summary of these experimental variables is presented in Table 20. In this chapter, this data are utilized to complete the description and tests of the model. The null hypothesis when testing a causal model is that the data do not materially depart from the model. Completion of such a test requires the estimation of path coefficients (Pji) and an assessment of the goodness of fit. With seven variables, the experimental data yielded twenty- one correlations (rij). The correlation matrix of observed results is presented in Table 21. Predicted values of seven of these correlations were constrained to be their actual values during estimation of the path coefficients and could not be used to test the model. These seven correlations are discussed in Part 5.1. In addition, the correlation between the exogenous variables of resource conflict (R) and organizational pressure (0) could not be examined. By utilizing equal cell sizes in the experiment, these variables were uncorrelated by definition. The remaining thirteen correlations were used to test the fit of the model, which is examined in Part 5.2. 82 DESCRIPTIVE STATISTICS FOR EXPERIMENTAL MEASURES 83 TAEHHB 20 THEORETICAL VARIABLE' was OBSERVE WE‘- fl' IEAN :3: MIN. MAX. III. MAX. ?;::::;; °°nfli°t 0.00 1.00 0.00 1.00 0.50 0.50 0.50 :zfigiizgd R°‘°“’°° 0.00 30.00 0.00 30.00 10.00 10.89 10.01 Per°°iv°d R?‘°“‘°° 0.00 1.00 0.00 1.00 0.00 0.46 0.50 Conflict (Binary) gigzziiztfggziry) 0.00 1.00 0.00 1.00 0.50 0.50 0.50 Perceived Organizational 0.00 1.00 0.00 1.00 0.0 0.28 0.45 Pressure (Binary) Frictionb -m +m -s.00 20.00 5.00 5.33 4.67 Interaction Qualitycz Average IQ -3.00 3.00 -0.87 2.84 1.80 1.69 0.69 Summary IQ -3.00 3.00 -1.37 3.00 2.00 1.68 0.97 Compatibility -3.00 3.00 -0.87 3.00 1.91 1.74 0.76 Work-Related Needs -3.00 3.00 0.00 3.00 1.75 1.65 0.67 IS-Attituded: Average IS-Attitude -3.00 3.00 —1.01 2.95 1.15 1.01 0.83 Summary IS-Attitude -3.00 3.00 -1.87 3.00 1.50 1.48 0.91 Relative Advantage -3.00 3.00 -l.64 3.00 1.53 1.36 0.99 Image -3.00 3.00 -3.00 2.88 0.00 -0.14 1.36 Ease of Use -3.00 3.00 -2.03 3.00 1.40 1.24 1.03 aAll measures are continuous within their ranges unless noted otherwise. bWhile the theoretical range is unbounded, the information given to the participants in the experiment was expected to drive their estimates of friction toward the range of $0.00 to $30.00. The observed range given here does not include the two dropped observations of -$30.00 and $425.00. cSee definitions on Table 17. dSee definitions on Table 18. 84 TAEHHB 21. THE MODEL'S MEASURED VARIABLES CORRELATION MATRIX PERCEIVED mmx- PERCEIVED INTER- REsuncs mmu- IS- RESGRCE ZATIGIAL flucnm ACTIGI cn'FLlc' CONFLICT PRESSURE 2"‘°“‘L QUALITY “IT'IUDE Peasants R R' o 0' F a A a 0.0000 0.0000 0.1612 -0.0400 ”0‘0208 0.5854 R' (0.0001) 0.0000 0.0683 0.0355 0.0000 0.0000 0 (1.0000) (1.0000) 0.0831 -0.0206 -0.0107 0.1158 0.1221 0.6255 °' (0.2610) (0.2361) (0.0001) ‘°°°33° '°°°171 6 0.2544 0.2916 0.0026 0.1665 -0 1289 (0.0133) (0.0043) (0.9801) (0.1088) ° 0 -0.1024 '0.2851 0.0531 '0.0815 -0.2481 (0.3207) (0.0049) (0.6074) (0.4291) (0.0159) A -0.0202 -0.1189 0.2030 0.0547 -0.0359 0.5195 (0.8452) (0.2484) (0.0473) (0.5969) (0.7312) (0.0001) Cell entries in the lower triangle are the observed Pearson Correlation Coefficients (p-values under null hypothesis that the true correlation is 0.00), uncorrected for measurement error. Cell entries in the upper triangle are the correlations predicted by the path model. 85 5.1 Path Coefficients Six direct relationships were hypothesized by the tested model. For each of these hypotheses, a structural equation describes more completely the relationship between the variables. The hypotheses, the related structural equations, and path coefficients are detailed below. Participants assigned to treatments where resource conflict and organizational pressure were high were expected to perceive less equity in their treatment than did those participants assigned to the low treatment conditions. Such perceptions of equitable treatment should have been reflected in the measures of perceived resource conflict (R') and perceived organizational pressure (0'), as stated by the following hypotheses. H1: The level of perceived resource conflict is positively correlated with the experimental manipulation of resource conflict. H2: The level of perceived organizational pressure is positively correlated with the experimental manipulation of organizational pressure. The structural equations for these first two paths (where s and t are error terms) are shown below. (1) R’=pR7RR+s=rRR/R+s (2) O’=p0700+ t=rw70+ t Since these variables each have only one predictor, the path coefficients were estimated to be equal to their correlations. That is, 86 (3) pR/R=IRR/=0.5854 (4) be=ray=O-5255 Both of these path coefficients were statistically significant (p=.0001). Perceived resource conflict and perceived organizational pressure were measures of the success of the experimental manipulations. The brevity and starkness of the experimental situation likely attenuated the correlations so that they are not as high as they would be in a more realistic situation. The levels of equity perceived in the PRC and PCP assessments were anticipated to cause participants to be more or less resistant to IS involvement. This resistance was measured by the friction (F) variable. Friction was expected to be a linear combination of the level of perceived resource conflict and perceived organizational pressure, as stated by Hypotheses 3 and 4. H3: The level of friction is positively correlated with the level of perceived resource conflict. H4: The level of friction is positively correlated with the level of perceived organizational pressure. The experimental results did find positive correlations between perceived resource conflict and friction (rR.F = 0.2916, p=0.004) and between perceived organizational pressure and friction (rb.F== 0.1665) although the latter correlation was below standard levels of significance (p=0.109). 87 The structural equation for friction is shown below. (The standard error of the estimate is symbolized by u.) (5) F=pFRIR’+pFo/O’+u Because friction had more than one cause in the model, the path coefficients were estimated by the standardized multiple regression coefficients. The formulas for these two coefficients utilized three of the observed correlations. r —r r (5) pm): RIF 3:" 0’1’=O.Z754 l-IR’O’ I'I-r/II/ (7) pF07= 0" R20 RF=O.1328 l‘IR/O/ Both path coefficients were positive as expected. The path from perceived resource conflict to friction is significant (p=0.008), but the impact of perceived organizational pressure upon friction is somewhat small and not significantly different from zero (p=0.200). Note that the coefficient for perceived resource conflict was more than double that for perceived organizational pressure. Higher levels of resistance to interaction on the systems project were expected to manifest as poorer relationships during the ensuing interaction. That is, higher levels of friction were expected to decrease interaction quality (Q) as stated in Hypothesis 5. H5: The quality of interaction between user and developer is negatively correlated with the level of friction. 88 Interaction quality was represented by the following structural equation (where error in measuring interaction quality is symbolized by the term v.) (8) Q=prF+ v=rFQF+V Since the tested model hypothesized only one factor affecting interaction quality, the path coefficient was estimated by the correlation between friction and interaction quality. (9) pQF=rFQ=-O.2481 This coefficient had the predicted sign and was statistically significant (p=0.02). Finally, the last causal link in the model was the impact of interaction quality upon the participant's attitude toward the developing system. The higher the level of interaction quality, the better the participant's attitude toward the system should have been, as stated in Hypothesis 6. H6: The user's attitude toward the information system under development is positively correlated with the quality of interaction between user and developer. The user's attitude (A) toward the information system being developed was represented by the following equation, which used the observed correlation between interaction quality and attitude. (Error of measurement in the attitude variable is symbolized by w.) 89 (10) A=pAQQ+w=rQAQ+w (11) pAQ=rQA=O .5195 As with the other single predictor paths, the path coefficient was estimated by the observed correlation. This coefficient displayed the expected sign and was statistically significant (p=.0001). Figure 4 summarizes the preceding parameter estimates by showing graphically how they relate to the model. Equation references on the figure relate the path estimates to the equations above. All path coefficients were statistically significant at the 0.05 level, except for pmy (p=0.20), and all have the expected signs. Three paths related to the manipulation checks were specifically excluded from the model because RC and OP were expected to be perceived as independent constructs. The excluded paths were between resource conflict and perceived organizational pressure (p = 0.261), between organizational pressure and perceived resource conflict (p = 1.000), and between perceived resource conflict and perceived organizational pressure (p = 0.236). These non-significant estimates suggest that the independence assumption might be valid. 5.2 Goodness of Fit The thirteen observable correlations not utilized in parameter estimation provided tests of the model. Two tests were conducted. First, the model was checked on an overall 90 .Hmpoe may DOTMTH 90: on camp 0:» .msna .Ammm.oumv unmowwwcmwn uoc ma teovwmwu mo nmnnmoe «a run; .roasz ms.va mo oaunaunun munsemuano n neanas 00608 nae» now naeeeanmu «0 0600 annoam era .Aom.umv coauowuu on wuanmwum Hmcoaumuwcmouo vm>wwoumm Scum summ 0n» u0m ummoxm Hm>ma mo.um 0:» Sodom no um acmoamacman mum Add .ncmwn pmuodnmum wwwsu huumo nounfiaumm spam Add on umummno cw acowumsvm 0:» cu umuou mammsucmumm cw mumnssc one "mmuoz wHJDwmm Om2_<._.m0 ”v MKDOI mmawwmmm mmmemmm ..__w0mma 0N0 o ..550 -m. A 83 295592. A memo- 20.5.”... 9: SC The 5 EOZEZOO 850me A 83 wwmwwwomw 828%.. 5 91 basis with the Chi-Square test of residuals. Secondly, correlations were examined individually. Both of these tests were conducted on the model using the factor-analyzed definitions of interaction quality and IS-attitude and again utilizing alternative definitions. 5.2.1 Tests using the Basic Variable Definitions The predicted correlations (fij) were obtained from the path estimates derived in the previous section. The equations for the predicted correlations and the numerical predictions derived from the experimental data are listed below. (12) fRF=pR/Rpm/=rmpm/=O . 16 12 (13) £01,: pO/opFOFroo/pFOF 0 . 0 8 3 1 (14) fRIQ=pm7pQF=pFR7rFQ= -O . O 6 83 (15) fRQ=pRIRpmeF=rRR7pFRIIFQ=-0 . 0400 (16) £079: P17049017: pFO’IFQ= - 0 . 0 3 3 0 (17) foQ=polopF0pQF=roopF07rFQ= — O . 02 0 6 (18) fFA=pQFpAQ=IFQrQA=—O .1289 (19) fR7A=pFRpQFpAQ=pFRIIFQIQA= —0 . O3 55 (2°) fRAsz’RpF'RI-pQFpAQ=IRR4pFR’rFQrQA:_0 ' 0208 (2 1) f0’71 :pFO’pQFpAoz-pFO’rFQIQA: " 0 ' 0 17 1 (2 2 ) f0.411 =pO’OpF0pQFpAozr oo’PFo’IFQI p.11: ' 0 ' O 1 07 92 An overall test of the model was performed using the residuals found in Table 22. These fourteen residuals, ek, were computed by subtracting each unconstrained observed correlation, r. from its associated predicted correlation, ij' Iij. If the residuals taken as a group are too large, the global test will reject the model. The formula for the test statistic is shown below, where n is the number of participants. (23’ x2=n2 e§=14.76 With fourteen degrees of freedom, this Chi-Square statistic is associated with a probability of 0.395. Thus, the data were not sufficient to reject the model on a global basis. However, since several of the residuals were quite large, this test must be regarded cautiously. The large residuals could indicate problems with specific portions of the model. The observed correlations were obtained from a sample of persons subjected to IS-development friction, not from the entire population of such persons. If the model is a good fit, the predicted correlations should fall within sampling error of the observed correlations. Table 23 presents the 95% confidence intervals for each of the observed, unconstrained correlations. Twelve of the fourteen predicted correlations fell within these limits; 93 TAIHHB 22 DIFFERENCES BETWEEN PREDICTED AND OBSERVED CORRELATIONS PERCEIVED ORGANI- PERCE'VED INTER- :‘ofi‘flifi‘. 33:3: 33$ 211%; mm mg, "#111... HESUE R R' o 0' P Q A R R. 0 0.00008 0.0000 0' -0.1158 P -0.0932 0.0805 Q 0.0624 0.2167 -0.0737 0.0487 A 0.0006 0.0834 -0.2138 -0.0718 -0.0930 aCell entries are the residuals derived by subtracting each observed correlation coefficient from its related predicted correlation coefficient. (See Table 21.) The Test of Residuals yields a Chi-Square statistic of 14.76. With 14 degrees of freedom, the associated probability is 0.395. two did not.9 While one out of fourteen correlations might have been out of bounds by chance, having two out of bounds suggests a need to closely examine the structure of the model. The two predicted correlations that were outside of the confidence interval are fng, the correlation between perceived resource conflict and interaction quality, and ‘?0A, the correlation between organizational pressure and IS-attitude. The predicted correlation, IEVQ, was well above the upper bound of the confidence interval around the 9This pattern was robust to changes in the interval. The twelve correlations that were within the 95% confidence interval were also within a narrower 90% interval. 94 TABLE 2 3 CONFIDENCE INTERVALS AROUND OBSERVED CORRELATIONS OBSERVED STANDARD 95: CONFIDENCE INTERVAL PREDICTED unumuun sun? tn£amwu1 LHERNNUI umuuuun rho 0.0000 0.1026 -0.2011 0.2011 0.0000 IRO' 0.1158 0.1012 -0.0825 0.3142 0.0000 rRF 0.2544 0.0960 0.0663 0.4425 0.1612 rho -0.1024 0.1015 -0.3014 0.0965 -0.0398 IRA -0.0202 0.1025 -0.2212 0.1808 -0.0205 rR,o 0.0000 0.1026 -0.2011 0.2011 0.0000 rmo -0.2851 0.0943 -0.4698 -0.1003 -0.0681b rRM. -0.1189 0.1012 -0.3172 0.0793 -0.0349 r0F 0.0026 0.1026 -0.l985 0.2037 0.0831 r00 0.0531 0.1023 -0.1474 0.2536 -0.0205 IDA 0.2030 0.0984 0.0102 0.3958 -0.0105c r0.Q -0.0816 0.1019 -0.2814 0.1181 -0.0328 roux 0.0547 0.1023 -0.1458 0.2551 -0.0169 ITA -0.0359 0.1025 -0.2367 0.1649 -0.1269 1-r2 n-l SHT= 8The standard error of the estimate is calculated as: bThe confidence level would have to be raised to 98.0% in order for the predicted correlation to be included in the interval. cThe confidence level would have to be raised to 97.1% in order for the predicted correlation to be included in the interval. 95 observed correlation. The interval would have to be widened to a 98.0% confidence level in order to include it. The second outlying predicted correlation, ?tm1 was below its associated lower bound. The interval around r0A would have to be expanded to a 97.1% confidence level for ‘IOA to fall within the limits. Furthermore, the sign of the observed correlation between organizational pressure and IS-attitude was positive, but the model predicted a negative relationship. Both of these findings have implications that will be explored in Part 5.4.2. 5.2.2 Tests with Alternative Variable Definitions The two outlying correlations discussed above involve either interaction quality or IS-attitude. Since the experiment collected data to support a variety of measures for both IQ and IS-attitude, the Chi-Square test of residuals and the examination of confidence intervals were repeated using different operational definitions of interaction quality and IS-attitude. Table 24 summarizes the results. Panel A of Table 24 repeats the results reported in Part 5.2.1. IQ and IS-Attitude were both measured using the factor-analyzed scales, which are unweighted averages of the items left in the scales after factor analysis. In Panel B, one or both of the summary measures for IQ and IS-Attitude were utilized. These measures were the 96 ITEHHB 24 SUMMARY OF STATISTICAL TESTS OF MODEL USING ALTERNATIVE DEFINITIONS OF INTERACTION QUALITY AND IS-ATTITUDE ‘2 PRWTIU 0F wRRELAnous 01mm“: IS-ATTITLDE snnsn P VALUE INSIDE 95% mom“ 10 1am mane ' CONFIDENCE C INTERVAL manu- THIS PANEL A Average IQa Average IS— 14.756 0.395 12/14 tmm, Attitude tOA PANEL B Summary IQb Summary IS— 10.594 0.718 14/14 None Attitude Average IQ Summary IS— 12.102 0.598 13/14 31210 Attitude Summary IQ Average IS- 12 . 227 0 . 588 13/14 rOA Attitude PANEL C Compatibilityc Relative 18.402 0. 189 12/14 two, Advantage tbA Compatibility Image 15.249 0.361 13/14 tR.Q Compatibility Ease of Use 18.678 0.178 12/14 ramv WA Needs Relative 13.545 0.484 13/14 rm Awareness Advantage Needs Image 10.638 0.714 13/14 tR'o Awareness Needs Ease of Use 16.117 0.306 13/14 I'R,A Awareness §Average IQ is the unweighted average of the ten items left in the scale after factor analysis. Average IS—Attitude is the unweighted average of the fourteen items remaining in the scale after factor analysis. bSummary IQ and Summary IS-Attitude are each one-item overall assessment measures reported by the user participant on the Exit Form. cFactor scores are the unweighted average of the Exit Form item scores. (See Tables 17 and 18.) 97 one-item assessments reported by participants. The global tests continue to support the model, and the incidence of outlying correlations was improved. However, the correlation between Summary IQ and Friction dropped to -O.153 (compared to -0.248 for Average IQ), and the correlation between Summary IQ and Summary IS-Attitude fell to 0.340 (compared to 0.520 for Average IQ and Average IS- Attitude). These lower correlations tend to make the model less interesting and relevant because they imply that the exogenous variables have very little impact upon the endogenous variables. The Summary measures also do not allow an assessment of the impact of measurement error, which is the topic of Part 5.3. Finally, in Panel C, individual factors were used to represent the more comprehensive IQ and IS-Attitude measures with very similar results. Overall, Table 24 presents a relatively uniform picture of the tests of the model, which appear to be robust to a variety of definitions of two of the endogenous variables. These results indicate that the model could not be globally rejected based upon the data, but that there were two outlying correlations that warranted further consideration. 5.3 Measurement Error Measurement error in any of the variables affects the path estimates derived from the experimental data. However, only if an intervening variable is measured with error will 98 the fit of the model be affected. The fitness test results reported above could have been influenced by measurement error in the variables of perceived resource conflict, perceived organizational pressure, friction, or interaction quality. Reliability estimates were available for the variables interaction quality and IS-attitude. These reliabilities were used to correct for measurement error in all correlations involving either variable. The formula to correct a correlation for attenuation due to measurement error is: (25) r’ =___£1__ Application of this formula to all correlations involving either interaction quality or IS-attitude yielded the correlation matrix found in Table 25. The interaction quality and the IS-attitude reliabilities, 0.846 and 0.878 respectively, are found on the diagonal. The Chi-Square goodness of fit test applied to residuals after correcting correlations for measurement error yielded a Chi-Square statistic of 17.24 (p=0.244). The data still did not reject the model, but the fit deteriorated. Repeating the comparison of predicted correlations to the 95% confidence intervals drawn around the corrected correlations yielded substantially similar results. The same two predicted correlations were outside the intervals: 99 1EABIJ3.25 THE MODEL'S MEASURED VARIABLES CORRELATION MATRIX-- CORRECTED FOR MEASUREMENT ERROR IN INTERACTION QUALITY AND IN IS-ATTITUDE PERCEIVED ORGAII- "WE'VE” 1111511- 339335 3:35: 33:; 3.1%.; nucnou 3m, m8}... PRESSURE R R ' 0 0 ' I" Q A R 0.0000 0.0000 0.1612 -0.0435 -0.0262 R' 0.5854 0.0000 -0.0743 -0.0448 0.0000 0.0000 0.0831 -0.0224 -0.0135 ' 0.1158 0.1221 0.6255 -0.0358 -0.0216 0.2544 0.2916 0.0026 0.1665 -0.1626 -0.1114 -0.3099 0.0577 -0.0888 -0.2697 0.8460 A -0.0216 -0.1269 0.2167 0.0583 -0.0383 0.6028 0.8780 Cell entries in the lower triangle are the observed Pearson Correlation Coefficients (p-values under null hypothesis that the true correlation is 0.00), corrected for measurement error in Interaction Quality and IS- Attitude measures. Cell entries in the upper triangle are the correlations predicted by the path model. The Test of Residuals yields a Chi-Square statistic of 17.24. With 14 degrees of freedom, the associated probability is 0.244. 100 (1) the correlation between perceived resource conflict and interaction quality and (2) the correlation between organizational pressure and IS-attitude. All other predicted correlations were within sampling error of the corrected correlations. These tests do not rule out measurement error as the cause of the outlying correlations. The two perceived variables and the friction variable almost certainly were measured with error. However, the experimental design did not contain provision for assessment of the error in these other variables. 5.4 Possible Explanations for the Findings In summary, the overall fit of the model was not rejected by the data although two indirect paths did not match the observed results. These two paths were: (1) The path from perceived resource conflict through friction to interaction quality. Although carrying the correct sign, the observed correlation, IR} , was much larger in absolute value than the model predicted. (2) The path from organizational pressure through perceived organizational pressure, friction, and interaction quality to IS-attitude. The observed correlation, rbA, was not only much larger in absolute value than predicted by the model, it also was the wrong sign. At least two potential explanations can be given for these results. Experimental or statistical artifacts might have been present, or the model as drawn might not be a true and full description of reality. 101 5.4.1 Experimental or Statistical Artifacts Three artifacts arising from an experiment or the statistical analysis of its data can cause poor results from model testing. The first of these is sampling error, the effect of which was explored in Part 5.2.1 by utilizing confidence intervals drawn around the obtained correlations. The second artifact is measurement error, and it was considered in Part 5.3 with respect to the two variables (interaction quality and IS-attitude) for which data allowed computation of corrected correlations. Still to be discussed is the measurement error impact upon other variables. Further, the third potential artifact, restriction in range, remains to be considered. 5.4.1.1 Restriction in Range Restriction in the range of values that a variable can assume may attenuate the correlations of that variable with others. Restriction in range can occur only in the exogenous variables of a model, which are resource conflict and organizational pressure in the model as tested here. Since both of these variables were constrained to a binary distribution, restriction in range was possible. However, in path analysis, if such restriction did occur, only the path coefficients would have been affected. There would have been no impact from restriction in range on the fit of the model. 102 5.4.1.2 Measurement Error in Friction Measurement error in intervening variables affects both the parameter estimates and the fit of a model. Friction was an intervening variable within the paths related to both of the outlying correlations. Since the operational measure was newly created for this experiment, friction was very likely measured with error. Due to constraints from the experimental environment, no data were gathered to allow for correcting the correlations as was done for interaction quality and IS-attitude in Part 5.3. However, sensitivity analysis of potential measurement error allows an assessment of its effect on the fit of the model. Table 26 presents correlations that would be attained if friction were corrected for potential measurement error in a range of magnitudes. The last column contains the new predicted correlations between perceived resource conflict and interaction quality. As noted on Table 23, the observed correlation between these two variables was -0.285 and the 95% confidence interval around that observation extends from -0.470 to -0.100. Thus, if the reliability of the friction measure was around 0.75 or lower, measurement error could be the answer to the model's first outlying correlation. The likelihood of the reliability of the friction measure being below .75 is probably high. It received little pretesting and relied upon the response to what must have seemed an unusual question to somewhat harried participants. 103 TAEHHB 26 SENSITIVITY ANALYSIS OF THE RESULTS TO MEASUREMENT ERROR IN FRICTION CORRECTED CORRECTED C0$13T11110N CORRELATION CORRELATION BETWEEN PERCEIVED BETWEEN PERCEIVED BETWEEN FRICTION RESOURCE CONFLICT RELIABILITY RESOURCE CONFLICT AND INTERACTION AND INTERACTION AND FRICTION‘ QUALITY“ QUALITY (Observed) 1.0 0.2916 -0.2697 -0.0786 0.9 0.3074 -0.2843 -o.0874 0.8 0.3260 -0.3016 -0.0983 0.7 0.3485 -0.3224 -0.1124 0.6 0.3765 -0.3482 —0.1311 0.5 0.4124 -0.3815 -0.1573 0.4 0.4611 -0.4265 -0.1966 0.3 0.5324 -0.4925 -0.2622 0.2 0.6520 -0.6031 -0.3933 0.1 0.9221 -0.8530 -0.7865 ‘This column contains corrected correlations between perceived resource conflict and friction, not path coefficients. The formula for correcting the observed correlation for attenuation due to measurement error in one or both variables is: r I” =-——"" "” 7:75. hThe correlations in this column are corrected for potential measurement error in friction and for actual measurement error in interaction quality. 104 Measurement error in friction cannot supply an explanation for the second outlying correlation. Correcting for attenuation makes correlations larger in absolute size but does not change their signs. Thus, measurement error in friction may be the problem with one of the outlying correlations but not the other. 5.4.2 Misspecified Model As discussed above, the statistical artifacts of the experiment--sampling error, measurement error, and restriction of range--do not adequately explain all of the results of the path analysis. Thus, concluding that the model needs to be adapted seems plausible. When correlations larger than predicted by path analysis are observed, the explanation might be that one or more mediating variables are missing from the model. If a mediating variable was influenced by perceived resource conflict and in turn affected interaction quality, the sub-model would look like Figure 5. The signs of the path coefficients connecting the new variable to perceived resource conflict and interaction quality would have to be opposites, so as to contribute to a relatively large and negative correlation. The identity of such a variable is unknown, but an examination of the various interaction quality measures served to provide a starting point for the search. Table 27 summarizes the IQ data available for the two overall IQ scores and the individual factor scores. 105 FL PERCEIVED RESOURCE CONFLICT (+) I FRICTION (-) 0R (+) > MEDIATOR (+) 0R (-) V (+) INTERACTION I QUALITY _.=J FIGURE 5: SUB-MODEL FOR PERCEIVED RESROUCE CONFLICT TO INTERACTION QUALITY 106 TAEHHB 27 ANALYSIS OF INTERACTION QUALITY MEASURES ON THE PERCEIVED RESOURCE CONFLICT SUB-MODEL OBSERVE OBSERVED . CORRELATION CORRELATION ass CONFIDENCE INI'ERVAL BETWEEN BETWEEN MEASURE Ol' FRICTION AM) PERCEIVED PREDICTED unmucnme numucnmfi Rummwn "mm“m’ ammy QUALITY 00“,”. CONFLICT AND “no". LATION' INTERACTION LOWER ROUND UPPER BOUND QUALITY 23erage -O.2481 -0.2851 0.0943 -0.4698 —0.1003 “-0.0723 gammary -0.1534 -0.1306 0.1009 -O.3282 0.0671 "-0.0447 9°9P?t‘ -0.2s94 -0.3278 0.0916 -0.5073 -0.l484 -0.0756 ibility Work- Related -0.2217 -0.2147 0.0979 -0.4065 -0.0229 -0.0646 Needs ‘The correlations in this column are not corrected for measurement error. "The standard error of the estimate of the correlation between perceived resource conflict and interaction quality is calculated as: 1-1:2 SE'= cThe predicted correlation is the product of the correlation between perceived resource conflict and friction (which is 0.29161 in all instances above) and the correlation between friction and the interaction quality measure. 107 As shown in column 3, Summary IQ does not relate to perceived resource conflict in the same way as the other two measures. In fact, if Summary IQ were the measure of interaction quality used for model tests, this path would no longer be troublesome (nor would it be very interesting since the path coefficient would be non-significant.) This variation in the perceived resource conflict to interaction quality relationship implies that when participants gave an overall estimation of the quality of interaction, they intuitively defined the construct differently from that derived from either the ten-item scale or its two factors. Further, this conceptual difference could be related to the omitted mediator, if in fact there is one. Equity theory suggests at least one candidate for an omitted variable. For example, suppose that the missing mediator in Figure 5 is the amount of time a person spends actively thinking about the problems presented in the systems development project. This is a construct that could be called "mental involvement." A person who is faced with a systems development project and who perceives a resource conflict (due to a potential negative change in personal equity) may decide to make some adjustments. One way the participant may improve the input/output ratio is to attend meetings but refuse to be mentally involved with the project at any significant level. If this situation occurred, perceived resource conflict and mental involvement would be negatively correlated. Mental involvement, in turn, is 108 likely to be positively correlated with interaction quality, as defined by the interaction quality scale. Whether or not the preceding explanation is the answer to the current dilemma is unknown, but it illustrates how a missing mediator could be integrated into the facts drawn from this experiment's data. The correlation observed between organizational pressure and IS-attitude was not only too large in absolute size, but it carried an unexpected positive sign. Thus, if there was a missing mediator affected by organizational pressure and in turn influencing IS-attitude, it must have been positively correlated with both variables.1° Furthermore, the one path in the model that is not significantly different from 0.000 is the path from perceived organizational pressure to friction. These two relationships indicate that the model, as it relates to organizational pressure acting upon friction, was misspecified. 10The missing variable could be negatively correlated with both organizational pressure and IS-attitude if the construct is "reverse— worded." A positive correlation can be assumed with appropriate definition of the construct. CHAPTER 6 - DISCUSSION 6.1 Summary of the Project This dissertation constructed a new model for the IS development setting using equity theory to predict behaVior. The model was tested experimentally and, except for the organizational pressure impact, was found to be supported. A summary of significant findings and contributions derived from the model construction and testing activities follows. 6.1.1 Model Construction This research contributes to the IS literature by constructing and testing a model of IS-development behavior based upon the MIS Law set forth by Cushing (1990). This law was previously undeveloped and untested. The new model captures the notion of IS project-related friction between users and system developers based upon the physical definition of friction. In this model, friction is viewed as resistance by a potential participant to interaction during IS development. Resource conflict and organizational pressure are causes of increased levels of friction while incentives can be used to decrease friction. The immediate consequence of friction in the IS- development environment is the impact upon interaction quality. Although high levels of friction will impair interaction quality, interaction quality can be higher than expected due to the use of countermeasures by someone in the 109 110 organization. Interaction quality directly affects one common measure of system success, the participant's attitude toward the new system. Since resistance to involvement can be attributed to issues of both utility and fairness, equity theory was utilized to define the model's exogenous variables and to explain the incidence of friction during IS development. The theory seemed to be especially suitable in light of comments made by participants during the experimental sessions, such as "Why would I want to do this Systems Lab project? It's not going to do anything for me," or "It's not fair that the OSU students get more money than we do." 6.1.2 Model Testing The model was tested experimentally by setting up a hypothetical firm, hiring students as employees and potential users of a new system, and exposing them to a system development project under a variety of conditions. Anecdotal evidence indicated that a good simulation of a real work environment was achieved. After debriefing, many participants expressed surprise that TMW was not a real firm. Kerlinger (1986, p. 367) lists the major strengths and weaknesses of most lab experiments for hypothesis testing. This experiment was no exception. Two major benefits arose from the use of a lab experiment for testing this new model. First, the operational definitions of all variables could be 111 quite specific. In particular, manipulation of resource conflict and organizational pressure could be uniformly handled for all participants. In a field study of resource conflict and organizational pressure, such controlled manipulation would not be possible, and participants might have distinctly different opinions about the meaning of the variables. The lab experiment avoided such interpretation problems. Secondly, experimental testing allowed a high level of control to be exerted over some potentially confounding extraneous variables. With respect to this model, the experimentally controlled variables included prior confederate bias, task structure uniformity, and IS- development group size. However, like most experiments, the conditions were somewhat contrived. This artificiality generally results in a lack of strength in the independent variables (Kerlinger, 1986). In this particular setting, the rewards to participants were relatively low. The work session was limited to one evening, and the comparison group was not personally known to the participants. Because of these conditions, the experimental results might be weaker than would be seen in real-world situations. 6.1.3 Construct Development Several new constructs were developed for this project, and the related operational measures are likely to benefit from further refinement and testing. 112 Resource conflict was conceptually defined as the amount of user-controlled resources requested by the developer that were not willingly given. In the experiment, the contested resource was participant time. Certainly this is not the only resource that could be constrained in the field. Future research should investigate both time and other constraints, such as effort, ability, and knowledge. Organizational pressure was developed as an equity comparison between the user and some other comparable party with respect to changes being brought about by the new system. The manipulation of organizational pressure in the experiment altered only the benefit to the other comparable party, and the other party was a class of persons not personally known to the participants. Additional investigation into alternative definitions of inequitable comparisons might examine the other components of the equity equation or equity comparisons with different classes of other parties. Participants were asked to report on their perceptions of the exogenous variables through their answers to one-item measures. These measures were somewhat less refined than might be desired and did not allow the assessment of measurement error. Participants might not have been clear in their own minds as to what their normal task was to be or what was being asked of them with respect to the system project. The results of these manipulation checks clearly showed that perceptions of the same stated time constraint 113 or specific comparative inequity differed across participants. Practical application of the model must allow for the fact that one individual will not view constraints or comparisons the same as another might. Future research could focus on the derivation of measures of both perceived resource conflict and perceived organizational pressure that could be utilized in the field. Without such measures, anticipation and control of friction will be difficult to achieve effectively. As an unexplored concept in the IS environment, friction was interpreted to be resistance by a potential participant to IS development activity. The amount of friction was measured as the dollar value that would move the participant from being negative or neutral about the prospect of interaction to being positive about it. As an operational measure, this construct has two very desirable properties: (1) It is easy to understand and (2) it is not context specific. However, several other properties keep the measure from being perfect. First, potential IS-development participants might need a period of time to contemplate their situations and consider the dollar value of their unhappiness. Experimentally, a realistic amount of time seemed difficult to provide, but in real-world situations this time constraint should not be present. In addition, the friction measure is susceptible to differences in utility functions held by participants. Knowledge of important utility-laden 114 factors would be very helpful to the assessment of costs and benefits underlying the resource-conflict construct. Finally, the friction measure needs to be more than a one- item question so that measurement error can be assessed. The last new measure developed for this project was the interaction quality instrument. The IQ construct underlying the instrument differed from prior works conceptually by being restricted to the participant's opinion regarding the process of development. The instrument developed was subjected to several pretests and confirmatory factor analysis. The instrument likely is context sensitive, and future research to expand and refine this instrument is desirable. Although not newly developed for this experiment, the Moore and Benbasat (1991) short instrument for measuring information technology innovation adoption was adapted and used to measure IS-attitude. IS-attitude was conceptually restricted to items about the systems product under development. The confirmatory factor analysis and results of the experiment yielded additional evidence of the reliability and validity of the Moore-Benbasat scale. 6.2 Major Conclusions After development and testing of the model, path analysis of the data indicated that the model was largely supported. All of the direct paths in the model (see Figure 115 4) were statistically significant except the path from perceived organizational pressure to friction. These path estimates relate to several important relationships. Time constraints that could hinder a worker's ability to realize expected equity do lead to perceived resource conflicts. Friction or resistance to IS- development does go up when a potential participant perceives a conflict over where to place efforts. And, friction has an impact upon the user's perception of how well IS development is proceeding. At the start of this dissertation, the user's opinion about IS development as a process was assumed to have a bearing upon the user's opinion about the eventual usefulness of the systems product generated by the development effort. This relationship also was found to exist. Since the user's attitude towards the system has been shown to affect the actual usage of the implemented system (Baroudi, Olson, and Ives, 1986; Maish, 1979; Lucas, 1978), allowing friction to arise in the systems environment will affect the organization's success with information systems. The data showed that perceived organizational pressure had no significant impact on friction, but anecdotal evidence suggested that experimental conditions contributed to the low path parameter in two ways. First, since the comparable others were unknown to the participants, the statement that the others received more money did not seem as unfair as it might if the others were actual coworkers. 116 Secondly, organizational pressure may need some time to work. Just as in physical terms, a fixed amount of pressure over the long term can be more destructive than a quick hit, the short term of the experiment may not have allowed pressure to accumulate. These two conditions--unknown other parties and little time to fume about inequity--seemed to make organizational pressure less salient to the participants than expected. Further research on this part of the model is especially warranted. Although the global test of the model did not reject it, the model seems to be incomplete. The strength of the effect Of perceived resource conflict upon interaction quality was not adequately explained by the model, nor was the positive relationship between organizational pressure and IS-attitude. Although explanations for these relationships were proposed in the dissertation, the actual nature of the disturbances is not known and remains for future research. 6.3 other Notable Observations In the course of conducting the experiment and analyzing the data, several additional relationships were observed. These observations are categorized below as individual differences and equity adjustments. 117 6.3.1 Individual Differences Consideration of individual differences was specifically omitted from the model at the start of this project so that organizationally controlled factors could be emphasized. However, the fact that such differences exist and might be important cannot be totally ignored. Personal factors, such as accounting ability, the desirability of working in groups, or a preference for or against use of computers to accomplish tasks, probably had an impact on the reactions of the experimental participants and may be important in real-world situations. For example, some evidence supports the notion that a participant's attitude regarding computers has an impact upon the friction assessment. Item 1 of Part C of the Exit Form (Appendix D, Exhibit 5) stated, "In general, I really enjoy working on system development projects." A moderately strong and negative correlation, r=-0.248 (p=0.016), between responses to that item and the measure of friction was observed. Whether this attitude arises out of distaste for working with computers, working in groups, or some other factor is unknown. In addition, of the fifteen IS-attitude items (Exit Form, Part B), only three items were significantly correlated (p<.05) with the friction amount. Two of these items belong to the Ease of Use factor. These findings suggest that pre-existing attitudes about the desirability of the participant's own use of computers might have 118 influenced the estimate of friction and also might have been a significant determinant of the attitude toward the computer system being developed. Future research regarding the impact of pre-existing attitudes would be desirable. Observations made during the course of the experiment suggested that high levels of resource conflict were greeted in more than one fashion. Participants placed in the high RC, low OP treatment (Case C) almost always seemed to work more diligently on the accounting lab project than participants in any other treatment group. However, not all of these participants indicated that they perceived a resource conflict. These behaviors could be tied to personal opinions about one's own ability. That is, those participants who thought they were good at accounting tasks did not perceive a resource conflict; they simply worked harder at the accounting task to make time for the systems lab task. Alternatively, those participants who felt inadequate to the accounting task were more likely to perceive a resource conflict. An evaluation of grade point averages, which were not collected, might have provided more insight into the ability difference and its impact upon resource conflict. Inclusion of a measure of personal ability in the base task in future research designs might be advisable. 119 6.3.2 Equity Adjustments When faced with an inequitable situation, one recourse available to the participant was to adjust--objectively or subjectively--one of the factors in the equity equation. Casual observation of participants suggested that such alterations occurred. For example, those participants who reported high levels of both PRC and POP (Case D) seemed to work less diligently that those participants who perceived only one of the variables to be at high levels. The Case D participants could have been adjusting their personal cost downward by decreasing their accounting lab effort in order to (a) make cost more nearly equal to their own benefit (perceived to be low) or (b) to make their personal equity ratios higher and more nearly equal to the comparable others' ratio. Most equity adjustments are surmised to be primarily subjective to the individual and probably unconscious. While it is very hard to capture such attitudinal adjustments at the time of their occurrence, some evidence suggested that they did occur. Table 28 summarizes this evidence. Out of the 96 participants, 76 initially said that they were unhappy or indifferent about systems interaction. Most of these participants changed their minds by the end of the session. Asked if they would be happy or unhappy about future participation in a similar systems development project under similar conditions, 57 of the 76 originally unhappy participants said they would be happy to 120 do so in the future. (Only three of the originally happy participants changed their minds.) However, judging whether this reversal came about because the Systems Lab project was less detrimental to the participants than originally expected or because of subjective equity revisions would require further investigation. TAJHHB 28 ATTITUDES OF PARTICIPANTS ABOUT PROSPECTIVE SYSTEMS DEVELOPMENT ACTIVITY Pre-Experimental Attitude Post- Ex§::;::::al Happy Ifigggggggzi Total Happy 16 57 73 Izggg¥2;;:; 3 19 22 No Answer 1 O 1 Total 20 76 96 6.4 Implications The results of this project have implications for both research and practice in the IS field. 6.4.1 Research Implications One of the deficiencies attributed to IS research in the past is the weakness of its theoretical base (Sethi and King, 1991; Cushing, 1990; Wong-on-Wing, 1988). While this model of friction in the IS-development environment cannot be applied to all IS issues, it has the potential to 121 aggregate a substantial amount of IS research within one framework. Furthermore, the equity theory underlying the model is capable of extension to the IS-usage environment. A model of friction's impact upon systems in use will be substantially more complicated than the IS-development model because of the likely presence of recursive variables, such as IS-attitude. 6.4.2 Practice Implications The results of this project have importance also in the practitioner's world. The results aid in understanding how friction occurs, how it can be controlled or treated, and what can be done if friction is too high for a given project. Friction is likely to affect the organization's realization of the benefits of computerization. If friction is too high, the firm may be undercomputerized. Alternatively, if friction is too low, the organization may have too much computerized activity occurring. Few organizations seem to have evaluated the consequences of allowing friction to be either too high or too low. Doing so would be a first step in the determination of how much an organization should spend to adjust the level of friction. Managers can accomplish such adjustments in many ways, but particularly through the informed construction of performance contracts and the allocation of resources and costs. 122 6.5 Research Extensions The above discussion alluded to many avenues for future research, such as the following: Were mediating variables omitted from the model? If so, what are they? Should the failure of organizational pressure to negatively impact IS-attitude be attributed to experimental design, or is the theoretical development of the model incorrect in this regard? How might the measures of perceived resource conflict, perceived organizational pressure, and friction be improved? How do the model variables react to individual participant characteristics? Can practical estimates of the path parameters be developed? In addition to exploring questions raised by the experimental results, future research could examine the roles of incentives and countermeasures. These variables were not included in the experimental design. However, both incentives and countermeasures are important to the conceptual model and possibly encompass preferred courses of action toward avoiding or mitigating friction in real-world situations. Questions such as the following are examples of potential research issues dealing with these variables: Can bonuses paid to one participant but not the other act as incentives without increasing friction perceived by the other party? Does it matter who provides the bonus? Is a countermeasure equally effective at improving user attitude toward the system than is an incentive of the same dollar value? 123 Along another line of inquiry, extensions of the equity theory framework could be used to investigate the effect upon friction and IS success of topics such as (a) top management's choice of participation or involvement with respect to IS activity, (b) developers' willingness to substitute their resources for those of the users under conditions requiring more or less specific knowledge of the user's task, and (0) users' differential friction amounts (likely based on risk assessments) when system costs and benefits are quantifiable versus when they are not. Finally, research comparing accounting information systems to other information systems would also be appropriate, especially with respect to organizational pressure. Accounting information systems have generally been among the first systems implemented in firms and are sometimes subjected to change due to the information needs of non-accountants. This situation seems to occur more often for accounting users than for other categories of IS users . LIST OF REFERENCES LIST OF REFERENCES Adams, J. S., "Inequity in Social Exchange," Advances in Experimental Social Psychology (1965) pp. 267-299. Adams, J. S., and S. Freedman, "Equity Theory Revisited: Comments and Annotated Bibliography," Advances in Experimental Social Psychology (1976) pp. 43-90. Antle R., and J. S. Demski, "The Controllability Principle in Responsibility Accounting," The Accounting Review (October 1988) pp. 700-718. Baker, G. P., M. C. Jensen, and K. J. Murphy, "Compensation and Incentives: Practice vs. Theory," The Journal of Finance (July 1988) pp. 593-616. Bailey, J. E., and S. Pearson, "Development of a Tool for Measuring and Analyzing Computer User Satisfaction," Management Science (May 1983) pp. 530-545. Baiman, S., and B. L. Lewis, "An Experiment Testing the Behavioral Equivalence of Strategically Equivalent Employment Contracts," Journal of Accounting Research (Spring 1989) pp. 1-20. Baroudi, J., M. Olson, and B. Ives, "An Empirical Study of the Impact of User Involvement on System Usage and Information Satisfaction," Communications of the ACM (March 1986) pp. 232-238. Christensen, D. M., "The User Interface: A Change in Communication," Journal of Informapion Systems Management (Spring 1991) pp. 71-75. Cohen, J., "A Coefficient of agreement for Nominal Scales," Educational and Psychological Measurement (1960), pp. 37-46. Cronan, T. P., and T. L. Means, "System Development: An Empirical Study of User Communication," Data Base (Spring 1984) pp. 25-33. Cushing, B. E., "Frameworks, Paradigms, and Scientific Research in Management Information Systems," The Joprnal of Information Systems (Spring 1990) pp. 38- 59. Doll, W. J., and G. Torkzadeh, "A Congruence Construct of User Involvement," Decision Sciences (1991) pp. 443- 453. 124 125 Dos Santos, B., "Information Systems: Similarities and Differences Across Organizations," Omega (1989) pp. 9-20. Dye, R. A., "Relative Performance Evaluation and Project Selection," Journal of Accounting Research (Spring 1992) pp. 27-52. The Encyclopedia Americana, International Edition, Volume 12, Grolier Incorporated, Danbury, CT (1991) pp. 90- 91. Galletta, D. P., and A. L. Lederer, "Some Cautions on the Measurement of User Information Satisfaction," Decision Sciences (1989) pp. 419-437. Gregson, T., "Communication Satisfaction: A Path Analytic Study of Accountants Affiliated with CPA Firms," Behavioral Research in Accounting (1990) pp. 32-49. Hawk, S. R., and N. S. Raju, "Test-Retest Reliability of User Information Satisfaction: A Comment on Galletta and Lederer's Paper," Decision Sciences (November/ December 1991) pp. 1165-1170. Henry, B., "Measuring IS for Business Value," Datamation (April 1, 1990) pp. 89-91. Ives, B., and M. H. Olson, "User Involvement and MIS Success," Management Science (May 1984) pp. 586-603. Jarvenpaa, S. L., and B. Ives, "Executive Involvement and Participation in the Management of Information Technology," MIS Quarterly (June 1991) pp. 204-227. Jensen, M. C., and W. H. Meckling, "Specific and General Knowledge, and Organizational Structure," in Contract Economics, L. Werin and H. Wijkander, eds., Blackwell Publishers, Oxford (1992), pp. 251-274. Joshi, K., "A Model of Users' Perspective on Change: The Case of Information Systems Technology Implementation," MIS Quarterly (June 1991) pp. 228- 242. Kaiser, K., and A. Srinivasan, "User-Analyst Differences: An Empirical Investigation of Attitudes Related to Systems Development," Academy of Management Journal (September 1982) pp. 630-646. Kerlinger, F. N., Foundations of Behavioral Research, Holt, Rinehart, and Winston, Inc., New York (1986). 126 Kieso, D. E., A. A. Arens, and D. D. Ward, Systems Understanding Aid for Financial Reporting, 3rd ed., Armond Dalton Publishers, Inc., Williamston, Michigan (1989). Kim, K. K., "User Satisfaction: A Synthesis of Three Different Perspectives," The Journal of Information Systems (Fall 1989) pp. 1-12. Lucas, H., "Empirical Evidence for a Descriptive Model of Implementation," MIS Quarterly (June 1978) pp. 27- 42. Maish, A. M., "A User's Behavior toward His MIS," MIS Quarterly (March 1979) pp. 39-52. Moore, G. C., and I. Benbasat, "Development of an Instrument to Measure the Perceptions of Adopting an Information Technology Innovation," Information Systems Research (September 1991) pp. 192-222. Newman, M., and R. Sabherwal, "Information System Development: Four Process Scenarios with Case Studies," The Journal of Information Systems (Spring 1991) pp. 84-101. O'Reilly, C., "Variations in Decision Makers' Use of Information Sources: The Impact of Quality and Accessibility of Information," Academy of Management Journal (December 1982) pp. 756-771. Sen, T., and J. A. Yardley, "Are Chargeback Systems Effective? An Information Processing Study," The Journal of Information Systems (Spring 1989) pp. 92- 103. Sethi, V., and W. R. King, "Construct Measurement in Information Systems Research: An Illustration in Strategic Systems," Decision Sciences (1991) pp. 455-472. Srinivasan, A., "Alternative Measures of System Effectiveness: Associations and Implications," MIS Quarterly (September 1985) pp. 243-253. Walster, E., G. W. Walster, and E. Berscheid, Eguity: Theory and Research, Allyn and Bacon, Inc., Boston (1978). Wong-On-Wing, B., "User Involvement in Systems Development: An Attributional Approach," The Journal of Information Systems (Spring 1988) pp. 3-14. APPENDIX A "The Materials Workshop" Releases: Recruiting and Debriefing Materials 127 APPENDIX A EXHIBIT 1 - Announcement ANNOUNCEMENT--VOLUNTEERS NEEDED The Materials Workshop (TMW), based in Ohio, is looking for accounting students to help with its research and development activities at Michigan State University during the period of October 28 through December 2, 1992. The work will be done locally under the supervision of Patricia Essex of MSU's accounting department. What would you be expected to do? Each student hired spends one three-hour session working on a current TMW project. Participants at the MSU sessions will be involved with field testing a portion of a business simulation activity being developed by TMW. Participants also are expected to complete some questionnaires and may be interviewed. If you decide to participate, your work performance and all of your responses will be held strictly confidential. None of the reports issued by TMW or by its personnel will identify you individually in any way. Your accounting professors will not be told about your performance in this workshop, but we will provide them with a list of participants. What would you gain from this experience? The simulation being tested contains more than ten different business forms which you must complete and handle appropriateLy. You will use three different journals and two ledgers. Participation in one of these sessions offers you an opportunity to learn a great deal about normal business forms and accounting practices under the direct supervision of an experienced accounting teacher. If you complete ‘the ‘three-hour session satisfactorily, you. will be compensated for your efforts. The average payment is $15 a session. (Almost all participants earn the average amount.) Light refreshments will be provided at the end of the session while you fill out some information about the TMW product(s) on which you worked. Who is eligible to participate? To be eligible, you should have completed your accounting principles courses (Ace 201 and Acc 202) satisfactorily. You do not have to be an accounting major. How do you sign up? If you are interested in participating, please pick up an application from Pat Essex or one of the secretaries in the Accounting Department (332 Eppley), fill it out, and return it to them. You will be contacted to finalize the location, date, and time. 128 EXHIBIT 2 - Application THE MATERIALS WORKSHOP APPLICATION -- T*M *W MICHIGAN STATE UNIVERSITY SESSIONS FOR BETTER LEARNING PART A: PERSONAL INFORMATION Name: Local Phone Number: Birthdate: Gender: Male Female PART B: WORK EXPERIENCE Please list your previous work experience below. (If more than three positions, please list the three most significant positions.) Employer Position Start End DateAvera e Hours er Week PART C: EDUCATIONAL BACKGROUND Please check off the accounting courses you have completed. Place a "C" on the line of any courses in which you are currently enrolled. The course numbers listed are the numbers used under the quarter system; the semester system numbers are listed in parentheses. Accounting 201 (201) Principles of Financial Accounting Accounting 202 (202) Principles of Managerial Accounting Accounting 230 (230) Survey of Accounting Concepts Accounting 251 (251) Honors Accounting Principles Accounting 300 (300) Intermediate Financial Accounting I Accounting 301 (301) Intermediate Financial Accounting II Accounting 434 (308) Governmental and Not-for-Profit Accounting 321 (321) Accounting Information Systems Accounting 303 (341) Cost and Managerial Accounting Accounting 410 (411) Auditing Accounting 401 (431) Federal Tax Accounting Other accounting courses (please list): Similarly, please check off the computer courses you have completed. Place a "C" on the line of any courses in which you are currently enrolled. CPS 100 (100) Using Computers CPS 115 (130) Introduction to Computing 129 Other computer-related courses (please list): Have you previously been awarded a college degree? Yes No If yes, what degree do you hold? PART D: AVAILABILITY All sessions are scheduled from 6:30 to 9:30 p.m., and sessions run Monday through Thursday, October 28 through December 10 (with some exceptions.) Space at each session is limited. Please circle the evenings which you generally have available. You will be contacted to schedule the specific date of your session. Mondays Tuesdays Wednesdays Thursdays 130 EXHIBIT 3 - New Employee Letter THE MATERIALS WORKSHOP 1048 North Main Street Suite 91 Bowling Green, OH 43402 T*M*W October 28, 1992 FOR BETTER LEARNING Dear New Employee: Welcome to The Materials Workshop! We are a research and development organization dealing with accounting educational materials. The materials we develop are field tested at colleges around the country. The Materials Workshop consists of two basic organizational components: an Accounting Lab, where most work deals with testing and research, and a Systems Lab, where most work is developmental in nature. Because of our research mission, we hire students for the Accounting Lab only on a very temporary basis--one session each. Currently we are satisfying several contracts. You will be working on our project, A4395. This project is the testing of a modified version of a popular business simulation. We must determine whether A4395 is complete and useable. You will receive your specific task assignment and materials from the Accounting Lab supervisor, Ms. Essex, who will be available to you for assistance if you should need it. Your task, in general terms, is to complete your copy of A4395. Please note any errors that you find, and feel free to make suggestions! Sometimes Accounting Lab employees are asked to lend assistance to ongoing projects of the Systems Lab. You may be one of these employees; your supervisor will let you know. Compensation for your services will occur after a review of your work. Your performance will be evaluated by the accounting lab supervisor, and you will be paid according to the contract that you receive with your work assignment. Before you leave the lab today, be sure to complete an Exit Form. This form allows us to more completely assess the projects, and it provides information about the distribution of your compensation. This form is so important that we will not pay anyone who does not complete it. Work completed in this workshop is used to provide data for research into productivity and personnel issues, as well as to supply information about the accounting materials being developed. Please note that participation in this workshop is voluntary and that all forms completed by you will remain confidential. No one but the Accounting Lab Supervisor will be able to connect your work to your name. Your accounting professors will not be notified of your performance in this workshop. Please sign the consent form at the bottom of this letter if you give your consent to use your completed materials for research purposes. The Accounting Lab Supervisor will collect it. 131 Thank you for joining us at TMW. We hope you have a productive and enjoyable session! Sincerely, Henry Jones Director of Accounting Projects I understand that participation in this workshop is voluntary and that my responses will be confidential. Any reports generated from this workshop will NOT identify me personally. I also understand that I may withdraw from the session at any time without penalty except that I will lose my right to be compensated. TSignature) (Date) 132 EXHIBIT 4 - Accounting Lab Assignment Form ACCOUNTING LAB ASSIGNMENT FORM Name Accounting Lab Project # A4395 Date Session Materials Packet # You have been assigned the following tasks. If you have problems, be sure to consult the Accounting Lab Supervisor. TASK LIST: Follow the instructions in the packet. Your goal is to complete all transactions through October 29. 133 EXHIBIT 5 - Compensation Agreement COMPENSATION AGREEMENT The Materials Workshop agrees to pay you the following compensation: You will receive $5 per hour for three hours provided that your performance is judged to be satisfactory. Approximately one hour of this time will be spent on administrative details; two hours will be used for your assigned tasks. In return for this compensation, you are expected to (1) complete your assigned tasks in the Accounting Lab, (2) share your knowledge with Systems Lab personnel, if asked to work on a Systems Lab project, and (3) complete several questionnaires. Your work performance will be evaluated by the Accounting Lab supervisor after the session is over. You will be penalized one-half of your pay for either of the following two items, which we equate with unsatisfactory performance: a. Extreme underachievement with respect to meeting the goals listed on the Accounting Lab Assignment Form. b. Failure to complete the Exit Form in a conscientious manner. Your pay will be available after December 2, 1992. Details about distribution of pay will be announced at the end of the session. Please sign below to acknowledge these terms; the Accounting Lab supervisor will collect this form. (Signature) (Date) 134 EXHIBIT 6 - Debriefing Letter MICHIGAN STATE UNIVERSITY GRADUATE SCHOOL OF BUSINESS ADMINISTRATION EAST LANSING - MICHIGAN - 48824-1121 DEPARTMENT OF ACCOUNTING - (517) 355-7486 CHAIRPERSON - (517) 355-3388 FAX (517) 336-1101 December 7, 1992 Dear Workshop Participant: Thank you for participating in The Materials Workshop sessions. Your donation of time and thoughtful work is appreciated. Your compensation money is included in this envelope. No one was penalized for not achieving the Accounting Lab goal. Your current accounting professors will receive a list of participants this week. At the end of the sessions, you were told that you had been misled with respect to certain items during the course of the evening. Foremost among these items is the fact that The Materials Workshop does not exist. The lab sessions supported part of my dissertation project; my dissertation committee chair is Dr. Severin Grabski. Financial support for the project was provided by Michigan State University. No sessions have been held on other campuses. This experiment was designed to test a model of friction during systems development. Friction is defined as resistance to systems project interaction. The research design required you to be busy at a primary task (testing the new practice set) and to be interrupted to work on a systems development project. I am interested in how several variables, discussed below, affect friction and how the level of friction affects (1) your evaluation of the development interaction and (2) your attitude about the system being developed. Two variables were investigated. They are called resource conflict and organizational pressure. Resource conflict has to do with the amount of time you were willing to spend on the development project, relative to the amount of time you were asked to spend. It was expected to vary depending upon whether you were to be paid with (or without) regard to your achieving the assigned Accounting Lab goal. Organizational pressure is a variable that deals with whether you get the same benefits, relative to your costs, as some comparable person. It was expected to vary by comparing some of you to students who had been paid additional money and others to students who had received no additional money for the development project. The practice set used in the Accounting Lab was developed by me especially for this experiment and borrows heavily from the work of Dr. Al Arens and Dr. Dewey Ward in their Systems Understanding Aid. Those of you who have had auditing recognized the similarity; those of you who will take auditing will find the resemblance later. In return for letting me base the task on their work, the practice set and the results of your testing are being given to Dr. Arens and Dr. Ward. 135 However, Dr. Arens and Dr. Ward did not take part in or request the development of the Classy Campers' practice set. The Systems Lab project, the automated homework system, was also conceived solely for this experiment. No plan exists to bring it to implementation (although your unexpectedly high assessment of its potential may generate such a plan in the future.) All of the systems developers except one are graduate students at MSU who were hired to help with the research. The remaining developer is my husband, Dr. Mark Asman, who is a professor of accounting and information systems at Bowling Green State University. Again, thank you for your help. If you have any questions about the project, please let me know. I will be happy to discuss it with you. Sincerely, Patricia Essex APPENDIX B Experimenter's Script 6:30 p.m. 136 APPENDIX B EXPERIMENTER'S SCRIPT Get settled. Handle miscellaneous details: Envelope--Participants should address it to themselves; it will be used to distribute payment. Ask participants to place their accounting professors names in stamp area. Letter and consent form--Discuss TMW's development, testing, and research objectives. Point out that data collected tonight might be used for research into productivity issues; ask them to sign the consent form. Compensation agreement--Discuss expectations and penalties. Assignment form--Point out the goal. Tell them that most participants take about 2 1/2 hours to reach this goal and that they will be stopped at 9 p.m. Start Accounting Lab work. Help participants as needed. Hand out Systems Lab assignments, and read the following announcement out loud. Remember from the letter that you read at the start of this session that TMW does both manual and compterized projects. The Systems Lab is working on the design of a new system and you are asked to provide them with information about accounting student life here at MSU. More details are found on this sheet that I have just distributed. With respect to the questions on the handout, I am instructed to make three comments: 1. The time spent on the Systems Lab project will mean that most of you will not get through the transactions of the 29th (your goal.) 2. You will get no learning benefit from working on the Systems Lab project. The Systems Lab employees are here to get your opinions, not to help you learn anything about systems. 3. TMW holds these sessions at other schools to test other cycles of this practice set. When considering your treatment compared to those students, you must assume that they receive the same benefits and costs--other than what is mentioned in the handout--as you do. That is, if you get extra credit from a prof for attending, you should assume that they did too. It is not uncommon for such an event to occur. 7:15 8:30 9:00 137 I cannot answer questions with respect to the Systems Lab project or to the other comments on the handout. Please read it, and respond carefully to the questions on it. When you are finished, set it aside and return to your work. I will pick up the handouts before 7:15. Participants complete questions on the Systems Lab Assignments. Collect them. Accounting Lab work resumes. Systems developers (confederates) start arriving. Last developer arrival. Snack. Collect Accounting Lab work. (Have participants circle where they left off.) Complete Exit Forms. Make the following closing comments: You all seem to have worked diligently and likely will not be penalized for underachievement. (However, I must check the materials to be sure.) In order to conduct some research into productivity matters tonight, some misleading items have been presented to you. Please do not discuss your experiences with others who have not yet attended a session; remember that accountants have a duty for confidentiality. If confidentiality is breached, you will waste tonight's experience for the researchers. A more detailed description of the actual research will be given to you with your pay. Announce details regarding distribution of compensation. APPENDIX C Systems Lab Instructions and Materials 13 8 APPENDIX c Systems Lab Instructions and Materials EXHIBIT 1 - Orientation Meeting Memo C ON P I D E N T I A L TO: Systems Lab Employees FROM: Pat Essex SUBJECT: Expectations DATE: October 20, 1992 The following items outline important aspects of the task you have agreed to undertake. I wish you luck in undertaking this project and I hope that you have fun assuming your role! (a) You are working in the Systems Lab of The Materials Workshop, based in Ohio. Your stated task is to sketch out the initial design for a new computerized product to be used by accounting students during completion of certain types of homework. In order to do this, you must occasionally meet with other students (participants) who are working in the Accounting Lab. These participants may or may not be happy to meet with you because they have other work that they are expected to perform. (b) Confidentiality is extremely important. You are required to keep your unstated role in this project a secret until after the participants have been debriefed (no earlier than December 3, 1992.) You are also required to treat all participant responses with confidentiality. That is, you may not at any time divulge information from this study in any way that identifies individual participants. (c) Follow meeting outlines closely but maintain realism. Every session must be handled as similarly to others as possible. Do your best to stick to the timetable given to you. (d) Make notes on every interaction. (Forms are provided for this purpose.) Write notes immediately upon returning to the Systems Lab after communicating with the Accounting Lab participant. (e) Since you will be interacting with two participants, you must keep a close watch on your schedules. A researcher is in the Systems Lab to help everyone adhere to the master schedule. (f) Whenever you arrive to talk to a participant, you must announce yourself to the Accounting Lab supervisor, who will tell the participant of your arrival. (9) Your attitude and actions can easily affect the attitude and actions of the participants with whom you work. Aim to be mildly positive about the project, to be neither dominant nor submissive in your interactions, and to keep your discussions on the business at hand. 139 EXHIBIT 2 - Systems Lab Materials DESIGN MEETING # 1 Request permission to tape the interviews. (Explain that sometimes when ideas get flying around quickly, it's hard to get good notes.) Show the participant the Systems Proposal, and discuss it with him or her. Ask the following questions: a. What do you think of the proposed system? b. Would you have used such a product when you were in intermediate accounting? c. Do you think the system would be helpful in classes other than intermediate accounting? d. The product needs a name (eg., Homework Helper, ShortForm.) Any ideas? 140 SYSTEMS PROPOSAL GENERAL PROJECT DESCRIPTION: The proposed system is an automated accounting tool to be used by students for their accounting homework. The system would be flexible enough to work with homework from a variety of accounting classes. Each use of the base system will require the user to select a chart of accounts (or use the default), enter beginning balances, and make transaction entries (after analyzing the transaction description provided by the homework problem). Upon the user's request, the system will provide a selection of automatic reports. Additions to the base system are planned; these additions would provide analytic capabilities (such as goal-seeking or what-if) for use by the student. Data files for the problems can be saved to disk, allowing the system to be restarted for a new problem even before being done with a previous problem. Our target users are those who have mastered transaction processing and preparation of accounting reports, that is, students who are past the accounting principles courses. The proposed system is expected to be a useful tool in much the same way as calculators are useful to students who have learned the basic arithmetic process. SYSTEM FEATURES: 0 Offers a variety of headers or title pages to identify the student, the class, and the homework assignment. 0 Uses a standard chart of accounts that can be quickly customized for each problem. 0 Accepts transactions, stores information, and produces standard financial and managerial accounting reports. 0 Restarts easily for a new assignment. Allows data files to be saved on a floppy disk. 0 Provides "what if" and other analyses in its peripheral modules. B Is affordable. (Some textbook publishers are interested in buying the copyright so that they can package the system with textbooks. However, distribution options are still being explored.) 0 Not a full capacity accounting system. No more than a limited number of transactions can be recorded at any one time. Only programmed reports are available. No more than two time periods can be handled in any one problem. 141 DESIGN MEETING # 2 Show and discuss with the participant the Structure Chart, Context Diagram, and Diagram 0 for the proposed system. Ask the following questions: a. Do you think the system should interact with entities (people, departments, etc.) other than the student? (For instance, the professor?) How many homework problems is a student likely to work on at one session? Do you think that the system should allow the user to ask queries about the stored data, in addition to its reporting functions? What kinds of questions might be asked with a query function? Should this product be able to export data into other software (e.g., spreadsheets or word processing)? 142 «00.5 35.3 Nd owning of .E to continue. Be sure to select at least one asset and at least one equity account. Note: You may add more accounts to the selected list later. If you want to edit an account title, click on the current account title, edit the title, and press when you are done. Note: Any title that is changed must keep the same account classification. For example, you could change "Accounts Receivable" to "Accounts Receivable- Over 90 days" since both titles are current assets. Changing "Accounts Receivable" to "Long-term Notes Receivable" should not be done because the first account is a current asset but the new one is a long-term asset and will be handled differently by the system. A changed title will be in effect for the current problem only. U ASSETS 0 Current Assets Cash Marketable securities Accounts receivable Notes receivable Merchandise inventory Purchases Raw materials inventory Work in process inventory Finished goods inventory Supplies inventory Prepaid expenses Investments Other current assets DUDUUDDUDDDDD 0 Long-term Investments D Marketable securities 0 Other investments B Property 0 Land Buildings Leasehold improvements Furniture and fixtures Equipment Other property Accumulated depreciation UDUDUU U Intangible Assets 0 Goodwill Patents Organization costs Other intangible assets Accumulated amortization DUDE] D Other assets 149 U LIABILITIES D Current Liabilities Accounts payable Wages payable Customers deposits and advances Employees' payroll deductions Current maturities of long-term liabilities Dividend payable Miscellaneous accrued liabilities Other current liabilities CID DDUDDD Long-term liabilities 0 Mortgage payable 0 Bonds payable D Discount/Premium on bonds 0 Pension obligations D Other long-term liabilities Deferred taxes Other liabilities U OWNERS' EQUITY D Paid-in capital Preferred stock Common stock Paid-in capital in excess of par value Other paid-in capital DUDE! Retained earnings 0 Appropriated retained earnings 0 Unappropriated retained earnings 0 Dividends declared D Other retained earnings Treasury stock Other equity items 0 REVENUES DECIDE] Sales revenue Service revenue Interest revenue Investment revenue Other revenue 0 EXPENSES DDDDDUDDDD Cost of goods sold Cost of goods manufactured Salaries and wages Payroll expenses Advertising Rent Property taxes Depreciation and amortization Interest Other expenses 150 OTHER INCOME STATEMENT ITEMS Ordinary gains/losses Unusual items Extraordinary items Gain/loss from discontinued operations 0 Gain/loss from operation of discontinued division 0 Gain/loss from disposal of discontinued division Cumulative effect on prior years of change in accounting principle DUDE! El TAXES 0 Federal income taxes D State income taxes 0 Local income taxes 151 PROPOSED LIST OF REPORTS Statement of Financial Position Statement of Changes in Financial Position Statement of Retained Earnings Aged Accounts Receivable Amortization Schedule 10-Column Worksheet Contribution Margin Income Statement Statement of Earnings Statement of Stockholders' Equity Statement of Cost of Goods Manufactured Bank Reconciliation Trial Balance Budget Performance Report Variance Analysis Report WW 152 EXHIBIT 3 - Systems Interaction History SYSTEMS INTERACTION HISTORY Systems Lab Project # Systems Developer Meeting Date AL Employee MEETING H 1 2 3 I Starting time of meeting ] Ending time of meeting H Conference Room “ FACTOR I: Dominant-Submissive FACTOR II: Positive-Negative FACTOR III: Serious-Expressive FACTOR IV: Conforming-Nonconforming Comments (Please identify meeting): 03-50! M 153 FACTOR SCORING GUIDELINES FACTOR A: Dominant-Submissive Very Dominant. More dominant than necessary, pushing issues, not allowing others to speak, shouting. Dominant. Continually initiating conversation, blocking path of previous conversation. Slightly Dominant. Any slight hint of dominance, approaching you with suggestions, occasionally initiating dialogue. Neutral. Slightly Submissive. The slightest bit submissive, hesitating, avoiding looking you. Submissive. Backing off, obviously avoiding you, speaking only when spoken to. Very Submissive. Cringing, running away. FACTOR B: Positive-Negative U'l l-‘wa Very Positive. Hugging, kissing, "gushing", other signs of extreme affection. Positive. Agreeing, smiling, encouraging, overt signs of friendliness. Slightly Positive. Showing the slightest signs of friendliness, smiling, being pleasant. Neutral. Slightly Negative. Not smiling, gloomy. Negative. Hostile, challenging, disagreeing. Very Negative. Nasty, angry. FACTOR C: Serious-Expressive .7 (Ab-#01 N Very Serious. Soberly involved in the development task with indications of high inertia, that is, it would be difficult to move the participant to a lighter vein. Indications of anxiety about turning from development task and returning to normal duties. Serious. Giving information or opinions that indicate serious involvement in the development task. Slightly Serious. Routine agreement or other indications that the participant is paying attention to the systems development work. Neutral. Slightly Expressive. Smiling or other indications that the participant finds the situation amusing and is not very involved. Expressive. Joking and laughing or other forms of relief from the tension of the serious nature of the task. Very Expressive. Giving support to you regardless of your own serious task performance. Obvious signs of flight from the task that make it difficult for you to do serious work on the development task. 154 FACTOR D: Conforming-Nonconforming 7 (Al-5U! to Very Conforming. Clear statements or action indicating that all Accounting Lab participants should act in line with TMW's expectations for systems interaction. Conforming. Seeking to be guided by TMW's expectations by asking for information or suggestions. Revealing a constriction of fantasy life and social patterns in line with TMW's expectations for systems interaction. Slightly Conforming. Acting in an accepted way for TMW, especially in response to requests for conformity to expectations. Neutral. Slightly Nonconforming. Shows tension or slight resistance to systems activity. Nonconforming. Acting in ways that are clearly different from TMW's expectations about systems interaction although within accepted limits for systems interaction as a whole. Very Nonconforming. Withdrawing from interaction, describing fantasies that reveal individual goals rather than group goals. APPENDIX D Data Collection Instruments 155 APPENDIX D EXHIBIT 1 - Systems Lab Assignment Form--Case A SYSTEMS LAB ASSIGNMENT Name TMW would like you to assist in the design of a computerized product. The proposed system is an automated accounting tool to be used by students for their accounting homework. The system would be flexible enough to work with homework from a variety of accounting classes. The expected users of this system are intermediate and advanced accounting students. However, this system is unlikely to be produced in time for your use. A Systems Lab employee will be arriving soon to discuss the project with you and ask your advice on a number of issues. These discussions should be easy for you; they do not require any knowledge of computer systems. Remember that your contract indicated that you might be expected to do such work. You are being asked to split your remaining time as follows: Accounting Lab project (the practice set): 90 minutes Systems Lab project (the computerized system): 30 minutes Participation in the Systems Lab project will be considered in your performance evaluation. If you work conscientiously on both the Accounting Lab project (the practice set) and Systems Lab project (the computerized homework system), your pay will NOT be penalized for underachievement. Assume that the Accounting Lab project and the Systems Lab project are equally attractive to you in terms of effort and interest. Given Egg effect on our com ensation described above, how many minutes would you voluntarily spend on the Systems Lab project? I l l l I if I 0 S 10 15 20 25 30 Minutes Minutes Recently, students in sessions of The Materials Workshop at the Ohio State University participated in the design of a similar computerized system. Their sessions were conducted in a SIMILAR manner, and they were compensated at the SAME level as you will be. Consider your expected benefits and expected costs from working on the Systems Lab project. Compare these benefits and costs to those of the Ohio State students who participated on a similar Systems Lab project. How does that comparison come out? a. You will be better off than the Ohio State students. b. You will be worse off than the Ohio State students. c. About equal. (OVER) 156 Are you happy about being required to spend 30 minutes on the Systems Lab project? a. No (or Indifferent) b. Yes IF YOU ANSWERED "no (or indifferent)", COMPLETE THE FIRST (UNSHADED) BOX BELOW. IF YOU ANSWERED "yes", COMPLETE THE SECOND (SHADED) BOX BELOW. There must be some amount of money that could be paid to you to make you happy to spend 30 minutes on this Systems Lab project. What is the LEAST amount of money (more than $0) that TMW would have to offer you to make you happy to spend 30 minutes on this Systems Lab project? Make your answer a lump—sum amount: Please explain how you determined this amount. YOU ARE NOW DONE WITH THIS QUESTIONNAIRE You might wish to participate more in this Systems Lab project. For example you might be willing to give up part of your pay from TMW or be willing to pay with your own money for the opportunity to spend more than 30 minutes with the Systems Lab development team. Would you be willing to pay to increase your participation level? (Circle your answer.) Yes No IF NO, YOU ARE NOW DONE WITH THIS QUESTIONNAIRE What is the MOST you would be willing to pay for an increased participation opportunity? Make your answer a lump-sum amount: Please explain how you determined this amount. YOU ARE NOW DONE WITH THIS QUESTIONNAIRE 157 EXHIBIT 2 - Systems Lab Assignment Form--Case B SYSTEMS LAB ASSIGNMENT Name TMW would like you to assist in the design of a computerized product. The proposed system is an automated accounting tool to be used by students for their accounting homework. The system would be flexible enough to work with homework from a variety of accounting classes. The expected users of this system are intermediate and advanced accounting students. However, this system is unlikely to be produced in time for your use. A Systems Lab employee will be arriving soon to discuss the project with you and ask your advice on a number of issues. These discussions should be easy for you; they do not require any knowledge of computer systems. Remember that your contract indicated that you might be expected to do such work. You are being asked to split your remaining time as follows: Accounting Lab project (the practice set): 90 minutes Systems Lab project (the computerized system): 30 minutes Participation in the Systems Lab project will be considered in your performance evaluation. If you work conscientiously on both the Accounting Lab project (the practice set) and Systems Lab project (the computerized homework system), your pay will NOT be penalized for underachievement. Assume that the Accounting Lab project and the Systems Lab project are equally attractive to you in terms of effort and interest. Given Egg effect on your compensation described above, how many minutes would you voluntarily spend on the Systems Lab project? I I I I I I l 0 5 10 15 2O 25 30 Minutes Minutes Recently, students in sessions of The Materials Workshop at the Ohio State University participated in the design of a computerized system. Their sessions were conducted in a SIMILAR manner. However, the OSU students were paid $7.50 in addition to any money they received for doing their Accounting Lab work. (That is, satisfactory performance on both projects resulted in a total payment of $22.50; unsatisfactory goal achievement in the Accounting Lab resulted in a total payment of $15.00.) Consider your expected benefits and expected costs from working on the Systems Lab project. Compare these benefits and costs to those of the Ohio State students who participated on a similar Systems Lab project. How does that comparison come out? a. You will be better off than the Ohio State students. b. You will be worse off than the Ohio State students. c. About equal. KWER) 158 Are you happy about being required to spend 30 minutes on the Systems Lab project? a. No (or Indifferent) b. Yes IF YOU ANSWERED "no (or indifferent)", COMPLETE THE FIRST (UNSHADED) BOX BELOW. IF YOU ANSWERED "yes", COMPLETE THE SECOND (SHADED) BOX BELOW. There must be some amount of money that could be paid to you to make you happy to spend 30 minutes on this Systems Lab project. What is the LEAST amount of money (more than $0) that TMW would have to offer you to make you happy to spend 30 minutes on this Systems Lab project? Make your answer a lump-sum amount: $ Please explain how you determined this amount. YOU ARE NOW DONE WITH THIS QUESTIONNAIRE 159 EXHIBIT 3 - Systems Lab Assignment Form--Case c SYSTEMS LAB ASSIGNMENT Name TMW would like you to assist in the design of a computerized product. The proposed system is an automated accounting tool to be used by students for their accounting homework. The system would be flexible enough to work with homework from a variety of accounting classes. The expected users of this system are intermediate and advanced accounting students. However, this system is unlikely to be produced in time for your use. A Systems Lab employee will be arriving soon to discuss the project with you and ask your advice on a number of issues. These discussions should be easy for you; they do not require any knowledge of computer systems. Remember that your contract indicated that you might be expected to do such work. You are being asked to split your remaining time as follows: Accounting Lab project (the practice set): 90 minutes Systems Lab project (the computerized system): 30 minutes Only your Accounting Lab work can be taken into account when your performance is evaluated. Participation in the Systems Lab project may hurt your chances of achieving the goals given to you. Remember that the penalty for underachievement is $7.50. Assume that the Accounting Lab project and the Systems Lab project are equally attractive to you in terms of effort and interest. Given Egg effect on our com ensation described above, how many minutes would you voluntarily spend on the Systems Lab project? I I I I I I I O 5 10 15 20 25 30 Minutes Minutes Recently, students in sessions of The Materials Workshop at the Ohio State University participated in the design of a similar computerized system. Their sessions were conducted in a SIMILAR manner, and they were compensated at the SAME level as you will be. Consider your expected benefits and expected costs from working on the Systems Lab project. Compare these benefits and costs to those of the Ohio State students who participated on a similar Systems Lab project. How does that comparison come out? a. You will be better off than the Ohio State students. b. You will be worse off than the Ohio State students. c. About equal. fiWER) 160 Are you happy about being required to spend 30 minutes on the Systems Lab project? a. No (or Indifferent) b. Yes IF YOU ANSWERED "no (or indifferent)", COMPLETE THE FIRST (UNSHADED) BOX BELOW. IF YOU ANSWERED "yes", COMPLETE THE SECOND (SHADED) BOX BELOW. There must be some amount of money that could be paid to you to make you happy to spend 30 minutes on this Systems Lab project. What is the LEAST amount of money (more than $0) that TMW would have to offer you to make you happy to spend 30 minutes on this Systems Lab project? Make your answer a lump-sum amount: S Please explain how you determined this amount. YOU ARE NOW DONE WITH THIS QUESTIONNAIRE 161 EXHIBIT 4 - Systems Lab Assignment Form--case D SYSTEMS LAB ASSIGNMENT Name TMW would like you to assist in the design of a computerized product. The proposed system is an automated accounting tool to be used by students for their accounting homework. The system would be flexible enough to work with homework from a variety of accounting Classes. The expected users of this system are intermediate and advanced accounting students. However, this system is unlikely to be produced in time for your use. A Systems Lab employee will be arriving soon to discuss the project with you and ask your advice on a number of issues. These discussions should be easy for you; they do not require any knowledge of computer systems. Remember that your contract indicated that you might be expected to do such work. You are being asked to split your remaining time as follows: Accounting Lab project (the practice set): 90 minutes Systems Lab project (the computerized system): 30 minutes Only your Accounting Lab work can be taken into account when your performance is evaluated. Participation in the Systems Lab project may hurt your Chances of achieving the goals given to you. Remember that the penalty for underachievement is $7.50. Assume that the Accounting Lab project and the Systems Lab project are equally attractive to you in terms of effort and interest. Given phg effect on your compensation described above, how many minutes would you voluntarily spend on the Systems Lab project? I I I I I I l O 5 10 15 20 25 30 Minutes Minutes Recently, students in sessions of The Materials Workshop at the Ohio State University participated in the design of a computerized system. Their sessions were conducted in a SIMILAR manner. However, the OSU students were paid $7.50 in addition to any money they received for doing their Accounting Lab work. (That is, satisfactory performance on both projects resulted in a total payment of $22.50; unsatisfactory goal achievement in the Accounting Lab resulted in a total payment of $15.00.) Consider your expected benefits and expected costs from working on the Systems Lab project. Compare these benefits and costs to those of the Ohio State students who participated on a similar Systems Lab project. How does that comparison come out? a. You will be better off than the Ohio State students. b. You will be worse off than the Ohio State students. C. About equal. KNER) 162 Are you happy about being required to spend 30 minutes on the Systems Lab project? a. No (or Indifferent) b. Yes IF YOU ANSWERED "no (or indifferent)", COMPLETE THE FIRST (UNSHADED) BOX BELOW. IF YOU ANSWERED "yes", COMPLETE THE SECOND (SHADED) BOX BELOW. —'—-————————_——-—_———————_——————_—_fi There must be some amount of money that could be paid to you to make you happy to spend 30 minutes on this Systems Lab project. What is the LEAST amount of money (more than $0) that TMW would have to offer you to make you happy to spend 30 minutes on this Systems Lab project? Make your answer a lump-sum amount: 5 Please explain how you determined this amount. YOU ARE NOW DONE WITH THIS QUESTIONNAIRE more than 30 minutes with the Systems Lab development team. ..... VWould you be willing to pay to increase your participation level? (Circle your answer. ) Yes _ No 163 EXHIBIT E - Exit Form EXIT FORM (SYSTEMS PRODUCT VERSION) Name Date Your answers to the items on this form can help us improve both our products and the way in which the Systems Lab and the Accounting Lab employees work together. Please be careful and honest with your answers. Indicate your response by marking an X on the line below each item. SECTION A: PLEASE RESPOND TO THE FOLLOWING ITEMS ABOUT YOUR INTERACTION WITH THE SYSTEMS LAB EMPLOYEE. 1. The Systems Lab employee was a good listener. I I I I I I I 1 L i 1 1 1 1 Agree Disagree 2. I had very little authority with respect to decisions about the Systems Lab project. I I I I 1 1 1 1 1 Agree Disagree r- 3. I found it difficult to work with the Systems Lab staff on the Systems Lab project. I I I i 1 1 1 Agree Disagree b— F— b- 4. Working on the Systems Lab project was fun. I I I I L 1 1 1 i I I I l I Agree Disagree 5.. Communications from the Systems Lab employee were Clear (vocabulary easy to understand, little jargon, explained well, etc. . . .) I I I I I I I 1 1 1 1 1 1 1 Agree Disagree 6. .As a typical student user, I should have had more power to direct the development of the Systems Lab product. I I I I I I 1 1 1 1 1 1 Agree Disagree P- 7. The needs of people like me are being considered by the Systems Lab. I I I Agree Disagree P- I.- p— #— 164 8. The Systems Lab employee should have let me participate in the Systems Lab project more than he or she did. I I, I I I I I Agree Disagree 9. The initiation of communications by the Systems Lab employee to me had a purpose. I I I I I I I Agree Disagree 11L The Systems Lab employee seems to be aware of problems that users might have with the system. I I I, I I I I Agree Disagree 11” The Systems Lab employee seemed to want me to work on the Systems Lab project more than I felt was necessary. I I I I I I I Agree Disagree 12. The Systems Lab employee was not very compatible with me. I I I, I I I I Agree Disagree 13. I felt that I was allowed to make changes in the design of the Systems Lab product. I I I I I I I l L l l I L I Agree Disagree 14w The Systems Lab employee anticipated my need for information about the Systems Lab project. I I I I I I I Agree Disagree 15. The Systems Lab employee's work habits seemed to fit well with my own. I I I I I I I Agree Disagree Overall, how would you rate the quality of the interactions that took place between you and the Systems Lab employee with whom you worked? I I 1 1 Excellent I I I I I 1 L 1 1 1 Average Poor ************************************************************************ 165 SECTION B: PLEASE RESPOND TO THESE ITEMS ABOUT THE SYSTEMS LAB PRODUCT BEING DEVELOPED AS IF YOU WILL BE USING THE PRODUCT AS A TYPICAL ACCOUNTING STUDENT. 1. Using the product will enable me to accomplish learning tasks more quickly. I I ' I I Agree Disagree I.- y- 2. Using the product will be compatible with many aspects of my accounting study. I I I I I I 1 1 1 1 I 1 Agree Disagree P— 3. People at my school who use this product will have more prestige than those who do not. I I I I Agree Disagree P- F— b- 4” My interaction with this product will be clear and understandable. I I I I I I I 1 1 1 1 1 1 1 Agree Disagree 5. Using the systems product will improve the quality of work that I do. I I I I I I I Agree Disagree 6. I think that using a product such as this fits well with the way that I like to work. I I I I I I J 1 1 L Agree Disagree P- h )- 7. People at my school who use products such as this have a high profile. I I I I I I L 1 1 L 1 1 Agree Disagree b- 8. I believe that it will be easy to get the systems product to do what I want it to do. I I I I I I I 1 1 1 1 1 1 1 Agree Disagree 9. Using the systems product will make it easier to learn accounting material and methods. I I I I J 1 I 1 Agree Disagree P- P— P— ILL Using the systems product will fit into my learning style. I I I I I I I 1 1 1 1 1 1 1 Agree Disagree 166 11» Using this systems product will be a status symbol at this school. I I I I I I 1 1 g 1 1 1 Agree Disagree I 1 12. Overall, I believe that the systems product will be easy to use» I I I I I I I I I l L l L Agree Disagree 13. ‘Using the systems product will increase my productivity. I I I, I I I I Agree Disagree l4. ILearning to utilize the systems product will be easy for me. I I I I I I I 1 1 1 1 1 1 1 Agree Disagree 15“ ‘Using the systems product will give me greater control over my learning activity. I I I I I 1 1 1 1 1 Agree Disagree b— b- What is your overall opinion of the eventual success of the Systems Lab project to which you were assigned? I I I I I 1 1 1 L L Very Successful Very Unsuccessful p.- p.— ************************************************************************ SECTION C: PLEASE ANSWER THESE ITEMS THE BEST THAT YOU CAN. 1. In general, I really enjoy working on system development projects. I I I I I I I Agree Disagree 2. Students in my major at other colleges would be more qualified than I was to help with this Systems Lab project. I I I I I I I 1 1 1 1 1 1 1 Agree Disagree 3. I had little relevant knowledge to contribute to the Systems Lab project. I I I I I I 1 1 1 1 1 1 Agree Disagree #- 4. Compared to the Accounting Lab task to which I was assigned, participation on the Systems Lab project was hard for me. I I I l I I I 1 1 1 1 1 1 L Agree Disagree 167 5. The Systems Lab project was a valuable learning experience for me. I I I I I I I 1 L l l l I I Agree Disagree 6. Considering my efforts, I got about the same benefits out of working on the Systems Lab project as the Ohio State University students probably received from doing a similar project. I I I I I 1 4 1 1 1 Agree Disagree I I 1 1 7. Students in my major at other schools probably are more interested than I am in system development projects. I I I I I Agree Disagree I'- I.- 8. The Ohio State University students received unfair compensation on their Systems Lab project, considering my own efforts and compensation tonight. I I I I I I 1 1 1 1 L 1 Agree Disagree 9. Considering the terms of my compensation agreement, I was happy to be able to cooperate on the Systems Lab project. I I I I I I I l l l l l l 1 Agree Disagree 10. ‘The cost to me of participation on the Systems Lab project was too high, considering the benefits I derived from it. I I I I 1 1 1 1 1 1 Agree Disagree b- WOuld you be interested in being involved with a SYSTEMS project such as this one again (under similar conditions)? a. Yes b. No (or Indifferent) IF YOU ANSWERED "no (or indifferent)", COMPLETE THE FIRST (UNSHADED) BOX ON THE NEXT PAGE. IF YOU ANSWERED "yes", COMPLETE THE SECOND (SHADED) BOX ON THE NEXT PAGE. 168 There must be some amount of money that could be paid to you to make you happy to participate in a similar SYSTEMS project. What is the LEAST amount of money (more than $0) that TMW would have to offer you to make you happy to spend 30 minutes on a similar systems proje ct? Make your answer a lump-sum amount: $ Please explain how you determined this amount. Would you be willing to pay to partiCipate? (Circle your answer. ) Yes . NO .. ., IF NO, YOU ARE NOW DONE WITH THIS BOX. I ill I S E I R AH R B I L