mflbhnrb-thl 1%; Ifl..{d...llr .K‘xb ‘ a1 ..3..... ..n .. E‘ 3 3... .... ... ......zgikxul 2.33.2.3 .53“... ..3..}. A. n...» v ..ILSR 1...“.mflmmus..nz : I... ha?!» a”. 5.?! 551 8.4.2.: ...-...!- 5.4982. lax!!!“ Huh ; ... ‘3“... z... z .. .5... i ......u...... ! lis- .. .x.» 1. lle-nivfltrali l. (345‘. it. ”NIL“... .... ...... .... . .. ... .3... . fififl?tf#fl. ...rwunkfl. ..J LIBRARY “Tm Michigan State 20 .3 g UI'IIVBI'SIIY This is to certify that the dissertation entitled AN EVALUATION OF THE IMPACT OF SUPPLY CHAIN INFORMATION MANAGEMENT SYSTEMS ON OPERATIONAL PERFORMANCE presented by Shawn Rocco Jones has been accepted towards fulfillment of the requirements for the Doctoral degree in Supply Chain Management wawfi MajoTProf sor’s Signature (103/443: 1 i 02 M 5/ Date MSU is an affirmative-action, equal-opportunity employer ..—---.-o-n-—.---»—-.-.-.-.-.—.---»-.- . PLACE IN RETURN BOX to remove this checkout from your record. TO AVOID FINES return on or before date due. MAY BE RECALLED with earlier due date if requested. DATE DUE DATE DUE DATE DUE 5/08 K‘IProlecc8Pres/ClRC/DateDue indd AN EVALUATION OF THE IMPACT OF SUPPLY CHAIN INFORMATION MANAGEMENT SYSTEMS ON OPERATIONAL PERFORMANCE By Shawn Rocco Jones A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Supply Chain Management 2008 ABSTRACT AN EVALUATION OF THE IMPACT OF SUPPLY CHAIN INFORMATION MANAGEMENT SYSTEMS ON OPERATIONAL PERFORMANCE By Shawn Rocco Jones Supply chain managers face increasing challenges to incorporate data from suppliers, customers, competitors, and internal processes in a format to aid decision making. As information management systems are developed to condense, contextualize, and calculate the critical decision variables for a firm, there is a significant risk of creating inaccuracies as the data is processed. This study investigates the impact of variations in the accuracy of decision information systems and completeness of data on managerial performance within a supply chain management system. The Supply Chain Decision Operations Environment simulation was adapted to provide a platform for this experiment. Afier randomly assigning participants to one of four categories of corrected / uncorrected information and condensed / uncondensed data, users executed several weeks of a simulated supply chain operation. The final performance measures include raw material inventory status, internal production efficiency, finished goods inventory status, market fulfillment rate, net profit, and supply chain contribution to profit. The accuracy of the information system had the most significant impact on the performance of the managers and is recommended as the most critical investment opportunity for information technology. Additional recommendations include key decision variables under varying degrees of information system accuracy. Dedicated to Beverly, Gwendolyn, and Evan. iii ACKNOWLEDGEMENTS This finished dissertation reflects the combined efforts of a number of individuals to whom I will forever be grateful. Dr. David Closs served a role far greater than simply the Chair of the Dissertation Committee. He opened the opportunity for a new phase of my life as the previous one closed. In addition, he invested countless hours in simulation planning, career advice, and personal development. For all of this and more, thank you. Dr. Ken Boyer, Dr. Martha Cooper, and Dr. Cheri Speier comprised the Dissertation Committee. I was truly privileged to have the opportunity to learn from these distinguished experts in each of their respective fields. Despite their overwhelmingly busy schedules, I always received the insight, direction, and advice that I needed to continue work. I sincerely appreciate all that you have done to help me achieve my goals. Dr. Fred Rodammer serves as Director of the Center for Leadership of the Digital Enterprise, which oversees the SCODE simulation as one of many successful projects. His willingness to include me in the SCODE project, flexibility to allow me to adapt the simulation to the experimental requirements, and trust to manage the project to fruition means more to me than he understands. Thank you. Cheryl Lundeen, Laurie Fitch, Sherry Danner, and Sheila Mead each took active roles in my progression. In three years, I never had a question that one of you couldn’t answer. You kept me on track, focused on the goal, and cheerful in the face of often overwhelming obstacles. Thank you each for your camaraderie and advice. iv Alice Kalush, Jim Murphy, and Ping Liang were essential to development and fielding of the SCODE simulation. Without your expertise, none of this would have been possible. Congratulations on a job well done and thank you for helping me achieve my goals. Lieutenant Colonel Michael Weeks first introduced me to the opportunities and challenges of academic research. I appreciate your guidance and advice before and during the program. Without the support of my family, none of this would have been possible. I thank my parents who set a strong example of hard work and dedication to goals. I hope you know how grateful I am for the tools, encouragement, and understanding you provided to help me through this program. My children, Gwendolyn and Evan, deserve a great deal of credit. They kept my life balanced, never complained when we took another trip to the office, and were always willing to reward my hard work with a trip to the MSU Dairy Store for ice cream. My wife and best friend, Beverly gets the final and largest share of the credit. Words cannot express my gratitude for everything you have done to make this dream come true and I am thankful every day that I have the opportunity to share my life with you. TABLE OF CONTENTS CHAPTER I: INTRODUCTION- 1 1 .0 INTRODUCTION .................................................................................................... 1 1.1 RESEARCH OBJECTIVE........................................ ................................................. 1 1.2 CONTRIBUTION OF RESEARCH ............................................................................. 3 I. 2. 1 Theoretical Significance ................................................................................. 4 1. 2. 2 Managerial Significance ................................................................................. 5 1.3 STRUCTURE OF THE DISSERTATION ...................................................................... 7 CHAPTER II: LITERATURE REVIEW - 8 2.0 INTRODUCTION .................................................................................................... 8 2.1 DEFINITIONS ........................................................................................................ 8 2.2 THE RESOURCE BASED VIEW OF THE FIRM ........................................................ 14 2.3 THE KNOWLEDGE BASED VIEW OF THE FIRM .................................................... 17 2.4 FACT-BASED KNOWLEDGE WITHIN FIRMS ......................................................... 19 2.5 INFORMATION COST, COMPLEXITY, AND PROCESSING ....................................... 22 2.6 DATA - IN FORMATION - KNOWLEDGE —- PERFORMANCE LINKAGES .................. 24 2.7 CONCEPTUAL MODEL AND RESEARCH HYPOTHESES ......................................... 26 CHAPTER III: RESEARCH METHODOLOGY- 34 3.0 INTRODUCTION ................................................................................................... 34 3.1 RESEARCH OBJECTIVE .............................. 34 3.2 SIMULATION MODELING ..................................................................................... 34 3.2.1 The Supply Chain Operations Decision Environment Model ....................... 36 3.2.2 Model Verification and Validation ............................................................... 40 3.3 RESEARCH DESIGN ............................................................................................ 45 3. 3. 1 Experimental Population .............................................................................. 45 3.3.2 Experimental Design ..................................................................................... 45 3.3.3 Experimental Variables ................................................................................ 48 3.3.4 Performance Variables ................................................................................. 5 7 3. 4 PERCEPTUAL MEASURES ................................................................................... 60 3.5 DATA ANALYSIS ............................................................................................... 63 CHAPTER IV: RESULTS - - -- 65 4.0 INTRODUCTION .................................................................................................. 65 4.1 EXPERIMENTAL POPULATION DESCRIPTIVE DATA ............................................. 65 4.2 STATISTICAL METHODS AND TESTS OF ASSUMPTIONS ...................................... 70 4.3 Two-WAY OVERALL RESULTS ........................................................................... 75 4.4 MAIN EFFECT RESULTS ...................................................................................... 88 vi 4.5 SURVEY AND NOMINAL GROUP TECHNIQUE RESULTS ....................................... 95 4. 5. 1 Survey Results ............................................................................................... 95 4.5.1 Nominal Group Results .............................................................................. 100 4.6 HYPOTHESIS TESTING ...................................................................................... 104 4. 6.1 Hypothesis 1 ................................................................................................ 104 4. 6.2 Hypothesis 2 ................................................... ‘ ............................................. 1 05 4. 6.3 Hypothesis 3 ................................................................................................ 107 4. 6. 4 Hypothesis 4 ................................................................................................ 108 4.7 DISCUSSION OF RESULTS .................................................................................. 109 CHAPTER V: CONTRIBUTIONS AND CONCLUSION 112 5.0 INTRODUCTION ................................................................................................ 1 12 5.1 RESEARCH CONTRIBUTIONS ............................................................................. 112 5.2 MANAGERIAL CONTRIBUTIONS ........................................................................ 1 13 5.4 LIMITATIONS AND OPPORTUNITIES FOR FUTURE RESEARCH ............................ 1 16 5.5 CONCLUSION .................................................................................................... 120 APPENDICES- - - 121 REFERENCES 146 vii CHAPTER I: INTRODUCTION 1.0 Introduction As supply chains continue to provide a challenging environment requiring agility and flexibility to take advantage of new opportunities, executives are increasingly reliant on management information systems to lead their companies. To keep pace with the increasingly complex supply chains, these information systems collect data from many sources to compile a hopefully meaningful set of decision metrics for managers. These systems frequently enable managers in one continent to oversee operations across the world. However, there are opportunities for the information to become corrupt, inaccurate, or completely wrong. Does the manager usually detect incorrect information prior to making decisions or is it only after developing a certain level of knowledge that the manager questions the accuracy of the system? Even if the inaccuracy is detected, does the manager know what other data are available to continue Operations? This research investigates these questions through studying manager’s decision making processes in a complex global simulation experiment. 1.1 Research Objective The objective of this study is to improve understanding of the decision making process in a supply chain environment and the balance between information accuracy and data compression. Specifically, this research explores the impact of varied data quantity and information quality in management information systems on a manager’s ability to make sound decisions in a time-constrained environment. In order to explore these questions, subjects participated in a structured experiment involving the Supply Chain Operations Decision Environment (SCODE) computer simulation. The participants were randomly assigned to one of four groups receiving different levels of feedback information and quantity of feedback data. The users are unaware of any difference between their feedback mechanism and any other players. After a series of rounds, or simulated weeks, the users complete a detailed survey about their experience and participate in a nominal group feedback session. Four research questions are proposed with hypotheses explained in detail in Chapter 11. Research Question 1 posits whether decision information systems mediate the impact of data saturation on a supply chain manager’s performance. Tools such as management dashboards are prevalent in industry to quickly highlight metrics or status levels that require attention by a manager. One question this research seeks to answer focuses on the manager’s ability to make micro-level decisions from the macro-level aggregate metrics presented by the dashboard and whether the dashboard provides the ability to process vast amounts of data into a quickly comprehendible Visual signal for management action. Research Question 2 inquires if the level of detail of the data available to a manager mediates the impact of inaccurate information on that manager’s decision- making ability. In the event that, whether purposely or accidently, the information management dashboard presents incorrect information, is it better for the manager to have access to vast amounts of data or compressed amounts of data? The study evaluates manager’s performance in each instance and proposes the right amount of data to be gathered and retained to minimize the impact of erroneous information systems. The third research question asks whether there is a pattern Of incorrect information provided by decision information systems before a manager seeks alternate decision variables. By studying a manager’s ability to detect errors in the information system and exposing the manager to various patterns of inaccurate information, insight is gained into the trust relationship between a manager and the computer system. The final research question inquires what decision variables managers identify as the most important and least important to monitor in a supply chain. Direct inquiry through a structured survey and a nominal group feedback period at the end of the experimental session contribute to capturing the user’s experience. Analysis of the group feedback provides Significant contributions to identifying the most critical data and information for management systems to capture accurately as well as offering a listing of the variables that may not need to be collected. This dissertation reports the findings of experimental sessions utilizing the SCODE computer Simulation within the design outlined in Chapter 3. These findings make a number of significant contributions for supply chain managers to increase the effectiveness and efficiency of their decision information systems as well as improving understanding of the processes by which supply chain operational decisions are made. 1.2 Contribution of Research This research provides important insight to both a practitioner and an academic audience. For the academic audience, this research refines understanding of the knowledge-based View (KBV) of the firm and expands previous research involving the bullwhip effect, information sharing among supply chain collaborative relationships, and information processing in a global supply chain. For the practitioners, this document contains information on the behavior of managers operating a simulated supply chain in a global environment. The insight gained assists practitioners in making both daily operational decisions as well as strategic investment and policy decisions. 1. 2. 1 Theoretical Significance In 1971, Andrews proposed that distinctive organizational competencies form the basis of competitive advantage for a firm. Wemerfelt later (1984) refined this proposition into the Resource Based View of the Firm. In both of these works, the authors argued that firms are comprised of bundles of capabilities. These capabilities could take a number of forms such as employees, equipment, methodologies, access to markets, or other traits. The firm’s position relative to their competition within these resources could make these a source of competitive advantage within the marketplace. In the 119905, a number of researchers (Huber, 1990; Grant, 1996; Zack, 1999) identified knowledge as the most Significant competitive advantage of the resources available to a firm. Davenport and Prusak (1998) propose processes for managers to gain knowledge from information available within the firm. In turn, this information is gained from data recorded throughout the firm. By progressing through the hierarchy, Davenport and Prusak argue that simple records of transactions can form the basis for evaluating new opportunities within the firm. However, this structure leaves two significant gaps explored by this research. First, if the processes to translate data into information are compromised, how is knowledge creation affected? Are managers expected to identify discrepancies in the information based on prior experience, or is new knowledge formed that incorporates the erroneous information? Second, if the processes are compromised, is it possible for the managers to create knowledge directly from data without the intermediate processing of information? The results proVide insight into the difficulty of identifying the knowledge that contributes to competitive advantage versus the knowledge gained through inaccurate information which could ultimately become a competitive disadvantage. In addition to building upon the work of Davenport and Prusak, this research focuses on the bounds of the Knowledge Based View of the Firm. In particular, the premise that more knowledge leads to greater absorptive capacity (Cohen and Leventhal, 1990) is evaluated under varying degrees of information accuracy and data compression. Similarly, a vast literature stream supports information Sharing as essential to reducing the phenomenon of order amplification throughout the supply chain, or the bullwhip effect (Lee, Padmanabhan, and Whang, 1997; F iala, 2005; Croson and Donohue, 2006; Disney and Towill, 2003). This research proposes sharing of specific information can be helpful, but sharing of too much information or complete sharing of data sources could be insufficient to mitigation of the bullwhip effect. Lastly, the research highlights differences in trust behavior in a human-human interaction and a human-computer interaction. 1. 2. 2 Managerial S ignificance Supply chain managers make daily decisions affecting the profitability, customer satisfaction, and supplier relationships of their firms. In order to make these challenging decisions, the managers must be aware of hundreds of items impacting the supply chain. These items include such diverse variables as raw material purchase prices, average transit time from the supplier to the manufacturing plant, available labor hours, distribution center finished goods status, and customer demand in a given market, to name only a few. In a global marketplace, managers cannot personally oversee every aspect of the supply chain. Instead, they rely on information systems to provide insight into the operational status at specific points in time. Difficulties can arise when these information systems are created and maintained by different agencies. The data may be recorded for another use within the specific location, but also absorbed by the information system. The data may also be modified without notifying the owner of the information system. For example, a supplier in Singapore may record the number of products available to sell to their customers. If a purchasing firm has a strong collaborative relationship with the Singapore supplier and adheres to recommended best practices in the industry, they likely share an information management system which accesses this availability level and provides it to a manager in Canada. After a period of time, the data system owner identifies missed sales Opportunities. The sales staff is turning away prospective buyers for lack of inventory immediately available in the warehouse. However, some buyers may request delivery in the subsequent week. In order to rectify the problem and increase sales, the Singapore supplier changes the format of the inventory status to reflect incoming shipments due within the week instead of those currently available in the warehouse. If the Canadian information system owner was not notified of the change, the manager could mistake the quantity immediately available for Shipment and make decisions adversely affecting the supply chain’s customers. This research highlights the potential for incorrect information presented by management systems despite the presence of accurate data within a supply chain. By understanding the key variables involved in decision making and the risk associated with management information systems, practitioners can better balance the substantial investment required to improve the data processing accuracy with the adverse effects of inaccurate information. 1.3 Structure of the Dissertation Chapter 1 provides an overview of the dissertation objectives, significance, and purpose. Chapter 2 presents a review of recent literature as it pertains to the study. The sections include definitions of essential terms used throughout this dissertation and explanation of the Resource Based View of the Firm and the Knowledge Based View of the firm, which form the theoretical basis for this work. Additional literature explores managerial data requirements, information processing, computer simulation in research, and details of the hypotheses supporting the research questions. Chapter 3 reviews the research design, methodology, and tools. Chapter 4 contains the data analysis, results, and research findings. Chapter 5 concludes the dissertation with implications of the research findings, contributions to practitioners and academics, limitations of the study, and opportunities for future research. CHAPTER I]: LITERATURE REVIEW 2.0 Introduction This chapter presents theoretical support for this research from the literature. Included in this discussion are the definitions, bounds of the study, previous research related to this work, and identification of specific gaps and opportunities explored in the dissertation. Specific research questions and hypotheses to address these gaps are presented as a precursor to the research design in Chapter III. 2.1 Definitions AS the basis for topic development and establishing a frame of reference, it is essential to present the definitions of key terms as they pertain to this study. In the field of information management, three terms are consistently presented to explain the process of transforming events into cognition. At the most basic level, data exists. Through processes, this data transforms into information. At the highest level, information is processed into knowledge. Various literature streams interpret the specific definitions differently and there is frequently a lack of clarity in the usage of the three terms. Therefore, further discussion is warranted to frame the definitions for this research. In order to provide this frame of reference, contrasting definitions are presented to illustrate the broad interpretation of these terms. Each paragraph discussing the usage of the terms then focuses on typical business literature usage of the terms and concludes with a clear statement of the definition used throughout this research. Data are broadly accepted as a representation of fact. Within specific contexts, data takes on slightly different characteristics. The Department of Defense specifies data are a “representation of facts, concepts, or instructions. . .suitable for communication, interpretation, or processing by humans or automatic means (DoD Dictionary, 2003). Marketing dictionaries identify data as “facts that become useful information when organized in a meaningful way or when entered into a computer (Imber and Toffler, 2007). In a more general form, the American Heritage dictionary presents the definition as “factual information” (2007), adding a level of confusion and lacking distinction between data and information. While various streams of literature adopt slightly different interpretations of the term, for the purposes of this research, data is defined as “a set of discrete, objective facts about events” (Davenport and Prusak, 1998). The explicit inclusion of the term “objective” in this definition is significant. Without additional input, data has no inherent meaning (Badii and Sharif, 2003). However, data is of vital importance to firms and researchers. The data collected through various processes are the raw materials critical to production of information or knowledge within an institution (Chow, Choy, and Lee, 2003). While some disparity exists in the definition of data, interpretation of the term “information” is considerably broader. The American Heritage dictionary again lacks specificity as it presents information as “knowledge derived from study, experience, or instruction” (2007) as the primary definition. The Department of Defense crosses the hierarchy in the other direction by including “facts, data, or instructions” to their definition of information, though their dictionary does expand the explanation to include “the meaning that a human assigns to data” (2003). Drucker emphasizes the human interaction when asserting that information is “data endowed with relevance and purpose” (1988). Drucker’s definition delineates the difference between data and information for this research. Davenport and Prusak propose five processes to add value to data in order to create information. This dissertation focuses on these fiVe processes in progressing through the hierarchy from data to information. The first is to contextualize the data in a way for the user to understand why the data exist. Second, data are categorized into components or similar units of analysis. Third, mathematical or statistical calculations are performed to add meaning and simplify the data into a comprehendible set of meaningful information. Fourth, errors are removed as the data are corrected. Lastly, data are condensed into a format easier for the user to comprehend or analyze (Davenport and Prusak, 1998). As an example to clarify these processes, the simple case of recording data obtained from a drive through window at a fast food restaurant is followed through the five steps. Data consisting of order input time and order output time are recorded for a month. A manager reviewing the data understands that these times are key performance measures of customer satisfaction, which is critical to the success of her franchise. Therefore the data has been given context to understand why it is important to the manager. These sets of data are differentiated from the order times taken within the restaurant, and so have been categorized by unit of analysis. The simple calculation of subtracting the completion time from the input time yields the total customer wait time. The manager can glean meaning from different wait times. If any of the wait times are negative, this would clearly be an error that would be corrected or omitted from review. Lastly, the manager would likely reduce the entire month’s customer wait times to an 10 average, standard deviation, maximum, and minimum. This step condenses the data into a usable metric to present to the workers of the restaurant. By taking these five steps, the manager was able to provide meaning and purpose to the data and, by the definition used in this study, created information. Figure 2.1: Processes to transform data to information to knowledge (Davenport and Prusak, 1998) I Information I I I'J.‘ ll ' 'n _ . ' "' ' -ll' “' ,_ . _. , . . ,r ‘ .‘d ‘ ' "'. ”in... wt. ... . l Knowledge I The highest level of the hierarchy is knowledge. Just as defining information became more difficult than defining data, establishing a consensus on what it means to know something is elusive. The dictionary attempt is circular and superficial by focusing on such definitions as “the state of knowing” and “specific information about something” (American Heritage Dictionary, 2007). The Department of Defense does not offer an official definition of knowledge. Even as one of the pioneers of the Knowledge-Based View (KBV) of the Firm, Grant evades a specific definition of knowledge. Instead, he notes that the question, “What is knowledge?” has “intrigued some of the world’s greatest thinkers from Plato to Popper without the emergence of a clear consensus” and, therefore, asserts that “there are many types of knowledge relevant to the firm” (1996, p. 111). While a clear consensus on the definition of knowledge is not found, business literature focuses on two aspects that distinguish knowledge from information. The necessity of human interaction with information is the first. Badii and Sharif (2003) discuss the creation of knowledge by managers reflecting on previous experiences. This reflection leads to a “justified true belief” (N onaka, 1994, p. 15) based upon the experience of the person evaluating the information. The second differentiating factor for knowledge stems from a person’s ability to comprehend new information and predict future consequences based upon prior experience and evaluation of available information (Davenport and Prusak, 1998). The definition used throughout this research combines these two key points. For the purposes of this dissertation, knowledge is the person’s justified true belief providing a framework for evaluating and incorporating new experiences and information (N onaka, 1994; Davenport and Prusak, 1998). As with processing data into information, Davenport and Prusak propose four ways to transform information into knowledge. The first is comparison with previous experience or standards. The second is for the user to understand the consequences of acting on the information or what can be forecast from the information available. Third, the user makes connections between one piece of information and others that are 12 available. Lastly, the user converses with other users with access to similar information to gain a broader understanding of the meaning (Davenport and Prusak, 1998). Continuing with the fast food drive through processing example, the information the manager previously created from the data set of order input and order completion times can form the basis of useable knowledge. Once she compares her average order completion time to a standard, she understands whether her restaurant is performing well or below standards. Additionally, her previous experience from other months as well as realizing that the standard is based upon many years of historical customer satisfaction records allows her to add significance to the month’s information. This is closely related to understanding the consequences of the information. Her previous experience linked profitability with higher customer satisfaction and linked higher customer satisfaction with lower than standard wait times. Therefore, she knows the consequences of the month’s drive-through transaction information. She connects this month’s performance with other information available to her to determine if corrective action or rewards are appropriate for the employees. Through conversations with other franchise owners, she detects trends that may be affecting the industry as a whole and can better form a basis for future decision making. By arriving at a final set of beliefs from the information, she is prepared to incorporate the next month’s information and evaluate the ongoing performance of her drive through as well as predict profit margins for the upcoming months. The definitions presented above for data, information, and knowledge help bound this dissertation. In the following subsections, the theoretical framework supporting the 13 research questions is developed. The underlying theory is derived from the Resource Based View (RBV) of the Firm. 2.2 The Resource Based View of the Firm As firms develop strategies to compete in their industry, many will target certain areas of strength within the company to exploit in the marketplace. In order to provide a sustainable competitive advantage, these strengths must yield a unique position that other firms within the industry cannot easily duplicate. These concepts form the premise of the Resource Based View of the Firm (Andrews, 1971; Wernerfelt, 1984). RBV proposes a causal relationship between the assets available to a firm and the products or services offered to the marketplace. Additionally, there is capacity to exploit these resources to improve performance and sustain competitive advantage (Conner, 1991). Evolving from the general strategy of identifying strengths within a firm, specific traits are associated with RBV as a strategic foundation. Specifically, the four resource‘traits are valuable, rare, inimitable, and non-substitutable (Barney, 1991 ). Each of these four aspects is discussed in greater detail below. The theory of RBV provides a perspective to understand different levels of performance among different firms with different asset portfolios. These assets available to a firm are often separated into different categories for analysis. The first is physical resources. These include geography, physical plants, equipment, technology, raw material access, and distribution networks. The second focuses on the human factors available to a firm. These comprise training, knowledge, opportunities, and experience. 14 The third category includes organizational routines such as processes, controls, and practices regularly occurring within the firm (Barney, 1991). To provide sustained strategic advantage, a resource must be of value to a firm. The value could occur in a number of different means. The resource could be access to a market allowing the firm to outperform the competition in speed to deliver new products to the consumer. The resource could be an investment in technology that reduces internal quality concerns that previously existed. Regardless of the specific resource, the value added to the firm must be greater than the expense to gain that resource and be compatible with the firm’s overall strategy (Barney, 1991; Mahoney and Prandian, 1992). In addition to creating value within a firm, the resource should be rare to support building long-term strategy involving the resource. As an early adopter of the rare resource, a firm stands to control access to the resource within the industry. Access to a mine yielding an extremely pure titanium source that requires significantly less processing time or cost than the competition faces is one example. Employing a group of scientists who bring together a unique set of skills to solve problems that other research groups cannot is another example. As with the value aspect, the scarcity of the resource should return a greater reward than the cost of acquiring the resource (Barney, 1991). While rarity reduces the access of competition to a resource, a resource that is inimitable cannot be economically repeated by others in the industry (Barney, 1991). The reasons for inimitability could be diverse. In the previous example, it is possible that the purity of the titanium as a raw material allows the firm to utilize a different method for production than others in the industry. Because of the rarity of the raw material and the unique processing capability of the firm, the processing resource is inimitable within 15 the field. There may be other processes that are inimitable due to the knowledge or culture of the workforce (Conner and Prahalad, 1996). The Toyota Production System is a widely understood process to empower employees, reduce waste in the system, and paces production to a just-in-time supply system. Despite Toyota’s willingness to allow competitors to observe their process, other automotive firms cannot duplicate Toyota’s efficiency or quality of production. The cultural customs and knowledge the workforce gained through years of experience are among two of the reasons this production system has proven inimitable within the industry (Sugimori, Kusunoki, Cho, and Uchikawa, 1977; Womack, Jones and Roos, 1990). lnimitability of the production system supports Dierickx and Cool’s work identifying characteristics of accumulating the resource (1989). Specifically, the authors identify the value of time, mass efficiencies, interconnectedness of similar resources, resource erosion, and causal ambiguity as critical to establishing inimitability of a resource (Dierickx and C001, 1989). The previous three traits are clearly important to return value to the firm owning the resource. However, if the competition can easily attain a substitution to avoid the rarity, value, or inimitability of the resource, the firm no longer holds a sustainable competitive advantage (Barney, 1991; Peteraf, 1993). The existence of substitutes, as established by Porter’s five forces model, reduces the monopoly’s advantage in the marketplace (Porter, 1980). One example of a substitutable resource is an alternate material or process that replicates the performance of the finished product without requiring the rare, valuable, or inimitable resource. Altemately, a firm that creates a completely different paradigm may eliminate the need for the resource entirely. Dierickx and C001 (1989) present the example of Canon’s inability to compete with the dominant 16 position Xerox established in servicing copiers. By investing in a series of copiers that no longer required significant servicing, Canon was able to substitute a better and lower maintenance design for Xerox’s valuable and inimitable service network. Viewed through the RBV perspective, a firm’s decision making process could be a source of competitive advantage. In the case of managers compiling data through commercially-available database management systems, the decision making process would be easily imitated. However, if the process involves a unique manner of processing data into management information systems and the manager’s analytical process evolved from distinct corporate cultural or experiential knowledge, the process is likely to be rare, inimitable, and valuable. While the actual decision making process remains an individual-based action, firms leveraging their decision making processes as competitive advantage are likely to have a reliable, sustainable, accessible, and highly accurate information system to share among the workforce (Szulanski and Jensen, 2004). 2.3 The Knowledge Based View of the Firm In the Knowledge Based View of the Firm, it is recognized that RBV identifies opportunities for sustained competitive advantage by focusing on the strength of resources within a firm. However, KBV extends this assertion one step further to identify knowledge as the single most important resource available to a company (Grant, 1996; Zack, 1999). Firms with the ability to identify and exploit specialized knowledge are expected to adapt their resources more quickly and effectively in the marketplace than their competitors. Additionally, the internalization of existing knowledge can foster creation of new knowledge, thus reinforcing a dynamic progression of the critical traits l7 (Zack, 1999). As in the RBV, KBV seeks to identify sustainable advantages for firms. Two types of knowledge are identified as essential to sustaining performance: explicit and tacit. Explicit knowledge is defined by written procedures, recordable events, and established processes that a firm employs. It is transferable from one location or section of a firm to another. However, this trait also makes explicit knowledge easier for competitors to imitate unless additional factors are involved (Zack, 1999). As soft drink companies introduce a new packaging container for their products or an online news service releases a new format for their material, the competition can swiftly adapt and adopt a similar process without investing the research and development time and costs into the new process. In the general case of all other variables being equal, unique explicit knowledge is not likely to sustain competitive advantage for a firm. Tacit knowledge refers to context-specific knowledge gained from a period of experience within a company’s complex organizational routines (Zack, 1999). The implied significance of experiencing time within the operating system, comprehension of the underlying connectivity of resources, and causal ambiguity align with Dierickx and Cool’s identification of traits critical to establishing inimitability of a resource (1989). This tacit knowledge held within the workforce is extremely difficult to reproduce in other settings due to its evolution as a complex resource requiring firm-specific assets in order to be successful (Peteraf, 1993). Therefore, knowledge-based resources can yield valuable, rare, and inimitable advantages to a firm. The final significant facet identified in the RBV requires knowledge to provide a sustainable advantage that rebukes efforts of competitors to find substitutes. In their 18 study of absorptive capacity, Cohen and Leventhal (1990) find evidence that the more a firm knows, the greater its capacity to learn and incorporate additional knowledge. Therefore, it is predicted to be more difficult for competitors to absorb and process new information or acquire new knowledge than for a firm with unique tacit knowledge and experience in the arena (Zack, 1999). This stream of literature supports knowledge as a critical resource capable of sustaining competitive advantage. At the individual worker level, the tacit knowledge gained from experience with an operational system increases the opportunity to process new information and gain additional knowledge. After reaching a level of proficient knowledge about the job requirements, the individual is capable of evaluating which additional knowledge is critical to success and what order to learn new processes. This self-reinforcing cycle leads to a generalization that accumulation of increasing amounts of knowledge is beneficial and should lead to higher performance in the marketplace (Zack, 1999). 2.4 Fact-based Knowledge Within Firms Companies operating within a global supply chain setting rely on information technologies to monitor performance of plants, warehouses, transportation, personnel, and various other aspects of the business from a centralized location due to constraints on time and travel budgets. Numerous studies support the assertion that companies who capitalize on their information technology capabilities can achieve improved performance among their collaborative partners and increase coordination within their own organizations (Shin, 1999; Min, Roath, Daugherty, Genchev, Chen, Amdt, and Richey, 2005). As companies cross traditional firm boundaries with information systems, real- 19 time electronic data interchange allows manufacturers to forecast retailer demand patterns and promote sharing and consolidation of forecasting responsibilities (Challener, 2000; Xu, Dong, and Evers, 2001). This centralization of information processing reduces workload requirements on both firms and should lead to reduced amplification of order uncertainty by essentially eliminating one link from the supply chain (Disney and Towill, 2003). The firms must ensure they have compatible information technology systems, have sufficient quality controls in place, and resolve cultural or legal concerns with sharing data across firm boundaries (Kahn, Maltz, and Mentzer, 2006). Within companies, centralization of certain functions realizes significant cost savings and control of scarce resources. An unintended consequence of consolidation for some companies is the improved opportunity to acquire and disseminate implicit knowledge among workers performing similar tasks. During a presentation to the Michigan State University Operations Management Summit (2007), Sara Lee executives discussed the recent alignment of purchasing fimctions into a centralized location. Despite requiring an immense investment in information management systems, the company is saving substantial costs through economies of scale and improved bargaining power in specific markets due to the centralized structure. Sara Lee is also able to better share best practices among purchasing agents by co-locating them within a centralized unit. These purchasing agents share solicitation strategies, pricing information, and promotional opportunities at a significantly increased rate when they are physically working within the same office. In a similar move, the United States Air Force is moving toward a centralized purchasing structure estimated to save over $1 billion per year. In addition to the dollar savings and reduced workload at the individual Air Force bases, 20 leadership cites increased asset visibility, improved accountability in the contracting system, and better support to the customer as key benefits made possible by a centralized information management system. The purchasing agents share best practices, operating information, and knowledge of successful methods to prOcess and track their orders at a greater rate than when operating from geographically-separated bases (Holmes, 2008). These are two examples of large organizations operating in a global supply chain environment with centralized designs to better facilitate knowledge creation and transfer. In their 2003 work, Lee and Choi investigated knowledge management factors such as collaboration, trust, learning, centralization, formalization, skills that are both deep and broad, and information technology support to better understand their impact on organizational performance. Their model identified four specific areas of knowledge creation processes, socialization, extemalization, combination, and internalization, to measure the effects. Of particular significance to this dissertation, Lee and Choi found support for information technology systems as facilitators of statistical knowledge development. Similar results have been attained in Britain as radio fiequency identification (RFID) tags are added to increasing quantities of inventories of food and drug products. The RF ID tags are performing their expected function of providing management with near-instantaneous monitoring of inventory, logistics timeliness, and freshness factors. Some unexpected benefits have been to focus traceability for theft, recalls, and other unforeseen problems with the food or drug supply chain. Workers monitoring the RFID tracking stations have become knowledgeable about expected patterns and are able to 21 rapidly detect variance from the norm (Jones, Clarke-Hill, Comfort, Hillier, and Shears, 2005) 2.5 Information Cost, Complexity, and Processing Based upon the seemingly clear benefits of accurate and encompassing information management systems for improved operational performance, it would seem that every business would possess the highest level of information processing systems. The most significant barrier to this level of information availability is the complexity and cost of acquiring and operating such systems. As with most business decisions, a balance must be attained to maximize the operational performance contribution of an information management system at the most economical cost possible. In terms of cost and complexity, the United States government manages one of the most challenging networks of information. In Fiscal Year 2006, they invested $66.2 billion in information technology systems. Focusing solely on the Department of Transportation during the same period, the government purchased a total of 74 budgeting and accounting systems and 38 operational performance systems to track metrics within the workplace (www.whitehouse.gov/omb/ budget/fy2008). The immense investment required to purchase and maintain the systems is apparent. In addition to the cost, the complexity of maintaining the vast number of systems is daunting. With this number of new systems introduced in a single year, confusion is certain to occur when updating older systems or integrating newer ones. The opportunities to inadvertently present inaccurate information to the user are plentiful. The users are presented with a 22 challenging environment of diverse computer interfaces and knowledge to manage in order to successfully complete his or her job. In addition to the cost and complexity of the system at the firm level, the individuals processing the information have a separate level of constraints. The human mind is limited in its ability to process information. Bounded rationality is the term Simon (1957) proposes to identify a number of dimensions that limit human rationality. Due to the financial and comprehension limits of solving complex problems or storing information, Simon recommends use of heuristics for daily decision processing rather than making efforts to optimize each decision. By employing these useful tools to approximate optimal solutions within a certain degree of accuracy, the computational time and cost may be significantly reduced. Consideration of time allocated for decisions is a critical factor in many supply chain settings. Managers are frequently saturated with human resources requirements, meetings to attend, production reports to review, and internal performance improvement processes. The time available to oversee seemingly routine purchasing orders is likely minimal. Given this restricted time, the manager must be afforded the tools to make quick and accurate decisions. One method for compensating for the negative performance impact of time pressure is information compression (Spurrier, Topi, and Valacich, 1994). In the contemporary management arena, company dashboards accomplish the same task as the heuristics proposed by Simon (1957) and serve as the means for information compression as proposed by Spurrier et al (1994). The analogy to the dashboard in an automobile is intentional as managers are encouraged to focus on the 23 road ahead while frequently glancing at a few key indicators to provide status updates, warning codes, and verification of progress. If the gauges are remaining within the expected range, no action is required. Once the warning indicators show a trouble area, the manager can divert attention to the necessary item and set a corrective course (Dennis, 2006). The ability of management to respond quickly to accurate decision variables is significant in the global supply chain setting. Daily decisions frequently represent actions occurring months into the future due to lengthy lead times, transportation limitations, customs clearance times, and intermodal final delivery requirements. In order to provide the manager the best opportunity to succeed, the quality of the information provided as well as the quality of the decision making process must be of the highest quality due to their direct relationship on the performance of the organization (Huber, 1990). 2.6 Data — Information — Knowledge - Performance Linkages As discussed in Section 2.1, Davenport and Prusak (1998) present a testable framework for this study. The addition of a link from knowledge to performance, as in Figure 2.2, completes the bounds of this study. A number of researchers established strong linkages between knowledge and performance. A brief discussion follows. 24 Figure 2.2: Processes to transform data to information to knowledge (Davenport and Prusak, 1998) with performance link (Choi, Poon, and Davis, 2008) II, _ a Lin, ..1’)’ '- (Davenport and Prusak, 1998) I Information I F.- (Davenport and Prusak. 1998) I Knowledge |‘ l Performance m n - . --_‘_.' , - ‘2 ', ;.\-',1U' .‘siu ‘ .. --"'.- '- '= a . ‘n'r-r" . "'-,_‘-‘-'i'..-!‘I Wm “'u: if"! ‘ I.’ -'a '- .' -' _ 1 , ' _ ‘ ‘1 ' , . ' ‘ .. ‘ ' I ‘ ‘ ' I. ' H .... ' . t‘ ....' ,“~ " -' ..‘L‘| a ‘ I“. ‘ .1. . 1 ‘c'i . .4» x, . t It; - 'L‘ ‘1 x a. I 'u ' m1 i m. ,v .. u“. -J'L v- a» - [all u Choi, Poon, and Davis (2006) utilize a novel theoretical approach involving complementarities of knowledge management strategies and operational performance. The primary supported finding involved firms using a combination of a tacit-internal strategy and an explicit-extemal strategy outperformed any of the other combinations. Their evaluation of 131 firms engaging in knowledge management practices revealed performance improvements in those firms capable of combining a strategy based upon following documented best practices in relationships outside the organization and capitalizing on intemally-acquired knowledge for within-firm decisions. In addition to the significance of the complementarities in Choi, Poon, and Davis’ (2006) work, the authors found support for differentiating operational performance based 25 upon knowledge management strategy. For a setting in which all parties have the same structure and opportunity for developing a knowledge strategy, the performance link is expected to be present and equal for all of the four groups in the study. Through different methodologies, researchers such as Sharma, Levy, and Evanschitzky (2007) and van Buren (1999) found similar linkages between knowledge management strategy employment and operational performance. Kanawattanachai and Y00 (2007) identified links between knowledge and operational performance through study of virtual teams. These researchers found the three factors of expertise location, task-knowledge coordination, and cognition-based trust all had varying impacts on operational performance over time. While virtual teams took longer to develop trust than co-located teams, once a member demonstrated a sufficient level of knowledge for the task, the location bias was eliminated. Additional research in the area includes Lee and Yang’s (2000) work on the knowledge value chain and the need to reach an established level of knowledge value in order to yield knowledge performance. Pathirage, Amaratunga, and Haigh (2007) evaluated the tacit knowledge present within construction workers and again identified a strong link between a sufficient level of task-oriented knowledge and operational performance. 2.7 Conceptual Model and Research Hypotheses With the research framework supported through extant literature and established theoretical grounds, the final evolution of Davenport and Prusak’s (1998) hierarchy positions this research. As seen in Figure 3, the research questions are established by holding three of the methods for transforming data to information constant as well as all 26 four of the methods for transforming information to knowledge. The result is a 2x2 experimental design in which the information is either corrected or includes a random erroneous factor and the data are either condensed into aggregate form or provided in full detail. This design yields four categories of information. Evaluation of the research questions involves hypotheses for each as explained below. Figure 2.3: Processes to transform data to information to knowledge (Davenport and Prusak, 1998) with performance link (Choi, Poon, and Davis, 2008) and experimental design. Data (Accurate) ** 2 ' Information 1, (Four categories) 3 1 4 ‘ 1v ~n_,n_5'.-'..‘.. -' “"’ ‘ --* i ‘3 I Performance | Knowledge l ‘ I (Financial/Operating) Using Davenport and Prusak’s (1998) model as a basis, the research model was developed to capture the experimental factors, environmental factors, operational decisions, and overall performance as it relates to the Supply Chain Operations Decision Environment (SCODE). The administrator set the supply chain environment for the 27 duration of the exercise. This environment directly impacts the criteria utilized by the participants for operational decision making as well as affecting the operation of the SCODE model during the weekly iterations of the scenario. Participants entered weekly decisions impacting their supply chain model and entered those into the SCODE simulation. After processing the input from both the administrative environmental settings and the participant’s operational decisions, SCODE provides the users with performance feedback. The subjects were randomly assigned to one of the four experimental factor groups which impacted whether the data was condensed or not and whether the information was corrected or not. The weekly performance measure then sets the baseline for future decisions as processed through the experimental factors for the subjects. Specifics of the methodology are presented in Chapter 3. 28 Figure 2.4: Research Model Supply Chain Environment - Location of Plants - Base and Variation of Demand - Reliability of Transportation - Raw Material and Labor Requirements 1 _(_)_perational Decisions - Select Supplier(s) - Purchase Raw Materials _, - Determine Speed of Production - Select Product Mix - Identify Markets to Service Demand - Select Transportation to Destination l SCODEF l Performance - Inbound - Market - Internal - Net Profit Margin - Outbound - Supply Chain Contribution l , Experimental Factors - Condensed / Uncondensed Data - Corrected / Uncorrected Information 29 Research Question 1 asks whether decision information systems mediate the impact of data saturation on a supply chain manager’s performance. Tools such as management dashboards are prevalent in industry to quiCkly highlight metrics or status levels that require attention by a manager. This research focuses on the dashboard’s ability to process vast amounts of data into a quickly comprehendible visual signal and evaluate the manager’s ability to make micro-level decisions from the macro-level aggregate metrics. The first Research Question explores the ability of information systems to mediate the impact of data saturation. Specifically, it states: RQI .' In a time-constrained setting, do decision information systems mediate the impact of data saturation on a supply chain manager’s performance? H1: Supply chain managers performing in an environment with uncondensed data, time constraints, and uncorrected decision information systems perform at a lower level than those with corrected decision information systems. This hypothesis delineates operating performance expectations for the two groups with uncondensed data treatments. Given complete data availability, the group with accurate decision information systems is expected to perform at a higher level than the group with inaccurate decision information systems. This finding should be expected, but becomes more interesting in light of the second research question and hypothesis. Research Question 2 focuses on the level of detail of the data available in the presence of inaccurate decision information systems. If the information management dashboard presents incorrect information, is it better for the manager to have access to 30 vast amounts of data or compressed amounts of data? The next two research questions and hypotheses follow: RQZ: In a time-constrained setting, does the level of detail of the data mediate the impact of inaccurate decision information systems on a manager’s decision making ability? H2: Supply chain managers operating in an environment of uncorrected decision information systems, time constraints, and uncondensed data perform at a lower level than those with condensed data. RQ3: Is there an interaction effect between the accuracy of the information system and the level of detail of the data? H3: There is an interaction between the level of detail of the data and the accuracy of the information system. The effect will result in the following performance levels for the various treatment groups: 1) Highest performance level: Corrected information / Uncondensed data 2) 2d highest performance level: Corrected information / Condensed data 3) 3d highest performance level: Uncorrected information / Condensed data 4) Lowest performance level: Uncorrected information / Uncondensed data The hypothesized interaction effect between the two variables marks a departure from Davenport and Prusak’s (1998) model, but finds support in literature. Simon’s assertion that bounded rationality limits the amount of data able to be processed supports the first hypothesis. In conditions where the information management dashboard is accurate, any additional insight the user receives directly from the data will augment performance. However, when the user is forced to abandon the information management dashboard due to inaccuracies, the additional data becomes burdensome and the user cannot distinguish the important variables from the noise of the data. 31 The fourth research question focuses on the user’s reaction to incorrect information in the dashboard. By manipulating the patterns of exposure to incorrect information, the following hypotheses are tested: RQ4: Is there a pattern of incorrect information provided by decision information systems before a manager seeks alternate decision variables? H4a: Managers provided consecutive reports from a decision information system containing incorrect information cease utilization of the information system and made future decisions based on another approach. H4b: Once managers seek an alternate approach to decision making, they do not return to utilizing the decision information system regardless of the accuracy of the system. These two hypotheses are based upon the user’s learning as the simulation progresses. The first time the user is presented with incorrect information, it is expected to take considerable time to identify alternate decision variables and establish a method to incorporate the raw data into a decision making framework. After the information management dashboard is consistently wrong enough to provide the user with sufficient motivation, the user identifies other variables, regardless of whether they are as useful as the corrected information could have been, and continues to use those variables throughout the remainder of the exercise. The final research questions focus on the crux of the question facing supply chain managers today. It states: RQ5: What do managers identify as the most important and least important data to monitor in a supply chain? 32 The final research question asks which decision variables the majority of managers identify as the most important and least important to monitor in a supply chain. Analysis of the group consensus contributes to identifying the most critical data and information for management information systems to capture accurately as well as offering a listing of the variables that may not need to be displayed. This opportunity to discuss the results of the experiment with the users immediately following the session contributed to capturing a more accurate image of the user’s experience. Further, by highlighting the most useful and least useful data, recommendations can be developed to focus information technology spending in improving accuracy of those critical areas rather than dispersing the funds across a wider and less useful collection strategy. 33 CHAPTER III: RESEARCH METHODOLOGY 3.0 Introduction This chapter describes the approach used to test the hypotheses proposed in the previous chapter. Following a pilot study to verify both the simulation environment and the experimental instruments, several sessions of the main study were conducted. The following sections describe the simulation program, validation and verification processes, participant selection, identification of variables, and the approach to data analysis. The analysis of data is presented in Chapter 4. 3.1 Research Objective This research examined the challenges faced by supply chain managers operating in an environment characterized by an overwhelming quantity of available data, information technology systems with varying degrees of reliability, and constrained time to complete all necessary tasks. Specifically, this study investigated the effects of varying quality of information and quantity of data on a manager’s ability to make decisions in a time-constrained setting. The questions to be answered by this research are listed in Chapter 2. The following sections explicate the experimental design focusing on these four questions. 3.2 Simulation Modeling Supply chain management research presents unique challenges when designing an experiment affecting financial and operational performance in a firm. Investigating phenomena in their naturally occurring environment is preferred, but not always possible. 34 The intricate nature of a supply chain, the inability to control the environment to completely account for confounding factors, and the potentially adverse effect on a firm’s performance make supply chain experiments difficult to accomplish in real world settings (Kelton, Sadowski, and Sturrock, 2003). Research involVing issues as complex as supply chain management can best be evaluated using dynamic approaches in a controlled setting (Ridalls, Bennett, and Tipi, 2000). Computer simulation is one recommended dynamic modeling method enabling the researcher to provide a controlled environment in a cost-effective manner without adversely impacting a firm’s livelihood (Helo, 2000). Providing research evidence to evaluate the above hypotheses in a global supply chain setting can therefore be appraised through use of a verified and validated computer simulation program. An area of less precedent in literature is the use of simulation to stimulate theory development. Davis, Eisenhardt, and Bingham (2008) explore the use of simulation methods to develop theory. These authors provide a number of guidelines as a roadmap for using simulation in theory development. Additionally, they present advantages simulation offers over competing methods. By starting the study with clearly defined research objectives rooted in established literature, simulation provides a unique opportunity to isolate specific interactions over time. Additionally, the precision and ability to measure numerous performance variables simultaneously lends simulation to refining theory. A common criticism for simulation studies stems from researchers viewing the setting as too artificial and the outcomes as predetermined. However, by following the recommended roadmap, these concerns are sufficiently addressed. Table 35 3.1 summarizes the steps to ground simulation studies while pushing the research envelope to expand or develop new theory. Table 3.1: Roadmap for Developing Theory Using Simulation Methods (Davis, Eisenhardt, and Bingham, 2008, p. 482) Step 1. Begin with a research question 2. Identify simple theory 3. Choose a simulation approach 4. Create computational representation 5. Verify computational representation 6. Experiment to build novel theory 7. Validate with empirical data This dissertation follows the recommended roadmap to build theory. Step 1 is presented in the introduction. Step 2 is discussed in Section 11. Steps 3 — 5 are discussed in this section. Steps 6 and 7 are explored in Section IV. 3. 2.1 The Supply Chain Operations Decision Environment Model The Supply Chain Operations Decision Environment (SCODE) is a “simulation training program to help executives and students learn the Digital Enterprise intricacies of international supply chain management” (www.bus.msu.edu/clodeo. Through this simulation, teams create supply networks to acquire, store, manufacture, market, and distribute goods throughout an international setting. 36 The general environment is depicted in Figure 3.1. The simulation administrator creates an environment designed to focus participant learning in a specific supply chain objective. During the training and experimental sessions for this research, the objective is to gain a better understanding of purchasing raw materials in a global supply chain environment. In general, the users may have the opportunity to manufacture up to four products, with each requiring a specific number of labor hours and a mix of up to four raw materials to produce one unit of finished goods. These raw materials may be purchased through contract or spot-market buys and may be found in any of four global regions designated by “North America,” “South America,” “Europe,” and “China.” There may be up to four manufacturing plants for the users to select from, which include co-located raw material warehouses and a finished goods warehouse, with a single plant in each of the previously identified four regions. Adding the region “India” to the previous list completes the five markets providing demand for the finished goods. Raw material and finished goods are transported throughout the supply chain by air, land, or sea channels. The participants have the opportunity to select the mode of travel based upon the speed required and price he or she is willing to pay to meet his or her objectives. The specifics involved in the experimental scenario are presented in Section 3.3. 37 Figure 3.1: SCODE Supply Chain Network Suppliers Suppliers Suppliers Suppliers (North America) (Europe) (South America) (China) l i v v v - 1 North America Europe South America China RM warehouse RM warehouse RM warehouse RM warehouse Plant Plant Plant Plant DC DC DC DC v j i v v v V r r DC (North DC DC DC DC America) (Europe) (South (China) (India) America) 1 r r v v Market (N. A.) The simulation operates at three distinct levels. The highest architectural level involves the “Administrator” decisions. These options define the environment for the users. The scenario designer can tailor any variables ranging from raw material price or availability to the market demand function curve and including over 1,000 other variables impacting all aspects of the supply chain. The second level is the supply chain “Design.” The design worksheets allow the users to set the initial conditions for their supply chain. 38 While some variables can be altered later in the game, usually at a substantial cost, others remain fixed for the duration of the exercise. The operational level involves the “Weekly” decisions. Each round simulates one week of elapsed time where the users make decisions about the product quantity and mix to manufacture, the transportation mode for raw materials or fixed goods, production acceleration to meet immediate market demand, and investment decisions for research and development or promotions, to name a few variables. The User’s Manual Executive Summary is included in Appendix A. Figure 3.2: SCODE model framework Strategic Decisions Administrator Decisions (One time team decisions at start) (Controlled by administrator) . Plant (own or contract) 0 Demand variation . Plant capacity . Production economies 0 DC (own or contract) . Transportation lead times a DC capacity oSupplier availability 0 Market & DC allocation oReport detail 0 Initialization Decision 0 Initial inventory 0 Scheduling l SCODE Operations Decisions (Week/y team decisions) oSourcing (plant, supplier, raw material, mode, quantity) oProduction (what to produce where ‘ RESULTS and when, quantity) oMarketing (R&D, promotion, price) oDistribution (source, DC, mode, quantity) 39 Feedback from the simulation occurs on two worksheets. The first is a financial report containing data ranging from revenue to detailed supply, plant, and distribution center costs for both the current week and year to date. This financial data lists aggregate data, data separated by region, and data ordered by product. The second form of feedback is the “Output” reports providing detailed analysis of the supply chain performance. The report contains status on seven different areas including raw material quantity and locations, production output and schedules, distribution center performance, location and expected receipts of finished goods, sales report, lost sales report, and system messages. As an additional aid to users, the output worksheet contains a decision information system which captures relevant data and recommends key variable input for the next week of operational decisions. For example, the interface detects the team scheduled production of a product requiring 100 units of a raw material, but there are only 20 units of this raw material in the warehouse. The display would recommend ordering 80 additional units of the raw material to meet the production schedule. The portions of SCODE directly affecting the experimental variables are discussed in greater detail in Section 3.3. 3. 2.2 Model Verification and Validation When creating a simulation model, two items are critical for success: verification and validation. Verification involves certifying the computer program is correctly designed and implemented (Sargent, 2000). Validation tests ensure the computer simulation is producing results that are consistent with possible results from the actual system operating in the business world (Sargent, 2000). Four categories of validation 40 include conceptual model validity, computerized model verification, operational validity, and data validity (Sargent, 2000). These are addressed individually below. Conceptual model validity requires evaluation of the theories and assmnptions upon which the model is built as well as determination that the model is reasonable for its experimental purpose. Two typical methods employed in conceptual model validity are face validity and exploring traces (Law and Kelton, 2000). In the case of SCODE, a panel of experts comprised of industry executives and academic professors envisioned the simulation environment. The vision was implemented by computer programmers and supply chain management students. This implementation team closely followed trace technique to evaluate every step of the simulation program while processing input and output. Comparison with hand calculations and expected outcomes highlighted discrepancies. The program was continually refined until the implementation team was confident with the operation and results SCODE provided. Throughout the creation of SCODE, the team was closely advised by the panel of executives and academics. The expert panel reviewed the operational environment and tested SCODE with realistic inputs and confirmed the model was based upon valid constructs and implemented in a way that met the purpose of creating SCODE. Therefore, conceptual model validity was achieved. Computerized model verification requires analysis of the computer programming and implementation of the model. Traces and error log reviews are typical methods utilized in computer model verification (Sargent, 2000). As previously discussed, meticulous trace examination of SCODE input and output screens provided evidence of an accurate computerized model. Further, SCODE outputs error messages to the user 41 whenever input is inconsistent with expectations. Close scrutiny of these error messages led to further refinement of SCODE. Once the trace examinations and error messages confirmed proper execution of the program, the computerized model verification requirement was met. Operational validity is attained when the output has sufficient accuracy for the input conditions. A number of tests exist for operational validity including extreme condition tests, internal validity checks, parameter variability-sensitivity analysis, and degenerate tests (Kelton et al, 2003). In determining the operational validity of SCODE, all four of these methods were used and explained in detail below. Extreme condition tests require input of values at the limit of operational conditions to ensure the simulation system reacts appropriately (Kelton et al, 2003). For example, exceedingly large or small quantities of orders were processed within the same week, large changes in warehouse capacity entered, and unusually long transportation delays created to examine the effects on SCODE. In each case, the program responded as expected. Internal validity checks are implemented by examining the internal stochastic variability across several model replications (Kelton et al, 2003). SCODE offers the ability to set a fixed random seed or variable random seed. For conducting internal validity checks, the random seed was varied and the coefficient of variation measured with the same set of input data. The model performed as expected, with the coefficient of variation remained less than three percent across all measured variables. This low degree of variation lends credibility to the operation of the simulation. 42 Parameter variability-sensitivity analysis involves varying known parameters to determine the impact on the output (Kelton et al, 2003). This process was performed in SCODE at numerous levels. By fixing the random seed and varying input, the expected results were compared with the experimental results. In One case, the likelihood of experiencing a transportation-related delay was significantly increased. As a result, the supply chain experienced shortages of all raw materials, production was reduced, and lost sales skyrocketed. Another case lowered labor hours required to produce each product. SCODE output showed the profits soared as all demand was fulfilled at a fraction of the previous production operating cost. Over a number of these test cases, SCODE performed as expected, lending another degree of operational validity support. The final technique employed degenerate tests to manipulate input parameters in a known way to create constraints or extremes in the system (Kelton et al, 2003). As SCODE’s warehouse capacity was decreased below the level of demand, the manufacturing plants lacked storage for finished goods. The demurrage charges escalated and lost sales increased. This display of expected results indicated passing marks for SCODE’s ability to handle degenerative inputs. The fourth and final validation category involves data validity. Expert review and internal consistency checks are the two most common methods of ensuring data validity (Kelton et al, 2003). The administrative settings in SCODE are obtained and updated according to publicly available statistics and values. As one example, the costs of transportation are checked monthly against figures released through the Department of Transportation. Expert analysis of all variables present in the administrative settings supported face validity of the constructs. Internal consistency checks similarly ensured 43 the absence of outliers or incorrect information encoded in the administrative settings of the program. SCODE underwent extensive pre-testing prior to initiating the experiment. During development, SCODE evolved incrementally from basic design to its current operational state. Separate teams of experts in computer programming, supply chain management, computer simulation, and research design alternately implemented and tested each facet of SCODE as well as validating other teams’ work. As the computer programmers introduced a new feature to SCODE, the simulation and supply chain management teams tested the functionality, incorporated the new features into existing scenarios, and validated the feature performed as designed. Periodic presentations to a research board and peer review also contributed to the verification and validation of SCODE. Experimental validation occurred with a group of users who had not previously utilized SCODE, but were considered highly qualified in supply chain management experience. These executive participants completed the entire experimental design as outlined below with one additional feedback session focused on comprehension and interpretation of the survey questions and the perceived effectiveness of the experiment to capture research variables. Following this validation, minor questionnaire and scenario design changes were implemented. 44 3.3 Research Design This section focuses on the technical aspects of the study. Sections include the target sample population, the pilot study, identification of the variables and controls, and method of analysis. 3. 3.1 Experimental Population The experimental participants include Master of Business Administration (MBA) students attending one of three programs offered through Michigan State University. The first session involved full-time traditional MBA students. The second included Executive MBA students attending supply chain management courses in the evening. The third and fourth sessions were comprised of Weekend MBA students attending a supply chain management course over a series of weekends. All of the MBA students were pre- screened to ensure a sufficient SCM background to have meaningfirl participation in this study. Power analysis and sample size calculations were performed using a statistical calculator designed by Lenth (2006) based upon the calculations of Cohen (1975). Given the 2 x 2 factor design of this study, a minimum sample size of 20 participants per block is required to provide a power of at least 0.9 given an alpha level of 0.01. 3. 3. 2 Experimental Design All SCODE sessions began with a one-hour presentation about a specific topic of supply chain management interest. In this study, the topic is Global Sourcing. The briefing (Appendix B) provided participants with a baseline level of knowledge about the 45 subject. The Global Sourcing training expanded upon the participant’s general management knowledge by identifying current trends in research, demonstrating common benefits and concerns about purchasing in a global environment, and enumerating best practices seen in industries. Classroom discussion was encouraged for the users to share their experience in the context of the session. Following the topic lesson, users were introduced to SCODE through a hands-on training session. This training session reinforced the topic area while introducing the users to SCODE through a reduced-scale practice scenario. Again following the Global Sourcing example, the scenario was tailored to include only four decision variables, simplified assignment of manufacturing locations and capacities, fixed sales prices of raw materials and finished goods, and additional assumptions to provide users an acclimation opportunity with the new software. Users were guided through the design input interface, the feedback information and location, and the decisions that are required to sustain their supply chain. The decision variables followed the classroom discussion to enhance the understanding of benefits and challenges to operating a supply chain in a global market. Following repeated rounds, or simulated weeks, of training, users had an opportunity to ask questions or discuss any issues with the software, scenario, or focused training area. After this session, the users reach a level of proficiency to understand the cause-effect relationship between their inputs and the feedback SCODE provides. The computers are reinitialized and the recorded experiment begins. The second scenario is more complex than the orientation problem and includes ten decision variables, dynamic market demand, supplier shortages, production quality deficiencies, and numerous other obstacles simulating industry challenges. This 46 increased level of difficulty provides users with a more realistic supply chain environment facilitating generalization of the study results to broader supply chain practices. A ten minute lecture introduces the specifics of the scenario and advises the teams of the strict time table to be followed for this simulation setting. The scenario hand-out is included in Appendix C. The participants operate a production facility in North America with the opportunity to manufacture four finished goods. Individuals may choose to limit the quantity of finished goods in favor of a simplified supply chain, but at the expense of potentially greater income. Such decisions are noted during data analysis and incorporated into the performance evaluation and data analysis portion of the research. Each participant has twenty minutes to create and input his or her supply chain design. Facilitators provide enough assistance to ensure the users are able to input the supply chain as designed, but do not provide advice on network creation. At the end of the design phase, participants have ten minutes to enter his or her decisions for the first week. The members simultaneously execute the first week’s run and again have ten minutes to analyze the results, make decisions, and input the next week’s operational instructions. This process continues until round six, when the time is reduced to five minutes between rounds for all remaining rounds. As with the design, facilitators provide help identifying the proper places to input the weekly operational decisions, but will not help locate information from the output or interpret the data. 47 3. 3.3 Experimental Variables Designing an experiment to measure differences in performance within an environment requires manipulation of certain variables while holding others constant. This approach increases the likelihood that any differences in measurement are due to actual differences in the desired variables rather than outside random fluctuations. A full factorial experiment involves manipulation of all experimental factors simultaneously (Rodrigues, 2004). In this research, two factors are varied in two dimensions each. The first is quantity of feedback data provided. The second is the quality of feedback information provided. Each of the SCODE participants is randomly assigned to one of four experimental groups. . The first treatment group received all available SCODE “Output” and “Financial” feedback data. Additionally, the decision information system on the “Weekly” worksheet always provides accurate information to assist with the next week’s raw material ordering requirements, labor force utilization, and warehouse inventory status. This group is identified as the “Uncondensed / Corrected” group. The second treatment group received the same feedback data as the first group. However, the decision information system provides incorrect information to the team 30 percent of the time. The information may include the wrong quantity of a material to order, mislead the team about the quantity of products capable of being produced in the next week, or incorrectly report the usage rates within the warehouses. This group is identified as “Uncondensed / Uncorrected” for complete data, but questionable information. Figures 3.3 and 3.4 illustrate the management information dashboard with correct and incorrect results for the same data set. 48 Figure 3.3: Management information dashboard with accurate feedback. When viewed in color, the areas requiring immediate attention of the user are highlighted in red. For example, this screen is telling the user to order 24,899 units of “Raw Material A” for the South American plant immediately to meet the production requirements for the week. lnformat ion Technology Interf ace Needed to Meet Scheduled Production for Next Week Raw Material North Amgrig _E_u_r_gpe Sgum Amgrigg Chin; A 0 0 24899 27090 B 0 0 -269045 -9525 C 0 0 403238 4091 D 0 0 -8948 0 Percentage of Labor Force Scheduled for Next Week North America Eurog South America China 0.0% 0.0% 88.4% 100.0% Percentage of Raw Material Warehouse in Use for Next Week 02mm gums mm M 0.0% 0.0% 110.8% 96.4% Percentage of Finished Goods Distribution Center in Use for Next Week North America Eurog South America China India 0.0% 0.0% 0.0% 0.0% 0.0% Percentage of Demand Satisfied This Week North America Eurou South America China India By Region: 62.4% 24.7% 25.8% 0.0% 0.0% nguid Regular Liguld Premium Dy Regular Du Premium By Product: 89.2% 4.4% 0.0% 57.9% Performance Metrics Weeks of Raw Material in the Warehouse (weekly) 0.6 Percentage of Production Completed (weekly) 33.4% Weeks of Finished Goods in Warehouse (weekly) 0.0 Order Fulfillment (year to date) 47.5% 49 Figure 3.4: Management information dashboard with inaccurate feedback. The data supporting this dashboard is identical to that in Figure 3.3. However, a single factor used for creating the dashboard has been randomly altered. Instead of placing the correct order (as in Figure 3.3 above), the user is recommended to place an order for 105,899 units of “Raw Material A” for the South American plant. lnformat ion Technology lnterf ace Needed to Meet Scheduled Production for Next Week Raw Matgrial North America Europg Sguth Amgrig Chlng A 0 0 105899 127090 B 0 0 -169545 90475 C 0 0 -64738 104091 D 0 0 9552 0 Percentage of Labor Force Scheduled for Next Week North America Eurom South America China 0.0% 0.0% 132.7% 150.0% Percentage of Raw Material Warehouse in Use for Next Week Loam m ME! 91m 0.0% 0.0% 110.8% 96.4% Percentage of Finished Goods Distribution Center in Use for Next Week North America Europ_e_ South America China India 0.0% 0.0% 0.0% 0.0% 0.0% Percentage of Demand Satisfied This Week W M MM; 2111—"! Me By. Region: 62.4% 24.7% 25.8% 0.0% 0.0% Liguld Rggulgr Liguld Premium Du Rggular m PQmium fly Product: 89.2% 4.4% 0.0% 57.9% Performance Metrics Weeks of Raw Material in the Warehouse (weekly) 0.6 Percentage of Production Completed (weekly) 33.4% Weeks of Finished Goods in Warehouse (weekly) 0.0 Order Fulfillment (year to date) 47.5% The error rate factor for the information management dashboard is based on refinement through the validation process as well as previous research. Kang and 50 Gershwin (2008) evaluated the accuracy of inventory systems as part of their study to reduce stock outs. In their review of 500 stores in various industries, the authors found the highest level of inventory accuracy to be between 75 — 80%. The inventory information systems averaged 51% accuracy across the stores. In a previous study, Raman, Deoratius, and Ton (2001) investigated over 370,000 stock keeping units within a single retailer and found over 65% of the inventory systems reflected inaccurate quantities. The initial experimental validation session for this research utilized a 50% error factor for the frequency of projecting incorrect information on the management information dashboard. Participants provided feedback indicating this level seemed unusually high and created the impression that the simulation program was not working correctly. The users expressed doubts about. the accuracy of the data, the input screens, and the delivery of raw material in addition to the information dashboard. The second experimental validation session reduced the error rate to 30%, placing it at the lowest level of Kang and Gershwin’s findings. The second experimental validation participants were better able to differentiate between the accuracy of the data and the inaccuracy of the management information dashboard than the first group. The second group did not relate incorrect information on the dashboard with simulation problems. During the feedback session at the end of the validation, the majority of users with incorrect information correctly assumed this phenomenon was part of the learning environment and expressed trust in the operating performance of the input, order processing, and delivery mechanisms of the simulation. The magnitude of the effect also combined experimental validation feedback with previous information systems research. In their 1995 work, Rohleder and Scudder 51 investigated methods to maximize the net present value as a scheduling performance measure. They hypothesized a significant concern with using such measures stemmed from inaccuracies in two of the key factors comprising the metric. Their research concluded that these performance metrics proved robust to error levels below 40% magnitude of change. In other words, even though the contributing factors to the metric varied by as much as 40% from accuracy, the metrics proved sufficiently resilient to recommend the proper course of action. Therefore, an initial error factor of 50% of the magnitude was selected for the experimental validation studies. This factor went largely unnoticed by participants in the validation studies. Therefore, a final range of 50% - 200% of the correct value was selected. Excel selects a random number as a multiplier for the dashboard and can provide erroneous information in the positive or negative direction. SCODE processes the data required for the management information dashboard by taking the week’s performance output data and comparing it with the next week’s scheduled production and expected demand. For example, to calculate the raw material required within a region for the next week, the equation is: 4 QAN = Z PNyRANy — WAN '- DAN y=1 where: Q AN is the quantity of raw material A required in North America Py is the quantity of finished good y scheduled for production for the next week (Dry/Liquid and Premium/Regular) RANy is the quantity of raw material A required to produce one unit of finished good Py in North America WAN is the quantity of raw material A currently available in the North American warehouse DAN is the quantity of raw material A forecast for delivery at the start of the next week at the North American warehouse 52 The error term is introduced by altering RANy by a factor of 50% and rounding to the nearest whole unit in the results. In a simplified example, if the user chose only a single finished good to produce, “Dry Regular,” and operated a plant in South America, the following calculations would be made. The order quantity to meet next week’s production is the amount of product to be produced, say 10,000 units, multiplied by the amount of raw material A required to produce one unit of “Dry Regular” finished good, say 2. Therefore, the quantity of raw material A needed is 20,000 units. If the user carried over 5,000 units from the previous week and expects a delivery of another 5,000 units, the management information dashboard will tell the user to order an additional 10,000 units to meet the next week’s schedule. For the users with an inaccurate dashboard, SCODE multiplies the raw material requirement by 1.5 to yield a raw material requirement of 30,000 units for the same 10,000 units of finished goods. Subtracting the 10,000 units in the warehouse and delivery status, the management information dashboard tells the user to order 20,000 units to meet the next week’s schedule. While the users in the experimental validation study were concerned about the apparent system problems triggered by the management information dashboard presenting erroneous information too often, the magnitude of the error was not a concern. Therefore, it was unchanged for the second experimental validation session. This group responded favorably to both the pattern and magnitude of the errors. The feedback indicated the error magnitude was sufficient to trigger awareness, but not too exaggerated to be unrealistic. 53 The third treatment group of users receives only condensed feedback data. Rather than having full access to the data provided in the “Financial” and “Output” worksheets, the teams receive a condensed version of the data only. The financial information by region or product is not available and the specific cost centers are shown in aggregate rather than in item-by-item detail. For example, Figures 3.5 through 3.8 presents the uncondensed and condensed data from two different financial and output screens. The decision information system is always accurate for this team. This group is identified as “Condensed / Corrected” for condensed data and a correct information dashboard. Figure 3.5: Production output data for team with uncondensed data. r—__Te7m Number ' 1] ~‘* C4...) .4...,..~,..,;,1._,.;»,_.-.. 9.14.! PRODUCTION REPORT Premium Premium Subtotal Figure 3.6: Production output data for team with condensed data. I Team Number 1] PRODUCTION REPORT Premium Premium 54 Figure 3.7: Financial output data for team with uncondensed data. SCM Budget Less: Supply Order Placement Purchase Raw Materials Purchase Price Variance Transportation ( Supplier-Plant) Inventory Carrying Contract Not Fulfilled Total Supply Less: Plant Acceleration Charge Excess P roduction Charge Production Variance Charges to Alter Production Capacity Warehousing Demurrage Plant Pro duction Facility Plant Facility Charges to Alter Warehouse Capacity Total Plant Less: DC Order Placement Transportation Inventory Carrying Facility Warehousing... Demurrage Charges to Alter Warehouse Capacity Total DC Less: Market Transportation Total SCM Expenses SCM Contri buuon 55 wrathmw . I ' a. I‘ ‘ 1." " M20 ' $0.00 $0.00 $0.00 $0.00 $25,61 1.72 $0.00 $25,61 1.72 $0.00 $52,548.05 $91,281.09 $143,829.15 $0.00 $0.00 $17,711.30 $2,465.46 $0.00 $20,176.76 $4,200.00 $193,817.63 $96,682.18 ‘.a. " $911 $119.70 $365,253.79 $31,762.47 $35,610.86 $64,318.03 $0.00 $497,064.84 $4,878.47 $126,851.51 $165,948.88 $297,678.86 $0.00 $0.00 $40,343.99 $7,422.72 $0.00 $47,766.71 $16,800.00 $859,310.40 $52,089.76 Figure 3.8: Financial output data for team with condensed data. SCM Budget Less: Supply Total Supply Less: Plant Total Plant Less: DC Total DC Less: Market Transportation Total SCM Expenses SCM Contribution ,, " .‘J’F‘. ’ $2901.81,848 ' $66,911.77 $155,524.75 $22,584.59 $0.00 $245,021.10 $1,567,827.80 8.0 $1,065,801.17 $444,220.14 $162,202.17 $0.00 $1,672,223.47 $4,669,935.43 The last treatment group receives only condensed information from the “Financial” and “Output” reports as well as occasionally inaccurate recommendations from the decision information system. This group is identified as “Condensed / Uncorrected” for condensed data and questionable information. A summary of the groups is shown in Figure 3.9. 56 Figure 3.9: Experimental groups. 'O 93 9 E . g 8 Uncondensed Condensed/ «E / Corrected Corrected .2 a E .9 E C .3 8 Uncondensed Condensed/ g g /Uncorrected Uncorrected 9 __2 Uncondensed Condensed “Financial” and “Output” Feedback Data 3. 3. 4 Performance Variables Six performance variables are recorded for every week of the experiment. These metrics were previously developed in business literature (Martin and Stanford, 2007; Robinson, Sahin, and Gao, 2008; White, 1996; Wacker, 1996; Bendoly, Rosenzweig, and Stratman, 2007) as common metrics monitored by supply chain managers and validated by members of the CLODE Research Board. These six are summarized in Table 3.2 and explained more completely following. 57 Table 3.2. Dependent variables, their equations, and significance. Performance Equation Measures condition of: Variables Weeks of raw Units RM in warehouse . . . . . . . - Efficrency of inbound materral 1n Unrts RM used 1n prevrous . supply charn warehouse week Percentage of Units produced Effectiveness of production completed Units of production scheduled production scheduling Weeks of finished goods in warehouse Finished units in warehouse Finished units shipped Efficiency of outbound supply chain Sales to market Demand satisfaction Order Fulfillment Sales to market + Lost sales effectiveness Net profit Amount of each sales Net Profit Margin C ' dollar left after all ompany revenue . expenses have been pard Ability to operate supply SCM Contribution SCM budget — SCM expenses chain efficiently from end- to-end The first performance variable is the number of weeks of raw material available in the warehouse. Excessive raw material inventory is one indicator of problems in the supply chain. This safety stock incurs unnecessary expenses for inventory carrying costs and variable warehouse costs. The most likely causes of inflated raw material storage is a poorly managed inbound transportation system, a struggling manufacturing plant, or a miscalculation of the anticipated demand for finished goods (Martin and Stanford, 2007). The percentage of production completed metric compares the scheduled production with the actual production. This ratio directly reports the ability of the team to execute its production plan and indirectly reflects the efficiency of the supply chain. In this simulation, there are only two reasons for the scheduled production to be greater than 58 the actual production. The first is if insufficient labor exists to manufacture the quantity scheduled. The second is a shortage of raw material to complete the product. In either case, the manufacturing plant is incurring expenses by planning a higher level of production than it can complete. This metric is reflective of difficulties in forecasting operational requirements with operational goals (Robinson, Sabin, and Gao, 2008; Joshi, 1982). The third metric is reflective of the first. The lower number of weeks of finished goods in the warehouse, the more efficient the supply chain performs. High levels of finished goods also incur greater amounts of inventory carrying costs and variable warehouse costs. The most likely cause of inflated finished goods is overestimation of the demand for the product (White, 1996; Gunasekaran, Patel, and Tirtiroglu, 2001). It is possible for situations of variable demand where the peaks exceed operational capability that the safety stock could be mistaken for poor performance in this metric. However, in the design of the scenario for the SCODE users, the plant production capacity exceeds all demand throughout the simulation. The order fulfillment metric records the percentage of total demand that is met each week. At first glance, this appears to mirror the previous metric. However, there are a number of settings where order fulfillment could be low despite high numbers of finished goods available in the warehouse. Since there are multiple products offered in the marketplace, a team could have high quantities of premium products in the warehouse, but insufficient regular products to satisfy demand. Additionally, the teams could have the finished goods in a warehouse serving different markets from those with unfulfilled demand. The ability to meet customer demands is a central metric throughout 59 management and business literature (Kaplan and Norton, 2000; Wacker, 1996; Slater, Olson, and Reddy, 1997). The final two metrics are financial performance measures. The first is the supply chain management contributiOn to the margin. This value represents the amount of the supply chain budget that is returned to the firm. The second is the net profit margin. The net profit margin represents the money remaining after all expenses are deducted. Therefore, these two values represent the efficiency and effectiveness of the entire supply chain (Blasberg, Vishwanath, and Allen, 2008; Bendoly, Rosenzweig, and Stratman, 2007) 3. 4 Perceptual Measures An important portion of this experiment is to measure the supply chain manager’s perceptions within SCODE. Following the final round of the scenario, the computers utilized for the SCODE training are turned off and memory sticks containing the input and output data are collected. While the data are being collected, each individual is provided a survey (Appendix D) to report their experience. The survey contains two distinct parts. The first contains open-ended questions utilized to facilitate nominal group technique discussion at the end of the session. The second is adapted from previous works in the information technology arena. The open-ended questions in the first part of the survey elicit group discussion and consensus. Nominal group technique is a brainstorming method encouraging input by all members of a group. By recording the written responses of all participants, trends become apparent and foster interaction among the individuals. By waiting until after the 60 experiment and questionnaires are complete for the group discussion, the researcher is able to reveal the different levels of feedback data and accuracy of the decision information system to the entire group, thus facilitating a rich discussion about the required information to effectively make the next week’s operational decisions. This group discussion is documented through recording information on a white board and on the individual surveys. The experimental session is concluded following the group discussion. The perceptual measures recorded in the second part of the survey capture the individual’s impression of the experiment. These items are derived from Speier (1996), Bailey and Pearson (1983), and Spurrier, Topi, and Valacich (1994). The questions focus on both the information qualities as well as the user’s confidence in his/her performance using the information. This portion of the survey captures feedback essential to determining the usefulness of the condensed data versus the uncondensed data, the perception of reliability of the decision information system, and the user’s ability to grasp the information and utilize it effectively. The constructs, specific questions comprising the construct, and derivation source are presented in Table 3.3. 61 Table 3.3. Survey Constructs Constructs Questions Derived from 25:22:;22 7A 7327“: 7D Bailey and Pearson (1983) Information Precision 8A 8B 8C 8D Bailey and Pearson (1983) [aggrllfigfyn 9A 9B 9C 9D Bailey and Pearson (1983) Conigfzhrengiisgness 10A 10B2210C 10D Bailey and Pearson (1983) Information Format 11A 11B 11C 11D Bailey and Pearson (1983) 13:31:22: 12A 13133 1226C 12D Bailey and Pearson (1983) Confidence 61‘8‘13613166C176D 81:11:11;ng IP99?)1d Rational Approach 15 20 25 $3121.12: at? €113 91,9?)1d Intuitive Approach 18 21 85:12:; a: €11) 91,923“! 13:22:18,123: 14281929223024 Speier (1996) 62 3.5 Data Analysis Data analysis occurs in three parts. The first is multivariate analysis of variance (MANOVA) of the weekly decisions. The second phase is analysis of variance (ANOVA) for the scaled survey questions. Third, the short-answer and group discussion information are examined by post-hoc analysis for trends. Statistical analysis of results divided by executive or MBA status captures and controls for any potential differences between the groups. MANOVA enables analysis of variance for multiple dependent variables. The dependent variables in this study include the weekly output variables provided at each SCODE week’s end. Since there is no single indicator of a “health of the supply chain” available in SCODE, evaluation of variables is used to identify whether the supply chain is operating efficiently and effectively. Table 3.2 summarizes the six dependent variables evaluated in Phase 1 of data analysis. ANOVA is performed on questions from the survey. By retaining group identity in the analysis of these responses, the four clusters representative of condensed / corrected, condensed / uncorrected, uncondensed / corrected, and uncondensed/ uncorrected are compared for statistical differences from and similarities to the entire population. Comparing the variance between the groups with the variance within each group will provide insight to the perceived difference in information availability and accuracy. The post-hoc analysis captured the remaining experimental data in a holistic perception of the four groups. The richness of information provided by open-ended 63 questions and group discussion significantly augments the quantitative data collected throughout the experiment. The detailed analysis of the data follows in Chapter 4. 64 CHAPTER IV: RESULTS 4.0 Introduction This chapter presents the results of the study. The first section introduces the experimental population and reveals the sample size, demographics, and assignment to experimental groups. The next sections explore the statistical methods for analysis, the assumptions required to utilize these methods, and the verification checks within the sample data sets. After ensuring the data fits the assumptions for the methods, the hypotheses are tested and post-hoe analysis highlights further stratification of the variables under study. 4.1 Experimental Population Descriptive Data The subjects in this study are currently enrolled in a MBA program at Michigan State University. The students represent three variants of the MBA options offered through Michigan State University. One session included traditional MBA (F-TMBA) students enrolled full-time in courses. Another session involved Executive MBA (EMBA) students attending courses in the evening. The final two sessions enrolled Weekend MBA (WMBAl and WMBA2) students who attend courses on the weekend. All participants completed at least one course in supply chain management prior to inclusion in this study. A total of 172 subjects participated in this study. Twelve of the subjects were excluded from analysis due to various causes. Two participants failed to complete enough weeks of SCODE to provide meaningful results. Six subjects did not return a questionnaire to complete the required data collection. Two subjects did not properly 65 identify his/her group number on the survey for identification with the computer simulation. The final two subjects were excluded because he/she did not submit the correct files from the computer simulation for study. The division of participants by session is in Table 4.1. Table 4.1. Participants by Session Session Number of Participants Percent Executive MBA , 42 ,, , 26.3 Full-Time MBA 29 18.1 Weekend MBA 1 42 26.3 Weekend MBA 2 47 29.4 Total 160 100.0 The included 160 subjects were randomly assigned to one of the four treatments within the experiment. The division of subjects by session is presented in Table 4.2. Table 4.2. Assignment of Treatment by Session Group Assignment EMBA F-T MBA- WMBA1 WMBAZ Frequency Percent Uncondensed / Corrected _9_ _ 6 15 8 _ 38 23.8 Uncondensed / Uncorrected 13 1 1 8 12 44 27.5 Condensed / Correct 12 5 7 11 35 21.9 Condensed / Uncorrected 8 7 12 16 43 26.9 Total 42 29 42 47 160 100.0 Chi-square tests investigated possible differences among group assignment and format of MBA education. The chi-square statistic was 9.82 (d.f. = 9) with a p-value of 66 .365. This indicates no significant differences among the treatments based the population’s method of MBA education. Of the 160 subjects in the study, 48 females and 112 males are included. The treatment by gender appears in Table 4.2. As with the MBA format, chi-square tests (1.99, d.f. = 3, p-value = .575) revealed no significant differences among the treatments based upon the gender of the participants within each group. Table 4.3. Assignment of Treatment by Gender Group Assignment Female Male Frequency Percent Uncondensed / Corrected 9 29 38 23.8 Uncondensed / Uncorrected 16 28 44 27. 5 Condensed / Correct 13 22 35 21.9 Condensed / Uncorrected 10 33 43 26.9 Total 48 112 160 100.0 Participants represented a wide variety of backgrounds. The self-reported fields of the survey included questions about the current position, years of supply chain experience, and highest degree earned to date. The diverse job positions represented in Table 4.4 are then clarified to show a more equal diversification of job-related experience among the participants. Table 4.5 condenses the current position to supervisory levels within the participant’s respective organization. The results follow in Tables 4.4 and 4.5. The chi-square tests for supervisory position within the treatments resulted in a test statistic of 6.76 (d.f. = 12, p-value = .873). 67 Table 4.4. Participant Employment Position Participant Position Frequency Percent Accountant 4 ' 2.5 Analyst 11 6.9 Associate 1 0.6 Banker 1 0.6 Buyer 1 0.6 CEO 1 0.6 CIO 2 1.3 Consultant 3 1.9 Director 6 3.8 Engineer 22 13.8 Finance 1 0.6 Leader 71 7 g 0.6 Management 47 29.4 Marketing fl ”1 7 70.6 7 President 1 0.6 Sales 4 2.5 Specialist 1 1 6.9 Student 37 23.1 Supervisor 1 0.6 VP 4 2.5 Total 160 100.0 Table 4.5. Participant Employment Level Participant Position By Supervisory Level Frequency Percent Senior Management 14 8.1 Management 49 30.6 Employee 60 37.5 . Student 37 23.1 Total 160 100.0 68 The self-reported experience level of participants varied greatly. The vast majority of subjects, 65.6%, reported no supply chain management experience. The division of subjects by experience across the treatment groups again reflects no statistical differences (chi-square statistic = 35.21, d.f. = 33, p-value = .364). The results are summarized in Table 4.6. Table 4.6. Participant Years of Supply Chain Experience Participant Years of Supply Chain Experience Frequency Percent 0 105 65.6 1 11 6.9 2‘ ,, 8 7 5.0 3 7 4.4 4 4 2.5 5 10 6.3 6 2 1.3 7 2 1.3 8 5 3.1 10 2 1.3 ‘ 15 3 1.9 20 1 0.6 Total 160 100.0 The final report of demographic information represents the highest degree earned by the participants. The chi-square statistic of 6.13 with d.f. = 9 and a p-value of .727 indicates no statistical difference among the treatments based upon educational level attained to date. 69 Table 4.7. Participant Years of Supply Chain Experience Participant Highest Degree Earned Frequency Percent High School 5 3.1 Bachelor's 87 g 54.4 Master's 65 40.6 Doctoral 3 1.9 Total 160 100.0 4.2 Statistical Methods and Tests of Assumptions MANOVA can be a powerful tool to compare groups created through assignment of different treatments to different groups. There are five assumptions for MANOVA results to be accurate. The first is that observations must be statistically independent from each other. Second, the dependent variables should follow a multivariate normal distribution. Third, the variance-covariance matrices of the dependent variable should be equal for all treatment groups. Fourth, the dependent variables must exhibit linearity and multicollinearity. Fifth, there should not be significant outliers in the population. In the case of this study, each observation is unique to the participant using the SCODE simulation. This is one of the more stringent assumptions for MANOVA. Typical violations of independence occur if repeated measures are taken for a single participant, as in a time-series study, or if a subset of the population is likely to have similar traits that are distinct from other members. As shown in the demographic tables in the first section of this chapter, the participants are in similar status within their MBA programs and each treatment represents a statistically uniform cross-section of the population based on gender, type of MBA program, and supervisory level. Therefore, independence of observations is met. 70 Multivariate normality refers to the assumption that the joint effect of two variables results in a normal distribution. For the case of MANOVA with large sample sizes, violations of this assumption have minimal impact on the effectiveness of the tests (Hair et al, 1998). Concern is warranted if non-normality is caused by outliers rather than skewness. The skewness and kurtosis statistics (Table 4.8) as well as frequency histograms (Figure 4.1) illustrate the variance from normality within the dependent variables. Table 4.8. Skewness and Kurtosis Statistics Skewness and Kurtosis of Dependent Variables Inbound Internal Outbound Market Profit SC Contrib Skewness 1.318 -1 .245 2.276 -0.586 -2.461 -1.581 Std. Error of Skewness 0.192 0.192 0.192 0.192 0.192 0.192 Kurtosis 0.886 0.331 4.936 -0.685 ,. 7 7.891 9.3047 Std. Error of Kurtosis 0.381 0.381 0.381 0.381 0.381 0.381 The third assumption of MANOVA is the equality of variance-covariance matrices. The assumption addresses substantial differences in the degree of variance within a single group compared to the others. As with the multivariate normality assumption, MANOVA is robust to violations of the variance-covariance matrices equality for equal sample sizes. For these data, Levene’s Test of homogeneity of variances returns significant results for all dependent variables at p<.05. Additionally, Box’s M-test results (Box’s M = 317.8, d.f. 1 = 63, d.f. 2 = 53,614, p < .001) are significant indicating the covariance matrices of the dependent variables are not equal across all treatment groups. The frequency histograms in Figure 4.1 graphically depict the absence of significant outliers from the population. This satisfies the fifth MANOVA assumption. 7] Figure 4.1. Frequency Histograms for the Dependent Variables Frequency l 0.0 2.0 4.0 6.0 Inbound 1005 80- a c 60' e :l e- 2 u. 40- 20— .. r": V I I I I T o 20 40 60 60 Internal 72 Figure 4.1 (cont’d). w- 60— Frequency 40... v I I I T l 2 4 6 3 Outbound fro-1 @— 6 r: O 3 6' 20- O L- ll. 10- 3 r o o Figure 4.1 (cont’d). Frequency Frequency I I 0 4mm 5000000 I SC Contrib 74 l 50111300 1 10000000 The final assumption of MANOVA is the linearity and multicollinearity among dependent variables. For general linear hypotheses models to correctly predict relationships among the variables, it is important that the variables have a linear association with other variables. Bartlett’s test of sphericity is used to evaluate the significance of intercorrelation among the dependent variables. The result indicates that significant intercorrelations do exist within the variables. Bartlett’s factor of 196.37 with d.f. = 15 yields a p < .001 for the experimental results. As a result of the assumption testing in this section, MANOVA is an appropriate method to evaluate the effect the treatment had on the subject’s ability to perform in a supply chain environment. 4.3 Two-way Overall Results This section progresses from the tests of the data ensuring assumptions of the statistical tools to analysis of the results from the experimental sessions. The four subject treatment groups are based on the manipulation of two variables: corrected information and condensed data. Each of the experimental variables has two levels as discussed in Chapter 3. The participant’s performance was evaluated on six distinct metrics measuring the effectiveness of various aspects of the supply chain. The six metrics include the efficiency of the inbound supply chain (Inbound), the effectiveness of manufacturing operations (Internal), the efficiency of the outbound supply chain (Outbound), the market satisfaction level (Market), net profit margin (Profit) for the entire operation, and the supply chain contribution to the overall profit (SCContrib). A 75 seventh overall metric utilized a standardized calculation of performance across all six metrics to determine overall supply chain performance by utilizing a weighted z-score. Table 4.9 provides more detailed description of the measurement method and the interpretation of each of the variables. Table 4.9. Performance Variable Scoring Performance Variable Measurement Method Interpretation Weeks of raw material in Inbound warehouse based on current Lower (zero) is better production requirement Percentage of production Internal scheduled for current week that Higher (100%) is better was actually completed. Weeks of finished goods in Outbound warehouse base on current Lower (zero) is better market demand requirement. Market Percentage .Of demand met in Higher (100%) is better prevrous week. Net profit margin from the Profit beginning of the simulation to Higher (100%) is better the ending state. 50% of the firm’s revenue is allocated for supply chain expenses. This measures the amount under budget (positive) or over budget (negative) that the supply chain returns. SCContrib Higher is better Equally weighted scaled combination of the above six metrics. Provides an overall . . . H1 r1 tter performance metric based on ghe S be deviation from the average scores of each metric. Overall 76 A final overall supply chain score (Overall) is computed based upon the equally weighted sum of the deviation from average values of each of the six metrics. This seventh metric provides an overall performance measure for the behavior of the entire supply chain over the simulation and provides an accurate gauge for supply chain behaviors beyond the experimental setting. For example, participants identified a supply chain goal at the beginning of the session. Most (55.0%) selected “Meet all demand” as the goal of their supply chain. This approach would be expected to earn the highest profits by meeting a higher percentage of demand, but meeting greater challenge with managing the inbound, internal, and outbound supply chain performance which could lead to lower supply chain contributions. Others created a more focused or efficient approach that might sacrifice profits for simplicity. It is expected that those groups may return a lower profit margin, but should have less difficulty maintaining an efficient inbound and outbound flow of goods and therefore returning greater supply chain contributions. Because of these trade-offs inherent in operating a supply chain, there is merit to comparing the performance of the subjects under different treatments with the six performance variables as well as separately with an overall combined supply chain variable. The participant’s supply chain goals were spread statistically equally across the treatments (chi-square value = 22.98, d.f. = 15, p = .085). 77 Table 4.10. Participant’s Supply Chain Goals Supply Chain Goal Frequency Percent All 2 1.3 Demand 88 55.0 Efficiency 34 I 21 .3 Focused 22 13.8 None 1 0.6 Profit 13 8.1 Total 160 100.0 Analysis of the data was performed utilizing the SPSS software package, version 15.0, with conversion to Excel for presentation of tables and some figures. There are four criteria presented for testing multivariate differences between groups. The criteria include Pillai’s Trace, Wilks’ Lambda, Hotelling’s Trace, and Roy’s Largest Root. Pillai’s Trace is the sum of variance that can be explained through calculation of discriminant variables and is robust in conditions where MANOVA assumptions have not been met. Wilks’ Lambda determines whether there is an overall difference between the levels of the independent variable relative to the dependent variables when considered as a whole. Wilks’ Lambda is also generally utilized when there are more than two dependent groups under study. Hotelling’s Trace is more commonly used for two dependent groups and includes the pooled ratio of effect variance to error variance. Roy’s Largest Root is generally most sensitive if population centers differ along a single dimension and is least sensitive otherwise. However, in cases where sample sizes are large, all four of these become asymptotically equivalent (Hair et al, 1998). As can be seen in Table 4.11, all return significant (p < .001) results for the difference in performance variables among the four experimental treatment levels. 78 Table 4.11. Multivariate Test Results for Treatment Levels Multivariate Test Results F Hyp df Error (If Sig. Part Eta Sq Power(a) Pillai's Trace 2.675 18 459 <0.001 0.095 0.998 Wilks' Lambda 2.726 18 428 <0.001 0.097 0.997 Hotelling's Trace 2.767 18 449 <0.001 0.100 0.998 Roy's Largest Root 5.847 6 153 <0.001 0.187 0.997 Computed using alpha = .05 The additional columns of the table include the Partial Eta Squared and Observed Power of the test. The Partial Eta Squared statistic measures effect size by identifying the proportion of total variability in the dependent variables that are accounted for by variation of the independent variable. The possible values range from 0, no effect, to 1, a very strong effect (Cohen, 1988). The Observed Power presents the ability of the test to reject the null hypothesis when false. Power in excess of .9 represents a strong test (Cohen, 1988). The multivariate test results in Table 4.10 represent the highest level of analysis. The significance of the multivariate tests indicate a statistical difference in performance across the six dependent variables based on the four different treatments of corrected or uncorrected information and condensed or uncondensed data. To better understand the effects within each of the treatment groups, additional analysis is required. The next step in analysis involves evaluating the between-subjects effects. This series of tests identifies the specific dependent variables affected by treatment level. As can be seen in Table 4.12, four of the seven metrics show significant differences in performance based upon 79 treatment. The inbound and internal metrics are significant at p < .01 level. The profit and overall metrics indicated significance at p < .05 level. Table 4.12. Tests of Between-Subject Effects for Treatment Levels Tests of Between-Subjects Effects Dependent Variable F at Sig. Part Eta Sq Power(a) Inbound 4.409 3 0.005 *** 0.078 0.868 Internal 4.377 3 0.005 *** 0.078 0.865 Outbound 1.626 3 0.185 7 0,030 7 0.421 Market 1.670 3 0.176 . 0.031 0.431 Profit 2.844 3 . 0040?: 7 70.052 0.673 SC Contrib 1.769 3 0.155 0.033 0.455 Overall 3.720 3 0.013 ** 0.067 0.799 a Computed using alpha = .05 For these four dependent variables with significant differences at the treatment level, the mean value of each metric by treatment is graphed and further analyzed to identify which specific treatments are statistically different from others. Figure 4.2 displays the inbound dependent variable by treatment. 80 Figure 4.2. Mean Value of Inbound Dependent Variable by Treatment Inbound Metric 2.5 _______._..__.. ..______.____ . ..___ .----____...___ ......----_-_-.-_...-._....__-._2.-..__-_-_____.___..._..-_.--.___.-._.._....__.----.__.__2; 1.5 Weeks of Raw Material In Warehouse Uncondensed / Uncondensed/ Condensed I Corrected Condensed / Corrected Uncorrected Uncorrected From visual examination, Figure 4.2 shows a substantial difference between the uncondensed / corrected treatment and the condensed / corrected-treatment. To validate this statistically, the Tukey HSD test is used. The Tukey HSD test performs a series of pair-wise comparisons of the mean values of each treatment. For the inbound dependent variable, the results are presented in Table 4.13. The Tukey evaluation confirms the visual indication from Figure 4.2, that the mean values are statistically different for the uncondensed / corrected treatment and the condensed / corrected treatment. In addition, the Tukey HSD test confirms the uncondensed / corrected treatment returns significantly different performance on the inbound metric than the condensed / uncorrected treatment. This series of findings indicates both uncondensed data and corrected information are 81 efficient management of the inbound supply chain. necessary to statistically improve the performance of the ordering, inventory levels, and Table 4.13. Tukey HSD Test for Inbound Dependent Variable by Treatment Tukey I-ISD Multiple Comparisons DV (1) Group (J) Group Sig. Inbound Uncondensed / Corrected Uncondensed / Uncorrected 0.185 Condensed / Corrected 0.005 *“ Condensed / Uncorrected 0.022 ** Uncondensed / Uncorrected Uncondensed / Corrected 0.185 Condensed I Corrected 0.439 Condensed / Uncorrected 0.797 Condensed / Corrected Uncondensed I Corrected 0.005 *** Uncondensed I Uncorrected 0.439 Condensed / Uncorrected 0.922 Condensed / Uncorrected Uncondensed / Corrected 0.022 ** Uncondensed / Uncorrected 0.797 Condensed / Corrected 0.922 Next, the internal metric is analyzed in greater detail. Figure 4.3 provides the 82 graphical depiction of the values of the dependent variable at different treatment levels. Figure 4.3. Mean Value of Internal Dependent Variable by Treatment Internal Production Metric 90 l 80 70 60 50 40 30 20 Percentage of Production Completed 10 Uncondensed / Uncondensed/ Condensed I Corrected Condensed I Corrected Uncorrected Uncorrected As with the inbound metric, the internal production metric appears to differ between the uncondensed / corrected treatment and both the condensed / corrected and the condensed / uncorrected treatments. Table 4.14 confirms both visual findings. The test indicates that a firm would need to expand the level of detail of the data and improve the quality of the information in order to make a significant improvement in performance for scheduling labor and raw material within the production facility. 83 Table 4.14. Tukey HSD Test for Internal Dependent Variable by Treatment Tukey HSD Multiple Comparisons DV (1) Group (J) Group Sig. Internal Uncondensed / Corrected Uncondensed / Uncorrected 0.528 Condensed / Corrected 0.026 ** Condensed ’/ Uncorrected 0.008 *** Uncondensed / Uncorrected Uncondensed / Corrected 0.528 Condensed / Corrected 0.382 Condensed / Uncorrected 0.220 Condensed / Corrected Uncondensed / Corrected 0.026 ** Uncondensed / Uncorrected 0.382 Condensed / Uncorrected 0.996 Condensed / Uncorrected Uncondensed I Corrected 0.008 *** Uncondensed / Uncorrected 0.220 Condensed / Corrected 0.996 The profit dependent variable showed significant values in Table 4.11. The graph of the mean value of the profit metric is presented in Figure 4.4. For this metric, the figure demonstrates the uncondensed / uncorrected and condensed / corrected treatments have mean values the furthest apart. 84 Figure 4.4. Mean Value of Profit Dependent Variable by Treatment Profit Metric 50 » 45 4o 35 3o 25 20 Net Profit Margin 15 10 Uncondensed / Uncondensed I Condensed I Corrected Condensed / Corrected Uncorrected Uncorrected Performing the Tukey HSD test for the profit dependent variable across all treatment levels returns significant differences between the uncondensed / uncorrected and the condensed / corrected treatments. This finding is interesting because it provides some evidence that a firm may be able to increase performance level for net profit margin by providing supply chain managers with a less detailed level of data as long as the information system provided accurate decision making metrics. 85 Table 4.15. Tukey HSD Test for Profit Dependent Variable by Treatment Tukey HSD Multiple Comparisons DV (I) Group (J) Group Sig. Profit Uncondensed / Corrected Uncondensed / Uncorrected 0.125 Condensed ‘/ Corrected 0.954 Condensed / Uncorrected 0.924 Uncondensed / Uncorrected Uncondensed / Corrected 0.125 Condensed I Corrected 0.037 ** Condensed / Uncorrected 0.361 Condensed / Corrected Uncondensed / Corrected 0.954 Uncondensed / Uncorrected 0.037 ** Condensed / Uncorrected 0.661 Condensed / Uncorrected Uncondensed / Corrected 0.924 Uncondensed I Uncorrected 0.361 Condensed / Corrected 0.661 The overall combined performance metric also was significant in between-subject effect testing. Figure 4.5 illustrates the overall scaled performance across all treatments. As with the first two metrics, the overall dependent variable shows uncondensed/ corrected and condensed / uncorrected as the treatments with the greatest difference in means. 86 Figure 4.5. Mean Value of Overall Dependent Variable by Treatment Overall Performance Metric 0.8 0.6 0.4 0.2 Scaled Performance Uncondensed / Uncondensed / Condensed I Corrected Condensed / Corrected Uncorrected Uncorrected Further analysis using the Tukey HSD test confirms the uncondensed / corrected and condensed / uncorrected treatments have a statistically significant difference in means. There is no surprise that the largest difference in overall performance would be between the treatment possessing more extensive data availability and accurate information feedback systems and the treatment possessing less data and inaccurate information systems. However, the lack of statistical difference of the other two treatments is surprising. This series of discoveries warrants further investigation through exploring each of the two factors separately. This analysis is covered in Section 4.4. 87 Table 4.16. Tukey HSD Test for Overall Dependent Variable by Treatment Tukey HSD Multiple Comparisons DV (1) Group (J) Group Sig.__ Overall Uncondensed / Corrected Uncondensed / Uncorrected 0.106 Condensed/ Corrected 0.344 Condensed / Uncorrected 0.007 *** Uncondensed / Uncorrected Uncondensed / Corrected 0.106 Condensed / Corrected 0.957 Condensed I Uncorrected 0.726 Condensed / Corrected (Uncondensed / Corrected fig 0.344 7 J Uncondensed / Uncorrected 0.957 Condensed I Uncorrected 0.449 Condensed / Uncorrected Uncondensed / Corrected 0.007 *** Uncondensed / Uncorrected 0.726 Condensed / Corrected 0.449 4.4 Main Effect Results Section 4.3 provided insight into the effects the combined treatment had on the seven performance variables. This section focuses on each of the treatment variables separately to evaluate the specific effects that condensed / uncondensed data and corrected / uncorrected information systems have upon the seven dependent variables. As with the two-way interactions, the analysis begins with the multivariate tests. Table 4.17 provides the multivariate test results for the main effects. The condensed / uncondensed treatment reports significant differences across the seven performance metrics. The corrected / uncorrected effect does not show significant impact across all of the metrics simultaneously. However, further investigation reveals significant effect within certain metrics as will be investigated later in this section. Due to the sample size and degrees of freedom of the test, the four multivariate tests asymptotically approach the same values. 88 Table 4.17. Multivariate Test Results for Main Effects Multivariate Tests c) IEffect F Hyp of Error df Sig Part Eta Sq Power(a) Corrected I Pillai's Trace 1.589 6 151 0.154 0.059 0.598 Uncorrected Wilks' Lambda 1 .589 6 151 0.154 0.059 0.598 Hotelling's Trace 1 .589 6 '151 0.154 0.059 0.598 RoYs flesf Root 1.589 6 151 0.154 0.059 0.598 Condensed I Pillai's Trace 5.344 6 151 <0.001 0.175 0.995 Uncondensed Wilks' Lambda 5.344 6 151 <0.001 0.175 0.995 Hotelling's Trace 5.344 6 151 <0.001 0.175 0.995 Roy's Largest Root 5.344 6 151 <0.001 0.175 0.995 a. Computed using alpha = .05 Further analysis through tests of between-subject effects provides additional clarification to the significant differences in dependent variables for each of the main effects. Figure 4.18 presents these test results. Table 4.18. Tests of Between-Subject Effects for Main Effects Tests of Between-Sub'ects Effects Source Dependent Variable F df Sig. Part Eta Sq Power(a) Corrected I Inbound 0.912 1 0.341 0.006 0.158 Uncorrected Internal 1.230 1 0.269 0.008 0.197 Outbound 3.622 1 0.059 0.023 0.473 Market 0.009 1 0.926 0.000 0.051 Profit 5.591 1 0.019 ** 0.035 0.652 SC Contrib 2.387 1 0.124 0.015 0.336 Overall 7.061 1 0.009 *** 0.043 0.752 Condensed I Inbound 9585 1 0.002 *** g 0.058 g 0.868 Uncondensed Internal 11.500 1 0.001 *** 0.069 0.921 Outbound 0.592 1 0.443 0.004 0.119 Market 2.534 1 0.113 0.016 0.353 Profit 2.221 1 0.138 0.014 0.316 SC Contrib 0.127 1 0.722 0.001 0.065 Overall 3.734 1 0.055 0.023 0.484 a Computed using alpha = .05 89 The two main variables show very different effects on the dependent variables. The corrected / uncorrected variable creates significant differences in the outbound, profit, and overall metrics. The condensed / uncondensed metric produces significantly different results in the inbound, internal, and overall metrics. Reviewing the mean values of each group on the same graph better illustrates the relationships. Figures 4.6 — 4.10 graphically depict the dependent variables with one or both main effects showing significant differences in the mean values of the results. Figure 4.6. Mean Value of Inbound Dependent Variable by Main Effect Inbound Metric 2.5 ,9 , , , . vvvvvv 0 § 2 2 f 3 E 1.5 (I t 3 ‘9‘ 3 1 II I! "6 in '§ 0.5 3 o No Yes +Condensed? - I- Corrected? The graph illustrates the significant difference with the condensed-yes versus the condensed-no (uncondensed) weeks of raw material in the warehouse. The accuracy of 90 the information system had little effect on the performance by itself. However, the participants receiving condensed data for ordering raw materials were able to Operate the supply chain more efficiently on the inbound side of the process. By reducing the extraneous data available to the uncondensed users, the condensed users performed at a higher level on this metric. Figure 4.7. Mean Value of Internal Dependent Variable by Main Effect Internal Production Metric 90 G) (II (D O \l 0" Percentage of Production Completed V O 65 No Yes +Condensed? - I- Corrected? The internal production metric shows a significant difference between the condensed / uncondensed treatment while the corrected / uncorrected treatment again shows no statistically significant difference in means. As in the raw material ordering and storage metric, the removal of extraneous data should allow the subjects to focus 9] attention on the critical data needed for decision making regardless of the accuracy of the management information system. The average participant in the condensed data treatment completed over 88% of the scheduled production as compared to under 75% for the average user in the uncondensed data treatment group. Figure 4.8. Mean Value of Outbound Dependent Variable by Main Effect Outbound Metric 12 0.8 is. 0.6 0.4 Weeks of Finished Goods In Warehouse No Yes .—e—Condensed? - I- Corrected? The graph of the outbound metric reflects the opposite trend as seen in the last two figures. The corrected / uncorrected treatment groups performed statistically differently within the outbound metric while the condensed / uncondensed treatment groups did not. The participants with corrected information in their management 92 dashboard performed statistically better at managing their finished goods inventories and matching production with demand. Figure 4.9. Mean Value of Profit Dependent Variable by Main Effect Profit Metric 40 Net Profit Margin N N 00 O 01 O ..s 01 10 No Yes +Condensed? - l- Corrected? The graphical depiction of the profit metric also shows a greater difference in the mean values of the corrected / uncorrected treatment than the condensed / uncondensed treatment. The participants in the group with corrected information on the management dashboard performed significantly better in terms of net profit margin than those with uncorrected information. 93 Figure 4.10. Mean Value of Overall Dependent Variable by Main Effect Overall Performance Metrics 0.5 0.4 0.3 0.2 0.1 -0.1 Scaled Performance -0.2 -0.3 -0.4 No Yes .—e—Condensed? - I- Corrected? For the overall supply chain performance metric, the condensed / uncondensed treatment group and the corrected / uncorrected treatment group performed significantly differently. The participants with uncondensed data recorded an overall higher score across all six metrics than the participants with condensed data. Further, subjects with corrected information on the management dashboard performed better across all six metrics than the subjects with uncorrected information. The study of main effects individually provides some greater insight into the behavior of the participants within each treatment group. The last analysis required prior to evaluating the hypotheses involves evaluating the written survey and nominal group technique discussion at the conclusion of each experimental session. 94 4.5 Survey and Nominal Group Technique Results The survey and nominal group technique portion of the session provided additional insight into the perception of the subjects and revealed additional significance to the numerical analysis presented in previous sections of this chapter. By understanding more about the decision making process of supply chain leaders, the argument for condensed versus uncondensed data and corrected versus uncorrected information becomes less of a binary decision and more of an understanding of the balance of the two variables along with complexity and cost of the decision information system. 4. 5.] Survey Results As presented in Table 3.3, the survey items werederived from constructs established in previous studies (Speier, 1996; Bailey and Pearson, 1983; Spurrier, Topi, and Valacich, 1994). Analysis of the survey results followed the same pattern as the simulation results. Namely, the surveys were first evaluated at the overall treatment level. Table 4.19 presents the significant survey items based upon ANOVA at the treatment group level. The constructs Information Correctness, Information Precision, Information Reliability, and Amount of Information are found in the significant list. 95 Table 4.19. Survey Significant Items at Treatment Level Treatment Level Effects W Mean Constructs Squares df Square F 8L Confidence - 1.477 3 g 0.492 0745 g -, g 0.527 Information Conectness 13.312 3 4.437 3.437 0.018“ Information Precision 17.127 3 5.709 3.442 ., 0.0138" lnfonnation Reliability 12.159 3 4.053 2.453 0.065 Infomiation Comprehensiveness 3.370 3 1.123 0.977 0.405 lnfonnation Format 2.542 3 0.847 0.480 0.696 Information Usefulness 4 4.021 g 3, 1.340 ,. 1.604 0.191 g Rational Approach 4.153 3 1 .384 1 .848 0.141 Intuitive Approach 2.515 3 0.838 0.828 -. 0.481 Amount of Information 1.891 3 0.630 1.003 0.393 Further evaluation of the significant items through Tukey HSD test reveals the specific pair or pairs of treatments with significant differences. Figure 4.11 presents these findings in a 4x4 matrix to facilitate comparison of the results. The corrected / condensed and uncorrected / condensed treatment groups produced statistically different responses to ten survey items. All of these questions focused on information correctness, information reliability, and quantity of information. The number of significant results from this pairing was expected due to the treatment providing the subjects with less data on the Output Report and Financial Report. Therefore, the participants were more reliant on the dashboard to make weekly operational decisions. For the groups with the corrected dashboard, the absence of detailed data was less critical than for the groups with an uncorrected dashboard. Further, for each of the ten questions returning significant differences, the corrected /condensed treatment groups had a more favorable impression of the accuracy, sufficiency, consistency, and precision of the dashboard. At the same time, the corrected / condensed 96 treatment groups expressed less concern with the quantity of data available on the Output Report and Financial report. The corrected / condensed treatment group also reported statistically different survey responses on three questions from the corrected / uncondensed treatment group. These three questions were subsets of the information precision and amount of information constructs. As with the previous pairing, the corrected / condensed treatment group expressed higher satisfaction with the precision and less concern about the quantity of data presented in the session. The corrected / condensed treatment group had two additional significantly different responses from the uncorrected / uncondensed treatment group. These two questions reflected statistically different perceptions about the reliability of the information presented in the dashboard as well as the quantity of data presented in the Output Report and Financial Report. The uncorrected / uncondensed treatment group found the reliability of the information presented on the dashboard to be less sufficient and the quantity of data presented to be more than he or she could use to make weekly operational decisions. The final statistically significant difference within the questions results from the corrected / uncondensed and the uncorrected / uncondensed treatment groups’ divergent perceptions of the confusion caused by the quantity of data presented in the Output Report and Financial Report. This difference was likely caused by the corrected / uncondensed treatment group relying on the dashboard for weekly decisions. 97 Figure 4.11. Tukey HSD Significant Results by Treatment Level Post-Hoe Test Sigiificance Summary (Tukey HSD) Corrected I Condensed “mm” Precision Uncorrected I Correctness Condensed Precision Corrected I Uncondensed Uncorrected I Uncondensed Corrected I Uncorrected I Corrected l Uncorrected I Condensed Condensed Uncondensed Uncondensed The final analysis of the survey data involved reviewing the two treatments separately to reveal distinctions within the main effects. Table 4.20 presents this group of significant results. As can be seen in the table, the corrected / uncorrected variable provided significantly different responses on 22 of the questions in three of the constructs while the condensed / uncondensed variable only provided a single difference in response. One possible reason for the disparity is the focus of the questions on the behavior of the dashboard. For the corrected / uncorrected variable, the range of constructs returning statistically significantly differences in the question responses included information correctness and information precision. As presented in the overall treatment analysis, the direction of the difference was the same in each case. The mean values of the participants in the corrected treatment groups indicated higher satisfaction with the information presented in the dashboard and less concern about the quantity of data presented in the 98 Output Report and the Finance Report. The corrected group means reflected a more rational approach and less intuitive approach to their decision making process. Of additional significance to the findings, the users were able to detect inaccuracies in the information provided to them with the uncorrected information management systems. Table 4.20. Tests of Between-Subjects Effects for Independent Variables Tests of Between-Subjects Effects Type In SumW mean Constructs Squares df Square F 8L Corrected / Confidence 0.565 1 7 0.5675A7 0.858 0.356 Uncorrected Information Correctness 7.559 1 7.559 7 5.765 0017“ Information Precision 6.800 1 6.800 3.993 0.047” Information Reliability 7.942 1 7.942 4.790 0.030” lnforrnation Comprehensiveness77 7 3.137 1 3.137 7 7 772.760 77 0.099 Information Format 2.387 1 2.387 1.370 707.244 Information Usefulness 2.447 1 2.447 2.7930 77#9.7089 7 Rational Approach 3.459 1 3.459 4.649 7 0.033 7 Intuitive Approach 0.058 77777 7 1 7 ' 7 0.058 0.057 7 770.812 Amount of Information 0.039 1 0.039 0.062 0.804 Condensed / Confidence 0.729 1 0.729 1.109 0.294 Uncondensed lnforrnation Correctness 0.000 1 0.000 0.000 0.991 Information Precision 1.050 1 1.050 0.603 0.439 Information Reliability 2.263 _ 1 2.263 1.336 0.250 Information Comprehensiveness 0.150 1 0.150 0.130 0.719 Infonnation Format 7 777i 0.181777 7 1 0.181777 0.103 0.748 Information Usefulness 0.590 1 0.590 0.696 0.405 Rational Approach 0.031 1 0.031 0.040 0.841 Intuitive Approach 0.070 1 0.070 0.069 0.794 Amount of lnfonnation 0.534 1 0.534 0.848 0.358 99 4.5.2 Nominal Group Technique Results The experimental sessions concluded with collection of the surveys and discussion among the participants concerning the three open-ended questions at the beginning of the survey. These three questions solicit the subjects to provide the three most important pieces of output data used to make decisions, the three least important pieces of output data, and three items the participants would like to have available to make the operational decisions. Once the surveys have been collected, the groups were encouraged to talk openly about the differences in condensed or uncondensed data in the Output Report and Finance Report as well as the corrected or uncorrected displays of information on the management dashboard. After refinement from the surveys and the discussions, the lists of responses by frequency are presented in Tables 4.21 — 4.23. By overwhelming majority, the participants identified raw material inventory as the most critical item to monitor when making the weekly operational decisions. Based on the global sourcing focus of the scenario, the reliance of the production system on raw material to execute the schedule, and variability in delivery timeliness, the fact that 120 of the 160 participants identified raw material inventory as the most critical item is expected. The next three highest items include monitoring the demand forecast, tracking the finished goods inventory, and the weekly production status. These items provide valuable insight to understanding which variables supply chain managers identify as critical. 100 Table 4.21. Most Important Decision Making Items as Reported by Participants Most Useful Items Uncondensed Uncondensed Condensed I Condensed I I Corrected I Uncorrected Corrected Uncorrected Count Count Count Count Total Raw Material Inventory 29 291 25 37 120 Demand Forecast 191 1 7 1 2 17 65 Finished Goods Inventory 1 5 14 1 1 19 59 Production Status 9 17 7 10 43 Costs 10 14 9' 3 36‘ Profit 7 6 8 6 27 Sales Report 4 3 9l 8 24 Lost Sales Report 4 5 3 8 20 Transportation 4 6 4 5 1 9 Dashboard 4 2 5 2 1 3 Calculators 4 1 OI 0 5 Total Responses 109 1 14 93 1 15I 431 Interestingly, the responses are statistically undifferentiated across the four treatment groups. Out of a total sample of 431 responses from the 160 participants, with each participant listing up to three answers for each of the open ended questions, none of the treatment groups yielded a significant difference in responses from the others. The same process identified the items that the participants found to be useless or distracting while making weekly operational decisions. These results are found in Table 4.22. There was far greater diversity in the answers provided to this question both on the written surveys and in the group discussion at the end of the session. The highest response rate involved the finished goods distribution center status. One of the metrics provided to the users identified the percentage of the warehouse currently in use. The responses from participants indicated that they were less concerned about the inventory 101 carrying costs or demurrage costs than they were about meeting the demand within the regions. The next highest response, market information, was a surprise to be identified on the least important list. The group discussion involving production status indicated that the participants focused on the raw material inventory and therefore were less concerned with the response from the market. Table 4.22. Least Important Decision Making Items as Reported by Participants Least Useful Items Uncondensed Uncondensed Condensed I Condensed I I Corrected I Uncorrected Corrected Uncorrected Count Count Count Count Total Distribution Center Status 6 5 2 8 21 Market Information 6 4 2 6 1 8 Production Status 2 6 1 8 17 Investments 7 2 2 4 1 5 Sales Reports 2 2 3 8 15 Warehouse Utilization 2 3 1 8 14 ‘ System Notifications 0. 3 6 5 14 Financial 3 5 1 4 1 3 Lost Sales Report 2 1 2 5 10 Too Much Data 2 2 3 2 9 Raw Material Inventory 3 2 1 1 7 Not Enough Time 3 2 0 1 6 Dashboard 3 2 0| 0 5 Transportation 1 1 0 3 5 Other 5 1 0- 0 6 Total Responses 47 41 24 63 1 75 The next highest three items include the production status, investments to bolster demand, and the sales report. In each case, the participants felt there was enough 102 complexity to manage across the supply chain without adding these variables to their weekly operational decisions. Again, analysis by treatment revealed no statistical differences among the respondents. The third open-ended question on the survey inquired about additional information the participant would have found helpful to making the weekly operational decisions. Table 4.23 presents the complete listing of responses. The most requested item was a more precise forecast of the demand. In the discussion, participants admitted that they have the same challenges when dealing with real-world supply chains and identified demand uncertainty as a major obstacle to efficiency. The next highest request was for more precise costs of transportation, production, and warehousing so they could make more informed trade-off decisions about sourcing. Of the remaining requested items, more time for decision making, more time for training the users on the details of managing this supply chain, and more precise information in general comprised the majority of the requests. Participants indicated the precise costs could have been helpful, but also included demand forecast, reliability of transportation, reliability of suppliers, and quality defect rate for the production facility as useful items. As with the previous two open-ended questions, there were no statistically significant differences among the treatment groups for the responses to this question. 103 Table 4.23. Items Needed to Improve Decision Making as Reported by Participants Needed Items Uncondensed Uncondensed Condensed I Condensed I I Corrected I Uncorrected Corrected Uncorrected Count Count Count Count Total POW“ 1o 11 8 11 40 0°33 4 9i 3 7 23 More training 1 8 3 5 17 More time 2 2 2 3 9 Transportation 0 2 01 2 4 01"" o o 2 2 4 7°“ ““9“” 17 32 18 3o 97 4.6 Hypothesis Testing This section presents evaluation of the hypotheses based upon the analysis of the previous sections in this chapter. The hypotheses are presented in the same order as they were introduced with a summary table at the end of this section. 4. 6.1 Hypothesis 1 The first hypothesis stated that in an environment of uncondensed data, supply chain managers with corrected information would perform at a higher level than those with uncorrected information. By isolating only the treatment groups with uncondensed data, performing ANOVA for the corrected / uncorrected treatment with the seven dependent variables reveals significant findings for five of the dependent variables as presented in Table 4.24. This hypothesis is strongly supported at a 95% confidence level for the overall supply chain metric and partially supported for the remaining six metrics. 104 Table 4.24. ANOVA for Corrected and Uncorrected Effects in an Uncondensed Data Environment ANOVA Sum of Squares df Mean Square F Sig. Inbound Between Groups _ 9.1951 17 9.195. 6.6581 0.011“ Within Groups 110.4897 80» ' 1.381’_ Total ' 119.685 81 ' Internal Between Groups 7 1223.40? 1‘ 12234047 3802 0.072 Within Groups 29643045 80 370.538; 9 ITotal ’ 30888449 ‘81! if i i i Outboundpetwesneroupsp _ _ 1&1, 11 _____1..46_1;_9_-§§5i ..__9.417 Within Groups _ 175.7737 807 7 2.197 Total 177.234, 811 7 7 Market Between Groups 7 645.0237 17 645.023 1.490T 0.226 Within Groups _ 34530-217; 801 432.8787 7 Total _ 35275240. 81 i 1 Profit Between Groups 1 2544.417: 1 2544.417, 2.9951 0.087 Within Groups f 67971399. 80; 849.842l * Total ‘ 70515815 81 ‘ . ‘ sc ContrilBetween Groups_ 3.1753137 1; 3.175913? 5.986 0.016” Within Groups 4.2441314 80 537049127 f 7 77 Total I " 4.561E+14: 81lm” i i Overall Between Groups 7 23.774T7 1 23.7747 4.9287 0.029“ Within Groups 7 385.9497 807 4.824 Total 1 409.723 81 . 1 4.6.2 Hypothesis 2 The second hypothesis stated that managers operating in an environment of uncorrected information would perform at a higher level with condensed data than with uncondensed data. Isolating the uncorrected information, ANOVA was performed to analyze the condensed versus uncondensed performance. As seen in Table 4.25, the investigation yields no statistically significant results. Additionally, within the perceptual results, the respondents-provided only moderate feedback of less certainty in their responses with condensed data and a moderately more difficult time performing the 105 weekly operations due tO excessive data in the Output Report and Finance Report. In the group discussions following the simulation portion of the experimental session, the respondents expressed satisfaction with the amount Of data provided even at the condensed level. One respondent from an uncondensed group explained that she occasionally found herself spending too much time sorting through the vast amounts of data. However, she further explained that the short amount Of time available between executing weeks of the simulation forced her to rely on the summary data. The hypothesis is not supported for any of the seven metrics. Table 4.25. ANOVA for Condensed and Uncondensed Effects in an Uncorrected Data Environment ANOVA —§finot Squares df Mean Square F Sig. inbound [Between Groups 7 1.88611; 1.8861 0.788I 777707.377 Within Groups 203.5861 851 2.395 1 77 Total 7 205.473 86 . Internal Between Groups ‘ 2473.719+ 1i 2473719 3.3451 0.071 Within Groups ‘ 62863618 85 739.5721 , Total 65337.337 86' ’ 1 Outbound BetweenGroups 1 2.8091 717 2.809 77 0.958: 270.3301 Within Groups ' 249.1617 85‘ 279311 1 Jotal 251.970 867 1 1 Market Between Groups I 3.5631 1: 3.5637 0.00612 70.938 . -_w1thin-9;<>ups.. 419789115. 851- -. 985-7541 ,___ Total . 49792678, 86. 1 ; Profit Between Groups 7 13933801 17 13933801 1.8721 0.175 Within Groups 63270177 85 744.355 1 Total f 646635577 861 SC Contrib Between Groups 1 5.545E+121 17 5.545E+121 0.8397 0.362 1WIthin Groups 5.618E+141 857 6.609E+12‘ Total 5.673E+14 86 1 Overall Between Groups 1 4.9317 11 4.931[ 0.8181 0.368 Within Groups ‘ 512.1497 85} f 6.025 Total 517.080 86T 106 4.6.3 Hypothesis 3 The third hypothesis stated there in an interaction between the level Of detail Of the data and the accuracy Of the information provided to supply chain managers. Specifically, the hypothesis suggested that the overall performance level would be ranked as follows: 1) Highest performance level: Corrected information / Uncondensed data 2) 2d highest performance level: Corrected information / Condensed data 3) 3d highest performance level: Uncorrected information / Condensed data 4) Lowest performance level: Uncorrected information / Uncondensed data Evaluating the analysis Of the seven dependent variables, there was a significant difference in the overall performance of the corrected / uncondensed treatment group and the uncorrected / condensed treatment group. However, the corrected / condensed treatment group and the uncorrected / uncondensed treatment group did not show significant differences between themselves or among the other groups. In the perceptual portion of the study, the respondents clearly identified differences between the corrected and the uncorrected information presented on the dashboard. However, as explored the Hypothesis 2, the respondents were less concerned about the quantity Of data presented because the summary data was available in all of the treatment groups. Therefore, this hypothesis is partially supported. 107 4. 6. 4 Hypothesis 4 The fourth hypothesis consisted of two parts pertaining to the behavior of supply chain managers when presented with incorrect information systems. The first hypothesis stated that managers presented with consecutive reports containing incorrect information would seek alternate decision variables to make his or her weekly operational decisions. In the experimental sessions, the uncorrected treatment groups received randomly incorrect information. From post-hoc analysis of the uncorrected treatment groups, 51 of the 87 participants in the uncorrected treatment groups received at least one instance of consecutive weeks of incorrect information. Analysis of this subset revealed a statistically significant (p < .05) difference in the responses to the perceptive data in the areas of information correctness, information precision, information reliability, and information usefulness than expressed by the remaining 36 participants receiving the uncorrected treatment but not experiencing consecutive weeks of incorrect information. Further, 24% (12 of 51) of the participants receiving consecutive weeks of incorrect information identified the dashboard as one of the three least useful items available for forming their weekly operation decisions. Conversely, 0% of the subjects identified the dashboard as one of the three most important items for making weekly operational decisions. The evidence supports the user’s dissatisfaction with the dashboard and reliance on alternative decision making variables to complete the weekly operations input. While there is not an empirical test to evaluate this hypothesis, the perceptual feedback and the post-hoe analysis of the participant’s performance after receiving consecutive weeks of incorrect information lends support to this hypothesis. 108 The second portion of the fourth hypothesis stated that once managers seek alternate decision variables, they do not use the uncorrected information system regardless of the accuracy of the system. As with the first part of this analysis, the perceptual feedback provides the insight into the subjects’ behaviors. The 87 participants in the uncorrected treatment groups received incorrect information on the dashboard in a total of 161 out of the 522 weeks exercised in the simulation for an overall 31% occurrence of incorrect information. In the remaining 69% of the weeks, the participants received corrected information on the dashboard. As presented earlier in this section, the uncorrected treatment groups reported statistically significant differences in the question responses included information correctness, information precision, information reliability, information comprehensiveness, information usefulness, rational approach, intuitive approach, and amount of information. The uncorrected treatment group participants indicated lower satisfaction with the dashboard and relied on a more intuitive approach than a rational approach to the decision making requirements. The group discussion revealed that the groups who identified the dashboard as misleading did not refer back to the dashboard in future weekly decision making. Again, there is not an empirical test to evaluate this hypothesis, but the perceptual evidence provides support and warrants further investigation of this behavior. 4.7 Discussion of Results The main objective of this research is to gain greater understanding into the impact that accurate decision information systems and compression of data has on the performance of a supply chain. In an environment typified by costly and highly complex 109 hardware and software requirements to detect, store, process, and display data and information about a supply chain’s status, there is great value to exploring the items managers identify as vital to operational decision making as well as those that they find least important. . Through both the detailed analysis of the seven dependent variables and the perceptive feedback of the users, the treatment groups with corrected information performed at a higher level. This treatment had a substantial impact on the decision making ability, perception of the simulation as a whole, and perception of being able to make rational operational decisions than the treatment with inaccurate information. The condensation of data by itself did not have a significant impact on the performance or perceptions of the participants in the study. Across all seven performance metrics and 48 survey questions, the treatment groups with condensed or uncondensed data differed in only two areas. One possible explanation for the lack of significant findings in this area is the inclusion of aggregated data for both treatment groups. If one group had only complete data without calculated summaries and the other treatment group received only the summary data, the study may have produced different results. This is addressed further in the limitations and opportunities for future research section of Chapter V. Despite the insignificant findings of the condensed treatment groups, the combination of corrected information and condensed data into four groups of treatments significantly affected the performance when viewed as a whole. The treatment groups with corrected information benefited most from the opportunity to add uncondensed data to augment their decision making variables. Conversely, the uncorrected information 110 treatment groups did not realize a substantial benefit from receiving condensed data to aid the decision making process. Collection of perceptual feedback from the users provides a useful list for managers. By identifying the raw material status, demand requirements, production status, and finished goods status as the most essential decision making items, the groups were able to condense nearly 1,000 variables into a few critical items for successful operational decisions. Equally important, identification of items that were unnecessary for operational decision making can help managers eliminate excessive data collection. As a whole, the items on the least useful variables list included areas traditionally outside of the supply chain environment. Inclusion of such items as financial reports, marketing information, production status, utilization rates, and sales figures on the list indicate the narrow focus sometimes adopted in managing complex supply chains in constrained timeframes. By broadening user’s scope of decision variables to include some of these items, greater returns in performance may be realized. The list providing the items participants requested to make better operational decisions is similarly valuable to researchers as well as managers. Identifying better methods for forecasting demand behavior, providing improved precision of costs and decision variables, and offering managers the luxury of time to consider all of these variables before making operational decisions are all areas that researchers and practitioners can collaborate to improve supply chain management as a science and a occupation. 111 CHAPTER V: CONTRIBUTIONS AND CONCLUSION 5.0 Introduction This chapter presents the implications of the research findings. The first section presents the research contributions of the study. The second discusses managerial contributions. This section is followed by a discussion of the limitations of this study and recommended directions for future research. 5.1 Research Contributions This study makes academic contributions to both theory refinement and experimental design for future research opportunities. The Knowledge Based View of the Firm provided a foundation for this study through identification of critical implicit knowledge required to perform complex functions and then detect and adapt new information into an existing framework. The study largely upheld the idea that greater levels of data converted into corrected information will help managers make operational decisions resulting in higher operational effectiveness. However, this study also raised questions about Davenport and Prusak’s (1998) processes to transform data to information and then to knowledge (Figure 2.1). The authors asserted condensing vast amounts of data was essential to knowledge creation. In this study, there was no statistical difference between the users in the uncorrected treatment group with uncondensed data and those with condensed data. This creates potential for future studies to resolve the apparent paradox. The approach undertaken in this study is unusual within the supply chain management arena. The triangulation of techniques to capture complex simulation data, 112 a formal experimental design, and essential perceptual feedback is not a common practice in the field. However, the insight gleaned from putting the empirical results together with human impressions and behaviors is critical to better understanding the research questions. This process also ensured the research remained pertinent to managers due to the close interaction with the population representing diverse levels of supply chain knowledge and experience. While some areas for refinement are addressed in the limitations section, future studies should incorporate similar design to provide the human behavioral reporting in conjunction with empirical results. 5.2 Managerial Contributions The opportunity to work closely with managers in the development of SCODE, the testing of the experimental design, and the experimental sessions ensured the pertinence of the study to managers. By design, SCODE is intended to bring together practitioners in the supply chain arena of multiple levels and create a cooperative learning environment. In addition to preparing for the experimental sessions, the validation opportunities with SCODE yielded numerous improvements for application of the simulation to the classroom and Executive Education environments. A few examples of the evolution of the program included development of a simpler user interface to control SCODE, deployment of the program from USB drives rather than full installation on a computer, and the addition of material requirement calculators to reduce the mathematical computation time between operational decisions. Through the refinement in conjunction with this study, SCODE has become a valuable tool to integrate with 113 traditional classroom learning in developing the skills and knowledge necessary for managers to successfully compete in the complex and challenging supply chain field. As a result of this research, a great deal of insight was gained into the behavior of managers under dynamic conditions within a supply chain. Though the experience involved a simulation in this case, these same phenomena have been realized at subsequent Executive Development sessions. As managers have the Opportunity to design or revise an existing supply chain, strategic goals of the company must be a priority consideration. Through identification and publication of goals focused on certain customer satisfaction levels, specific profit margins, return on assets, or other measures, the personnel employed within the supply chain environment are immediately conditioned to certain behaviors. Senior management must ensure that the metrics, goals, and reward system within the company promote broad-visioned employees rather than people who optimize local metrics at the expense of the overall supply chain. For example, in this research, participants could easily ensure 100% production efficiency for their metric by increasing safety stock and setting a desired fill rate of 80% of customer demand. However, the efficiency and effectiveness of the entire supply chain suffers. Similarly, senior managers must carefully balance the employees’ span of control reward system with the overall needs of the company. Further developing the previous recommendation, broader training requirements within the supply chain environment are clearly needed. Discussions with managers and researchers support the assertion that many employees in supply chain related positions have limited comprehension of tactical operational requirements across the supply chain. Many employees have a broad understanding of how procurement, transportation, 114 warehousing, production, distribution, and marketing fit into the overall health of the organization, but few grasp the effect that a decision in one area can have throughout the supply chain. Numerous participants in this study discussed the eye-opening experience of failing to enter a raw material order correctly and then watching production, transportation costs, and demand satisfaction suffer over the next few weeks of the simulation. Development of supply chain professionals at all levels must be a recurring process within firms. A third recommendation focuses on the approach to this training. By integrating the simulation into structured lessons with specific objectives, the participants were able to reinforce the concepts from the lessons in a first-hand environment without jeopardizing the health of the firm in the real world. Completing the learning loop by sharing lessons and insight among groups, the managers had the opportunity to express their perceptions at different challenges of the simulation and learn from other participant’s successes and failures. Whether the simulation is SCODE, another computer-based simulation, or a completely different hands-on learning tool, this educational process has strong merit and should be adopted by managers seeking return on their training investments. The fourth area of research contributions focuses on the linkage of accurate information and uncondensed data with higher performance levels. While the most complete data sets combined with accurate decision information systems yielded significantly higher performance in some areas, there was not a statistically significant difference between the corrected / uncondensed, corrected / condensed, and the uncorrected / uncondensed groups at an overall supply chain level. Given an unlimited budget and the ability to integrate extensive quantities of data into a management 115 dashboard, a firm would be wise to get the most accurate and complete system available. However, firms that must make trade-offs due to costs, computing skills, storage capacity, time required to record data, or any variety of other constraints affecting most businesses should make investments into the most accurate decision information systems available, even at the expense of completeness of data. The fifth managerial contribution is discussed in greater detail at the end of Chapter IV (pp. 109 — 110) and will be only briefly mentioned in this section. The creation of lists identifying the most important, least important, and desired items for managers to make supply chain decisions should be invaluable. By focusing on these items and gaining a better understanding of significance to a particular firm, managers should be able to make immediate impact in operational performance through addressing the shortfalls, reinforcing the most important items, and making a decision to either eliminate recording or improve education related to the least important items. 5.4 Limitations and Opportunities for Future Research As with any research involving a simulation study, certain assumptions are adopted in the creation of the simulation to limit the breadth of topics that can be covered in a short amount of time. The assumptions also serve to focus the variability of the interaction with respondents in an effort to detect significant differences in a few effects. In the SCODE simulation, there are numerous assumptions created to provide an environment where the participants can quickly adapt to the interface and learn valuable supply chain management lessons within a short amount of time. The operating environment was created to capture the most critical aspects of managing a supply chain 116 without becoming overburdened with the thousands of details occurring every day in real-world supply chains. SCODE provides a dynamic learning environment for users to be exposed to a variety of supply chain challenges. While it is Excel-based for ease of interface with the users, it shares a common limitation with many software packages. Specifically, SCODE has a steep learning curve for new participants. All of the subjects in the study were new users of SCODE. A 3-hour timeframe was extremely compact to provide a global sourcing lecture, distribute USB drives of the software and ensure compatibility with diverse personal computers, train users on the input-output cycle of SCODE, execute the scenario, complete the survey, and participate in the nominal group session. With additional time or multiple sessions, the results of this study may have changed, particularly with the condensed / uncondensed treatment groups. If the users were more proficient at reviewing the available data and integrating more of the variables into their weekly decision making routine, there would likely have been greater significance in the findings between the groups with only condensed data and those with extensively detailed data. Starting with the basic design of SCODE, there are realistic, but simplified, parameters affecting the interactive environment. The structuring of the simulation into weeks of play accelerates the input-output effect, but forces assumptions about behavior and performance. Since each set Of operational weekly decisions requires the participants to invest considerable time and effort in the process, some of the details that could normally occur in a firm are overlooked. The raw material orders are placed at the beginning of the week. However, the users will not receive feedback about exact 117 quantity to arrive or the timeliness of the shipment until the following week. Further, if the user schedules production which exceeds the labor force capabilities, the feedback occurs in the following week when shortages are apparent. When there are shortages affecting the user’s ability to meet demand in a market, there are no backorders. All of these assumptions create an environment suitable for understanding and developing the concepts critical to operating an effective supply chain network, but will not mimic all aspects of real-world supply chains. Another limiting aspect when compared to real world supply chains is the architecture of the scenario. In this case, the users operated a single production facility and could source raw materials from four locations to distribute finished goods to five markets. This process is an enormous simplification of a real-world supply chain, but again creates a manageable framework for the participants to operate within. The characteristics of the products created in this scenario could have a limiting factor on the results. The general design incorporates a chemical company manufacturing artificial sweetener for bulk consumers. The design includes liquid and dry powder variants of a standard grade and a premium grade of the product. Each of the products requires four or fewer raw materials to create. The weight and packaging requirements favor shipment by land or sea by making air transportation significantly more expensive. Given the design, these are reasonable assumptions. However, users who operate in a drastically different supply chain in the real world will bring certain preconceptions to SCODE. If the participant’s background involves high-cost electronics with a short shelf life and intense competition in the global market, he or she will be accustomed to utilizing air transportation for inbound and outbound shipping, will meet 118 all demand at any cost to preserve market share, and is likely to schedule the production mix to favor the premium lines of products. This may be in stark contrast to a participant with a background in low value, bulk goods with an extended shelf life and a steady market share. While the design cannot simultaneously mimic both extremes of a supply chain in a single setting, it could be a limiting factor to the broader application of the results. At the same time, the design of the scenario provides opportunities for future studies. The first recommendation for future research includes developing a range of scenarios with different product focus to investigate the changes in behavior over diverse supply chains. One iteration could incorporate the lightweight, high value item discussed in the previous paragraph with a design favoring air shipment and complete satisfaction of demand. Another scenario could include more complex supply chains consisting of multiple production facilities, complex demand curves, and supply chain interruptions. The behavior of participants under the different circumstances may be significantly different than under the scenario provided for this study. A second research opportunity involves refinement of the current study. By creating additional layers of correctness of information and condensation of data, a more refined model of the trade-offs could be developed. In future iterations of this study, levels could include corrected information with no data, condensed data with no information dashboard, a between-group level of data condensation, and completely uncondensed data without including the condensed summary data. Repeating the study with these additional levels of treatment would require substantially more participants, but could shed additional light on the balance required between costly and complex data 119 collection and information management systems and the absence of quality information or enough data to make informed operational decisions. Another opportunity for future research involves testing additional aspects of Davenport and Prusak’s (1998) model. By controlling the quantity of data and the quality of information throughout the simulation, variables addressing the contextualization, categorization, and calculations of the data could be manipulated and the study repeated. While less focused on the information technologies required to process data, these variables would lend insight into the cognitive processes of integrating supply chain data into the information required to make knowledgeable Operational decisions. 5.5 Conclusion This study set out to investigate the process of converting supply chain data into useable information yielding the knowledge necessary to achieve a high level of performance. This chapter summarized the study as it pertains to the goal. This study also provided empirical results and perceptions about the variables in question. This chapter summarized the previous four chapters and recommended opportunities to further develop the framework into additional research. The research and managerial implications of this study provide the opportunity to benefit from this study as well as create internal improvement opportunities to build a stronger sense of supply chain theory and improve operational decision making. 120 APPENDIX A: SCODE USER’S MANUAL EXECUTIVE SUMMARY IllI LINIFRICDRIIAIMR>1HP C31 1111 DICE ITAI INIIIF’RISI SCODE SIMULATION MODEL EXECUTIVE SUMMARY BROAD [I INISK ftllSL.-.I.IKLAN STATE I N‘s \ E It s I 1 \ April 8, 2008 121 INTRODUCTION SCODE is a computerized logistics simulation designed to provide realistic training on integrated decision making in a competitive supply network environment. SCODE is executed as a supply chain in which users compete against an environment controlled by game administrators. SCODE demonstrates the importance of integrated decision making. Participants make several decisions including sourcing, supply base collaboration, production scheduling, marketing, and distribution. SCODE also provides a learning environment to apply the concepts presented in educational seminars to a realistic setting. Learners are grouped into teams. Each team is expected to discuss and make strategic and operational decisions in order to compete effectively in SCODE. Due to time constraints for making and entering decisions, each team is required to work efficiently. Moreover, SCODE goes beyond a simple training exercise -— it provides users an opportunity to enjoy the challenge of achieving top performance in a highly competitive environment. DECISION OVERVIEW SCODE teams are expected to develop and execute an integrated supply chain strategy in a competitive environment. Each team makes two sets of decisions: strategic and operational. Strategic decisions relate to supply chain network design. Simulation initialization decisions (intended to get the simulation process started) are made during strategic decision phase. Operational decisions involve weekly supply chain activities. SCODE administrators control the simulation environment by manipulating production ’ economies, material availability, transportation lead times, and demand variation. Thus, as Figure 1 shows, there are three categories of decisions fed into SCODE: (i) Strategic decisions, (ii) Operational decisions, and (iii) Administrator decisions. (Refer to Sections 2.0, 3.0, and 5.0 of the User’s Manual for a detailed discussion of each decision phase). The administrator runs SCODE simulation and then gives results to teams to help them make their next weekly decisions. The results are in form of reports (Refer to Section 4.0 for detailed discussion of SCODE output reports and interpretation). 122 Figure 1. SCODE model framework Strategic Decisions (One time team decisions at start) 0 Plant (own or contract) Administrator Decisions (Controlled by administrator) 0 Demand variation e Production economies '0 Transportation lead times oSupplier availability oReport detail 0 Plant capacity 0 DC (own or contract) 0 DC capacity 0 Market & DC allocation 0 Initialization Decision 0 Initial inventory 6 Scheduling SCODE V Omrations Decisions (Weekly team decisions) oSOurcing (plant, supplier, raw material, mode, quantity) 0 Production (what to produce where A and when, quantity) oMarketing (R&D, promotion, price) oDistribution (source, DC, mode, quantity) RESULTS GEOGRAPHIC OVERVIEW SCODE environment consists of five geographic regions, namely, North America, Europe, South America, China, and India. The team’s manufacturing plant will be located in one of these regions. Raw materials can be sourced from any of the available suppliers. A distribution center is required to be open in the region where the manufacturing plant is located and a team may open additional DCs in other regions. There are five market segments, representing the five geographic regions. Teams can locate DCs in each market segment or in select segments. A DC can serve more than one market segment. Figure 2 provides a pictorial depiction of the entire SCODE supply chain network. In any given scenario, portions of the network may be unavailable in order to emphasize certain learning objectives. 123 Figure 2. SCODE Supply Chain Network Suppliers Suppliers Suppliers Suppliers (North America) (Europe) (South America) (China) 1 1 L T v W North America Europe South America China RM warehouse RM warehouse RM warehouse RM warehouse Plant Plant Plant Plant DC DC DC DC 6 v t J 1 DC (North DC DC DC DC America) (Europe) (South (China) (India) America) 1 v 11 v Market (N. A.) 124 PRODUCT OVERVIEW SCODE consists of two product categories: bulk industrial product (liquid) and packaged consumer product (dry). Each of these products has a regular and premium brand. Each product brand has different raw material requirements. Teams may decide to deal with one product line (industrial versus consumer product), one brand of each product category (regular versus premium), or all of them (both product categories and brands of each category). The brands are: Table 1: Products and Raw material components Product Category Brand Raw Materials Industrial (Liquid) Regular Each product will Premium consist of a unique Consumer (Dry) Regular mixture of raw materials Premium from the four types TRANSPORTATION OVERVIEW Three Options are available for to transport raw materials to the manufacturing plant, transport finished goods to additional DCs, or in transport finished goods to the market. The options, cost, reliability rates for on-time deliveries, and expected delivery times are shown in Table 2. Table 2: Transportation Mode Cost Reliability Expected Transit Time (weeks) Air High High 0 (arrives in same week shipped) Land Low High 1 Sea Low Low 2 FEEDBACK SCODE will provide the teams feedback in a number of Excel worksheets. This data includes raw material levels, production output, DC finished goods levels, quantities sold to market, lost sales, financial performance, transportation costs, plant Operating costs, and raw material costs to name but a few. Successful interpretation of these indicators will affect the next week’s decisions and help produce a sustainable supply chain. ADMINISTRATOR ROLE SCODE administrators will identify key training objectives for an educational session. Based upon these learning objectives, scenarios will be designed to mimic real-world 125 situations. The teams will have the opportunity to apply their personal knowledge as well as integrating the information provided during the classroom education portion of their training in the SCODE environment. Through the use of the “Administrator” workbook, administrators will have the ability to alter the very nature of the SCODE experience to focus attention on one or two topics. The learning cycle is closed by providing both in- depth debrief on the decisions and outcomes of a team’s performance as well as a group- wide debrief on the learning objectives from the lesson. Additional information about the variables, scenario design, and expected outcomes are available in the Administrator’s Manual. 126 APPENDIX B: SCODE GLOBAL SOURCING BRIEFING Global Sourcing with the Supply Chain Operations Decision Environment 6 — 7 June 2008 Shawn Jones E'L‘s'fi'té‘é Introduction - SCODE Overview - Executive Education Scenario: Training with Global Sourcing Focus . Execution - Metrics / Feedback 127 Installation Of the Software - Insert USB drive - Copy entire SCODE folder to your C:\ - Double click on C:\SCODE\SCODE.exe - Should open SCODE window - If so, close the window - If not, let me know - Pass the drive on _ Introduction: SCODE Simulation - Interactive learning tool designed for Executive Education and MBA courses - Setting includes: - Individuals or teams Operating a global supply network - Up to four raw materials - Up to four finished goods - Over 600B possible approaches - Over 1,000 administrator-controlled variables 128 1115mm SCODE Product Overview - Industrial (Bulk) - Consumer (Package) — Liquid — Dry - Premium - Premium - Regular - Regular Why Source Globally? Access to markets Cost/Price Benefits Access to Technology Quality Access to Only Source Available Introduce Competition to Domestic Suppliers React to Buying Patterns of Competitors Establish a Presence in a Foreign Market 129 Barriers to Global Sourcing ° Lack of understanding Of International procedures - Lack of knowledge of documentation - Resistance to change - Domestic market nationalism Longer leadtimes and material pipelines Increased uncertainty and risk Barriers tO Global Sourcing - Logistical, political, financial risks - Planning, coordination, information sharing - Lack of knowledge of foreign business practices - Language and cultural differences - Expertise and labor . Negotiations can be difficult - Engineering changes, changes in general difficult 130 1:; Global Sourcing Futur - Continued refinement/development of the global sourcing process Develop or obtain human resources with higher-level skills and a willingness to view the supply network from a worldwide perspective Develop global performance measures Establish integrated systems between worldwide units and with suppliers aumm Training (Global Sourcrng) Scenario Learning Objectives — Introduce SCODE - Orientation to input/output — Increase understanding of cause-effect in SCODE — Secondary objective: learn about trade-offs of global sourcing Simulation Model Proxies — Variable transportation reliability — Significant price difference for regions — Bulky, heavy goods expensive to transport - Variable product availability 131 - Supply chain goal: - Selected suppliers because: - Marketing approach: - Other notes: lil'tlk'té‘é Operations Commentary - Challenges to meet your goal: - How were the challenges handled? - What worked well? - Other notes: 132 APPENDIX C: SCODE SIMULATION SCENARIO fl “\d IIIF C IPxITIRI