TEAM ERROR MANAGEMENT TRAINING: THE ROLE OF MONITORING, BACKING UP BEHAVIOR, PLANNING, AND PSYCHOLOGICAL SAFETY IN TEAM TRAINING AND TEAM ADAPTIVE PERFORMANCE By Robert Gray A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Psychology—Master of Arts 2019 ABSTRACT TEAM ERROR MANAGEMENT TRAINING: THE ROLE OF MONITORING, BACKING UP BEHAVIOR, PLANNING, AND PSYCHOLOGICAL SAFETY IN TEAM TRAINING AND TEAM ADAPTIVE PERFORMANCE By Robert Gray While many studies and meta-analyses have examined error management training at the individual level, this study is the first to apply these principles at the team level. Specifically, the study explores the relationships between error management and team learning perceptions, team communication, and team psychological safety, respectively, and their effect on team adaptive performance. Ninety-seven teams of three members were trained. Results show a positive impact on adaptive team efficiency, as well as increased team psychological safety and perceptions of team learning related to errors. Applied and theoretical implications are discussed. TABLE OF CONTENTS LIST OF TABLES .......................................................................................................................... v LIST OF FIGURES ....................................................................................................................... vi INTRODUCTION .......................................................................................................................... 1 LITERATURE REVIEW ............................................................................................................... 4 Errors, Error Consequences, and Error Detection ....................................................................... 4 Two Types of Training: Error Management and Error Avoidant ............................................... 6 Error Management Research ..................................................................................................... 10 Diagram of Model ..................................................................................................................... 21 Team Performance and Error Management .............................................................................. 21 Adaptive Performance and Critical Thinking ........................................................................... 25 Research Justification ................................................................................................................ 28 METHODS ................................................................................................................................... 31 Simulation-based Training ........................................................................................................ 31 Participants ................................................................................................................................ 32 Materials .................................................................................................................................... 33 Procedure ................................................................................................................................... 33 Measurement ............................................................................................................................. 36 Communication ..................................................................................................................... 36 Psychological safety .............................................................................................................. 37 Team learning ........................................................................................................................ 37 Adaptive performance and efficiency .................................................................................... 39 RESULTS ..................................................................................................................................... 41 Hypotheses ................................................................................................................................ 41 DISCUSSION ............................................................................................................................... 46 Limitations ................................................................................................................................ 47 Practical Implications ................................................................................................................ 49 Future Directions ....................................................................................................................... 50 APPENDICES .............................................................................................................................. 54 APPENDIX A TEAM PSYCHOLOGICAL SAFETY SURVEY ........................................... 55 APPENDIX B PHASES OF EXPERIMENT ........................................................................... 56 APPENDIX C DEMOGRAPHICS ........................................................................................... 58 APPENDIX D ERROR MANAGEMENT TRAINING MANIPULATIONS ......................... 59 APPENDIX E TABLES............................................................................................................ 60 APPENDIX F FIGURES .......................................................................................................... 63 iii REFERENCES ............................................................................................................................. 64 iv LIST OF TABLES Table 1. Intercorrelation Matrix of Measures ............................................................................... 60 Table 2. Means (and Standard Deviations) of Measures .............................................................. 61 Table 3. Standardized Loadings for Four-factor Structure of Team Learning and Team Psychological Safety ..................................................................................................................... 62 v LIST OF FIGURES Figure 1. Model of relationships of hypothesis at the team level ................................................. 21 Figure 2. Model 2 .......................................................................................................................... 63 Figure 3. Model 3 – Final Path Model with Only Error Factor of Team Learning ...................... 63 vi INTRODUCTION Errors manifest in all areas of organizational life (Reason, 1990). When they surface, organizations tend to approach them by either viewing them as a source of shame and a signal of failure or embracing them as a chance to learn a valuable lesson. Studies have shown that companies who embrace an “error acceptance culture” are not weakened by welcoming errors at all. Workers in these environments respond more positively and successfully to threats and face novel situations with more ingenuity and persistence. There is even cursory evidence that these companies earn more profits when compared to similar companies with more traditional “error as failure” philosophies (Van Dyck, Frese, Baer, & Sonnentag, 2005). Despite these findings at the macro and micro level, there has been little research done at the meso level (Bell & Kozlowski, 2011). Teams—defined as groups of individuals with interdependent roles who work for a shared outcome—and more specifically, teamwork—the “how” of that interdependent process—have largely been ignored by error research, especially empirically-based research. Much can be gained by investigating the impact framing errors as positive information and training teams to explore errors as sources of valuable information has on learning and adaptive performance. On September 19, 2014 over the course of 15 minutes, Omar Gonzalez scaled an 8-foot fence, then traveled across the expansive lawn and into the White House where he was finally captured (“White-House Fence Jumper,” 2014). This breach in security was not the result of an evil mastermind or super-human athlete—Gonzalez was a 42-year-old veteran who suffered a foot amputation and was taking strong medication for depression and PTSD. Neither did it result from lack of resources—the White House budget was over $20 million in 2014. It was the result of individual and team errors. 1 Reports discuss multiple critical failures by security during the incident. One of these security features, a team of plainclothes Secret Service agents was assigned to the outside area as an “early warning” where Gonzalez jumped the fence, but the team failed to notice him. Another security feature was an officer at the North Lawn guard booth who was trained to release a dog and call the SWAT team in such events, but he failed to follow protocol. Additionally, an alarm box near the front entrance of the White House had recently been muted because of a request by an usher. Had protocol been followed, the alarm would have triggered, notifying staff, and locking key doors. Once inside the building, Gonzalez rushed past a guard and overpowered another guard before finally being physically taken down by an off-duty agent. Following the event, government agencies used these errors to analyze the security strategy for the nation’s capital and made improvements such as adding anti-climbing fences, installing new sensors, and redesigning training measures for the security officers. There have been many similar events in recent world history when error management approaches may have reduced the costliness of collective efforts: the Southwest Airlines Flight 1455 crash, the Mars Climate Orbiter malfunction, the Deepwater Horizon explosion, and Fukushima Nuclear Power Plant melt down, to name a few. Teams in these events have more at stake and the errors they make can be costly not only to their organizations, but also to the national and international communities. By training teams in ways that make errors more salient and reinforcing communication norms that encourage information sharing about errors and inefficiencies teams may be better-suited to engage in tasks where errors and inefficiencies can be especially costly. Weick and Sutcliffe (2001) offer a useful sub-category of organizations (and teams within these organizations) called high-reliability organizations (HRO). They argue HROs operate in high-risk environments in which learning and adaptability are more impactful when 2 compared to more traditional environments. This is just one example of an existing area of organizational scholarship that can benefit from additional insights into error related research. To thrive in these more perilous and error-filled settings, HROs value and promote five core competencies: sensitivity to operations, commitment to resilience, deference to experience, reluctance to simplify, and preoccupation with failure. All five of these subdomains involve errors and reward identifying, processing, and adapting to errors. For instance, a part of sensitivity to operations means constantly seeking to understand and monitor one’s own behaviors as well as the dynamic environment. Due to the precarious nature of HRO work, minor failures can lead to more costly losses. This is only one example of how holistic and positive views of errors enables successful behaviors and outcomes in high-stakes situations. This present study contributes to the existing literature in the three ways. First, it replicates previous error management training studies. Second, it extends the methods used during individual level error management training to the team level. Third, the primary focus of this study explores error management training at the team level. More particularly, it identifies three team behaviors – planning, monitoring, and backing up – as critical to error management training and examines the positive relationship between increased use of these behaviors and adaptive performance compared to a training procedure without error management training principles. To my knowledge, no past studies have aimed to systematically test these specific team behaviors to improve adaptive performance through error management training. 3 Errors, Error Consequences, and Error Detection LITERATURE REVIEW In a discussion of error management training, it is helpful to define what is and is not meant by the term error and describe relevant context for the discussion of errors in the world of work. An essential feature of organizational behavior is goal-oriented actions. Individuals and teams plan, act, and reflect to accomplish the objectives of an organization. Errors can be defined as potentially avoidable actions that cause a person or team to unintentionally fail to achieve a goal (Hoffmann & Frese, 2015). First, errors are not intentional (Reason, 1990). A person or team may decide to intentionally fail to achieve a goal and this is not an error. For example, a truck driver may have a safety protocol that calls for a checklist to be consulted before starting the engine of his truck, but in an emergency, he may decide to ignore the list to save time. By sacrificing the low priority goal, the driver may arrive at his destination sooner, thus achieving a higher priority goal. Next, inefficiencies can be errors if efficiency is part of a broader goal (Hoffmann & Friese, 2015). For instance, a team member may be tasked to produce a written report by a certain deadline for part of a larger project. If the team member fails to produce the report on time, another member of the team may create the report after the group is made aware of the failure. This not only delays the report but also unexpectedly diverts members away from their other duties, resulting in lowered overall productivity. Now that some of the basic properties of errors have been defined, it is useful to look at error detection, since error detection is a prerequisite for improving performance when errors occur. Frese and Hoffman (2015) make the distinction between error signals and error consequences to isolate sub steps of error detection and to make it easier to discuss error 4 management strategies. They define error signals as the information noticed by people when an error occurs, while they define error consequences as any outcomes resulting from an error. They note that error can have multiple consequences that can unfold across time. For instance, a document with financial information may be created, an error in one of the rows made during that creation, and each time that information is applied to a new project it could negatively impact the outcome. Accordingly, error detection is valuable to consider because it is recognized as the starting point for regulating and correcting errors (Frese & Hoffmann, 2015; Reason, 1994). Error detection is defined as an awareness that an error has taken place. This is distinct from understanding the cause of an error or predicting the consequences of an error (Zapf et al., 1994; Zapf & Reason, 1994). Error detection can be seen in many discussions, theories, and frameworks in organizational psychology that do not explicitly focus on errors because errors are ubiquitous in the world of work. Self-regulation theory, for example, describes people as open systems that are constantly performing goal-directed behavior while gathering information from themselves and their environments (Karoly, 1993). While moving towards desires or completing projects, people compare their actions to their desired actions, termed self-observation or self-monitoring. According to this theory, self-monitoring can be both automatic (e.g. noticing a mistake without purposefully planning to monitor one’s actions) and purposeful (e.g. planning to allocate some of one’s limited cognitive resources on attending mindfully to goal-relevant information). It makes sense to apportion some of one’s cognitive resources to monitoring if the costs of errors are high or if the possibility of errors is likely. 5 In fact, teams posit error detection (and more generally team process monitoring) as a strength of teams compared to individual work, since teams have a pool of cognitive resources to allocate towards a shared goal. Team monitoring has been shown to increase team performance through increased coordination and feedback (Marks & Panzer, 2004). In teams, monitoring is a prerequisite to changes in strategy or reallocation of resources in response to errors, which allow teams to complete objectives despite errors (Marks, Mathieu, & Zaccaro, 2001). Two Types of Training: Error Management and Error Avoidant One purpose of training is to prepare workers for future job performance. Successful training involves transfer, defined as when a person can generalize learned behavior to the context of the job and maintain the behaviors over a period of time (Baldwin & Ford, 1988). To further focus the scope of training, Kraiger, Ford, and Salas (1993) identify three types of learning outcomes during training: 1) knowledge, 2) skill, and 3) affect. These three types of outcomes are present in nearly all training agendas, allowing for an analysis of different styles of training. Two types of training are commonly compared in error management literature: error avoidant training and error management training. According to Bandura (1986), error avoidant training assumes errors are a source of pain (and negative attitudes) for trainees, thus, to be avoided. This perspective on training typically involves three common steps: (a) provide the context and steps needed to master a skill or complete a task, (b) allow the trainee to apply these steps to internalize and form the necessary ability, and (c) support any deviations with prompt feedback to minimize exposure to mistakes and digressions. Social learning theory, specifically vicarious learning, supports this point of view. Within this framework, humans are distinct among living beings in their ability to use modeled behavior as a tool to transmit skills from person to person (Bandura, 2001). By 6 bypassing trial and error learning, people can sidestep potential mistakes and transfer knowledge, skills, and abilities more directly. As learning increases in complexity, mistakes become “more costly and hazardous” according to Bandura. This point of view has been explored in some depth in the behavior modeling training research. Borrowing heavily from social learning theory, behavior modeling training sequences training in a way that clearly defines the desired behaviors to learn, models effective use of those behaviors, allows trainees to practice, and gives trainer feedback and social reinforcement (Taylor, Russ-Eft, & Chan, 2005). According to Bandura (1977;1989), during the process of learning, people often become fearful and defensive when they encounter errors or experience slow, gradual growth. This kind of hazard-prone trial and error process can be mitigated or removed completely by having learners observe correctly-modeled behavior and learn vicariously from people who demonstrate effective behaviors and the consequences of such behaviors without directly experiencing the negative affect from personally struggling to perform the desired behaviors. Behavior modeling training aims to focus the attentional resources of a trainees on key characteristics and behaviors. By drawing attention to central characteristics and behaviors needed for successful future performance, a trainee can better encode these items (Taylor, Russ- Eft, & Chan, 2005). This allows for more effective mental rehearsal during practice and ultimately recalling these behaviors when they are needed on the job. Additionally, during practice, the trainer provides immediate corrective feedback coupled with social reinforcement (i.e. praise), leveraging the social nature of people as a mechanism for improved transfer. Although Bandura (1977) argued that one of the benefits of this kind of learning was avoiding mistakes and ineffective behaviors, Baldwin (1992) found that behavior modeling 7 training incorporating mix (negative and positive) models generalized learning better to novel situations compared to those trained in positive-only models (i.e. models demonstrating correct behaviors only). Compared to a control group who experienced only positive modeling strategies, trainees who experienced positive and negative models showed superior performance and generalization. This may be explained by the fact that learners experienced more contrasting information to define key behaviors, showing that negative experiences are neither purely wasteful nor harmful to learning. These findings support the depth of processing view (Mann & Decker, 1984) of learning, supporting the idea that desired behaviors must be memorable to be reproduced. Mann and Decker argue that not only successful, repeated, and verbalized behaviors but also unsuccessful behaviors and even exaggerated behaviors contribute to the creation of internal schemas from which successful behaviors are generated. In other words, people benefit from both positive models and the feedback generated by inefficient and error laden experiences to form distinctions leading to successful performance (Salas, Rhodenizer, & Bowers, 2000). In contrast, error management training views errors as a potential source of positive information. Instead of being given quick corrective feedback, learners are encouraged to explore mistakes, making sense of them and using them to learn about the broader causal elements of an area of interest (Keith & Frese, 2005). This activity is believed to be a valuable step towards critical thinking (Halpern, 1998), making the learner more prepared for future events beyond the controlled training environment and without the immediate support of trainer feedback. Because errors tend to be novel and unexpected events, learners are often surprised and can ruminate on the situation. Therefore, errors can induce mindfulness and stimulate learning (Bell & Kozlowski, 2009). 8 Error management training also aims to build a positive climate for learners to make mistakes and use errors as a source of growth (e.g. have positive affect towards errors or deviations from expected actions or outcomes). First, individuals and teams must be willing to acknowledge failure rather than conceal it. In the context of team performance, this may be more salient because actions and the success of outcomes are often highly interdependent and require coordination and communication actions from more than one team member. Additionally, in team operations each team member must take interpersonal risks of raising a concern without fear that it will be held against him or her (Cannon & Edmondson, 2001; Burke et al., 2004). Psychological safety is defined as the shared belief that the team is a safe space for interpersonal risk taking (Edmondson, 1999; Rosen et al., 2011). By promoting a supportive environment with psychological safety, team members will be able to share information about errors necessary to promote error exploration and adaptive behavior. Thus, team performance may benefit from error management training because of the impact of improved psychological safety and its ability to facilitate more information sharing. Because errors are ubiquitous and expected, other training models and systems discuss errors in various ways. One example is guided error training, a training method in which errors are purposefully included in the training experience (Wilson, Burke, Priest, & Salas, 2005). By manipulating the training task to include exposure to errors, trainees receive the additional knowledge of the errors and exploration afterwards as a type of feedback for skills-based training (Lorenzet, Salas, & Tannenbaum, 2005), resulting in better performance compared to error-free training. Because many error management training approaches have employed planned error exposure as part of the training plan, including this line of research in the overview of error management training helps to explain the benefits of error management training. Crew resource 9 management (CRM) is another training style in which errors are systematically viewed and incorporated into training. Because the current literature of CRM training shares some overlap with error management training, a more in-depth discussion of CRM training is included later in the review of error management research. To summarize, both training styles (error management and error avoidant) have been shown to be valuable (Keith & Frese, 2008). The context and type of skill should inform the type of training used. For routine, highly repetitive tasks, error avoidant training may be the most direct path to desired outcomes. If adaptability is desired, error management has been shown many times to be superior because by exploring errors and uncovering deeper underlying patterns and relationships, the learner has a stronger foundation for responding to unanticipated contexts that are beyond the scope of initial training (Keith & Frese, 2008). Error Management Research Over the last 25 years there has been substantial research on the effectiveness of error management training compared to error avoidant training. The first wave of studies attempted to define error management training and empirically test it in experimental lab settings. After finding positive evidence of the value of error management training, further studies expanded the training contexts and explored the relationship of error management training to other variables of interest such as adaptive performance and efficacy. Most recently, because of the breadth and depth of error management studies, meta-analyses have been conducted and theoretical work has been done to predict error management training in new domains (e.g. team level). A brief overview of these initial studies is useful for understanding the context in which error management has been shown to be successful. The numerous studies can be classified into three distinct categories: computer-related tasks, simulated driving tasks, and professional skills 10 (in aviation and medicine). The variety of contexts is promising evidence that other areas such as teamwork- related skills and complex, interdependent tasks may also benefit from error management training because teams can leverage behaviors such as planning, monitoring, and backing up to detect and adapt to errors during operations. First, numerous studies employed various computer-related tasks, such as designing slides for Microsoft PowerPoint (Keith & Frese, 2005), creating databases for Microsoft Access (Debowski, Wood, & Bandura, 2001), searching for information using electronic library software (Wood et al., 2000), using word processing software (i.e. WordStar) to do basic tasks (Frese et al, 1991), and using software (i.e. SPSS) to demonstrate basic knowledge in statistics (Dormann & Frese, 1994). These studies form a baseline for what effective error management training can look like because they consistently employ the same protocols, such as instructional design, environmental cues, and trainer standards regarding feedback. Next, the use of computer technology to simulate real experiences has been used to demonstrate the effectiveness of error management training in driving tasks. Driving is an activity where errors can be life-ending, and self-perception of ability is often erroneous compared to actual ability (Marottoli & Richardson, 1998). There is a consensus that people tend to be overconfident and optimistic about their ability (and most other positive traits), and this overconfidence can lead to unnecessary risk taking and errors (Camerer & Lovallo, 1999). Error management training was used in two driving simulator tasks where 80 participants practiced various driving tasks (Ivancic & Hesketh, 2000). Ivancic and Hesketh (2000) found that individuals given error management training (compared to error avoidant training) made fewer mistakes (such as going through red lights and speeding) and tended to repeat errors made in training less often during a performance trial. 11 Furthermore, the individuals exposed to error management training displayed more accurate levels of confidence on a post-training measurement demonstrating that exposures to errors can help rectify inflated perceptions of ability that people may enter training with. Because team performance is a result of the actions and interactions of individuals within the team, similar outcomes may be found in team performance when exposed to error management training. Meanwhile, many medical and aviation training studies have demonstrated error management training is beneficial compared to error avoidant training (Helmreich, 2000; Sexton, Thomas, & Helmreich, 2000). Work in medicine and aviation share a common theme: errors can cause catastrophic consequences. The environments are complex. Successful performance involves high levels of expertise, highly technical tools and equipment, and coordination among multiple agents to accomplish collective outcomes. In medicine, human factors researchers created an error management model designed to systematically account for errors and provide a strategy to apply when errors emerge during operations (Helmreich, 2000). They classify errors as threats and identify both overt (i.e. environmental factors, organizational factors, individual factors, team factors, and patient factors) and latent (i.e. national culture, organizational culture, and professional culture). Many medical training programs use simulation-based training to purposefully expose novices to errors, allowing them to explore workable solutions without the risks and expenses of real-world training environments. One example of error management training conceptual framework is the Threat and Error Management model (Helmreich & Musson, 2000). This model was created as an adaptation based on aviation error training to be used to train medical personnel. It defines any source that increases the likelihood of an error as a threat. It further categorizes both these sources of errors 12 as overt (i.e. environmental, organizational, and team) and latent (i.e. culture and policies) threats. From this step, Helmreich and Musson (2000) suggest common responses tailored to the type of problem. As another area that has used error-related training principles, the aviation industry has made many strides to incorporate error management into their training and day-to-day operations. For instance, Thomas (2004) analyzed the performance of a Southwest Asian airline and determined that threats and errors are major sources of decreased performance in the day-to- day activity of airline crew. He summarized both contextual factors and nontechnical skills (both cognitive and interpersonal skills) related to improved performance in the face of errors. Similarly, Helmreich (2000) identifies errors caused by fatigue, workload, overloaded attentional demands, poor team communication, imperfect information processing, and imprecise decision making. He then reviews a host of error management tactics used by the professional aviation agencies, suggesting a view of errors as inevitable. This leads him to conclude that errors must be managed and systematic efforts (such as training) should be used to reach desired outcomes. One area where researchers and practitioners have applied error-related principles in aviation is crew resource management (CRM). CRM is a training protocol in which crew members use a training protocol highlighting threats and risks which can commonly lead to errors. CRM training emphasizes how to manage errors in critical situations at both the individual task level and at the team level. Other industries, including medicine, offshore oil production, and nuclear power, have adopted CRM training approaches based on the success in aviation training (Salas et al., 2006). These efforts support the idea that organizations acknowledge the impact of errors. Many industries now use error taxonomies and error 13 management strategies in both training and active performance to aid employees in both team work and task work. CRM training literature is useful to review when considering the role of error management training to a more generalized theoretical perspective, but there are important distinctions that must be considered. First, a recent review of CRM training in applied settings concluded that this type of training lead to positive attitudes across studies, but showed mixed results in learning of actual competencies (Salas et al., 2006). For instance, among five studies across multiple industries that measured changes in skill, three found mixed results (i.e. various positive, neutral, and negative outcomes), one found neutral results, and the last study found negative results. Additionally, part of CRM training addresses power distance and the impact of cultural collectivism on team member communication and coordination. For example, a janitor may be unlikely to share important information about an error with a captain due to social norms related to speaking to people of higher rank. Similarly, a captain may disregard the request to share information from a lower ranking team member altogether (Helmreich, Merritt, & Wilhelm, 1999). This highlights the complex team dynamics potentially present in environments where errors can impact team processes and performance. While this may be useful to consider in specific practical contexts where power distance negatively impacts performance, it is beyond the scope of this study to examine the issues of power distance and cultural collectivism related to error management training and adaptive performance. Instead, this study will look at teams comprising of members with similar power distance and focus on performance outcomes. Although external leaders may exert their influence by allocating resources, monitoring behaviors, and adjusting performance (Zaccaro, Rittman, & 14 Marks, 2002), many teams operate under shared leadership principles (Carson, Tesluk, & Marrone, 2007) because many team work contexts rely on members who have high levels of expertise to apply their skills in autonomous ways. Katz and Kahn (1978) suggest that team members spontaneously volunteer their attention and resources towards shared goals and this shared leadership function can free up organizational resources as a benefit of this kind of team activity. This view of team leadership and organization further suggests that power distance isn’t inherent in all teams. In fact, team performance isn’t inhibited by decentralized, less formal leadership, and in some cases, has been shown to improve team performance (Mehra, Smith, Dixon, & Robertson, 2006). So, explicit leadership and power distance concerns are not necessary to attend to during error management training, even at the team level. Since the initial studies used to demonstrate that error management training is beneficial to achieving learning objectives, more nuanced studies of error management have expanded our understanding of error management. In two instances, studies looked at younger (17 – 20 years old) participants (Nordstrom, Wendland, & Williams, 1998) and older (40 years old or more) workers (Chillarege, Nordstrom and Williams, 2003) to examine the question of how error management training and type of training goal affected performance. The findings demonstrated that participants in the error management conditions sought out more help from others and improved performance overall in both younger participants and older workers. A similar study by Carter and Beier (2010) demonstrated that older and more-skilled workers showed improved immediate and delayed performance in a low structure and error management condition compared to training without error management instructions. 15 Additionally, individual differences such as cognitive ability, conscientiousness, and openness have been examined since not all people use exploration and discovery as behavioral strategies when engaged in training (Eysenck, 1996; Snow, 1986). Gully and colleagues (2002) found that individuals with higher levels of openness had more positive training outcomes. This finding supports the notion that exploring behavior in environments that view errors positively can create new opportunities to learn and improve. Similarly, individuals higher in cognitive ability showed more effective training outcomes in error management training, such as greater task performance and greater self-efficacy. Finally, highly conscientious individuals showed lower self-efficacy in error management training, likely due to their natural discomfort with ambiguity and failure associated with error exploring behaviors. As discussed earlier, error management training is thought to promote adaptive transfer. By encountering novel situations during training, people are confronted with the need to use critical thinking skills such as evaluating current behavior effectiveness, exploring new possibilities, and adjusting their performance. For example, Keith and Frese (2005) explicitly trained participants to use metacognitive planning, monitoring, and evaluating along with emotional control techniques (i.e. regulating negative emotions through self-talk) during a computer task in addition to error management training and observed an increase in performance. They posited that the enhanced emotional control and increased metacognitive activity promoted during training would mediate the effects of the training. They found support that both hypotheses, enhanced emotional control and increased metacognitive activity, mediated the effects of training. Enhanced emotional control is a form of self-regulation. Self-regulatory processes can mitigate stress and reduce time-off-task due to performance anxiety and worry (Kanfer, Ackerman, & Heggestad, 1996) allowing individuals to 16 focus their efforts on the task on hand. Similarly, metacognition (i.e. planning, monitoring, and evaluating one’s progress towards a goal) acts as a sort of feedback mechanism that helps learners identify and manage their actions in relation to their task at hand and the external environment (Pintrich & De Groot, 1990). In environments with more learner control, less external structure, and less formal feedback meta-cognition is believed to be particularly useful as a source of information and feedback (Schmidt & Ford, 2003). Furthermore, Keith and Frese’s (2005) post hoc analyses revealed planning, monitoring, and evaluating as three subcategories of metacognition that predicted adaptive performance. As for the macro level, some research applied error management training principles to organizational culture. Van Dyck and colleagues (2005) describe error management culture as shared practices and procedures related to detecting errors, communicating and sharing error knowledge, and helping others in error situations. The sampled employees (i.e. workers and managers) from many distinct industries using an adapted questionnaire supplemented by interview statements to measure error management culture. From these methods, Dyck and company concluded that systematic organizational initiatives such as promoting lower blame and punishment along with higher empathy responses when individuals erred were present in some organizations. They found some support for the idea that introducing error management culture predicted profitability above and beyond the effects accounted for by past performance. They do acknowledge the difficulty of precision in measuring this construct at the organizational level such as the influence of broader economic factors and using managers as a proxy for worker perceptions during parts of their study. While this current study will not look at larger organizational culture, it is useful to see how error management training principles may fit into a larger organization that promotes similar values. 17 A meta-analysis of 24 studies by Keith and Frese (2008) found a significant positive effect (d = .44) of error management training. Keith and Frese also suggest that effect sizes were even larger (d = .56) when looking at post-training transfer (i.e. performance boosts measured after training is completed) compared to within-training transfer (i.e. performance boosts measured during training). Additionally, transfer of training to structurally distinct tasks (i.e. adaptive transfer) was higher (d = .80) than on similar tasks (i.e. analogous transfer). Both differences are valuable to the goals of organizations, since the environments are not static and adaptive performance is beneficial for long term success. Error management training has been shown to consistently enable improved learning and post training outcomes such as performance and scores on tests given after training (Keith & Frese, 2008) compared to other training styles such as proceduralized or error avoidant types at the individual level. Furthermore, it has been demonstrated that metacognitive behaviors such as planning and monitoring along with attitudinal factors such as positive error framing can positively impact the adaptive performance in future uses of skills acquired during error management training. With this in mind, applying these principles to team level training with attention to similar behaviors contributes incremental scientific understanding to the study of both training and teams. While task complexity is often poorly described and conceptualized, Liu and Li (2012) offer a synthesis of the construct’s use and a framework for understanding how to think about task complexity. They point out that for a task to be ‘objectively complex’ (Liu & Li, 2012, p. 557) there must be a property that is intrinsic to the task that has this feature. I would argue that task complexity is transjective, in the sense that the complexity results from an interaction between the objective features of the task (and broader environment or context) and the 18 subjective knowledge, skills, abilities, and other individual factors of an individual or team. It is neither wholly subjective nor wholly objective. It is possible to anticipate what a team or individual may experience as complex if enough is known about both the task and the individual or team. For instance, knowing the training of a group of soldiers and the details of a mission, one can often reasonably anticipate if the team has the requisite understanding and abilities to accomplish their objectives. This is even more likely to be the case in a simulation, where the environment is more controlled compared to the field. While task complexity is not an intrinsic property of a task, Liu and Li (2012) offer a set of dimensions that are often observed in the broader research that can help to identify common themes. In particular, they mention variety and size, which capture the idea of amount of components and task steps, ambiguity, variability and novelty, which both point to an unknown element of the state of things or a changing state of things as experienced by the individual or team, and action complexity, which encompasses the technical and physical requirements inherent in the task. By training in one environment with a particular goal and set of tasks used to accomplish the goal and then performing in a novel environment with more technical elements and changes in some of the features of the environment, one can think of the new environment as more complex in these ways. Furthermore, performance adaptation can be conceptualized in a more specific manner. Baard, Rench, and Kozlowski offer a definition in a review of performance adaptation as “cognitive, affective, motivational, and behavioral modifications made in response to the demands of a new or changing environment, or situational demands” (2014, p. 50). In their framework, the current study would be considered a domain-specific approach that is often used in the expertise and skill acquisition domains. This class of performance adaptation relies on 19 adapting through knowledge and skill and is often fostered in training with the goal to prepare individuals or teams for novel tasks, situations, or environments. They point out that the need for increased understanding in this area is relevant to team research and that understanding the nature of the processes, tasks, and characteristics can aid in the conceptual clarity of these streams of research. They use the input-process-output framework (Hackman, 1987; Ilgen, Hollenbeck, Johnson, & Jundt, 2005) for analyzing this phenomenon. Training is an input and adaptive performance is the outcome or transfer effect. By defining the task complexity using a framework such as Liu and Li’s (2012), the kinds of contexts where error management training principles may apply well may be seen more clearly. To examine the effects of training and transfer in novel contexts, team adaptive performance can be observed using this perspective on task complexity. With teams who have little direct experience prior to training that may be relevant, the training experience forms the basis of their skills and understanding in an environment or with a set of tasks. Changes in variety and novelty, such as adding new elements, rearranging elements, or modifying elements of the environment may increase the cognitive and coordinative demands on a team. Similarly, the situational demands many be increased by changing other features of the environment such as the technical ability, strengths and weaknesses, and behavioral patterns of competitive agents. Team processes and performance may require increases in communication and coordination as well as integration of new information with existing understanding of how to approach tasks and goals. This may highlight the role of errors and inefficiencies in team performance in adaptive performance cycles after training in a more stable and understood environment. 20 Diagram of Model Figure 1. Model of relationships of hypothesis at the team level Team Performance and Error Management Teams are increasingly utilized in organizational contexts. Teams are defined by their interdependent actions towards shared goals. Teams are often chosen to achieve success in organizational outcomes due to their ability to synergistically leverage the knowledge, skills, and abilities of multiple expert members to achieve collective goals in dynamic environments (Pulakos, Arad, Donovan, & Plamondon, 2000). Team training is a common strategy employed by organizations to improve team performance, therefore understanding the potential impacts of different training approaches, such as error management training, adds to the organizational science literature both practically and theoretically. As previously mentioned, aviation and medical training have already embraced error management paradigms and use errors as data to systematically prevent the impacts of errors on performance and outcomes. CRM training identifies and incorporates many team functions and errors: communicating errors to the rest of the team when they are noticed, discussing alternative 21 procedures, planning for contingencies, and monitoring and evaluating the situation and other team members’ activities. Aviation training can include adopting a non-punitive attitude towards errors, providing formal training in teamwork, and providing feedback on both technical and teamwork skills Little empirical evidence exists showing which team-related knowledge, skills, and abilities (KSAs) may be positively impacted by error management training. Wilson and colleagues (2005) suggest KSAs related to team behaviors that are important to high reliability teams (e.g. teams that operate in complex and potentially hazardous environments) should focus on anticipation, communication, and support. Teams with inclusive and supportive attitudes (i.e. psychological safety) are better able to communicate information about errors and inefficiencies to promote adaptive performance (Edmondson, 1996). As a result, teams use regulation processes such as coordination and monitoring of mutual performance as a form of higher-order regulation (Bell & Kozlowski, 2011) to execute actions towards goals (i.e. tasks and subtasks). This study focuses on information exchange related to errors and inefficiencies (facilitated by psychological safety and positive error framing)—specifically backing up behavior, monitoring, and planning as mechanisms for improved team functioning during action and planning phases through error management training. Druskat and Pescosolido (2002) assert that “heedful interrelating”, defined as awareness of interdependence among team members is critical to catching mistakes during active team performance. Additionally, planning phases of training will be used to promote error management-related values to ensure team members are acknowledging human errors and to reinforce the development of these shared behaviors and attitudes (i.e. psychological safety, monitoring, and backing up behaviors). 22 Psychological safety is defined as a shared belief that, within the context of the team, it is safe for members to take interpersonal risks (Edmondson, 1999). Implicit in this definition is the notion that the team shares mutual respect and trust among its members, and that team members will not embarrass, reject, or punish each other for speaking up. Previous research has shown that negative affect—such as fear of lowered social regard, negative self-image, or embarrassment— can cause team members to withhold information relevant to inefficiencies and errors. Fostering an environment of mutual respect can undue this fear (Edmondson 1999, 2004; Carmeli & Gittell, 2008). Improved quality of working relationships connects people with distinct, yet interdependent roles, allowing for increased communication and coordination. People can feel safe expressing what they think, including information about inefficiencies and errors made by themselves or other team members (Kahn, 2007). Hypothesis 1: Team error management training is positively related to average level of team psychological safety. Monitoring is defined as tracking progress towards goals by detecting and transmitting information relevant to team goals to other team members (Marks, Mathieu, & Zaccaro, 2001). Salas, Sims, and Burke (2005) assert that mutual performance monitoring is critical to catching mistakes as a precursor to applying an appropriate strategy to adjust team behavior. Monitoring is a form of intra-team feedback, functioning as an information exchange mechanism during action phases byusing other members’ attentional resources to examine other actions. This attention examines other members’ actions by comparing what they are doing to what they should be doing, then providing verbal and behavioral feedback when gaps are detected (Marks & Panzer, 2004). By focusing on monitoring, team members can identify potential sources of errors before they result in actual errors. Additionally, errors may be detected more often and 23 earlier, allowing the team to adjust their behavior accordingly. Team monitoring behavior has been shown to improve task mental model accuracy, which is known to positively relate to team performance (Kwei-Harh, Valker, Haerem, & Lervik, 2016). Backing up behavior is defined as one team member providing effort to another team member when it is apparent that an error is made or failure to reach an intended goal is imminent (Porter, Hollenbeck, Ilgen, & Ellis, 2003). It is considered another essential aspect of teamwork (McIntyre & Salas, 1995) because it is how monitoring behaviors are transformed into performance gains (Salas, Rosen, Burke, & Goodwin, 2009). By focusing on backing up behavior, team members are often able to facilitate adaptive performance and function more consistently and proficiently in complex environments. Mutual performance monitoring and backing up behaviors serve as a redundancy system, enabling teams to prevent and manage errors more effectively (Bell & Kozlowski, 2011). Hypothesis 2: Team error management training increases average team communication related to errors and inefficiencies. While it is beyond the scope of this study to measure cognitive processes, it is useful to briefly examine the underlying cognitive processes that may be responsible for explaining how error management principles are enacting positive changes in team behavior. Shared mental models are “common and overlapping cognitive representations of tasks requirements, procedures, and role responsibilities” (Cannon-Bowers, Salas, & Converse, 1990, p. 222). A meta-analysis by DeChurch and Mesmer-Magnus (2010) of 23 independent empirical studies of 1,511 teams (N = 5,668) showed non-trivial (r = .29) effects of shared mental models on team performance. 24 Research suggests that shared understanding of task and team responsibilities enables teams to identify and correct errors earlier (Salas, Sims, & Burke, 2005). In a sense, teams are living systems who encounter their environment like individuals in the sense that they work towards goals and continually process information related to their goals. When one member of the team communicates a need or shares information about an error, they are facilitating the rest of the team to anticipate what members need and enabling a common view of what is currently happening and what is likely to happen next (Mohammed, Ferzandi, & Hamilton, 2010). This allows teams to adapt to the demands of the task and environment, including errors from within the team. Team planning and aligning the understanding of taskwork within the team has also been shown to improve teamwork accuracy and taskwork prediction (Mathieu, 2000). For instance, Edmondson’s (1999) team learning model suggests many factors which contribute to team learning behaviors, such as seeking feedback, sharing information, and talking about errors. These behaviors (and others) allow teams to adapt to unexpected demands of the environment, which facilitates adaptive performance. As teams communicate effectively and frequently about how the team acts in its environment, performance is improved (Mohammed & Dumville, 2001). Adaptive Performance and Critical Thinking Teams in organizational contexts face more complex and uncertain environments due to globalization, technological change, and political and economic influences. This environment rewards teams in which adaptive behavior is fostered through training, leadership, and organizational climate. Adaptive performance is distinct from learning in two salient ways: 1) learning is essentially cognitive and 2) learning doesn’t necessarily lead to change (Burke et al., 2006). The increasing need to understand the antecedents of adaptive team performance has led 25 to many valuable insights in team adaptation (Rosen et al., 2011; Kozlowski, Watola, Jensen, Kim, & Botero, 2009; Burke, Salas, & Diaz, 2008; Chen, Thomas, & Wallace, 2005; Kozlowski, Gully, Nason, & Smith, 1999) In other words, understanding how to prepare teams for adaptive performance is of substantial concern to both academic research and applied work. Burke et al. (2006), after surveying previous models of adaptive performance, offer a model of adaptive team performance that conceptualizes team adaption in the context of time, emergent states, and multi-level perspective. Burke and associates review six commonly-used definitions of adaptability and then define adaptive performance as “an emergent phenomenon which complies over time … to functionally change current cognitive or behavior goal directed action” (p. 1192). Burke posits four phases (i.e. processes) teams engage in to achieve successful adaptive performance: 1) situational assessment, 2) plan formulation, 3) plan execution, and 4) team learning. Relevant to this study, plan execution (phase 3) involves monitoring, communication, and backing up behavior. Burke argues that these behaviors lead to improved team situational awareness and improved shared mental models. Because this cycle is recursive, not only do teams have the opportunity to manage errors in their performance during the execution phase, but also during the learning phase, as well as the next assessment and formulation phases when they have the time to reflect on their recent performance and share insights with each other. Members of teams use their pooled resources (including information gained from monitoring and information known about errors) to modify their behaviors according to contextual requirements. Additionally, teams may identify errors and adjust their behaviors during a planning phase between action phases (Marks, Mathieu, & Zaccaro, 2001). Self-correction during 26 planning phase allows teams to diagnose errors during previous performance and identify specific knowledge to use during future performance episodes (Kozlowski et al., 1996). Prior research has established a link between increased psychological safety and increased adaptive performance (Hirak, Peng, Carmeli, & Schaubroeck, 2012; Baer & Frese, 2003, Van Dyck, Frese, Baer, & Sonnentag, 2005). For instance, Hirak and associates (2012) observed significant positive correlations among psychological safety, learning from failure, and performance at both the team and individual level. Based on work by Edmonson (1996), they reasoned that teams need to be able to express themselves without fear of negative consequences (i.e. ridicule or punishment). Teams who share information more freely and more openly are able to address errors and inefficiencies without the constrain of fear of stigma or reprisal. This in turn allows teams to adapt in the face of errors and inefficiencies observed by the team and leverages the cognitive resources (i.e. monitoring and backing up behaviors) of the team more completely. Hypothesis 3: Team error management training is positively related to perceptions of team learning. Hypothesis 4a: Team communication is positively related to perceptions of team learning. Hypothesis4b: Team communication mediates the relationship between team error management training and perceptions of team learning. Hypothesis 5a: Team psychological safety is positively related to perceptions of team learning. Hypothesis 5b: Team psychological safety mediates the relationship between perceptions of team learning, and team error management training. 27 Hypothesis 6a & 6b: The impact of team psychological safety (a) and team communication (b) on team adaptive performance are mediated by perceptions of team learning. Although there is no agreed upon definition of critical thinking in organizational psychology literature, organizations view individuals and teams as having knowledge, skills, and abilities (KSAs) which are resources targeted for training with the goal of transferring the training experiences to improved future performance (Blume, Ford, Baldwin, & Huang, 2010). Error management training, by providing learners with additional knowledge of errors and their consequences, seems promising as a way to expose teams to errors and error management principles that will improve their ability to perform in novel, challenging contexts better than other types of training. Teaching teams to view errors as a source of feedback, not failure, and promoting monitoring behavior capacities of team members to increase the capacity to notice errors and communicate this information in a timely manner should create more accurate and up- to-date mental models, which in turn allow teams to function despite mistakes or precursors of mistakes. Since this training style is often not the natural focus of training, there is an opportunity to improve training environments for future teams. Acceptance of failure and openness to experience are critical to learning, growing, and thriving in complex real-world environments. Research Justification This study attempts to demonstrate that four key aspects of the error management training intervention are linked to observable changes in key team attitudes and behaviors: shared psychological safety, shared perceptions of team learning, monitoring and communication, planning, adjustment, and backing up. 28 First, setting expectations that team members’ perspective, actions, and communication are valued, along with encouraging individuals to value learning from errors and sharing observations relevant to other team members’ roles can increase positive shared attitudes about these behaviors and events. By framing the environment as a psychologically safe place to think and act, teams are more able to feel supported and give support to other members. Additionally, encouraging teams to consider errors, inefficiencies, and failures during planning phases can result in increased related behaviors during the planning phase of performance. This behavior should allow teams to have improved foresight, develop contingencies, and create more effective plans for future performance. Also relevant, this exploring behavior is a component of critical thinking, leading to improved adaptive behavior in new and challenging situations. Second, encouraging teams to monitor for and communicate about potentially avoidable actions, inefficiencies, and failures which may cause the team to fail to achieve a goal can result in increased monitoring and communication behaviors related to errors, inefficiencies, and potentially avoidable actions. These behaviors should improve overall team performance by allowing teams to adjust to these events. Third, encouraging teams to view errors as something positive reduces stress and harmful associated negative feelings, such as fear of judgement and loss of status among teammates because of sharing information about errors. The increased awareness of the value of sharing errors and the removal of negative connotations which otherwise would motivate team members to withhold information will lead to improved communicating behaviors of the team, which are necessary for awareness and adjustment of team functions. 29 Fourth, encouraging backing up behavior allows teams to use inactive resources or divert active resources to more impactful actions during team processes, which will increase overall performance when encountering potentially avoidable actions, inefficiencies, and failures. 30 Simulation-based Training METHODS Simulation-based training is a type of training medium that uses computer simulation to create virtual simulations of environments. It allows people to experience immersive, real-world situations for a low cost and in a risk-free environment by generating a lifelike computer experience that mirrors the future environments of users. Simulation-based training allows trainers to control and modify almost any element of the experience, making it ideal for error- based training (Rosen, Salas, Wilson, & King, 2008). Trainers can embed errors within the training curriculum and allow trainees to explore the environment without jeopardizing safety or company resources. Additional opportunities to try new behaviors and adapt to challenges and mistakes have the potential to increase learning and encourage an attitude of errors as a positive source of information. Additionally, simulation-based training allows for more data to be captured due to the environment existing within a computerized version of reality. Researchers can more objectively compare the experiences of different actors in different situations and observe the details of events unfolding in minute detail. Furthermore, because of the higher degree of control and creation, mistakes can more easily be embedded in situations that are digitally designed. Increasing the task characteristics variables such as complexity, interdependence, and skills central to success allows for systematic exploration of how individuals and teams function during training. In this experiment, Artemis Space Bridge Simulator (Artemis) will be used as the training environment. Artemis is a space mission simulator in which a team of 2 to 6 individuals pilot a spaceship to accomplish various missions, such as rescuing space outposts from attacks, 31 negotiating hostile alien surrender, or combating alien attacks. Despite the imaginative nature of the game, each mission involves using a diverse array of skills parallel to real world team functioning. Task work such as managing limited resources, deciding among various choices of action to accomplish goals, planning future actions, and performing routinized procedures is used throughout each ‘mission’ of gameplay. Teamwork is also essential for success. Each member has information about various aspects of the ship’s functioning beyond the scope of their own duties and knowledge of the other roles of the team. In this training exercise, a team of three people will take on the roles of a) helm, b) weapons and engineering, and c) science and communication. The helm steers the ship. Weapons and engineering selects which weapon will be used, who it will be fired at, and can see the various ship systems’ status and adjust these systems. Science and communications identifies friendly and hostile ships and space stations, sends and receives messages, and can get enemy ships to surrender. Many of the actions of the game involve interdependent actions (such as communicating with, docking a ship at, and reloading munitions at a friendly space station) and many of the ship’s activities can be observed by the main monitor, regardless of position. This allows for skills such as monitoring, backing up behavior, coordination, and communication to impact the success and failure of the team. Participants This study was conducted on 100 teams consisting of 3 team members per team (50 teams per condition). Three teams were removed from the final analysis because of technical issues during the experiment or incomplete data capturing, leaving the rest of the teams (a total of 291 students) for analysis at the group level (N = 97). A power analysis was conducted (Soper, 2017) for an a priori recommended sample size of 91 teams with an effect size of .15 (Keith & Frese, 2008). This estimate is similar to other studies similar to the present study. Each 32 team consisted of 3 team members (3 participants). Participants were recruited using the SONA research pool system, earning course credit as compensation. The demographics of participants consisted mostly of undergraduate psychology students from a large midwestern university and represented at typical mixture of age, sex, and ethnic/racial composition. The roles of the participants and the condition were randomly assigned using computer-generated random numbers for each of the trials. The average age of each participant was 19.48 years (SD = 1.37) with 38.83% male and 61.27% female. The class standings of participants were 38.83% freshman, 37.80% sophomore, 31.27% junior, 20.97% senior, and 0.10% identifying as none of the above. The average GPA (self-reported estimate) was 3.35 (SD = 0.62) and the average amount of weekly videogame play (self-reported estimate) was 4.25 hours per week (SD = 1.05). Materials Participants in both conditions had access to a computer station with a computer and an 8” x 12” summary of key aspects of their role and the roles of the rest of their team. The error management condition also had multiple 2’ x 3’ posters within view of the computers with positive messages on the role of errors to success. A planning area with a map was within view of the computer stations for both conditions. Procedure Two conditions were compared. The first condition was the control training condition, which lacks any error management training elements, and received general training on game play and group functioning as well as individual training on the role(s) a member was responsible for within the game. The second condition was the error management training condition, which had the same elements as the control training condition, with the addition of the posters. Each session 33 of the study took approximately 2.5 hours. Individuals in both conditions began by viewing training videos on their assigned roles as well as shorter videos on the roles of the rest of their team. Following the videos, participants were given a quiz to assess their understanding of the material, with feedback given if they failed to demonstrate understanding of any part of the training. Participants could not exit the training until demonstrating competency on the review questions at the end of the training material. Participants recorded basic demographic information including their prior experience playing video games, particularly space simulators and teams-based games. Participants also self-reported measures of current grade point average and admissions tests score approximations and completed a personality questionnaire to allow for post hoc analysis of individual differences. The error management group also received explicit instructions on the positive value of errors and was encouraged to look for errors, share error information, and monitor and back up team mates when errors or inefficiencies were observed. No additional error training was given to the control condition. To ensure equal time between groups, the control group was given information about the ship they would be piloting, but this information was not relevant to the skills involved in the experiment. After collecting background data and completing the training videos, participants were given instructions on a 20-minute training mission (mission #1). All members of each team were offered up to 5 minutes to plan their strategy at a “planning area” with a map and a black marker, which they were told they may use during the planning time. The error management group received explicit instructions on the positive value of errors and was encouraged to discuss potential errors, contingencies if errors or inefficiencies were encountered, and past errors and inefficiencies from the previous mission that may be relevant to 34 success in the following mission. Specifically, as part of the error management training process, the error management training group was told multiple times throughout the training session and two training missions to “care about errors and inefficiencies” and that “errors and inefficiencies are a positive source of information” to instill a sense of importance about errors and inefficiencies and to prime the team to notice this kind of information and events. Participants then completed the 20-minute training simulation. Following the training mission (mission #2), participants were instructed on a 20-minute mission objective and given the same planning opportunity as before. The second mission had the same goals and parameters as the first mission, serving as an opportunity to reinforce skills described during the training material. After the second mission, participants were instructed on another 20-minute mission (mission #3), with an objective distinct from the first two missions. They were given the same 5- minute planning opportunity as well. During this mission, resources were reduced, and hostile enemies behaved differently (including using more challenging strategies) than during the first two missions in order to capture adaptive performance. These changes represent an increase in task complexity in which the team must adapt their behaviors in response to this complexity in order to perform well. Under the framework offered by Liu and Lu (2012), the dimensions of size and variety can be seen in the features of the environment which are more numerous, in different locations, and in different relationships to each other. For example, the game zone created more neutral objects (i.e., debris) and had a different configuration for the friendly bases and enemy ships in the final mission. In terms of novelty and variability, the enemies were set to a higher in game difficulty level, resulting in new patterns of behaviors in the context of movement and engagement with the player team. This in turn created a need for more complex 35 actions. In other words, the technical performance of the team needed to be higher to see the same level of success as in the training missions. Upon completing the third mission, each participant completed additional surveys to measure training reactions. The first two missions were explicitly framed as the training phase and the third mission was explicitly framed as the performance phase. Measurement Communication Each team member was responsible for his or her own role as well as the overall success of the mission. Communication was operationalized as the number of times a team member communicates information to another team member relevant to a role outside of their own. For instance, the science and communications officer suggesting to the weapons and engineering officer “the upcoming ship is an enemy craft” during an encounter with an unknown ship would count as an act of monitoring behavior. Each communication utterance related to errors or inefficiencies was recorded by a trained observer as well as captured by audio recording devices during each of the planning phases and mission phases for a total of 6 observation periods. During the planning phase, the total time used by the team (up to 5 minutes) was recorded. The team was informed they have up to 5 minutes to plan for each mission. They were also informed that they may terminate the planning phase at any time before the five minutes. This is consistent with other measures of communication related to backing up behavior and monitoring in similar research (Porter et al., 2003; Barnes et al., 2008). To calculate the score used in the model, three steps were done. First, A random sample of 20 percent of the recorded audio from all six phases was completely transcribed. It was then used to create five categories for the number of communications about errors and inefficiencies 36 from 1 ("No communication about errors or inefficiencies") to 5 ("Very frequent communication about errors or inefficiencies") for each of the missions and planning phases. Next, raters were trained based on ten examples to identify and rate audio of team communication. Then raters listened to each audio file and assigned a score to each item. Finally, 20% of the communication logs were scored by two raters to ensure consistency in ratings, resulting in 90% interrater reliability (100% of ratings were within 1 value when scored by two raters). The average of each of the scores was computed for each team and used in the model. Psychological safety Perceptions of shared psychological safety were measured using the 7-item scale (α = .82) developed by Edmonson (1999). To assess whether there was agreement within members of a team, an estimate of within-group interrater reliability rWG was calculated (Baer & Frese, 2003; James, Demaree, & Wolf, 1984). The within group agreement for psychological safety was rWG = .91, which is considered strong team agreement (LeBrenton & Senter, 2008). A follow up analysis to detect outliers in agreement was conducted and no groups were below the threshold of rWG = .70, indicating a lack of outliers (Liu, Amini, Babakus, & Stafford, 2011). Additionally, ICC1 and ICC2 were calculated. For team psychological safety, there was good support to aggregate ratings to the team level, F(94,190) = 1.62, p < .001 (ICC1 = .17, ICC2 = .38). Team learning Perceptions of shared team learning were measured using a 15-item scale (α = .93) developed by Savelsbergh and colleagues (2009). To assess the level of agreement within members of a team, an estimate of within-group interrater reliability rWG was calculated. The within group agreement for perception of team learning behaviors was rWG = .89, which is considered strong team agreement (LeBrenton & Senter, 2008). A follow up analysis to detect 37 outliers in agreement was conducted and six groups were below the threshold of rWG = .70 (.43, .44, .51, .69, .69, .69), indicating six potential outliers (Liu, Amini, Babakus, & Stafford, 2011). Because the overall rWG values were high and the differences in standardized path coefficients of both models tested were low (less than 1% difference in actual estimate in both models), all outliers were included in the final analysis. Additionally, ICC1 and ICC2 were calculated. For team learning, there was good support to aggregate ratings to the team level, F(96,194) = 1.67, p < .01 (ICC1 = .18, ICC2 = .40). To verify that the constructs of team psychological safety and team learning were conceptually distinct, a factor analysis was conducted. The scree plot showed inflexion points at 4 factors, and a four-factor model was used to extract four components, which accounted for a combined 64% of the variance explained. Factor one and two included all of the items except one from the team learning measurement. Savelsberg and colleagues’ (2009) original scale suggested multiple dimensions and the current factor one loads the items on error communication and error analysis onto one factor and reflection on outcomes and reflections on processes onto the second factor. A third factor contains all but one of the psychological safety measure items and the fourth factor contains two items, one from each survey. The fourth factor is not interpretable, leaving three distinct factors, psychological safety and two dimensions of team learning (reflective and error related). These three factors will be used to examine the models and hypothesis. This demonstrates that the two constructs, team learning and team psychological safety, are conceptually distinct. See table 3 in Appendix E for the complete correlation matrix of standardized loadings. 38 Adaptive performance and efficiency Teams that can take their training experiences and apply them to effectively meet novel, difficult, and complex environments compared to the training environment are more effective(Chen, Thomas, & Wallace, 2005). Teams were informed that mission 3 was the performance mission. Team scores on the third mission were used to measure adaptive performance. Information about mission performance scores appeared on the computer screen after completing all mission goals and was recorded. Team adaptive performance efficiency was calculated at the end of the final mission for each team by creating a ratio of the primary and secondary weapon usage to the number of enemy targets destroyed or surrendered (i.e., a lower value results in more efficient performance of the task during the mission, which was stated as a shared objective by the proctor before the start of each mission). Enemy ships were destroyed based on the following parameters: they had “health” points, which much be reduced to zero, they could be engaged by directly aiming at them with ship weapons or at less optimal angles of engagement, which do only a percentage of weapon damage, and they were damaged less the further away they are from the attacking ship. Enemy ships could surrender (i.e., they cease fighting) if they had been damaged and the communication officer requested a surrender. Their chance to surrender upon request was increased based on the amount of damage they sustained at the time the surrender request was initiated. Teams needed to modify their behaviors to succeed in this new mission because of the increased task complexity to succeed on the final mission. For example, teams had to update their understanding of the requirements to destroy an enemy ship, since the enemies had increased defenses and increased tactical abilities, which may have surprised teams at first. Similarly, teams encountered enemies in a new map, with different amounts (i.e., variability and size) of friendly, neutral, and hostile elements. These changes in 39 task complexity required more observation and incorporation of these new elements and information to successfully navigate the ship, engage with enemies, and manage resources. The changes may have also caused teams to doubt their understanding of other elements of the mission, taxing them cognitively and motivationally in this way as well. It should be noted that the dependent variable in this study was efficiency and not just number of enemies killed or surrendered. We can conceptualize performance by considering that an agent or team of agents has a stated goal. In the pursuit of their goal, they apply their skills, abilities, and knowledge towards moving closer to and ultimately attaining the stated goal. In this sense, inefficiencies are a sort of error in that they are actions that result in suboptimal performance. In teams, team members can observe this performance, comment on it, and the team can adjust either in the moment, or during a subsequent performance. This more broadly construed definition of errors emphasizes increased efficiency, including a more effective use of resources based on observation, team correction, and learning. Therefore, this study focuses on efficiency as its dependent variable rather than a just looking at the number of enemies destroyed or surrendered. This aligns with the stated goals each team was given during both of the training missions as well as the final adaptive performance mission that highlighted a focus not only on the destruction and acquiescence of hostile agents, but also on overall efficiency throughout the mission. 40 RESULTS Table 1 displays the intercorrelation matrix for condition (error management or control), team past general gaming experience, team past space gaming experience, team average SAT score, team psychological safety, team learning (error related factor), team learning (reflective factor), team communication, final mission performance, team training performance efficiency, and adaptive team performance efficiency. Table 2 displays the means and standard deviations for the error management condition and the control condition for each of the items in the intercorrelation matrix. Upon examining adaptive team efficiency and training team efficiency, the data was shown to be right skewed data and was log transformed for all analysis (Manikandan, 2010). Untransformed values for this variable are reported in Table 1 and Table 2. Hypotheses Hypothesis 1 stated team error management training will be positively related to average level of team psychological safety. An independent samples t-test was conducted to compare the team psychological safety level between the error management condition and the control condition. The results of the t-test, t(94) = -3.67, p < .01, demonstrate a difference in team psychological safety between the control condition and the experimental condition, showing support for hypothesis 1. Hypothesis 2 stated team error management training will increase average team communication related to errors and inefficiencies. An independent samples t-test was conducted to compare the team communication level between conditions. The results of the t-test, t(94) = - 1.16, p = .25, showed no statistically significant difference between the amount of communication between groups, showing a lack of support for hypothesis 2. 41 Hypothesis 3 stated team error management training is positively related to perceptions of team learning. An independent samples t-test was conducted to compare the perceptions of team learning level for both factors, between the error management condition and the control condition. The result of the t-test for the team learning error related factor is t(94) = -2.66, p < .01, and the result for the t-test for team learning reflection factor is t(94) = -2.18, p < .05. This demonstrates a difference in perceptions of team learning between the control condition and the experimental condition, showing support for hypothesis 3. Hypothesis 4a stated team communication is positively related to perceptions of team learning and hypothesis 4b stated team communication mediates the relationship between perceptions of team learning and team error management training. A mediation analysis was conducted on both learning factors, following the recommended steps by Baron and Kenny (1986). In step 1, the regression of perceptions of team learning error related factor on error management training, ignoring the mediator, was significant, b = .37, t(94) = 7.87, p < .01. In step 2, the regression of perceptions of team learning error related factor on the scores of the mediator, team communication, was significant, b = .31, t(94) = 31.99, p < .001. In step 3 of the analysis, controlling for the mediator, team communication, error management training was still a significant predictor of perceptions of team learning error related factor, b = 0.30, t(94) = 20.02, p < 0.01, indicating partial mediation. For the second factor of reflective aspects of team learning the same process was conducted. In step 1, the regression of perceptions of team learning reflective related factor on error management training, ignoring the mediator, was significant, b = .31, t(94) = 5.30, p < .05. In step 2, the regression of perceptions of team learning reflection related factor on the scores of the mediator, team communication, was significant, b = .27, t(94) = 21.95, p < .001. In step 3 of 42 the analysis, controlling for the mediator, team communication, error management training was still a significant predictor of perceptions of team learning error related factor, b = 0.26, t(94) = 13.20, p < 0.01, indicating partial mediation. This shows support for hypothesis 4a and 4b. Hypothesis 5a stated team psychological safety is positively related to perceptions of team learning and hypothesis 5b stated team psychological safety mediates the relationship between perceptions of team learning and team error management training. A mediation analysis was conducted testing both factors of perceptions of team learning, following recommended steps by Baron and Kenny (1986). In step 1, the regression of perceptions of team learning related to errors on error management training, ignoring the mediator, was significant, b = .37, t(94) = 7.87, p < .01. In step 2, the regression of the perceptions of team learning related to errors on the scores of the mediator, team psychological safety, was also significant, b = .46, t(92) = 11.67, p < .01 . In step 3 of the analysis, controlling for the mediator, team psychological safety, error management training was still a significant predictor of team learning, b = 0.36, t(91) = 8.09, p < 0.01, indicating partial mediation. For the second factor of reflective aspects of team learning the same process was conducted. In step 1, the regression of perceptions of team learning reflective related factor on error management training, ignoring the mediator, was significant, b = .31, t(94) = 5.30, p < .05. In step 2, the regression of perceptions of team learning reflection related factor on the scores of the mediator, team psychological safety, was significant, b = .40, t(92) = 8.45, p < .001. In step 3 of the analysis, controlling for the mediator, team psychological safety, error management training was still a significant predictor of perceptions of team learning error related factor, b = .18, t(91) 5.09, p < 0.01, indicating partial mediation. This shows support for hypothesis 5a and 5b. 43 Additionally, in the final model below (Model 2; see Appendix F) one thousand boot strapped samples were taken to compute the 95% confidence interval for the indirect effects of error management training on team learning through team psychological safety did not include zero (95% CI = [.29, .68]). This analysis provides additional support for hypothesis 5a and 5b. Hypothesis 6a and 6b stated the impact of team psychological safety (a) and team communication (b) on team adaptive performance are mediated by perceptions of team learning. A regression of the team adaptive team performance on the scores of team psychological safety was tested and found to not be significant, b = 0.01, t(92) = 0.01, p = 0.91. A regression of the team adaptive performance on the scores of team communication was tested and found to not be significant, b = -0.04, t(94) = 1.23, p = 0.27. This shows a lack of support for hypothesis 6a and 6b. The overall model (Model 1) proposed for this study was tested using path analysis techniques with the R lavaan package (Oberski, 2014). Model fit indices were calculated (χ2 = 5.426, df = 2, p = .066; CFI = .939; SRMR = .051; RMSEA = 0.135), indicating poor overall model fit. Next, a path analysis model (Model 2; see Appendix F) was tested in which team psychological safety mediates the relationship between both team learning and error management condition while team adaptive performance is impacted by both direct effects from error management training and mediating effects of both team learning factors. This model also includes training efficiency and number of enemies destroyed or captured, both variables acting as control variables to assess more accurately the variables of interest. Model fit indices were calculated (χ2 = 125.06, df = 9, p < .01; CFI = .317; TLI = -.517; SRMR = .167; RMSEA = 44 0.376[95% CI = .319, .436], indicating poor fit. All reported values in Figure 2 (Model 2; see Appendix F) are standardized fit estimates. An additional analysis was conducted because analysis with the two factors resulted in a non-significant relationship between the reflection factor and the team error management condition (p = .34) and the adaptive performance variable (p = .10) when examining Model 2. This was removed from the model and another path analysis was conducted with the error component of team learning only. Model fit indices were calculated (χ2 = 11.285, df = 6, p = .080; CFI = .901; TLI = .766; SRMR = .072; RMSEA = 0.098[95% CI = .000, .186], indicating improved fit (Kline, 2005; Hooper et al., 2008) based on criteria of CFI > .90, and χ2 p-value > .05 and the 95% confidence interval for RMSEA including zero. All reported values in Figure 2 (Model 3; see Appendix F) are standardized fit estimates. 45 DISCUSSION Error management training principles have been shown to be positively correlated with individual performance across multiple contexts including simulations, computer tasks, and other areas of performance (Keith & Frese, 2008). The current study shows error management training principles may be applicable at the meso level to positively impact teams who perform in environments where errors and inefficiencies can be costly to team effectiveness. Because error management training has not been systematically studied in laboratory or applied settings at the team level in the past, the current study may offer insights into the role of team learning and positive error framing on team efficiency and adaptive performance. Like studies at the individual level, error management training literature suggests that an environment promoting exploration can promote learning and transfer by encouraging people to view errors as relevant information. As teams practiced their skills in training in an environment that promoted the incorporation of errors into their experiences, members may have experienced more thorough learning and felt more inclined to explore their roles and the environment. This fits with the notion that teams who experiment and explore are enacting behavior which may lead to a broader range of skills and alternative responses useful in novel situations (Kozlowski & Bell, 2008). Despite the lack of increased quantity of communication in the present study, there may have been qualitatively different communication patterns that were impacted by a more psychologically safe environment (Edmondson, 2003), leading to improved coordination or efficiency. As the world of work increasingly becomes more team-focused and filled with the need for adaptive performance in response to uncertainty and environmental complexity, approaches to training and efficiency which may benefit these contexts may be increasingly 46 relevant. Understanding the strengths and limitations of error management training can benefit science and practice alike. In the context of the simulation software used, many types of common errors and inefficiencies are possible across all of the roles. For example, the ship navigator could dock the ship slowly when attempting to resupply the weapons payload. This information would be available to the entire team, since all members have access to the map and the speed, direction, and location of the ship. Noticing this error can be accomplished by any member of the team and communicated to the member whose role it is to pilot the ship, allowing them to adjust in the moment or at the next attempt. Another example of team relevant error or inefficiency may be in how the team attacks an enemy. The weapons are more effective when the ship is aimed directly at the enemy and when the weapons are fired in specific distances from the enemy ship. Any team member may notice these patterns of engagement, how the ship is currently positioned, and other details relevant to maximal use of weapons, since all team members are both given this information during training and also may notice the effects throughout their experience in the simulation. Then, they may share their insights into how the team is engaging with the enemy, either in the moment or during less active times (such as the planning phase), resulting in the opportunity to increase efficiency. Limitations One methodological limitation of the current study is that the simulation was primarily created as an entertainment videogame, not to train individuals or teams. While the two missions used in the study may have been more engaging due to this factor, it differs from typical training simulations in this respect. This may impact the generalizability to team training contexts in many ways. For instance, there are many features of the missions, such as anomalies and 47 background debris, which serve to make the game more immersive, but would sometimes confuse teams because they were only briefly discussed in training due to their irrelevance to the roles of each participant. Similarly, there were friendly bases that teams could dock at to replenish weapons and gain information, but this was not central to the training or necessary for the success of either the training or adaptive mission. Future studies should control for features not central to the research questions involved to minimize distraction and undesired complexity. Another methodological limitation of this study is the use of undergraduates as participants, which may not generalize to an adult workforce (Henrich, Heine, & Norenzayan, 2010). A typical undergraduate participant will not only vary in age from an average worker, but also in life experiences (including work history), skills, and motivation. The participants in this study were working for course credit and may have lacked the serious attitude a person may feel towards work training in both ad hoc and real teams. Secondly, this study employed multiple survey measures (i.e., self-report), which may be impacted by common method bias. One attempt to reduce this source of error was to use behavioral measures (e.g., audible communication) and objective performance measurements (e.g., in-game resource allocation, timing, and goal completion). Future research using behavioral operationalization for team training outcomes (i.e, performance, affect, and cognition) may allow for a more accurate assessment of participant affective and cognitive responses to team error management training. Thirdly, this study was conducted in a lab using a series of events that took approximately two hours to complete from the start of training to the adaptive transfer of training skills. In the workforce, the time length between training and application is often much longer and consists of periods of non-work as well. This study demonstrates that performance after a 48 short period of training may be impacted, but long-term consequences of error management training need to be explored. Nevertheless, past error management research has used similar protocol to incorporate post training and adaptive events (Frese & Keith, 2015). An additional limitation of the current study is lack of previous empirical research using the error management training principles to inform the study design. While theoretical insights as to the source of errors detection and interpretation relevant to the team level or the amount of signal to noise ratio in the environment (Kiesler & Sproull, 1982) have been given cursory analysis, the underlying processes and psychological states at the team level are largely lacking empirical support. For instance, when a manager mis-frames nonproblematic events or objects in the environment as a problem, this could possibly encourage wasteful acts by the team before identifying the correct assessment of the context. The degree to which error management training frames and reinforces team orientation towards errors, the amount of coordination required to adjust to errors once they have been discovered and shared with the team (i.e., brainstorming, expanding attention, and evaluation), and task complexity (i.e., relevant skill or expertise required) are all areas for future exploration (Hammond & Farr, 2011). Practical Implications High risk organizations and organizations that already promote climates and cultures of the positive values of errors may benefit from incorporating error management principles into their training practices. The present study may clarify and corroborate that these ideas can benefit the performance of not only individuals, but also teams in the pursuit of increased performance. The data also supports the idea that error management training principles may increase shared perceptions of team learning by highlighting the use of errors as a valuable source of 49 feedback. Although the data did not support the relationship between psychological safety as a mediator between the training style and team performance directly, promoting psychological safety did relate to the error management training condition, which may have relevance to other more distal performance or organizational goals. Teams in complex and novel environments as well as teams in environments where adapting to errors may improve team functioning can elicit the error management perspective to see improved team performance. Future Directions As teams continue to perform core functioning in organizations, the impact of training protocols has increased potential. As work becomes increasingly complex, the impact of errors and inefficiencies, and remedies to these things, can be a competitive advantage to organizations who capitalize on novel training approaches. The current study provides empirical support for the notion that error management training is related to increases in team adaptive performance. Furthermore, the construct of team learning seems to play a role in facilitating this improvement. Although psychological safety was shown to be related to team learning and error management training, no direct effects were observed between it and the outcome of adaptive performance. Therefore, it may be useful to explore this relationship in the future, using different training protocols that may benefit from the increased psychological safety. While the total amount of communication about errors and inefficiencies was similar between the experimental and control condition, there is evidence from other streams of research on team communication that there may be a qualitative shift as teams improve their ability to solve problems and work in a coordinated fashion (Van Dyck, Frese, Baer, & Sonnentag, 2005; Sexton, Thomas, & Helmreich, 2000; Barrett et al., 2001). 50 Team communication is not always equally distributed between team members and can also display different patterns of interaction at different times in a team’s lifecycle (Gersick, 1988). For instance, in one case a team may each contribute to plan or discuss a relevant point while in another case a single member or minority of members may contribute while others only agree, disagree, or abstain entirely. Both demographic and functional differences are also known to impact team communication (Bell et al., 2011), so looking at the patterns of communication among roles, surface fault lines (Mohammed & Angell, 2004) may yield insights into how teams communicate about errors and afford insights into where communication can be derailed. Although this data was not captured in the current study, it may be useful for future research to investigate these areas with a specific focus on error-related communication and performance. A second area for future research is the role in organizational support and other broad contextual factors that may impact how error management training impacts far transfer and transfer after longer periods of time. What role does supervisor support, organizational norms and values, and other conditions in the broader organization play in employing and maintaining these skills? There is consistent research on the importance of support from within the team, leadership, and the organization to reward or punish outcomes of training (Smith-Jentsch, Salas, & Brannick, 2001) and the need for opportunities to enact and apply skills (Ford, Quinones, Sego, & Sorra, 1992). A third area to explore in future research is to target training of team communication and coordination related to errors and information about errors and inefficiencies specifically. Teamwork skills are thought of as distinct from task work skills and a focus on helping to facilitate these events during performance missions as well as communication during planning and debriefs may have been a barrier to the potential of error management training’s impact on 51 adaptive performance (Ellis et al., 2005). By creating specific norms and teaching team members how and when to enact communication about errors and how to better incorporate this type of information as the team is acting towards its goals, this information may be more appropriately processed and integrated into team functioning. A psychologically safe environment may be necessary, but not sufficient for achieving greater benefits from applying error management training principles to team training. More specific and targeted behaviors and group norms may be required to evoke the potential of error management training on team functioning. Finally, research into the nature of task complexity and adaptive performance has many frameworks and theories that offer rich and nuanced understanding of these constructs (Liu & Ii, 2012; Baard, Rench, & Kozlowski, 2014). Additionally, the definitions of and relationship to task complexity and adaptive performance are another area where research can seek to clarify and enhance our understanding of performance in these conditions. While the error management paradigm has reported gains in adaptive performance (Keith & Frese, 2008), this term has been applied ambiguously and the rigor of more recent frameworks may yield insights into how error management training impacts behaviors and the more specific contexts in which it is most useful. The nature of the processes at work at both the team and individual level and the objective features of the environment and the transjective interaction of the individuals with the environment in these adaptive performance episodes can benefit from increased analysis and experimentation. The current study can serve as an example of how these frameworks can start to be applied a more comprehensive and purposeful manner, and future studies can apply and control these aspects to greater degrees. 52 It is likely that error management principles can offer unique opportunities to team training programs and many challenges wait for future researchers to continue to refine the understanding and application of this set of ideas to team training contexts 53 APPENDICES 54 Appendix A Team Psychological Safety Survey 1. If you make a mistake on this team, it is often held against you. 2. Members of this team are able to bring up problems and tough issues. 3. People on this team sometimes reject others for being different. 4. It is safe to take a risk on this team. 5. It is difficult to ask other members of this team for help. 6. No one on this team would deliberately act in a way that undermines my efforts. 7. Working with members of this team, my unique skills and talents are valued and utilized. 55 Appendix B Phases of Experiment Greeting & Team Formation Phase: (6 minutes) Informed consent. Describe training process, introduce team members, and assign roles. Collect Demographics Training Phase 1: (40 minutes) General Training Instructions. Error Management Prime. Role specific Training Videos & Competency Check Practice Scenario 1 Introduction. Planning 1. Training Mission 1. Training Phase 2: (30 minutes) Error Management Prime. Practice Scenario 2 Introduction. Planning 2. Training Mission 2. Adaptability Phase: (30 minutes) Error Management Prime. 56 Adaptability Scenario Introduction. Adaptability Planning. Adaptability Mission. Conclusion: (3 minutes) Debrief Total Time for Experiment: 1 hour & 49 minutes 57 Appendix C Demographics Demographics collected during greeting and team formation phase: Gender: (M / F) Age: SAT score: ACT score: Year in college: Approximate GPA: Briefly describe experience with video games, especially space simulation games: (i.e. how often do you play, types of games played, amount of time played in the last year) 58 Appendix D Error Management Training Manipulations Error management prime: To reduce the negative perception of errors and explain to participants that errors are positive sources of information and necessary for learning trainees will be presented with the follow four error management instructions: “I have made an error. Great!” “There is always a way to leave the error situation.” “Look at the screen to understand what’s happening and see when an error is made.” “I watch what is on the screen and what is changing.” (adapted from Heimbeck, Frese, Sonnentag, & Keith, 2003) Additionally, to promote psychological safety and the sharing of errors and inefficiencies with other team members the trainees will be presented with the following four instructions: “Sharing information about errors and inefficiencies is welcome. Each person is important, and everyone is respected.” “In this team you aren’t rejected for being yourself or stating what you think.” “Team members want your view and communicating errors is valued.” “Team members care about your input and you can freely express yourself.” (adapted from Edmondson, 1999) 59 Appendix E Tables 4 5 6 0.11 0.01 0.34** 0.08 0.28** 0.83** -0.06 0.11 0.01 -0.07 0.08 0.11 -0.03 -0.03 0.09 -0.16 0.50** 0.44** 0.09 0.10 -0.14 -0.04 7 8 9 -0.19 10 Table 1. Intercorrelation Matrix of Measures 1. Condition 2. Team Past Gaming Experience 3. Team Space Gaming 1 -0.11 0.01 2 0.25* 3 Experience 4. Team Average SAT Score 5. Team Psychological Safety 6. Team Learning (Error 0.13 0.35** 0.27** 0.13 -0.07 -0.22 -0.11 -0.20 -0.03 Related) 7. Team Learning (Reflective) 8. Team Communication 9. Final Mission Performance 10. Training Efficiency 11. Adaptive Performance 0.23** -0.18* -0.16 0.21* -0.02 -0.29** -0.02 0.12 -0.03 -0.08 -0.12 -0.12 0.05 -0.03 -0.06 -0.25** -0.23* -0.18 -0.38** 0.27** Efficiency Note. *p < .01, **p < .001. The first condition is the control condition, and the second condition is the error management training condition. For training efficiency and adaptive performance efficiency, a lower score indicates better performance efficiency 60 Table 2. Means (and Standard Deviations) of Measures 1.70(0.49) 1.10(0.64) 1.84(0.49) 1.10(0.62) 1211.06(150.89) 1238.15(142.37) Error Management Condition Control Condition Team Past Gaming Experience Team Space Gaming Experience Team Average SAT Score Team Psychological Safety Team Learning (Error Related) Team Learning (Reflective) Team Communication Final Mission Performance Training Efficiency Adaptive Performance Efficiency Note. For training efficiency and adaptive team performance efficiency, a lower score indicates better performance efficiency. 4.97(0.46) 3.87(0.56) 3.57(0.57) 3.17(0.94) 5.03(3.49) 4.62(0.44) 3.49(0.75) 3.26(0.77) 2.90(1.23) 5.23(3.47) 1090.75(2451.52) 176.06(206.85) 774.34(1664.93) 85.77(86.66) 61 Table 3. Standardized Loadings for Four-factor Structure of Team Learning and Team Psychological Safety Factor 1 Factor 2 Factor 3 Factor 4 0.66 0.53 0.68 0.72 0.76 0.98 0.81 0.68 Item 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 Note. Scores below .3 are not shown. Item 1 – 15 are team learning measurement items and item 16 - 22 are team psychological safety measurement items 0.35 0.60 0.65 0.66 0.99 0.43 0.63 0.63 0.39 0.98 0.52 0.36 0.98 0.98 62 Appendix F Figures Figure 2. Model 2 Figure 3. Model 3 – Final Path Model with Only Error Factor of Team Learning 63 REFERENCES 64 REFERENCES Baer, M., & Frese, M. (2003). Innovation is not enough: Climates for initiative and psychological safety, process innovations, and firm performance. Journal of Organizational Behavior, 24(1), 45-68. Baldwin, T. T., & Ford, J. K. (1988). Transfer of training: A review and directions for future research. Personnel Psychology, 41(1), 63-105. Baldwin, T. T. (1992). Effects of alternative modeling strategies on outcomes of interpersonal- skills training. Journal of Applied Psychology, 77(2), 147. Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84(2), 191. Bandura, A. (1985). Model of causality in social learning theory. In Cognition and psychotherapy. 81-99. Springer US. Bandura, A. (1989). Human agency in social cognitive theory. American Psychologist, 44(9), 1175. Bandura, A. (2001). Social cognitive theory: An agentic perspective. Annual Review of Psychology, 52(1), 1-26. Barnes, C. M., Hollenbeck, J. R., Wagner, D. T., DeRue, D. S., Nahrgang, J. D., & Schwind, K. M. (2008). Harmful help: the costs of backing-up behavior in teams. Journal of Applied Psychology, 93(3), 529. Barrett, J., Gifford, C., Morey, J., Risser, D., & Salisbury, M. (2001). Enhancing patient safety through teamwork training. Journal of Healthcare Risk Management, 21(4), 61-69. Bell, B. S., & Kozlowski, S. W. (2010). Toward a theory of learner-centered training design: An integrative framework of active learning. Learning, training, and development in organizations, 263-300. Bell, B. S., & Kozlowski, S. W. (2011). Collective failure: The emergence, consequences, and management of errors in teams. Errors in organizations, 113-141. Bell, S. T., Villado, A. J., Lukasik, M. A., Belau, L., & Briggs, A. L. (2011). Getting specific about demographic diversity variable and team performance relationships: A meta- analysis. Journal of Management, 37(3), 709-743. Blume, B. D., Ford, J. K., Baldwin, T. T., & Huang, J. L. (2010). Transfer of training: A meta- analytic review. Journal of Management, 36(4), 1065-1105. 65 Burke, C. S., Salas, E., & Diaz, D. (2008). The role of team learning in facilitating team adaptation within complex environments: Tools and strategies. Work group learning: Understanding, improving and assessing how groups learn in organizations, 217-241. Burke, C. S., Salas, E., Wilson-Donnelly, K., & Priest, H. (2004). How to turn a team of experts into an expert medical team: guidance from the aviation and military communities. Quality and Safety in Health Care, 13(1), i96-i104. Camerer, C., & Lovallo, D. (1999). Overconfidence and excess entry: An experimental approach. The American Economic Review, 89(1), 306-318. Cannon, M. D., & Edmondson, A. C. (2001). Confronting failure: Antecedents and consequences of shared beliefs about failure in organizational work groups. Journal of Organizational Behavior, 22(2), 161-177. Cannon-Bowers, J. A., Salas, E., & Converse, S. A. (1990). Cognitive psychology and team training: Training shared mental models and complex systems. Human Factors Society Bulletin, 33(12), 1-4. Carmeli, A., & Gittell, J. H. (2009). High‐quality relationships, psychological safety, and learning from failures in work organizations. Journal of Organizational Behavior, 30(6), 709-729. Carson, J. B., Tesluk, P. E., & Marrone, J. A. (2007). Shared leadership in teams: An investigation of antecedent conditions and performance. Academy of Management Journal, 50(5), 1217-1234. Carter, M., & Beier, M. E. (2010). The effectiveness of error management training with working- aged adults. Personnel Psychology, 63(3), 641-675. Chen, G., Thomas, B., & Wallace, J. C. (2005). A multilevel examination of the relationships among training outcomes, mediating regulatory processes, and adaptive performance. Journal of Applied Psychology, 90(5), 827. Chillarege, K. A., Nordstrom, C. R., & Williams, K. B. (2003). Learning from our mistakes: Error management training for mature learners. Journal of Business and Psychology, 17(3), 369-385. Debowski, S., Wood, R. E., & Bandura, A. (2001). Impact of guided exploration and enactive exploration on self-regulatory mechanisms and information acquisition through electronic search. Journal of Applied Psychology, 86(6), 1129. DeChurch, L. A., & Mesmer-Magnus, J. R. (2010). Measuring shared team mental models: A meta-analysis. Group Dynamics: Theory, Research, and Practice, 14(1), 1-14 Dörner, D., & Schaub, H. (1994). Errors in planning and decision‐making and the nature of human information processing. Applied Psychology, 43(4), 433-453. 66 Druskat, V. U., & Pescosolido, A. T. (2002). The content of effective teamwork mental models in self-managing teams: Ownership, learning and heedful interrelating. Human Relations, 55(3), 283-314. East Room - White House Museum. (2017). Retrieved from http://www.whitehousemuseum.org/floor1/east-room.htm Edmondson, A. C. (1996). Learning from mistakes is easier said than done: Group and organizational influences on the detection and correction of human error. The Journal of Applied Behavioral Science, 32(1), 5-28. Edmondson, A. (1999). Psychological safety and learning behavior in work teams. Administrative Science Quarterly, 44(2), 350-383. Edmondson, A. C. (2003). Speaking up in the operating room: How team leaders promote learning in interdisciplinary action teams. Journal of Management Studies, 40(6), 1419- 1452. Edmondson, A. C. (2004). Learning from failure in health care: Frequent opportunities, pervasive barriers. Quality and Safety in Health Care, 13(2), ii3-ii9. Ellis, A. P., Bell, B. S., Ployhart, R. E., Hollenbeck, J. R., & Ilgen, D. R. (2005). An evaluation of generic teamwork skills training with action teams: Effects on cognitive and skill‐ based outcomes. Personnel Psychology, 58(3), 641-672. Eysenck, H. J. (1996). Personality and the experimental study of education. European Journal of Personality, 10(5), 427-439. Ford, J. K., Quiñones, M. A., Sego, D. J., & Sorra, J. S. (1992). Factors affecting the opportunity to perform trained tasks on the job. Personnel Psychology, 45(3), 511-527. Frese, M., Brodbeck, F., Heinbokel, T., Mooser, C., Schleiffenbaum, E., & Thiemann, P. (1991). Errors in training computer skills: On the positive function of errors. Human-Computer Interaction, 6(1), 77-93. Frese, M., & Keith, N. (2015). Action errors, error management, and learning in organizations. Annual Review of Psychology, 66, 661-687. Frese, M., & Zapf, D. (1994). Action as the core of work psychology: A German approach. Handbook of industrial and organizational psychology, 4, 271-340. Geiser, C. (2012). Data analysis with Mplus. Guilford Press. Gersick, C. J. (1988). Time and transition in work teams: Toward a new model of group development. Academy of Management Journal, 31(1), 9-41. 67 Gully, S. M., Payne, S. C., Koles, K. L. K., & Whiteman, J. A. K. (2002). The impact of error training and individual differences on training outcomes: An attribute-treatment interaction perspective. Journal of Applied Psychology, 87(1), 143-155. Halpern, D. F. (1998). Teaching critical thinking for transfer across domains: Disposition, skills, structure training, and metacognitive monitoring. American Psychologist, 53(4), 449. Hackman, J. R. (1987). The design of work teams. In J. W. Lorsch (ed.), Handbook of organizational behavior (pp. 315-342). Hammond, M. M., Farr, J. L., & Sherman, P. O. C. (2011). The role of errors in the creative and innovative process. Errors in organizations, 67-96. Helmreich, R. L. (2000). On error management: Lessons from aviation. BMJ: British Medical Journal, 320(7237), 781. Helmreich, R. L., Merritt, A. C., & Wilhelm, J. A. (1999). The evolution of crew resource management training in commercial aviation. The International Journal of Aviation Psychology, 9(1), 19-32. Helmreich, R. L., & Musson, D. M. (2000). Threat and error management model: Components and examples. British Medical Journal, 9, 1-23. Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world?. Behavioral and Brain Sciences, 33(2-3), 61-83. Hirak, R., Peng, A. C., Carmeli, A., & Schaubroeck, J. M. (2012). Linking leader inclusiveness to work unit performance: The importance of psychological safety and learning from failures. The Leadership Quarterly, 23(1), 107-117. Hofmann, D. A., & Frese, M. (2011). Errors, error taxonomies, error prevention, and error management: Laying the groundwork for discussing errors in organizations. Errors in organizations, 1-38. Hooper, D., Coughlan, J., & Mullen, M. (2008, September). Evaluating model fit: a synthesis of the structural equation modelling literature. In 7th European Conference on research methodology for business and management studies (pp. 195-200). Ilgen, D. R., Hollenbeck, J. R., Johnson, M., & Jundt, D. (2005). Teams in organizations: From input-process-output models to IMOI models. Annual Review of Psychology, 56, 517- 543. Ivancic IV, K., & Hesketh, B. (2000). Learning from errors in a driving simulation: Effects on driving skill and self-confidence. Ergonomics, 43(12), 1966-1984. Kahn, W. A. (2007). Meaningful Connections: Positive Relationships and Attachments at Work. 68 Kanfer, R., Ackerman, P. L., & Heggestad, E. D. (1996). Motivational skills & self-regulation for learning: A trait perspective. Learning and Individual Differences, 8(3), 185-209. Karoly, P. (1993). Mechanisms of self-regulation: A systems view. Annual Review of Psychology, 44(1), 23-52. Katz, D., & Kahn, R. L. (1978). The social psychology of organizations. New York: Wiley. Keith, N., & Frese, M. (2005). Self-regulation in error management training: emotion control and metacognition as mediators of performance effects. Journal of Applied Psychology, 90(4), 677. Keith, N., & Frese, M. (2008). Effectiveness of Error Management Training: A Meta-Analysis. Journal of Applied Psychology, 93(1), 59-69. Kenny, D. A., Kaniskan, B., & McCoach, D. B. (2015). The performance of RMSEA in models with small degrees of freedom. Sociological Methods & Research, 44(3), 486-507. Kiesler, S., & Sproull, L. (1982). Managerial response to changing environments: Perspectives on problem sensing from social cognition. Administrative Science Quarterly, 548-570. Kline, R. B. (2015). Principles and practice of structural equation modeling. Guilford publications. Kraiger, K., Ford, J. K., & Salas, E. (1993). Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. Journal of Applied Psychology, 78(2), 311. Kwei-Narh, P. A., Valaker, S., Hærem, T., & Lervik, J. E. (2016, January). Monitoring and team performance: The mediating role of Task mental model accuracy. In Academy of Management Proceedings (Vol. 2016, No. 1, p. 17012). Academy of Management. Kozlowski, S. W., Gully, S. M., McHugh, P. P., Salas, E., & Cannon-Bowers, J. A. (1996). A dynamic theory of leadership and team effectiveness: Developmental and task contingent leader roles. Research in Personnel and Human Resources Management, 14, 253-306. Kozlowski, S. W., Gully, S. M., Nason, E. R., & Smith, E. M. (1999). Developing adaptive teams: A theory of compilation and performance across levels and time. Pulakos (Eds.), The changing nature of work performance: Implications for staffing, personnel actions, and development, 240, 292. Kozlowski, S. W., Watola, D. J., Jensen, J. M., Kim, B. H., & Botero, I. C. (2009). Developing adaptive teams: A theory of dynamic team leadership. Team effectiveness in complex organizations: Cross-disciplinary perspectives and approaches, 113-155. LeBreton, J. M., & Senter, J. L. (2008). Answers to 20 questions about interrater reliability and interrater agreement. Organizational Research Methods, 11(4), 815-852. 69 Leonning, Carol. (2014, September 29). White House fence-jumper made it far deeper into building that previously known. Retrieved from https://www.washingtonpost.com/politics/white-house-fence-jumper-made-it-far-deeper- into-building-than-previously-known/2014/09/29/02efd53e-47ea-11e4-a046- 120a8a855cca_story.html?utm_term=.98bc158400bd Liu, G., Amini, M. M., Babakus, E., & Stafford, M. B. R. (2011). Handling observations with low interrater agreement values. Journal of Analytical Sciences, Methods and Instrumentation, 1(02), 9. Locke, E. A., & Latham, G. P. (2002). Building a practically useful theory of goal setting and task motivation: A 35-year odyssey. American Psychologist, 57(9), 705. Lorenzet, S. J., Salas, E., & Tannenbaum, S. I. (2005). Benefiting from mistakes: The impact of guided errors on learning, performance, and self‐efficacy. Human Resource Development Quarterly, 16(3), 301-322. Mann, R. B., & Decker, P. J. (1984). The effect of key behavior distinctiveness on generalization and recall in behavior modeling training. Academy of Management Journal, 27(4), 900- 910. Marks, M. A., Mathieu, J. E., & Zaccaro, S. J. (2001). A temporally based framework and taxonomy of team processes. Academy of Management Review, 26(3), 356-376. Marks, M. A., & Panzer, F. J. (2004). The influence of team monitoring on team processes and performance. Human Performance, 17(1), 25-41. Marottoli, R. A., & Richardson, E. D. (1998). Confidence in, and self-rating of, driving ability among older drivers. Accident Analysis & Prevention, 30(3), 331-336. Mathieu, J. E., Heffner, T. S., Goodwin, G. F., Salas, E., & Cannon-Bowers, J. A. (2000). The influence of shared mental models on team process and performance. Journal of Applied Psychology, 85(2), 273. Manikandan, S. (2010). Data transformation. Journal of Pharmacology and Pharmacotherapeutics, 1(2), 126. Mehra, A., Smith, B. R., Dixon, A. L., & Robertson, B. (2006). Distributed leadership in teams: The network of leadership perceptions and team performance. The Leadership Quarterly, 17(3), 232-245. Mohammed, S., & Angell, L. C. (2004). Surface‐and deep‐level diversity in workgroups: Examining the moderating effects of team orientation and team process on relationship conflict. Journal of Organizational Behavior: The International Journal of Industrial, Occupational and Organizational Psychology and Behavior, 25(8), 1015-1039. 70 Nordstrom, C. R., Wendland, D., & Williams, K. B. (1998). “To err is human”: An examination of the effectiveness of error management training. Journal of Business and Psychology, 12(3), 269-282. Oberski, D. (2014). Lavaan survey: An R package for complex survey analysis of structural equation models. Journal of Statistical Software, 57(1), 1-27. Pintrich, P. R., & De Groot, E. V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82(1), 33. Porter, C. O. L. H., Hollenbeck, J. R., Ilgen, D. R., Ellis, A. P., West, B. J., & Moon, H. (2003). Backing up behaviors in teams: The role of personality and legitimacy of need. Journal of Applied Psychology, 88(3), 391-403. Pulakos, E. D., Arad, S., Donovan, M. A., & Plamondon, K. E. (2000). Adaptability in the workplace: development of a taxonomy of adaptive performance. Journal of Applied Psychology, 85(4), 612. Reason, J. (1990). Human error. Cambridge university press. Rosen, M. A., Salas, E., Wilson, K. A., King, H. B., Salisbury, M., Augenstein, J. S., ... & Birnbach, D. J. (2008). Measuring team performance in simulation-based training: adopting best practices for healthcare. Simulation in Healthcare, 3(1), 33-41. Rosen, M. A., Bedwell, W. L., Wildman, J. L., Fritzsche, B. A., Salas, E., & Burke, C. S. (2011). Managing adaptive performance in teams: Guiding principles and behavioral markers for measurement. Human Resource Management Review, 21(2), 107-122. Salas, E., Rhodenizer, L., & Bowers, C. A. (2000). The design and delivery of crew resource management training: exploiting available resources. Human Factors, 42(3), 490-511. Salas, E., Rosen, M. A., Burke, C. S., & Goodwin, G. F. (2009). The wisdom of collectives in organizations: An update of the teamwork competencies. Team effectiveness in complex organizations. cross-disciplinary perspectives and approaches, 39-79. Salas, E., Sims, D. E., & Burke, C. S. (2005). Is there a “big five” in teamwork?. Small Group Research, 36(5), 555-599. Salas, E., Wilson, K. A., Burke, C. S., & Wightman, D. C. (2006). Does crew resource management training work? An update, an extension, and some critical needs. Human Factors, 48(2), 392-412. Savelsbergh, C. M., van der Heijden, B. I., & Poell, R. F. (2009). The development and empirical validation of a multidimensional measurement instrument for team learning behaviors. Small Group Research, 40(5), 578-607. 71 Schmidt, A. M., & Ford, J. K. (2003). Learning within a learner control training environment: The interactive effects of goal orientation and metacognitive instruction on learning outcomes. Personnel Psychology, 56(2), 405-429. Sexton, J. B., Thomas, E. J., & Helmreich, R. L. (2000). Error, stress, and teamwork in medicine and aviation: Cross sectional surveys. Bmj, 320(7237), 745-749. Smith-Jentsch, K. A., Salas, E., & Brannick, M. T. (2001). To transfer or not to transfer? Investigating the combined effects of trainee characteristics, team leader support, and team climate. Journal of Applied Psychology, 86(2), 279. Snow, R. E. (1986). Individual differences and the design of educational programs. American Psychologist, 41(10), 1029. Soper, D. (2017). Free statistical calculator. Retrieved from https://www.danielsoper.com/statcalc/default.aspx. Taylor, P. J., Russ-Eft, D. F., & Chan, D. W. (2005). A meta-analytic review of behavior modeling training. Journal of Applied Psychology, 90(4), 692-709. Thomas, M. J. (2004). Predictors of threat and error management: Identification of core nontechnical skills and implications for training systems design. The International Journal of Aviation Psychology, 14(2), 207-231. Van Dyck, C., Frese, M., Baer, M., & Sonnentag, S. (2005). Organizational error management culture and its impact on performance: A two-study replication. Journal of Applied Psychology, 90(6), 1228. Wilson, K. A., Burke, C. S., Priest, H. A., & Salas, E. (2005). Promoting health care safety through training high reliability teams. Quality and Safety in Health Care, 14(4), 303- 309. Wood, R., Kakebeeke, B., Debowski, S., & Frese, M. (2000). The impact of enactive exploration on intrinsic motivation, strategy, and performance in electronic search. Applied Psychology, 49(2), 263-283. Zaccaro, S. J., Rittman, A. L., & Marks, M. A. (2002). Team leadership. The Leadership Quarterly, 12(4), 451-483. Zapf, D., Brodbeck, F. C., Frese, M., Peters, H., & Prümper, J. (1992). Errors in working with office computers: A first validation of a taxonomy for observed errors in a field setting. International Journal of Human‐Computer Interaction, 4(4), 311-339. 72