ADAPTIVE GUIDANCE WITH TEAMS: SHIFTING FROM TASKWORK TO TEAMWORK By Stanton Mak A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Psychology – Doctor of Philosophy 2017 ABSTRACT ADAPTIVE GUIDANCE WITH TEAMS: SHIFTING FROM TASKWORK TO TEAMWORK By Stanton Mak This study sought to advance understanding of how to effectively train knowledgebuilding and decision-making teams in both taskwork and teamwork skills by comparing the effectiveness of two training strategies. The first is a traditional holistic strategy that focuses trainee attention to both taskwork and teamwork skill development simultaneously. The second is a strategy inspired by Kozlowski et al.’s (1999) normative model of team development that shifts the focus of attention from taskwork first and then teamwork skill development. Although the shifting focus strategy resulted in superior team knowledge outcomes, no differences were found in decision-making performance. Potential reasons for lack of support for hypotheses were suggested, and a re-design of the study was proposed. ACKNOWLEDGMENTS I would like to extend my extreme gratitude to my advisor, Dr, Steve Kozlowski, who made this work possible. His patience, guidance, and expert advice throughout the research process have been invaluable. I would also like to thank my committee members, Daisy Chang, Kevin Ford, and Georgia Chao, for spending time on careful reading of my dissertation, and providing guidance in improving the research in the present study. Finally, I am thankful to my family and friends for all of the love, support, and guidance they provide me. iii TABLE OF CONTENTS LIST OF TABLES ......................................................................................................................... vi LIST OF FIGURES ...................................................................................................................... vii INTRODUCTION ...........................................................................................................................1 Literature Review.................................................................................................................6 Dimensions of Team Performance...........................................................................6 Teamwork Behaviors ...............................................................................................8 Team Training ........................................................................................................11 Regulation Processes .............................................................................................13 Adaptive Guidance to Develop Taskwork and Teamwork ....................................17 Past Research on Adaptive Guidance ....................................................................18 Team Development ................................................................................................22 Research Goals.......................................................................................................25 Hypotheses .........................................................................................................................26 Attention to Knowledge Acquisition Taskwork ....................................................29 Quality of Knowledge Acquisition Taskwork .......................................................30 Attention to Knowledge Application Taskwork ....................................................32 Quality of Knowledge Application Taskwork .......................................................33 Attention to Knowledge Acquisition Teamwork ...................................................35 Quality of Knowledge Acquisition Teamwork ......................................................36 Attention to Knowledge Application Teamwork ...................................................38 Quality of Knowledge Application Teamwork......................................................40 Team Knowledge Outcomes ..................................................................................42 Team Decision Outcomes ......................................................................................43 METHOD ......................................................................................................................................45 Sample................................................................................................................................45 Procedures ..........................................................................................................................45 Experimental Task .............................................................................................................46 Experimental Manipulation ...............................................................................................50 Measures ............................................................................................................................56 Demographics ........................................................................................................56 Knowledge Acquisition Taskwork Practice...........................................................56 Knowledge Application Taskwork Practice ..........................................................57 Knowledge Acquisition Taskwork Study Time .....................................................57 Knowledge Application Taskwork Study Time ....................................................57 Knowledge Acquisition Teamwork Practice .........................................................58 Knowledge Application Teamwork Practice .........................................................58 Knowledge Acquisition Teamwork Study Time ...................................................59 Knowledge Application Teamwork Study Time ...................................................59 Quality of Knowledge Acquisition Taskwork .......................................................59 Quality of Knowledge Application Taskwork .......................................................60 iv Quality of Knowledge Acquisition Teamwork ......................................................61 Quality of Knowledge Application Teamwork......................................................62 Team Knowledge Outcome ...................................................................................62 Team Decision Accuracy .......................................................................................63 RESULTS ......................................................................................................................................66 Knowledge Acquisition Taskwork Hypotheses .................................................................66 Knowledge Application Taskwork Hypotheses.................................................................71 Knowledge Acquisition Teamwork Hypotheses ...............................................................75 Knowledge Application Teamwork Hypotheses ...............................................................79 Team Knowledge Outcomes Hypotheses ..........................................................................83 Team Decision Outcomes Hypotheses ..............................................................................85 DISCUSSION ................................................................................................................................87 Key Findings ......................................................................................................................88 Limitations and Future Directions .....................................................................................94 Conclusion .........................................................................................................................96 APPENDICES ...............................................................................................................................97 APPENDIX A: Study Questionnaires ...............................................................................98 APPENDIX B: Supplemental Analyses ............................................................................99 REFERENCES ............................................................................................................................101 v LIST OF TABLES Table 1. Description of embedded adaptive guidance prompts to promote taskwork ...................52 Table 2. Description of embedded adaptive guidance prompts to promote teamwork .................53 Table 3. Individual-level descriptive statistics and correlations ....................................................64 Table 4. Team-level descriptive statistics and correlations ...........................................................65 Table 5. Coefficient estimates from MRCM for knowledge acquisition taskwork efficiency across first four performance trials ................................................................................................70 Table 6. Coefficient estimates from MRCM for knowledge acquisition taskwork accuracy across first four performance trials ...........................................................................................................70 Table 7. Coefficient estimates from MRCM for number of relative correct route scores across first four performance trials ...........................................................................................................74 Table 8. Coefficient estimates from MRCM for number of relative asset choices across first four performance trials ..........................................................................................................................74 Table 9. Coefficient estimates from MRCM for knowledge sharing communication across last five performance trials ...................................................................................................................77 Table 10. Coefficient estimates from MRCM for knowledge sharing coordination across last five performance trials ..........................................................................................................................78 Table 11. Coefficient estimates from MRCM for communication to facilitate decision-making during the last five performance trials ...........................................................................................81 Table 12. Coefficient estimates from multilevel logistic regression for knowledge application teamwork during the last five performance trials ..........................................................................82 Table 13. Coefficient estimates from MRCM for team knowledge outcomes during the last five performance trials ..........................................................................................................................84 Table 14. Coefficient estimates from multilevel logistic regression for team decision outcome during the last five performance trials ...........................................................................................85 Table 15. Number of prompts triggered in the shifting focus condition .......................................92 Table 16. Number of prompts triggered in the holistic focus condition. .......................................92 vi LIST OF FIGURES Figure 1. Typology of Taskwork/Teamwork Requirements of Knowledge-Building Teams .......27 vii INTRODUCTION A 6.7 magnitude earthquake had just struck the Los Angeles area. Hundreds of people were now trapped underneath debris and fallen buildings, and their lives were in grave jeopardy. Without help, many of them would have suffocated or starved to death in a few short days. The situation was critical and time was short. In response, the L.A. County Fire Department hastily dispatched multiple urban search and rescue teams. Their objectives were to analyze the situation, search for victims, and conduct rescue operations to save as many people as possible. To succeed, the teams needed to harness the expertise of many specialists, including structure collapse engineers, engineering specialists, doctors, and dog handlers. By analyzing the problem space and sharing their expert knowledge with the rest of the team, these specialists developed a full understanding of where the survivors were and how best to save them. In the end, these teams of experts transformed themselves into expert teams through effective teamwork. The rescue operation was successful, and many were saved. A woman was admitted to a hospital with a life-threatening bowel obstruction. An emergency colostomy was performed to bypass the affected area of her colon. However, when the surgeon sewed her up, he made an error and left behind some necrotic bowel. Over the next two weeks, the patient ate almost nothing and had signs of bowel dysfunction. The team of specialists working on her case (internal medicine physicians, kidney doctors, home care nurse, and surgeons) could not agree about her health or nutritional status, and this was exacerbated by the fact that they primarily communicated with each other through notes in the patient's chart, and rarely face-to-face. When a physician friend of the patient suspected that she had a significant bowel problem and was slowly starving to death, he met with the medical care team 1 and convinced them to examine the condition of the colon more carefully. Only then were they able to diagnose the issue and eventually perform surgery to resolve it. What went wrong? According to investigators, a proper understanding of the patient’s health situation required the collective knowledge and expertise of everybody on the medical care team. However, her illness was continuously misdiagnosed due to failures of oversight, coordination, and communication among the medical specialists. Anyone who has ever watched a high-performance team in action can appreciate how important it is for teams to communicate, coordinate, and integrate dynamically to achieve results. This is especially true for problem-solving teams such as emergency/rescue teams, military units, and surgical teams. As illustrated by the above examples, these teams are composed of members with highly specialized roles who must complete a task that requires pooling information from multiple sources in order to make consequential decisions under time pressure (e.g., Salas & Cannon-Bowers, 1997; Cannon-Bowers & Salas, 1998). Team success is dependent on the ability of members to work together as an interdependent unit so that all relevant information can be exchanged and integrated, which allows for an accurate shared understanding of the problem space and potential solutions. However, teams are not successful simply by the virtue of their existence. Indeed, as demonstrated by the voluminous research on hidden profile problems, teams consistently fail to share or consider all relevant information, resulting in suboptimal decisions (Stasser & Titus, 2003). These challenges contribute to the broader need for improving team training effectiveness in organizations to ensure that teams have the requisite knowledge and skills needed to achieve success. In the past, training research had primarily been oriented towards individual change; models of training design were predicated on the acquisition of individual competencies, and so 2 theorizing and research were conducted at the individual-level of analysis (Salas, Dickinson, Converse, & Tannenbaum,1992). However, the success of many teams hinges on the ability of members to integrate their individual performances to meet interaction demands, and to adapt their behaviors in a coordinated fashion and work as an interdependent unit (Hollenbeck, DeRue, & Guzzo, 2004). Therefore, the past decade has seen an emergence of team training interventions that focus on team-level learning, and provide opportunities for members to train and interact together in order to simulate the conditions under which their trained knowledge and skills will be used. Despite progress in the literatures on training (e.g., Colquitt, LePine, & Noe, 2000), selfregulation (e.g., Kanfer, 1990; Mitchell & Daniels, 2003), and work teams (e.g., Kozlowski & Bell, 2003), our understanding of how to design training interventions to help teams to develop the requisite skills to perform their task effectively is incomplete. As described by Salas, Nichols, and Driskell (2007), “there are a large number of theoretical and conceptual efforts describing needed work [on team training] and a much smaller subset of empirical studies” (p. 485). Three limitations in the existing literature on team training are substantial contributors to this ambiguity. First, team training research tends to focuses solely on the development of either individual technical skills (i.e., taskwork skills) within a team setting, or on the team’s ability to work together to ensure collective success (i.e., teamwork skills) (Salas, Dickinson, Converse, and Tannenbaum, 1992; Salas & Cannon-Bowers, 2001). There is little research that focuses on the development of both taskwork and teamwork skills. This omission is unfortunate because teamwork skills are clearly necessary for teams in which success is dependent on the ability of their members to coordinate their actions and work as an interdependent unit. At the same time, 3 teams are not entities unto themselves; they are composed of individuals. Individuals enact teams, and so individual taskwork skills are clearly crucial to team performance. Thus, examination of how to effectively train taskwork skills, and how to bring those individual skills and talents together into a smoothly coordinated and integrated team, remains an important yet largely underdeveloped area of the team-training literature (Kozlowski, Gully, McHugh et al., 1996; Chen & Klimoski, 2007). Second, although feedback is a critical characteristic of motivation interventions, feedback interventions have yet to receive adequate attention in the context of the shift to teambased work structures. As documented by DeShon, Kozlowski, Schmidt, Milner, and Wiechmann (2004), there is voluminous research on feedback effects at the individual level, yet surprisingly little research on the effects of feedback on learning and performance at the team level. A large part of the problem is that the team training literature has historically lacked a framework that explains how individual team members use feedback to improve individual and team performance (DeShon et al., 2004). However, recent theoretical and empirical research suggests that self-regulatory processes function similarly across the individual and team level (Kozlowski & Klein, 2000; Chen and Kanfer, 2006). This suggests that basic principles of selfregulation can account for feedback effects on learning, motivation, and performance at both the individual and team level of analysis. Finally, team training research has largely neglected the role of time in intervention delivery (Salas, DiazGranados, Klein, Burke, Stagl, Goodwin, & Halpin, 2008). Although the theoretical literature on team development recognizes that teams form, mature, and evolve over time, the design of team training has generally neglected to take a developmental perspective. The normative model of team development by Kozlowski, Gully, Nason, and Smith (1999) 4 views team development as a progressive process across four phases, with each phase focused on distinct content, learning processes, and outcomes. This has implications for what kind and when leverage can be bested exerted in team training. Given the different primary concerns of teams through the developmental process, some factors are thought to be more important than others during specific phases. Therefore, team training interventions should target the development of performance capabilities appropriate to the different phases so that teams can more effectively navigate the developmental process and display better team performance. Accordingly, the purpose of the present research is to examine one type of feedback intervention that has shown to have potential for aiding trainees at the individual level – adaptive guidance – and evaluate its effectiveness in developing the skills that are crucial for members in teams. Adaptive guidance is designed to provide tailored feedback to meet the differing needs of individuals. By using computer technologies to monitor and assess individuals’ learning and performance, individuals are provided individualized recommendations regarding what learning objectives to focus on and what future actions to take to improve performance. Bell and Kozlowski (2002) found that providing trainees with guidance while learning a complex radar control task improved their study and practice, self-regulation, and performance. The present study builds on this prior work in two important ways. First, I examine adaptive guidance for teams, and evaluate its effectiveness in developing both taskwork and teamwork skills. By making a distinction between taskwork skills and teamwork skills, this study provides an understanding of how effective adaptive guidance can be in developing the fundamental individual and team skills necessary for effective team performance. Second, I use Kozlowski et al.’s (1999) normative model of team development to inform the design of adaptive 5 guidance for teams; adaptive guidance should be sensitive to the team’s current development phase, and targeted skills should be acquired at the appropriate phase of development. Although issues of taskwork and teamwork training are relevant to all kinds of teams, the current study is particularly focused on knowledge-building teams composed of members with distributed, specialized expertise who must actively acquire information from the environment and each other to adequately capture a problem space, and then integrate it and apply it to solve complex and challenging problems. Thus, the teamwork skills which facilitate coordinated actions across team members are a crucial component of team effectiveness for this type of team (Klienman & Serfaty, 1989). This is a very generalizable problem-space, and is consistent with many team tasks involving knowledge building and decision-making. Literature Review Dimensions of Team Performance To understand how team training can enhance team learning and performance, it is important to first clearly define what is meant by a team. Kozlowski and Ilgen (2006, p.79) define teams as being composed of two or more individuals who to some degree (1) interact socially, (2) possess common goals, (3) are interdependent in terms of workflows or outcomes, (4) have distinct roles and responsibilities within the team, and (5) are embedded in a larger organizational context that they influence and are influenced by. Interdependency refers to the structure and degree to which members are interconnected and rely upon each other to complete the task (Salas, Rosen, Burke & Goodwin 2009). Oftentimes, teams with high task interdependence consist of members with distributed expertise, such that each team member may have a different specialization in knowledge and skills. A basic challenge for these teams is the need for team members to successfully integrate their individual knowledge and skills to produce 6 desired collective outcomes. Indeed, it is this integration of diverse expertise that allows teams to complete tasks that exceed the capability of any individual member. Interaction among team members, such as collecting and interpreting information, passing on information, providing situation updates, and resolving opposing interpretations, may have crucial impact on overall team performance for highly interdependent tasks (Fleishman & Zaccaro, 1992; Hackman, 1987). In such contexts, team members need to perform two forms of activity: taskwork and teamwork (Burke, Wilson & Salas, 2003; Morgan, Glickman, Woodward, Blaiwes, & Salas, 1986). Taskwork refers to activities that individual team members perform while completing their own part of the team task; these activities do not rely on other team members’ input or interdependent interaction within the team (Salas, Cooke, & Rose, 2008). The knowledge and skills required to undertake taskwork are known as taskwork competencies. This includes understanding the nature of the task, how to interact with equipment, and how to follow proper policies and procedures. These individual competencies are important for team performance, and novice team members must acquire knowledge about the key components of their individual tasks and how to perform them (Morgan et al., 1986). For instance, several studies have found that teams comprising individuals with higher levels of task proficiency outperformed teams that were made up of members that were less task proficient and had lower abilities (Kabanoff & O’Brien, 1979). On the other hand, teamwork consists of the interdependent interactions among team members that must be performed to accomplish the team objectives (Cannon-Bowers, Tannenbaum, Salas, & Volpe, 1995). These activities help facilitate effective interactions, such as coordinating action between members and monitoring each other’s performance to ensure that 7 tasks are accomplished correctly (Kozlowski & Bell, 2003). The knowledge and skills required to undertake teamwork are known as teamwork competencies. It is important to make a distinction between taskwork and teamwork because teams that have the appropriate taskwork skills (e.g., taking a patient’s history) may not be similarly equipped with the teamwork skills (e.g., communication with other members of the emergency medical team) necessary for effective team performance. Indeed, team members who have the requisite technical expertise are still vulnerable to poor team outcomes if they cannot coordinate effort and provide mutual assistance to accomplish the broader team objectives (Mathieu, Maynard, Rapp, & Gilson, 2008). Therefore, it is important for team training to encompass both taskwork and teamwork. Teamwork Behaviors There have been many proposed teamwork taxonomies that attempt to delineate the different teamwork processes important for team effectiveness (e.g., Fleishman and Zaccaro, 1992; Helmreich and Foushee, 1993; McIntyre and Dickinson, 1992; Salas, Sims, and Burke, 2005; Marks, Mathieu, & Zacarro, 2001). Despite lack of complete agreement in the processes that influence team performance, most of these taxonomies highlight coordination, cooperation, and communication as key behavioral team processes that have been empirically found to predict team performance in both laboratory and applied settings (Kozlowski & Bell, 2003; Tannenbaum, Beard, & Salas, 1992; Rousseau, Aube, & Savoie, 2006). As task interdependence increases, the need for smooth interaction among team members increases due to a greater demand for coordination, cooperation, and communication (Thompson, 1967; Saavedra, Earley, & Van Dyne, 1993). Conversely, when task interdependence is low, the need to share or 8 exchange information is reduced, as is the need for smooth coordination, cooperation, and communication. Coordination can be defined as the act of managing interdependencies among team members to achieve a goal (Kozlowski & Bell, 2003). It is an important team capability when successful completion of the team objective requires the numerous contributions or efforts by every team member (i.e., integration), and when successful contributions by one participant are contingent on an orderly and timely contribution by another participant (i.e., temporal entrainment) (Argote & McGrath, 1993). Coordination can take several forms, which can vary as a function of interdependence that the task entails (Guastello & Guastello, 1998); specific workflow structures impact the degree to which tasks or sub-goals need to be completed simultaneously or in sequence, and at what appropriate times (Kanki & Palmer, 1993). As demonstrated in a range of lab and field studies, coordination is an important predictor of team performance. For example, Stout, Salas, and Carson (1994) examined the effects of coordination on team performance for teams operating a flight simulation task. The dimensions of coordination that they examined included behaviors such as making long- and short-term plans, assigning tasks, and altering behavior to meet situational demands. Results indicated that coordination ratings corresponded with higher mission performance. More generally, Stewart (2006) conducted a meta-analysis of 93 studies, and found that coordination positively predicted team performance. Cooperation refers to “the willful contribution of personal efforts to the completion of interdependent jobs” (Wagner, 1995, p. 152). It is reflected in the performance of extrarole behaviors that benefit a teammate or the team as a whole, but are not typically defined as their responsibility (Weldon, Jehn, & Pradhan, 1991). This includes backup behaviors, such as mutual 9 performance monitoring (i.e., observation of activities and performance of other members), error detection and correction, and compensatory performance (i.e., assisting another member when they experience overload or failure) (Nieva, Fleishman, & Rieck, 1978). Such behaviors are vital to team effectiveness because teammates must often put aside their own goals and share their capabilities in order to achieve the collectives’ goals (Morgan, Salas, & Glickman, 1993). Cooperation is positively related to knowledge regarding the strengths and weaknesses of other team members, knowledge of the tasks and roles of other members, and a positive orientation toward the team as a unit (McIntyre & Dickinson, 1992). Empirical research has established cooperative behaviors as positively related to team performance. For example, Mitchell and Silver (1990) found that cooperative feelings and behaviors mediated the influence of goals on task performance. Teams that were high in cooperative strategies did better than teams low in cooperative strategies. Morgan et al. (1986) found that cooperative skills like performing tasks outside one’s job, assisting others when they were having difficulty, and covering other members when they were overworked, are important skills that distinguish ineffective from effective teams. Smith et al. (1994) examined the effect of cooperative behaviors in top management teams and found that cooperation positively predicted both sales growth and return on investment. Finally, Eby and Dobbins (1997) showed that team cooperation mediated the relationship between team orientation and team performance. Communication involves the exchange of information between two or more team members in a prescribed manner using proper terminology (McIntyre & Dickinson, 1992). It is often examined as a means of enabling the critical team processes of coordination and cooperation. Effective communication allows members to exchange task-related information, to monitor each others’ performance, to share the workload when help is needed, to explicitly 10 coordinate tasks in real time, and to adjust team strategy. Therefore, communication is important to team performance, and effective communication skills should be developing during training in a team context. Research suggests that team communication is generally associated with team performance. Many of these studies take place within the context of aviation, health care, or the military (Helmreich, Merritt, & Wilhelm, 1999; Smith-Jentsch, Zeisig, Acton, & McPherson, 1998). For example, voice data has demonstrated that flight crews that communicated more overall tended to perform better, and those crews that exchanged more information about flight status committed fewer errors (Helmreich & Foushee, 1993). They also found that more standard and more predictable forms of communication are related to lower error rates. Similarly, Stout et al. (1994) rated the quality of communication within teams operating a flight simulator, and found it to be associated with higher levels of team performance. However, communication frequency alone does not always relate positively to team performance. For example, Smith and colleagues (1994) found that communication frequency was negatively related to performance in top-management teams, and suggested that greater communication frequency may be indicative of higher levels of conflict. Therefore, research should focus on the type of communication (i.e., whether they are being done in the context of coordination and cooperation) as well as frequency when examining its influence on team effectiveness. Team Training Given the importance of teamwork for interdependent tasks (i.e., workflow that emphasizes distributed expertise and coordination), the literature on team training has developed a wide range of techniques delivered to intact teams and targeted at the team level. The results of this work have clearly supported team training as an effective means of producing team learning 11 and performance improvement (Salas et al., 2007). In a recent meta-analytic investigation, Salas et al. (2008) examined the efficacy of interventions targeting teamwork skills such as mutual performance monitoring, coordination, communication, and decision making (e.g., CannonBowers et al., 1995; Stagl, Salas, & Fiore, 2007). Team coordination and adaptation training is one example of a team training intervention that is focused on providing teamwork skills content. The results showed that teamwork interventions led to enhancements in team performance. Additionally, teams exhibited improvements in cognitive outcomes (e.g., declarative knowledge gains), affective outcomes (e.g., trust, confidence, positive attitudes), and behavioral outcomes (e.g., decision making, coordination, situational assessment) following teamwork training (Salas et al., 2008). Although the evidence demonstrates that team training is effective in promoting team learning and performance, with very few exceptions (e.g., Chen, Thomas, & Wallace, 2005; DeShon et al., 2004) the extant research on team training has primarily focused on teamlevel outcomes (e.g., team performance, team mental models, etc.) to the neglect of individuallevel learning processes or outcomes within these team-training contexts (Chen & Klimoski, 2007; Dierdorff & Ellington, 2012). This lack of work exists even though researchers have suggested that a thorough understanding of any training intervention requires consideration of influences from multiple levels (Mathieu & Tesluk, 2010). As previously noted, team performance for highly interdependent tasks is a collective consequence of taskwork and teamwork. Therefore, team training needs to focus on how to help members master the knowledge and skills required by their own roles in teams, in addition to helping them become experts at executing its collective tasks. In the following section, I review the core theoretical 12 processes – individual and team regulation – that are likely to influence learning and performance at both levels. Regulation Processes Over the past 25 years, the training literature has embraced self-regulation as the dominant model of learning, motivation, and performance at the individual level (Kanfer, Chen, & Pritchard, 2008; Karoly, 1993). There are a variety of specific self-regulation theories, including control theory (Carver & Scheier, 1998; Klein, 1989), goal setting theory (Locke & Latham, 1990), and social cognitive theory (Bandura, 1997). Across these specific theories, they share a core set of concepts and processes that have developed a broad base of empirical support (Karoly, 1993). In essence, individuals regulate their attention and effort around goals, where goals are “mental representations of desired states” (Vancouver & Day 2005, p. 158). Feedback allows individuals to compare their current performance relative to the desired goal state. If a discrepancy exists between the individual’s desired goals and their actual performance, it creates a motivational force to adjust their effort or strategies to reduce the discrepancy (Carver & Scheier, 1998). This process operates in a cyclical and iterative fashion to regulate behavior and increase the probability of goal attainment, and accounts for individual learning, motivation, and performance over time. Feedback plays a critical role in this regulatory process. Although virtually all research on self-regulation has focused on individuals, recent research indicates that the regulatory model can be extended to the team level to account for team learning, motivation, and performance. DeShon et al. (2004) noted that interdependent team tasks require team members to regulate cognitive and behavioral resources around both individual and team goals. In such tasks, team members need to strive to achieve their own 13 individual goals, but they also must provide assistance to team members to achieve distinct team objectives. Individual team members need to make resource allocation decisions with respect to how much attention and effort they will devote to individual goals relevant to one’s role in the team, while also contributing to the team goal by coordinating collective effort. In their conceptual model, there are two goal-feedback loops referencing both individual and team goals. The feedback loop for the individual goal monitors discrepancies between current performance and individual goal state, whereas the feedback loop for the team goal monitors discrepancies between current performance and the team goal state. Research suggests that, all things being equal, individuals would focus regulatory resources on the goal loop with the largest discrepancy (Schmidt & DeShon, 2007). However, DeShon et al. (2004) proposed that the nature of the situation or surrounding environmental context may increase the salience of either of the feedback loops, which can bias individuals to prioritize either the individual or team goal. Based on this conceptual multiple-goal model, DeShon et al. (2004) developed a process model that treats regulation, learning, and performance in teams as homologous across the individual and team level. The meaning of this homology is that motivational constructs and goal processes are functionally similar across levels, in that the individual-level relationships among self-efficacy, individual effort regulation, and individual performance are highly similar (i.e., homologous) to the team-level relationships among team efficacy, team effort regulation, and team performance. Therefore, the homologous model can account for learning and performance at both the individual and team level of analysis. Additionally, they proposed that performance feedback that focuses on their individual goal would bias individuals to allocate resources towards their individual goal, whereas performance feedback that focuses on their team goal 14 would bias individuals to allocate resources towards their team goal. In a laboratory experiment that examined 79 teams of three, DeShon et al. (2004) demonstrated that regulatory processes responsible for individual effort allocation and individual performance also substantially hold at the team level. The provision of individual and team feedback was the primary factor in influencing resource allocation decisions that ultimately contributed to individual or team performance. Team members who received only individual feedback focused their attention and effort on individual performance, which resulted in the highest level of individual performance by the end of training. Conversely, team members who received only team-level feedback were more likely to focus on team performance, which resulted in the highest team-oriented performance by the end of training. Provision of both types of feedback encouraged tradeoffs between individual and team goals, indicating that individuals have difficulty regulating at both levels simultaneously (DeShon et al. 2004). Chen et al. (2005) also examined the multilevel regulation process as team members pursue both individual and team objectives. However, their focus was on how the regulatory processes accounted for skill transfer and adaptation following training (i.e., an additional performance trial characterized by greater complexity and difficulty compared to prior trials). Building on the work by Marks, Mathieu, and Zaccaro (2001), they proposed that team work is characterized by recurring cycles of goal-directed activity that can be divided into two distinctive phases: transition and action phase. In the transition phase, teams engage in evaluation or planning activities designed to foster goal attainment. In the action phase, teams perform work activities that directly contribute to goal accomplishment. Over time, teams repeatedly cycle through these transition and action phases to accomplish their team goals. This concept is consistent with the distinction made by self-regulation theories between goal choice and goal- 15 striving processes at the individual level. Chen et al. (2005) proposed that goal choice and goalstriving activities would mediate the relationship between individual training outcomes (individual knowledge, skill, and efficacy) and individual adaptive performance, whereas transition and action processes would mediate the relationship between team training outcomes (team knowledge, skill, and efficacy) and team adaptive performance. In general, they showed that these post-training regulation processes related similarly to adaptive performance at both the individual and team level, lending further support to the view that regulation is a multilevel process in the team context. In sum, there is emerging support that the process of self-regulation can be extended to the team level to account for team learning and performance. This suggests that, as individuals engage in taskwork training, they must self-regulate so that they focus their attention on their individual task, monitor progress toward their goal using feedback, and making behavioral adjustments based on the feedback as needed. An analogous but qualitatively distinct process may occur as teams engage in teamwork training. Team members must monitor the behaviors of other team members and the team as a unit to ensure that the team goal is achieved. For example, team members might need to monitor other team members’ performance to detect team goal discrepancies and provide mutual error correction to reduce discrepancies (Fleishman & Zacarro, 1992). The team also needs to develop a shared interpretation of team feedback so that the team has a common understanding of what discrepancies exist and how to collectively resolve them. Thus, despite the fact that taskwork skills are more cognitive in nature whereas teamwork skills are more social in nature, the learning processes are similar across levels. 16 Adaptive Guidance to Develop Taskwork and Teamwork The previous discussion highlighted the fact that the regulatory processes responsible for taskwork learning may be highly similar (i.e., homologous) to the regulatory processes responsible for teamwork learning. Therefore, it is possible that training interventions designed to facilitate self-regulation and taskwork development can also be generalized to the team level to promote team regulation and teamwork development. The focus of the present research is to examine whether adaptive guidance, which is a potent feedback strategy that has been shown to help individuals develop complex skills (Bell & Kozlowski, 2002), can be used to develop both taskwork and teamwork skills in teams. It can be expected that adaptive guidance targeted at taskwork deficiencies should focus individuals’ attention toward taskwork skill development, whereas adaptive guidance targeted at teamwork deficiencies should focus the team’s attention toward teamwork skill development. This perspective is consistent with the results by DeShon et al. (2004), which demonstrated that provision of individual-level feedback biased individuals to devote resources towards individual goals, whereas provision of team-level feedback biased individuals towards team goals. However, the findings by DeShon et al. (2004) suggest that individuals generally are not able to use individual and team feedback to simultaneously improve individual and team performance because regulatory resources are limited and therefore tradeoffs must be made. Thus, the proposed research contributes to the team training literature by examining how to sequence adaptive guidance in order to develop individual taskwork skills, while still ensuring that the critical teamwork skills are effectively developed. This requires taking a team developmental perspective. In the following section, I review the literature on adaptive guidance. 17 I then review the normative model of team compilation (Kozlowski et al., 1999), and discuss its implications for taskwork and teamwork development. Past Research on Adaptive Guidance Past research has indicated that trainees given high levels of control over their learning in technology-based learning environments often display poor learning outcomes. In general, research has consistently shown that individuals are not good judges of what or how much they need to learn and practice (Tennyson & Rothen, 1979; Tennyson & Buttery). As a result, trainees often terminate study and practice prematurely, and overlook important learning opportunities (e.g., Brown, 2001). However, research has shown that guiding or advising individuals as they progress through training enhances their ability to make effective learning decisions (Tennyson, 1980). With advisement, trainees are provided with information necessary to make good decisions about what to study and practice, but still allows them to retain a sense of control over their learning. Adaptive guidance is an extension of the advisement strategy approach by utilizing selfregulation as a theoretical foundation and extending advisement research to more cognitively complex tasks (Bell & Kozlowksi, 2002). Adaptive guidance does not replace traditional performance feedback. Instead, it supplements it with additional diagnostic and interpretive information to make effective learning decisions in the future. There are three key features of adaptive guidance: (1) It provides diagnostic information about current and desired levels of performance (2) it provides individual recommendations on what to focus on and what behaviors to practice, and (3) it focuses on not only the content but also the sequence of trainees’ study and practice. 18 First, adaptive guidance provides evaluative information that allows trainees to better understand the discrepancies between current and desired levels of performance. In traditional advisement research, trainees can only infer their current learning progress by the amount of instruction that is recommended. Adaptive guidance, however, provides specific information regarding the extent to which trainees have learned certain concepts or performed certain behaviors well, which enables trainees to better understand which skills and strategies still need improvement, as well as where proficiency is already high so that the trainee’s cognitive and behavioral resources can be directed accordingly (Bell & Kozlowksi, 2002). Second, adaptive guidance provides learners with diagnostic and interpretive information that enables them to understand how to overcome their deficiencies. It uses computer technologies to monitor individuals’ progress, and provides trainees with tailored recommendations in order to meet the different needs of individuals. Based on participants’ prior performance, the system adaptively suggests what materials to study and what behaviors to focus on to achieve task mastery. This includes recommendations on what they should think about and how to think about it (e.g., metacognitive strategies; Nelson, Dunlosky, Graf, & Narens, 1994), and what behaviors they should practice next (Early, Connolly, & Ekegren, 1989). This additional information allows trainees to make effective decisions about how best to deploy their attentional resources and effort, and promotes the integration of concepts and the development of task strategies. As a result, training effectiveness will be enhanced. Third, adaptive guidance does not only identify areas in need of improvement, it also sequences learning objectives as individuals progress through training. Learning is a gradual process in which the mastery of fundamental and basic knowledge and skills is a prerequisite to learning more difficult and complex knowledge and skills. Trainees must develop basic skills 19 first before proceeding to more complex and strategic aspects of the task that builds on those basic skills. If learning objectives are improperly sequenced, trainees may fail to develop critical skills, which can inhibit the acquisition of more complex skills. Therefore, adaptive guidance suggests a sequence of learning to help individuals focus attention on complex skills and tactics only after the fundamental knowledge and skills have been achieved. In an experimental study, Bell and Kozlowski (2002) compared a learner control condition with an adaptive guidance condition in a radar simulation learning environment. Participants in both conditions completed multiple training trials, each consisting of a cycle of study, practice, and performance feedback. However, the adaptive guidance condition also provided trainees with guidance advice to supplement the performance feedback. It compared trainees’ performance to normative standards in order to differentiate whether or not they were performing well on different aspects of the task. These standards were used to determine the guidance a trainee received, although trainees were not aware of the cutoff scores or percentiles. Based on the standards, adaptive guidance provided evaluative information to help the trainee calibrate current progress and then suggested actions the trainee could take to improve deficiencies. Results showed that adaptive guidance helped trainees make better learning decisions than those who received no guidance. Trainees who received adaptive guidance studied and practiced the material in a more appropriate sequence, exhibited higher levels of basic and strategic knowledge, and ultimately demonstrated higher levels of performance than those in the pure learner control condition. Although this preliminary research suggests that adaptive guidance may be a valuable tool for developing complex skills in individuals, questions remain about whether adaptive guidance can be used to effectively develop both individual and team competencies. The purpose 20 of the present research, then, is to examine whether adaptive guidance can be used to improve both individual skills (i.e., taskwork) and team skills (i.e., teamwork), which are the two critical components of team performance. Based on the results of DeShon et al. (2004), I expect that adaptive guidance targeted at taskwork should focus individuals’ attention toward taskwork skills that require further development, whereas adaptive guidance targeted at teamwork should focus the team’s attention toward teamwork processes that still need improvement. However, DeShon et al. (2004) demonstrated that providing both individual and team feedback may overwhelm individuals’ regulatory capacity, resulting in suboptimal performance for both individual and team goals. Similarly, I expect that teams that receive adaptive guidance targeted at both taskwork and teamwork simultaneously will not be able to maximize taskwork and teamwork skill development during training. Thus, the challenge is how to design training interventions in order to maximize the development of both components of team performance. One critical, unresolved factor in developing adaptive guidance for team training is how feedback and training content should be sequenced. As previously mentioned, Bell and Kozlowski (2002) emphasized that learning objectives within adaptive guidance must be sequenced appropriately to maximize training effectiveness. Unfortunately, as noted by numerous researchers, team training research has largely neglected the role of time in intervention delivery; research has generally been nonchalant with respect to how training interventions should be sequenced over the course of team development (Salas et al., 2008). This represents an important gap in our understanding of team training effectiveness. Thus, in the following section, I review the literature on team development to understand how teams develop and evolve over time across their lifespan. 21 Team Development Team development models seek to explain how team form, mature, and evolve over time (Morgan, Salas, & Glickman, 1993). Research on team development can help us identify the critical phases of a team’s lifespan, how they build upon and transition from one to another, and the primary processes that take place as teams progress in their development. One perspective is offered by Kozlowski et al.’s (1999) normative model of team compilation, which synthesizes a diverse literature base to identify content, processes, and outcomes that are relevant at different phases of the team’s lifespan. There are two key dynamic features of the theory. First, the cyclical and iterative nature of team tasks provide opportunities for learning and skill acquisition, and is linked to a regulation cycle. Consistent with the Marks et al.’s (2001) portrayal of team tasks as “episodic” patterns of transition and action, Kozlowski et al. (1999) suggests that teams cycle from low load prior to task engagement, to high load as the team engages the task, and back to low load at the end of the task episode. This regulatory process allows teams to set developmental goals during low load, monitor performance during task engagement, and interpret feedback and diagnose errors as the task cycles back to low load. Second, teams progress through a series of developmental phases, during which they (a) shift attention to learning increasingly more advanced skills, and (b) progress from an individual self-focus (e.g., what do I need to do to perform my task) to a focus on the team as a collective (e.g., how do we coordinate and adapt) as team capabilities are acquired. Thus, team compilation is viewed as a learning process across levels and time (Kozlowski et al. 1999). With repeated episodes, individual knowledge and skills compile upwards across focal levels to encompass team learning, skill acquisition, and performance. 22 Their normative model highlights four distinct developmental phases, and different content, processes, and outcomes are proposed to be relevant at each of the different phases. Initially, during the team formation phase, members focus on establishing an interpersonal foundation and shared understanding of the norms and goals with each other through informal socialization processes. Then, during the task compilation phase, members focus on developing individual task competencies and self-regulation skills. After they achieve individual task mastery, they shift their focus to role compilation as members focus on developing a good understanding of their own and teammates’ roles and responsibilities, and begin to establish routines to guide interactions. Finally, they transition to the team compilation phase, where the team focuses on how they can best manage interdependencies in both routine and novel situations by attending to the network of relationships that connect team members to each other. In sum, Kozlowski et al.’s (1999) normative model provides a prescriptive foundation for the design of training interventions that would promote the development of performance capabilities throughout the team’s lifecycle. The model makes clear the different knowledge and skills that teams must acquire during each developmental phase. Moreover, because the developmental phases are progressive and enable the development of increasingly more advanced skills, teams that fail to effectively engage in the critical processes at earlier phases of development and acquire the necessary cognitive, affective, and behavioral outcomes will face significant challenges in learning and performance at later phases of development. Thus, training interventions should be sensitive to the different primary concerns of teams during each phase of their development to help them effectively navigate the compilation process and promote higher levels of team performance. 23 There are no comprehensive tests of this meta theoretical framework, likely because it is complex and dynamic, which makes it difficult to study in its entirety. However, preliminary research has examined some of its core propositions by testing more constrained models derived from the broader theoretical framework. Dierdorff, Bell, and Belohlav (2011) used a business simulation to examine how different facets of psychological collectivism (i.e., preference, reliance, concern, norm acceptance, and goal priority) predicted team performance across the team lifespan. Results indicated that teams composed of members high on Preference and Concern were better able to navigate through the team formation phase and have stronger initial team performance than teams composed of members low on Preference and Concern. This is because individuals socialize into the team and try to develop an understanding of the new team situation during the team formation phase, and Preference and Concern reflect affinity toward the team and its members. In contrast, they found that teams with members high in Norm Acceptance and Goal Priority displayed higher levels of end-of-state team performance. This is because Norm Acceptance and Goal Priority promote harmony within the team and encourage team members to prioritize team goals over personal goals, which are facets that are particularly important in the last phase of compilation (i.e., team compilation). DeShon, Kozlowski, Schmidt, Wiechmann, and Milner (2001) provided initial support for the proposition that team members should shift from an individual focus to a team focus during team training. Using a team task simulation, they asked participants in the team building condition to focus on both individual and team goals throughout training. On the other hand, participants in the team compilation condition were instructed to only focus on individual goals early in training, and to only focus on team goals later in training. Results showed that trainees in the team compilation condition exhibited superior individual performance early in training, 24 superior team performance later in training, and better adaptive performance at the end of training relative to trainees in the team building condition. These preliminary results have relevance to the design of adaptive guidance for teams, as it supports the notion that the focal level of development shifts across levels over time as skills compile from the individual to the team level. Therefore, adaptive guidance should be sequenced appropriately to augment this developmental transition. Specifically, adaptive guidance should help individuals focus attention and effort on the development of taskwork competencies early in training. As trainees master taskwork skills, adaptive guidance should shift their attention to the team level and to focus on developing teamwork competencies. Research Goals The purpose of the current study is to examine the effects of a shifting focus adaptive guidance relative to a holistic focus adaptive guidance on taskwork and teamwork skill development. Self-regulation research suggests that adaptive guidance focused on both taskwork and teamwork simultaneously would hinder the development of both individual and team skills because individuals have difficulty regulating at both levels simultaneously. In contrast, a shifting focus adaptive guidance that shifts in focus from taskwork to teamwork over time should be more effective; individuals should be able to devote their attentional and behavioral resources towards learning the critical taskwork skills early in training, and this provides a foundation for learning the fundamental teamwork skills later in training (Kozlowski et al., 1999). Therefore, it is predicted that individuals in the shifting focus condition should improve their taskwork skills at a faster rate early in training, improve their teamwork skills at a faster rate later in training, 25 and have the highest level of team performance by the end of training relative to the holistic focus condition. To examine this research question, the current study uses an adaptive guidance intervention that is more sophisticated than the one used by Bell and Kozlowski (2002). Guidance advice is similarly provided at the end of each training trial to help individuals calibrate current progress and suggest what materials to study and what behaviors to practice for the next scenario. However, corrective recommendations are also embedded within the training trials, and delivered in real-time and as needed in response to inefficient or ineffective taskwork or teamwork behaviors. Thus, adaptive guidance provides team members with immediate feedback on deficiencies and suggestions on how to overcome them. Hypotheses Before I present the hypotheses that are examined in the current study, I will first denote the specific taskwork and teamwork requirements that are central to the current problem domain. Cannon-Bowers et al. (1995) suggested that team competency requirements vary as a function of the task performed and the type of team activity required. By denoting the specific taskwork and teamwork requirements, I can specify underlying mechanisms by which adaptive guidance improves team performance. Figure 1 depicts my proposed typology to distinguish the fundamental taskwork and teamwork requirements for collaborative knowledge-building in teams. In this task context, team members possess distributed and specialized expertise, and must extract information from the environment and integrate it with team members’ information within a given time frame. Team members must share their information with other team members so that all relevant information 26 is usable by all team members (Fiore, Cuevas, Scielzo, & Salas, 2007). The team then needs to collaboratively integrate and synthesize the unique knowledge in addition to common knowledge, thereby increasing the team’s ability to develop a shared understanding of the problem space and to generate effective solutions to the problem. Problem-Solving Phase Teamwork Competency Taskwork Knowledge Acquisition • Individual technical skills required to learn and share information that is within domain of specialization • Team coordination skills required to effectively share information with the rest of the team Knowledge Application • Individual’s ability to integrate the learned information and make individual judgment about the best course of action • Team’s ability to integrate the different perspectives of team members and achieve agreement on a solution to the problem Figure 1. Typology of Taskwork/Teamwork Requirements of Knowledge-Building Teams. The team competencies can broadly be distinguished in terms of two dimensions to yield a 2 x 2 typology. The first dimension represents which problem-solving phase the team is in: knowledge acquisition or knowledge application. The knowledge acquisition phase centers on the acquisition of relevant information and the compilation of a pool of knowledge for the team. Next, the team transitions to the knowledge application phase, which is focused on using the information that the team learned and arriving at a team consensus on a solution. This distinction is consistent with the theoretical framework of team learning (macrocognition) in collaborative 27 contexts proposed by Fiore, Rosen et al. (2010), which focuses on delineating the knowledge building processes and problem-solving phases for these types of teams. The second dimension of the typology represents the taskwork and teamwork components within each of the phases; that is, this typology proposes that both the knowledge acquisition phase and the knowledge application phase contains unique taskwork and teamwork components. In brief, the team knowledge building process requires team members to learn information within their unique domain of specialization so that it can be shared with the rest of the team. Therefore, they must have the technical knowledge and skills necessary to effectively and efficiently complete those parts of the team task (knowledge acquisition taskwork). In team contexts with distributed expertise though, individuals also learn information from other team members that would otherwise be unavailable due to lack of accessibility or expertise. Therefore, coordinative effort is required by the team to prepare for, communicate, and interpret new information to develop a comprehensive collective understanding of the problem space (knowledge acquisition teamwork). After team members have acquired as much information as they can within the given time frame, they transition to the knowledge application phase. First, members assign weights to each piece of information, which represents its perceived relevance or importance based on the individual’s expertise or past experiences. They then integrate all acquired information to produce an individual judgment about what is the most effective solution to the problem (knowledge application taskwork). Finally, the team needs to take into account the perspectives and insights of each individual member and achieve agreement on a solution to the problem (knowledge application teamwork). The taskwork and teamwork components are discussed in greater detail in the following sections. 28 Attention to Knowledge Acquisition Taskwork One important taskwork component that underlies the knowledge acquisition capabilities of teams with distributed expertise is the need for team members to learn information within their unique domain of specialization (Fiore et al., 2007). This is exemplified by a physician in an emergency medical team reading an EKG test to acquire information about a patient’s cardiac health. In this case, the team member learns information that he or she had access to, and can learn without input from other team members. Once team members learn the information, they can then share it with the rest of the team. Therefore, it is of critical importance for team members to develop the individual technical knowledge and skills required to learn and share information that only they can access or interpret due to their role specialization. For example, a sonar technician on a submarine must learn how to reliably assess information about underwater threats using his listening equipment, and to ensure that all potential threats are accounted for. Additionally, in order for team members to exchange information with each other, they must develop the technical knowledge and skills required to operate any equipment important for information sharing. For example, submarine crew members must learn how to operate the undersea telephone to communicate with the rest of the team, and how to operate the radio to communicate with distant command centers. Based on the prescription foundation provided by Kozlowski’s (1999) team development model, the shifting focus condition is designed to sequence the focus of self-regulatory activity from taskwork to teamwork over the course of training. During the early parts of training, individuals are provided with adaptive guidance related to knowledge acquisition taskwork, and this should focus individuals’ attention towards practicing knowledge acquisition taskwork. Additionally, at the end of each trial, it focuses their study time on developing the knowledge 29 acquisition taskwork competencies that still need improvement. On the other hand, individuals in the holistic focus condition are provided with adaptive guidance related to both taskwork and teamwork throughout the training; from a self-regulation perspective, providing feedback on knowledge acquisition taskwork and teamwork simultaneously can be expected to hinder individuals’ attention and effort towards knowledge acquisition taskwork skill development because tradeoffs must be made in the pursuit of both goals (DeShon et al., 2004). Therefore, it is expected that individuals in the shifting focus condition would devote greater attention towards knowledge-acquisition taskwork early in training, relative to the holistic focus condition. Hypothesis 1(a): During the early stages of training, individuals in the shifting focus condition will devote greater attention to practicing knowledge acquisition taskwork compared to individuals in the holistic focus condition (averaging across early trials). Hypothesis 1(b): During the early stages of training, individuals in the shifting focus condition will devote greater study time to improve knowledge acquisition taskwork compared to individuals in the holistic focus condition (averaging across early trials). Quality of Knowledge Acquisition Taskwork Adaptive guidance does not merely draw attention to performance deficiencies. It also provides information on how to overcome them. The adaptive guidance related to taskwork delivers corrective recommendations in real-time and as needed in response to ineffective knowledge acquisition taskwork behaviors. These recommendations are designed to help individuals develop the technical knowledge needed to perform basic task functions that underlie the learning and sharing of information, and ensure that individuals are effectively and efficiently learning all relevant information that is within their unique domain of specialization. 30 Additionally, based on a participant’s prior performance, adaptive guidance adaptively suggests which individual knowledge acquisition taskwork skills need improvement so that their study attention can be directed accordingly. Through practice and experience, declarative knowledge begins to be compiled or proceduralized and trainees will be able to execute knowledge acquisition taskwork activities quicker and with fewer errors (Anderson, 1982). However, it is expected that individuals in the shifting focus condition will demonstrate greater improvement in knowledge acquisition taskwork during the early stages of training, compared to the holistic focus condition. As previously mentioned, individuals in the holistic focus condition are likely to focus on both knowledge acquisition taskwork and teamwork simultaneously. This can draw regulatory resources needed for mastery of knowledge acquisition taskwork early in training. On the other hand, individuals in the shifting focus condition are likely to primarily regulate attention and effort around knowledge acquisition taskwork during the early stages of training. Therefore, I predict that individuals in the shifting focus condition will demonstrate faster improvement in the efficiency (how fast they can acquire all relevant information within their unique domain of specialization) and accuracy (how much incorrect/irrelevant information is learned or shared) of their knowledge acquisition taskwork. Additionally, they should have higher levels of knowledge acquisition taskwork by the midpoint of training. Hypothesis 2(a): During the early stages of training, individuals in the shifting focus condition will demonstrate faster improvement in the efficiency and accuracy of their knowledge acquisition taskwork compared to individuals in the holistic focus condition. 31 Hypothesis 2(b): By the midpoint of training, individuals in the shifting focus condition will demonstrate higher levels of efficiency and accuracy of knowledge acquisition taskwork compared to individuals in the holistic focus condition. Attention to Knowledge Application Taskwork Like the knowledge acquisition process, the knowledge application process also contains both taskwork and teamwork components. This conceptualization is consistent with multilevel theory of team decision-making (Hollenbeck et al. 1995), which posits that the decision-making process of teams with distributed expertise consists of both individual and team processes. Within this approach, team members need to first systematically gather information deemed to be relevant to the problem at hand. Once the information is gathered, each team member assigns weights to each piece of information, which represents its perceived relevance and importance based on the individual’s expertise or past experiences. They then need to integrate all of the acquired information to form an individual judgment about effective solutions to the problem. Members may have different information because not all information has been properly shared to everyone. Members may also have the same information, yet assign different weights based on their own expertise and other situational factors. Only after all team members have integrated the information and formed individual judgments can the team focus on making a consensual judgment. Therefore, team members need to have the proper training to make effective individual judgments based on the expertise that they bring to the team. During the early parts of training, individuals in the shifting focus condition are provided with adaptive guidance related to knowledge application taskwork, which will focus individuals’ attention and effort towards practicing knowledge application taskwork. Additionally, at the end 32 of each trial, it focuses their study time on developing knowledge application taskwork competencies that still need improvement. Conversely, the holistic focus condition provides individuals with adaptive guidance related to both knowledge application taskwork and teamwork simultaneously throughout training; thus, they would be forced to make tradeoffs between focusing on developing knowledge application taskwork or teamwork skills. It is expected that individuals in the shifting focus condition would devote more attention towards knowledge application taskwork early in training, relative to the holistic focus condition. Hypothesis 3(a): During the early stages of training, individuals in the shifting focus condition will devote greater attention to practicing knowledge application taskwork compared to individuals in the holistic focus condition (averaging across early trials). Hypothesis 3(b): During the early stages of training, individuals in the shifting focus condition will devote greater study time to improve knowledge application taskwork compared to individuals in the holistic focus condition (averaging across early trials). Quality of Knowledge Application Taskwork Adaptive guidance delivers corrective recommendations that is aimed at improving individuals’ ability to carry out effective knowledge application taskwork. Of most relevance to knowledge application taskwork is the ability of each team member to use their expertise to accurately weigh the importance of all pieces of information/cues that the team has gathered, which enables them to make accurate judgments about what is the best course of action to take from their perspective. The embedded adaptive guidance provides real-time suggestions on whether individuals are integrating the information properly, and provides information on how to overcome these deficiencies. Additionally, based on a participant’s performance, adaptive 33 guidance adaptively suggests which individual knowledge application skills need improvement so that their study attention can be directed accordingly for the next trial. Over time, participants should be able to improve their ability to integrate the information properly and make accurate judgments. It is expected that individuals in the shifting focus condition will demonstrate greater improvement in knowledge application taskwork during the early stages of training, compared to the holistic focus condition. Because individuals in the holistic focus condition are likely to focus on improving both knowledge application taskwork and teamwork, it will draw regulatory resources needed to achieve knowledge application taskwork mastery. On the other hand, individuals in the shifting focus condition are encouraged to focus solely on knowledge application taskwork during the early stages of training. Therefore, I predict that individuals in the shifting focus condition will demonstrate faster improvement in the relative quality of their knowledge application taskwork (i.e., how accurate individual judgments are, given the information acquired). Additionally, they should have higher relative quality knowledge application taskwork by the midpoint of training. Hypothesis 4(a): During the early stages of training, individuals in the shifting focus condition will demonstrate faster improvement in the quality of their knowledge application taskwork compared to individuals in the holistic focus condition. Hypothesis 4(b): By the midpoint of training, individuals in the shifting focus condition will demonstrate higher quality knowledge application taskwork compared to the holistic focus condition. 34 Attention to Knowledge Acquisition Teamwork Individuals in a team can acquire knowledge and increase their personal understanding of a given domain by interfacing with information sources they can access directly. However, members of knowledge-building teams commonly possess distributed and specialized expertise, and so the team must rely on one another to relay task-critical information that no one else in the team could access or interpret (Sundstrom, De Meuse, & Futrell, 1990). Hence, the team must engage in knowledge acquisition teamwork to share and distribute uniquely held knowledge to others within the team so that the team can reach a collectively shared and agreed upon understanding of all relevant knowledge in the team task environment. The sharing process requires coordinative effort on the part of the sender and receiver of information (Grand, Braun, Kuljanin, Kozlowski, & Chao, 2016). The team member who wants to share information must ensure that other team members are ready to receive information. Then, he or she must communicate the information to others, and then confirm that the shared information has been received and understood. Conversely, team members with whom information is shared need to be attentive, engage in effortful processing to internalize the communicated knowledge, and acknowledge that the new information has been received (i.e., closed-loop communication). Thus, teams must develop routines and procedures so that all team members are prepared to receive, communicate, and interpret new information when needed. During the later parts of training, the shifting focus condition introduces the adaptive guidance related to knowledge acquisition teamwork. Because individuals have become proficient in knowledge acquisition taskwork early in training, it should become more proceduralized and require less attentional resources to execute. As a result, they can subsequently shift their attention to practicing knowledge acquisition teamwork. Additionally, 35 they can focus their attention on performance feedback metrics and recommendations related to knowledge acquisition teamwork competencies that still need improvement. On the other hand, individuals in the holistic focus condition are provided adaptive guidance related to knowledge acquisition taskwork and teamwork throughout the training, which can inhibit the mastery of knowledge acquisition taskwork skills early in training. To the extent that individuals in the holistic focus condition have not yet become proficient in knowledge acquisition taskwork, it is expected that they will continue to make resource tradeoffs between developing knowledge acquisition taskwork and teamwork skills. Therefore, teams in the shifting focus condition will devote more attention towards developing knowledge acquisition teamwork skills later in training, relative to the holistic focus condition. Hypothesis 5(a): During the later stages of training, teams in the shifting focus condition will devote greater attention to practicing knowledge acquisition teamwork compared to teams in the holistic focus condition (averaging across later trials). Hypothesis 5(b): During the later stages of training, teams in the shifting focus condition will devote greater study time to improve knowledge acquisition teamwork compared to teams in the holistic focus condition (averaging across later trials). Quality of Knowledge Acquisition Teamwork I previously highlighted the important teamwork components that underlie the knowledge acquisition capabilities of teams with distributed expertise. Adaptive guidance related to knowledge acquisition teamwork delivers a set of corrective recommendations to improve the team’s ability to carry out these teamwork components (e.g., promoting consistent participation across members, minimizing missed communications). The embedded adaptive guidance 36 provides real-time suggestions in response to inefficient or ineffective sharing behaviors observed during the knowledge sharing process. The interventions were specifically designed to encourage the team to develop a commonly held procedure and strategy for sharing information and acquiring shared information, and to facilitate more equal participation among members during information sharing. Additionally, task coordination behaviors rely heavily on communication. Researchers have suggested that communication is a necessary component of coordination (e.g., Brannick, Roach, & Salas, 1993) and teams that experience communication breakdowns are likely to experience problems with their coordination processes (Marks et al., 2001). Therefore, adaptive guidance encourages team members to develop communication skills to facilitate the coordination process. Finally, based on the team’s prior performance, adaptive guidance adaptively suggests which knowledge acquisition teamwork skills need improvement so that their study attention can be directed accordingly for the next trial. It is expected that individuals in the shifting focus condition will demonstrate greater improvement in knowledge acquisition teamwork during the later stages of training, compared to the holistic focus condition. The team compilation model suggests that individuals must first develop skills essential for knowledge acquisition taskwork before they can effectively develop knowledge acquisition teamwork competencies. Because individuals in the holistic focus condition are less likely to fully master knowledge acquisition taskwork skills early in training, it will hinder their efforts to develop their knowledge acquisition teamwork skills later on. On the other hand, individuals in the shifting focus condition are more likely to achieve knowledge acquisition taskwork mastery earlier in training compared to individuals in the holistic focus condition; therefore, teams will focus attention on knowledge acquisition teamwork when they can maximally benefit from the teamwork recommendations provided by 37 the adaptive guidance. So, although it might be expected that their knowledge acquisition teamwork skills will be relatively poor during the early parts of training (prior to the introduction of adaptive guidance related to teamwork), they should demonstrate a faster improvement in communication skills during the later stages of training. Because communication is a means of enabling coordination among team members, it is expected that teams will demonstrate greater improvement in coordination skills as well. This should result in higher communication and coordination skills by the end of training, relative to teams in the holistic focus condition. Hypothesis 6(a): Teams in the shifting focus condition will initially demonstrate lower levels of communication skills for coordination (prior to the introduction of teamwork adaptive guidance), relative to teams in the holistic focus condition. However, during the later parts of training, they will improve their communication skills at a faster rate. Hypothesis 6(b): Teams in the shifting focus condition will initially demonstrate lower levels of coordination skills (prior to the introduction of teamwork adaptive guidance), relative to teams in the holistic focus condition. However, during the later parts of training, they will improve their knowledge application teamwork skills at a faster rate. Hypothesis 6(c): By the end of training, teams in the shifting focus condition will demonstrate higher levels of communication and coordination skills compared to teams in the holistic focus condition. Attention to Knowledge Application Teamwork Team members play a role in the knowledge application process by providing the team with information and recommendations based on their interpretation of the data available (knowledge application taskwork). However, in teams without a formal hierarchy (e.g., 38 leaderless groups), it is the collective responsibility of the team to integrate this information, work together to derive a consensus, and select or propose the final recommendation. Therefore, training is needed to ensure that team members can effectively combine acquired information, and have the communication skills necessary to reach a consensus decision accurately. Communication is essential for consensus-making because, among other things, it allows group members to (a) pool their individual perspectives and recommendations (Barnlund, 1959), (b) check for errors and reject incorrect suggestions (Taylor & Faust, 1952), and (c) exert positive influence over the decisional preferences of others (Riecken, 1958). During the later parts of training, the shifting focus condition introduces the adaptive guidance related to knowledge application teamwork. Because individuals have become proficient in knowledge application taskwork early in training, they can subsequently shift their attention to practicing knowledge application teamwork. Additionally, they can focus their attention on performance feedback metrics and recommendations related to knowledge application teamwork competencies that still need improvement. On the other hand, individuals in the holistic focus condition are provided with adaptive guidance related to knowledge application taskwork and teamwork throughout the training, which can inhibit the mastery of knowledge application taskwork skills early in training. To the extent that individuals in the holistic focus condition have not yet become proficient in knowledge application taskwork, it is expected that they will continue to make resource tradeoffs between developing knowledge application taskwork and teamwork skills. Therefore, teams in the shifting focus condition will devote more attention towards knowledge application teamwork later in training, relative to the holistic focus condition. 39 Hypothesis 7(a): During the later stages of training, teams in the shifting focus condition will devote greater attention to practicing knowledge application teamwork compared to teams in the holistic focus condition (averaging across later trials). Hypothesis 7(b): During the later stages of training, teams in the shifting focus condition will devote greater study time to improve knowledge application teamwork compared to teams in the holistic focus condition (averaging across later trials). Quality of Knowledge Application Teamwork Adaptive guidance is designed to improve the core mechanisms associated with knowledge application teamwork (e.g., ensuring that all information from team members are accurately combined, promoting communication to arrive at a consensus). The embedded adaptive guidance provides real-time feedback whenever information is incorrectly combined and interpreted, and provides information on how to overcome these deficiencies. Additionally, the embedded adaptive guidance encourages team members to communicate with each other whenever the team has difficulty in reaching an accurate consensual decision. Finally, based on the team’s prior performance, adaptive guidance adaptively suggests which knowledge application teamwork skills need improvement so that their study attention can be directed accordingly for the next trial. It is expected that individuals in the shifting focus condition will demonstrate greater improvement in knowledge application teamwork during the later stages of training, compared to the holistic focus condition. Individuals in the holistic focus condition are less likely to develop proficiency in knowledge application taskwork skills early in training; according to the team compilation model, this will hinder their efforts to develop their knowledge acquisition teamwork skills later in training, On the other hand, individuals in the shifting focus condition 40 are more likely to achieve knowledge application taskwork mastery earlier in training, and will therefore maximally benefit from the teamwork recommendations provided by the adaptive guidance. So, although it might be expected that their knowledge application teamwork skills will be relatively poor during the early parts of training (prior to the introduction of adaptive guidance related to teamwork), they should demonstrate a faster improvement in communication skills critical to the knowledge application process during the later stages of training, along with faster improvement in their knowledge application teamwork skills (i.e., how accurate team judgments are, given the information learned). This should result in higher levels of communication skills and knowledge application teamwork quality by the end of training, relative to teams in the holistic focus condition. Hypothesis 8(a): Teams in the shifting focus condition will initially demonstrate lower levels of communication skills related to knowledge application (prior to the introduction of teamwork adaptive guidance), relative to teams in the holistic focus condition. However, during the later parts of training, they will improve their communication skills at a faster rate. Hypothesis 8(b): Teams in the shifting focus condition will initially demonstrate lower quality knowledge application teamwork (prior to the introduction of teamwork adaptive guidance), relative to teams in the holistic focus condition. However, during the later parts of training, they will improve their knowledge application teamwork skills at a faster rate. Hypothesis 8(c): By the end of training, teams in the shifting focus condition will demonstrate higher levels of communication and knowledge application teamwork quality compared to teams in the holistic focus condition. 41 Team Knowledge Outcomes The focus of this study is on teams composed of members with distributed, specialized expertise who must actively acquire relevant problem domain information and use it to solve unique, complex, and challenging problems. These knowledge-building teams must strive to reach a collectively shared and agreed upon understanding of all relevant knowledge in order to adequately capture a problem space. It is proposed that teams in the shifting focus condition will have a positive effect on the level of knowledge held by teams. The shifting focus adaptive guidance directs individual attention towards knowledge acquisition taskwork. Since the shifting focus condition only focuses on resolving taskwork errors in the beginning, trainees will be more likely to absorb the embedded guidance information and be more confident in improving their taskwork skills. During the later stages of training, the shifting focus condition directs the team’s attention towards knowledge acquisition teamwork. Since teams have achieved individual taskwork mastery, trainees are able to better use the adaptive guidance information in improving their knowledge acquisition teamwork skills. Overall, then, trainees in the shifting focus condition should demonstrate a faster increase in team knowledge outcomes later in training compared to the holistic focus condition. Additionally, it is expected that teams in the shifting focus condition will have the highest levels of team knowledge outcomes by the end of training. Hypothesis 9(a): Teams in the shifting focus condition will initially demonstrate lower levels of team knowledge outcomes (prior to the introduction of teamwork adaptive guidance), relative to teams in the holistic focus condition. However, during the later parts of training, they will improve their team knowledge outcomes at a faster rate. 42 Hypothesis 9(b): By the end of training, teams in the shifting focus condition will demonstrate higher levels of team knowledge outcomes compared to teams in the holistic focus condition. Team Decision Outcomes The ultimate goal of knowledge-building teams is to actively acquire relevant problem domain information so that it can be used to inform themselves of what actions to take to solve complex problems. Therefore, this study also examines the extent to which the shifting focus condition will enhance the decision-making capabilities of the team compared to the holistic focus condition. The shifting focus adaptive guidance directs individual attention towards knowledge application taskwork early in training. This allows trainees to improve the accuracy in which they can interpret acquired information, which provides the basis for accurate team decisions. During the later stages of training, the shifting focus condition directs the team’s attention towards knowledge application teamwork, so that the team develops a better ability to combine the team members’ unique perspectives into a consensual decision. Additionally, because teams in the shifting focus condition are more likely to acquire all relevant knowledge in a problem space, their chosen solution are more likely to be based on a proper understanding of the problem space, and are therefore more likely to be correct. Overall, then, trainees in the shifting focus condition should demonstrate a faster increase in team solution accuracy later in training compared to the holistic focus condition. Additionally, it is expected that teams in the shifting focus condition will have the highest levels of team solution accuracy on the final trial. Hypothesis 10(a): Teams in the shifting focus condition will initially be less likely to make the accurate team choice (prior to the introduction of teamwork adaptive guidance), relative to 43 teams in the holistic focus condition. However, during the later parts of training, they will improve their probability of making the accurate team choice at a faster rate. Hypothesis 10(b): By the end of training, teams in the shifting focus condition will be more likely to make the accurate team choice compared to teams in the holistic focus condition. 44 METHOD Sample Data were collected from 369 undergraduate psychology and management students from a large Midwestern university (k = 123 teams). There were two conditions: a shifting focus condition and a holistic focus condition. A total of 180 students (60 teams) participated in the shifting focus condition and 189 students (63 teams) participated in the holistic focus condition. Twelve teams were dropped from the study for failing to complete all required portions of the study. As a result, the final sample consisted of 333 students (k = 111 teams), with 54 teams in the shifting focus condition and 57 teams in the holistic focus condition. Overall, the sample was 53 percent female and 47 percent male. Participants on average were 19.1 years of age (sd = 2.4). All participants were recruited through the online psychology and management Sona system and received course credit for participation in the study. Procedures Participants logged into the Sona system, and selected the “CRONUS” experiment from a list of all possible active psychology and management experiments. Upon selecting the study, students completed an online consent form (or opted out of the experiment). At this point, participants were asked to watch a 10-minute online instructional video introducing the objectives and task mechanics of CRONUS, and were given a list of possible in-person lab sessions to sign up for. After participants selected a time and date for the in-person lab session, they completed surveys for demographics. 45 When participants arrived for the in-person lab session, they were greeted by the experimenter and were assigned into three-member teams. Teams were randomly assigned to either the shifting focus condition or the holistic focus condition. Team members were not physically seated together so that they could not communicate or interact with each other except via the computer task interface. Therefore, the teams were composed of spatially distributed team members who initially had low familiarity with each other, and low experience with working together as part of a team. After all participants arrived and were seated in front of their assigned lab computer, they were shown a 20-minute instructional video on the basic functions as well as the rules and objectives of the CRONUS computer task. The instructional video reiterated the information presented in the online instructional video, and also provided more detailed instructions on how to perform specific task functions (e.g., switching between maps, locating and posting obstacles, sharing obstacle information, etc.). At the conclusion of the instructional video, the participants logged into the computer task with their unique university identification number and completed two practice trials. These practice scenarios allowed teams to familiarize themselves with the basic task mechanics of CRONUS. After completing the practice scenarios, the teams completed eight performance trials. After all scenarios had been completed (or participation for 2.5 hours), the students were provided a debriefing form that explained the purpose of the present study and were dismissed from the session. Experimental Task Teams engaged in the Crisis Relief Operation Naval Unit Simulation (CRONUS), which is a timed team learning and decision-making task that was built for a three-person team with distributed expertise; team members with different role specializations must find, learn, share, 46 and integrate information in order to make critical team choices across a number of scenarios, each with a time limit of 8 minutes. CRONUS simulated a crisis relief effort that the Navy and other government organizations engage in after a natural disaster such as the devastating earthquake in Japan in 2011. It presented a series of problem scenarios to teams; each scenario consisted of a mission, such as transporting medical supplies from A to B, or escorting physicians from A to B. The primary objective for each scenario was for teams to select one of three possible routes identified on a map in order to travel between a starting point and some ending destination. To make this choice, teams needed to learn about the presence of cues relevant to decision-making (i.e., obstacles along each route that could have impeded their travel). Each team member was assigned an explicit team role (i.e., Transport, Intel, Engineer) that allowed them to identify and learn certain obstacle information that no other team members could access (unique information), although there was also certain information that all team members could access (common information). Team members were aware they each possessed distinct expertise and sources of information and needed to work together to fully acquire the necessary knowledge for the team to make effective decisions. Additionally, after 6 minutes have elapsed, members were no longer be able to acquire or share any more information, and could only enter their final choices. A countdown timer was displayed on each member’s screen indicating the amount of time remaining in the trial. During each scenario, team members were able to communicate with each other via an in-game chat box. To identify and learn about obstacles, each team member had a Specialist Map screen that only he or she could view and access. The map was divided into a 3 by 3 grid. There was also an information center on the top-right part of the screen and a bank of all possible obstacles 47 on the bottom-right part of the screen. Clicking in a grid square revealed any obstacles that were present on the routes passing through that grid square. The team member’s Specialist Map contained information about two classes of obstacles: common obstacles and unique obstacles. Common obstacles were accessible to all members of the team (e.g., all members can initially learn about obstacles related to jungle terrain). Unique obstacles were only accessible to a single member of the team because of their specialized task role (e.g., only the Transport member can initially learn about obstacles related to poor road conditions). These unique obstacles located on members’ specialist map represented the unique expertise that the members brought to the team. Therefore, no single individual was able to locate and learn about all the obstacles present on the map, and needed to rely on other teammates to relay critical information about other obstacles that existed. Each scenario contained 15 obstacles total, three of which were common and 12 of which were unique across members (each of the three members had access to 4 unique obstacles). Once a member identified that a particular obstacle was present, they were able to “post” this information to their Specialist Map. By posting obstacles, team members documented that they learned about the presence of an obstacle. It also made obstacles permanently visible, which allowed them to easily determine what obstacles were on each route when they had to make decisions. Individuals posted obstacles by clicking on the relevant obstacle icon in the icon bank at the bottom right corner of the game screen, and then chose the appropriate route and grid space that they wished to post the obstacle on. Individuals also had access to a commonly shared workspace labeled the Mission Map screen, which allowed them to share information about obstacles with other team members. When team members posted obstacle information to the Mission Map, other team members had 48 access to that shared information. Once obstacle information was shared, the other team members had a limited period of time (10 seconds) to view that information before it disappeared and needed to be shared again. Team members needed to view the shared information and post that information on their own Specialist Map in order to acknowledge that they have learned about the presence of the shared obstacle. Therefore, teams needed to develop an effective coordination process so that all relevant knowledge could be efficiently shared to other team members before time was up. After learning and sharing all obstacles, or when time was up and the team was no longer able to post obstacles, team members were required to integrate the information to select the proper course of action to take using their Mission Command screen. Each team member had two unique assets that they could have used to travel to the destination point. For example, the Transport role had access to two Transport assets to resolve cues (obstacles). However, each team member was only able to choose to bring one asset, and each asset differed in its strength and weaknesses against the obstacles encountered along the routes. Thus, based on what obstacles the team has learned are on each route and how effective each of their asset was in overcoming the obstacles, teams needed to decide which route to follow and which assets to bring that would have most effectively enabled the team to reach the desired destination point. The Mission Command screen provided information about how effective each of their asset was in overcoming each type of obstacle (represented by an effectiveness score, with a higher number being better), along with a summary chart of what obstacles have been identified by the team to be present on each of the three routes. Using this information, individual team members first calculated and inputted how effective each of their asset was in traveling along each route. Afterwards, each team member selected the most effective asset for each of the three routes, and 49 then selected which asset and route combination was most effective for him or her overall. Once an asset and route combination had been chosen, they confirmed their choice by clicking the “Confirm Personal Choice” button located on the right side of the screen. Finally, the team selected the route they thought they should take, given all members’ asset choices and effectiveness scores. The Mission Command screen provided information to all participants about what scores their teammates had calculated for each route. For the team choice, participants selected the route with the overall highest effectiveness score across all three team members. Clicking on the “Confirm Team Choice” button on the right side of the screen brought up a pop-up box that asked the team whether they were certain that the route selected is the one that they wished to team to choose. When all team members selected “Yes”, the scenario ended. At the end of each scenario, teams were provided feedback on a variety of metrics that were relevant to their knowledge acquisition taskwork and teamwork knowledge, their knowledge application taskwork and teamwork, as well as their overall team performance outcomes (team knowledge outcomes and team solution accuracy). Additionally, if individuals clicked on a particular feedback metric, they were provided with general recommendations on how to improve their performance on the chosen metric. For example, if they clicked on the metric about the accuracy of their information integration process, they were provided advice on how to accurately calculate the effectiveness of their assets for each route given the obstacles learned. While reviewing feedback at the end of each trial, team members were also able to communicate with each other via a chat box. Experimental Manipulation I developed adaptive guidance prompts that were designed to improve teams’ ability to carry out taskwork or teamwork behaviors, and they were embedded within each scenario. 50 Specifically, one set of embedded adaptive guidance prompts provided real-time recommendations to individuals in response to ineffective knowledge acquisition and knowledge application taskwork behaviors (e.g., how to learn/post obstacle information, how to share obstacle information, which obstacles are irrelevant obstacles, how to calculate route scores and make route selections). Another set of embedded adaptive guidance prompts provided real-time recommendations to the team in response to ineffective or inefficient knowledge acquisition or knowledge application teamwork behaviors (e.g., how to coordinate/cooperate through communication, how to integrate information to make accurate route selections). The prompts were delivered through a pop-up window on the screen of team members that performed the triggering action; when the window appeared, individuals read the recommendation provided, closed the window, and then continued on with the task. These prompts were based on prior work (Grand et al., 2016), but have been modified and extended for the current study. Table 1 and Table 2 provides a description of the adaptive guidance prompts to facilitate taskwork and teamwork, respectively. 51 Table 1. Description of embedded adaptive guidance prompts to promote taskwork. Competency Knowledge Acquisition Taskwork Prompt Name Avoid Distractors Prompt Description Triggered when a member posted an irrelevant obstacle to the SM or MM. Advised to delete the post and not post this obstacle type in the future. Purpose Help team member understand which obstacles are irrelevant and that they should ignore them Repeat Common Obstacle Triggered when a member posted a common obstacle to the MM. Advised to delete the post and not post this obstacle type to the MM in the future. Help team member understand which obstacles are known by all team members, reducing ambiguity over presence of information Help players understand that they have to be thorough in learning obstacles on their SM, ensuring no unique information is forgotten Help players understand when and why they have learned or shared an incorrect obstacle. Help players understand that they should post obstacles shared to them onto their SM, not back onto the MM. Help players understand how to properly choose their personal route choice Incomplete Specialist Map Incorrect Post Wrong Map Incorrect Choice/Scores Knowledge Application Taskwork Relative Decision Incorrect Triggered when a member posted an obstacle to the MM before learning all available obstacles on the SM. Advised to search SM further before posting additional obstacles to MM. Triggered when a member posted an obstacle to the SM or MM that did not exist. Advised to delete the post and to double check the route and obstacle type. Triggered when a member posted an obstacle on the MM that was previously shared by another team member. Triggered when a member’s personal route choice is incorrect, given the asset scores that they inputted. Advised to double check route scores and asset choices Triggered when a member selects an incorrect asset choice for any of the three routes, given the obstacles learned. Note. SM = Specialist Map; MM = Mission Map 52 Help players understand how to properly calculate asset scores and choose the correct assets for each route Table 2. Description of embedded adaptive guidance prompts to promote teamwork. Competency Prompt Name Prompt Description Triggered when a member shares an obstacle, but not all team members have learned all obstacles on their SM. Advises all team members to let the rest of the team know when ready to learn from others. Triggered when team members collectively encounter high number of missed post notifications. Advise team members to develop and follow a strategy during the sharing process, and to communicate with each other in the process Triggered when a member missed an obstacle shared by teammate. Advised to monitor notification that indicates obstacle information has been shared by team member Triggered when a member posted an obstacle to the SM or MM that did not exist. Advised to delete the post and to double check the route and obstacle type. Purpose Encourage team members to communicate with each other so that everybody on the team is ready to share obstacles with each other. Encourage team members to develop a common sharing strategy to reduce the number of missed sharing attempts Decision Too Soon Triggered when team members are making decisions, but not all obstacles are fully learned, and there are at least two minutes left. Help players to be more thorough in their sharing process before they start making decisions. Double Check Decisions Triggered when a team member makes an incorrect asset-route choice. Asks other members on the team to double check each others’ work. Encourage members to help each other out when someone on the team has difficulty with decision-making Decision Too Slow Triggered when 30 seconds are remaining in scenario and not all team members have calculated scores for all routes yet. Advise the team to budget time better in the future, and to remind them to help out team members if they struggle in decision-making Remind members to budget time better, and to help team members to quickly calculate scores and make asset choices. Communication: Ready to Share Communication: Sharing Coordination Knowledge Acquisition Teamwork You Missed Teammate Missed Knowledge Application Teamwork 53 Help players understand how to look for information shared by team members Help players understand when a team member missed an obstacle, and remind them to re-share obstacle to them. Table 2 (cont’d). No Consensus Triggered when at least one member of the team chose a different team route choice from the rest of the team. Advise all team members to use the chat box to reach a consensus Triggered if the team route choice is incorrect, given all team members’ asset scores. Advise team members to double check scores, and remind them that they should pick the route with the highest overall score Note. SM = Specialist Map; MM = Mission Map Incorrect Relative Team Choice 54 Encourage team members to interact with each other when they fail to reach a consensus, facilitating the consensusmaking process Help teams calculate team scores correctly and make accurate team decisions In addition to the adaptive guidance prompts that were embedded within each scenario, I also developed adaptive guidance prompts that were provided at the end of each trial while team members reviewed their performance feedback. These prompts took the form of on-screen indicators that highlighted which metrics the trainees were performing low on so that they were able to focus on improving those aspects of performance. Specifically, one set of embedded adaptive guidance prompts highlighted which knowledge acquisition and knowledge application taskwork metrics needed improvement, thereby encouraging team members to focus on studying the general recommendations on how to improve their performance on those specific taskwork metrics. Another set of adaptive guidance prompts highlighted which knowledge acquisition and knowledge application teamwork metrics needed improvement, encouraging the team to focus on studying the general recommendations designed to improve performance on those specific teamwork metrics. As team members focused on teamwork metrics that need improvement, they were also able to chat with each other via the chat box in order to strategize about how to improve for the next scenario. To examine the extent to which team learning and performance could be facilitated if adaptive guidance transitions from a self-focus to a team-focus over time, participants were randomly assigned into either a holistic focus condition or a shifting focus condition. The adaptive guidance prompts were introduced at the start of the first performance scenario for both conditions. However, the order in which they were presented differed between conditions. Teams in the holistic focus condition were presented with both taskwork and teamwork adaptive guidance at the start of the first performance scenario, and throughout the rest of the experiment. On the other hand, teams in the shifting focus condition were presented with embedded guidance prompts and feedback recommendations that focused on knowledge acquisition and knowledge 55 application taskwork during the first four performance scenarios. After the fourth performance scenario, embedded guidance prompts and feedback recommendations that focused on knowledge acquisition and knowledge application teamwork were introduced. Measures The CRONUS task recorded a log file containing every task-relevant action (i.e., mouse click) for all trials. Actions were automatically categorized and time-stamped, thus making it possible to track precisely when, what, and how knowledge was being acquired and distributed during each trial. Additionally, the log file recorded every message that team members deliver to each other via the in-game chat box. Demographics Demographic information was collected online prior to the lab experiment. Information that were assessed included academic major, gender, age, and year in school. Knowledge Acquisition Taskwork Practice For the CRONUS task, the primary knowledge acquisition taskwork competencies that needed to be acquired were the technical knowledge and skills to effectively learn and share relevant obstacle information (i.e., how to identify and learn obstacles that only the individual can access due to his or her role specialization; how to share the information with team members). Therefore, I assessed individuals’ attention towards practicing knowledge acquisition taskwork by counting the number of actions made that were relevant to locating and learning information that was unique to the individual’s expertise; also included were the number of 56 actions made that were relevant to correcting taskwork mistakes, which included learning or sharing incorrect/irrelevant obstacle information. These actions were summed for each trial. Knowledge Application Taskwork Practice For the CRONUS task, knowledge application taskwork required individuals to identify how effective each of their unique assets were at overcoming the obstacles found along each of the three routes. This allowed them to decide which route they would personally take and which asset they would bring to most effectively travel to the destination point. Therefore, I assessed individuals’ attention towards knowledge application taskwork practice by counting and summing together the number of actions made that were relevant to inputting, modifying, and confirming these individual choices during each trial. Knowledge Acquisition Taskwork Study Time The data collected by the feedback page provided information about what recommendations each participant studied after each trial and how long each recommendation was studied for. An individual’s attention to studying knowledge acquisition taskwork was calculated by summing the total amount of time (in seconds) that each individual spent studying recommendations related to knowledge acquisition taskwork for each trial. Knowledge Application Taskwork Study Time The amount of attention that individuals devoted to studying knowledge application taskwork was calculated by summing the total amount of time (in seconds) that each individual spent studying recommendations related to knowledge application taskwork. The 57 recommendation of greatest relevance to knowledge application taskwork related to whether or not individuals made the correct individual route choice, given all the obstacles that they identified were present (i.e., obstacles that were posted on the specialist map). Knowledge Acquisition Teamwork Practice For the CRONUS task, the primary knowledge acquisition teamwork requirements included sharing obstacle information with other team members, and learning obstacle information from other team members. Therefore, I assessed the team’s attention towards practicing knowledge acquisition teamwork by counting and summing together the number of actions that the team collectively made that were related to sharing obstacle information to team members (i.e., posting obstacles on the mission map), identifying obstacle information that is shared, and learning obstacle information that is shared (i.e., posting obstacles shared to you on the specialist map). Knowledge Application Teamwork Practice The amount of attention that a team devoted to practicing knowledge application teamwork was represented by summing the number of task-relevant actions performed by the team collectively that are related to knowledge application teamwork. For the CRONUS task, this included actions such as inputting, modifying, and confirming team selections during each trial. 58 Knowledge Acquisition Teamwork Study Time The amount of attention that teams devoted to studying knowledge acquisition teamwork was calculated by summing the total amount of time (in seconds) that the team collectively spends studying recommendations related to knowledge acquisition teamwork. Recommendations that were relevant to such teamwork included advice on how to improve the number of obstacles shared to team members, and how to reduce the number of times team members missed a sharing attempt. Knowledge Application Teamwork Study Time The amount of attention that teams devoted to studying knowledge application teamwork was calculated by summing the total amount of time (in seconds) that the team collectively spent studying recommendations related to knowledge application teamwork. The primary recommendation that was relevant to such teamwork was whether the team made the correct route choice, relative to what had been posted on everybody’s specialist map. Quality of Knowledge Acquisition Taskwork Two distinct measures were used to capture an individual’s technical ability to effectively learn and share relevant obstacle information. The first measure captured the efficiency of knowledge acquisition taskwork. This was measured by assessing the time (in seconds) until the individual learned all relevant obstacle information within their unique domain of specialization. Individuals who located and posted on their Specialist Map all relevant obstacle information in a shorter amount of time were said to be more efficient in fulfilling their knowledge acquisition taskwork duties. 59 The second measure captured the accuracy of knowledge acquisition taskwork. This was measured by assessing the number of incorrect or irrelevant obstacles that were learned or shared to teammates. Individuals who posted on the specialist map or the mission map fewer incorrect or irrelevant obstacles were said to be more accurate in their knowledge acquisition taskwork. Quality of Knowledge Application Taskwork In the CRONUS task, an individual’s knowledge application taskwork quality was reflected by the accuracy to which team members were able to combine all of their information about obstacles on each route, determine which of their own unique assets would be most effective for each route, and determine which route they would personally take and which asset they would bring to most effectively travel to the destination point. Therefore, two measures were used to assess the quality of knowledge application taskwork. First, I assessed the number of route scores that the individual calculated correctly, relative to the obstacle information learned on the Specialist Map. For example, if an individual had learned only two obstacles on their Specialist Map, but calculated the route scores correctly based on those two obstacles, then the calculated route score would be correct relative to what obstacles had been learned. This measure was used to represent the degree to which individuals accurately combined all information about obstacles on each route, given the information learned. Second, I assessed the number of correctly chosen assets. This measure was used to represent the degree to which individuals accurately determined which assets were most effective for each route, given the information learned. 60 Quality of Knowledge Acquisition Teamwork In the CRONUS task, teams were required to coordinate the sharing process so that information can be shared, received, and interpreted. If an individual shared information by posting an obstacle to the mission map, the other team members had a limited amount of time to view that information before it disappeared and needed to be shared again. Communication is a necessary component of coordination (e.g., Brannick et al., 1993); therefore, adaptive guidance encouraged the development of communication skills to address the coordinative challenges of the information sharing process. Team members were allowed to communicate with one another via a chat box in the task interface, and the team’s chat history was recorded by the task. Therefore, I reviewed the messages in the chat log and coded each instance of communication relevant to the coordination process (e.g., “I am ready to share my obstacle information next”). The number of messages relevant to coordination that the team collectively sent to each other was used as a proxy for the communication skills of the team for knowledge acquisition teamwork. Coordination skills of the team was measured by assessing the proportion of sharing attempts that the team missed, relative to the total number of sharing attempts. Teams that had fewer missed sharing attempts were said to have better coordination skills for knowledge acquisition teamwork. Missed sharing attempts slowed down the sharing process because that information needed to be shared again to the individual who missed the shared information. Since teams had a limited amount of time to gather all relevant information before selecting what action to take, they needed to develop an effective coordination process so that all information could be learned by all members of the team before time was up. 61 Quality of Knowledge Application Teamwork In the current study, a team’s knowledge application teamwork quality was conceptually defined as the degree to which team members were able to combine all of the individual team member’s insights and perspectives, and come to a consensus about what route the team should take and what assets they should utilize. Therefore, two measures were used to assess knowledge application teamwork. First, I assessed the communication skills that the team possessed that enabled them to come to a consensus on what action to take, and resolve disagreements. To do so, I reviewed the messages in the chat log and coded each instance of communication relevant to the knowledge application process (e.g., “We should take Route A because the assets we chose traverse that route the best”). Second, I assessed the quality of knowledge application teamwork by examining whether the team selected the correct route to traverse and the correct assets to bring, given the obstacle information that was available to the members of the team. Team Knowledge Outcome Team knowledge was conceptualized as the amount of relevant obstacle information that the team collectively knows. This was captured by the number of obstacles learned by all three members of the team by the end of each trial. An obstacle was considered “learned” by a team member if he or she correctly identified an obstacle by learning its specific type and location from the specialist map (no communication needed from other members to learn) or the mission map (communication needed from other members to learn), and posted that obstacle on the specialist map. By posting the obstacle on the specialist map, they documented that they have learned about the presence of the obstacle. Thus, once all three team members posted the obstacle on their Specialist map, it reflected team-level knowledge of the existence of that 62 obstacle. Each scenario contained 15 total obstacles, representing the total knowledge pool to be acquired by the team. Team Decision Accuracy In the CRONUS task, teams needed to select which route to follow and which asset each team member should bring in order to most effectively enable the team to reach a desired destination point. A team choice was correct if the team selected the correct route to take, and each team member chose to bring the most effective asset (coded as 1). Otherwise, it was considered an incorrect decision outcome (coded as 0). Additionally, failure to make a performance action within the allotted time was considered as “bad” as making an inaccurate team choice, and therefore was treated as an incorrect decision outcome as well. 63 Table 3. Individual-level descriptive statistics and correlations. M SD 1 2 3 1 Gender .47 2 3 Age K. Acquisition Taskwork Practice 4 K. Acquisition Taskwork Study 5 K Application Taskwork Practice 6 K Application Taskwork Study 7 K Acquisition Taskwork Quality (Time) 8 K Acquisition Taskwork Quality (Incorrect Obstacles) 9 K Application Taskwork Quality (Route Scores) 10 K Application Taskwork Quality (Assets) 11 Experimental Condition (1 = Shifting Focus) Note. (n = 333 individuals) *p < .05 4 5 6 7 8 9 19.08 2.40 -.05 32.12 6.28 .06 .03 11.18 3.56 -.10 .00 .37* 12.25 4.21 .03 .20* .22* .18 10.18 3.11 .11 -.15* .16* .12* .09 80.96 22.78 -.09 .12* .10 .14 -.07 .36* 1.34 .68 -.06 -.08 -.45* -.21* -.11 .05 -.02 2.31 .32 .141 -.03 .15* -.02 .44* .14* -.02 -.32* 2.55 .21 -.05 .21* .16* -.01 .27* .12* -.00 -.11* .37* - - .07 -.05 .08 .13* -.03 .18* -.04 -.19* -.06 64 10 .01 11 Table 4. Team-level descriptive statistics and correlations. M SD 1 2 1 Team Knowledge 7.86 3.10 3 4 5 6 7 8 9 2 Decision Accuracy .22 .47 .32* 3 K. Acquisition Teamwork Practice 13.12 4.23 .46* .27* 4 K. Acquisition Teamwork Study 10.23 4.11 .12* -.09 .25* 5 K Application Teamwork Practice 3.53 2.54 .06 .06 .00 -.02 6 K Application Teamwork Study 9.69 3.51 -.04 .12 .10 .02 .11 7.18 4.62 .22* -.03 .24* .17* .01 .12 8.94 3.73 -.20* -.21* .16* -.18* .00 .00 -.25* 2.56 0.93 -.16 .12 .05 -.03 -.03 -.00 .02 .04 .49 .15 .11* .48* .05 -.00 .09 .13 .06 -.19* .06 - - .14* -.07 .08 .14* .01 .18* -.03 -.26* -.00 7 K Acquisition Teamwork Quality (Communication) 8 K Acquisition Teamwork Quality (Missed Sharing) 9 K Application Teamwork Quality (Communication) 10 K Application Teamwork Quality (Relative Correct) 11 Experimental Condition (1 = Shifting Focus) Note. (n = 111 teams) *p < .05 65 10 .08 11 RESULTS Because the data in the present study are at multiple levels (time is nested within individuals, which are nested within teams), this clustering must be incorporated into the analyses to avoid inflated Type I errors (Bryk & Raudenbush, 1992). Therefore, multilevel random coefficient modeling ( MRCM) was used when appropriate. Additionally, based on the nature of the proposed effects, it made the most sense to fit the data to a discontinuous functional form. That is, rather than fitting one continuous functional form to the entirety of the data, we have a “break” in the middle to better represent the observed relation (Singer & Willett, 2003). For discontinuous growth models, separate slopes are fit for different segments in the overall trajectory. There are three basic discontinuities that can occur: (a) changes in level, but not in slope; (b) changes in slope, but not in level; and (c) changes in both level and slope. The first two scenarios were not included in the primary analyses because they were familiarization trials prior to the introduction of the adaptive guidance prompts. To examine whether participants in both conditions were relatively equal in performance at the start of the experiment, I conducted a t-test to examine whether teams in each condition differed in the number of obstacles learned by all three members on the second practice scenario. Results showed no significant difference in the means between the two conditions; t(423.33) = 0.78, p = ns. Knowledge Acquisition Taskwork Hypotheses The first set of hypotheses was designed to determine whether individuals in the shifting focus and holistic focus condition differed in their attention towards, and quality of, their knowledge acquisition taskwork during the early half of training. 66 With respect to individuals’ attention towards knowledge acquisition taskwork, Hypothesis 1a predicted that individuals in the shifting focus condition would practice knowledge acquisition taskwork to a greater extent than individuals in the holistic focus condition during early parts of training. To test this hypothesis, the number of actions that individuals performed that were related to practicing knowledge acquisition taskwork was averaged across the first four performance scenarios. This measure was at the individual-level; thus, a two-level MRCM (Level-1: individual, Level 2: team) was used to examine mean differences between the two experimental conditions. Results of the MRCM analysis revealed that individuals in the shifting focus condition performed significantly more actions related to knowledge acquisition taskwork practice across the first four performance trials, relative to individuals in the holistic focus condition (b = 2.18, SE = .512, p < .001). Therefore, Hypothesis 1a was supported. This result indicates that, in the early parts of training, individuals provided with only taskwork adaptive guidance were more likely to practice knowledge acquisition taskwork compared to individuals that were provided with both taskwork and teamwork adaptive guidance. Hypothesis 1b predicted that individuals in the shifting focus condition would study knowledge acquisition taskwork to a greater extent than individuals in the holistic focus condition during early parts of training. To test this hypothesis, the time (in seconds) that individuals spent studying knowledge acquisition taskwork recommendations on the feedback page was averaged across the first four performance scenarios. The mean difference between the two experimental conditions was compared using a two-level MRCM (Level-1: individual, Level 2: team). 67 Results of the MRCM analysis revealed that individuals in the shifting focus condition spent significantly more time studying recommendations related to knowledge acquisition taskwork across the first four performance trials compared to individuals in the holistic focus condition (b = 5.45, SE = .67, p < .001). Therefore, Hypothesis 1b was supported. This result indicates that, at the beginning of training, individuals that were provided with only taskwork adaptive guidance tended to study knowledge acquisition taskwork recommendations longer than individuals that were provided with both taskwork and teamwork adaptive guidance. It was predicted that, because individuals in the shifting focus condition would have higher levels of attention towards practicing and studying knowledge acquisition taskwork during the early parts of training, they would also perform knowledge acquisition taskwork better. Individuals who locate and post all relevant obstacle information that can be found on the Specialist Map in a shorter amount of time can be said to be more efficient in fulfilling their knowledge acquisition taskwork duties. Additionally, individuals who post on the Specialist Map or the Mission Map fewer incorrect or irrelevant obstacles can be said to be more accurate in their knowledge acquisition taskwork. Hypothesis 2a examined whether individuals in the shifting focus condition would have a faster rate of improvement in the efficiency and accuracy of their knowledge acquisition taskwork during the first four performance scenarios. The influence of the experimental manipulation on changes in the efficiency and accuracy of individuals’ knowledge acquisition taskwork across the early scenarios was analyzed using a three-level MRCM to account for the nested data structure (Level-1: scenario; Level-2: individual; Level-3: team). Results from the MRCM analyses for efficiency and accuracy are reported in Table 5 and Table 6 respectively. Table 5 shows that there were no significant intercept differences between 68 conditions for efficiency in knowledge acquisition taskwork (b = 4.81, SE = 9.98 p = ns). Although a significant main effect was observed for the time term (b = -10.83, SE = 2.81, p < .001), there was no significant interaction between condition and time (b = -6.78, SE = 4.01, p = ns). These findings indicate that individuals’ efficiency in knowledge acquisition taskwork in both conditions tended to be the same on the first performance scenario, and individuals in both conditions became more efficient at equal rates over the course of the first four performance scenarios. With respect to knowledge acquisition taskwork accuracy, Table 6 shows that no significant intercept differences were found between conditions (b = -.11, SE = .08 p = ns), although there was a significant main effect for the time term (b = -.07, SE = .03, p < .001). These findings indicate that individuals in both conditions tended to be equally accurate in their knowledge acquisition taskwork on the first performance scenario, and became more efficient over time. However, this main effect was qualified by a significant interaction between conditions (b = -.26, SE = .03, p < .001), demonstrating that individuals in the shifting focus condition increased their knowledge acquisition accuracy at a faster rate than individuals in the holistic focus condition early in training. In sum, providing individuals with only taskwork adaptive guidance (versus providing them with both taskwork and teamwork guidance) early in training did not necessarily make them more efficient in knowledge acquisition taskwork. However, results indicate that focusing individuals on knowledge acquisition taskwork early in training improved their accuracy faster, as measured by the amount of incorrect or irrelevant obstacles posted. Thus, Hypothesis 2a was partially supported. 69 Table 5. Coefficient estimates from MRCM for knowledge acquisition taskwork efficiency across first four performance trials. Variables b Std. Error t-value DV: Time (in seconds) to post all relevant obstacles on SM 7.04 Intercept 119.39* 16.96 2.81 Trial -10.83* -3.86 9.98 Condition 4.81 0.50 4.01 Trial*Condition -6.78 -1.59 * p < .05 Note. Condition is a dummy coded variable (0 = holistic focus; 1 = shifting focus). Table 6. Coefficient estimates from MRCM for knowledge acquisition taskwork accuracy across first four performance trials. Variables b Std. Error t-value DV: Number of incorrect or irrelevant obstacles posted .06 Intercept 2.11* 35.46 .03 Trial -.07* -2.57 .08 Condition -.11 -1.27 .04 Trial*Condition -.26* -6.25 * p < .05 Note. Condition is a dummy coded variable (0 = holistic focus; 1 = shifting focus). Hypothesis 2b examined whether individuals in the shifting focus condition would demonstrate higher levels of efficiency and accuracy of knowledge acquisition taskwork by midpoint of training (scenario 4). This was tested using a two-level multi-level model for each dependent variable (Level 1: individual; Level 2: team) for scenario 4. 70 Results showed no difference between conditions in knowledge acquisition taskwork efficiency (b = -9.82, SE = 10.13 p = ns) at the midpoint of training. However, there was a significant difference in knowledge acquisition taskwork accuracy, such that individuals in the shifting focus condition made fewer knowledge acquisition taskwork mistakes than individuals in the holistic focus condition at the midpoint of training (b = -.97, SE = .11 p < .001). In sum, the results mirror those of Hypothesis 4a; providing individuals with only taskwork adaptive guidance (versus providing them with both taskwork and teamwork guidance) early in training did not necessarily make them more efficient in knowledge acquisition taskwork by the midpoint of training. However, individuals were significantly more accurate in their knowledge acquisition taskwork. Thus, Hypothesis 2a was partially supported. Knowledge Application Taskwork Hypotheses The shifting focus condition was also designed to focus individuals on improving their knowledge application taskwork early in training. The next set of hypotheses examine whether individuals in the shifting focus condition paid more attention to knowledge application taskwork during the early half of training, and whether this lead to higher quality knowledge application taskwork. With respect to individuals’ attention towards knowledge application taskwork, Hypothesis 3a predicted that individuals in the shifting focus condition would practice knowledge application taskwork to a greater extent than individuals in the holistic focus condition during early parts of training. To test this hypothesis, the number of actions that individuals performed that related to knowledge application taskwork practice was averaged 71 across the first four performance scenarios. A two-level MRCM (Level-1: individual, Level 2: team) was used to examine the impact of the experimental manipulations on this measure. Contrary to predictions, results of the MRCM analysis revealed no significant differences between conditions in the amount of knowledge application taskwork practice that individuals performed (b = -.92, SE = .50, p = ns). On average, individuals in both conditions practiced knowledge application taskwork to an equal extent across the first four performance scenarios. Thus, Hypothesis 3a was not supported. Hypothesis 3b predicted that individuals in the shifting focus condition would study knowledge application taskwork early in training to a greater extent than individuals in the holistic focus condition. To test this hypothesis, the time (in seconds) that individuals spent studying knowledge application taskwork recommendations on the feedback page was averaged across the first four performance scenarios. A two-level MRCM (Level-1: individual, Level 2: team) was used to examine the impact of the experimental manipulation on this measure. Results showed that individuals in the shifting focus condition spent significantly more time studying recommendations related to knowledge application taskwork across the first four scenarios compared to individuals in the holistic focus condition (b = 5.42, SE = .66, p < .001). Thus, Hypothesis 3b was supported. At the beginning of training, individuals that were provided with only taskwork adaptive guidance tended to study knowledge applications taskwork recommendations longer than individuals that were provided with both taskwork and teamwork adaptive guidance. Hypothesis 4a proposed that individuals in the shifting focus condition would have a faster rate of improvement in the quality of knowledge application taskwork during the early parts of training compared to individuals in the holistic focus condition. Two measures were 72 used to assess the quality of knowledge application taskwork: the number of route scores that the individual calculated correctly relative to what obstacles they had found (maximum 3 correct), and the number of correctly chosen assets relative to what obstacles they had found (maximum 3 correct). The influence of the experimental manipulation on changes in these measures across the first four performance scenarios was analyzed using a three-level MRCM to account for the nested data structure (Level-1: scenario; Level-2: individual; Level-3: team). Results from the MRCM analyses for the number of relative correct route scores and relative correct asset choices are reported in Table 7 and Table 8 respectively. Table 7 shows that there were no significant intercept differences between conditions for the number of relative correct route scores (b = .05, SE = .12 p = ns). There was also a significant main effect for the time term (b = .05, SE = .02, p < .001), although there was no significant interaction between condition and time (b = -.02, SE = .04, p = ns). Table 8 also shows similar results for the number of relative correct asset choices. No intercept differences were found between conditions (b = .02, SE = .08 p = ns). A significant main effect for the time term was found (b = .06, SE = .02, p < .001), but no significant interaction between condition and time (b = .00, SE = .03, p = ns). In sum, Hypothesis 4a was not supported. Providing individuals with only taskwork adaptive guidance (versus providing them with both taskwork and teamwork guidance) early in training does not appear to be beneficial. Instead, individuals in both conditions tended to be equally accurate in their knowledge application taskwork quality on the first performance scenario, and they also improved at equal rates over the course of the first four performance scenarios. 73 Table 7. Coefficient estimates from MRCM for number of relative correct route scores across first four performance trials. Variables b Std. Error t-value DV: Number of relative correct route scores .08 Intercept 1.60* 19.01 .02 Trial .05* 2.22 .12 Condition .05 0.39 .04 Trial*Condition -.02 -.48 * p < .05 Note. Condition is a dummy coded variable (0 = holistic focus; 1 = shifting focus). Table 8. Coefficient estimates from MRCM for number of relative correct asset choices across first four performance trials. Variables b Std. Error t-value DV: Number of relative correct asset choices .06 Intercept 1.23* 21.65 .02 Trial .06* 2.98 .08 Condition .02 0.31 .03 Trial*Condition .00 .17 * p < .05 Note. Condition is a dummy coded variable (0 = holistic focus; 1 = shifting focus). Hypothesis 4b examined whether individuals in the shifting focus condition would demonstrate higher levels of knowledge application taskwork quality by midpoint of training (Scenario 4). Two-level MRCM analyses (Level 1: individual; Level 2; team) were used to examine the impact of the experimental manipulation on each knowledge application taskwork quality measure for Scenario 4. 74 Results showed no difference between conditions in the number of relative correct route scores (b = .25, SE = .16 p = ns) on Scenario 4, nor was there difference between conditions in the number of relative correct asset choices (b = .11, SE = .07 p = ns) on Scenario 4. Overall, individuals in both conditions demonstrated similar levels of knowledge acquisition taskwork quality at the midpoint training, and so Hypothesis 4b was not supported. Knowledge Acquisition Teamwork Hypotheses The next set of hypotheses examined whether teams in the shifting focus and holistic focus condition differed in their attention towards, and quality of, their knowledge acquisition teamwork during the later parts of training. Hypothesis 5a tested whether teams in the shifting focus condition would practice knowledge acquisition teamwork to a greater extent than teams in the holistic focus condition during the later parts of training. To test this hypothesis, the number of actions that team collectively performed that related to the practicing of knowledge acquisition teamwork was averaged across the last four performance scenarios. This outcome variable was captured at the team level. Thus, a two-sample t-test was conducted to examine mean differences between the two conditions. Results showed a significant difference in the means between the two conditions; t(110.11) = 2.93, p < 0.01. As predicted, examination of the means revealed that teams in the holistic focus condition (M = 15.27) performed significantly fewer actions related to knowledge acquisition teamwork than teams in the holistic focus condition (M = 18.82). Therefore, Hypothesis 5a was supported. 75 Hypothesis 5b predicted that teams in the shifting focus condition would study knowledge acquisition teamwork to a greater extent than teams in the holistic focus condition during the later parts of training. To test this hypothesis, the time (in seconds) that the team collectively spent studying knowledge acquisition teamwork recommendations on the feedback page was averaged across the last four performance scenarios. The mean difference between the two conditions was examined using a two-sample t-test. The results revealed a significant mean difference between the two conditions; t(100.38) = -3.62, p < 0.01. As expected, an examination of the means revealed that teams in the shifting focus condition (M = 12.77) spent more time studying knowledge acquisition teamwork recommendations across the later parts of training than teams in the holistic focus condition (M = 7.81). Thus, Hypothesis 5b was supported. By introducing the taskwork adaptive guidance prompts before the teamwork prompts, team members were able to focus on developing their knowledge acquisition taskwork skills first before shifting their focus on developing their knowledge acquisition teamwork skills later in training. Hypothesis 6a predicted that teams in the shifting focus condition would initially have lower levels of communication to facilitate team knowledge sharing at the midpoint of training compared to the holistic focus condition, but would improve faster over time during the later parts of training. To test this hypothesis, I coded the number of messages relevant to coordination of sharing processes that the teams collectively sent to each other. I was not aware of the experimental condition that teams were in during the coding of messages The influence of the experimental manipulation on changes in team communication across the last five scenarios was analyzed using MRCM to account for the nested data structure (Level-1: scenario; Level-2: team). 76 Results from the MRCM analysis are reported in Table 9. A significant intercept difference was found between conditions, indicating that teams in the shifting focus condition initially had lower levels of communication to facilitate team knowledge sharing at the midpoint of training (b = -.87, SE = .44, p < .05). The interaction between condition and time was in the expected direction, suggesting that teams in the shifting focus condition increased their knowledge sharing communication at a faster rate; however, this result was only marginally significant (b =.39, SE = .20, p < .10). Therefore, Hypothesis 6a was partially supported. Table 9. Coefficient estimates from MRCM for knowledge sharing communication across last five performance trials. Variables b Std. Error t-value DV: Number of messages related to knowledge sharing by the team .30 Intercept 8.60* 28.31 .14 Trial -.14 -1.04 .44 Condition -.87* -1.97 .20 Trial*Condition .39 1.94 * p < .05 Note. Condition is a dummy coded variable (0 = holistic focus; 1 = shifting focus). Hypothesis 6b predicted that teams in the shifting focus condition would initially have lower levels of coordination for knowledge sharing at the midpoint of training compared to the holistic focus condition, but would improve faster over time during the later parts of training. To test this hypothesis, MRCM (Level 1: Time; Level 2:Team) was performed to examine change in the number of missed sharing attempts by the team during the last five performance scenarios. As shown in Table 10, a significant intercept difference was found between conditions (b = 2.53, SE = 1.09, p < .01). Although there was no significant main effect for the time term (b = -.05, SE = .22, p = ns), there was a significant interaction between condition and time (b = -1.90, 77 SE = .31, p < .01). Therefore, Hypothesis 6b was supported; the results show that teams in the shifting focus condition tended to have a higher number of missed sharing attempts at the midpoint of training, before the introduction of the teamwork prompts. However, when the teamwork prompts were introduced, teams in the shifting focus condition demonstrated a faster reduction in the number of missed sharing attempts during the last four performance scenarios. Table 10. Coefficient estimates from MRCM for knowledge sharing coordination across last five performance trials. Variables b Std. Error t-value DV: Number of missed sharing attempts by the team .75 Intercept 6.42* 8.57 .22 Trial -.05 -.25 1.09 Condition 2.53* 2.32 .31 Trial*Condition -1.90* -6.09 * p < .05 Note. Condition is a dummy coded variable (0 = holistic focus; 1 = shifting focus). Hypothesis 6c examined whether teams in the shifting focus condition would demonstrate higher levels of communication and coordination for knowledge sharing by the end of training (final performance scenario). Two-sample t-tests were conducted to examine mean difference between conditions for each dependent variable at the last performance scenario. Evaluation of the t-test results indicated that the experimental manipulation exerted significant effects on the level of communication for knowledge sharing during the final performance scenario; t(111.66) = -2.93, p < .05. Specifically, teams in the shifting focus condition had higher levels of communication for knowledge sharing (M = 9.29) than teams in the holistic focus condition (M = 7.91). Additionally, there was a significant mean difference between the two conditions for the number of missed sharing attempts on the final performance 78 scenario; t(110.11) = -2.93, p < .01. An examination of the means revealed that teams in the shifting focus condition had significantly fewer missed sharing attempts (M = 3.27) than teams in the holistic focus condition (M = 6.83). In sum, these results indicate that knowledge acquisition teamwork is enhanced when adaptive guidance shifts from taskwork to teamwork, thus supporting Hypothesis 6c. Knowledge Application Teamwork Hypotheses The shifting focus condition was also designed to focus teams on improving their knowledge application teamwork later in training. The next set of hypotheses examined whether teams in the shifting focus condition paid more attention to knowledge application teamwork during the latter half of training, and whether this led to higher quality knowledge application teamwork. With respect to teams’ attention towards knowledge application teamwork, Hypothesis 7a examined whether teams in the shifting focus condition would practice knowledge application teamwork to a greater extent than teams in the holistic focus condition during the later parts of training. To test this hypothesis, the number of actions that teams performed that related to the practicing of knowledge application teamwork was averaged across the last four performance scenarios. A two-sample t-test was performed to examine mean differences between the two conditions. Results showed no difference between conditions in the practicing of knowledge application teamwork during the latter half of training; t(99.92) = 1.25, p = ns). The amount of knowledge application teamwork practice in the shifting focus condition (M = 3.21) was similar to that of the holistic focus condition (M = 2.86), and so Hypothesis 7a was not supported. 79 Hypothesis 7b predicted that teams in the shifting focus condition would study knowledge application teamwork to a greater extent than teams in the holistic focus condition during the later parts of training. This hypothesis was tested by averaging the time (in seconds) that the team collectively spent studying knowledge application teamwork recommendations on the feedback page across the last four performance scenarios. The mean difference between the two conditions was examined using a two-sample t-test. Results showed a significant difference between conditions in the amount of time spent studying knowledge application teamwork during the latter half of training; t(95.26) = 5.56, p < .01). As predicted, the amount of time spent studying knowledge application teamwork in the shifting focus condition (M = 12.71) was greater than that of the holistic focus condition (M = 6.55). Therefore, Hypothesis 7b was supported. By introducing the taskwork adaptive guidance prompts before the teamwork prompts, team members were able to focus on studying knowledge application teamwork recommendations later in training. Next, it was predicted that teams in the shifting focus condition would perform knowledge application teamwork better than teams in the holistic focus condition. Specifically, Hypothesis 8a predicted that teams in the shifting focus condition would initially have lower levels of communication to facilitate the team decision making process at the midpoint of training compared to the holistic focus condition, but would improve faster over time during the later parts of training. To test this hypothesis, I coded the number of messages relevant to team decision-making that the team collectively sent to each other. Differences in the rate of change in communication to facilitate team decision making during the last five performance scenarios was analyzed using MRCM to account for the nested data structure (Level-1: scenario; Level-2: team). 80 Results from the MRCM analysis are reported in Table 11. No significant intercept difference was found between conditions (b = -.36, SE = .23, p = ns), nor was there a significant main effect for the time term (b = .08, SE = .06, p = ns) or interaction with condition (b = .04, SE = .09, p = ns). These findings indicate that the average number of messages that team members sent to each other that were relevant to team decision-making tended to be the same at the midpoint of training, and did not increase much during the later parts of training for either condition. Therefore, Hypothesis 8a was not supported Table 11. Coefficient estimates from MRCM for communication to facilitate team decision-making during the last five performance trials. Variables b Std. Error t-value DV: Number of messages related to team decision-making by the team .16 Intercept 1.60* 10.13 .06 Trial .08 1.25 .23 Condition -.36 -1.55 .10 Trial*Condition .04 .46 * p < .05 Note. Condition is a dummy coded variable (0 = holistic focus; 1 = shifting focus). Additionally, Hypothesis 8b predicted that teams in the shifting focus condition would initially have lower levels of knowledge application teamwork accuracy at the midpoint of training compared to the holistic focus condition, but would improve faster over time during the later parts of training. I assessed the accuracy of knowledge application teamwork by examining whether the team selected the correct route to traverse and the correct assets to bring, relative to the obstacles that were learned by the team. A two-level multilevel logistic regression analysis 81 (Level 1: trial; Level 2: team) was conducted to examine change over time during the last five performance scenarios. As shown in Table 12, no significant intercept differences were found between conditions (b = -.53, SE = .27, p = ns), nor was there a significant main effect for the time term (b = .10, SE = .08, p = ns). Although the interaction between condition and time was in the expected direction, it was only marginally significant (b = .22, SE = .12, p < .10). These findings indicate that teams in both conditions were equally likely to make the correct relative team choice at the midpoint of training, and there was no difference in the rate of improvement in the relative correct team choice across the later performance scenarios. Therefore, Hypothesis 8b was not supported. Table 12. Coefficient estimates from multilevel logistic regression for knowledge application teamwork during the last five performance trials. Variables b Std. Error z-value DV: Whether the team made the correct relative team choice .19 Intercept .41* 2.17 .08 Trial .10 1.17 .27 Condition -.53 -1.94 .12 Trial*Condition .22 1.85 * p < .05 Note. Condition is a dummy coded variable (0 = holistic focus; 1 = shifting focus). Hypothesis 8c examined whether teams in the shifting focus condition would demonstrate higher levels of communication for knowledge application teamwork by the end of training, and also accuracy of knowledge application teamwork by the end of training (final performance scenario). A two-sample t-test was conducted to examine the mean difference in level of communication between the two conditions on the final performance scenario. 82 Additionally, logistic regression was used to compare accuracy of knowledge application teamwork between the two conditions on the final performance scenario. T-test results showed no significant mean differences between conditions in the number of messages sent for knowledge application teamwork on the final performance trial; t(103.12) = -.18, p = ns). Additionally, logistic regression results indicated that there were no significant differences between conditions on whether or not the team made the relative team choice correct (b = -.22, SE = .42, p = ns) on the final performance trial, and so Hypothesis 8c was not supported. Team Knowledge Outcomes Hypotheses Hypothesis 9a predicted that teams in the shifting focus condition would initially have lower levels of team knowledge outcomes compared to teams in the holistic focus condition at the midpoint in training. However, it was expected that teams in the shifting focus condition would improve faster over time during the later parts of training. The influence of the experimental manipulation on changes in team knowledge outcomes (number of obstacles learned by all three team members) across the last five performance trials was analyzed using MRCM (Level-1: scenario; Level-2: team). Table 13 shows the results from the MRCM analysis. Although the shifting focus condition had a lower intercept than the holistic focus condition, this difference was not statistically significant (b = -1.04, SE = .76, p = ns). This indicates that teams in both conditions generally learned an equal number of obstacles during the midpoint of training. However, there was a significant interaction between condition and time (b = .88, SE = .16, p < .01), indicating that teams in the shifting focus condition increased team knowledge outcomes at a faster rate 83 during the later parts of training compared to the holistic focus condition. Thus, Hypothesis 9a was partially supported. Table 13. Coefficient estimates from MRCM for team knowledge outcomes during the last five performance trials. Variables b Std. Error t-value DV: Number of obstacles learned by all three team members .52 Intercept 9.62* 18.52 .12 Trial .17 1.52 .76 Condition -1.04 -1.37 .16 Trial*Condition .88* 5.61 * p < .05 Note. Condition is a dummy coded variable (0 = holistic focus; 1 = shifting focus). Additionally, it was predicted that teams in the shifting focus condition would demonstrate higher levels of team knowledge outcomes by the end of training (final performance scenario). The mean difference in team knowledge outcomes between the two conditions for the final performance scenario was examined using a two-sample t-test. Results showed a significant difference between conditions in the number of obstacles learned by all three team members during the final performance scenario; t(110.05) = 2.05, p < .05). As predicted, the number of obstacles learned by teams in the shifting focus condition (M = 12.23) was greater than that of the holistic focus condition (M = 10.71). These results indicate that team knowledge outcome is enhanced when adaptive guidance shifts from taskwork to teamwork, thus supporting Hypothesis 9b. 84 Team Decision Outcomes Hypotheses Hypothesis 10a predicted that teams in the shifting condition would initially be less accurate in their team decision outcomes at the midpoint of training, but would demonstrate faster improvement during the later parts of training. Multi-level logistic regression (Level 1: trial; Level 2: team) was used to examine change over time in the team’s decision accuracy during the last five performance scenarios. Results are reported in Table 14. No significant intercept differences were found between conditions (b = -.73, SE = .21, p = ns), which indicates that both conditions were generally equal in its accuracy in team decision at the midpoint of training. Additionally, although the interaction between time and condition was in the expected direction, this effect was not significant, suggesting that teams in the shifting focus condition were not improving faster over time in their team decisions compared to teams in the holistic focus condition. Therefore, Hypothesis 10a was not supported. Table 14. Coefficient estimates from multilevel logistic regression for team decision outcome during the last five performance trials. Variables b Std. Error z-value DV: Whether the team made the correct team choice .21 Intercept -.73* -3.50 .10 Trial .13 1.24 .29 Condition .22 .79 .14 Trial*Condition .15 1.11 * p < .05 Note. Condition is a dummy coded variable (0 = holistic focus; 1 = shifting focus). Additionally, it was predicted that teams in the shifting focus condition would demonstrate higher levels of accuracy in their team decision outcome by the end of training 85 (Scenario 8). Logistic regression was used to compare accuracy of team decisions between the two conditions on Scenario 8. The results showed no statistically significant difference in the probability of getting the correct team decision on the final performance trial. Although teams in the shifting focus condition had a higher estimate, this result was not statistically significant (b = .44, SE = .40, p = ns). Therefore, Hypothesis 10b was not supported. 86 DISCUSSION One of the challenges of designing training interventions for teams is the fact that learning is inherently an individual-level psychological process. At the same time, for many types of team tasks, individual taskwork does not directly yield team performance. Teamwork skills are also necessary when team success is dependent on the ability of members to coordinate actions and work together as an interdependent unit. Therefore, it is important to understand how to effectively train taskwork skills, and bring those individual skill together to form coordinated and integrated teams. The current study sought to advance understanding of the role of time in the delivery of taskwork and teamwork training interventions. In contrast to the vast majority of team training research which tends to neglect the role of time in intervention delivery (Salas et al., 2008), I draw on the normative model of team development (Kozlowski et al., 1999) to understand whether and how these training interventions should be sequenced. This normative model would suggest that team learning and performance is a multilevel developmental process by which individual skills compile to the team level. Additionally, individual and team regulation provides a means by which individuals shift from an individual-focus to a team-focus. Thus, the primary goal in this study was to compare a shifting focus and holistic team training strategy through the delivery of adaptive guidance feedback. The holistic team training strategy is consistent with many traditional team training approaches, which endeavor to focus trainee attention to both individual (taskwork) and team (teamwork) skill development simultaneously. In contrast, the shifting focus training strategy is designed to sequence the focus of self-regulated attention from taskwork to teamwork skills. From a self-regulation perspective, teams are unable to regulate at both the individual and team level simultaneously, and a failure to 87 develop individual taskwork skills will hinder the development of teamwork skills. Based on this rationale, it was expected that the shifting focus adaptive guidance would result in better taskwork skills early in training, better teamwork skills later in training, and have better performing teams overall. Key Findings It was predicted that the shifting focus adaptive guidance would benefit knowledge acquisition taskwork skills during the initial stages of training relative to holistic focus adaptive guidance. Indeed, the results showed that individuals in the shifting focus condition practiced knowledge acquisition taskwork to a greater extent than individuals in the holistic focus condition during the first half of training. Additionally, they also spent more time studying knowledge acquisition taskwork recommendations. These results suggest that shifting focus condition was successful in focusing their regulatory effort on improving their knowledge acquisition taskwork skills. The results also showed that these individuals tended to make fewer knowledge acquisition taskwork mistakes than those in the holistic focus condition, although they were not more efficient in performing their knowledge acquisition taskwork duties. Overall, the benefits of the shifting focus condition on developing knowledge acquisition taskwork skills early in training was generally supported. Additionally, the shifting focus adaptive guidance resulted in superior knowledge acquisition teamwork later in training. Teams in the shifting focus condition focused more of their attention on studying knowledge acquisition teamwork recommendations during the latter half of training. Additionally, although teams in the shifting focus condition initially had lower levels of knowledge sharing communication and coordination at the midpoint of training, they 88 improved at a faster rate once the teamwork prompts were introduced, resulting in the highest level of communication and coordination by the end of training. The successful learning and development of knowledge acquisition taskwork skills set the stage for successful learning and development of knowledge acquisition teamwork skills later in training. Taken as a whole, these results demonstrate the efficacy of the shifting focus approach to the knowledge acquisition component of team performance, as compared to a traditional holistic focus approach. Indeed, teams in the shifting focus condition improved their team knowledge outcomes at a faster rate during the latter half of training, and had the highest level of team knowledge outcomes by the end of training. Although these results provide some evidence supporting the idea that the sequencing of attention from taskwork to teamwork is beneficial, an alternative explanation for these results may be that participants in the holistic focus condition were cognitively overloaded by the adaptive guidance prompts delivered to them. Specifically, participants in the holistic focus condition were provided with more than double the amount of adaptive guidance prompts in the first half of the performance scenarios, and this may have cognitively overwhelmed the particpants. Techniques have been developed to reduce cognitive load, such as part-whole training, in which essential subtasks are practiced before performing the whole task. The shifting focus training strategy may be analogous to part-whole training, and its beneficial effects may have been due to a reduction in cognitive load and not because of the actual sequencing of skill acquisition. I performed supplementary analyses to examine whether this is a likely alternative explanation (see Appendix B). MRCM analysis was conducted to examine whether there were differences between the conditions in the number of cumulative clicks (actions performed in the CRONUS task, which can serve as a measure of attention and task engagement) that teams 89 performed across all eight performance scenarios. Results showed that the number of cumulative clicks were not significantly different between conditions across the performance scenarios. Additionally, MRCM was used to examine whether there were differences between the conditions in the number of prompts that were triggered across the eight performance scenarios. Results showed that the shifting focus condition initially had fewer prompts triggered early in training; however, the holistic focus condition reduced the number of prompts triggered by teams at a faster rate than the shifting focus condition, and the number of prompts triggered during the last four performance scenarios appear to be lower in the holistic focus condition (see Table 15 and Table 16). Therefore, it is unlikely that these results were because of the prompts causing cognitive overload in the holistic focus condition. In contrast to the hypotheses regarding knowledge acquisition skill development, the predictions related to knowledge application taskwork were generally not supported. Although individuals in the shifting condition spent more time studying knowledge application taskwork recommendations during the first half of training, they did not practice knowledge application taskwork to a greater extent than individuals in the holistic focus condition. Additionally, contrary to expectations, individuals in both conditions performed equally well in their knowledge application taskwork quality during the first half of training. Thus, focusing individuals’ attention on improving knowledge application taskwork early in training did not appear to be beneficial. Likewise, the predictions related to knowledge application teamwork were not supported. In general, it was found that teams in the shifting focus condition spent more time studying knowledge application teamwork recommendations during the latter half of training, but they did not practice knowledge application teamwork to a greater extent than teams in the holistic focus 90 condition. Moreover, results showed no differences between conditions in the level of communication to facilitate team decision making, nor quality of knowledge application teamwork. In fact, they were generally lower for teams in the shifting focus condition at the end of training, although not significantly so. Overall, teams in the shifting focus condition did not make significantly better team decisions during the latter half of training compared to the holistic focus condition. There are at least four potential reasons why there was a lack of support for some of these hypotheses. First, our previous research indicated that the knowledge application prompts only had noticeable beneficial effects on the quality of team decisions after teams were exposed to them for four or five scenarios. Therefore, if the shifting focus condition only introduced the knowledge application teamwork prompts towards the latter half of training (with only four scenarios left in training), the beneficial effects of the knowledge application teamwork prompts may have been attenuated because some of the effects may have still be developing. Therefore, future research should compare these training strategies across longer performance periods, as this may allow relationships not found in this study to emerge. A second reason is the fact that the nature of the CRONUS task focused everyone on taskwork as a priority, and therefore some of the effects I am seeking are subtle. For example, during the knowledge acquisition portion of a scenario, team members needed to know how to learn at least some obstacles on their own Specialist Map before they could share any information with fellow team members. Therefore, regardless of whether individuals were in the shifting focus or holistic focus condition, the nature of the task required all individuals to initially practice knowledge acquisition taskwork to some extent. Although the results showed that individuals in the shifting focus condition did indeed focus more attention on knowledge 91 acquisition taskwork during early parts of training, it is likely that some of the effects were subtle. This might explain why the results showed no difference between conditions in knowledge acquisition taskwork efficiency during the early parts of training (Hypothesis 2b). Likewise, for the knowledge application portion of a scenario, teams were not able to make a team decision (knowledge application teamwork) until individual decisions were made (knowledge application taskwork). Therefore, individuals in both conditions would have been practicing knowledge application taskwork to a certain degree before practicing knowledge application teamwork. To support this potential explanation, Table 15 and 16 presents the mean number of prompts related to each of the four competency categories that teams triggered during each trial in the shifting focus condition and the holistic focus condition respectively. Table 15. Number of prompts triggered in the shifting focus condition. Knowledge Knowledge Knowledge Knowledge Acquisition Application Acquisition Application Taskwork Taskwork Teamwork Teamwork Trial 1 20.29 10.82 Trial 2 13.36 12.21 Trial 3 9.81 7.84 Trial 4 5.72 6.37 Trial 5 4.68 4.95 13.11 3.32 Trial 6 4.52 3.83 10.46 3.98 Trial 7 4.85 4.23 5.59 1.54 Trial 8 4.39 2.97 4.43 2.43 Table 16. Number of prompts triggered in the holistic focus condition. Knowledge Knowledge Knowledge Knowledge Acquisition Application Acquisition Application Taskwork Taskwork Teamwork Teamwork Trial 1 18.03 10.22 4.78 0.36 Trial 2 14.01 11.31 7.52 0.47 Trial 3 13.44 7.18 10.43 1.11 Trial 4 9.21 5.97 9.76 2.09 Trial 5 6.42 4.18 4.17 2.31 Trial 6 5.67 4.11 4.34 1.95 Trial 7 4.18 3.76 4.29 2.07 Trial 8 4.46 3.06 4.33 1.27 92 Total 31.11 25.57 17.66 12.09 26.08 22.79 16.22 14.23 Total 33.40 33.32 32.17 27.05 17.09 16.08 14.3 13.14 As can be seen, the number of taskwork prompts that were triggered during the first two performance trials in both conditions appear comparable. Additionally, the number of knowledge acquisition teamwork prompts that were triggered in the holistic focus condition peaks on trial 3, suggesting that participants generally focus on knowledge acquisition teamwork only after the first couple of trials. These results are consistent with the explanation that participants in both conditions focus on taskwork skills to some extent during early parts of training, likely because of the way in which the task is structured. A third reason is the fact that the CRONUS task forced team members to begin the process of making their individual and team decisions when two minutes remained in a scenario. This was originally implemented to ensure that teams progressed in a timely fashion, and that they did not focus all of their attention and effort on learning and sharing obstacles. However, in the current study, this task design may have essentially forced team members in both conditions to practice knowledge application taskwork and teamwork for all performance scenarios. This is supported by the results that demonstrate that team members in both conditions practiced knowledge application taskwork at equal rates during the initial stages of training. A final issue is the fact that individuals were not able to practice knowledge application taskwork and teamwork until they had finished learning and sharing obstacles with their team members. Decision cues needed to be gathered before decisions could be made. This is the nature of any task, rather than a characteristic that is specific to CRONUS. However, this was potentially a significant issue in the current study’s design because teams were under significant time pressure to gather as much information about obstacles as possible in six minutes, but only had two minutes to make decisions. Therefore, even with a strong manipulation, the effects may not have been as strong as expected because two minutes likely only allowed individuals in both 93 conditions enough time to practice knowledge application taskwork during the early scenarios. As a result, the differences between conditions may have been attenuated because of the time differential chosen. Limitations and Future Directions Despite some interesting findings in the current study, a number of limitations exist that need to be addressed in future research. First, it is important to acknowledge that the degree to which the guidance in the present study adapted to the progress of team members was constrained to a certain extent in the shifting focus condition. That is, the shifting focus condition was sequenced to focus on taskwork first and then teamwork, and this switch occurred at the midpoint of training for all teams, even if not all team members had adequately developed their taskwork skills. Conversely, it is likely that there were many teams that mastered the taskwork requirements well before the shift in focus occured, thereby reducing the beneficial impact of the shifting focus condition relative to the holistic focus condition. As a result of this constraint, future research should examine another form of sequencing that adapts to the teams’ taskwork and teamwork progress. Second, this study was conducted in a laboratory context with undergraduate students performing a synthetic task, which limits the potential generalizability of the findings. The purpose of this study was to establish whether a shifting focus training strategy could yield a specific pattern of results for this decision-making task structure. The use of undergraduate students limits the degree to which these findings can be extrapolated to organizational teams. Third, an important boundary condition is the type of teamwork skills that this study focused on training. Cannon-Bowers et al. (1995) delineated the difference between task-and 94 team-specific teamwork skills and task- and team-generic teamwork skills. In contrast to task and team-generic skills, which refer to general competencies that are transportable to a variety of contexts, this study focused on developing task- and team-specific skills that are important for teams in the context of CRONUS specifically or tasks similar to CRONUS. Although the CRONUS task follows a theoretical structure that is generalizable, the particular way in which CRONUS functions is fairly unique. Therefore, the specific skills and strategies that team members learned and developed as a result of the guidance may only be useful for tasks similar to the CRONUS task, and may not be applicable to other team or task types. Task- and teamspecific training is most beneficial for teams with fairly stable membership or perform a small range of tasks. Additional studies are needed to be able to generalize the results to contexts that are less task or team-specific. Given some of the proposed reasons for why some of the results were not as expected, a re-design of this study is warranted. I would first increase the number of performance trials so that some of the subtle effects, particularly with respect to knowledge application, may begin to manifest over longer periods of time. As previously mentioned, it is likely that four scenarios are not enough for the full benefits of the knowledge application guidance prompts to manifest for the shifting focus condition. Increasing the number of performance trials to at least ten would strengthen the manipulation. Second, given that some of the effects were likely subtle, a larger sample size is needed. Some of the small differences between conditions may become significant with a larger sample size (e.g., difference between conditions in knowledge acquisition taskwork efficiency). Adding more scenarios in the experiment may also strengthen the power of some of the effects, and some of the hypotheses that proposed interactions with time may become significant. 95 Third, I would increase the time that teams are able to practice knowledge application taskwork and teamwork during a scenario. As previously mentioned, team members had only two minutes within each scenario to practice knowledge application taskwork and teamwork. As a result, individuals in both conditions generally only had enough time to practice knowledge application taskwork for a short while before time ran out, reducing the strength of the manipulation. By increasing the scenario time to nine or ten minutes, and allowing team members three or four minutes to practice knowledge application taskwork and teamwork, it would allow individuals in the holistic focus condition enough time to practice both taskwork and teamwork if they so desire. Conversely, it would allow individuals in the shifting focus condition the ability to spend extra time in mastering the knowledge application taskwork during the early scenarios. Therefore, I would expect the differences between the two conditions to be more significant as a result of this change. Conclusion This study sought to advance understanding of how to effectively train knowledgebuilding and decision-making teams in both taskwork and teamwork skills by comparing the effectiveness of two training strategies. The first is a traditional holistic strategy that focuses trainee attention to both taskwork and teamwork skill development simultaneously. The second is a strategy inspired by Kozlowski et al.’s (1999) normative model of team development that shifts the focus of attention from taskwork first and then teamwork skill development. Although the shifting focus strategy resulted in superior team knowledge outcomes, no differences were found in decision-making performance. Potential reasons for lack of support for hypotheses were suggested, and a re-design of the study was proposed. 96 APPENDICES 97 APPENDIX A Study Questionnaires Demographics Questionnaire Please provide as much of the following information as it is applicable. It is important to understand that these scores will be kept confidential and used only for research purposes. If you do not remember your exam scores, please put a zero in that space. Gender:_____(M/F) Age:_______ Year in College:_____ Major:_______________________ Your GPA:___________ SAT score:___________ ACT score:___________ 98 APPENDIX B Supplementary Analyses Two supplemental analyses were conducted to examine whether participants in the holistic focus condition were cognitively overloaded compared to participants in the shifting focus condition. This was a potential alternative explanation for some of the findings. First, I examined whether there were differences in the number of cumulative clicks that teams performed across the eight performance trials. Clicks constitute actions in the CRONUS task, and can serve as a measure of engagement and attention to the task. It is reasonable to expect that if participants in the holistic focus condition were indeed cognitively overloaded, they would have performed fewer actions overall. The influence of the experimental manipulation on clicks performed by teams across the experiment was analyzed using MRCM to account for the nested data structure (Level-1: scenario; Level-2: team). Results showed a main effect of time (b = 8.61, SE = 1.24 p < .05), indicating that teams increased the number of clicks performed across scenarios. However, there was no main effect of condition (b = 14.95, SE = 7.77, p = ns), nor was there an interaction between condition and time (b = -2.11, SE = 1.75 p = ns). These findings demonstrate that the number of cumulative clicks were not significantly different between conditions, and so the teams in both conditions were equally engaged with the task. Second, I examined whether there were differences in the total number of prompts that were triggered by teams across the eight performance trials. If teams in both conditions triggered relatively the same number of prompts, it is unlikely that the difference in prompts would have caused cognitive overload in the holistic focus condition. Results of the MRCM analysis showed 99 that there was a main effect of condition, indicating that the shifting focus condition initially had fewer prompts triggered than the holistic focus condition (b = -8.56, SE = 3.16 p < .05). There was also a main effect of time (b = -3.54, SE = 0.48 p < .05); which was qualified by a significant interaction between time and condition (b = 1.79, SE = 0.68 p < .05). This interaction demonstrates that the holistic focus condition reduced the number of prompts triggered by teams across scenarios at a faster rate than the shifting focus condition. Although the shifting focus condition initially had fewer prompts triggered, the holistic focus condition was able to quickly reduce this difference. Indeed, an examination of the means reveals that the shifting focus condition appeared to have more prompts triggered than the holistic focus condition during the last four performance scenarios (see Table 15 and Table 16). Overall, these two analyses show that it is unlikely that the study results were due to the fact that the number of prompts caused cognitive overload in the holistic focus condition. 100 REFERENCES 101 REFERENCES Anderson, J. R. (1982). The acquisition of cognitive skill. Psychological Review, 89, 369-406. Argote, L., & McGrath, J. E. (1993). Group processes in organizations: Continuity and change. International Review of Industrial and Organizational Psychology, 8, 333–389. Bandura, A. (1997). Self efficacy: The exercise of control. New York, NY: W. H. Freeman & Company. Barnlund, D. C. (1959). A comparative study of individual, majority, and team judgment. Journal of Abnormal and Social Psychology, 58, 55-60. Bell, B. S., & Kozlowski, S. W. J. (2002). Adaptive Guidance: Enhancing Self-Regulation, Knowledge, and Performance in Technology-Based Training. Personnel Psychology, 55, 267-306. Brannick, M. T., Roach, R. M., & Salas, E. (1993). Understanding team performance: A multimethod study. Human Performance, 6, 287-308. Brown, K. G. (2001). Using computers to deliver training: Which employees learn and why? Personnel Psychology, 54, 271-296. Bryk, A.S., & Raudenbush, S. W. (1992). Hierarchical linear models: Applications and data analysis methods. Newbury Park, CA: Sage Publications. Burke, C. S., Wilson, K. A., & Salas, S. (2003). Teamwork at 35,000 feet: Enhancing safety through team training. Human Factors and Aerospace Safety, 3(4), 287–312. Cannon-Bowers, J.A. & Salas, E. (Eds.) (1998). Making decisions under stress: Implications for individual and team training. Washington, D.C.: APA Books. Cannon-Bowers, J. A., Tannenbaum, S. I., Salas, E., & Volpe, C. E. (1995). Defining competencies and establishing team training requirements. In R. Guzzo and E. Salas (Eds.), Team effectiveness and decision making in organizations, (pp. 333-380). San Francisco: Jossey Bass. Carver, C.S., & Scheier, M.F. (1998). On the self-regulation of behavior. New York: Cambridge University Press. Chen, G., & Kanfer, R. (2006). Toward a systems theory of motivated behavior in work teams. Research in Organizational Behavior, 27, 223–267. Chen, G., & Klimoski, R. J. (2007). Training and development of human resources at work: Is the state of our science strong? Human Resource Management Review, 17(2), 180–190. 102 Chen, G., Thomas, B., & Wallace, J. C. (2005). A multilevel examination of the relationships among training outcomes, mediating regulatory processes, and adaptive performance. Journal of Applied Psychology, 90, 827-841. Colquitt, J. A., LePine, A., & Noe, R. A. (2000). Toward an integrative theory of training motivation: A meta-analytic path analysis of 20 years of research. Journal of Applied Psychology, 85, 678– 707. DeShon, R. P., Kozlowski, S. W. J., Schmidt, A. M., Milner, K. R., & Wiechmann, D. (2004). A multiple goal, multilevel model of feedback effects on the regulation of individual and team performance. Journal of Applied Psychology, 89, 1035-1056. DeShon, R. P., Kozlowski, S. W. J., Schmidt, A. M., Wiechmann, D., & Milner, K. A. (2001, April). Developing team adaptability: Shifting regulatory focus across levels. Paper presented at the 15th annual conference of the Society for Industrial and Organizational Psychology, New Orleans, LA. Dierdorff, E. C., Bell, S. T., & Belohlav, J. A. (2011). The power of “we”: Effects of psychological collectivism on team performance over time. Journal of Applied Psychology, 96, 247-262. Dierdorff, E. C., & Ellington, J. K. 2012. Members matter in team training: Multilevel and longitudinal relationships between goal orientation, self-regulation, and team outcomes. Personnel Psychology, 65(3): 661–703. Early, P.C., Connolly, T., & Ekegren, G. (1989). Goals, strategy development, and task performance: Some limits on the efficacy of goal-setting. Journal of Applied Psychology, 74, 24-33. Eby, L. T., & Dobbins, G. H. (1997). Collectivistic orientation in teams: An individual and group-level analysis. Journal of Organizational Behavior, 18, 275–295. Fiore, S. M., Cuevas, H. M., Scielzo, S., & Salas, E. (2003). Training individuals for distributed teams: Problem solving assessment for distributed mission research. Computers in Human Behavior, 18, 125-140. Fiore, S.M., Rosen, M.A., Smith-Jentsh, K.A., Salas, E., Letsky, M., & Warner, N. (2010). Toward an understanding of macrocognition in teams: Predicting processes in complex collaborative contexts. Human Factors, 52, 203-224. Fleishman, E. A. and Zaccaro, S. J. (1992). Toward a taxonomy of team performance functions. In R. W. Swezey and E. Salas (Eds.), Teams: Their Training and Performance (pp. 3156). Norwood, NJ: Ablex. 103 Grand, J. A., Braun, M. T., Kuljanin, G., Kozlowski, S. W. J., Chao, G. T. (2016) The dynamics of team cognition: A process-oriented theory of knowledge emergence in teams. Journal of Applied Psychology, 101(10), 1353-1385. Guastello, S. J., & Guastello, D. D. (1998). Origins of coordination and team effectiveness: A perspective from game theory and nonlinear dynamics. Journal of Applied Psychology, 83(3), 423-437. Hackman, J. R. (1987). The design of work teams. In J. Lorsch (Ed.), Handbook of organizational behavior (pp. 315-342). Englewood Cliffs, NJ: Prentice-Hall. Helmreich, R. L., & Foushee, H. C. (1993). Why Crew Resource Management? Empirical and theoretical bases of human factors training in aviation. In E. Wiener, B. Kanki, & R. Helmreich (Eds.), Cockpit Resource Management (pp. 3-45). San Diego, CA: Academic Press. Helmreich, R.L., Merritt, A.C., & Wilhelm, J.A. (1999). The evolution of Crew Resource Management in commercial aviation. International Journal of Aviation Psychology, 9, 19-32. Hollenbeck, J., DeRue, D., & Guzzo, R. (2004). Bridging the gap between I/O research and HR practice: Improving team composition, team training, and team task design. Human Resource Management, 43(4), 353-366. Hollenbeck, J. R., Ilgen, D. R., Sego, D. J., Hedlund, J., Major, D. A., & Phillips, J. (1995). Multilevel theory of team decision making: Decision performance in teams incorporating distributed expertise. Journal of Applied Psychology, 80, 292–316. Kabanoff, B., & O'Brien, G. E. (1979). Cooperation structure and the relationship of leader and member ability to group performance. Journal of Applied Psychology, 64, 526-532. Kanfer, R. (1990). Motivation theory and industrial and organizational psychology. In M. D. Dunnette (Ed.), Handbook of industrial and organizational psychology (Vol. 1, 2nd ed., pp. 75-130). Palo Alto, CA: Consulting Psychologists Press. Kanfer, R., Chen, G., & Pritchard, R. D. (Eds.) (2008). Work motivation: Past, present, and future. New York: Taylor and Francis Group. Kanki, B., & Palmer, M. (1993). Communication and crew resource management. In E. Wiener, B. Kanki, & R. Helmreich (Eds.), Cockpit Resource Management. (pp. 99- 136). San Diego, CA: Academic Press. Karoly, P. (1993). Mechanisms of self-regulation: A systems view. Annual Review of Psychology, 44, 23–52. 104 Klein, G. A. (1989). Recognition-primed decisions. In W. B. Rouse (Ed.), Advances in manmachine systems research (Vol. 5, pp. 47–92). Greenwich, CT: JAI Press. Kleinman, D. L., & Serfaty, D. (1989). Team performance assessment in distributed decisionmaking. In R. Gibson, J. P. Kincaid, & B. Goldiez (Eds.), Proceedings of the Interactive Networked Simulation for Training Conference (pp. 22-27). Orlando, FL: Naval Training System Center. Kozlowski, S.W.J. & Bell, B. (2003). Work groups and teams in organizations. In W.C. Borman, D.R. Ilgen, & R.J. Klimoski (Eds.), Industrial and Organizational Psychology (pp. 333375). Chichester: John Wiley & Sons. Kozlowski, S.W.J., Gully, S.M., McHugh, P.P., Salas, E., & CannonBowers, J.A. (1996). A dynamic theory of leadership and team effectiveness: Developmental and task contingent leader roles. In G.R. Ferris (Ed.), Research in personnel and human resource management (Vol. 14, pp. 253–305). Greenwich, CT: JAI Press. Kozlowski, S. W. J., Gully, S. M., Nason, E. R., & Smith, E. M. (1999). Developing adaptive teams: A theory of compilation and performance across levels and time. In D. R. Ilgen & E. D. Pulakos (Eds.), The changing nature of work performance: Implications for staffing, personnel actions, and development (pp. 240–292). San Francisco, CA: JosseyBass. Kozlowski, S.W.J, & Ilgen, D.R. (2006). Enhancing the effectiveness of work groups and teams. Psychological Science in the Public Interest, 7, 77-124. Kozlowski, S.W.J., & Klein, K. J. (2000). A multilevel approach to theory and research in organizations: Contextual, temporal, and emergent processes. In K. J. Klein & S.W.J. Kozlowski (Eds.), Multilevel theory, research and methods in organizations: Foundations, extensions, and new directions (pp. 3-90). San Francisco, CA: Jossey-Bass. Locke, E.A., & Latham, G.P. (1990). A theory of goal setting and task performance. Englewood Cliffs, NJ: Prentice-Hall. Marks, M. A., Mathieu, J. E., & Zaccaro, S. J. 2001. A temporally based framework and taxonomy of team processes. Academy of Management Review, 26(3): 356-376. Mathieu, J. E., Marks, M. A., & Zaccaro, S. J. (2001). Multi-team systems. In N. Anderson, D. Ones, H. K. Sinangil, & C. Viswesvaran (Eds.), International handbook of work and organizational psychology (pp. 289 –313). London: Sage. Mathieu, J., Maynard, M.T., Rapp, T., & Gilson, L. (2008). Team effectiveness 1997-2007: A review of recent advancements and a glimpse into the future. Journal of Management, 34, 410-476. 105 Mathieu, J. E., & Tesluk, P. E. (2010). A multilevel perspective on training and development effectiveness. In S. W. J. Kozlowski & E. Salas (Eds.), Learning, training, and development in organizations, Volume 1 (pp. 405-442). New York: Routledge. McIntyre, R. M., & Dickinson, T. L. (1992). Systemic assessment of teamwork processes in tactical environments (N61339-91-C-0145). Norfolk, VA: Old Dominion University. Mitchell T. R., & Daniels, D. (2003). Motivation. In W. Borman, D. Ilgen, & R. Klimoski (Eds). Handbook of psychology, vol. 12: industrial/organizational psychology (pp. 225–254). New York: Wiley. Mitchell, T. R., & Silver, W. W. (1990). Individual and group goals when workers are interdependent: Effects on task strategies and performance, Journal of Applied Psychology, 75, 185-193. Morgan, B. B.; Glickman, A. S.; Woodard, E. A.; Blaiwes, A. S. & Salas, E. (1986). Measurement of Team Behaviors in a Navy Environment [Technical Report No. NTSC TR-86--014]. Orlando, FL: Naval Training Systems Center. Morgan, B. B., Salas, E., & Glickman, A. S. (1993). An analysis of team evolution and maturation. General Psychology, 120(3). Nelson, T. O., Dunlosky, J., Graf, A., & Narens, L. (1994). Utilization of metacognitive judgments in the allocation of study during multitrial learning. Psychological Science, 5, 207-213. Nieva, V. F., Fleishman, E. A., & Rieck, A. (1978). Team dimensions: Their identity, their measurement and their relationships. Washington, DC: Response Analysis Corporation, Advanced Research Resources Organization. Riecken, H. W. (1958). The effect of talkativeness on ability to influence group solutions of problems. Sociometry, 21, 309–321. Rousseau, V., Aube, C., & Savoie, A. (2006). Teamwork behaviors: A review and an integration of frameworks. Small Group Research, 37, 540 –570. Salas, E., & Cannon-Bowers, J. A. (1997). Methods, tools, and strategies for team training. In M. A. Quinones & A. Ehrenstein (Eds.), Training for a rapidly changing workplace (pp. 249-280). Washington, D.C.: American Psychological Association. Salas, E. & Cannon-Bowers, J. A. (2001). The Science of Training: A Decade of Progress. Annual Review of Psychology, 52, 471–499. Salas, E., Cooke, N. J., & Rosen, M.A. (2008). On teams, teamwork, and team performance: Discoveries and developments. Human Factors, 50(3), 540–547. 106 Salas, E., DiazGranados, D., Klein, C., Burke, C. S., Stagl, K. C., Goodwin, G. F., & Halpin, S. M. (2008). Does team training improve team performance? A meta-analysis. Human Factors: The Journal of the Human Factors and Ergonomics Society, 50(6), 903-933. Salas, E. Dickinson,T. L., Converse, S.A. and Tannenbaum, S. I.(1992).Toward an understanding of team performance and training. In R.W. Swezey & E. Salas (Eds.) Teams: Their training and performance (pp.3-29), Norwood, NJ:Ablex. Salas, E., Nichols, D. R. & Driskell, J. E. (2007). Testing three team training strategies in intact teams: A meta-analysis. Small Group Research, 38, 471-488. Salas, E., Rosen, M.A., Burke, C.S., & Goodwin, G.F. (2009). The wisdom of collectives in organizations: An update of the teamwork competencies. In E. Salas, G.F. Goodwin, & C.S. Burke (Eds.), Team Effectiveness in Complex Organizations. Cross-Disciplinary Perspectives and Approaches (pp. 39- 79). New York: Psychology Press. Salas, E., Sims, D. E., & Burke, C. S. (2005). Is there a big five in teamwork? Small Group Research, 36, 555–599. Schmidt, A. M., & DeShon, R. P. (2007). What to do? The effects of discrepancies, incentives, and time on dynamic goal prioritization. Journal of Applied Psychology, 92, 928 –941. Smith, K. G., Smith, K. A., Olian, J. D., Sims, H. P., Jr., O’Bannon, D. P., & Scully, J. A. (1994). Top management team demography and process: The role of social integration and communication. Administrative Science Quarterly, 39, 412-438. Smith-Jentsch, K. A., Zeisig, R. L., McPherson, J., & Acton, B. (1998). Team dimensional training: A strategy for guided team self-correction. In J. A. Cannon-Bowers & E. Salas (Eds.), Decision making under stress: Implications for individual and team training (pp. 271-297). Washington, DC: American Psychological Association. Stagl, K. C., Salas, E., & Fiore, S. M. (2007). Best practices in cross training teams. In D. A. Nembhard (Ed.), Workforce cross training handbook (pp. 155-179). Boca Raton, FL: CRC Press. Stasser, G., W. Titus. (2003) Hidden profiles: A brief history. Psychological Inquiry, 14, 304313. Stewart, G.L. (2006). A meta-analytic review of relationships between team design features and team performance. Journal of Management, 31, 319-335. Stout, R. J., Salas, E., & Carson, R. (1994). Individual task proficiency and team process behavior: What is important for team functioning. Military Psychology, 6, 177–192. Sundstrom, E., De Meuse, K.P., Futrell, D. (1990). Work teams: Applications and effectiveness. American Psychologist, 45, 120-133. 107 Tannenbaum, S. I., Beard, R. L., & Salas, E. (1992). Team building and its influence on team effectiveness: An examination of conceptual and empirical developments. In K. Kelley (Ed.), Issues, theory, and research in industrial/organizational psychology (pp. 117-153). Amsterdam: Elsevier. Taylor, D. W. & Faust W. L., (1952). Twenty Questions: Efficiency in Problem Solving as a Function of Size of Group, Journal of Experimental Psychology. 44, 360-368. Tennyson, R. D., & Buttery, T. (1980). Advisement and management strategies as design variables in computer-assisted instruction. Educational Communications and Technology Journal, 28, 169– 176. Tennyson, R. D., & Rothen, W. (1979). Management of Computer-Based Instruction: Design of an Adaptive Control Strategy. Journal of Computer-Based Instruction, 5(3), 63– 71. Vancouver, J. B., & Day, D. V. (2005). Industrial and organization research on self-regulation: From constructs to applications. Applied Psychology: An International Review, 54(2), 155-185. Wagner, J. A. (1995). Studies of individualism-collectivism: Effects on cooperation in groups. Academy of Management Journal, 38, 152–172. Weldon, E., Jehn, K. A., & Pradhan, P. (1991). Processes that mediate the relationship between a group goal and improved group performance. Journal of Personality and Social Psychology, 61, 555–569. 108