IDEAL PARTNERSHIPS: SELF-ORGANIZING CAPACITY AS A FACILITATING FACTOR OF INTER-PROFESSIONAL COLLABORATION By Erin R. Watson A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Psychology 2012 ABSTRACT IDEAL PARTNERSHIPS: SELF-ORGANIZING CAPACITY AS A FACILITATING FACTOR OF INTER-PROFESSIONAL COLLABORATION By Erin R. Watson Over the last three decades there has been a growing concern regarding the poor outcomes for the large number of children and youth with serious emotional disturbances (SED) in the United States. The System of Care model has been adopted as a central strategy to address the fragmented, costly, and inaccessible nature of the mental health service delivery systems that has been attributed to these poor outcomes. Yet despite years of research and millions of dollars in federal support, many system of care efforts struggle in transforming their systems to adequately meet the needs of SED youth. One strategy to improve these efforts is the promotion of inter-professional collaboration. While research suggests that inter-professional collaboration is associated with positive outcomes for SED youth, front-line service providers often lack the capacity to engage in these collaborative processes (Fraser & Greenhalgh, 2001; Walker & Shutte, 2005) and there is little research available to guide communities in how to facilitate the process of inter-professional collaboration in their own systems. Drawing upon theories from complexity science, this study aimed to address this gap by introducing the IDeAL Partnerships Framework, an integrated model describing how interprofessional collaboration comes about through the development of four self-organizing capacities (knowledge of the service delivery system, knowledge of provider differences, reflective dialogue skills, and beliefs about collaboration) and the initiation of transformative learning exchanges. The IDeAL Partnerships Framework was used to develop an intervention to increase service providers’ self-organizing capacity, transformative exchanges, and collaborative behavior. The IDeAL Partnerships Intervention was implemented with front-line service providers and was evaluated using a longitudinal experimental design. Two waves of data were collected from a total of 33 providers (17 in the intervention, and 16 in a control group). Results found some support for the overall conceptual model within the IDeAL Partnerships Framework: all four self-organizing capacities were predictive of transformative learning exchanges, and transformative learning exchanges were predictive of collaborative behaviors. However, transformative learning exchanges did not fully mediate the relationship between capacity and collaboration as proposed in the framework, and analyses suggested additional relationships between variables in the model. Results also provided some support for the intervention’s efficacy. Quantitative results suggested a trend indicating that intervention participants experienced increases in two of the four self-organizing capacities (knowledge of the service delivery system and knowledge of provider differences) and their case-related transformative learning exchanges compared to control group participants. While intervention participants’ collaborative behaviors also increased compared to the control group in these analyses, the interaction was driven by a decrease in the control group’s behaviors over time. These findings were triangulated by focus groups and follow-up interviews with intervention participants. In addition, these qualitative findings expanded upon the quantitative findings by suggesting that at least some intervention participants also gained reflective dialogue skills and shifted their beliefs about collaboration as a result of the intervention. Follow-up interviews also suggested that many intervention participants were still using many of the selforganizing capacities they gained from the intervention to improve their collaborative exchanges 4 months after the intervention ended. The quantitative and qualitative findings from the study are discussed, as well as their implications for future research. ACKNOWLEDGEMENTS It has been a long road to the completion of this doctorate degree, and there have been many people who have provided invaluable support along the way. I would like to thank my advisor and chair Dr. Pennie Foster-Fishman. Your support of this project was unfailing, and your consistent encouragement helped me to push through until the very end. I have learned so much from you over the years, not only about promoting community change but also about what it takes to be an engaged scholar. You have played a central role in molding me into a powerful change agent, and I will always appreciate the opportunity to work and learn alongside you in the many communities we have grown to love. I would also like to thank my committee members Dr. Jennifer Neal, Dr. Julie Brockman, and Dr. Kevin Ford for helping me make this project the absolute best it could be. Your theoretical and methodological guidance was incredibly beneficial – as were the pep talks and encouraging words along the way. I would like to thank my friends and colleagues who have played such key roles throughout my graduate school experience: Dr. Lauren Lichty, Charles Collins, Jason Forney, Dr. Giannina Cabral, Dr. Rebecca Campbell, Jen Mortensen, Jeremy Dowsett, Dr. Bill Davidson, and Heather Sprague. In particular, I would like to thank Megan Greeson who, since our initial thesis brainstorming meetings all the way to our dissertation debriefing sessions, has been my intellectual partner and confidant. I cannot imagine this experience without you all. Finally, none of this would have been possible without the love and support from my family and husband. To my parents, Leana and Raymond Droege: from my first day of kindergarten until the night before my dissertation defense, you have always believed in me and encouraged me to pursue my dreams. Your love and support helped me to finish this long race iv and will continue to help me as I journey down future paths. To my husband, Gabriel: you have been my true partner in this venture. Not only have you helped me to think about new questions and ways to refine my wild ideas over the years, but you have also strengthened my confidence and perseverance during the times when I needed it most. I could not have done this without you. v TABLE OF CONTENTS LIST OF TABLES……………………………………………………………………………. viii LIST OF FIGURES………………………………………………………………………….. x OVERVIEW …………………………………………………………………………………. Inter-professional Collaboration within the System of Care Model………………… Self-organizing Capacity as a Means of Promoting Inter-professional Collaboration…………………………………………………………………………. The IDeAL Partnerships Intervention………………………………………………... Summary……………………………………………………………………………… 1 3 INTRODUCTION……………………………………………………………………………. History of the System of Care Approach…………………………………………..... Importance of Inter-Professional Collaboration within the System of Care Model…. Relation between Self Organizing and Inter-Professional Collaboration…………… Eoyang’s Theoretical Model of Self Organizing …………………………………….. IDeAL Partnerships Framework……………………………………………………… IDeAL Partnerships Intervention …………………………………............................. Research Questions…………………………………………………………………... Summary……………………………………………………………………………… 8 8 11 21 28 39 49 51 52 RESEARCH DESIGN AND METHODS……………………………………………………. Overview……………………………………………………………………………… Intervention Design and Implementation…………………………………………….. Setting………………………………………………………………………………… Sample………………………………………………………………………………... Research Design……………………………………………………………………… Procedures……………………………………………………………………………. Measures……………………………………………………………………………… 54 54 54 67 68 68 69 78 RESULTS…………………………………………………………………………………...... General Analysis Strategy …..……………………………………………………….. Data Entry ……………………………………………………………………………. Descriptive and Diagnostic Information……………………………………………... Relation between Self-Organizing Capacities and Initiation of OTEs………………. The Mediating Role of OTEs on Collaborative Behaviors…………………………... The Intervention’s Influence on Participants’ Self-Organizing Capacity……………. The Intervention’s Influence on Participants’ Initiation of OTEs……………………. The Intervention’s Influence on Participants’ Collaborative Behavior………………. Validation Analyses………………………………………………………………….. Contamination Effects……………………………………………………………….. Triangulation Analyses………………………………………………………………. 93 93 94 94 100 102 105 108 111 112 115 116 vi 4 5 6 DISCUSSION……………………………………………………………………………........ Integrity of IDeAL Partnerships Framework…………………………………………. Effectiveness and Viability of the IDeAL Partnerships Intervention………………… Study Limitations……………………………………………………………………... Future Research Directions…………………………………………………………… 138 138 140 152 155 CONCLUSION……………………………………………………………………………….. 158 APPENDICES………………………………………………………………………………... Appendix A: Intervention Meeting Agendas………………………………………… Appendix B: Presentation Template for Meeting 2………………………………….. Appendix C: Weekly Reflection Forms……………………………………………… Appendix D: Wave 1 Survey Consent Form…………………………………………. Appendix E: Survey Measures……………………………………………………….. Appendix F: Description of Study Used During Recruitment Presentation………… Appendix G: Protocol for Validation Case Study Interviews……………………….. 162 162 177 178 179 181 189 190 REFERENCES……………………………………………………………………………….. 197 vii LIST OF TABLES Table 1. Summary of self-organizing capacities and strategies within the IDeAL Partnerships intervention…………………………………………………………………….. 56 Table 2.Staff who signed up to participate in the study……………………………………… 71 Table 3. Participants who withdrew from the study………………………………………… 73 Table 4. Final intervention and control group demographics………………………………… 74 Table 5. Participants participating in validation interviews………………………………….. 77 Table 6. Item descriptives for knowledge of containers measure wave 1…………………… 80 Table 7. Item descriptives for knowledge of containers measure wave 2…………………… 81 Table 8. Item descriptives for knowledge of differences measure wave 1………………...… 82 Table 9. Item descriptives for knowledge of differences measure wave 2……….………..… 83 Table 10. Item descriptives for reflective dialogue skills measure wave 1……..…………… 84 Table 11. Item descriptives for reflective dialogue skills measure wave 2………………… 84 Table 12. Item descriptives for beliefs about collaboration measure wave 1………………. 86 Table 13. Item descriptives for beliefs about collaboration measure wave 2………………... 86 Table 14. Item descriptives for OTE measure wave 1……………………………………….. 87 Table 15. Item descriptives for OTE measure wave 2………………………………..……… 88 Table 16. Factor analysis results for OTE scale…………………………………….……….. 88 Table 17. Item descriptives for collaborative behaviors measure wave 1…………………… 89 Table 18. Item descriptives for collaborative behaviors measure wave 2……………..…….. 90 Table 19. Variable descriptives for intervention group participants……………………….. 95 Table 20. Variable descriptives for control group participants……………………………... 96 Table 21. Variable descriptives for entire study sample…..…………………………………. 97 viii Table 22. Correlation matrix between study variables for full sample of 33……………...… 98 Table 23. Summary of mean differences between intervention and control groups on validation measures…...……………………………………………………………………… 115 ix LIST OF FIGURES Figure 1. The IDeAL Partnerships Framework……………………………………………… 49 Figure 2. Proposed influences of the IDeAL Partnerships Intervention..…………………… 50 Figure 3. Distribution of intervention participants’ meeting attendance…………………….. 75 Figure 4. Significant regression relationships within the IDeAL Partnerships Framework…………………………………………………………………………………… 104 Figure 5. Average response to knowledge of containers measure for each group over time... 106 Figure 6. Average response to knowledge of differences measure for each group over time.. 106 Figure 7. Average response to reflective dialogue skills measure for each group over time... 107 Figure 8. Average response to beliefs about collaboration measure for each group over time…………………………………………………………………………………………………...………… 108 Figure 9. Average response to OTE measure for each group over time…………………….. 109 Figure 10. Average response to case-related OTE subscale measure for each group over time…………………………………………………………………………………………... 110 Figure 11. Average response to diffusion OTE subscale measure for each group over time......................................................................................................................................... 110 Figure 12. Average response to collaborative behaviors measure for each group over time... 111 x Overview Over the last three decades there has been a growing concern among communities, government agencies, and researchers regarding the poor outcomes for children and youth with serious emotional disturbances (SED) in the United States (National Advisory Mental Health Council, 2001; Power, 2009; U.S. Department of Health and Human Services, 2003). SED is a diagnosable disorder that substantially interferes with or limits a youth’s functioning in family, school or the community (National Dissemination Center for Children with Disabilities, 2010). An estimated 4.5-6.3 million youth in the United States experience SED (Friedman, Katz-Leavy, Manderscheid, & Sondheimer, 1999). Not only do these young people experience severe outcomes such as school drop-out and delinquency, but some reports indicate that less than 1/3 of these youth receive needed mental health services (Zahner, Pawelkiewicz, Di-Francesco, & Adnopoz, 1992). This disconnect has been in part attributed to the fragmented, costly, and inaccessible nature of most service delivery systems for SED youth and their families (Hodges, Nesman, & Hernandez, 1999; Mankiam, 2002; Power, 2009). In response, increasing pressure has been put on local systems to transform their traditional mode of operation in order to better meet the needs of this population. The System of Care model has been adopted as a central strategy by local, state, and federal entities for enacting these system reforms (Hodges, Ferreira, Israel, & Mazza, 2006). A system of care is a comprehensive approach to transform the service delivery system for youth and families by improving, among other critical factors, collaborative relationships between organizations and system stakeholders (Hernandez & Hodges, 2003). Systems of care often engage multiple public service agencies such as child welfare, community mental health, schools, and juvenile justice (Rossen, Bartlett, & Herrick, 2008); however, recent calls have been 1 made for increased involvement by informal forms of support in the community (Cook & Kilmer, 2010a). One key strategy within the system of care model is to promote collaboration between service agencies, a goal that has taken on greater relevance with the growing realization that the complex problems faced by youth with SED cannot be solved by agencies working in isolation (Hodges et al., 2006; Odegard, 2007). While collaboration is meant to operate at all levels of the system of care, to date the central focus in the system of care literature has been on organization-level collaboration; collaboration among front-line providers, referred to as inter-professional collaboration, has received far less attention (Hernandez & Hodges, 2003; Stelk & Slaton, 2010). A bias for examining collaboration at the organizational level of analysis is also seen in other literatures outside of the system of care field (Clancy, 2009; Nelson, Batalden, Huber, Mohr, Godfrey, Headrick, Wasson, 2002; Smith & Mogro-Wilson, 2008). Despite this lack of attention, empirical evidence suggests that inter-professional collaboration can serve as an important strategy for improving consumer outcomes across a variety of settings (Lemieux-Charles & McGuire, 2006; Suter & Bruns, 2009). However, wide-spread implementation of practices like inter-professional collaboration rarely succeed through simply the adoption of policies or practices at the organization-level of analysis (Lipsky, 1980; Sandfort, 1999; Smith & MogroWilson, 2008) and often requires attention to process issues like the readiness and capacity of all stakeholder (Durlak, & DuPre, 2008; Klein & Knight, 2005). As a result of this disconnect, there have been numerous calls for more research on the adoption and implementation of interprofessional collaboration (Clancy, 2009; Nelson et al., 2002; Okamato, 2001). The current study addressed this gap by examining processes to promote the implementation of inter-professional collaboration within a system of care. 2 Inter-professional Collaboration within the System of Care Model The literature has struggled to put forth a clear and consistent definition of interprofessional collaboration, most likely due to the variation in the practice across different contexts (Eason, Atkins, & Dyson, 2000). One simple yet effective definition describes it as the structuring of collective action towards consumer needs among diverse service providers (Martin-Rodrigues, Beaulieu, D’Amour, & Ferrada-Videla, 2005). Systems of care efforts place great emphasis on engaging youth and families within the process of inter-professional collaboration (Burns & Goldman, 1999). As a result, many efforts have attempted to adopt the wraparound approach, a formalized model of inter-professional collaboration that is centered around youth and family voice (Winters & Metz, 2009). Despite its growing popularity, many communities struggle to implement collaborative approaches like wraparound, often due to barriers such as limited provider capacity and unsupportive system or organizational contexts (Bruns, Suter, Burchard, Leverentz-Brady, 2004; Walker & Shutte, 2005). In addition, there remains a gap in understanding regarding the process through which these complex collaborative behaviors develop and function (Butt, Markle-Reid, Browne, 2008; D’Amour, Ferrada-Videla, Rodriguez, & Beaulieu, 2005; Polivka, Dresbach, Heimlick, & Elliot, 2001), leaving communities with little guidance as how to facilitate the implementation of inter-professional collaboration. This study aimed to address this gap by examining the process through which inter-professional collaboration develops, specifically by: 1) using complexity theories to propose a set of self-organizing capacities and exchange behaviors underlying the practice; and 2) evaluating the effects of an intervention designed to build these capacities and behaviors among front-line providers and to address system-level barriers to inter-professional collaboration. 3 Self-organizing Capacity as a Means of Promoting Inter-professional Collaboration One useful theory for conceptualizing the process of inter-professional collaboration is self-organizing (Allred, Burns, & Phillips, 2005; Butt et al., 2008). Self-organizing is an emergent process through which agents establish new patterns of behavior that are adaptive to shifting conditions within the system (Anderson, 1999). Inter-professional collaboration is similar to self-organizing because it involves providers adapting their behaviors and services to the shifting needs of a youth with SED and their family. Eoyang (2001) and Eoyang and Olson (2001) proposed a theoretical model self-organizing that can be used to guide the promotion of inter-professional collaboration in a system of care. This model suggests that inter-professional collaboration can be maximized through a self-organizing process where service providers engage with one another in transformative learning exchanges that take advantage of their important differences. These transformative learning exchanges can eventually lead to the emergence of collaborative behavior patterns between providers that are adaptive to addressing the needs of SED youth (i.e., inter-professional collaboration). The concept of transformative exchange is supported by similar models in the organizational learning literature that define learning as a process where individuals align their practices in response to information about changing environmental conditions and to the socially constructed differences (in perspective, practice, skills) between collaborating individuals. By drawing upon Eoyang’s (2001) model and the inter-professional collaboration and organizational learning literature, this study proposed four self-organizing capacities that facilitate transformative exchanges and in turn promote effective inter-professional collaboration between stakeholders. The first self-organizing capacity relates to providers’ knowledge of which stakeholders across the service delivery system are, or could be, relevant to their work 4 with SED youth and their families. This capacity is important to define the boundaries within which the self-organizing behaviors can take place (Eoyang, 2001). The second self-organizing capacity relates to providers’ knowledge of the relevant differences between themselves and other stakeholders who are collaborating around the needs of SED youth. This knowledge allows providers to take advantage of their relevant difference whereby encouraging the emergence of unique and adaptive solutions to address the dynamic needs of SED youth (Olson & Eoyang, 2001). The third self-organizing capacity relates to providers’ skills in reflective dialogue. Reflective dialogue allows for the effective exchange of relevant information and learning between collaborating providers that can lead to adaptive practices (Butt et al., 2008). The fourth self-organizing capacity relates to providers’ beliefs about the practice of collaboration itself. Positive beliefs about collaboration are important because they can increase stakeholders’ readiness to engage in new practices like inter-professional collaboration (Armanakis, Bernerth, Pitts, & Walker, 2009), and facilitate a shared purpose to guide the collaboration between diverse stakeholders (Van den Vegt & Bunderson, 2005). The current study introduced the Integrating Differences to Amplify Learning (IDeAL) Partnerships Framework, an integrated model describing how the emergence of collaborative behavior patterns among stakeholders comes about through the development of self-organizing capacities and the initiation of transformative learning exchanges. Specifically, the framework proposes that four self-organizing capacities predict the initiation of transformative learning exchanges, and that the initiation of transformative learning exchanges plays a mediating role between self-organizing capacity and collaborative behaviors. This study conducted a series of analyses to investigate the relationships within the IDeAL Partnerships Framework. The IDeAL Partnerships Intervention 5 The IDeAL Partnerships Framework provided the foundation for an intervention to develop self-organizing capacities among front-line service providers in a system of care. There were three main elements of the intervention. First, the intervention utilized various strategies to build participants’ self-organizing capacities, such as facilitated discussions, role playing, and skill-building exercises. Second, the intervention involved an action research process to support participants in identifying organizational barriers to inter-professional collaboration and in diffusing self-organizing capacities to their direct colleagues. Third, the intervention included capacity-building activities related to the use of simple rules. Simple rules are based on the idea that an individual’s behavior is guided by a set of simple rules or, in social psychology terms, simple schemata (Anderson, 1999; Bonabeau & Meyer, 2001). These simple rules can be intentionally developed and promoted across groups of individuals to shape and guide system behavior (Olson & Eoyang, 2001). Drawing from the self-organizing literature, the intervention developed the following set of simple rules to guide stakeholders’ collaborative behaviors: 1) connect with critical partners; 2) integrate relevant differences; 3) adapt to change. The study piloted and evaluated the IDeAL Partnerships Intervention using a longitudinal experimental design. The intervention involved 17 front-line service providers addressing the needs of SED youth from four different public service agencies within a system of care in Saginaw, Michigan. These participants engaged in a series of 6 meetings (approximately twice a month) that lasted approximately 2 hours each. The intervention participants were compared with a control group of 16 service providers from the same 4 service agencies. Summary In summary, the purpose of this study is to contribute to our current understanding of the process through which inter-professional collaboration develops in a system of care context. 6 Specifically, the study investigated the relationships between self-organizing capacities, the initiation of transformative learning exchanges, and collaborative behaviors as defined by the IDeAL Partnerships Framework. The study also examined whether an intervention based on the framework successfully increased self-organizing capacity, transformative exchanges, and collaborative behaviors for participating service providers. The study answered the following research questions to assess these areas: 1. To what extent are self-organizing capacities related to the initiation of transformative learning exchanges? 2. To what extent does the initiation of transformative learning exchanges mediate the relationship between self-organizing capacities and collaborative behaviors? 3. Does the IDeAL Partnerships Intervention have a significant influence on participants’ selforganizing capacities? 4. Does the IDeAL Partnerships Intervention have a significant influence on participants’ initiation of transformative learning exchanges? 5. Does the IDeAL Partnerships Intervention have a significant influence on participants’ engagement in collaborative behaviors? 7 Introduction The following literature review begins by providing a historical context for understanding the need for inter-professional collaboration within the system of care model. This is followed by an overview of the research on inter-professional collaboration, and a theoretical framing of inter-professional collaboration as a self-organizing phenomenon. Finally, the review discusses a series of self-organizing capacities hypothesized to promote inter-professional collaboration as well as an intervention to foster this capacity among front-line service providers in a system of care. History of the System of Care Approach A series of reports came out beginning in the late 1980’s that documented the large gap between the mental health needs of youth, particularly those with severe emotional disturbances (SED), and the service delivery system’s ability to meet those needs (Knitzer, 1982; Stroul & Friedman, 1986). For example, some estimates reported that less than one-third of the youth who need mental health services receive them (Zahner et al., 1992). Considering that an estimated 4.5-6.3 million youth experience SED in our country (Friedman et al., 1999), comprising up to 50% of the children in child welfare and 80% of the children in juvenile justice (National Advisory Mental Health Council, 2001), these low rates of service utilization are incredibly concerning. Years after these initial reports came out there is still widespread recognition that the needs of SED youth and their families are not adequately addressed by mental health organizations and professionals (Annie E. Casey Foundation, 1999; Institute of Medicine, 2001; U.S. Department of Health and Human Services, 2003). This failure is due in part to the fact that the service delivery system for youth with SED continues to remain fragmented, costly, and difficult to access for those who need it (Hodges et al., 1999; Mankiam, 2002; New Freedom 8 Commission on Mental Health, 2003; Power, 2009). As a result, in many communities the available services for youth with SED are ineffective, overly restrictive, insufficient in quantity or quality, or simply inappropriate (U.S. Department of Health and Human Services, 1999). This situation is compounded by the fact that many youth with SED and their families require services or support from multiple systems, including mental health, special education, child welfare, and juvenile justice (Stroul & Friedman, 1986; Rossen et al., 2008). The inability of the service delivery system to meet the needs of SED youth has been associated with such negative outcomes as school drop-out and involvement in the juvenile justice or child welfare systems (Institute of Medicine, 2005). The system of care approach was developed in response to this research as a means to improve outcomes for youth with SED and their families. The Child Mental Health Initiative under the Substance Abuse and Mental Health Services Administration (SAMHSA) was created in 1993 to facilitate the development of system of care efforts around the county (Center for Mental Health Services, 2008) and over the years has invested over $1.5 billion into this endeavor. The original definition of a system of care proposed in the 1980’s referred to the model as: a comprehensive spectrum of mental health and other necessary services which are organized into a coordinated network to meet the multiple and changing needs of children and adolescents with severe emotional disturbances and their families” (Stroul & Friedman, 1986, p. 3). This definition has evolved over the years to focus more on systems change, with a more recent definition stating: A system of care is an adaptive network of structures, processes, and relationships grounded in system of care values and principles that provides children and youth with serious emotional disturbances and their families with access and availability of necessary services and supports across administrative and funding jurisdictions (Hodges et al., 2006). 9 Overall, the goal of the system of care approach is to replace the traditionally fragmented, restrictive, and professionally-driven service delivery system with one that is coordinated, flexible, family-driven, community-based, and better able to meet the complex needs of the community (Hodges et al., 2006; Stroul & Friedman, 1986). The key organizations typically involved in a system of care include schools, community mental health centers, child welfare agencies, and the juvenile justice system (Hodges et al., 2010). While this representation has grown over the years, Cook & Kilmer (2010a) recently critiqued system of care efforts for their lack of engagement of informal and natural sources of support in the community, such as faith-based organizations and extended family. Taking an ecological approach, these authors argue that without integrating these natural community supports into a child’s care, any positive service outcomes will likely be short lived after services end (Cook & Kilmer, 2010a). Thus, one goal for a system of care is to intentionally extend the boundary around which stakeholders are to be involved in the care of SED youth to include more community-based supports (Foster-Fishman & Droege, 2010).The current study will take place in a system of care initiative in Saginaw, Michigan, that has to date engaged four public sectors related to education, community mental health, juvenile justice, and social services. Despite the enthusiasm and funding focused on developing system of cares, many communities struggle to implement and embed the system of care practices into their systems. (Bruns et al., 2005; Cook & Kilmer, 2010b; Epstein, Nordness, Kutash, Duchnowski, Shrepf, Benner, & Nelson, 2003). Research suggests that one of the reasons underlying this struggle is that many communities adopt particular changes to their service delivery systems without fully considering or attending to the dynamic processes needed to actually implement those changes (Klein & Knight, 2005; Wandersman et al., 2008; Smith & Mogro-Wilson, 2008). These 10 processes often relate to issues of readiness, capacity, diffusion, and sustainability as they unfold over the course of a change effort (Foster-Fishman & Watson, 2010b). Thus, change agents can potentially play an important role in helping communities move from “principles to practice” (Kilmer, Cook, & Palamaro Munsell, 2010) by working to embed key system changes into the daily practice of service delivery stakeholders. Importance of Inter-Professional Collaboration within the System of Care Model One of the major change targets in a system of care is interagency collaboration, a concept that has taken on greater relevance with the growing realization that the complex and multidisciplinary problems faced by children with SED require the coordination and collaboration of services between a diverse array of agencies and providers (Stroul & Friedman, 1986; Hodges et al., 1999; Hodges et al. 2006; Odegard, 2007). This is because children and youth with SED often require a range of diverse services including special education, social services, health and mental health, vocational training (Rossen et al., 2008). Interagency collaboration has be defined as “a process in which organizations exchange information, alter activities, share resources, and enhance each other’s capacity for mutual benefit and a common purpose by sharing risks, responsibilities, and rewards” (Himmelman, 2004, p. 3). While interagency collaboration implies coherent patterns of collaborative behavior at all levels of analysis (Barr, Koppel, Reeves, Hammick, & Freeth, 2005) the majority of existing studies and corresponding measures examining the promotion of interagency collaboration within service delivery systems focus solely on the organizational-level of analysis (Dedrick & Greenbaum, 2010; Friedman, Reynolds, Quan, Call, Crusto, & Kaufman, 2007; Smith & MogroWilson, 2008). There is an assumption in much of the interagency collaboration literature that the adoption of collaboration at the organizational level implies the adoption of collaboration 11 among individual providers within those organizations (Smith & Mogro-Wilson, 2008). However, research shows that the adoption of a policy or practice (e.g., collaboration) at the organizational level of analysis is often insufficient for ensuring the implementation of that same practice at the frontline staff level (Lipsky, 1980; Sandfort, 1999; Smith & Mogro-Wilson, 2008). In response, there have been numerous calls for more research on the development and implementation of collaboration at the individual level of analysis (referred to as interprofessional collaboration) across a variety of settings (Clancy, 2009; Lemieux-Charles & McGuire, 2006; Nelson et al., 2002; Okamato, 2001). This is reinforced by a growing recognition that inter-professional collaboration plays a central role to improving patient care and outcomes (Institute of Medicine, 2001; Miller, Freeman, & Ross, 2001). The literature does not contain a clear or consistent definition of what behaviors specifically constitute collaboration at the inter-professional level of analysis, most likely because this collaboration is extremely context specific and can vary dramatically in terms of its focus, diversity, depth of involvement and time scale (Easen et al., 2000; Robinson, 2005). For example, inter-professional collaboration can take place in the context of a formalized team of providers who consistently work together over a period of years (Drinka & Clarke, 2000; Lemieux-Charles & McGuire, 2006) or in the context of a loosely structured group that is created to serve the temporary needs of a client (Robinson, 2005). In broad terms, Martin-Rodrigues et al. (2005) describes it as the structuring of collective action towards consumer needs and Darlington et al. (2005, p. 1086) defined it as “shared work in relation to a client”. In their review of the literature D’Amour et al. (2005) identified several common keywords that cut across the numerous definitions of the concept which included: a) sharing (i.e., shared responsibilities, shared decision-making, shared data, shared planning and intervention); b) 12 partnering (i.e., collegial-like relationship, open and honest communication, mutual trust and respect, pursuing set of common goals or outcomes); c) interdependency (i.e., mutual dependence, accept and capitalize on disciplinary differences to problem-solve around complex issues); and d) shared power (i.e., power is shared and based on knowledge and experience versus title). In the context of a system of care addressing youth with SED and their families, this type of collaboration could occur between providers who directly share cases, whose agencies share cases, or who have a shared interest in improving the care provided to consumers in the service delivery system. Inter-professional collaboration could also occur throughout the multiple stages of a case, from helping a family gain access to families, to jointly planning services, to problem-solving around the developing needs of the case, to helping the youth and family transition out of services. The inclusion of youth and family consumers within the inter-professional collaborative process is also addressed in the literature, although there are differing opinions on what form this involvement should take and how this participation should come about (D’Amour et al., 2005). Within the system of care literature, in contrast, youth and family engagement is highlighted as a central goal and guiding principle, particularly within the wraparound model of service planning utilized within many of these efforts (Burns & Goldman, 1999; Van den Berg & Grealish, 1996; Winters & Metz, 2009). While it is recognized that the ultimate goal in a system of care effort is to fully involve families as equal partners within inter-professional collaboration (Stroul & Friedman, 1986), there is evidence suggesting that systems may proceed slowly towards this goal through a series of developmental steps as opposed to immediately through the adoption of a new policy. For example, Hodges, Hernandez, & Nesman (2003) conducted in-depth case studies of separate system of care efforts across the United States and found that before these systems 13 could establish genuine family-professional partnerships they first needed to take actions to establish effective collaborative relationships between professionals. More specifically, the study found that as providers within these efforts learned how to collaborate with each other they began viewing children and their families more “holistically in the context of their families and communities rather than categorically from the perspective of the agency providing services” (Hodges et al., 2003, p 301). In turn, this recognition of the need to see children and families more holistically opened these providers up to the idea of more fully integrating families within this collaborative process (Hodges et al., 2003). Attempts to engage families prior to this realization were generally unsuccessful in these systems (Hodges et al., 2003). This developmental process is supported by research suggesting that the implementation of any change, such as collaborative behaviors, is dynamic and requires different types of readiness and capacity over time (Foster-Fishman & Watson, 2010b). The current study will focus exclusively on the inter-professional phase of this developmental process as a means of building the system’s capacity to eventually implement genuine family-professional partnerships. One of the ways in which inter-professional collaboration takes place within a system of care is through the wraparound approach, a collaborative model endorsed by most system of care efforts (Burns & Goldman, 1998; Van den Berg & Grealish, 1996; Winters & Metz, 2009). Wraparound is a family-centered, team-driven planning process where youth, family members, service providers, and natural supports collaboratively problem-solve around the dynamic needs of the youth and family’s case (Burns & Goldman, 1998; Hyde, Burchard, & Woodworth, 1996). While inter-professional collaboration can occur in many different forms within a system of care (e.g., through multidisciplinary teams, informal partnerships, case management), within the wraparound process these collaborative behavior patterns are formalized and managed by a 14 trained wraparound coordinator (Burchard, Bruns, Burchard, 2002). The engagement of youth and family members in the planning process is central within the wraparound model (Burns & Goldman, 1998), more so than in other forms of inter-professional collaboration described in the literature. However, there remains a focus on promoting effective collaboration between professionals in the wraparound approach as evidenced by the inclusion of an inter-professional collaboration construct in the Wraparound Observation Form (Nordness & Epstein, 2003), a widely used wraparound fidelity assessment tool. While the research base for wraparound has lagged behind other mental health interventions (Suter & Bruns, 2009), there is growing empirical evidence suggesting that when implemented effectively the wraparound approach can lead to better outcomes for SED youth and their families (Bruns, Suter, Force, & Burchard, 2005; Farmer, Dorsey, & Mustillo, 2004; Myaard, Crawford, Jackson, & Alessi, 2000; Suter & Bruns, 2008). Suter and Bruns (2009) recently conducted a meta-analysis of wraparound studies that utilized control group designs, including both experimental and quasi-experimental. While a previous review by Suter and Bruns (2008) identified 36 wraparound outcomes studies conducted to date, only seven of these studies fit the experimental or quasi-experimental criteria utilized within the meta-analysis. Collectively the seven studies examined the effects of wraparound for a total of 802 children and adolescents (mean age of 13.43). The mean effect size of wraparound across the seven studies was 0.33, and the average youth receiving wraparound had better outcomes than 63% of comparison youth receiving conventional services. Within specific domains, there were significant effect sizes found for wraparound’s effect on mental health outcomes (0.31) and school and juvenile-justice related functioning (0.25), and a marginally significant effect size for living situation (.44). While still limited in the number of included studies, the results of this 15 meta-analysis suggest the collaborative processes within wraparound can serve as an effective strategy for improving youth outcomes. Additional empirical research outside of the wraparound context has found evidence that other forms of inter-professional collaboration are also associated with perceived effectiveness of services and objective ratings of consumer outcomes in a variety of primary health care settings, most often focused on adult consumers. For example, Simmons, Coid, Joseph, Marriot, & Tyrer (2001) reviewed the experimental and quasi-experimental research between 1966 and 1998 on multidisciplinary community mental health teams (involving professionals only) treating adults aged 18-65 with severe mental illness. The review found that consumers receiving services from the collaborative teams had fewer subsequent hospital admissions, fewer cases of dropping out of services, and fewer deaths by suicide and suspicious circumstances compared to standard care (Simmons et al., 2001). The Simmons et al. (2001) review also found evidence that interprofessional teams reduced the costs associated with services compared to standard care. Lemieux-Charles and McGuire (2006) conducted a review of studies conducted on a wider range of inter-professional teams within health care settings between 1985 and 2006 and found evidence that this type of collaborative practice was associated with better patient outcomes (e.g., lower rates of depression symptoms, mortality, and readmission) and higher staff satisfaction compared to uncoordinated sequential care. Despite these optimistic findings, many communities struggle to effectively implement collaborative approaches like wraparound, particularly in terms of utilizing creative problemsolving processes and engaging families and natural supports as partners (Bruns et al., 2004; Burchard et al., 2002; Farmer, 2000; McGinty, McCammon, & Koeppen, 2001; Walker, Koroloff, & Schutte, 2003; Walker & Shutte, 2004, 2005). Two key barriers to the successful 16 implementation of these collaborative approaches that have been identified in the literature include: 1) a lack of provider capacity; and 2) an unsupportive system or organizational context. First, there is a growing recognition that the vast majority of front-line service providers are not properly trained to operate within highly collaborative contexts like wraparound and as a result have difficulty delivering services in this manner (Fraser & Greenhalgh, 2001; Hanley and Wright, 1995; McGinty, McCammon, & Koeppen, 2001). As suggested by Hodges et al. (2003), efforts may need to be taken to build providers’ capacity to effectively collaborate with each other (e.g., engage in inter-professional collaboration) before they can begin engaging youth and families in different types of collaborative processes. In terms of these former capacities, MartinRodrigues et al. (2005) and Xyrichis and Lowton (2008) reviewed the empirical research (both qualitative and quantitative) and identified several factors that have been associated with effective inter-professional collaboration including communication skills (e.g., active listening, open communication), a willingness to collaborate, and an understanding of and respect for the expertise and perspectives of diverse professionals. The current study will build on this research to identify and evaluate capacities that can facilitate inter-professional collaboration. Second, the successful implementation of wraparound and other inter-professional collaboration approaches has been challenged by system and organizational contexts that are unsupportive of the practice (Walker & Koroloff, 2007; Clark, Lee, Prange, & McDonal, 1996; Malekoff, 2000; McGinty et al., 2001; Glisson, 2002). These barriers can take the form of obstructive policies, a lack of administrative support, entrenched organizational routines for providing services, rigid bureaucratic cultures that resist innovation, organizational climates leading to depersonalization and emotional exhaustion, professional and educational socialization that promotes isolationism, and normative beliefs or skepticism toward 17 collaboration (D’Amour, Sicotte, & Levy, 1999; Glisson & James, 2002; Martin-Rodrigues et al., 2008; Walker & Koroloff, 2007). For example, a number of researchers have argued that features of the organizational context such as organizational culture and climate not only influence rates of staff turnover, work attitudes, service quality, and service outcomes (Glisson, 2007; Glisson & Hemmelgarn, 1998; Glisson & James, 2002), but also influence whether new service innovations (e.g., new approaches to inter-professional collaboration) are adopted, implemented, and sustained within an organization or system (Hoagwood, Burns, Kiser, Ringeisen, & Schoenwald, 2001; Hohmann & Shear, 2002; Jensen, 2003; Schoenwald & Hoagwood, 2001). Fortunately, there is evidence that organizational contexts within mental health service agencies can become more supportive of innovation through intentional interventions involving stakeholders within the organizational setting (Glisson, Dukes, and Green; 2006). This study will explore whether engagement in an action research process focused on addressing organizational barriers can shift providers’ ability to initiate behaviors that facilitate inter-professional collaboration. As of 1998 there were an estimated 200,000 wraparound teams operating in this country, making it a primary strategy for addressing the needs of SED youth (Faw, 1999). This number continues to grow as wraparound has become identified as a promising best practice (Burchard et al., 2002). However, while some exemplary system of care efforts have been able to provide wraparound for essentially every SED child within the service delivery system (e.g., Wraparound Milwaukee), there is great variation in the degree to which communities have the resources to comprehensively offer wraparound (Burns & Goldman, 1998). Considering that there are approximately 4.5-6.3 million youth with SED in our country (Friedman et al., 1999), in most communities the majority of SED youth continue to be served through more traditional forms of 18 service delivery that are often fragmented across system sectors (Power, 2009). In fact, outside of formal wraparound teams, there is extremely limited understanding of the extent to which inter-professional collaboration occurs within children’s mental health settings (Odegard, 2007). Outside of children’s mental health systems, research conducted in primary health and hospital settings (e.g., general internal medicine, nursing) in the United States has found rates of interprofessional collaboration and communication to be very low (Reeves, Rice, Gotlib, Lee-Miller, Kenaszchuk, & Zwarenstein, 2009a; Rice, Zwarenstein, Conn, Kenaszchuk, Russell, & Reeves, 2010). In addition, Darlington et al. (2005) found that even if professionals from different agencies are in contact with each other, this does not necessarily imply collaboration. A number of education-based interventions have been developed to increase interprofessional collaboration, the majority of which occur within primary health settings. These interventions often bring together professionals from more than one discipline to learn about improving inter-professional collaboration and client outcomes within service delivery systems (Goldman, Zwarenstein, Bhattacharyya, & Reeves, 2009). Most of these interventions are extremely short, such as a single day-long workshop or a series of short sessions over several weeks or months (Holman and Jackson, 2001; Carr, Brockbank, & Barrett, 2003; Kilminster et al., 2004; Goldman et al., 2009). Inter-professional collaboration interventions have provided training on variety of topics including collaborative practices, continuous quality improvement, goal-setting, the development of action plans, and communication skills (Lemieux-Charles & McGuire, 2006; Reeves, Zwarenstein, Goldman, Barr, Freeth, Hammick, & Koppel, 2009b). While the research on these interventions is limited, there is evidence that within primary health settings they can improve team processes and patient outcomes (Barr et al., 2005; LemieuxCharles & McGuire, 2006; Zwarenstein & Reeves, 2006). For example, Zwarenstein, Reeves, 19 and Perrier (2004) reviewed 14 experimental studies examining whether interventions to promote inter-professional collaboration affected the quality of care and patient outcomes. The review found that in 11 of these studies the interventions were associated with improved client outcomes (e.g., lower mortality rates, improved healthy functioning) or more effectively delivered care in a variety of health settings (e.g., geriatric, neonatal, emergency room, sexually transmitted infections screening). However, research has found that interventions are less likely to make lasting impacts on participants’ practice when there is a lack of support from senior management (Reeves, Freeth, Glen, Leiba, Berridge, & Herzberg, 2006) or when the intervention is too brief or subtle to counteract the daily routines of participants (Rice et al., 2010). Despite the increasing popularity of interventions promoting inter-professional collaboration, there remains a gap in understanding regarding the process through which these complex collaborative relationships and behaviors develop and function (Butt et al., 2008; D’Amour et al., 2005; Polivka et al., 2001). This suggests that the literature must go beyond simply identifying factors that are associated with inter-professional collaboration and begin exploring the mechanisms underlying the practice. Unfortunately, research on collaboration at the front-line service provider level has mainly focused on the process through which professionals develop their relationships with consumers instead of the process through which collaboration develops among groups of providers delivering care (Nelson et al., 2002; Robinson, 2005). Existing research on inter-professional collaboration has also primarily focused on how the practice takes place in the context of formal meetings (Cott, 1997); as a result, there is a dearth of research investigating inter-professional collaboration taking place outside of this context (Butt et al., 2008). Thus while the literature contains varying conceptual descriptions of the characteristics and associated factors of inter-professional collaboration, there is a lack of 20 “broadly applicable, pragmatic information” guiding systems of care in how inter-professional collaboration, in all its diverse forms, is implemented in the real world (Robinson, 2005, p 115). The current study aims to address this gap in the literature by exploring practical processes for promoting inter-professional collaboration within a system of care. Relation between Self Organizing and Inter-Professional Collaboration One way to conceptualize inter-professional collaboration is as a process through which service providers collectively learn how to organize and adjust their behaviors to best meet the dynamic needs of SED youth and their families. One useful framework for conceptualizing the process of inter-professional collaboration is self-organizing (Allred et al., 2005; Butt et al., 2008). The following section will first introduce the concept of self-organizing and then describe its relation to the process of inter-professional collaboration. Conceptualizing self-organizing. The concept of self-organizing comes out of the burgeoning, multidisciplinary field of complexity science and complex adaptive systems (CAS) (Edgren, 2008). A “complex” system is an arbitrarily defined group of independently acting agents (individuals, organizations, sub-systems) who are densely entangled and interrelated with one another (Eoyang, 2001). A system can contain numerous subsystems operating within it that can be large (e.g., a network of organizations) or very small (e.g., a team of collaborating professionals). As agents interact with one another in a complex system, the accumulated effect of their behaviors can generate system-wide patterns that are often non-linear and unpredictable (Holbrook, 2003). Contemporary service delivery systems can be seen as examples of complex systems as they are made up of many diverse agents (e.g., agencies, providers, consumers) who are interrelated based on their efforts to serve overlapping populations of consumers (Hodges et al., 2006; Nelson et al., 2002). Similarly, individuals collaborating around the needs of a case can 21 also be considered a complex adaptive system as the group is made up of diverse individuals (therapists, parents, social workers, etc) who are interrelated based on their efforts to serve the dynamic needs of the case (Allred et al., 2005). Interactions between agents in complex adaptive systems can cause reactions that lead to dramatic outcomes. For example, the formation of a new collaborative relationship between several different professionals serving the same family could lead to synergistic thinking and the development of a broad range of ideas that would not have emerged from each professional working in isolation. Self-organizing is a process in which complex systems adapt to shifting environmental conditions that render previous patterns of behavior obsolete (Comfort, 1994; Anderson, 1999). These adaptations can result from shifts in internal conditions (new leadership within a coalition, consolidated departments, increased capacity of staff within a team of collaborating professionals, county-wide hiring freeze), or external conditions (reduction of government funding for services, formation of coalition, new needs of a youth receiving services from an interdisciplinary team of professionals) within the system’s environment (Comfort 1994). Selforganizing promotes local patterns of behavior that allow agents to adapt to the environment; these local patterns eventually evolve to form coherent and adaptive system-wide patterns (Olson & Eoyang, 2001). The foundation of self-organizing is formed by the interactions among interdependent agents who are each trying to adapt to their own perceived opportunities and demands in the local environment (Anderson, 1999; McDaniel & Driebe, 2001). These interactions are guided by each agent’s internalized schema or mental model regarding how he or she understands the best way to succeed in the world (Anderson, 1999; Senge, 2006). Out of these interactions emerge distinct local patterns of behavior among groups of agents across the system. For 22 example, in response to increased financial strain a group of providers in a service delivery system may develop a pattern of collaborating with one another as a means to reduce duplication of services. Over time, system-wide communication allows for local patterns all over the system to “compete” with each other for the most stable and successful pattern given the context (Comfort, 1994). Through this process system-wide patterns emerge (e.g., normative practices) which in turn influence local behavior patterns in an ongoing feedback cycle (Eoyang, 2001). A key characteristic of self-organizing is that the restructuring of the system is not imposed by an outside force or agent; rather, it emerges from an internal process of competition between a variety of potentially adaptive behavior patterns (Comfort, 1994; Edgren, 2008). Self-organizing is considered a neutral process according to complexity theory (Anderson, 1999), yet when applied to the real world certain outcomes of this process are usually more desired than others. For example, an outcome of collaboration would be more desired in the system of care context than an outcome of fragmentation. Change agents often attempt to influence (versus control) the emerging patterns of behavior in a system to better align with particular goals, for example by facilitating communication between key actors across the system or by highlighting different perspectives for how to do the work (Eoyang, 2001). One explicit goal for a system of care is the emergence of collaboration between agents at all levels of the system. System theorists refer to this state as system coherence where local behavior patterns are consistent at all levels (both vertically and horizontally) across a system, and local patterns are reinforcing and no longer competing (Eoyang, 2001). In order to generate coherence, change agents can attempt to identify and influence interfering patterns throughout the system. For example, coherent system-wide collaboration may be disrupted by an internal conflicting pattern of service providers avoiding to work with stakeholders from outside agencies because they 23 distrust the quality of care in other systems. Nelson et al. (2002) suggest that many efforts like systems of care struggle to promote coherent systems change because they focus exclusively on higher levels of analysis and do not give sufficient attention to supporting effective and interconnected groups or “microsystems” of collaborating providers. This study will evaluate an intervention targeted at promoting collaborative behavior among providers in various microsystems with the hope that these behavior patterns would eventually diffuse and lead to coherent system-wide patterns. Conceptualizing inter-professional collaboration as a self-organizing process. While it is recognized that a system of care is indeed an approach to promote systems change, there are few examples in the literature where specific theories and research from the systems field have been used to inform system of care efforts (Foster-Fishman & Droege, 2009). While many authors have compared complex adaptive systems with health care organizations (Anderson, Issel, & McDaniel, 2003; Crabtree, 2003; McDaniel & Driebe, 2001; Plsek &Wilson, 2001; Zimmerman, Lindberg, & Plsek, 1998) and specifically with systems of care (Friedman, 2010; Hodges et al., 2006), to date no system of care efforts have adopted an explicit self-organizing framework to guide system of care implementation. A self-organizing framework could be particularly relevant for supporting the implementation of inter-professional collaboration because inter-professional collaboration can itself be thought of as a self-organizing process (Allred et al., 2005; Butt et al., 2008). For example, inter-professional collaboration involves a group of service providers (interdependent agents) who collectively engage in problem-solving efforts (interact) in order to continually respond (adapt) to the unique and changing conditions of a particular case (local environment). 24 In a system of care context, the complexity of providing services to youth with serious emotional disturbances necessitates a collaborative approach that is responsive and adaptive to unpredictable changes in the youth’s situation and service delivery context (Hodges et al., 1999). For example, when a youth with SED enters the service delivery system, the professionals working with the youth and his or her family form a new subsystem that is defined by the unique needs of that youth and family. The boundary of this subsystem has the potential to include many different types of interacting actors (service providers, family members, natural community supports) at different points in time who could have a role in improving outcomes for that case, thus giving it the potential for complexity. To add to this complexity, each actor will have a unique perspective of the case that will influence his or her behavior and interactions within the system (Williams & Hummelbrunner, 2010). As a youth’s circumstances change over time, the opportunities and demands within their subsystem, perhaps even the subsystem boundaries themselves, also change requiring the actors to adapt (self-organize) their strategies in order to remain relevant and effective. Unfortunately, service providers from different agencies who are included together within the boundary of a youth’s subsystem (by virtue of their current or potential role in the case) are often unaware of each other or isolated in their work (Hodges et al., 1999), preventing the systems from adequately adapting to the needs of youth in a coordinated and effective manner. This study will seek to discover processes for integrating the actors within these youth systems and building their capacity to adapt their strategies and services to better meet the needs of youth and their families. The connection between self-organizing and inter-professional collaboration is clearly demonstrated in the wraparound approach as described above (Burns & Goldman, 1998; Van den Berg & Grealish, 1996; Winters & Metz, 2009). Wraparound creates a subsystem around a youth 25 where the youth and family can work with a group of service providers and natural supports to collaboratively problem-solve around the dynamic needs of their case. Members are invited to the team based on their relevance to the child and are inherently connected to one another based on their shared goal of improving the child’s outcomes. Once convened, members of the wraparound team combine their expertise (i.e., self-organize) to create unique and flexible patterns of care that are adaptive to the specific opportunities and needs associated with the family’s dynamic circumstances. These collaborative interactions continue until the family is no longer in need of them. It is important to note that the emergence of the self-organizing process in these types of collaborative settings ultimately depends on the quality of communication and diverse interactions among agents, not on the control or management of one central manager or agency (Allred et al., 2005; Edgren, 2008). A system of care effort could draw upon different theoretical concepts from complexity science to identify and foster conditions that encourage this type of constructive, self-organizing behavior among professionals inside or outside of the wraparound context. The literature suggests that self-organizing occurs naturally as interdependent agents who share the same goals interact with each other in an attempt to adapt to their environment (Olson & Eoyang, 2001). Unfortunately, these conditions can be hard to come by within many service delivery systems (Hodges et al., 1999). Several reasons can be found for this disconnect. First, as mentioned above, providers are often unaware of who is or should be included within the subsystems of the youth with SED on their caseload (Hodges et al., 1999). Second, even when providers are aware of each other, they often continue to focus their efforts around the goals within their specific agency instead of developing a collective identity across agency partners (Peck, Sheinberg, & Akamatsu, 1995). Fragmented infrastructure and norms of independence and competition – 26 common characteristics of service delivery systems – can add to this isolationism and reduce the communication and interactions between providers (Lippitt & Van Til, 1981). Third, the literature suggests that even if professionals manage to overcome these barriers and engage in collaboration, self-organizing is unlikely to occur unless they can engage in dialogue and “sensemaking” to effectively integrate their diverse perspectives in pursuit of shared goals (Allred et al., 2005). Unfortunately, providers in health and social service delivery systems are rarely given adequate training in how to engage in dialogue with colleagues from outside agencies (Fraser & Greenhalgh, 2001). These descriptions suggest that if inter-professional collaboration is to occur between providers in a system of care, intentional efforts must be made to facilitate this self-organizing behavior. The literature suggests that fostering the conditions that support self-organizing – such as a system awareness, collective identity, and integrative communication across agencies - may be a more effective strategy for promoting inter-professional collaboration than attempting to force or control behavior change through mandates or policies (Allred et al., 2005; Brown & Crawford, 2003; Cilliers, 2001; Edgren, 2008; Uhl-Bien, Marion & McKelvey, 2007). This is because collaboration’s emergent and dynamic characteristics are poorly matched with a reliance on rigid management practices (Nelson et al., 2002; Olson & Eoyang, 2001). Instead of controlling emerging processes like collaboration, change agents should articulate goals for the system (e.g., coordinated and integrated services) and then foster conditions that will support agents in self-organizing ways to achieve that goal (e.g., through inter-professional collaboration). The current study will develop an intervention based on self-organizing principles to build providers’ capacity to engage in collaborative behavior patterns that are adaptive to the 27 goals of a system of care. The next sections will describe the framework that will guide this intervention. Eoyang’s Theoretical Model of Self-Organizing A useful model of self-organizing was proposed by Eoyang (2001) and later expanded upon by Olson and Eoyang (2001). While the model has primarily been applied to organizational settings, it can easily be translated to the self-organizing process of inter-professional collaboration in a system of care. Eoyang’s (2001) model proposes three interacting elements that affect the self-organizing process: containers, differences, and exchanges (CDE). The following will introduce Eoyang’s (2001) CDE model of self-organizing and describe its relevance to inter-professional collaboration. Containers. The first element with Eoyang’s (2001) model relates to the diverse set of boundaries around particular groups of agents within a system. Eoyang refers to these boundaries as containers because they define which agents are likely to interact with one another. As individuals are brought together through different types of containers, their interactions have the potential to give rise to new adaptive patterns of behavior (Olson & Eoyang, 2001). For example, containers pertinent to service providers in a system of care could take the form of a shared vision, collective identity, agency affiliation, location, a shared case, participation in the same phase of a case (e.g., assisting a family gain access to services), a social network, shared language, or trust. An individual agent can be included in multiple containers at the same time (Eoyang, 2001). For example, a parole officer could participate in a container with other staff at the court by virtue of their shared agency affiliation, as well as containers with groups of providers from outside agencies or community stakeholders who are connected to his or her cases. The literature suggests that a shared vision and collective identity may serve as 28 particularly effective containers to promote self-organizing processes (Butt et al., 2008; Easen, 2000; Salmon, 2004). The main function of the container within the self-organizing process is to determine the intensity and frequency of interactions in which an agent will engage; it also serves to hold agents together as they undergo a process of change and self-organizing (Olson & Eoyang, 2001). Agents in small containers (e.g., several providers problem-solving around a case, staff working within a small service unit in an agency) are more likely to engage in concentrated interactions, a condition that would encourage the development of new behavior patterns among agents (Eoyang, 2001). Large containers (e.g., all providers working within a service delivery system), on the other hand, are likely to encourage a wide range of less intense interaction and as a result are less likely to support the development of stable behavior patterns (Eoyang, 2001). To promote self-organizing, service providers can interact with individuals in currently identified containers (e.g., individuals who have previously helped families on their caseload gain access to services) or they can develop new containers in which to engage with stakeholders (e.g., seek out individuals who could eventually assist them in helping families transition out of services). Differences. Agents in a container can differ from one another in many ways including knowledge, background, skills, expertise, perspective, resources, social connections, and energy (Eoyang, 2001). Differences can provide agents with multiple perspectives for how to understand needs and opportunities in the environment, as well as multiple options for how to best adapt to these conditions (Olson & Eoyang, 2001). As a result, differences create a “potentially generative tension” that can promote new types of interactions and can eventually lead agents to adopt new patterns of adaptive behavior (Eoyang, 2001 p.36). For example, differences in ideas for how best to address the needs of a youth or family could influence the adaptive behavior 29 patterns among collaborating providers related to a case. Support for the importance of differences can be seen across the literature. For example, the literature on multidisciplinary teams suggests that “diversity creates an atmosphere for enhancing group performance” (Mannix & Neal, 2005, p 1) and there is evidence from the teamwork literature that a team’s diversity in expertise and knowledge is related to the emergence of innovative and adaptive solutions (i.e., self-organizing; Ancona & Caldwell, 1992; O’Connor, 1998; Tsui, Egan, & Xin, 1995; West, Borrill, & Unsworth, 1998; Wiersema & Bantel, 1992). The organizational learning literature also suggests that cognitive diversity and differences in perspective can be important facilitators of learning across individuals within an organization (Bapuji & Crossan, 2004; Bogenrieder, 2002). The idea of focusing on significant differences is echoed in the inter-professional collaboration literature, for example Edgren (2008) suggested that: If health and social service providers are to meet changing demands and expectations from patients or users, they must be able to move quickly to find mutually acceptable, locally developed forms of integration at their points of intersection, that is, where their separate services should be coming together (p 2). Yet differences can have their limits. For example, while containers with agents who do not differ from one another will be less likely to provide the necessary tension to promote new forms of interaction and exchange, containers that focus on too many differences between agents – regardless of how many agents are included within that container - will also be less likely to promote new forms of behavior because actors will be unable to converge on a unified pattern (Olson & Eoyang, 2001; Milliken & Martins, 1996). To overcome this potential for overload, agents within a container must focus on the differences that are most relevant and significant to the adaptive needs of the group, or in other words the “differences that make a difference” (Olson & Eoyang, 2001, p89). For example, the self-organizing process among a group of providers would probably lead to more adaptive 30 solutions for a youth with SED if they were able to identify and focus on their differences that were most relevant to a case, such as their previous experiences with the youth, unique service ideas, different professional backgrounds, or community connections - as well as focusing on how these different perspectives could inform a diverse array of solutions for the case - rather than focusing on their less relevant differences in gender, age, authority, or historical agency conflicts. This idea is supported by research on diverse multidisciplinary teams that has found differences in skills, knowledge, and ability to be more positively related to creativity and group problem-solving than demographic differences in race, gender, or age (Mannix & Neale, 2005). While less relevant differences in demographics, authority, or agency politics should not become the focus of the self-organizing process, they cannot be ignored either because they shape the ongoing interactions between actors in a container (e.g., long-standing conflict between the directors of two agencies could hinder the interactions between providers representing these agencies; Olson & Eoyang, 2001). Instead of avoiding them, Olson and Eoyang (2001) suggest that actors must be able to acknowledge and work within these differences in order to open up space to concentrate on more relevant differences for selforganizing. This resembles the call within the organizational learning literature for a greater acknowledgement of organizational politics and power within the social learning process (Coopey, 1995). Unfortunately, working through these differences is often difficult to achieve in practice (Bunderson & Sutcliffe, 2002), as will be described below. In order to constructively take advantage of their relevant “points of intersection,” and work through their less relevant and potentially distracting differences, collaborating providers must engage with each other in the final element within the CDE model: transformative exchanges. 31 Exchanges. The exchange element within Eoyang’s (2001) model relates to information, energy (e.g., enthusiasm), or materials that are exchanged between agents as they interact in a container. Exchanges occur through many mediums (e.g., face-to-face conversations, emails, phone calls, memos, or the delivery of goods) and can occur naturally or can be intentionally structured (Olson & Eoyang, 2001). It is important to note that this definition moves beyond the literature’s predominant focus on the exchanges occurring between stakeholders within formal meeting settings. Exchanges are considered “transforming” when they promote learning and new behaviors within an individual agent (Eoyang, 2001). For example, provider A initiates an exchange of information with provider B about ideas for how to better address the needs of a youth on their shared caseload. This exchange would become transformative if provider B responded to the exchange of ideas by shifting his own beliefs or behaviors related to the case. Transformative exchanges form feedback loops where new behavior patterns are amplified or dampened through the ongoing exchange of information or materials between agents (Olson & Eoyang, 2001). A potential outcome of transformative exchanges is the emergence of new collaborative behavior patterns (i.e., inter-professional collaboration) that are adaptive and beneficial for SED youth. For the purposes of this study, a transformative exchange will be defined as an interaction that has the potential to promote learning (including shifted ideas, beliefs, and perspectives) and/or new behaviors within agents. Research suggests that transformative exchanges are important to self-organizing because they improve actors’ ability to collectively learn and adapt their behaviors, beliefs, and plans to the changing demands of the environment (Butt et al., 2008; Edmondson, 1999). There is great emphasis in the literature on how the quality of interaction and information flow between agents influences their learning or adaptive behavior (Anderson, 32 Crabtree, Steele, McDaniel, 2005; Bapuji & Crossan, 2004; Dasgupta & Gupta, 2009; EasterbySmith & Araujo, 1999; Liebeskind, 1996; Nelson et al., 2002; Yeo, 2006). The concept of transformative exchange overlaps with some of the conceptualizations of learning within the organizational learning literature as well. For example, the role of transformative exchanges within self-organizing resembles the technical perspective of learning within the organizational learning literature, for example theories from Argyris and Schön (1978) and Levitt and March (1988). This perspective describes learning as a process where individuals detect and correct errors within their practice to better align with reality (in the system of care context, this would be within professional’s practice with youth and families). On the other hand, the role of transformative exchanges also overlaps with the social learning perspective within the organizational learning literature, for example theories from Lave and Wenger (1991) and Strauss (1993). This perspective describes learning as a socially constructed process that is “situated” in social practice where individuals develop their practice through their interactions with others within the organizational setting (in the system of care context, this social practice would involve interactions with stakeholders in pertinent containers). There are aspects from both the technical and social perspectives that are relevant for understanding how providers collaboratively adapt around their shared cases. The idea of transformative exchanges within self-organizing practice incorporates elements from both the technical and social perspectives of organizational learning by describing a process where providers align their practices in response to: 1) technical information about the changing conditions within the case; and 2) the socially constructed differences (in perspective, practice, skills) between providers within the container. 33 What factors influence whether an interaction or exchange between providers has the potential to promote adaptive learning and behavior change related to improving outcomes for youth and families (i.e., whether the exchange is transformative)? As described above, bringing together partners with differences in perspective has been suggested in multiple literatures as serving a facilitating role within adaptive learning. In addition to differences, two other key elements are salient across the self-organizing, inter-professional collaboration, and organizational learning literatures: 1) collective identity; and 2) reflective dialogue. The first factor fosters opportunities for transformative exchanges, while the second facilitates the quality of the exchange itself. Collective Identity. A collective identity facilitates transformative exchanges by helping stakeholders to work within or around their less relevant (but perhaps not less salient) differences that can hamper learning. Some of the key agency differences that the inter-professional literature has found to be most associated with this misunderstanding and conflict include agency policies, procedures, cultures, roles, specific language and jargon, limitations, educational socialization and professional paradigm, and commonly perceived stereotypes associated with each other’s agency (Alter & Hage, 1993; Bullock & Little, 1999; Caplan & Caplan, 1993; Easen et al., 2000; Hallett & Birchall, 1992; Miller & Ahmad, 2000; Renade, 1998; Rivard, Johnsen, Morrissey, & Starrett, 1999; Waxman et al., 1999). Histories of previous working relationships between agencies can also become a distracting difference to collaborating (Challis, Fuller, Henwood, Klein, Plowden, Webb, Whittingham, & Wistow, 1988; Salmon, 2004). Research suggests that the process of working within or around differences can be quite difficult (Drinka & Clarke, 2000). For example, people tend to form homogenous groups due to their preference to interact with individual who share their social categories (termed homophily; McPherson, 34 Smith-Lovin, & Cook, 2001). According to social categorization theory, individuals utilize homophily to bolster their self-esteem by attributing positive characteristics to their own groups and less favorable characteristics to other social categories (Bunderson & Sutcliffe, 2002; Tajfel & Turner, 1979; Williams & O’Reilly, 1998). This social comparison makes it easier for individuals to avoid interactions (i.e., transformative exchanges) with members of different groups and to assume that these members “just don’t understand” (Van Der Vegt, 2005, p.535). To counter these effects, and create potential opportunities for transformative exchange and learning, research suggests that diverse stakeholders must reduce their intergroup bias and increases their emotional attachment to the partnership by establishing a collective identity – a view or belief that utilizing the diverse qualities (i.e., relevant differences) across stakeholders within a container are essential for meeting shared goals (Butt et al., 2008; Easen, 2000; Salmon, 2004; Hallet & Birchall, 1992; Huo, 2003; Johnson, 1992; Mariano, 1989; Wenger, 2000). In other words, collective identity allows stakeholders to bridge their differences without eradicating the distinct value each member’s difference brings to helping the group reach their goal (Brewer, 1995; Thomas & Ely, 1996), providing stakeholders with a sense of “feeling different yet similar” (Mannix & Neal, 2005, p 47). For example, Van der Vegt and Bunderson (2005) found that collective identity significantly moderated learning in industry related multidisciplinary teams. Specifically, the research found that in teams with low collective identity, expertise diversity across stakeholders was negatively related to group learning; in contrast, in teams with high collective identity, expertise diversity across stakeholders was positively related to group learning. It is likely that these findings could transfer to the system of care context. While collective identity opens the opportunity for many types of exchanges to 35 occur, stakeholders must engage in reflective dialogue to ensure that these exchanges become transformative. Reflective dialogue. Reflective dialogue involves sharing and actively exploring stakeholders’ perspectives (e.g., their ideas, experiences, knowledge, opinions), and integrating these perspectives into coherent and adaptive solutions. Reflective dialogue can increase the likelihood that an exchange will result in learning or behavior change and involves three steps. First, exchanges are more likely to promote learning when stakeholders share a range of unique ideas or perspectives that are relevant to a common goal, for example meeting the needs of youth and families (Bunderson & Sutcliffe, 2002; Drach-Zahavy & Somech, 2001; Van der Vegt & Bunderson, 2005). These ideas or perspectives should tap into the diverse expertise of collaborating partners, instead of focusing solely on information that is commonly held by all stakeholders, to avoid what is termed the “collective information sampling bias” (Wittenbaum, Hubbell, & Zuckerman, 1999). This exchange of perspectives is important for learning because it creates a “cross-fertilization” of ideas (Van der Vegt & Bunderson, 2005). This research suggests that focusing on relevant differences is positively related to engaging in effective transformative learning exchanges. The literature suggests that not only should these exchanges be diverse, but they should also be non-critical (Sessa & London, 2007; Senge, 2006) and transparent in terms of disclosing the assumptions and reasoning underlying one’s perspectives (Friedman, 2003). Transparency is important because it is common for collaborating stakeholders to assume they understand each others’ reasoning and assumptions when in actuality they do not (Basadur, 2004; Bullock & Little, 1999; Kutash & Duchnowski, 1997). Second, exchanges are more likely to promote transformative learning when they involve honest dialogue about the differences between and accuracy of stakeholders’ shared ideas and 36 perspectives (Basadur, 2004; Edmondson, Bohmer, and Pisano 2001). The organizational learning literature suggests that learning is promoted when stakeholders ask open questions and are able to challenge each others’ underlying assumptions, confront their differences straightforwardly, identify gaps and contradictions in their reasoning, evaluate alternatives for action, and avoid trying to “win” an argument (Argyris & Schon, 1996; Edmondson, 1999, 2002; Fraser & Greenhalgh, 2001; Gibson & Vermeulen, 2003; Preskill & Torres, 1999). These types of exchanges often do not come naturally to stakeholders and require capacity building (Argris, 1991; Basadur, 2004), an issue that will be discussed again in later sections. Third, exchanges are more likely to lead to learning and behavior change when they include dialogue that integrates stakeholders’ diverse ideas into a coherent, shared understanding (Argote, Gruenfeld, & Naquin, 2000; Burke, Salas, & Diaz, 2007; Clark, 1994; Gruenfeld, & Hollingshead, 1993; Krauss & Morsella, 2000; Shon, 1983). Specifically, this involves stakeholders using dialogue to “reframe” and integrate their initial perceptions of group members’ perspectives and examine their own perceptions or mental models in light of this integrated whole (Burke, Salas, & Diaz, 2007). Reframing is facilitated by group level reflection and metacognition where stakeholders reconsider how they are approaching a problem and processing information (Burke, Salas, & Diaz, 2007; Edmondson, 1999). Research suggests that integrating stakeholders’ perspectives promotes collective action (Crossan, Lane, & While, 1999) and fosters the creation of effective solutions to complex problems (Allred et al., 2005; Paul & Peterson, 2001; Weick, 1995). Reflective dialogue is thought to form a positive feedback loop with collective identity as it promotes mutual respect, sharing, and mutual trust (Henneman, Lee, & Cohen, 1995). 37 As agents engage in transformative exchanges and mutually adjust or learn from each other in a container over time, they can collectively develop patterns of behavior (i.e., selforganize) that are adaptive to the environment. As mentioned above, this study will focus on how transformative exchanges could lead to learning and in turn promote collaborative behavior patterns among professionals that are adaptive to the evolving needs of the youth and families with whom they work. In particular, the study will focus on two types of transformative exchanges that are related to: 1) addressing the needs of a specific case; or 2) embedding interprofessional collaboration across the service delivery system. In relation to a specific case, transformative exchanges might include sharing ideas about how to problem solve around the needs of a youth, discussing ways to integrate services for the case, or exchanging information about helping a family access community resources. In relation to embedding inter-professional collaboration within the service delivery system, a transformative exchange might include sharing ideas or strategies with colleagues about how to better initiate or engage in collaborative behaviors, discussing how to problem solve barriers to collaborating (i.e., incompatible policies or practices across agencies), and diffusing self-organizing capacities to their colleagues. As mentioned above, the specific behavior patterns that result from transformative exchanges are influenced by the unique combination of differences and containers related to the exchanges. For example, exchanges between agents who do not differ from one another can become redundant and will be less likely to lead to new adaptive behavior patterns, where as exchanges between agents who differ from one another in relevant ways are likely to lead to more adaptive, innovative behavior patterns (Eoyang, 2001). As suggested above, exchanges are also influenced by the type of container in which the exchanges occur. Containers that have too few actors will limit the degree of diversity among actors and stifle the emergence of adaptive 38 behavior patterns. Containers that have too many actors, on the other hand, will make it difficult for actors to focus on their relevant differences and drown out the emergence of adaptive behavior patterns (Eoyang, 2001). Morgan (2003) suggests that the optimal number of actors within an interdisciplinary community mental health team (i.e., container) is 6-8. Resulting Behavior Patterns. According to Eoyang’s (2001) CDE model, collaborative behavior patterns that are adaptive to the needs of youth and families can emerge through a selforganizing process where providers in the same container (e.g., connected in some way to a case, working for the benefit of SED youth) engage with one another in ongoing transformative learning exchanges (e.g., discussing ways to improve outcomes for a youth, sharing ideas for overcoming barriers to collaboration) that are based on relevant differences (e.g., different perspectives on effective strategies for youth, different ideas about how to embed collaborative practices across the system). The resulting collaborative behavior patterns that could emerge from this self-organizing process might include collectively developing and revising service plans for shared cases or using each other’s ideas or suggestions more broadly in their work with youth and families. These resulting behavior patterns could prompt the emergence of additional transformative exchanges. For example organizing a joint planning meeting could lead to future exchanges of ideas and information between providers, creating an ongoing feedback loop. The next section will describe a framework that can be used to build stakeholders’ capacity to embed Eoyang’s (2001) three self-organizing elements of containers, differences, and exchanges into their daily practice. IDeAL Partnerships Framework Front-line service providers could develop the capacity to integrate the CDE model elements into their daily practice, whereby fostering their initiation of learning exchanges that 39 could eventually lead to self-organizing behaviors related to addressing the complex needs of youth and families (i.e., inter-professional collaboration). Specifically, this capacity could promote transformative exchanges that are based on relevant differences within pertinent containers, hereon referred to as optimized transformative exchanges (OTEs). As mentioned above, OTEs can be focused on developing ideas and solutions to better meet the needs of a specific case, or on embedding inter-professional collaboration across the system by addressing contextual barriers and diffusing self-organizing capacities to colleagues. OTEs are important because they serve as an antecedent to the emergence of self-organizing collaborative behavior patterns among providers. Capacity building efforts around the CDE model elements is important because service providers are often woefully unprepared to initiate or engage in OTEs due to the fact that most training approaches do not teach providers how to work and communicate with individuals from other disciplines (Fraser & Greenhalgh, 2001; Institute of Medicine, 2003). While Eoyang’s (2001) CDE model describes the processes related to self-organizing, it does not explicitly define the capacities required for stakeholders to initiate OTEs. This study aimed to address this gap by introducing the IDeAL (Integrating Differences to Amplify Learning) Partnerships Framework. The IDeAL Partnerships Framework incorporates key capacities suggested in the literature on self-organizing, inter-professional collaboration, implementation, and group learning as necessary to initiate OTEs. These capacities include: 1) the ability to locate stakeholders in pertinent containers within a service system; 2) knowledge of the differences between partnering stakeholders that are most relevant to meeting the needs of youth and families, as well as the differences that are potentially distracting toward this goal; 3) the ability to promote reflective dialogue between stakeholders in relevant containers; and 4) 40 beliefs about collaboration as an appropriate practice for serving youth and families and as it relates to a collective identity. The IDeAL Partnerships Framework suggests that the four self-organizing capacities promote the initiation of transformative exchanges, and that the initiation of transformative exchanges mediates the relationship between capacity and collaborative behaviors. This framework could be used to guide efforts to promote new patterns of inter-professional collaboration between service providers in a system of care. In addition to improving providers’ ability to collaborate with other professionals, the IDeAL Partnerships Framework could also be used to help providers eventually transition from engaging solely in professional partnerships to partnering with more fully with youth and families in the developmental process described by Hodges et al.’s (2003) research. The following sections will describe each of the four capacities within the IDeAL Partnerships Framework and how they relate to the initiation of OTEs and collaborative behaviors (see Figure 1). Capacity 1: knowledge of pertinent containers. In order to initiate OTEs, providers must gain the capacity to identify stakeholders who are - or could be - included in pertinent system containers that can facilitate their work with youth with SED and their families. This could include providers who directly share cases with participants, or individuals who intersect in other ways within their work, such as staff who could assist a family in gaining access to an agency’s services or connecting with community supports. Being able to identify stakeholders in pertinent containers involves an understanding of who knows what, who does what, and with whom in the service delivery system or broader community. It also involves acknowledging that who is included in the specific pertinent containers may shift over time as youth and family circumstances or needs change. The literature refers to this knowledge as network cognition and 41 suggests that it provides agents with a roadmap to identify and navigate potentially beneficial relationships (Kilduff, 2006; Krackardt, 1987). Research suggests that individuals within an organization or system often vary in the accuracy of their perceptions of the network (Krackardt, 1987). Thus, it is essential that providers have a realistic understanding of the service delivery system in order to navigate pertinent containers and initiate OTEs related to their work with youth and families. The study investigated whether having knowledge of which stakeholders are or could be included in pertinent containers was related to the initiation of OTEs using the following hypothesis: H1: higher levels of knowledge about pertinent containers will be related to more frequent initiations of OTEs. Capacity 2: knowledge of differences. Initiating OTEs also requires providers to have an understanding of the differences between system stakeholders that are most relevant for developing adaptive solutions to address the complex needs of youth and families (Olson & Eoyang, 2001). Learning about relevant differences could create an overlapping knowledge base across providers that encourage the process of self-organizing (Allred et al., 2005). The literature suggests that inter-professional collaboration is facilitated when providers understand each others’ differences in roles, competencies, and scope of practice (Sargeant, Loney, & Murphy, 2008). In terms of a system of care for youth with SED and their families, specific relevant differences could include: access to information about a case; social connections within the service delivery system; skills or expertise related to serving youth and families; specific services offered; ideas and strategies for addressing the needs of a case; or ideas for embedding collaborative practices across the system. By identifying relevant differences between stakeholders in each of their pertinent containers, providers will be able to use transformative 42 exchanges to take advantage of how their differences can lead to more adaptive behavior (Bunderson & Sutcliffe, 2002; Olson & Eoyang, 2001; Van der Vegt & Bunderson, 2005). In addition to differences that are relevant for developing adaptive solutions, stakeholders must also identify and understand those differences that could potentially distract them from engaging together in collaborative self-organizing (Olson & Eoyang, 2001). In the system of care context, these distracting differences could include a range of specific agency policies, legal mandates, service procedures, technical terms, and resource limitations that affect the way stakeholders carry out their jobs and delivery services to youth and families. By learning about the different conditions that affect and limit the way providers delivery services in their respective agencies, stakeholders could be more capable of using transformative exchanges to work through their distracting differences and focus instead on the relevant differences they bring to adaptively solve problems (Olson & Eoyang, 2001; Milliken & Martins, 1996). This study investigated whether knowledge of the relevant and distracting differences between providers in the service delivery system was related to the initiation of OTEs as defined by the IDeAL Partnerships Framework using the following hypothesis: H2: higher levels of knowledge about the differences between providers in the system will be related to more frequent initiations of OTEs. Capacity 3: skills in reflective dialogue. The empirical and conceptual literature on inter-professional collaboration suggests that communication skills and a knowledge of the language, terms, and jargon used in different agencies are key factors in facilitating exchanges between service providers (Akhavain, Amaral, Murphy, & Uehlinger, 1999; Braithwait et al. 2007; Easen et al., 2000; Felker et al., 2004; Haley et al., 2004; Henneman, Lee and Cohen, 1995; Miller & Ahmad, 2000; Nichols, DeFriese, & Malone, 2002; Quinn & Cumblad, 1994). 43 While these skills are important, the inter-professional collaboration literature has for the most part ignored the skills in reflective dialogue suggested by the group learning, organizational learning, and self-organizing literatures. Reflective dialogue requires participants to: 1) engage in non-critical and transparent exchanges of different ideas; 2) use dialogue to actively explore perspectives; and 3) integrate ideas into a cohesive whole. The following sections will summarize some of the capacities the literature suggests support each of these actions. Capacities to foster non-critical and transparent exchange of different ideas. To communicate new ideas transparently and non-critically, the literature suggests that providers need several types of capacity. First, providers must learn how to encourage each other to share new and different ideas to avoid the collective information sampling bias described above (Wittenbaum et al., 1999). Edmondson (2003) suggests that this can be accomplished by learning how to communicate to others , both verbally and non-verbally, that their input is “explicitly needed and desired” (p 1424). Second, providers must learn how to articulate their ideas and perspectives clearly and simply in order to reduce the “fuzziness” that plagues many exchanges (Basadur, 2004). This requires providers to learn how to effectively explain their terms, definitions, and technical language so that all collaborating stakeholders have a consensus on what this language means in relation to their shared goals (Kutash & Duchnowski, 1997; Miller & Ahmad 2000; Ranade, 1998). Third, providers need to gain the capacity to openly and respectfully communicate the reasoning and assumptions behind their opinions, beliefs, and actions to diverse stakeholders (Friedman et al., 2003; Sargeant et al., 2008). Transparently communicating one’s ideas is important because it increases the likelihood that others will also provide honest disclosure of their reasoning (Friedman et al., 2003). This skill can be difficult for most people because, as Argyris (1991) describes, most people have developed “defensive 44 reasoning” that encourages them to “keep private the premises, inferences, and conclusions that shape their behavior” (p. 103). However, Argyris (1991) argues that people can be taught how to identify the reasoning behind their beliefs and actions, recognize the contradictions within their “espoused and actual theories of action” (p. 106), and communicate this effectively to others through intentional capacity-building efforts. Fourth, exchanging ideas in a non-critical manner requires providers to gain the capacity to genuinely listen, maintain an open mind, and separate the process of idea generation from the process of evaluating those ideas (Basadur, 1994; Sargeant et al., 2008; Senge, 2006). This prevents providers from jumping to solutions too quickly and facilitates the exchange of a broad array of diverse ideas and perspectives (Basadur, 2004). Capacities to actively explore perspectives through dialogue. To engage in explorative dialogue, providers must be able to inquire into others’ perspectives without trying to win an argument (Burke et al. 2007; Senge, 2006). Specifically, providers need to learn how to ask genuine, probing questions in order to clarify assumptions and honestly evaluate the different perspectives offered by all stakeholders, including themselves (Edmondson, 2002; Gibson & Vermeulen, 2003; Sterman, 2006; Zweben, 1993). Senge (2006, p 186) offers several examples of these types of probing questions. For example, how did you arrive at your conclusion? Are there gaps or inconsistencies in any of our perspectives? Are you using information to inform your perspective that is different than what I have considered? What additional information might we need to inform our perspectives? Actively exploring perspective through dialogue also requires providers to have the capacity to speak openly to diverse stakeholders during meetings or other interactions, to bring up conflict directly (instead of letting it fester), and to keep an open mind (Basadur, 2004; 45 Edmonson, 1999). Because these conversations have a high potential of generating conflict, providers also need skills in negotiation in order for all members to adequately express their opinions and retain mutual influence (Drach-Zachavy & Somech, 2001; Van Offenbeek & Koopman, 1996) Capacities to integrate ideas into cohesive whole. Once their ideas are clarified, providers must have the capacity to integrate their different perspectives in the service of shared goals (Burke et al., 2007). Unfortunately, providers in health and social service delivery systems are rarely trained with these types of skills (Fraser & Greenhalgh, 2001). Specifically, providers must learn how to identify ways in which their diverse perspectives are related to each other (Clark, 1994; Shon, 1983), connect and situate these perspectives in “a larger context in which they make sense together” (e.g., their work with youth and families; Arrow & Henry, 2010, p 864), and dialogue about how these perspectives could be integrated in order to derive solutions that better meet the shared goals of the collaborating stakeholders (Madhavan & Grover, 1998; Shon, 1983; Van der Vegt & Bunderson, 2005). According to Swan, Newell, Scarbrough, and Hislop (1999), this takes place through a dynamic and interactive cycle of continuously recreating and re-constituting the knowledge of the group. The application of these capacities is reliant upon the ability to develop a collective identity across stakeholders, a topic to be discussed next. The current study investigated whether various skills in reflective dialogue were related to the initiation of OTEs as defined by the IDeAL Partnerships Framework using the following hypothesis: H3: higher levels of reflective dialogue skills will be related to more frequent initiations of OTEs. 46 Capacity 4: beliefs about collaboration. The literature suggests that an individual’s decision to implement a behavior, such as the initiation of OTEs, is influenced by their beliefs about the behavior itself (Armanekis et al., 2007; Bandura, 1986; Klein & Sorra, 1996; Oetting, Donnermeyer, Plested, Edwards, Kelly, & Beauvais, 1995; Rogers, 2003). The literature has highlighted several types of beliefs that could play a key role in a providers’ decision to initiate OTEs. For example, research suggests that providers will be more likely to initiate OTEs if they believe: there is a need to change current practices (Bartunek, Rousseau, Rudolph, & DePalma, 2006; Kotter, 1995); inter-professional collaboration is appropriate to meet this need (Butt et al., 2008; Lin, 2000; Miller & Ahmad, 2000); and that implementing inter-professional collaboration will lead to anticipated benefits (van Dam, 2005; Bartunek et al., 2006). Beliefs about collaboration are also important as they relate to developing a collective identity with stakeholders who share similar goals. Again, a collective identity helps providers to work within their distracting differences in order to engage in reflective dialogue focused on the differences that are relevant to addressing the needs of youth and families. Developing collective identity beliefs require providers to recognize the “complementarity” and interdependence of diverse stakeholders in addressing the needs of SED youth (Bushnell & Dean, 1993; Evans, 1994; Gage, 1998; Mariano, 1989; Pike, McHugh, Canney, Miller, Reiley, & Seibert, 1993; Satin, 1994; Siegler & Whitney, 1994; Stichler, 1995; Way & Jones, 1994; Way, et al., 2000). For example, the literature suggests that stakeholders must see themselves as sharing mutual goals with other stakeholders (Salmon, 2004) and believe that they have more to gain than lose through this interdependent relationship (Bronstein, 2003) in order to engage in learning exchanges. Davidson & White (2007) suggest this identify is facilitated by a shared “conceptual organizing principle”– such as the goal of client recovery - that defines the commonality of 47 purpose across diverse stakeholders and provides justification and motivation for interprofessional collaboration. This can be thought of as the development of shared mental models between collaborating stakeholders (Senge, 2006). In addition, research suggests that providers must not only learn to respect each others’ roles and contributions, but they must learn how to demonstrate that respect through their interpersonal interactions with collaborating partners (Sargeant, et al., 2008). The current study investigated whether beliefs about collaboration were related to the initiation of OTEs as defined by the IDeAL Partnerships Framework using the following hypothesis: H4: increased beliefs in support of collaboration will be related to more frequent initiations of OTEs. Mediation role of Transformative Exchanges. The IDeAL Partnerships Framework suggests that OTEs mediate the relationship between self-organizing capacities and collaborative behaviors. The literature suggests that engaging in OTE-like behaviors can lead to collaborative behaviors by influencing stakeholders’ understanding of the opportunities and demands within their environment and their plans for how to adapt to these conditions (Butt et al., 2008; Easterby-Smith & Araujo, 1999; Edmondson, 1999; Nelson et al., 2002; Yeo, 2006). Through this process, front-line service providers who engage together in OTEs could develop a shared understanding of what behaviors are necessary to successfully adapt to the shifting demands and opportunities related to their SED cases. These behaviors could include collaborative behaviors, such as jointly developing new service strategies that better address the multiple demands and opportunities within the case, co-designing services to reduce conflicts between the family’s multiple service plans, and working together to provide follow-up to a case in order to immediately identify changes in family circumstances. In other words, engaging in OTEs could 48 illuminate the types of self-organizing, collaborative behaviors that could help stakeholders succeed as service providers, and foster the alignment of providers’ actions as a result of this shared understanding. The four self-organizing capacities could foster the initiation of OTEs and as a result activate the potential for these learning exchanges to lead to collaborative behaviors. The current study investigated whether the initiation of OTEs mediated the relationship between self-organizing capacities and collaborative behaviors as defined by the IDeAL Partnerships Framework using the following hypothesis: H5: the initiation of OTEs will significantly mediate the relationship between selforganizing capacities and collaborative behaviors Figure 1. IDeAL partnerships framework. Self-Organizing Capacities Knowledge of Containers Optimized Transformative Exchanges (OTEs) Knowledge of Differences Addressing Specific Cases Skills in Reflective Dialogue Self-Organizing, Collaborative Behaviors Embedding Collaboration Beliefs towards Collaboration IDeAL Partnerships Intervention The current study implemented and evaluated a three-month intervention based on the IDeAL Partnerships Framework. The purpose of the intervention was to build service providers’ self-organizing capacity to initiate OTEs and collaborative behaviors related to their work with youth with SED and their families. Figure 2 illustrates the proposed role of the intervention (shown in bold lines) on these outcomes. 49 Figure 2. Proposed influences of the IDeAL Partnerships Intervention. Shifts in Self-Organizing Capacities IDeAL Partnerships Intervention Knowledge of Containers Knowledge of Differences Initiating Optimized Transformative Exchanges Addressing Specific Cases Skills in Reflective Dialogue Beliefs towards Collaboration SelfOrganizing, Collaborative Behaviors Embedding Collaboration The current study investigated whether participating in the IDeAL Partnerships Intervention increased participants’: 1) knowledge of pertinent containers; 2) knowledge of differences; 3) skills in reflective dialogue; 4) beliefs toward collaboration; 5) initiation of transformative learning exchanges; and 6) initiation of collaborative behaviors. To investigate these influences, the study tested the following hypotheses: H6: Participants in the intervention will have greater knowledge of pertinent containers after the intervention compared to participants in the control group. H7: Participants in the intervention will have greater knowledge of provider differences after the intervention compared to participants in the control group. H8: Participants in the intervention will have greater reflective dialogue skills after the intervention compared to participants in the control group. H9: Participants in the intervention will have stronger ratings of positive beliefs toward collaboration after the intervention compared to participants in the control group. 50 H10: Participants in the intervention will initiate OTEs more frequently after the intervention compared to participants in the control group. H11: Participants in the intervention will engage in more collaborative behaviors after the intervention compared to participants in the control group. Research Questions and Hypotheses The current study attempted to answer the following research questions: 1. To what extent are self-organizing capacities related to the initiation of transformative learning exchanges? 2. To what extent does the initiation of transformative learning exchanges mediate the relationship between self-organizing capacities and collaborative behaviors? 3. Does the IDeAL Partnerships Intervention have a significant influence on participants’ selforganizing capacities? 4. Does the IDeAL Partnerships Intervention have a significant influence on participants’ initiation of transformative learning exchanges? 5. Does the IDeAL Partnerships Intervention have a significant influence on participants’ collaborative behaviors? The following hypotheses were also explored in this study: H1: Higher levels of knowledge about pertinent containers will be related to more frequent initiations of OTEs. H2: Higher levels of knowledge about the differences between providers in the system will be related to more frequent initiations of OTEs. H3: Higher levels of reflective dialogue skills will be related to more frequent initiations of OTEs. 51 H4: Increased beliefs in support of collaboration will be related to more frequent initiations of OTEs. H5: Initiations of OTEs will mediate the relationship between self-organizing capacities and collaborative behavior. H6: Participants in the intervention will have greater knowledge of pertinent containers after the intervention compared to participants in the control group. H7: Participants in the intervention will have greater knowledge of provider differences after the intervention compared to participants in the control group. H8: Participants in the intervention will have greater reflective dialogue skills after the intervention compared to participants in the control group. H9: Participants in the intervention will have stronger ratings of positive beliefs toward collaboration after the intervention compared to participants in the control group. H10: Participants in the intervention will initiate OTEs more frequently after the intervention compared to participants in the control group. H11: Participants in the intervention will engage in more collaborative behaviors after the intervention compared to participants in the control group. Summary Despite years of research and millions of dollars in federal support, there continues to be an urgent need to facilitate the success of the system of care movement for youth with SED and their families. One strategy to improve the service delivery system’s ability to meet the needs of these youth is the promotion of inter-professional collaboration. Unfortunately, many providers lack the capacity to engage in collaborative processes (Walker & Shutte, 2005), and there is little research available to guide communities in how to facilitate the process of inter-professional 52 collaboration in their own systems (Butt et al., 2008; D’Amour et al., 2005). Drawing upon theories from complexity science, this study aimed to address this gap by testing the processes defined within the IDeAL Partnerships Framework and by evaluating an intervention based on the framework to increase front-line service providers’ capacity to self-organize around the needs of SED youth and their families. The evaluation of both the IDeAL Partnerships Framework and the intervention aimed to contribute to the system of care field by informing future capacitybuilding efforts related to collaboration. 53 Research Design and Methods Overview In order to investigate the proposed research questions and hypotheses, the study used a longitudinal experimental design to examine the relationships within the IDeAL Partnerships Framework and the effects of participating in the IDeAL Partnerships Intervention over time. The first section will describe the design and implementation plans for the IDeAL Partnerships Intervention. Following sections will describe the study setting, sample, research design, procedures, and measures. Intervention Design and Implementation The IDeAL Partnerships Intervention was carried out over a series of six meetings that were held approximately every other week for 3 months. The intervention incorporated the same self-organizing principles underlying inter-professional collaboration described above. For example, the intervention brought together providers from different agencies within a small container setting with the purpose of exchanging ideas and collectively gaining capacity to initiate both types of OTEs with colleagues (whereby facilitating the eventual emergence of inter-professional, self-organizing collaborative behaviors). Several of the intervention’s capacity-building processes were also designed to self-organize. For example, although the author acted as facilitator, participants were able to influence which types of information exchanges were most critical to include in the intervention in order to help them learn how to initiate OTEs within the current conditions of the service delivery system. Research suggests that capacity-building processes that utilize these types of non-linear learning styles can stimulate learning (Elwyn, Greenhalgh, & Macfarlane, 2000) and build providers’ capacity to address “messy, fuzzy, unique, and context embedded problems” that are common within their work 54 (Fraser & Greenhalgh, 2001, p 801). In addition, the intervention provided participants with the opportunity to help diffuse the practice of inter-professional collaboration to other stakeholders in the system. This took place through an action learning process where participants worked with their peers to develop strategies to facilitate inter-professional collaboration across the system. The following sections will first introduce several strategies from the transfer of training literature that were incorporated into the intervention, and then describe how the intervention built each of the four self-organizing capacities described in the introduction (see Figure 2). A summary of the specific intervention strategies employed to build the four self-organizing capacities is described in Table 1. A full description of the meeting agendas is also included in Appendix A. Transfer of learning considerations. Despite considerable investment by organizations in staff training or learning opportunities, the literature suggests that typically only about 10% of training content is transferred by participants back to their work practice (Karl & Ungsrithong, 1992). Many intervention strategies have been shown to increase this transfer (Baldwin, Ford, & Blume, 2009; Taylor, Russ-Eft, & Chan, 2005). Three of these strategies were utilized within the IDeAL Partnerships Intervention including: 1) trainee generated scenarios; 2) negative and positive behavior models; and 3) goal setting and behavioral intentions. First, the intervention provided opportunities for participants to practice their self-organizing capacity by engaging in practice scenarios that they generated, as opposed to practice scenarios generated by the author. A practice scenario refers to a hypothetical situation that approximates the conditions participants encounter in their every day work experience; these situations can be used to practice or model new behaviors through discussions or role plays (Baldwin et al., 2009). Taylor et al. (2005) conducted a meta-analysis on transfer of training strategies and found evidence that 55 Table 1. Summary of self-organizing capacities and strategies within the IDeAL Partnerships intervention. Capacity Knowledge of Pertinent System Containers  Specific Strategies within IDeAL Partnerships Intervention Participant-led presentations describing agency structure and functioning.  Facilitated discussion about connecting with providers in pertinent containers and sharing this learning with colleagues.  Development of cross-agency resource guide. Knowledge of Differences     Reflective Dialogue Skills Non-critical and transparent exchange of ideas Active exploration of perspectives     Integration of ideas into cohesive whole Beliefs about Collaboration      Participant-led presentations describing typical range of relevant differences across providers from each agency. Facilitated discussion about identifying and utilizing relevant differences between providers, and sharing this learning with colleagues. Participant-led presentations describing potentially distracting differences affecting the way providers carry out their work in each agency. Development of cross-agency resource guide. Facilitated discussion and documentation of agencyspecific jargon. Establishment of group norms to promote non-critical dialogue, and discussion of strategies for establishing similar norms with other collaborating partners outside of intervention. Training on use of probing questions relevant to exchanges with collaborating partners. Role playing exercise to practice using questions in participant generated scenarios. Card sorting exercise to practice integrating ideas. Opportunities for participants to practice skills during facilitated discussions, modeled behavior of integration skills. Facilitated discussion on benefits of integrating providers’ strengths and perspectives. Activity to reflect upon and define common purpose across collaborating providers. Facilitated discussion on working through distracting differences with collaborating partners. 56 trainings utilizing trainee-generated scenarios are associated with higher training transfer, theoretically because the scenarios are more relevant to the realistic work environments in which participants work. Second, the intervention exposed participants to both positive and negative models of collaborative behavior, a strategy that has been found to increase participants’ ability to retain and generalize what they learn during training (Baldwin, 1992). For example, when learning about ways to engage in reflective dialogue participants discussed both effective and ineffective ways to actively explore each others’ perspectives. Finally, the intervention helped participants set goals and develop behavioral intention strategies to overcome anticipated challenges as they attempted to meet those goals. The combination of setting goals and behavioral intentions has been found to be more effective than either strategy in isolation (Gist, Bavetta & Stevens, 1990). These three strategies are highlighted in the following section as they apply to the specific training components. Building knowledge about pertinent system containers. The IDeAL Partnerships Intervention aimed to increase participants’ knowledge of which stakeholders are or could be involved in pertinent system containers by developing their “network cognition” or roadmap of the system. Participants had the opportunity to exchange information about the structure of their respective agencies at intervention meeting two (see Appendix A). Specifically, participants prepared brief presentations about their agency and gave them to their colleagues using a prepared template (see Appendix B). These exchanges focused on describing characteristics related to each agency including the following: the roles of providers within various departments; the range of services offered by providers; different staff who are involved with youth and families as they receive services, including during any agency-specific processes (e.g., intake, service planning, behavioral safety planning, discharge, etc); and staff who 57 information about accessing services and resources in the community. These exchanges also included the distribution of contact information for these individuals (e.g., handing out directories). In addition, there was time set aside to share information about how to identify and connect with stakeholders outside the system of care partner agencies (e.g., department of health, faith-based communities, non-profit or private organizations) who might be relevant to their work with SED youth and families. Throughout these exchanges the author facilitated discussions about how providers could determine which stakeholders are or could be involved in pertinent containers to their work, strategies for how to effectively connect with these stakeholders, and ideas on how to share this learning (i.e., initiate OTEs related to embedding self-organizing capacities across the service delivery system) with their colleagues outside the intervention. Summaries of the presentations and group discussion were created and distributed to participants through email after each meeting. In addition, the information on system containers (agency services, staff listings, organizational charts, and service flow charts) was developed into a resource guide. Specifically, the author compiled agency information shared by participants during each meeting and organized it into an integrated document. A draft version of this document was given to each participant in the intervention group by the third meeting, and the document continued to be edited throughout the duration of the intervention. In an attempt to reduce any intervention contamination effects on the control group, participants were explicitly asked not to share the guide with any of their colleagues outside of the intervention group until the end of the study in order to provide their agency leadership time to approve the guide for general distribution; everyone agreed to this request. 58 Building knowledge about stakeholder differences. The IDeAL Partnerships Intervention used a process similar to that just described to increase participants’ knowledge of relevant and potentially distracting differences between providers in each agency. In terms of relevant differences, during meeting two the participants shared information as a means to expose each other to the typical range of relevant differences across providers from each agency, and worked together to develop strategies for learning about relevant differences with new stakeholders they meet in pertinent containers. The information brought up in these discussions was based on the differences participants viewed as most relevant for developing adaptive solutions to better meet the evolving needs of youth and families. For example, participants exchanged information about the following characteristics typically associated with providers in each agency: access to certain types of information (e.g., related to services or a particular case), skills, perspectives, expertise related to serving youth with SED and their families; and aspects of their jobs that contributed to their work with SED youth. To help participants put this knowledge into action, the author facilitated a discussion about how participants can: identify relevant differences between stakeholders in the pertinent containers related to their own work; utilize relevant differences to address the needs of youth and families (this will also be discussed in relation to reflective dialogue skills described below); and spread this learning to their colleagues outside the intervention (i.e., initiate OTEs related to embedding self-organizing capacities across the service delivery system). See Appendix A for a full description of this process. In addition to relevant differences, participants also engaged in a discussion during meeting three to identify some of the distracting differences they had previously encountered in their work (e.g., the legal, financial, or policy limitations affecting how providers can deliver services, conflicting elements within providers’ approach to delivering services). For example, 59 participants exchanged information about the following conditions that affect the way they deliver services in their respective agencies: state and federal legal mandates determining when and how providers delivery services, agency specific policies that determine the way services are delivered, the time and resource limitations providers encounter in their work, and the specific procedures affecting the way services are delivered in each agency. During this discussion the author ensured that participants described these differences in enough detail so as to clarify any misconceptions and provide explanation for why the differences exist. A summary of the presentations and group discussion during meetings two and three was created and distributed to participants through email after each meeting. This information was also incorporated into the resource guide described above. Building reflective dialogue skills. The IDeAL Partnerships Intervention attempted to increase participants’ capacity to engage in reflective dialogue with professionals outside of their agency in several ways. The following sections describe the intervention strategies used to promote the three capacities related to reflective dialogue. Non-critical and transparent exchange of ideas. The intervention built participants’ capacity to exchange ideas in non-critical and transparent ways using two strategies. First, research suggests that gaining knowledge of agencies’ terms and jargon and creating a form of shared language can help providers’ engage with each other in dialogue (Kutash & Duchnowski, 1997; Miller and Ahmad 2000; Ranade, 1998). The intervention helped familiarize participants with these terms by creating opportunities to discuss and document unfamiliar language. When confronted with unfamiliar language during each meeting, participants were able to write any critical terms or jargon on large sheets that were taped to the wall for the group to access. These terms were also included in the meeting minutes. Second, to encourage participants to practice 60 using transparent and non-critical dialogue, the author helped participants establish norms during the first meeting to foster a setting that encouraged non-critical communication. Research suggests that by establishing this type of psychologically safe environment facilitators can increase the likelihood that group members will exchange information and engage in non-critical dialogue (Burke et al., 2007; Edmondson, 1999). In addition to setting norms within the group during meeting one, the author facilitated a discussion during meeting three about how participants can establish similar norms that promote non-critical dialogue with collaborating partners related to their work with SED youth. Actively exploring perspectives through dialogue. The intervention aimed to build participants’ capacity to actively explore outside perspectives using a role playing exercise. First, the author trained participants in how to use a set of probing questions to explore each others’ perspectives and ideas during meeting three. Specifically, a list of explorative, probing questions was developed by adapting questions suggested in the literature for facilitating open dialogue (Senge, 2006; see Appendix A). These questions related to advocating a view, inquiring into others’ views, and overcoming conflict. These sample questions were written on handouts and described to the group; participants were also provided an opportunity to add their own suggestions to the list. After finalizing the list, the author led participants through a role playing exercise to practice the questions in real dialogue with a partner. The activity involved both positive and negative models of probing questions (a transfer of training strategy), and was followed by a brief discussion to summarize the training and brainstorm how to transfer this skill to other situations. The handout of questions was brought to each subsequent meeting to provide participants with a reference, and participants were encouraged to practice the questions during these meetings. 61 Capacities to integrate ideas into cohesive whole. The intervention aimed to build participants’ capacity to integrate perspectives using two strategies. First, participants engaged in an activity during meeting one where they had an opportunity to practice integrating ideas across group members. This activity proceeded in several phases. First, participants individually wrote answers to the following question on note cards: what are some words you would use to describe your past experiences collaborating with providers from outside your agency? After finishing their answers, participants collectively taped their cards up on a wall and grouped them within categories they felt were most relevant. After this process, the group examined the data for themes related to the experiences that led to either positive or negative outcomes for a case. After this discussion, participants repeated this cycle with a new question: what needs to happen in order for providers to collaborate in ways that lead to positive outcome for youth and families? After grouping this set of cards on the wall, the group discussed their opinions for how providers might address the ideas listed on the cards. This exercise not only gave participants an opportunity to practice skills in integrative discourse, but it also provided them with guidance and feedback as they developed these skills. Second, the intervention helped participants develop their capacity to initiate integrative discourse by giving them opportunities to practice these skills during meeting discussions. Every meeting had at least one planned group discussion that engaged participants in integrating ideas across group members. The author facilitated this process, modeling how to identify links between participants’ different ideas and perspectives, and guiding group reflection. Shifting beliefs about collaboration. The intervention aimed to promote participants’ positive beliefs toward collaboration in two ways. First, the intervention encouraged participants to develop a collective identity across agency affiliation. Research suggests that dialogue about 62 mutuality, specifically how the process of integrating different strengths and perspectives is beneficial for achieving shared goals, can encourage individuals to arrive at a collective identity that supports collaboration (Bronstein, 2003; Espevik, Johnson, Eid, & Thayer, 2006; Gage, 1998; Hallet & Birchall, 1992; Herzog & Hertwig, 2009; Kerr & Tindale 2004; Salmon, 2004; Way, et al., 2000). This type of dialogue was facilitated in meeting two as participants explored the potential benefits of integrating their relevant differences in relation to their work with SED youth and their families. Meeting three provided an opportunity for further reflection by engaging participants in collectively defining a shared purpose or goal to guide their work. Davidson and White (2007) suggest that this process of defining common purpose can support the development of shared mental models between collaborating stakeholders and encourage a motivation for inter-professional collaboration. Participants developed the following common purpose to guide their work: empower families to be healthy and self-sufficient. This purpose was referred to during discussions at subsequent meetings, and was included in the meeting minutes. Second, the intervention attempted to shift participants’ beliefs by providing a chance for them to work through some of the potentially distracting differences meeting in meeting three. Using the list of distracting differences identified in the first part of meeting three (see section above on differences), participants shared strategies they have developed to work around these distractions. After this discussion, participants engaged in the “left-hand column exercise” developed by Agryis (1991) in order to become more aware of how their own mental models and assumptions can interfere with the collaborative process. This activity provided an opportunity for participants to reflect upon the assumptions underlying their beliefs and actions within a hypothetical collaborative conversation. Participants filled in a worksheet individually and then 63 discussed their reactions to the exercise with a partner. After the paired sharing time, participants engaged in a group conversation to share some of their unique hypothetical scenarios (a transfer of training strategy) and to talk about what they learned from the activity. Facilitating OTEs and collaborative behaviors. In addition to building participants’ self-organizing capacity, the IDeAL Partnerships intervention also aimed to directly facilitate their initiation of OTEs and collaborative behavior through a process modeled after action research. Action research is a method of promoting change that engages ordinary people in iterative problem-solving cycles to address problems in their local context (Stringer, 2007). Participants in action research cycles collectively analyze the conditions related to a shared problem (e.g., a lack of OTEs or collaborative behaviors related to their work with youth and families), plan and carry out actions to address the problem (e.g., attempt to initiate OTEs or collaborative behaviors with relevant providers), assess the efficacy of these actions (e.g., were these initiated attempts successful?), and if needed explore ideas to adjust future actions to address remaining problem conditions (e.g., how can we overcome barriers to initiating OTEs or collaborative behaviors?; Foster-Fishman & Watson, 2010). These action research cycles continue until the presenting problem is solved (e.g., there are adequate OTEs or collaborative behaviors) or, more commonly, until a new problem develops (e.g., need to diffuse a new practice throughout the system). Action research was embedded into the intervention to achieve two main purposes. First, as described above, the process was included to help participants problem solve around their attempts to initiate transformative exchanges and collaborative behaviors. At times this problem solving process was related to specific inter-personal barriers, such as strategizing ways to deal with resistant collaborative partners. At other times, the action research process highlighted 64 contextual barriers to initiating new behaviors, such as the existence of wide-spread stereotypes about each agency and engrained practices related to communication and referrals. While some of the contextual barriers (e.g., agency policies) were outside the scope of the group, participants were able to develop strategies for addressing many of the barriers among their colleagues (e.g., encouraging new practices among colleagues around leaving specific information in voicemails to other agencies). Second, the action research process was included in the intervention in order to influence participants’ beliefs about collaboration by allowing them to hear first-hand how other participants had integrated collaboration into their mental model of service delivery. Cutcher-Gershenfeld and Ford (2005) suggest that gaining an accurate awareness of the degree to which others have integrated a new practice can facilitate learning and influence the development of new mental models. Meetings four through six provided participants with opportunities to discuss their successful and unsuccessful attempts to initiate OTEs and collaborative behaviors related to improving their cases and to facilitating collaboration across the system (see Appendix A). During these meetings, participants had opportunities to share their successful and unsuccessful attempts to initiate these behaviors during the previous weeks. Throughout this sharing, the author facilitated a discussion to help participants reflect on their experiences and problem solve around barriers to initiating OTEs and collaborative behaviors (including ways to support the enactment of any solutions to these barriers). To facilitate action learning, participants were given weekly reflection forms to fill out between meetings to make sense of their collaborating experiences (see Appendix C). These forms provided space for participants to keep track of their attempts to initiate OTEs or collaborative behaviors and to reflect upon the success of those attempts. In addition to these conversations, the intervention facilitated the initiation of OTEs 65 and collaborative behaviors by providing an opportunity for participants in meeting four to identify goals related to these new behaviors and to create behavioral intention plans for achieving those goals (a transfer of training strategy). To further facilitate OTEs, the IDeAL Partnerships intervention introduced participants to the concept of simple rules. Simple rules are based on the idea that coherent behavior can emerge from the interactions of independently acting individuals when these individuals follow a shared set of simple rules or, in social psychology terms, simple schemata (Anderson, 1999; Bonabeau, 2001; Olson & Eoyang, 2001). This phenomenon can be seen in many types of systems, from a flock of birds [simple rules: 1) match your speed to others in flock; 2) avoid running into others; 3) fly toward the center] to a highway of traffic [simple rules: 1) match speed to others; 2) stay in my lane; 3) leave enough space between cars] (Olson & Eoyang, 2001). In applied contexts, simple rules can be intentionally developed and explicitly used to shape and guide system behavior (Olson & Eoyang, 2001). Based on the factors associated with self-organizing identified in the literature reviewed above, participants were introduced to the following set of simple rules to guide their initiation of OTEs: 1) connect with critical partners; 2) integrate relevant differences; 3) adapt to change. These three rules were presented to participants during meeting 1 (see Appendix A), and the rules were written on a large piece of paper that was taped to the wall at each subsequent meeting. The simple rules were also written on the top of participants’ weekly reflection forms, and participants’ received a small magnet with a picture of the simple rules to hang as a reminder at their desk. In addition to these strategies, the simple rules were embedded into the facilitated discussion questions during meeting 2, 4, 5, and 6. This encouraged participants to keep the rules salient in their minds and to integrate the rules with other material in the intervention. 66 Setting The setting for the study was a service delivery system in Saginaw County, Michigan. Saginaw County has a population of 200,745, of whom 72% are Caucasian, 18.5% are African American, 7.4% are Hispanic, and 1% is Asian (US Census Bureau, 2008). While Saginaw includes rural and suburban sections, the majority of its population (70%) lives in the county’s densely populated urban areas. The community context in which the Saginaw service delivery system operates is defined by high rates of poverty (22.2% of families with children under 18, and 24% of families living in the urban city of Saginaw; US Census Bureau, 2009; Michigan State Police-Criminal Justice Information Center, 2009), unemployment, and violence (due to its th high violent crime rate, Saginaw was recently considered the 6 most dangerous city in the United States; CQ Press, 2008). Since 2007 the county’s service delivery system has suffered from massive funding cuts due to Michigan’s economic crisis. The county currently provides services to less than 1/3 of youth aged 0-18 who are estimated to have an SED. The Saginaw System of Care effort was started in 2007 with funding from a state block grant as an attempt to develop local infrastructure to better meet the needs of youth with SED and their families in the county. The author served in a collaborative partner role with the organizations participating in the system of care (co-facilitating meetings, supporting the development of cross-agency relationships, promoting system-wide learning) from the beginning of the effort. Over the years the effort developed several action learning teams at different levels of the service delivery system to promote systems change. One of these teams was made up of approximately 30 key middle managers, supervisors, and front-line staff (i.e., social workers, probation officers, therapists, early childhood workers, foster care coordinators, and child protective services staff) from the relevant units within each of the key public sector 67 organizations. The main purpose of this team was to learn about service delivery challenges and promote care coordination across partner organizations. Through an ongoing process of learning, action, and reflection, this team made great progress in fostering systems change. The experience also affected the capacities and behaviors of the staff on this team, including increasing their knowledge of the system (Foster-Fishman & Watson, In Press). While staff directly involved in this team gained capacity related to collaboration, the majority of front-line staff in the system did not have the opportunity to participate in these teams. If the Saginaw community hopes to attain a coherent and effective system of care, this front-line level of the system must also be addressed. This study tested whether the IDeAL Partnership Intervention could serve as a promising strategy to promote inter-professional collaboration among additional front-line service providers within this system. Sample This study involved 33 front-line service providers (e.g., social workers, child and adolescent mental health therapists, parole officers, foster care coordinators, child protective services staff) within four key public service agencies involved in the Saginaw System of Care. These agencies include the Saginaw County Community Mental Health Administration (CMH), th Department of Human Services (DHS), 10 Circuit Family Court, and the Saginaw Independent School District (SISD). Research Design The study used a longitudinal experimental design to evaluate the IDeAL Partnerships Intervention and test the relationships within the IDeAL Partnerships Framework. Front-line providers from the system of care were recruited as participants and randomly assigned to an intervention and control group. These groups were compared before and after the intervention for 68 differences in participants’ levels of capacity and frequency of OTEs and collaborative behaviors. The control group did not participate in any part of the intervention except for the data collection. Procedures The current study involved two waves that required separate recruitment and data collection procedures. The recruitment and data collection procedures for each of these waves are described below. Wave one recruitment. The baseline data for the study was drawn from a survey given to service providers in four key public service agencies involved in the Saginaw System of Care. One additional agency was invited to participate in the survey, but ultimately declined due to time constraints affecting their providers’ availability to engage in the study. To recruit participants for the baseline survey, agency leaders from the four key public agencies were asked to provide lists (including name and contact information) of all front-line staff in their agency who worked with youth with SED and their families (e.g., therapists, social workers, probation officers, foster care workers, child protective services workers). From these requests, 111 service provider staff were nominated for the survey. Leadership staff sent an email to all front-line staff nominated for the study notifying them that the author would be visiting their agency in the next week to administer a voluntary survey as part of the Saginaw System of Care. The email also notified staff that as incentive for participating in the survey, all participants would receive a $15 gift card to a local grocery store. Wave one data collection. On the date of the data collection, interested staff were directed to a conference room and given a packet containing a consent form (see Appendix D) and a survey (see Appendix E). Each of these documents was labeled with a random ID number 69 and recorded on a master tracking list. Once all staff received a packet, a consenting process was carried out where the author described what the project entailed, the risks and benefits of participation, and the right to refuse to participate at any time. Consenting participants signed the consent form, and all forms were collected and stored in a separate envelope. Duplicate copies of the consent forms were provided to participants. Consent forms and the master tracking list were kept in locked file cabinets. After the consenting process, staff who consented to participate completed the paper-andpencil survey contained in their packet. Across the 4 agencies, 87 staff filled out the survey in person. To minimize risk to participants and encourage honest responses, all data were kept confidential. Participants did not put their names on the surveys; rather the surveys were labeled with an ID number that corresponds to the ID on their consent form. The author was on hand during the survey to answer any questions and assist participants as needed. The survey took approximately 25 minutes to complete. The author mailed a packet containing two copies of the consent form and survey (each of these documents were labeled with a random ID number) to the 24 staff who missed the inperson survey administration in order to give them the opportunity to participate. A cover letter in the packet directed respondents to read the consent form prior to beginning the survey. In order to preserve the confidentiality of the data, a postage-paid envelope was included for participants to return their survey and a signed copy of their consent form. All participants were notified that they would receive a gift certificate for $15 as an incentive for returning their completed mail survey. To increase response rates, an email was sent to all staff who received a mail survey within one week to remind them of the survey; the author called any staff who had not responded to this email to remind them of the survey and to answer any questions. In total, 70 13 out of 24 staff returned the mail survey across the four agencies (a 54% return rate for mail surveys). Overall, across in person and mail methods, 100 out of 111 front-line providers filled out the wave 1 survey (a 90% response rate). Wave two recruitment. The intervention project was described to staff during the survey administration, specifically the intervention’s purpose, duration (approximately 3 months), conditions (control and intervention), obligations (intervention would attend bi-monthly meetings and fill out a survey, the control group would just fill out a survey), and incentive for participating (free lunch at each meeting, including the meetings at which they would fill out the surveys). A copy of the slide that was used to describe the study is included in Appendix F. As participants handed in their completed surveys, they were asked if they were interested in participating in the project. Participants who received a survey in the mail were sent an email describing the intervention project and inviting their participation; if a response was not received within three business days, these staff were given a follow-up phone call to discuss their interest in the intervention. Through these two recruitment efforts, 48 providers from across the four agencies signed up to participate in the intervention project, representing 48% of the total number of providers (n=100) who filled out the wave 1 survey. Table 2 summarizes the proportion of staff from each agency who signed up to participate in the project. Table 2. Staff who signed up to participate in the study. Agency CMH DHS Courts SISD Total Staff recruited for survey 24 67 7 13 111 Staff who completed survey 24 57 7 12 100 Staff who signed up to participate in project 11 26 5 6 48 71 % of staff who signed up to participate in project out of staff who completed survey 46% 46% 71% 50% The 48 providers who signed up to participate were randomly assigned to an intervention group (n=27) and control group (n=21). The intervention group was assigned five more participants than the control group in order to protect the study’s power in the event that participants decided to withdraw from the intervention. The sample was stratified so approximately equal numbers of participants from each agency were randomly assigned to each group. A t-test was used to compare the wave 1 survey results for each group to ensure that intervention and control participants were approximately equivalent in terms of their initial capacity levels. Levene’s test showed that the variances between the groups were not significantly different, therefore the t-test results were examined under the assumption of equal variances. Results showed that the two randomly assigned groups did not differ significantly in their knowledge of containers (t(44)= -.09, p=n.s.), knowledge of differences (t(44)= -.29, p=n.s.), skills in reflective dialogue (t(44)= -.17, p=n.s.), beliefs about collaboration (t(44)=.27, p=n.s.), transformative exchanges t(44)= -.55, p=n.s.), or collaborative behaviors t(44)= -1.15, p=n.s.). Once the intervention and control groups had been assigned, the 27 intervention participants were again randomly assigned to three groups with nine participants in each group; this process was also stratified so approximately equal numbers of participants from each agency were randomly assigned to each intervention group. Each intervention group was scheduled to meet on separate days; however, all three intervention meetings were scheduled during the same week. Subject participation and retention rates. Over the duration of the study, 10 participants dropped out of the intervention group and 5 participants dropped out of the control group. This created a final sample size of 17 intervention and 16 control group participants. Table 3 describes the staff who left the study from each agency and the reason for discontinuing their participation. 72 The primary reason participants gave for withdrawing from the intervention group was that their workload had become too overwhelming for them to continue attending the bi-monthly meetings (n=6). Other reasons for leaving the intervention group included being appointed to a new position that no longer involved youth with SED (n=2), and never attending a meeting or responding to the author’s contact attempts (n=2). All ten of these participants had withdrawn from the study by the third intervention meeting. The primary reason for withdrawing from the control group was never attending a meeting or responding to the author’s contact attempts (n=3); other reasons included being appointed to a new position that no longer involved youth with SED (n=1) and leaving their job due to an illness (n=1). Table 3. Participants who withdrew from the study. Participants Intervention Group CMH : 1 DHS: 6 Courts: 2 Schools: 1 Control DHS: 4 Courts: 1 Reason for withdrawing from study Never attended a meeting, no response (1) Appointed to new position (2) Workload became too overwhelming (3) Never attended a meeting, no response (1) Workload became too overwhelming (2) Workload became too overwhelming (1) No response (3) Moved out of town (1) Left position shortly after study began due to illness (1) A t-test was also run on this final sample to ensure that the 17 intervention participants and 16 control participants were approximately equivalent in terms of their initial capacity levels. Again, Levene’s test showed that the variances between the groups were not significantly different so the t-test results were examined under the assumption of equal variances. Results showed that the two groups did not differ significantly in their knowledge of containers (t(31)= .39, p=n.s.), knowledge of differences (t(31)= -.61, p=n.s.), skills in reflective dialogue 73 (t(31)=.30, p=n.s.), beliefs about collaboration (t(31)=.40, p=n.s.), transformative exchanges t(31)= .08, p=n.s.), or collaborative behaviors t(31)= -.95, p=n.s.). Table 4 describes the demographics of the final sample of participants in the intervention and control groups. Table 4. Final intervention and control group demographics. Demographic Characteristics Agency CMH DHS Courts Schools Gender Female Male Age Range Mean, SD Race/Ethnicity Black or African American White Hispanic or Latino Multiracial Years worked in Agency Range Mean, SD Intervention (n=17) Control (n=16) 5 9 1 2 5 7 1 3 13 4 12 4 28-57 42.75, 8.86 26-64 44.94, 11.59 4 10 2 0 2 8 2 4 2-20 9.7, 5.8 .5-25 8.7, 8.17 Three strategies were used to increase the dosage of intervention exposure that each participant received. Previous research has suggested that dosage can play a significant role in the efficacy of an intervention (Nation, Crusto, Wandersman, Kumpfer, Seybolt, MorrisseyKane, & Davino, 2003). First, intervention group participants were sent 3 reminder emails between intervention meetings (one 4 days after the prior meeting, one 4-5 days before the next meeting, and one the day before the meeting) in order to ensure participants remembered the meeting date. Any participant who had not given notice of their availability for the next meeting was also contacted by phone. Second, while participants were asked to attend the meeting dates 74 for their assigned group, they were also given the option to attend another group’s meeting for that week if they had a scheduling conflict. Most of the intervention participants’ schedules were wildly unpredictable due to the crisis-oriented nature of their work, making it difficult for them to consistently attend their assigned group’s meeting date. Personal follow-up emails were sent to any participants who unexpectedly did not attend a meeting in order to remind them that they could attend another group’s meeting for that week. While the alternative meeting option was essential to maintain a high intervention dosage for all participants, it may have influenced participants’ experience of the intervention as they were exposed to different group members at each meeting. This consideration will be covered in the discussion section. Third, any participant who was unable to attend either their assigned group meeting or another group’s meeting for a particular week was contacted and given an overview of the meeting content during a one on one meeting. Over the course of the study, intervention participants attended on average of 5 out of 6 meetings (SD=.85). Figure 3 shows a distribution of the number of meetings each participant attended. The average number of participants attending each intervention meeting was 5.64. Number of Participants Figure 3. Distribution of intervention participants’ meeting attendance. 8 7 6 5 4 3 2 1 0 1 2 3 4 5 Number of Meetings Attended Over Duration of Study 75 6 Wave two data collection. Four forms of data collection occurred during wave two of the study: surveys, structured validation interviews, focus groups, and 4-month follow-up interviews. First, survey data were collected from study participants within two weeks of the final meeting date using the same survey measures given at baseline. Participants who were present at the sixth meeting had an opportunity to complete the survey during the meeting. A separate meeting was scheduled for control group participants to take the survey. All participants were given a free lunch at the meeting for completing the survey. Participants who were unable to attend either of these meetings were sent a survey in the mail. The survey took approximately 15 minutes to complete. All participants in the intervention group (n=17) and control group (n=16) who had not previously withdrawn from the study completed the wave 2 survey (100% response rate for both groups). Second, validation interviews were conducted with selected study participants to provide a check for the survey measures. Specifically, 50% of the participants who had signed up for the study were randomly selected to receive an invitation for a structured validation interview (n=23) within one month of the wave 1 survey. These participants were stratified across agency and intervention group, and all 23 agreed to participate. The interview protocol was adapted from the survey questions and asked participants to answer questions about their capacity and behavior in relation to a recently completed case involving a youth receiving services from at least 2 agencies in the system of care (see Appendix G for interview protocol). The interview took approximately 15-20 minutes and participants were offered a $5 gift card as incentive for participating. These same participants were invited to participate in another interview within one month of the end of the intervention. Of those still participating in the study (n=16), 14 agreed to participate in this second round of interviews (88% response rate); the 14 participants 76 interviewed in this second round represent approximately 42% of the final study sample of 33. Table 5 summarizes the demographic information for these participants. Table 5. Participants participating in validation interviews. Demographic Characteristics Agency CMH DHS Courts Schools Gender Female Male Age Range Mean, SD Race/Ethnicity Black or African American White Hispanic or Latino Multiracial Years worked in Agency Range Mean, SD Intervention (n=7) Control (n=7) 2 5 1 0 2 3 0 1 5 2 6 1 28-57 39.50, 10.03 26-62 43.71, 11.00 1 4 2 0 2 2 1 2 2-11 6.14, 4.30 2-16 6.29, 6.04 Third, focus group sessions were carried out in order to triangulate participants’ responses to the survey. The focus group sessions took place at the end of the final intervention meeting; all 17 intervention participants took part in one of three focus group discussions. Participants were asked the follow questions during the focus group session: 1) what were the most significant things you gained from the intervention experience?; 2) what about the intervention experience facilitated these gains?; and 3) have these gains influenced your practice? These three questions were supplemented by probe questions to facilitate dialogue during the discussion. 77 Finally, as an additional form of triangulation, all intervention participants were invited to take part in a follow-up interview approximately 4 months after the intervention had finished. The purpose of these interviews was to assess participants’ transfer of the self-organizing capacities they had gained from the intervention to their work in order to triangulate the survey and focus group results. These semi-structured interviews asked participants two primary questions: 1) what, if anything, are you actually using from the project in your day to day work; and 2) how, if at all, has the project shifted the way you work with other providers to meet the needs of youth and families in Saginaw. The interviews took approximately 15-20 minutes to complete. Overall, 16 out of 17 participants participated in the follow-up interviews; the participant who did not participate declined due to feeling overwhelmed with her workload. Confidentiality. To protect respondents' confidentiality, all identifying information was removed from raw data and participants were coded with ID numbers only. The electronic master list of ID numbers and corresponding names were kept in a locked filing cabinet and/or electronic file stored separately from raw data. All physical information gathered from respondents (e.g. paper surveys) were kept in locked filing cabinets within a locked room where only senior staff members had access. All electronic information (e.g. entered data, interview transcripts) were kept in password protected electronic folders and computers with access given to project staff only. Only trained and IRB certified staff had access to any identifying information. After completion of the data collection phase, master files were permanently stored in a locked filing cabinet and electronic file in spaces separate from the data files. Measures Measure development. Several scales were developed for this study due to the lack of existing measures related to the self-organizing processes within the CDE model. The new 78 measurement scales were developed through a collaborative process with stakeholders within the Saginaw System of Care. Specifically, staff and supervisors from each of the represented system of care agencies were invited to provide ideas and commentary during three Saginaw System of Care meetings. During the first meeting, stakeholders were asked to identify factors related to the successful implementation of inter-professional collaboration. Three distinct factors were identified including: an awareness of the service delivery system, an attitude of problem-solving versus agency blaming, and the skills to work and communicate with other professionals. The second meeting allowed stakeholders to refine these ideas and give more specific examples that could be used to directly inform the creation of survey items. At the third meeting, stakeholders were provided with drafts of the developed measures and asked to give their feedback. This feedback was integrated into the final version of the survey measures used in the survey. Finalized survey measures Knowledge of system containers. Participants’ knowledge of relevant system containers was measured by a 7-item scale developed for this study. Items in this scale were developed for this study based on the theory within Eoyang’s CDE model, as well as the suggestions from system of care stakeholders. The scale included two subscales. The first subscale was composed of four items measuring participants’ ability to locate and initiate contact with staff from outside agencies who were relevant to their work with youth and families (i.e., relevant staff who could generally assist me in coordinating services for the youth and families on my case load; see question 3 a-d in Appendix E). The second subscale was composed of three items measuring participants’ familiarity with the services available and typical procedures affecting youth with SED and their families at each agency (i.e., service eligibility requirements; see question 4 a-c in 79 Appendix E). The scale assessed the comprehensiveness of participants’ knowledge using a 6point Likert scale ranging from 0=“not at all” to 5=“a great deal”. The literature suggests that the first step in determining the strength of a measure is to examine the intercorrelations between the items in each subscale (Churchill, 1979; DeVellis, 2012; Field, 2005). Some researchers suggest that any item that correlates less than .40 with all other items in a scale or subscale should be removed prior to a factor analysis to ensure that the items are theoretically tied to a common domain (Armenakis et al., 2007; Hinkin, 1998; Kim & Mueller, 1978). Table 6 and 7 show the intercorrelation matrices for the two subscales for both wave 1 and wave 2 surveys. The matrices revealed that all items except 3c (“I have the ability to locate and initiate contact with various providers who work with youth and families at different points in the time from when they first enter services to when they finish services”) had significant correlations above .40 for both wave 1 and 2. Thus, item 3c was removed from the measure. Table 6. Item descriptives for knowledge of containers measure wave 1. Item Mean SD 1 Subscale 1: Knowledge of Providers 3a 3.18 1.24 3b 3.42 0.87 .66** 3c 2.97 1.26 .67** 3d 2.88 1.02 .54** Subscale 2: Knowledge of Services 4a 2.18 1.24 4b 1.91 1.16 .82** 4c 1.94 1.25 .84** Note: * p<.05, **p<.01 2 3 4 .53** .55** .63** - .88* - 80 Table 7. Item descriptives for knowledge of containers measure wave 2. Item Mean SD 1 Subscale 1: Knowledge of Providers 3a 3.58 1.03 3b 3.64 1.17 .78** 3c 3.27 1.23 .36* 3d 3.61 1.00 .83** Subscale 2: Knowledge of Services 4a 2.58 1.17 4b 2.33 1.22 .80** 4c 2.60 1.25 .82** Note: * p<.05, **p<.01 2 3 4 .38** .81** .42* - .89** - A principle components factor analysis using a varimax rotation was conducted on the remaining 6 items (conducted separately for both waves) to determine factor structure. All items for each wave loaded cleanly onto two factors corresponding with the subscales defined above (item loadings were all above .70 on their primary factor and below .38 on the secondary factor). Based on the factor analysis results, a mean score was calculated for the two subscales; these subscales were then averaged to create an overall scale of the participants’ knowledge of relevant system containers. The alpha for the two subscales in wave 1 were .79 and .94; in wave 2 the subscale alphas were .92 and .94. The alpha for the overall scale was .84 for the wave 1 survey, and .88 for the wave 2 survey. Knowledge of differences. Participants’ knowledge of relevant and distracting differences was measured by a 9-item scale developed for this study. Items in this scale were developed based on the theory within Eoyang’s CDE model, as well as the suggestions from system of care stakeholders. Items assessed the participants’ knowledge of the relevant differences related to the providers they work with from outside agencies (i.e., “I am familiar with the full range of their skills and expertise related to serving youth and families”; see question 9 a-e and i in Appendix E), as well as the distracting differences affecting providers in 81 each agency (i.e., “I am familiar with the limitations (legal, financial, policy) affecting their ability to deliver services”; see questions 9 f-h in Appendix E). Items were measured using a 6point Likert scale ranging from 0=“not at all” to 5=“a great deal”. The intercorrelations between the items were examined to assess the internal consistency of the measure. Tables 8 and 9 show the intercorrelation matrix for these items. All of the items were correlated with all the other items at or above the .40 cutoff level for both waves 1 and 2, except for item 9f. Specifically, item 9f (“I am familiar with the technical language or jargon outside providers use related to serving youth and families”) was correlated below .40 with items in wave 2. Thus, based on the recommendations from the literature (Armenakis et al., 2007; Hinkin, 1998; Kim & Mueller, 1978), item 9f was removed from the scale. Following the correlation analysis, a principle components factor analysis using a varimax rotation was conducted on the remaining 8 items to determine factor structure. All 8 of the items loaded cleanly on to one factor for each wave (item loadings were all above .75 for wave 1 and above .63 for wave 2). Based on the factor structure, a mean score was calculated across the 8 items to indicate the participants’ knowledge of relevant differences of providers from outside agencies. The alpha for the wave 1 survey was .93, and .92 for the wave 2 survey. Table 8. Item descriptives for knowledge of differences measure wave 1. Item Mean SD 1 9a 2.38 1.31 9b 2.06 1.03 .73** 9c 1.94 1.14 .61** 9d 1.88 1.02 .63** 9e 2.27 1.18 .57** 9f 2.42 1.06 .50** 9g 2.21 1.22 .46** 9h 2.15 1.48 .54* 9i 1.55 1.30 .45* Note: * p<.05, **p<.01 2 3 4 5 6 7 8 9 .69** .69** .60** .69** .69** .67** .70** .71** .57** .59* .66** .69** .74** .62** .51** .60** .47** .61* .75** .78** .58** .51* .85** .59* .71** .59** .65** .62* - 82 Table 9. Item descriptives for knowledge of differences measure wave 2. Item Mean SD 1 9a 2.36 1.03 9b 2.33 0.99 .71** 9c 2.36 1.08 .66** 9d 2.39 0.93 .66** 9e 2.70 1.02 .59** 9f 2.39 0.97 .23 9g 2.42 0.90 .61** 9h 2.27 1.23 .41* 9i 1.67 1.24 .42* Note: * p<.05, **p<.01 2 3 4 5 6 7 8 9 .76** .70** .70** .41* .71** .67** .55** .72** .61** .37* .64** .70** .60** .76** .52** .72** .64** .41* .57** .76** .62** .40* .48** .40* .53** .62** .47** .41* - Skills in reflective dialogue. Participants’ communication skills were measured by an 8item scale. Four items were adapted from a scale by Drach-Zahavy (2001) and four items were developed for this study based on the strategies proposed by Senge (2006) and Argyris (1991) for promoting reflective dialogue. Items assessed participants’ perceived ability to communicate in ways that balanced inquiry with advocacy (i.e., integrate and utilize diverse perspectives and expertise when problem-solving around a case; see question 5 a-h in Appendix E) using a 6-point Likert scale ranging from 0=“not at all” to 5=“a great deal”. The internal consistency of the items within the shared construct domain was examined using the items’ intercorrelation matrix (see Tables 10 and 11). For both waves 1 and 2, all of the items were correlated with each other above the .40 cutoff level suggested in the literature (Armenakis et al., 2007; Hinkin, 1998; Kim & Mueller, 1978). A principle components factor analysis using a varimax rotation was conducted on the 8 items and produced a single factor (item loadings were all above .70 for both waves). Given these results, a mean score was calculated across the 8 items to indicate the participants’ skills in reflective dialogue. The alpha for this scale in wave 1 was .95, and .92 for the wave 2 survey. 83 Table 10. Item descriptives for reflective dialogue skills measure wave 1. Item Mean SD 5a 2.73 1.46 5b 3.52 1.20 5c 3.30 1.29 5d 3.12 1.24 5e 3.70 1.19 5f 3.22 1.18 5g 2.94 1.34 5h 3.09 1.40 Note: * p<.05, **p<.01 1 .81** .61** .67** .67** .67** .70** .61** 2 3 4 .72** .65** .68** .73** .67** .57** .76** .76** .73** .73** .77** .75** .83** .71** .78** 5 6 .74** .67** .74** .66** .69* 7 8 .78** - Table 11. Item descriptives for reflective dialogue skills measure wave 2. Item Mean SD 5a 2.73 1.33 5b 3.48 1.09 5c 3.45 1.06 5d 3.39 1.03 5e 3.76 1.15 5f 3.52 1.12 5g 2.91 1.28 5h 3.12 1.22 Note: * p<.05, **p<.01 1 .59** .49** .56** .59** .64** .57** .56** 2 3 4 5 6 7 8 .53** .58** .50** .48** .55** .45** .49** .56** .66** .51** .46** .80** .71** .76** .71** .81** .83** .60** .71** .73** .75** - Beliefs about collaboration. Participants’ beliefs about collaboration were measured by a 13-item scale adapted from scales by Armenakis et al., (2007), Thompson, Perry, & Miller (2007), and Flaspohler, Anderson-Butcher, Bean, Burke, & Paternite (2008). Items assessed the extent to which participants agreed with the belief statements using a six-point Likert scale ranging from 0=“not at all” to 5=“a great deal”. This scale contained three subscales. The first subscale included 4 items related to participants’ beliefs about the appropriateness of interprofessional collaboration (e.g., I believe collaborating with service providers from other agencies will have a favorable effect on my practice; see question 2 a-d in Appendix E). The second subscale included 4 items measuring whether participants believed inter-professional collaboration would benefit them personally (e.g., increasing my collaboration with providers in 84 other agencies will benefit me in my job; see question 2 f-h in Appendix E). The third subscale included 5 items measuring whether participants believed they have a common shared goal or mutuality with other providers in Saginaw (e.g., I will better meet the needs of youth and families on my caseload by working with other service providers in Saginaw than working alone; see question 2 i-m in Appendix E). The items in each subscale were examined for their intercorrelations. Table 12 and 13 show the intercorrelation matrices for both wave 1 and wave 2 surveys. Using the .40 cutoff criterion (Armenakis et al., 2007; Hinkin, 1998; Kim & Mueller, 1978), the matrices revealed that all items within the first subscale (appropriateness of collaboration) for both wave 1 and 2 had significant correlations above .40. However, the items within the other two subscales (benefits of collaboration and mutuality) did not all correlate above .40 with all the other subscale items for the two waves; the inter-item correlations for these two subscales were particularly low in wave 2, and even negative in some cases. Thus, the items within the second and third subscales (2e-3m) were removed from the measure. The items within the first subscale (2a-d) were entered into a principle components factor analysis using a varimax rotation to determine factor structure. All items for each wave loaded cleanly onto one factor (item loadings were all above .70). A mean score was calculated on the four items within the appropriateness subscale to represent the final beliefs about collaboration scale which had an alpha of .89 for the wave 1 survey, and .86 for the wave 2 survey. 85 Table 12. Item descriptives for beliefs about collaboration measure wave 1. Item Mean SD Subscale 1: Appropriateness 2a 4.21 1.14 2b 4.18 1.01 2c 3.85 1.25 2d 3.91 .98 Subscale 2: Benefits 2e 4.00 1.15 2f 4.16 .92 2g 4.09 .95 2h 3.72 1.55 Subscale 1: Mutuality 2i 3.91 1.21 2j 3.24 1.41 2k 3.48 1.28 2l 2.79 .96 2m 4.09 1.18 Note: * p<.05, **p<.01 1 2 3 4 .86** .70** .55** .74** .55* .68** - .39* .78** .01 .39* .49** .08 - .67* .29 -.29 .24 -.15 .18 -.21 -.07 .17 .44* 5 - Table 13. Item descriptives for beliefs about collaboration measure wave 2. Item Mean SD Subscale 1: Appropriateness 2a 4.24 .83 2b 4.06 .90 2c 4.09 .95 2d 4.09 .91 Subscale 2: Benefits 2e 4.00 1.09 2f 4.15 1.03 2g 3.97 .88 2h 4.09 1.26 Subscale 1: Mutuality 2i 3.88 1.17 2j 3.82 .88 2k 3.82 1.24 2l 2.91 .95 2m 4.20 .81 Note: * p<.05, **p<.01 1 2 3 4 .52** .65** .42* .76** .72** .53** - .00 .68** .02 -.10 .64** -.138 - -.30 -.10 .36* .29 .66** -.02 .12 -.04 -.06 .29 5 - Optimized transformative exchanges. Participants’ initiation of optimized transformative exchanges was measured by a 10-item scale. Four of these items were adapted from scales by 86 Drach-Zahavy et al. (2001) and Mellin, Bronstein, Anderson-Butcher, Amorose, Ball, & Green (2010), and 6 items were developed for this study based on the theory within Eoyang’s CDE model as well as the suggestions from system of care stakeholders. These items were combined into 2 subscales. The first subscale included six items measuring OTEs related to specific cases (e.g., suggest ways to integrate your different perspectives or expertise for the sake of problemsolving around a case; see question 6 a-f in Appendix E). The second subscale included four items measuring OTEs related to encouraging (diffusing) system-wide collaboration (e.g., discuss ways to overcome barriers to collaborating with providers from different agencies; see question 7 a-d in Appendix E). Items assessed how frequently providers initiated various types of exchanges with individuals from outside agencies using a six-point Likert scale ranging from 0=“less than quarterly” to 5=“daily”. An intercorrelation matrix was calculated for each subscale to assess the internal consistency of each domain. Table 14 and 15 show the intercorrelation matrices for both wave 1 and wave 2 surveys. The matrices revealed that all items for both wave 1 and 2 had significant correlations above the .40 cutoff point. Table 14. Item descriptives for OTE measure wave 1. Item Mean SD 6a 3.24 1.09 6b 3.09 1.33 6c 2.91 1.53 6d 2.92 1.45 6e 2.06 1.97 6f 2.30 1.78 7a 2.55 1.75 7b 2.68 1.61 7c 2.39 1.68 7d 2.14 1.95 Note: * p<.05, **p<.01 1 .82** .73** .80** .55** .63** .71** .77** .81** 2 3 4 5 6 .84** .70** .54** .54** .77** .42* .55** .53** .69** .82** - .82** .74** .83** - 87 Table 15. Item descriptives for OTE measure wave 2. Item Mean SD 6a 3.30 1.26 6b 3.24 1.25 6c 3.03 1.45 6d 3.18 1.31 6e 2.03 1.69 6f 2.33 1.49 7a 2.58 1.62 7b 3.00 1.32 7c 2.64 1.39 7d 2.55 1.52 Note: * p<.05, **p<.01 1 .89** .79** .71** .57** .64** .71** .62** .68** 2 3 4 5 6 .86** .78* .65** .72** .90** .72** .78** .70** .74** .74** - .85** .78** .84** - All 10 items were entered into a principle components factor analysis using a varimax rotation. All items for each wave items except items 6e (“When interacting with outside providers, how often do you evaluate each others’ ideas in order to improve effectiveness?”) and 6f (“When interacting with outside providers, how often do you share reflections on the way services are being delivered to a case?”) loaded cleanly on two factors associated with each subscale (item loadings were all above .70). Table 16 summarizes the factor loadings for the two subscales across wave 1 and 2. Table 16. Factor analysis results for OTE scale. Item 6a 6b 6c 6d 6e 6f 7a 7b 7c 7d Wave 1 Components 0.19 0.90 0.31 0.90 0.39 0.87 0.48 0.78 0.55 0.62 0.65 0.57 0.48 0.75 0.35 0.82 0.24 0.90 0.26 0.92 Wave 2 Components 0.23 0.88 0.04 0.93 0.02 0.91 0.30 0.85 0.55 0.59 0.65 0.54 -0.02 0.86 0.09 0.92 0.34 0.85 0.24 0.88 88 Based on the factor analysis results, a mean score was calculated for the two subscales; these subscales were then averaged to create an overall scale of participants’ OTEs. The alpha for this scale was .95 for the wave 1 survey, and .93 for the wave 2 survey. Collaborative behaviors. Participants’ collaboration was measured by a 7-item scale. Four of these items were taken from Mellin et al.’s (2010) collaboration scale, and three items were developed for this study based on the theory within Eoyang’s CDE model as well as the suggestions from system of care stakeholders. Items assessed the extent to which collaborative behaviors resulted from their interactions with individuals from outside agencies (e.g., As a result of working together, services/supports for youth are delivered in new ways; see question 8 a-g in Appendix E) using a six-point Likert scale ranging from 0=“not at all” to 5=“a great deal”. The internal consistency of the items within the collaborative behaviors domain was examined using the item’s intercorrelation matrix (see Tables 17 and 18). Across waves 1 and 2, all of the items except 8a (“New practices related to working with youth occur as a result of the diversity of ideas among collaborating providers”), 8e (“Service plans are jointly created by providers who share cases”), and 8g (” Providers work together to provide ongoing follow-up to a shared case”) were correlated with each other above the .40 cutoff level. Thus, these three items were removed from the scale. Table 17. Item descriptives for collaborative behaviors measure wave 1. Item Mean SD 8a 1.97 1.19 8b 2.30 1.26 8c 2.03 1.31 8d 2.18 1.47 8e 1.94 1.32 8f 1.70 1.31 8g 2.41 1.19 Note: * p<.05, **p<.01 1 .70** .66** .56** .62** .68** .62** 2 3 4 5 6 7 .71** .75** .57** .42** .32 .79** .63** .66** .42** .60** .61** .38** .69** .46** .70** - 89 Table 18. Item descriptives for collaborative behaviors measure wave 2. Item Mean SD 8a 1.91 1.16 8b 2.18 0.98 8c 2.03 1.29 8d 2.24 1.03 8e 1.58 1.03 8f 1.39 1.06 8g 2.27 1.01 Note: * p<.05, **p<.01 1 .73** .32 .62** .31 .36* .18 2 3 4 5 6 7 .44* .60** .42* .53** .20 .60** .58** .68** .45** .36* .43* .24 .87** .30 .48** - The remaining four items were entered into a principle components factor analysis to determine factor structure. Using a varimax rotation, the analysis showed that all items for each wave loaded cleanly onto one factor (item loadings were all above .70). Given this factor structure, a mean score was calculated across these items to indicate the participants’ overall frequency of collaboration. The alpha for this scale was .89 for the wave 1 survey, and .82 for the wave 2 survey. Validation measures. To validate the self-reported survey responses, two sets of structured interviews were conducted with a randomly selected subset of intervention and control group participants (n= 14) one month prior to the intervention started and one month after the intervention finished. When applicable, the response options given to participants during the interviews corresponded with the response options included in the survey. See Appendix G for a copy of the structured interview protocol. Four validation measures were created based on these interviews that assessed participates’ knowledge of containers, knowledge of differences, transformative exchanges, and collaborative behaviors related to a recently completed case involving a youth receiving services from at least one additional system of care partner agency. Specifically, participants’ responses to the structured interview questions were assigned values (either rank values to illustrate increasing levels of capacity and behaviors, or dichotomous 90 values to illustrate the presence or absence of a particular capacity or behavior) based on the response options from the survey. These values were then entered into a data base. One categorical and three continuous measures were developed from this process that were used to validate the survey responses related to knowledge of containers, knowledge of differences, exchanges, and collaborative behavior. The four measures are described below. Knowledge of system containers. One categorical measure was created to validate participants’ knowledge of containers based on the following interview question: Did you ever contact another agency for assistance with the case, and if so, who did you first contact for this assistance? This question was assessed using a 0-1 scale were 0 indicated the participant did not attempt to contact a provider within an external agency container and 1 indicated the participant did attempt to make contact with an external container. Knowledge of differences. Another measure was created to validate participants’ knowledge of outside providers’ differences. The measure was based on the following 3 interview questions asking participants about each outside provider connected to the case: 1) did this person have any information (e.g., about the case, available services, etc.) you did not have that was useful for improving outcomes for the case?; 2) did this person have any connections in Saginaw you did not have that were useful for improving outcomes for the case?; and 3) did this person suggest any service ideas you alone did not come up with that were useful for improving outcomes for the case? These questions were assessed using a 0-1 scale. Participants’ answered the questions for each of the providers connected to the case (on average there were 1.75 outside providers connected to participants’ cases in wave 1, and 1.64 outside providers connected to their cases in wave 2). Responses were averaged across providers within each response category 91 to create three separate information, connection, and idea subscales. These three subscales were summed to create an overall differences measure (ranging from 0-3). Transformative Exchanges: One measure was created to validate participants’ initiation of exchanges with the outside providers connected to the case based on the following interview question: How often did you talk with this provider about the case? Participants’ answers were quantified on a scale from 1-8 where 1=never, 2= less than every 4 months, 3=every 2-3 months, 4= monthly, 5= every other week, 6=once a week, 7= a few times a week, and 8=daily. Participants’ answered this question for each of the providers connected to the case, and these responses were averaged to create an overall exchange measure based on a 1-8 scale. Collaborative behaviors. One measure was developed to validate participants’ collaborative behaviors based on the following interview questions: 1) Did you come up with new approaches for the case through working with the other providers connected to the case?; and 2) did you and the other providers connected to the case come up with new ways of working together? These two questions were assessed using a 0-1 scale, and participants’ responses were summed to create an overall collaboration measure based on a 0-2 scale. 92 Results General Analysis Strategy A series of quantitative and qualitative analyses were used to examine the research questions and hypothesis posed in this study, as well as to examine the validity of participants’ responses. Research question 1 was answered using regression to examine the extent to which the four self-organizing capacities were related to participants’ initiation of OTEs. The study used a mediation analysis with bootstrapping (Preacher & Hayes, 2004) to answer research question 2 by examining the mediating effect of OTEs between self-organizing capacity and collaboration. Several additional regression analyses were conducted to supplement the mediation findings. Mixed-design repeated measures analysis of variance (ANOVA) was used to answer research questions 3-5 by testing the differences in self-organizing capacities, OTEs, and collaborative behaviors between wave 1 and wave 2 for participants in the intervention and control groups. The structured case study interviews from both control and intervention participants were correlated with the survey measures as a form of validation. Finally, the focus group sessions and follow-up interviews were content analyzed to triangulate the survey findings. The results of these analyses are described below. Based on the final sample size that resulted from the drop in participation over the course of wave 1 and wave 2 data collection, the estimated power of the study was .10 for a small effect size of the intervention, .30 for a medium effect size, and .65 for a large effect size. Given that the anticipated effect size of the intervention was most likely to be small to moderate, and that all hypotheses were one-directional, the repeated measures ANOVA and regression analyses described below were conducted using a one-tailed significance test in order to maximize the power of the study. 93 Data Entry Data collected through surveys were entered into SPSS manually by research assistants. Several steps were taken to protect against human error in this process. First, a survey codebook was created that included detailed instructions for how data were to be entered for each scale. Assistants were trained in how to use this codebook. Second, research assistants demonstrated their data entry ability in a practice task and one-on-one meetings were held with each assistant to discuss mistakes. Third, all surveys were entered independently into two separate databases by two separate assistants. These databases were then merged and cross-referenced for errors. Any inconsistencies or errors were investigated and resolved. Descriptive and Diagnostic Information Prior to conducting the primary study analyses, descriptive information on all variables was analyzed to examine distributions of the data, homogeneity of variance, and outliers. A summary of descriptive information for all variables is presented in Tables 19-21. Table 22 includes a correlation matrix of all the study variables for the full sample. Again, all variables were measured on a 0-5 scale. First, the Kolmogorov-Smirnov test was used to examine the distributions of the 6 study variables at both time points for the intervention and control groups. Analysis showed that the following variables were not normally distributed: control participants’ knowledge of containers variable for wave 2 had a bi-modal distribution (D(16)=.22, p=.05); control participants’ knowledge of differences variable for wave 1 was positively skewed (D(16)=.22, p<.05); and control group participants reflective dialogue skills variable for wave 1 was negatively skewed (D(16)=.22, p<.05). To further examine these distributions, the skewness and kurtosis statistics were transformed into z scores and examined for significant values (an absolute value greater than 1.96 is considered significant at the p<.05 level). Two variables 94 Table 19. Variable descriptives for intervention group participants. Statistics Wave 1 N Mean Std. Dev. Range Min. Max. Skewness Kurtosis Wave 2 N Mean Std. Dev. Range Min. Max. Skewness Kurtosis Knowledge of Containers Knowledge of Differences Reflective Dialogue Skills Beliefs about Collaboration 17 2.53 .82 2.67 1.33 4.00 .27 -1.10 17 1.95 .91 3.13 .50 3.63 .20 -1.11 17 3.25 1.18 3.88 1.13 5.00 -.50 -.61 17 4.10 .94 3.50 1.50 5.00 -1.44 2.50 17 2.76 1.30 4.63 .38 5.00 .20 -.66 17 1.87 1.05 3.75 .25 4.00 .38 -.13 17 3.17 .81 2.67 1.83 4.50 .08 -.96 17 2.50 .75 2.88 .88 3.75 -.41 -.08 17 3.40 .98 3.00 1.63 4.63 -.44 -.79 17 4.32 .65 2.25 2.75 5.00 -1.10 .75 17 3.14 .99 3.63 1.38 5.00 -.36 -.37 17 2.06 .83 3.00 .50 3.50 .13 -.70 95 Initiation of OTEs Collaborative Behavior Table 20. Variable descriptives for control group participants. Statistics Wave 1 N Mean Std. Dev. Range Min. Max. Skewness Kurtosis Wave 2 N Mean Std. Dev. Range Min. Max. Skewness Kurtosis Knowledge of Containers Knowledge of Differences Reflective Dialogue Skills Beliefs about Collaboration 16 2.65 .89 3.83 .50 4.33 -.56 1.31 16 2.16 1.08 4.50 .50 5.00 1.10 2.30 16 3.13 1.08 3.25 1.25 4.50 -.44 -1.14 16 3.97 1.00 3.00 2.0 5.00 -.71 -.93 16 2.72 1.36 4.50 .00 4.50 -.40 -.64 16 2.25 1.26 4.00 .00 4.00 -.59 -.71 16 2.94 1.00 3.33 1.00 4.33 -.42 -.93 16 2.12 .92 2.75 .88 3.63 .30 -1.50 16 3.19 .91 3.13 1.50 4.63 -.16 -.95 16 3.91 .81 2.50 2.50 5.00 .10 -1.08 16 2.73 1.07 3.25 1.25 4.50 -.16 -1.26 16 1.86 .95 3.25 .50 3.75 .75 -.09 96 Initiation of OTEs Collaborative Behavior Table 21. Variable descriptives for entire study sample. Statistics Wave 1 N Mean Std. Dev. Range Min. Max. Skewness Kurtosis Wave 2 N Mean Std. Dev. Range Min. Max. Skewness Kurtosis Knowledge of Containers Knowledge of Differences Reflective Dialogue Skills 33 2.60 .84 3.83 .50 4.33 -.15 -.09 33 2.05 .99 4.50 .50 5.00 .76 1.10 33 3.19 1.12 3.88 1.13 5.00 -.43 -.88 33 3.06 .90 3.50 1.00 4.50 -.31 -.68 33 2.31 .85 2.88 .88 3.75 -.10 -1.17 Beliefs about Collaboration 33 3.30 .94 3.13 1.50 4.63 -.27 -.95 Collaborative Behavior 33 4.04 .95 3.50 1.50 5.00 -1.01 .27 33 2.74 1.31 5.00 .00 5.00 -.10 -.72 33 2.05 1.15 4.00 .00 4.00 -.11 -.86 33 4.12 .75 2.50 2.50 5.00 -.46 -.89 97 Initiation of OTEs 33 2.94 1.03 3.75 1.25 5.00 -.27 -.88 33 1.96 .88 3.25 .50 3.75 .41 -.59 Table 22. Correlation matrix between study variables for full sample of 33. Variable Study Condition Knowledge of Containers Wave 1 Knowledge of Differences Wave 1 Reflective Dialogue Skills Wave 1 Beliefs about Collaboration Wave 1 OTEs Wave 1 Collaborative Behaviors Wave 1 Knowledge of Containers Wave 2 Knowledge of Differences Wave 2 Reflective Dialogue Skills Wave 2 Beliefs about Collaboration Wave 2 OTEs Wave 2 Collaborative Behaviors Wave 2 Note: * p<.05, **p<.01 1 -.07 -.11 .06 .07 .01 -.17 .13 .23 .11 .28 .40* .11 2 3 4 5 6 7 8 .65** .49** .03 .12 .23 .68** .38* .36* .11 .40* .17 .22 -.14 .29 .37* .32 .30 .24 .15 .41* .09 .16 .18 .29 .33 .58** .75** .17 .38* .52* -.23 .18 .02 .15 .02 .44* -.12 .10 .23 .21 .12 .23 .08 .50** .01 .03 .34 .28 .38* .40* .58** .47* .39* .25 .48** .20 98 9 10 11 12 .64** .41* .21 .57** .50** .35* .54** .55** .18 .39* 13 - showed significant values from this analysis: beliefs about collaboration and knowledge of differences. Specifically, the intervention group’s wave 1 beliefs scale had significant negative skewness and positive kurtosis; their wave 2 beliefs scale also had significant negative skewness. The control group’s knowledge of differences variable had significant positive skewness and positive kurtosis. These distributions are most likely the result of a small sample size and limited variability in item responses (e.g., the beliefs about collaboration scale showed a ceiling effect for both groups). Three separate transformations were conducted on the variables in an attempt to address the non-normal distributions described above including the square root, log, and inverse transformations. A constant of 1 was added to all variables prior to the transformation to account for values of zero. The Kolmogorov-Smirnov, skewness, and kurtosis tests described above were used to assess the distributions for the transformed variables. These analyses showed that all transformations were unsuccessful at normalizing the distributions of the effected variables, except for the square root transformation of the knowledge of differences variable. However, the transformed knowledge of differences variable did not produce significantly different results in the subsequent analyses than the non-transformed variable. Thus, in light of these findings, none of the study variables were transformed. While the transformations proved unsuccessful, several other tests suggested that the data was still appropriate for the planned regression and repeated measures analyses. For example, Levene’s test produced non-significant coefficients for all the variables in all waves, implying that despite the evidence of non-normality the variance of scores was not significantly different across the intervention and control groups in each wave. The data were also examined for possible outliers by transforming all variables into z scores and identifying any occurrences of 99 absolute values above 3.29 (greater than 2 standard deviations above the mean). No outliers appeared in the in any of the variables for wave 1 or wave 2. Given the results of these two tests, and the fact that both repeated measures ANOVA and multiple regression analyses are robust to non-normal samples, the decision was made to move forward with the planned analyses. Relation between Self-Organizing Capacities and Initiation of OTEs Regression was used to answer research question 1 (to what extent are self-organizing capacities related to the initiation of transformative learning exchanges?) and hypotheses 1-4. Specifically, four separate regression models were defined where knowledge of containers, knowledge of differences, skills in reflective dialogue, and beliefs about collaboration each individually predicted the OTE variable at time 2. Study condition (control versus intervention) was also entered into each of these models as a way of controlling for group assignment. Prior to examining the models, a series of diagnostics were run on the data to examine evidence for the violation of assumptions including: cases with unreasonable influence on the model, multicolinearity, the independence of errors, and the normal distribution of errors. First, a leverage statistic was calculated to assess any unreasonable influence of certain cases on the models. Hoaglin and Welsch (1978) suggest an unreasonable leverage cut-off point as above the value of 2(k+1)/n where k=predictors and n=sample; upon examining the data, all cases were below this value for the regression models indicating no case had unreasonable influence on the model. To further test for influence of particular cases on the model, Cook’s distance was estimated. None of the cases had a Cook’s distance value above 1, indicating there was no abnormal affect of a single case on each of the models (Cook & Weisberg, 1982). Second, two tests were used to examine collinearity between predictors in the regression models: the Variance Inflation Factor (VIF) and Tolerance. The VIF statistic was below the threshold of 10 (Myers, 100 1990) and Tolerance statistic was above the threshold of .2 (Menard, 1995) for the models, indicating no significant collinearity between the predictors. Third, the Durbin-Watson test was used to assess independence of error across variables and produced statistics that were within the reasonable limit (above 1 and below 3) for all the models indicating the assumption of independent errors was met. Finally, the Kolmogorov-Smirnov, skewness, and kurtosis tests were applied to the standardized residuals in each of the regression models; none of the three tests showed significant coefficients indicating that the errors were normally distributed. Because the data met these key assumptions, the decision was made to move forward with the planned analysis. Four separate regression analyses were run on the full 33 participant sample where each self-organizing capacity variable at time 2 independently predicted the OTE variable at time 2. In order to control for participants’ group assignment (control or intervention), study condition was also entered as an independent variable into each of these regression models. Each of these analyses used a one-tailed significance test. The model including knowledge of containers and 2 study condition was significant (F(2,30)=5.11, p<.01, r =.25) with knowledge of containers significantly predicted OTEs (t(30)=2.93, b=.53, p<.01, ES=.47); study condition was not significant in this model (t(30)=.90, b=.029, p=n.s.). When knowledge of differences and study condition were entered as predictors of OTEs, the overall model was also significant 2 (F(2,30)=7.21, p<.001, r =.33) with knowledge of differences significantly predicting transformative exchanges (t(30)=3.55, b=.66, p<.001, ES=.54); again, study condition was not significant in this model (t(30)=.51, b=.16, p=n.s.). The regression model with skills in reflective 2 dialogue and study condition as predictors was significant (F(2,30)=5.46, p<.01, r =.27), and 101 skills in reflective dialogue significantly predicted transformative exchanges (t(30)=3.04, b=.52, p<.01, ES=.49); study condition was not significant in this model (t(30)=.95, b=.30, p=n.s.). The model with study condition and beliefs about collaboration as predictors was also significant 2 (F(2,30)=2.31, p=.05, r =.13), and beliefs about collaboration significantly predicted OTEs (t(30)=1.79, b=.44, p<.05, ES=.31) while study condition was not significant in the model (t(30)=.64, b=.23, p=n.s.). Overall, the results of these analyses supported hypothesis 1-4. The Mediating Role of OTEs on Collaborative Behaviors A series of mediation tests with percentile bootstrapping was used to answer research question 2 (To what extent does the initiation of transformative learning exchanges mediate the relationship between self-organizing capacities and collaborative behaviors?) and hypotheses 5. Prior to conducting this analysis, the degree to which wave 2 OTEs predicted wave 2 collaborative behaviors after controlling for group assignment was assessed using regression with a one-tailed significance test. This overall regression model was significant (F(2,30)=2.77, 2 p<.05, r =.17) with OTEs significantly predicting collaborative behaviors (t(30)=2.25, b=.33, p<.05, ES=.38); study condition was not significant in this model (t(30)=.21, b=.06, p=n.s.). Given that the regression analyses described in previous sections suggested that each of the four wave 2 self-organizing capacity variables significantly predicted the wave 2 OTE variable, there was evidence for a possible mediated relationship in the IDeAL Partnerships framework. Four separate bootstrapped mediation analyses were conducted on the full sample where the OTE variable mediated the path between each of the four capacity variables (independent variables) and collaborative behaviors (the dependent variable). Bootstrapping is a nonparametric approach that estimates the mediation effect on a series (e.g., 10,000) of random samples from the overall sample and computes the indirect effect for each sample; these effects 102 are then compiled into a bootstrapped distribution which is used to develop a 95% confidence intervals (Preacher & Hayes, 2004). If the 95% confidence interval does not contain zero, it can be concluded that the indirect mediated effect is significant at p<.05 (Preacher & Hayes, 2004). This approach is preferred over other traditional mediation tests because it does not assume that the distribution of mediated values are normally distributed and as a result has increased power with small samples (Preacher & Hayes, 2004). The four bootstrapped mediation tests found that OTEs did not significantly mediate any of the relationships between the capacity variables and collaborative behaviors because the 95% confidence intervals for each test included zero. Specifically, the 95% confidence interval for the mediation tests related to knowledge of containers (B=.17, bias-corrected 95% CI: -.02 to .37), knowledge of differences (B=.09, biascorrected 95% CI: -.01 to .27), reflective dialogue skills (B=.09, bias-corrected 95% CI: -.08 to .26), and beliefs about collaboration (B=.15, bias-corrected 95% CI: -.06 to .35) suggest that hypothesis 5 was not supported by the data. Given that none of the mediation analyses were significant, several additional regression analyses were conducted to explore the relationships between the self-organizing capacity variables and collaborative behaviors in the IDeAL Partnerships Framework. The four capacity variables were each entered into separate regression models predicting collaboration, with study condition entered as a control variable for group assignment. Results indicated that only knowledge of differences and reflective dialogue skills had significant overall regression models 2 2 (F(2,30)=6.27, p<.01, r =.30 and F(2,30)=6.70, p<.01, r =.31, respectively). Knowledge of differences significantly predicted collaborative behaviors in its individual model (t(30)=3.46, b=.57, p<.01, ES=.53), as did reflective dialogue skills (t(30)=3.60, b=.51, p<.001, ES=.55). 103 Study condition was not significant in either of these models (t(30)=-.06, b=-.17, p=n.s., and t(30)= .35, b= .09, p=n.s., respectively). In contrast, the regression models with knowledge of 2 containers (F(2,30)=.74, p=n.s., r =.05) and beliefs about collaboration (F(2,30)=.59, p=n.s., 2 r =.04) as predictors had very poor fit and were not significant; these two variables did not significantly predict collaboration in the models (t(30)= 1.03, b= .18, p=n.s., and t(30)= .88, b= .19, p=n.s., respectively), nor did the study condition variable (t(30)= .50, b= .16, p=n.s., and t(30)= .37, b= .12, p=n.s., respectively). Figure 4 summarizes the 7 significant linear regression relationships within the IDeAL Partnerships Framework. Figure 4. Significant regression relationships within the IDeAL Partnerships Framework. Initiating Optimized Transformative Exchanges (OTEs) Knowledge of Pertinent Containers Knowledge of Differences Addressing Specific Cases Skills in Reflective Dialogue Beliefs towards Collaboration Engagement in Self-Organizing, Collaborative Behaviors Embedding Collaboration A multiple regression model was defined with the four wave 2 self-organizing capacity variables, OTEs, and study condition all entered as predictors of wave 2 collaborative behaviors; this model was created in order to assess the relative strength of each predictor variable when all the other variables were held constant. The overall multiple regression model was significant 2 (F(6,26)=2.70, p<.05, r =.38), although reflective dialogue skills was the only variable that significantly predicted collaboration (t(26)=1.65, b=.32, p=.05, ES=.31). However, there was a trend toward knowledge of differences also significantly predicting collaborative behaviors in 104 this model (t(26)=1.47, b=.35, p=.08, ES=.28). In contrast, knowledge of containers (t(26)= -.79, b= -.14, p=n.s.), beliefs about collaboration (t(26)= -.19, b= -.04, p=n.s.), OTEs (t(26)= .56, b= .10, p=n.s.), and study condition (t(26)= .02, b= .01, p=n.s.) did not significantly predict collaboration in this model. Overall the bootstrapped mediation analyses suggest that OTEs did not fully mediate the path between the self-organizing capacities and collaborative behaviors. The fact that reflective dialogue skills and knowledge of differences had meaningful relationships with collaborative behaviors even when OTEs were held constant further supports the lack of a fully mediated model within the IDeAL Partnerships Framework. However, the linear regression findings suggest that OTEs may still play a partial mediation role in the model. This potential area for future research will be explored in the discussion section. The Intervention’s Influence on Participants’ Self-Organizing Capacity Four mixed-design repeated measures ANOVA tests were used to answer research question 3 (does the IDeAL Partnerships Intervention have a significant influence on participants’ self-organizing capacities?) and hypotheses 6-9. As mentioned above, all of these tests used a one-tailed significance test. Figures 5-8 display graphs for each of the four tests described below. Knowledge of containers. The first model was defined with knowledge about pertinent containers over the two time points as the within-subjects variables and group assignment as the between-subjects factor. Results show that participants’ knowledge of containers significantly increased across the two time points for the overall sample, F(1,31)=14.88, p<.001). There was a trend toward a significant interaction effect where intervention participants increased their 105 knowledge of containers more over time than control participants (F(1,31)=2.06, p=.08). This trend provided some support for hypothesis 6. Survey Response Figure 5. Average response to knowledge of containers measure for each group over time. 5 4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 Intervention Control Wave 1 Wave 2 Knowledge of differences. The second model was defined with knowledge about differences over the two time points as the within-subjects variables and group assignment as the between-subjects factor. Results showed that participants’ knowledge of differences did not significantly increase across the two time points for the overall sample, F(1,31)=1.50, n.s.). Survey Response Figure 6. Average response to knowledge of differences measure for each group over time. 5 4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 Intervention Control Wave 1 Wave 2 106 However, there was a strong trend toward a significant interaction effect where intervention participants increased their knowledge of differences more than control participants over the study duration (F(1,31)=2.58, p=.06). This strong trend supported hypothesis 7. Skills in reflective dialogue. The third model was defined with skills in reflective dialogue over the two time points as the within-subjects variables and group assignment as the between-subjects factor. Results show that participants’ reflective dialogue skills did not significantly increased across the two time points for the overall sample, F(1,31)=.563, p=n.s.). There was no significant interaction effect between condition and time for reflective dialogue skills (F(1,31)=.114, p=n.s.), meaning the groups did not differ on their growth in this area. Thus, there was no support in the data for hypothesis 8. Survey Response Figure 7. Average response to reflective dialogue skills measure for each group over time. 5 4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 Intervention Control Wave 1 Wave 2 Beliefs about collaboration. The fourth model was defined with beliefs about collaboration over the two time points as the within-subjects variables and group assignment as the between-subjects factor. Results show that belief in collective identity did not significantly increase across the two time points for the overall sample, F(1, 31)=.24, n.s.). Results showed no significant interaction effect between condition and time (F(1,31)=.77, n.s.), indicating there was 107 no difference in how the groups increased over time. Thus, hypothesis 9 was not supported by the data. However, it should be noted that the intervention and control groups both had high scores on the beliefs measure at time 1, suggesting there may have been a ceiling effect to their growth over time. This ceiling effect will be explored more in the discussion section. Survey Response Figure 8. Average response to beliefs about collaboration measure for each group over time. 5 4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 Intervention Control Wave 1 Wave 2 The Intervention’s Influence on Participants’ Initiation of OTEs To assess research question 4 (does the IDeAL Partnerships Intervention have a significant influence on participants’ initiation of transformative learning exchanges?) and hypothesis 10, a mixed-design repeated measures ANOVA was used to test the differences in OTEs between participants in the intervention and the control groups. Specifically, this analysis defined OTEs over two time points as the within-subjects variables and group assignment as the between-subjects factor. Results show that participants’ OTEs did not significantly increase across the two time points for the overall sample, F(1, 31)=.87, p=n.s.). There was no significant interaction between condition and time for this variable (F(1,31)=.80, p=n.s.), indicating the two groups did not differ in their growth over time. See Figure 9 for a graph of this change over time. According to this analysis, there was no support in the data for hypothesis 10. 108 Survey Response Figure 9. Average response to OTE measure for each group over time. 5 4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 Intervention Control Wave 1 Wave 2 To further examine this research question, the OTE subscales related to cases and diffusion were each examined separately in order to understand whether the intervention had a different effect on these two categories of OTEs. The alpha for the case-related subscale was .95 for the wave 1 survey and .93 for the wave 2 survey; the alpha for the diffusion subscale was .93 for the wave 1 survey and .92 for the wave 2 survey. The two subscales were each entered into a mixed-design repeated measures ANOVA where OTEs over two time points were defined as the within-subjects variables and group assignment was defined as the between-subjects factor. For the OTE subscale related to cases, results showed that participants’ case related transformative exchanges did not significantly increase across the two time points for the overall sample (F(1,31)=.44, p=n.s.), however there was a trend toward a significant interaction between condition and time where intervention participants increased their case-related transformative exchanges more than control group participants (F(1,31)=1.93, p=.09, ES=.24). This finding provided some support for hypothesis 10. See Figure 8 for a graph of these changes over time. 109 Survey Response Figure 10. Average response to case-related OTE subscale measure for each group over time. 5 4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 Intervention Control Wave 1 Wave 2 Results for the OTE subscale related to diffusion showed that the overall sample of participants did not significantly increase their diffusion of collaboration across the system over the study period (F(1,31)=.70, p=n.s.). There was also no significant interaction between condition and time (F(1,31)=.07, p=n.s.) indicating the two groups did not differ in their growth over the course of the study. See Figure 9 for a graph of these changes. Survey Response Figure 11. Average response to diffusion OTE subscale measure for each group over time. 5 4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 Intervention Control Wave 1 Wave 2 110 Overall the repeated measures ANOVA findings implied that the intervention had somewhat of an effect on participants’ exchange behavior related to discussing their personal cases, while no effect on their behavior related to spreading collaboration across the system. This effect will be further explored in the discussion section. The Intervention’s Influence on Participants’ Collaborative Behavior To assess research question 5 (does the IDeAL Partnerships Intervention have a significant influence on participants’ engagement in collaborative behavior?) and hypothesis 11, a mixed-design repeated measures ANOVA was used to test the differences in collaborative behavior between participants in the intervention and the control groups. Results show that participants’ collaborative behaviors did not significantly increase across the two time points for the overall sample, F(1, 31)=.37, p=n.s.). There was a significant interaction between condition and time for collaborative behaviors (F(1,31)=3.21, p<.05), however this was because Survey Response Figure 12. Average response to collaborative behaviors measure for each group over time. 5 4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 Intervention Control Wave 1 Wave 2 intervention participants maintained a constant level of collaborative behaviors over the study period while the control group decreased their collaborative behaviors over time. Thus, while this interaction finding supports hypothesis 11, it is due to the control group’s decrease in 111 behavior instead of an increase in behavior by the intervention participants. Figure 10 displays a graph of this analysis. Validation Analyses Case Study Validation. As described in the methods section, four measures were developed from the structured interviews that were conducted with a subset (n=14) of the overall sample at time 1 and time 2. During the interviews participants were asked to select a recent case where the youth was receiving services from at least one additional public agency (CMH, Courts, Schools, DHS) outside of their own agency; participants answered the structured interview questions based on their selected case. These measures were used to validate the survey measures by correlating participants’ interview responses to their responses on the survey. The correlation results for each of the four measures are summarized below. A summary of the means for these measures are included in Table 23. Knowledge of containers. As described above in the methods section, the validation measure for knowledge of containers was based on the structured interview question asking participants whether they had attempted to contact anyone within an outside public agency that had provided services to the youth (i.e., within a relevant container) for assistance with the case. Ten out of the 14 participants in the wave 1 interviews attempted to contact a provider in a relevant container, while 8 out of 14 attempted to make this contact in the wave 2 interviews. The resulting dichotomous measure (attempted contact = 1, did not attempt contact = 0) was correlated with the knowledge of containers survey measure for this subset of 14 participants. The point-biserial correlation results showed that the validation interview measure was not significantly related with the overall knowledge of containers survey measure for wave 1 (rpb=.22, p=n.s.) or wave 2 (rpb= -.34, p=n.s.). The validation measure was also not significantly 112 correlated with the knowledge of containers case-related subscale for either wave 1 (rpb=.26, p=n.s.) or wave 2 (rpb=.16, p=n.s.), or the service-related subscale for wave 1 (rpb=.30, p=n.s.) or wave 2 (rpb=.01, p=n.s.). Thus, the survey measure was not validated by the structured interview results. However, it is possible that participants’ actual attempts to contact an individual in a relevant container – as captured by the structured interview protocol - may have been distinct from their knowledge of relevant containers – as captured by the survey measure - given the specific needs of the targeted case. These implications will be further described in the discussion. Knowledge of differences. The knowledge of differences validation measure was based on three questions asking interview participants about their knowledge of the information, connections, or service ideas held by outside providers connected to the targeted case. Fourteen out of 14 interview participants answered these questions, and the resulting continuous measure (0-3 scale) was correlated with the knowledge of differences survey measure. Results showed that the validation interview measure was not significantly related to the survey measure for wave 1 (r=.26, p=n.s.) or for wave 2 (r=.27, p=n.s.). Thus, the structured interview results did not validate the knowledge of differences survey measure. However, participants’ responses to the structured interview questions about knowledge of differences may have been contingent on the actual range of ideas, connections, or service ideas held (or willingly shared) by the targeted outside providers connected with the cases. It may have also been impacted by the degree to which the targeted case required participants to tap into these specific differences. Both of these issues will be further described in the discussion section. Transformative exchanges. The validation measure for transformative exchanges was developed from a structured interview question asking participants about how often they spoke with the other providers connected with their targeted case. All 14 interview participants 113 answered this question. The relation between this measure and the survey measure on OTEs was examined using a correlation. Results showed that the validation measure was not significantly related to the overall OTE survey measure for wave 1 (r=.31, p=n.s.) or for wave 2 (r=.11, p=n.s.), nor was it related to the OTE case-related subscale for wave 1 (r=.24, p=n.s.) or wave 2 (r=.21, p=n.s.). The case study measure was significantly related to the OTE diffusion subscale for wave 1 (r=.48, p<.05), but not for wave 2 (r= -.09, p=n.s.). Thus, despite the one significant correlation, overall the OTE survey measure was not validated by the structured interview results. Collaborative behaviors. The validation measure for collaborative behaviors was developed from the structured interview question about whether participants came up with new approaches to the case or new ways of working together as a result of their interactions. All 14 interview participants answered this question. The resulting measure was dichotomous where 1=developed new approaches or ways of working together and 0=did not develop these approaches or ways of working together. A point-biserial correlation examined the relation between the validation measure and the survey measure for collaborative behaviors. Results showed the validation measure was not significantly related to the survey measure for wave 1 (rpb=.03, p=n.s.) or for wave 2 (rpb=.13, p=n.s.). Thus, the collaborative behaviors survey measure was not validated by the structured interview results. However, here again participants’ answers to this question may have been contingent on the collaboration needs of their specific case targeted within the structured interview. This issue will be further addressed in the discussion section. 114 Table 23. Summary of mean differences between intervention and control groups on validation measures. Validation Measure Knowledge of Container N included in analysis Wave 1 Mean, SD Wave 2 Mean, SD Intervention Control 7 .71, .49 .71, .49 7 .71, .49 .43, .53 Knowledge of Differences N included in analysis Wave 1 Mean, SD Wave 2 Mean, SD 7 1.55, 1.07 2.79*, .27 7 1.33, .76 1.71*, .81 Exchanges N included in analysis Wave 1 Mean, SD Wave 2 Mean, SD 7 3.02, 1.98 5.50, 1.41 7 4.29, 1.98 4.21, 1.89 Collaborative Behaviors N included in analysis 7 7 Wave 1 Mean, SD 1.20, .84 .50, .84 Wave 2 Mean, SD 2.00*, .00 1.17*, .75 Note: *=significant differences between means at p<.05 level Contamination Effects Contamination effects between the intervention and control groups over the duration of the study were examined using the repeated measures analysis of the OTE diffusion subscale related to sharing ideas or information about collaboration with colleagues (e.g., share information as a means to promote collaboration between providers from different agencies; exchange ideas about how to better tap into the unique expertise of providers from different agencies; items 7a-d). At time 1, participants in both the control and intervention groups on average indicated that they shared ideas and information about collaboration with colleagues on roughly a quarterly basis; as described above in the repeated measures ANOVA results, there was no significant change in this rate for either group at time 2. Seeing that the intervention only lasted 3 months, this analysis suggests that the diffusion between intervention and control groups 115 was most likely to be low during the study. However, it is still possible that intervention participants may have influenced members of the control group by talking generally about their experiences in the intervention – interactions that may not have been captured by the survey items on diffusion. Participants may have also influenced their colleagues by simply modeling new forms of collaborative behavior, another aspect that was not captured in the measure. The possible effects of contamination will be further described in the discussion section. Triangulation Analyses The focus group sessions and follow-up interviews conducted with intervention participants were analyzed to triangulate the survey findings. Recordings of the focus groups and interviews were transcribed by research assistants and the author verified the quality of the transcriptions by listening to each recording and making appropriate corrections where necessary. Participants’ comments were coded using deductive content analysis, an approach that involves using an existing theory or framework to code data for similar patterns and themes (Patton, 2002). Specifically, codes were developed based on the theoretical descriptions of the four self-organizing capacities and transformative exchanges, and the key intervention strategies (see Table 1). These codes were applied to the focus group and interview transcripts in order to analyze the data for themes. To aid in managing and displaying the data, coded sections were organized using a coding matrix (Miles & Huberman, 1994). Focus groups. The focus groups captured comments from all 17 participants about their experiences during the intervention. Participants spoke about through the intervention experience they had gained new forms of knowledge about the service delivery system and about each other, as well as new dialogue skills and beliefs about collaboration. They also discussed ways in which the intervention encouraged them to begin shifting some of their exchange behaviors with other 116 providers. The comments provided triangulation support to the survey findings, and confirmed some of the key design elements of the intervention process. The following is a summary of the key themes within the focus group data. Knowledge of containers. When asked about the most significant things they gained from the intervention experience, participants said the intervention had played an important role in improving their ability to find stakeholders in the system who were relevant to their cases. Participants said they gained this knowledge through the process of talking in-depth about how each of their agencies were structured, and compiling this information into a resource guide. As described above in the methods section, the resource guide was co-created throughout the study by the participants (who contributed information and agency documents at each meeting) and the author (who compiled this information into an integrated document between meetings). By the end of the intervention, the resource guide was a 26 page document that included information about each agency’s staff listings, organizational charts, service flow charts (to help participants identify the relevant providers who were working with youth at a particular stage in the service process), key contacts, available services, and relevant programs and agencies in the community. Participants said this document was a “very useful tool” that improved their ability to find the right people to meet their clients’ needs and made their understanding of how the service delivery system functioned more “clear.” For example, one participant from DHS recognized that while she still had much to learn about CMH, she could now locate the appropriate contacts to gain the information she needed: CMH does offer a lot, a variety of programs. I’m sure I only tapped into the surface of it, of understanding what’s available. But at least I know who to go to if I have questions. (Participant 51) 117 The participants’ value for this guide – and their new ability to navigate the system – was also demonstrated in their agreement to meet together every six months to update the guide after the intervention ended. Overall, the focus group discussions suggest that the intervention helped participants gain the knowledge and information they needed to effectively navigate and locate relevant stakeholders within the service delivery system. This finding is aligned with the repeated measures ANOVA results that showed a trend toward a significant interaction where intervention participants increased their knowledge of containers over the study period. However, the focus group data does not explain why the control group participants also significantly increased their knowledge of containers over the two time points (albeit less than the intervention group). Participants were explicitly asked not to share the guide until the end of the intervention to provide their agency leadership time to approve the guide for general distribution; everyone agreed to this request. Despite this agreement, contamination could still have occurred as participants modeled new behaviors or spoke to their colleagues about their experiences in the intervention. These considerations will be further discussed in later sections. Knowledge of differences. In addition to finding relevant stakeholders, participants said the intervention also gave them a better understanding of “what exactly each agency does” in the service delivery system, the different roles and responsibilities providers have in each agency, and how agency policies, timelines, caseloads, and legal mandates affect the way providers can delivery services. For some, the intervention provided participants with “clarification” around their “vague ideas” about how each agency functioned, challenged their assumptions, and gave them a better and more “nuanced” understanding of how other providers carry out their work: 118 My biggest thing for years is people not returning my phone calls. But now I have a better understanding of what your caseloads are, and how much work you guys actually do. (Participant 31) [The project helped us with] understanding each other’s jobs. Not everybody understood that about each other and their agencies. That was for me anyway. [It helped] when we sat and listened to what everyone brought in and the frustration at not knowing what other people were doing. I think a lot of it was just a lack of communication and understanding. (Participant 117) One participant believed that this knowledge brought about a new respect for “other agencies and the work that they do” in the group which allowed members to “become more connected.” Similarly, the intervention provided some participants with a better understanding of the common misconceptions about their own agencies and jobs. For example: Because I thought it was common knowledge that we [at DHS] aren’t sitting at our desk all day, you know. We’re in and out all day, and some days we’re not even in maybe 2030 minutes because we have so many families to see. So, I thought that they knew, but they didn’t. (Participant 56) One thing that really surprised me as we were going over all the different programs that we [at CMH] run is you guys were like ‘you’re doing this?’ I was like “how can you guys not know this!?” It’s just assumed by our administration that we talk to each other…I was just kind of surprised that that [information] wasn’t out there. (Participant 31) As with their knowledge of containers, the focus group comments suggest that the intervention significantly influenced many participants’ knowledge of the relevant and potentially distracting differences between providers. This data validates the repeated measures ANOVA findings that indicated there was a strong trend toward the intervention group significantly increasing their knowledge in this area compared to the control group. Not only did the focus group comments suggest that participants had gained a better understanding of provider differences in the intervention, but also that this knowledge was beginning to influence participants’ work behavior. For example, participants spoke about how their knowledge of provider differences was encouraging them to be more “respectful,” to “work 119 together and help each other,” and to intentionally address potential misconceptions of each other’s agencies. The following dialogue illustrates this shift: Participant 34: Yeah, but that was really helpful to me [in the group] to actually hear the worker say, ‘well, we [at DHS] really can’t do it.’ Cuz I hear them on the phone and then personally I’m like, yeah, you’re just sitting there, feet on the desk doing god knows whatever, shopping for shoes on the internet. [Group: laughing]…But actually coming here and hearing it, it’s like, alright, I got it. Participant 39: I think knowing everybody’s perspectives, where they’re coming from helps because you know, like Participant 34 said, we [at DHS] said we can’t do it…but that’s all. We could have said well we’re not able to because of this and this and this, you know, maybe actually explain a little more to maybe reach out to where other people are, to help where they’re coming from, to better fill in the gaps…After this, I feel a little more accepted and accept a little bit more, like I’m more open to what everybody’s coming to the table with. Instead of, ‘oh well, you’re CMH, we’re CPS, we’ll stay over here and hang out in our own corners.’ Participant 34: Mm-hm, more accommodating, and more respectful toward each other, I agree. One participant said that being able to take other providers “timeframes and duties and frustrations into consideration” allowed her to establish a new “flow” in her dealings with outside providers. Another participant noted that gaining knowledge about what services outside providers could offer shifted the way she advised and referred families in her practice: I enjoyed hearing some of the services that CMH offered. I felt like I didn’t have a full understanding of what was available. Even to tell my clients, oh you have a CMH worker? You might want to ask about respite. That kind of thing. And other things I could advise them to ask about. (Participant 51) These behavior shifts could represent the preconditions leading providers to engage in transformative exchanges and collaborative behaviors. These comments also give support to the regression analysis findings that participants’ knowledge of differences significantly predicted their engagement in transformative exchanges. Overall, the focus group data suggests that the intervention successfully shifted many participants’ knowledge of each other’s differences, and as a result influenced their interactions with each other and with their clients. This finding supports the repeated measures ANOVA 120 trend toward a significant interaction where the intervention participants increased their knowledge of differences more than the control group participants. It also supports the regression findings that participants’ knowledge of differences survey scores significantly predicted transformative exchanges. Reflective dialogue skills. Several participants in the focus groups talked about how the intervention had increased their confidence to engage in reflective dialogue with other providers. This primarily dealt with learning new ways to gather necessary information and to explore different perspectives. For example, participants said that learning more about each other’s organizations “opened a door to a new way to communication” and gave them a better idea about “what questions we should be asking” to get the information they needed for their cases. Learning these key questions was important because it allowed participants to better navigate through unfamiliar agency jargon and service access procedures. In addition to learning better questions, some participants also spoke about using specific strategies from the intervention to improve communication and reflective dialogue with other providers: One of the things that I liked was that exercise we did about how to talk about differences…where you might say, ‘maybe you can help me explain the gaps in my logic, I’m not coming to the same conclusion you are.” You know, little techniques like that where it diffuses it and brings it out a little. Yeah, like, “hey, I’m confused, I don’t want to fight, you know, I want you to have a say and you can maybe lead me around to see how you’re thinking. (Participant 36) Through learning these new questions and skills, some participants spoke about becoming more comfortable dialoging with providers from outside agencies about how to effectively and efficiently address the needs of a case. For example, one participant spoke about how her experiences in the intervention – primarily the experience of gaining accurate knowledge about other providers’ jobs - had made her more comfortable talking with workers from DHS about their shared cases: 121 I feel more comfortable now when I call the CPS workers and say,…’what exactly do you want to know and what do you think I could do to help?... I feel more comfortable now saying that, knowing that they’re not shopping for shoes and they really are working, that they’re also as frustrated as I am… I feel more comfortable doing that now. And I have done it with new workers. (Participant 34) As with knowledge of differences, the last part of this comment suggests that some participants had started using these skills in their actual practice. Overall, the focus group data suggests that the intervention had a positive effect on some participants’ reflective dialogue skills. Because the repeated measures ANOVA results showed no significant growth in these skills over time for the intervention (or control) participants, the focus group comments suggest that some of the shifts in reflective dialogue skills may not have been captured by the survey. It may also suggest that the intervention had different effects on different individuals. These issues will be further described in the discussion. On the other hand, the focus group finding that some participants had started using their skills to promote transformative exchanges in their work supports the regression findings that participants’ skills in reflective dialogue significantly predicted their engagement in transformative exchanges. Beliefs about collaboration. Participants talked about two main things they gained from the intervention related to beliefs about collaboration: beliefs about the benefits of collaboration, and beliefs about the similarities between providers. First, the focus group comments describe how the intervention shifted some participants’ beliefs about the benefits of collaboration. For example, the intervention helped one participant think differently about the ways in which collaboration can increase productivity through encouraging providers to “respect each other’s time” and “get back to each other a whole lot faster.” Others discussed how the intervention had influenced their beliefs about the importance of collaboration in preventing job burnout. The 122 following comments come from two participants discussing this shift; the first comment was verbally confirmed by other members of the focus group: Participant 21: I think for me personally [the project has] given me the ability to…maybe chill out a little bit. Working together is much easier and less stressful on me. I have the shortcoming of taking a lot on myself, and [the project has] given me the opportunity to share the pain…It’s definitely a work in progress - it’s gonna take a lot of being mindful of what I need to do and not repeating some of my own habits which is leading down that road of burnout. You know what I mean? Group: Yeah. Absolutely. It’s almost like letting go a little…the biggest thing for me I think, having done this for so long, is I have this idea of how things go. And then to hear from other workers different ideas and different approaches, for me to take a step back and say, okay, I don’t have to make those decisions. I can get everybody involved and we can make a decision. I don’t need to make my job harder than it is. So that’s been helpful. (Participant 54) In addition to recognizing the importance of collaboration, some participants also talked about how the intervention had shifted their beliefs about the existence of a collective identity among providers. For some participants the idea of collective identity was related to a new perspective about the commonality of goals and motivations across different agencies: Participant 90: [I gained] the idea that there are common goals between the different programs - even with our different perspectives. Participant 56: That’s a good one. We all want the same thing, what’s best for the family. You know, I think part of it is just like seeing like they’re real people. And I don’t mean that in a derogatory way, but knowing that…you have an interest in what’s going on and how to make things better, and we do too. So it’s like we’re on the same page, we do different jobs but we’re on the same page. (Participant 34) For others, the intervention influenced their beliefs about collective identity by showing them that most providers in the service delivery system share similar work frustrations – regardless of their agency affiliation. This was particularly important for some participants because it gave them more patience when working with outside providers. This shift is captured in the following comments; both comments include a response from the group: 123 Participant 54: I think to give a sense too that everyone experiences the same frustrations Group: mm-hm. Participant 54: It’s not just us. So that was helpful. Participant 40: I just thought it was really interesting from the get-go to know that we’re all kind of on the same boat, same level, you know. We have our different ideas of the agencies, but coming together we’re all very similar with [our] frustrations Group: yeah. Facilitator: More similar than you thought? Participant 40: Yeah, a lot more similar than what I thought. Overall, the focus group data suggested that the intervention shifted at least some of the participants’ beliefs about the benefits of collaboration and the commonalities across providers by the end of the experience. While the repeated measures ANOVA results indicated that participants did not significantly increase their beliefs over the two time points, the focus group comments support the slight positive shift in the intervention group’s mean scores. It is important to note that the final survey measure for beliefs about collaboration used in the quantitative analyses did not include items about collective identity (these items were removed due to low inter-item correlations), beliefs about similarities between providers’ frustrations experienced in their work, or beliefs about the importance of collaboration for preventing burnout. The fact that focus groups participants talked about experiencing shifts in all three of these areas indicates that the final survey measure did not capture some of the shifts in beliefs experienced by the group. These considerations will be elaborated on in the discussion section. Transformative exchanges. Several participants spoke in the focus group discussion about how the intervention had influenced their exchanges with other providers related to shared cases and diffusing collaboration across the system. For example, several participants discussed how the intervention had encouraged them to engage in more “timely communication” in response to providers’ requests for information about shared cases; this shift in behavior occurred as a result of participants gaining a better understanding of how timely exchanges of 124 information – even if just to say there is no new information – can drastically improve external perceptions of their agency. The following comment illustrates how one provider shifted her exchanges with other providers connected to her case as a result of the intervention: Trying to get back with the person within 24 hours, even if it’s just a quick email saying ‘hey I got it, I haven’t had time to address it, but I’ll get with you as soon as I’m free’…. I was trying to do it [before the project], but I’m making a more concerted effort to do it now. (Participant 56) This participant also talked about how she had developed a new information tracking system for her clients that included all the involved providers and family members, and used the system to quickly update contacts when she learned of any changes to the case. It should be noted that the OTE subscale measure related to case-related exchanges did not include items about initiating more timely exchanges as described by focus group participants. Despite this gap, the participants’ focus group comments are at least consistent with their growth in the other caserelated OTE exchange behaviors over time as found in the repeated measures ANOVA. In addition to shifting exchange behaviors related to their cases, two providers also talked about how the intervention had encouraged them to facilitate collaboration in the system more generally by connecting providers with colleagues and correcting false assumptions in their agency: The last couple weeks…I’ve gotten calls on my line here from different agencies who were trying to get through to the main number or were trying to get a hold of a worker and couldn’t. And I’m like, ‘well, our main line is working, but how can I help you? Let me find out who the assigned worker is.’ And anything else I could give them I would give them. So I kind of noticed myself wanting to be a little more helpful and open and saying, “ok, I’m gonna help you with this too.” So that’s really helped. (Participant 40) Since that time, I’ve been able to go back [to my colleagues at CMH] and say, you know [providers at DHS] really can’t do it. And they’re like, ‘really?’And I’m like, ‘no, they can’t.’ And they say ‘wow, I thought they could’. [And I say,] ‘Yeah, me too.’ (Participant 34) 125 While suggesting a positive change for these two participants, the above two comments were the only ones given in the focus group sessions related to diffusion; the isolated nature of these reports supports the repeated measures ANOVA findings that as a whole the intervention participants did not significantly increase their OTEs related to diffusing collaboration across the system over time. This also lends support to the low contamination between control and intervention groups during the course of the study. Overall, the focus group data suggests that the intervention influenced both the frequency and nature of some providers’ transformative exchanges. While the repeated measures ANOVA findings for the overall OTE measure (including both subscales) found no significant growth over time for the intervention group compared to the control group, the comments support the increase in the intervention group’s average OTE rating over time (2.76 in wave 1 and 3.14 in wave 2). As described above, the focus group participants’ comments about increasing the timeliness of their exchanges also supported the trend toward a significant increase in the intervention group’s case-related exchange behaviors over time, despite the fact that the survey measure did not include items about more timely exchanges. The lack of overlap between the focus group comment and survey items will be discussed in more detail in later sections. Facilitating processes. Participants talked about three particular intervention processes that influenced their gains in capacity: 1) participant-led presentations; 2) facilitated discussions that promoted interpersonal engagement; and 3) setting norms and characteristics. First, participants talked about ways in which the participant-led presentations aided their learning experience. For example, participants said that compiling and sharing information about their agencies – especially when everyone could participate in writing it on the white board - was helpful for their learning because it allowed them to “visualize” the information, support each 126 other in the sharing process, get feedback, and ensure that the agency information was complete and accurate. Second, participants said that the facilitated group discussions throughout the intervention improved their learning by giving them opportunities to engage in deep interpersonal dialogue with outside providers – sometimes for the first time in their careers. Participants said the discussions were especially important because they gave them first-hand accounts from actual front-line staff about how things worked in each other’s agencies (as opposed to getting information from an agency representative) which made the information more believable, meaningful, and personal. We have staff meetings here where people from outside agencies come and say what they do. But I think with yours [the intervention] it was a little more…personal level where instead of just having a speech it was seeing first-hand the front-line stuff. (Participant 40) For me it was really beneficial to hear from the actual workers who do the work about what exactly they do, because our perceptions are different. But hearing it from the horse’s mouth so to speak was enlightening for me. It helped me understand, and now I can go back and go like, ‘yeah they don’t do that, you know?’ (Participant 34) It’s always been different people from different agencies [at the meetings]. Getting that perspective first-hand. Not just you [the author] saying, ‘well, CMH is saying that according to this, or the schools are saying’ - no, it’s that person directly. (Participant 54) Third, participants spoke about how the characteristics of the setting itself facilitated their learning. For example, participants talked about how important it was that the setting felt “casual”, “informal”, and “respectful” for them to be able to engage in deep conversations. Each of these elements was included on the list of group norms developed during meeting one, suggesting that the process of co-creating these norms played a role in determining the positive climate of the setting. Participants also described the intervention meetings as a type of alternative setting within the service delivery system where they were able to separate themselves from the stress of their work for two hours and spend some time “just being with 127 other agencies.” For some, this alternative setting was one of the few places they felt comfortable letting their “guard down” with outside providers. The following comments illustrate this point: Participant 77: I’m going to miss it [the intervention] because it was a couple hours during the day when you’re away from clients, away from the phone, and you can discuss things that are going to help you with dealing with them. It’s a little different thing to do for that day. And I’m really going to miss it. Participant 54: yeah, definitely. Participant 77: So you get a star. It’s been very interesting and very informative and I really enjoyed it. It’s more about getting together with these players. So for me, it’s the casual contact where you’re just kind of, you’re not dealing with all the stresses of the moment in the case, just kind of letting the guard down. (Participant 36) In addition, participants’ comments suggested that the prolonged duration of the setting was important because it provided an ongoing space for participants to work through some of the distracting differences they had experienced interacting with outside providers in the past. Many participants had never attempted to discuss distracting differences with outside providers in such an intimate space before, and it took them several meetings to learn how to process their frustrations in more respectful and constructive ways. The following comments illustrate this aspect of the setting: Participant 51: In that first meeting I almost felt personally attacked, and you know it was just the agency name. At least for me I felt on the defensive…I did feel like it started out that way, like a gripe session. Facilitator: Do you feel like people became less on the attack the farther we went? Participant 51: Yeah. I think some people just need to get it out of their systems. And then we could put it aside and look at the bigger goal [of serving youth and families]. Participant 31: We’re good now though. Participant 51: [laughing] Hugs. These comments suggest that future interventions to promote inter-professional collaboration should provide enough time (e.g., meetings) for individuals to work through their frustrations and develop new ways of engaging constructively with other providers. Overall, the focus group comments provided support for the effectiveness of some of the key processes within the 128 intervention. Specifically the comments indicated that participants’ learning was aided by the creation of a safe and casual setting that allowed members to separate themselves from the stress of their work and take time to work through their differences. Follow-up interviews Approximately 4 months after the end of the intervention, brief follow-up interviews were conducted with 16 of the 17 intervention participants to ask them what if anything they were still using from the project at that point in time. The primary purpose of these interviews was to provide an additional source of triangulation to the survey data, as well as to assess evidence for the transfer of the intervention to their everyday jobs. One of the 16 interviews was not included in the final analysis because the participant said she had been forced to take a leave of absence within weeks of the intervention ending due to a family health emergency and was thus unable to comment on her job-related behaviors since the intervention. The remaining 15 interviews were transcribed and content analyzed for key themes using the same deductive approach used in the focus group analysis. Specifically, the analysis focused on how participants were using the capacities they had gained from the intervention to influence their transformative exchanges and collaborative behaviors with other providers. Interview participants described many ways in which they were still using the capacities they gained from the intervention to influence their practice. For example, the majority of participants were still using their knowledge of containers – specifically the resource guide – to assist them in navigating the service delivery system. Roughly half of the participants were using their knowledge about relevant and distracting provider differences to guide their interactions with outside providers, and approximately three-quarters of the participants were using their skills in reflective dialogue in their exchanges. In contrast, very few participants were applying 129 the new beliefs they gained from the intervention in their work. The following sections provide a summary of these themes as well as a description of several of the providers’ job situations changed since the end of the intervention. Changes in providers’ job situations after the intervention. Some participants described several key changes that had affected their job situations in the follow-up interviews, and thus influenced their ability to apply the intervention content to their work. For example, 4 out of the 16 participants interviewed had actually shifted positions since the end of the intervention. While these participants were still engaging with other providers in their work, two of them had begun working in medical clinics and were no longer collaborating with agencies within the system of care. Other participants spoke about shifts in agency morale that had influenced their job satisfaction, and the introduction of new programs to support inter-professional collaboration (e.g., a training around team-problem solving). These changes illustrate the complexity of designing interventions within dynamic systems and will be touched upon in the discussion section. Knowledge of containers. Twelve out of the 15 (80%) participants included in this analysis said they were still actively using their knowledge of relevant containers from the intervention – particularly the resource guide - to help them navigate the service delivery system and initiate contact with relevant stakeholders in outside agencies four months after the intervention. Participants said the resource guide had “opened up a lot of resources” they hadn’t known about before and as a result had better equipped them to “redirect” families to new agencies and services in response to changes in family circumstances. The following comments illustrate well how some providers used this knowledge to better adapt (self organize) to meet their clients’ needs: 130 Since the project…I’m able to respond a little more effectively [to changes in a case] because I know who the service providers are, who can meet he need when a change arises. And if I don’t know I can go to my resource directory and look up, okay who can meet this need? (Participant 56) So being aware of what’s out there, of who can help you, give a name, a number, that made it easier to change with the clients [when their situations changed] because I knew. It wasn’t like boom, there’s a change, what are we going to do! I felt like I had somewhere I could direct them to better help them. (Participant 39) Five intervention participants said they had also shared the final version of the guide with colleagues after the intervention finished, and were assisting them in using the resource. Three participants decided not to use the resource guide after the intervention. For example, two participants thought the guide was a good resource, but had not “transitioned” from using the collection of information they had been using prior to the intervention due to old habit. Another participant said she was not using the guide because it no longer applied to her new position working exclusively with medical clinics; however, while the guide was no longer relevant to her work, she said she was instead using the ideas she learned in the intervention about connecting to relevant stakeholders to help her navigate her new system. Overall, the follow-up interview comments suggest that the intervention successfully built the majority of participants’ knowledge of containers and provide triangulation support for the survey analyses (particularly the trend toward a significant interaction effect in the repeated measures ANOVA and the significant regression results) and the focus group findings. Given the high rates of use after the intervention, the comments also suggest that if participants had been given the knowledge of conditions measure at this later point in time, results may have shown a significant increase in the intervention group’s knowledge of containers compared to the control group (versus just a trend). Knowledge of differences. Eight out of the 15 participants interviewed (53%) said that they were using their knowledge of provider differences to help them in their transformative 131 exchanges 4 months after the intervention. For example, five participants said that gaining an understanding of the relevant provider differences in the intervention - “how people do their jobs,” “what each agency can and cannot do,” where providers are “coming from,” and “agency goals” – was helping them to work more effectively with outside providers. One of these participants said that his new understanding of each agency’s goals had influenced his motivation and ability to align service plans to satisfy the goals of all providers involved. For others, this knowledge influenced the way they gathered information from outside providers sharing cases because it made it easier to “know what to ask about” and what information was available. For one provider in particular, this knowledge fundamentally shifted the way he approached his work, encouraging him to seek out multiple perspectives and resources to address the needs of his cases and avoid a state of burnout: I think being part of the project kind of opened my eyes to look at things a little bit differently, and it happened to be at a time when I was very overwhelmed with my job. Getting the perspective of looking at each case from different viewpoints - other than the one I had been stuck in - provided me with feedback on how to include other agencies, other departments, other resources that I may not have previously tried to encounter. (Participant 90) Seven providers also spoke about gaining a new understanding of potentially distracting provider differences from the intervention – such as different agency policies, requirements, and assumptions – and using this knowledge in their work. For example, for some participants this knowledge helped them to give others “more benefit of the doubt” and be more patient when working with outside providers: More patience with other providers. If they couldn’t give me information for this or that, I had a better understanding of why they couldn’t, and where they were coming from. So I wasn’t automatically like ‘well, they just don’t want to give me the information”. (Participant 39) [The intervention helped me to] understand how people do their jobs that I didn’t know that much about…I think it makes me stop a little bit and not be so quick to judge or become so upset. I stop and will think first before I react to a situation. (Participant 117) 132 One of the participant from the school said that her new knowledge of distracting differences was not only aiding her in her exchanges with outside providers, but was also helping her to confidently promote inter-professional collaboration among her colleagues: I’m aware of the guidelines and procedures from being in the meeting with those specific agencies…now I understand why they operate the way they do, and what they can and cannot do when I’m contacting them… By having that information it makes me feel a little bit more secure in my decision-making because I’m giving them [my colleagues] information that I know is solid information from the agencies. Some of the teachers will say, ‘well, why didn’t they do anything in regards to a specific student?’ And I can tell them, ‘they have to go through this specific procedure in order to report that child to the family.’ Now I can kind of give them the information that I’ve learned from the partnership. (Participant 4) In contrast, 7 out of the 15 providers said that they were not using their new knowledge of provider differences in their jobs 4 months after the intervention. There were three reasons cited for this lack of usage. First, four participants said that since the intervention they only had opportunities to work with a limited pool of outside providers who they had known for a long time; the information about differences was not as relevant in these situations because they said they were already very familiar with these providers. Second, the two participants who switched positions to work in medical clinics said that the knowledge they gained in the intervention was no longer applicable to this “new set of system players.” Finally, one participant said that while at first this knowledge of relevant differences had shifted her outlook, over time this affect went away as the intervention was “out of sight out of mind” and she became distracted by other aspects within her job. This provider said she was particularly influenced by the low morale occurring within her organization since the intervention, and this may have also affected the transfer of this knowledge to her work. Overall, just over half of the 15 participants interviewed four months after the intervention were using their knowledge of relevant and distracting provider differences in their work. While the interview comments provide triangulation support to the survey and focus group 133 findings, the comments also suggest that the transfer of this knowledge to participants’ work was less consistent due to a lack of opportunities for application. The implications of this lack of opportunity will be explored in the discussion section. Reflective dialogue skills. Eleven out of the 15 participants interviewed (73%) said that they had gained reflective dialogue skills through the intervention and were using these new skills in their transformative exchanges with other providers 4 months after the intervention. Participants were using these skills in a variety of ways, such as to ask better questions, tap into other providers’ perspectives, test assumptions, and better articulate themselves. Three participants gave specific examples of how they had utilized materials from the intervention in their practice. For example, one of these participants said that he actually keeps the “communication cheat sheet” that was developed in the intervention at his desk and, to the amusement of his colleagues, references it every time he talks with an outside provider about a case. The following comments illustrate how two other participants used specific strategies from the intervention in their transformative exchanges: I listened to her input and used the cheat sheet that we had. I asked her what do you think, how can we best service these kids. Ever since then it’s been a better working relationship. (Participant 56) Before the project I would just bitch, loudly, everyday…[Now] I can articulate much better what I want to say. I look for other ways to make the change rather than just being adamant about it… I’ve tried to use the simple rules. In calling, I make the connection, this is my take on the case, they tell me their take on the case, and then we go from there…I know this much, what can you fill in that I don’t have here. And that’s helped. (Participant 34) Several other participants said they were using more general reflective dialogue skills from the intervention in their exchanges with outside providers; in a few cases, participants’ knowledge about provider differences and beliefs about common goals had a synergistic effect on their use of these skills. For example, one participant was using the general reflective dialogue skills in 134 combination with her knowledge of provider differences to be more “aware” and “cognizant” of how she presented information to outside providers, and to ask better questions to get access to the information she needed. For others, the reflective dialogue skills helped them to tap into the opinions of other stakeholders (aided by an understanding of their relevant differences) and to check their assumptions as a way to find what is best for the case (aided by beliefs about a common goal among providers) as illustrated in the following comment: I feel like I ask more clarification questions now. I want to know how they think things are going. And in specific how they think things are going - not just in general. I really want their opinion… Not assuming what they’re saying - or how I’m taking what they’re saying - is what they are actually meaning to say… I want to give them my perspective and then check their perception to see if they are seeing the same thing. (Participant 78) Similarly, one participant said that the general reflective dialogue skills she gained from the intervention had influenced her reporting practices, specifically by helping her to gather outside perspectives and including them in the service plans she creates for the youth on her caseload. Asking for an opinion, or what do you think in this situation we should be doing. Many times I haven’t put that in [to the service plans]. But many times now I’m putting that in. (Participant 77) One participant spoke about more subtle changes in her practice as a result of gaining these skills. Specifically, she said her reflective dialogue skills in combination with her knowledge of distracting differences were helping her to “take a step back” when outside providers fail to respond to her requests for information and to courteously make a second phone call - instead of becoming defensive and “firing off a response that might be inappropriate.” In contrast, four participants said that they had not applied skills in reflective dialogue to their work since the intervention. Three reasons were cited for this lack of transfer. First, one participant said that she had not come across a situation that required the skills she learned in the intervention. While it is difficult to verify whether this assessment is accurate, the participant said she was willing to use the skills if a relevant situation presented itself in her work. Second, 135 one participant said that he had used the strategies at first but over time had fallen away from using them because he had become disconnected with his work due to feelings of burnout and low morale. Third, two participants said they hadn’t applied any new reflective dialogue skills from the intervention in their work because they felt they already had these skills prior to the intervention. Overall, well over half of the 15 participants interviewed were using the reflective dialogue skills they had gained through the intervention in their transformative exchanges four months after the intervention. Like the focus group results, this finding is on contrast to the survey analyses showing the intervention group did not significantly increase their skills over time. This contrast is intriguing seeing that most of the participants’ interview comments are closely aligned with the survey questions about reflective dialogue skills. A discussion of the difference in these findings will be described in later sections. Beliefs about collaboration. Only three of the 15 participants interviewed (20%) discussed how the beliefs about collaboration they had gained from the intervention were still influencing their transformative exchanges after the intervention. One of these participants talked about how he was using his new beliefs about collaboration that he had gained from the intervention to take a new, less stressful approach in his work: Before I always thought that it’s on me, mental health is kind of the catalyst for everything else to work. And now I feel there’s some shared responsibility - share the workload if you will…change is something inherent in what we do… I don’t get so worked up and personalize and internalize it as I once did… before I would take it all on myself to solve everything. Now it’s more of a systems collaborative approach. That is certainly something I did get out of the project that was great for me and my own sanity and mental health, you know - that we’re not in this alone. (Participant 21) Two other participants spoke about how their new beliefs about their common “commitment to the population that we serve” had motivated them to continue engaging in collaboration – despite the barriers and frustrations. This sentiment is captured in the following comment: 136 Hearing that everybody pretty much had a common goal … I think it kind of just refreshed us. It was a new perspective. You know, well everybody does have the same ideas or the same goal. And in my opinion I think everyone just had a new perspective on it, and was ready to work again. (Participant 39) The fact that so few participants talked about their beliefs in relation to their collaborative exchanges after the intervention may help explain why there was very limited growth in this variable on the survey. These issues will be further addressed in the discussion section. Overall, the follow-up interviews provided triangulation for the quantitative and qualitative findings and confirmed that at least some participants were using the capacity they gained from the intervention 4 months after the intervention ended. Specifically, the follow up interviews supported the repeated measures findings related to knowledge of containers, knowledge of differences, and transformative exchanges, as well as the focus group data around reflective dialogue skills. The interviews also gave support to the linear regression findings that self-organizing capacity is positively related to transformative exchanges. These findings, along with the results from the other quantitative and qualitative analyses described above, will all be explored further in the discussion section. 137 Discussion The literature suggests that inter-professional collaboration can serve as an important strategy for improving consumer outcomes across a variety of settings (Lemieux-Charles & McGuire, 2006; Suter & Bruns, 2009). However, there remains a gap in the literature regarding the process through which these complex collaborative behaviors develop and function (Butt et al., 2008; D’Amour et al., 2005; Polivka et al., 2001), leaving communities with little guidance as how to facilitate the implementation of inter-professional collaboration. This study sought to address this gap by examining the effectiveness of an intervention designed to increase interprofessional collaboration among front-line service providers in a service delivery system. The results of this study have contributed to the current literature by highlighting strategies for building key self-organizing capacities for promoting collaboration, as well as providing evidence for the process through which these capacities influence providers’ initiation of collaborative exchanges and behaviors. Integrity of IDeAL Partnerships Framework This study examined the relationship between four self-organizing capacities – knowledge of containers, knowledge of differences, skills in reflective dialogue, and beliefs about collaboration – and the initiation of transformative exchanges. It also examined the relationship between transformative exchanges and collaborative behaviors, as well as the mediating role of transformative exchanges between self-organizing capacity and collaborative behaviors. In regards to the first set of relationships, study results suggested that the four selforganizing capacities within the IDeAL Partnerships were positively related to the initiation of transformative exchanges. Specifically, regression results found that each of the four capacity variable significantly predicted the initiation of transformative exchanges even when controlling 138 for group assignment. The focus group and follow-up interview findings also strongly supported relationships between three of the capacity variables (knowledge of containers, knowledge of differences, and reflective dialogue skills) and transformative exchanges and served as a form of qualitative triangulation for the regression results. In contrast, only a few participants spoke about the relationship between beliefs about collaboration and transformative exchanges in the focus groups and follow-up interviews, providing less robust support for the regression findings. These findings are supported by the inter-professional collaboration literature. For example, knowledge of relevant system containers (e.g., network cognition) has been found to help individuals to identify and navigate potentially beneficial exchange relationships (Kilduff, 2006; Krackardt, 1987). There is prior evidence that understanding each others’ differences in roles, competencies, and scope of practice allows stakeholders to initiate productive exchanges (Sargeant, Loney, & Murphy, 2008). Skills in reflective dialogue have been found to encourage stakeholder engagement in learning exchanges including encouraging others to share their perspectives (Wittenbaum et al., 1999), articulating ideas clearly (Basadur, 2004), clarify assumptions (Friedman et al., 2003; Sargeant et al., 2008), and integrating different perspectives (Burke et al., 2007). Research has also suggested that engaging in behaviors such as transformative exchanges requires stakeholders to hold beliefs that inter-professional collaboration is appropriate for their situation (Butt et al., 2008; Lin, 2000; Miller & Ahmad, 2000) and that implementing inter-professional collaboration will lead to benefits (van Dam, 2005; Bartunek et al., 2006). The survey measures and qualitative data included direct references to each of these concepts. In addition to providing strong support for the relationships between the four selforganizing capacities and transformative exchanges, the study also provided evidence that 139 transformative exchanges significantly predicted collaborative behaviors (even when controlling for group assignment) as suggested in the IDeAL Partnerships Framework and supported by Eoyang’s (2001) CDE model of self organizing. However, despite these five predictive relationships, the results of the bootstrapped mediation test did not support the full mediation role of transformative exchanges between capacity and collaboration as hypothesized within the framework. This finding was reinforced by the multiple regression analysis showing that reflective dialogue skills (and to a modest extent knowledge of differences) significantly predicted collaboration even when transformative exchanges were held constant. Thus, results suggest that the relationships within the IDeAL Partnerships Framework may be better captured by a partial mediation role for transformative exchanges instead of a fully mediated model. Findings also indicate the existence of additional direct relationships between self-organizing capacity and collaborative behaviors. Implications for future research in this area will be discussed below. Effectiveness and Viability of the IDeAL Partnerships Intervention Effectiveness at building self-organizing capacity. This study examined the effects of the IDeAL Partnerships Intervention on participants’ self-organizing capacity. The results from the repeated measures ANOVA revealed a strong trend supporting a significant increase over the study duration in intervention participants’ knowledge of containers and knowledge of differences compared to the control group. These findings were triangulated by the focus group and interview comments. In fact, the 4-month follow up interviews suggest that participants actually continued to increase their knowledge of system containers after the intervention finished. This indicates a potential reinforcing feedback loop where the more information participants gained about the system, the better they became at continuing to learn about relevant 140 containers within their work. On average, participants came into the intervention with only “somewhat” of an awareness of system containers and provider differences according to the survey results (approximately 63% and 88% endorsed knowledge scores below the “mostly aware” level for containers and differences, respectively). This lack of awareness was documented in the intervention meeting discussions, focus groups, and interviews as participants commented on their lack of system awareness and misconceptions about each other prior to the intervention. This lack of knowledge is consistent with accounts in the inter-professional collaboration and system of care literature documenting how front-line staff are typically unaware of who is or should be connected with the youth on their caseload (Hodges et al., 1999). Thus, the intervention successfully targeted a critical gap in these providers’ capacity prior to participating in the intervention. On the other hand, the repeated measures ANOVA analyses suggested that the intervention participants did not significantly increase their skills in reflective dialogue or beliefs about collaboration over the study period compared to the control group. In contrast to their initially low levels of knowledge about system containers and provider differences, participants came into the intervention with relatively high levels of reflective dialogue skills and beliefs about collaboration. For example, on average participants came into the intervention at least “mostly” confident in the quality of their reflective dialogue skills (only about 30% endorsed scores below the “mostly” response option). Similarly, upon starting the intervention participants on average held positive beliefs about collaboration at least “quite a bit” according to the survey results (only about 12% of participants endorsed scores below the “mostly” response option). These levels are somewhat higher than normal according to the literature on inter-professional collaboration (Peck et al., 1995; Fraser & Greenhalgh, 2001). One explanation is that some of the 141 providers in the intervention – particularly social workers and therapists – had received preexisting training around dialogue and collaborative practice models which may have inflated their scores in these areas. The participants’ scores may have also been influenced by prior system of care capacity-building efforts in the community. While the repeated measures ANOVA results showed no significant change over time in beliefs about collaboration for the intervention group compared to the control group, the focus group findings suggested that at least some participants shifted their beliefs as a result of the intervention. This disconnect can be explained by the fact that the survey captured beliefs about the appropriateness of collaboration, where the focus group comments described shifts in beliefs related to collective identity, similarities between providers’ work frustrations, and the importance of collaboration for preventing burnout. Thus, it is likely that some shifts in participants’ beliefs may have gone undetected by the repeated measures results. However, in contrast to the focus group comments, very few participants spoke about these beliefs impacting their work 4 months after the intervention. This may be because these beliefs gradually became less salient over time as participants were removed from the intervention setting (the “out of sight out of mind” phenomenon mentioned by some participants), or because the application of new beliefs is less straightforward than applying new knowledge or skills. Future research could examine strategies to assist providers in sustaining their new beliefs over time and in applying these beliefs within their work context. As with beliefs about collaboration, the focus groups and interviews also suggested that some of the intervention participants increased their reflective dialogue skills over the course of the study. This finding is in contrast to the repeated measures ANOVA results showing no significant growth in this area. This disconnect is surprising considering many of the focus group 142 and interview comments closely overlapped with the survey items measuring reflective dialogue skills. One reason for this may have to do with measurement issues within the survey instrument (as will be described below). Another reason may be that participants filled out the wave 2 survey prior to engaging in the focus groups and follow-up interviews. It could be possible that participants became more aware of their reflective dialogue skills as they took part in (and potentially used their skills in) the focus group and interviews conversations. As suggested in the literature, the extent to which participants increased their capacity during the intervention may have also been influenced by their initial views about the necessity of this capacity for their job, and their motivation to gain these skills (Mathieu and Martineau, 1997; Yelon, Sheppard, Sleight, and Ford, 2009). It became clear through the meeting discussions, focus groups, and follow-up interviews that participants definitely came to the intervention with different perceptions of their own initial capacities, and with different ideas about what capacities were required for their specific jobs. For example, some interview participants specifically said that they had not increased their reflective dialogue skills through their participation in the intervention because they already possessed those skills prior to the study. While these assessments may have been accurate, other interview participants made comments suggesting that the individuals claiming to have skills prior to the intervention actually had quite poor reflective dialogue skills in practice. This disconnect resembles what Argyris and Schön (1996) refer to as “espoused theory” and “theory in practice”. Espoused theory refers to individuals’ internalized beliefs and attitudes about how they should behave in practice, while theory in practice refers to the theory individuals actually use in their practice. Not only is there often an inconsistency between these two theories, but individuals are rarely conscious of the disconnect (Argris, 1993). This tension may have been compounded by 143 cognitive dissonance related to the fact that reflective dialogue skills were central to many of the participants’ self-identity of themselves as successful service providers. Thus, these findings support the idea within the literature that in order to increase their capacity participants must first gain the readiness to identify the issues within their true practice and see new skills as necessary and appropriate to address that need (Armanekis et al., 2007). In response, future efforts could be designed to target participants’ readiness prior to engaging in the intervention in order to promote more gains in capacity. In addition to targeting participants’ readiness prior to the intervention, the follow-up interviews also illuminated several transfer of training strategies to consider after the intervention to sustain participants’ capacity over the long term. For example, the follow-up interviews suggested that more participants were successful at transferring their knowledge of system containers than any of the other self-organizing capacities targeted within the intervention. Participants’ ability to transfer this knowledge to their work context may have been supported by the co-creation of the resource guide based on the information shared in the intervention meetings. This guide provided a tangible reference that, according to participants’ comments, was credible, easy to use, and relevant to their work with SED youth. Each of these perceived qualities (credible, accessible, and necessary) have been found to promote successful training transfer according to Yelon et al. (2009). In contrast to knowledge of containers, there was less robust transfer of the other 3 self-organizing capacities to participants’ work after the intervention. For example, 5 out of 15 participants had not transferred their knowledge of differences or reflective dialogue skills to their jobs because they lacked opportunities to use these capacities. This lack of opportunity barrier to transfer has also been cited in the transfer of training literature (Ford, quinines, Sego, & Sorra, 1992). Other participants said that they were 144 unable to transfer their knowledge of differences and reflective dialogue skills because they had switched into positions that no longer required this information, or had fallen away from using their new skills because they were overwhelmed with their work and the training was “out of sight out of mind.” Both of these situations illustrate the challenges of intervening within this public service organization context where providers frequently shift jobs and are under constant levels of high work-related stress. Future intervention efforts could attempt to increase transfer of training by creating tangible reference materials for participants related to the self-organizing capacities, providing follow-up trainings to prevent the material from going “out of sight out of mind”, and addressing system barriers (e.g., high rate of staff shifting job positions, lack of opportunities to use skills) with organizational leadership. Effectiveness at promoting transformative exchanges and collaborative behaviors. This study also investigated the intervention’s effectiveness at increasing participants’ transformative exchanges and collaborative behaviors. Repeated measures ANOVA results showed a trend (p=.09) toward a significant increase in intervention participants’ case-related transformative exchanges compared to the control group. These findings were supported by the qualitative analyses showing intervention participants had started shifting some of their exchange behaviors related to their cases by the end of the intervention. Given the relatively short duration between wave 1 and wave 2 data collection and the small sample size, case-related OTEs (along with the self-organizing capacities) can be considered as a successful intermediate outcome of the intervention. In contrast, there was no evidence for significant growth in intervention participants’ diffusion-related transformative exchanges according to the repeated measures ANOVA results, and only limited evidence in the focus groups that these behaviors had increased (only two 145 participants mentioned engaging in these behaviors). However, the lack of exchanges related to spreading collaborative practice across the system (diffusion) may be due in part to the fact that the participants were still in the process of gaining their capacities and were not ready to begin diffusing them by the second wave of data collection. Overall, both the quantitative and qualitative findings imply that the intervention had somewhat of an effect on participants’ exchange behavior related to discussing their personal cases, while no effect on their behavior related to spreading collaboration across the system. It is interesting to note that, according to the qualitative findings, the intervention may have influenced participants’ transformative exchanges in different ways depending on the individual. The focus group and interview results suggested that the intervention not only affected the frequency of participants’ exchanges (as measured by the survey) but also the actual nature of their exchanges. For example, gaining knowledge about provider differences made some providers more patient and gentle in their exchanges, while it made others more confident and forward in their interactions. Gaining skills in reflective dialogue also had differential effects on participants’ exchange behavior, leading some to use specific strategies to promote dialogue within their exchanges while leading others to shift their behaviors more generally by helping them “take a step back” and become more “cognizant” in their interactions. The different ways in which the intervention influenced participants’ behaviors may be related to the diversity of job experiences and personalities represented across participants. This diversity even extended to participants from within the same organization, as some individuals worked more frequently with certain types of outside agencies or service issues than others. Given this diversity, the construct of transformative exchanges may be better understood and measured along a qualitative continuum that more accurately captures the wide range of unique expressions of the 146 behavior instead of by a frequency count. This finding could inform future studies examining the role of transformative exchanges within service providers’ practice. In contrast to the transformative exchange results, the repeated measure ANOVA findings related to collaborative behaviors were more complex. While the results indicated a significant interaction between time and condition for collaborative behaviors, this effect was driven primarily by a decrease in the control group’s frequency of behaviors. The intervention group’s collaborative behaviors increased only slightly across the two time points according to the survey measure (from 1.87 to 2.06). However, while the quantitative survey results indicated little growth in collaborative behaviors, the qualitative findings suggest that at least some intervention participants started to shift their collaborative behaviors as result of their experience in the intervention. For example, some participants said that the intervention had shifted the way they developed service plans with outside agencies, helping them take more perspectives into account and better address the needs of multiple agencies. Participants linked these behavior shifts back to their improved exchanges with outside providers, due in part from their gains in reflective dialogue skills and knowledge of differences. Other participants talked about behaving more adaptively with other providers as a result of the intervention. Specifically, they said that gaining the capacity to locate relevant providers (e.g., in relevant containers) and initiate exchanges more quickly had equipped them to “respond more effectively” and better “redirect” families in response to changes in family circumstances. These comments align closely with the self-organizing concepts presented in Eoyang’s (2001) CDE model. The study findings overall suggest that some participants increased their transformative exchanges and collaborative behaviors over the course of the intervention. However, both quantitative and qualitative findings also suggest that other participants simply did not increase 147 these behaviors. There may be several reasons for this lack of growth in addition to an ineffective intervention design. First, the second wave of data collection may have occurred too early to detect the changes in participants’ transformative exchanges and collaborative behavior. Given that some intervention participants said they continued to develop their self-organizing capacities after the intervention finished, they also may have continued to increase their transformative exchanges and collaborative behaviors based on the regression findings. Future studies could attempt to capture these developing behavioral shifts by collecting additional waves of data from study participants. Second, participants may have had difficulties increasing their transformative exchanges or collaborative behaviors because the system context in which those behaviors were to be carried may have been unsupportive of the new practice. The influence of contextual barriers – such as obstructive policies, a lack of administrative support, entrenched organizational routines for providing services, or rigid bureaucratic cultures that resist innovation - on the implementation of new behaviors has been well documented in the inter-professional collaboration literature (Walker & Koroloff, 2007; Clark et al., 1996; Malekoff, 2000; McGinty et al., 2001; Glisson, 2002). While the intervention attempted to identify and begin addressing some of these barriers, the three-month time frame and limited scope of the effort prevented the intervention from totally removing these contextual barriers. Thus, the remaining contextual barriers may have influenced participants’ ability to carry out new exchange and collaborative behaviors. Future intervention efforts could attend to this issue by testing alternative methods for addressing contextual barriers before, during, and after the intervention. Finally, some participants may not have increased their transformative exchanges or collaborative behaviors if the particular stakeholders connected to their cases lacked the 148 readiness and capacity to engage with them in these behaviors. As suggested in the organizational learning literature, learning occurs through a socially constructed process where individuals develop their behaviors through their interactions with others in the organizational setting (Lave & Wenger, 1991; Strauss, 1993). Given the importance of these interactions, prior research has also demonstrated that system-wide skepticism toward collaboration can obstruct stakeholders from initiating new forms of behavior (Glisson & James, 2002). If the low to moderate levels of transformative exchanges (roughly a 2.7 on a scale of 0-5) and collaborative behaviors (roughly a 2 on a scale of 0-5) endorsed by study participants prior to the intervention were representative of the system as a whole, it is likely that participants may have encountered some resistance when attempting to implement these new behaviors in their daily jobs. Future interventions could be designed to better equip participants to diffuse collaboration among their colleagues in order to promote this readiness. Future efforts could also attempt to influence the readiness of non-participant stakeholders through system-wide social marketing efforts as suggested in the literature (Hastings & Saren, 2003). Intervention implementation. There are two aspects of the intervention’s implementation process that may have affected its influence on participants’ capacities and behaviors. First, each intervention meeting was for the most part focused on a separate component of the IDeAL Partnerships Framework instead of focusing on all of the components during each session. For example, meetings one and two were focused on participants’ knowledge of containers and differences while meeting three was focused on reflective dialogue skills. While there was certainly some overlap and reference to multiple components across meetings, the meetings were implemented in a fairly structured manner due to time limitations. While this implementation design ensured participants experienced each component of the 149 framework by the end of the intervention, it may have decreased their ability to integrate the four capacities together. Future efforts could test the effects of different implementation designs on participants’ outcomes. Second, the intervention’s implementation allowed participants to deviate from their assigned group meeting and attend a different group’s meeting in order to accommodate to their wildly unpredictable work schedules. While this implementation design definitely increased the training dosage for each participant, it also affected the group of individuals each participant was exposed to in each meeting. This in turn may have affected their intervention experience – such as the degree to which they could develop trust with other participants – and as a result may have influenced their learning outcomes. However, it is likely that the variation in group composition from week to week was more representative of the variable conditions typical within participants’ collaborative interactions and may have actually provided a more realistic intervention experience. This may have had a positive influence on the participants’ learning as the literature suggests the degree to which participants can generalize or apply aspects of a training to their actual job scenario can influence their transfer of that training (Yelon and Ford, 1999). Again, future implementation efforts could test different variations of group composition on participant outcomes. Viability of the intervention within real word settings. According to Bronfenbrenner’s (1979) Ecological Systems Theory, settings and processes are influenced by external factors within the broader environment. There were four main factors within the messy context of the service delivery system that influenced the intervention’s influence on participants’ capacity and behaviors. First, the intervention’s influence was impacted by the intense workloads typically assigned to providers in the system. These intense workloads not only impacted who could 150 participate in the intervention (several providers declined to participate or dropped out of the study due to feeling overwhelmed by their work), but according to the qualitative findings it also influenced the energy and motivation participants had to apply the intervention content to their work between intervention meetings and after the intervention finished. While participants commented that the intervention setting itself served as a form of refuge from their stressful work days and was important for their learning process, their attempts to implement their new capacities and behaviors were definitely affected by the intense nature of their work. Second, the intervention’s influence on participants’ outcomes was affected by the issue of participants’ variable work schedules. This had the most impact on the scheduling of intervention meetings. For example, while school social workers were only able to meet in the middle of the day due to the school day schedule, probation officers often found this time difficult to attend due to the scheduling of court hearings. In addition, the extreme fluctuations – often last minute and unanticipated – in participants’ schedules made it difficult for them to attend all six of the intervention meetings. While efforts were made to respond to this variability (the flexible attendance policy described above, conducting one-on-one following up meetings with participants who were unable to attend a meeting) it was an ongoing challenge that influenced the effects of the intervention. Third, the intervention’s influence on participants was impacted by the variability of participants’ cases. It became clear through the meeting discussions and other qualitative methods that participants encountered very different types of cases in their work. For example, while some participants dealt a large range of outside providers in their cases, others dealt with a very small and consistent pool of external agencies. Similarly, while some of the participants’ cases involved youth who were experiencing incredible amounts stress and behavioral issues, 151 other cases involved children who were experiencing less severe risk factors. This variability was seen both across agencies and colleagues, but also within each provider’s own practice. For example several participants said that it was not uncommon for the nature of their cases to shift drastically over time, sometimes even within a 6 month period. As a result of this variability, there were not always uniform or consistent opportunities for participants’ to apply what they had learned in the intervention to their immediate work environment. This was seen clearly in the follow-up interviews when some participants said they had not transferred their knowledge of differences or reflective dialogue skills because their current cases did not require these capacities. This is a key limitation to the viability of the intervention within this context. Finally, the viability of the intervention in the service delivery system was affected by the agency practice of frequently reassigning staff to new job positions, sometimes without much advance warning or preparation. On one hand, this practice affected the retention of participants within the intervention (two participants dropped from the intervention group because they had been reassigned to new departments that no longer dealt with direct service provision). On the other hand, this practice also influenced the sustainability of the intervention’s effects. For example, 4 out of the 16 participants in the follow-up interviews had shifted positions since the end of the intervention and had begun working with new types of outside providers (e.g., doctors). As a result, these participants said they had struggled to apply some of the intervention content to their new contexts. This suggests that the capacities targeted within the intervention may be specific to particular range of settings and interactions instead of a generalizable set of capacities and behaviors. Future research could explore ways to increase the generalizability of the intervention content to attend to this system context. Study Limitations 152 Interpretations of the study findings must take into consideration several limitations. First, the study included a small sample size which affected the power within the quantitative analyses. Despite multiple efforts to increase and maintain the sample size in the study, only 33 participants were included in final sample. The drop in participation was in part influenced by a myriad of factors within the service delivery system, such as high workloads and frequent job reassignments. In response to the small sample size, analyses adopted a one-tailed significance test and were triangulated by two forms of qualitative data (focus groups and follow-up interviews). Second, there were several limitations to the primary survey measures used within the study. For example, some of the measures did not include elements brought up in the focus group and follow-up interviews which may have influenced the sensitivity of the survey to detect changes in participants’ capacity and behaviors. This disconnect occurred because the survey instrument was developed prior to when the qualitative data was collected, and also because some of the survey items were not included in the final scales due to low intercorrelations and factor loadings. While these issues are natural consequences of using quantitative measures and adopting a positive epistemological orientation, they suggest the need for additional measure development. In addition, some of the measures used in the study were limited by the relatively low degree of variability in participants’ item responses, and by the existence of ceiling effects (particularly in the beliefs about collaboration measure). This may have influenced the survey’s ability to distinguish between participants in terms of their capacity and behavior levels. Third, findings were limited by the fact that the survey measures related to knowledge of containers, knowledge of differences, transformative exchanges, and collaborative behaviors were not correlated with the validation measures based on the structured interview conducted 153 with a subset of participants before and after the intervention. However, this lack of correlation may have been influenced by several issues within the validation measures themselves. For example, the validation measure related to knowledge of differences was based on questions about whether participants were aware of any information, connections, or service ideas outside providers brought to the exchange. In addition to their actual knowledge of these factors, participants’ ability to answer this question may have been influenced by whether or not the outside providers they were working with possessed these assets or were able to activate them in their exchanges with participants. Similarly, participants’ answers to the validation measures related to knowledge of containers, transformative exchanges, and collaborative behaviors may have also been influenced by the characteristics of the particular case they had selected to review for the structured interview. As mentioned above, qualitative findings revealed that participants typically dealt with a diverse range of cases in their work, often requiring very different types of actions and services. For example, some of the participants’ cases involved several outside agencies and required them to initiate contact and interact frequently with providers in relevant containers, where other cases involved only one outside agency that was not heavily involved in the service process. It may have been that while participants’ survey scores accurately represented their knowledge and behaviors on average, these qualities may not have been activated consistently across all of their cases. Because the structured interviews only focused on one case due to time limitations, the validity measures may have provided an inaccurate summary of participants’ overall case-related behaviors. Thus, while the survey measures were not validated by the structured validity interviews, this lack of support should be considered in light of the limitations within the validity measures themselves. 154 Fourth, there were limitations to the study’s research design in terms of preventing any contamination of the control group by intervention participants. While efforts were taken to diminish this contamination, for example by preventing the dissemination of the resource guide until after the last intervention meeting, it is inevitable that some contamination could have occurred due to the close interactions of staff within these agencies. However, the repeated measures ANOVA results for the diffusion-related OTE scale suggest that participants’ exchange of information related to promoting collaboration occurred infrequently (roughly quarterly according to results) both before and after the intervention. While this suggests that the intervention was not successful at shifting this behavior, it does provide support to the conclusion that the contamination between groups was likely low. Yet even if participants didn’t intentionally share information from the intervention with colleagues, their modeling of new behaviors related to exchange and collaboration could have affected the diffusion of new practices to their peers in the control group. The risk of contamination is a natural consequence of conducting research in real world settings where it is impossible to control all aspects of participants’ interactions. Future Research Directions The findings from this study suggest several future directions for research related to both the IDeAL Partnerships Framework and IDeAL Partnerships Intervention. First, future research could explore additional relationships between the variables in the IDeAL Partnerships Framework. For example, the focus group and interview data suggest that in some cases the selforganizing capacities may actually build on each other to create a synergy, such as when participants talked about blending their knowledge of differences with their reflective dialogue skills to better promote transformative exchanges. Future research could begin to capture these 155 combined effects and explore the interactions between the constructs over time using multivariate autoregressive models. The study also suggested that some of the self-organizing capacities were directly related to collaborative behaviors – as opposed to being fully mediated by transformative exchanges. Future research could assess these direct relationships in larger service provider samples in order to inform intervention efforts related to building selforganizing capacity. In addition, future research could explore whether a non-linear relationships exists between transformative exchanges and collaborative behaviors, specifically whether the relationship between the two variables becomes stronger over time as participants increase the frequency of their transformative exchanges. Second, future research could continue to refine the quantitative survey measures used within the study. For example, the qualitative data suggested that the constructs included in the IDeAL Partnerships Framework are multidimensional. This was captured in the intervention group’s meeting discussions (and documented in the 36 page resource guide) as well as participants’ focus group and interview comments. While it was beyond the scope of current study to capture the full range of these multidimensional constructs within the survey, future research could explore additional dimensions to include within survey measures to better capture how the constructs play out in the everyday work of service providers. Third, future research could examine more fully how the relationships within the framework may play out differently for different individuals. The focus group and follow-up interviews indicated that many participants experienced the intervention in slightly different ways which in turn uniquely influenced how the intervention affected their capacity and behaviors. Future research could examine how the constructs defined within the IDeAL Partnerships Framework manifest themselves differently depending on individuals’ job-related 156 tasks, social contexts, stage of development, and personal characteristics. This inquiry would be especially tailored to the use of qualitative research methods. Finally, future research could continue to explore questions related to the effects of the IDeAL Partnerships Intervention. For example, future studies could follow participants of the intervention over longer periods of time to assess whether they continue to develop their selforganizing capacities, OTEs, and collaborative behaviors after the end of the intervention. The follow-up interviews suggested that this phenomenon may have occurred with participants as they continued to develop their knowledge of containers after the intervention was finished. In addition, future research could explore questions related to sustaining the impact of the intervention over time, particularly in contexts such as service delivery systems. This research could draw upon literature on sustainability and transfer of training to explore strategies to support participants in successfully applying and adapting the self-organizing capacities and collaborative behaviors within their real world work environments. 157 Conclusion There is an urgent need to improve the life outcomes for millions of children with SED in the United States (Power, 2009). Inter-professional collaboration could play a key role in promoting these outcomes by helping front-line service providers effectively adapt services to better meet the shifting needs of youth with SED and their families. While research suggests that inter-professional collaboration is associated with positive outcomes for SED youth, front-line service providers typically lack the capacity to engage in these collaborative processes (Fraser & Greenhalgh, 2001; Walker & Shutte, 2005). Despite the millions of federal dollars that have been invested into systems of care efforts, little research is available to guide communities in how to build these capacities to facilitate the process of inter-professional collaboration in their own systems. This study introduced the IDeAL Partnerships Framework as a means of addressing this gap. The framework defines how inter-professional collaboration comes about through the development of four self-organizing capacities (knowledge of the service delivery system, knowledge of provider differences, reflective communication skills, and beliefs about collaboration) which foster the initiation of transformative learning exchanges. The study provides preliminary evidence for some of the key relationships within the framework. Specifically, service providers with high levels of self-organizing capacity were more likely to initiate transformative exchanges which in turn lead to higher initiations of collaborative behaviors. While these core relationships were supported by the findings, the study found that transformative exchanges did not fully mediate the relationship between self-organizing capacity and collaborative behaviors as suggested by the framework. Instead, findings suggested the existence of additional direct relationships between some of the self-organizing capacity 158 variables and collaborative behaviors. Taken together, these findings offer initial guidance on potential areas for service delivery systems to target through capacity-building interventions. The study also conducted an initial experimental assessment of the effectiveness of the IDeAL Partnerships Intervention. The intervention was based on the key relationships within the IDeAL Partnerships Framework and was carried out over the course of three months with frontline service providers from a system of care effort. Quantitative findings revealed a strong trend suggesting that intervention participants increased their knowledge of the service delivery system and their knowledge of the differences between service providers over the course of the study compared to a control group of service provider peers. Qualitative findings confirmed these shifts and also suggested that participants experienced gains in reflective dialogue skills and shifted their beliefs about collaboration as a result of the intervention. Follow-up interviews indicated that many intervention participants were still using their self-organizing capacities 4 months after the intervention ended. Quantitative results also showed a trend suggesting that intervention participants increased their case-related transformative learning exchanges over the three month study period compared to control group participants, a finding that again was confirmed by qualitative results. In contrast, intervention participants did not increase their collaborative behaviors over the course of the intervention but instead held them at roughly a steady rate while the control group participants decreased their collaborative behaviors over time. It is possible, however, that due to the short duration of the study the findings did not capture shifts in intervention participants’ capacity and behaviors occurring after the intervention finished. Future research could follow participants for longer periods of time in order to capture these potential long-term changes. 159 In summary, the primary goal of a system of care effort is to improve outcomes for youth with SED and their families. Self-organizing capacity provides a promising mechanism for communities to activate self-organizing, inter-professional collaboration as a means of achieving this goal. The findings of this study – regarding both the IDeAL Partnerships Framework and Intervention - can serve as a guide for communities as they attempt to promote these capacities and collaborative behaviors among front-line service providers. 160 APPENDICES 161 Appendix A: Intervention Meeting Agendas Meeting 1 Agenda I. Introductions/ice breaker (10 minutes) Have everyone introduce themselves and briefly talk about a recent win in their work with SED youth and families. (5 minutes) Thumb Wrestling Exercise Purpose:  To expose and explore our implicit assumptions about competition and collaboration. Process:  Have participants pair up. Ask everyone if they have ever thumb wrestled before. Demonstrate to those who are unfamiliar.  Say that the goal of this activity is to collect as many points as you can in one minute. To get a point, one partner pins the thumb of the other partner. Offer a bag of candy to the first and second placing teams.  Have participants tap their thumbs three times and then begin.  After one minute stop the game. Debriefing Questions:  Ask each team how many points they’ve gained. If you have a pair that scored extremely high – ask them how they did it (will probably be through collaboration)  What are our typical assumptions about thumb wrestling? (one person wins)  How do these assumptions relate to collaborating around the needs of SED youth?  What was an unintended consequence of competition in this game? (both lose)  How can we shift our focus to see these types of interdependencies in our work – in other words to see opportunities for collaboration? II. Establishing Group Norms & Jargon Sheet (10 minutes) Example Norms  Begin on time – end on time  Practice being open-minded  Monitor your own airtime  Listen first  Probe ideas, don’t criticize people  Show respect for views of others  Avoid side conversations 162  Challenge by Choice: people can participate at their own comfort level. Silence or passing is always acceptable. No one should feel pressed to disclose more than they feel comfortable. Introduce the jargon sheet, a large piece of paper that will be taped to the wall during each meeting. It will be a space to write up any jargon brought up in meetings that is unfamiliar to participants. III. Overview of the project (15 minutes) Briefly introduce concept of a system of care. Talk about why research and leaders see collaboration as important. Get their reactions. Talk about idea of youth as a small system, and how the system must adapt to the dynamic complexity within this system. Talk about what capacities the research says can support this type of collaboration. How project is structured to build these things IV. Simple Rules (15 minutes) People are driven by an internalized belief system or set of “simple rules”. Often these are unconscious. This can be seen in many examples. Simple Rules for Flock of birds: 1) match your speed to others in flock 2) avoid running into others 3) fly toward the center Simple Rules for Highway of Traffic: 1) match speed to others 2) stay in my lane 3) leave enough space between cars It is often useful to reflect on the current simple rules driving your behavior, what patterns are coming out of our current simple rules, and think about which rules might lead to more effective patterns for addressing the needs of SED youth and families. What are current simple rules? What behavior patterns result from those rules? To what extent are these patterns the most effective for addressing the needs of SED youth and their families? Suggested Simple Rules for Addressing Needs of SED Youth 1) connect with critical partners 2) integrate relevant differences 3) adapt to change. 163 I will write the simple rules on a large piece that will be taped to the wall at each subsequent meeting. V. Activity (40 minutes) This activity will allow providers to reflect upon their previous experiences collaborating with other providers, as well as how these interactions can lead to better outcomes for youth and families. The activity will involve two cycles related to two separate questions. Each cycle will allow group members to brainstorm ideas on their own, theme them into categories, and discuss the resulting data. Question 1: what are some words you would use to describe your past experiences collaborating with providers from outside your agency?  Group members write their words on index cards (5 minutes)  Group members use tape to group their cards into categories on the wall (5 minutes)  The group will engage in a discussion about the themed data using the following discussion questions (10 minutes): o As you think back, which of these experiences led to positive outcomes for a youth or family – why? o Which experiences did not lead to more positive outcomes – why? o What types of things determine whether a collaborative experience leads to positive outcomes? o Overall, what are some of the benefits to collaborating? What are some of the risks or challenges? Question 2: what needs to happen in order for providers to collaborate in ways that lead to positive outcome for youth and families?  Group members write their words on index cards (5 minutes)  Group members use tape to group their cards into categories on the wall (5 minutes)  The group will engage in a discussion about the themed data using the following discussion questions (10 minutes): o What are some of the overall messages we see across these ideas? o How might these needs be addressed? Note that they have just learned skills related to integrating perspectives. VI. Wrap-up and Next Steps (15 minutes) Introduce weekly reflection forms Introduction of presentation homework: participants from the same sector will be given the assignment of creating a brief 10-minute presentation on their organization based on a provided template (see Appendix B). 164 Meeting 2 Agenda I. Welcome and Ice breaker (5 minutes) Pen perspective exercise (5 minutes; taken from Systems Playbook)  Purpose: to get people thinking about how their perspective depends on where they “sit”. How our perspective colors how we see the system and events, and how different people can have different perspectives of the same case, event, etc – both of which can be useful for understanding and problem-solving.  Process: o Ask everyone pick up a pen and hold it straight up in the air. o Have everyone pretend to draw a circle with the pen on the ceiling in a clock-wise direction. o Ask them to continue drawing the circle clock-wise, slowly bringing it down a few inches at a time until it is in front of their faces, and then continue down until they are looking at the top of it. o Ask the group what direction the pen is moving (it will be counterclockwise).  Debrief Questions: o So what happened? o What was your initial reaction - what are the first thoughts that came to mind? o Do your immediate reactions provide any insights into your own process of forming assumptions? o If we imagine the pen to be a case involving a youth with SED, what might the consequences be if we based all of our solutions on only the initial view? o What does this tell us about seeking out alternative perspectives when looking at the complex circumstances of SED youth? II. Agency Presentations (70 minutes) Introduction to presentations: Each agency team will give their presentation following the presentation template. I will write key points from these presentations on large pieces of paper that will be taped to the wall. I will interject two questions half-way through as shown in italics. 1.What is the range of services that staff at your agency provides for SED youth and their families at your agency? What are the eligibility criteria for these different services? 2.Describe the different staff members who are involved with SED youth and their families as they receive services at your agency – at each stage from access to discharge. a. Are there any specific procedures (e.g., intake, IEP, behavioral safety planning, etc) that these staff are involved with during this process? 3.Who at your agency has access to information that could be useful for: 165 a. coordinating services for specific cases involving SED youth and their families? b. helping SED youth and their families access services or community resources? Who else within your agency might be relevant for outside providers to collaborate with related to their cases? How can outside providers identify and connect with staff in your agency who might be relevant to their cases? 4. What skills, perspectives, or expertise do providers in your agency typically have that could be useful in problem-solving for SED youth and their families? Are there any other types of differences that might be important to utilize in this problemsolving? III. Summary Discussion Questions (35 minutes) 1. How can you figure out which stakeholders might be important to engage – at different points in time - related to your work with your SED cases? 2. Any ideas for how to identify the relevant skills, perspectives, or expertise of new colleagues? 3. How might you utilize these different skills, perspectives, and expertise across providers to create better and more adaptive solutions for SED youth and their families? a. Any specific ideas on how to integrate these differences for the purpose of addressing the needs of SED youth? 4. What community stakeholders outside all of your agencies might be relevant for providers to collaborate with related to their cases (e.g., faith-based communities, non-profit community organizations, natural supports)? a. What characteristics might make them relevant? b. How can providers identify and connect with these community stakeholders? 5. How can help colleagues outside of our group learn about these things – specifically how to identify and connect with critical system partners and how to integrate relevant differences to arrive at better solutions? IV. Summary Discussion Questions (5 minutes) Who would like to share any summarizing thoughts or conclusions about what we’ve discussed today. Has today’s conversation shifted the way you approach the simple rules we talked about at our last meeting? V. Wrap-up and Next Steps 166 Meeting 3 Agenda I. Welcome and Icebreaker (5 minutes) Folded Arms Exercise  Purpose: get participants to think about how recognizing and altering habitual ways of thinking can lead to great insights and learning – but is often uncomfortable and awkward feeling. Get them to reflect on the self-imposed challenges to changing the way they think. Give them a physical analogy of stepping out of their mental ruts and comfort zone.  Process: o Have everyone fold their arms the way they would if they were bored, with one arm naturally falling on top of the other. Have them look at their arms and notice which one is on top. Have them reflect on how this feels – is it comfortable, does it feel normal? o Then have them uncross their arms and fold them again the other way, with the other arm on top. Ask: how does that feel? What do you notice?  Debrief Questions: o How does our need to be comfortable and secure and avoid feeling awkward potentially get in the way of our learning? II. Balancing Inquiry and Advocacy (30 minutes; Sense, 2006) Last meeting we talked about some benefits to integrating differences across providers in the work with SED youth. I’m going to show you some communication strategies to help utilize these differences. We are all good at advocating for our own position. We do this on our jobs and in our personal life. Problem is, in situations where we need to tap into the insights of other – when we need to learn from others – we often don’t know how to balance advocating for our points while genuinely inquiring into the points of others. I’m sure you’ve experienced this in decision-making situations. It’s like an arms race where each side continues to advocate their point more and more strongly. Imagine a therapist and a social worker who are stuck in this cycle as they try to come up with a plan for a youth. The more a therapist argues her point, the more the social worker feels threatened. The social worker counters by arguing his point even more fiercely. Neither asks each other questions, rather they just continue arguing their points. What usually happens is both sides become so exacerbated that they abandon the partnership because it’s just too much trouble. What’s the remedy? Learning is more likely when everyone can make his or her thinking clear and open for public examination. It creates an environment of “genuine vulnerability”. No one advocates a view without making the reasoning behind that view open for public scrutiny. 167 When partners are just advocating, the goal is to win the argument. When partners balance advocacy with inquiry, the goal is to find the best argument. The point isn’t to convince others – it is to work together to objectively evaluate the potential value or weaknesses of everyone’s reasoning. This involves being transparent and exposing all of the underlying reasoning behind a point – not just those parts that make our case, but also those areas where we think our points might be weak. Some things to consider:  Only ask genuine questions – if you are not really interested in the answer, you will only make things worse.  Keep an open mind – listen first, then evaluate.  Collaboration is about finding effective solutions – not winning an argument.  When you don’t agree with someone – instead of arguing against their point, inquire into it.  You have to be willing to expose the limitations of your own thinking – be willing to be wrong. Only this will make it safe for others to do the same.  It may help to emphasize that your partners’ input is desired and needed to come up with effective solutions. Sample Questions:  Positive Models: o Here is how I arrived at my view (state reasoning and information view is based on) –  Do you see any gaps in my reasoning?  Do you have any different information that might inform this conclusion?  Do you make any different conclusions from this information o What are your views?  What information did you use to you arrive at your view?  Are you using information to inform your perspective that is different than what I have considered?  Here are some of the assumptions I’m making based on what you’ve told me – are they accurate? o What additional information might we need to inform our perspectives?  Negative Models: o I appreciate your point, but let me tell you why your idea won’t work. o This is just the way things are (as opposed to this is the way I see things) o You wouldn’t understand. Other suggestions for positive or negative examples of probing questions: 168 Practice 10 minutes  Have participants break up into groups of 2 or three and discuss how to address a hypothetical problem related to a case involving a SED youth. Have them practice using these types of questions in their conversations. Group Discussion Questions (10 minutes)  What are your reactions to these strategies?  Do you think this would work in a real world conversation? If yes, how and when could you see yourself using these strategies?  How can providers establish norms that promote this type of non-critical dialogue with collaborating partners? (refer to norms we established in first meeting)  What are some obstacles or challenges to using these strategies? How can these be managed? III. Common purpose (10 minutes) We’ve talked a lot about the importance of integrating different viewpoints to arrive at better solutions. Some providers in other communities have found it helpful to come up with a shared vision or common purpose to unify their work. For example, in a service delivery system around substance abuse, providers choose the common purpose of “recovery” to help unify their separate efforts. Another system working with survivors of domestic violence chose the purpose of “empowerment” to guide their efforts. What are some words to describe our common purpose serving SED youth? e.g., positive development, growth, We can draw upon this common purpose throughout the rest of today’s meeting and as we continue meeting. IV. Discussion of Potentially Distracting Differences (20 minutes) While the relevant differences we talked about during our last meeting are the most useful in terms of serving SED youth and their families, let’s be honest – these aren’t the only differences that we have between us. There are many other types of differences, some of which can distract from our common purpose What are some of these differences? Differences in service approach or philosophy they take in addressing the needs of youth and families. 169 Differences in the limitations (legal, financial, policy, service capacity) affecting how providers within your agency are able to deliver services. Differences in agency procedures for delivering services to youth and families. How have these differences influenced your past interactions with stakeholders from outside agencies? Key is not to ignore these distracting differences – but find ways to work around them so we can put more focus on the differences that can make a difference. Any strategies people have used to work around or through these differences to open up space to focus on more relevant differences? How could you draw upon the common purpose we talked about to help partners focus on larger goals instead of distracting differences? V. Left-Hand Column Exercise (35 minutes) Purpose:  Have participants reflect upon how their hidden assumptions can drive their communications; practice questioning hidden assumptions. Process  5 introduction  8-10 minutes individual reflection  8-10 minutes pair discussion  10 minutes group discussion Here is an exercise to help reflect on some of the personal biases that may make it hard to focus on more relevant differences. This is an exercise that was created by an action researcher named Chris Argyris and has been widely used by many organizations. Pick a specific situation where you are interacting with another providers in a way that isn’t working – that isn’t producing any learning and isn’t moving ahead. Take a piece of paper and divide it into two columns. On the right side write “what is said” and list out the dialogue between the two of you. On the left side write “what I am thinking” and list out everything you are thinking during this conversation – but not saying. This left hand column is a good way to bring out some of your hidden assumptions that influence your behavior. I’ll give you an example of a hypothetical conversation between two colleagues from different agencies in the system of care regarding a case. 170 What Provider A is Thinking, but not Saying What is Actually Said I noticed you never returned my phone call about Jimmy’s case. You really left me hanging on this one. I can’t believe how inconsiderate you can be. Provider A: How is work going, pretty busy lately? I wonder if you have even noticed how bad Jimmy’s situation has gotten over the last month. Provider A: So how do you think we should respond to Jimmy’s situation? It’s getting pretty intense for him and his family lately. Provider B: Ah, as busy as usual I guess. Provider B: I’ve been talking with some people from the mentoring program at Jimmy’s school. I think that might be a really good source of support for him right now. Are you trying to go behind my back in this? I thought we were collaborating on this case? I’m not sure I trust you to get this information since you can’t even get your act together to call me back. Provider A: Well, I guess we will have to see. I don’t know anything about that mentoring program, and I wouldn’t want to make things worse for Jimmy by connecting him with an ineffective organization. Pair Discussion Questions  Why didn't I say what was in the left-hand column? What prevented me from acting differently?  What assumptions was I making about the person? What are my beliefs about them/the situation?  How might these assumptions be incorrect? What else may be contributing to provider B’s behavior?  How might my comments have contributed to the difficulties/outcomes? Group Discussion:  What are some costs of hiding behind your assumptions?  How can I use the left-hand column insights to improve this situation and/or my communication?  How can all of us work on speaking openly & bringing up conflict directly? VI. Wrap-up and Next Steps 171 Meeting 4 Agenda VII. Welcome and Recap from last meeting (5 minutes) Frames Purpose:  Reflect upon how their perspective depends on where they “sit”. How our perspective colors how we see the system and events, importance of expanding our perspective to aid problem-solving. Process:  Have participants create a small aperture with their thumb and finger. Have everyone hold their viewing hole at arm’s length and focus on a pen I am holding.  Ask the following questions, allowing 10 seconds after each for participants to ponder their answers: o What do you see within this frame? o What questions could you answer with the information available to you through your frame? o What actions could you take to influence the objects or the processes that you see?  Have participants bring their viewing hole halfway toward their eye and repeat this process.  Have participants bring their viewing hole as close as they can to their eye while keeping the object centered in the whole. Debriefing Questions:  We often fail to consider what frame we are using and skip straight to problem solving. How do you usually frame problems related to your work serving SED youth?  What is the link between the questions you ask and the framing you choose?  What framing best captures the problem of collaboration? VIII. Sharing of Recent Attempts to Initiate OTEs Participants will collectively discuss their successful and unsuccessful attempts to initiate collaborative learning exchanges related to improving outcomes for youth and families with colleagues during the previous month. Specific discussion/probing questions:   Have you been able to identify or connect with any stakeholders who are or could be relevant to your work? Has anyone attempted to share information, brainstorm, or problem solve with other providers around your work with SED youth? o Have you been able to integrate your relevant differences in these situations to help problem solve around the needs of SED youth and their families? 172     IX. Has anyone tried using the strategies we discussed at our last meeting – the left-hand-column or balancing inquiry and advocacy? How did it go? Have you come across any barriers (e.g., lack of capacity, agency policies) as you’ve tried to collaborate with stakeholders during the last month? o Can you describe these barriers in detail? For example, how do they occur, when do they occur, do they always occur, etc. o What might be causing these barriers? For example, what conditions within the service delivery system might be contributing to these barriers? What gets in the way of you overcoming these barriers? Do you perceive the source of these barriers differently than others you work with? o What are the consequences of these barriers? o Using this understanding, are there any ideas for potential solutions to address these barriers? o Any ideas to support the enactment of these solutions? What have been your experiences using the simple rules?** Has anyone tried to share any of the learning from our group with your colleagues? o What were their reactions to this sharing? o Did you encounter any barriers or challenges to sharing? o Any ideas for overcoming those barriers? o What are some additional ideas to help promote learning among your colleagues? Goal setting and behavioral intentions (30 minutes) Selecting Goals Has anyone ever set goals before? Goals have been used in many different contexts – from athletes setting goals for their performance to kids setting goals for their academics. Setting goals is a popular process in many organizations because like the exercise – goals influence action. Goals help keep ideas salient in your mind instead of getting lost in the chaos of daily work. Relate to simple rules ideas. Research has found that goals are especially effective when they are specific and challenging. What are some specific and challenging goals related to collaboration in your work? But just setting goals alone isn’t always enough. Many people fail to meet their goals because they run into obstacles along the way. 173 People have studied ways to help people meet their goals and have found one good way is to problem-solve around obstacles before they happen. Based on our conversation today, what are some of the obstacles you anticipate possibly encountering as you attempt to collaborate or share information with colleagues? What are some ideas for how to overcome those obstacles? Specifically, what will you do to overcome these obstacles? How will you ensure that you follow-through with these intentions? X. Wrap-up and Next Steps Meeting 5 Agenda I. Welcome and Recap from last meeting (5 minutes) II. Sharing of Recent Attempts to Initiate OTEs Participants will collectively discuss their successful and unsuccessful attempts to initiate collaborative learning exchanges related to improving outcomes for youth and families with colleagues during the previous month. Specific discussion/probing questions:      What are your experiences with the goals we set at last month’s meeting? Have you been able to identify or connect with any stakeholders who are or could be relevant to your work? Has anyone attempted to share information, brainstorm, or problem solve with other providers around your work with SED youth? o Have you been able to integrate your relevant differences in these situations to help problem solve around the needs of SED youth and their families? Has anyone tried using the strategies we discussed at our last meeting – the left-hand-column or balancing inquiry and advocacy? How did it go? Have you come across any new or additional barriers (e.g., lack of capacity, agency policies) as you’ve tried to collaborate with stakeholders during the last month? o Can you describe these barriers in detail? For example, how do they occur, when do they occur, do they always occur, etc. o What might be causing these barriers? For example, what conditions within the service delivery system might be contributing to these 174    XI. barriers? What gets in the way of you overcoming these barriers? Do you perceive the source of these barriers differently than others you work with? o What are the consequences of these barriers? o Using this understanding, are there any ideas for potential solutions to address these barriers? o Any ideas to support the enactment of these solutions? Anyone try to enact the solutions we proposed from the last meeting? o How did it go? o Do you need additional ideas for moving forward? What have been your experiences using the simple rules?** Has anyone tried to share any of the learning from our group with your colleagues? o What were their reactions to this sharing? o Did you encounter any barriers or challenges to sharing? o Any ideas for overcoming those barriers? o What are some additional ideas to help promote learning among your colleagues? Wrap-up and Next Steps Next month is our last meeting. I want you to start thinking about some ideas for how to continue this type of learning we’ve done so far after next month’s meeting. We will have a discussion on this next time. Meeting 6 Agenda I. Welcome and Recap from last meeting (5 minutes) III. Touching base about actions over previous month (50 minutes)   How did it go during the last month – anyone have any collaborating successes? Have you come across any barriers (e.g., lack of capacity, agency policies) as you’ve tried to collaborate with stakeholders during the last month? o Can you describe these barriers in detail? For example, how do they occur, when do they occur, do they always occur, etc. o What might be causing these barriers? For example, what conditions within the service delivery system might be contributing to these barriers? What gets in the way of you overcoming these barriers? Do you perceive the source of these barriers differently than others you work with? o What are the consequences of these barriers? o Using this understanding, are there any ideas for potential solutions to address these barriers? o Any ideas to support the enactment of these solutions? 175 IV. Fill out survey (20 minutes) V. Focus Group (30 minutes) I will facilitate a focus group with each of the intervention groups that will allow them to reflect on the intervention’s influence on their self-organizing capacity and/or their initiation of transformative exchanges related to improving outcomes for youth and families. VI. Final Debrief and Wrap-up (30 minutes) What are some ideas for how to formally or informally continue learning after today’s meeting? [I will have already spoken with the system of care leaders about options for continuing support for learning opportunities. This conversation will build on this information.] 176 Appendix B: Presentation Template for Meeting 2 1. What is the range of services that staff at your agency provides for SED youth and their families at your agency? What are the eligibility criteria for these different services? 2. Describe the different staff members who are involved with SED youth and their families as they receive services at your agency – at each stage from access to discharge. a. Are there any specific procedures (e.g., intake, IEP, behavioral safety planning, etc) that these staff are involved with during this process? 3. Who at your agency has access to information that could be useful for: a. coordinating services for specific cases involving SED youth and their families? b. helping SED youth and their families access services or community resources? 4. What skills, perspectives, and expertise do providers in your agency typically have that could be useful in problem-solving for SED youth and their families? 177 Appendix C: Weekly Reflection Forms Name: _______________________ Collaboration Log Use this log to record your collaboration with outside providers or stakeholders related to your work with youth and their families. We will use this log for the discussion at our next meeting. What Date With whom? Regarding what? Barriers encountered? So What What resulted from the interaction? Successful? Now What Lessons learned for future collaborations? Adapted from Eoyang, G.H. (2006, January). What? So what? Now what? Attractors, Info-letter of the Human Systems Dynamics Institute, 3(1). www.hsdinstitute.org/lear-more/read-the-latest/attractors/archive/18-ATTRACTORS-Jan-2006.pdf 178 Appendix D: Wave 1 Survey Consent Form Consent Form for Saginaw System of Care Survey The Saginaw System of Care effort has been working since 2007 to strengthen and improve the service delivery system in Saginaw for children with severe emotional disturbances (SED) and their families. The Saginaw System of Care’s main goal is to help youth and families be safe, healthy, at home, and fully involved and included in their community. This fall we are sending out a survey to learn more about the current state of the service delivery system in Saginaw. The survey is also part of a research study to learn more about what types of knowledge, skills, and relationships could help providers meet the System of Care’s goals. You were selected to receive this survey because your agency is currently a partner in the Saginaw System of Care effort. We are hoping that you will complete this survey and provide us with a description of the work you do with youth and families. Your responses are important. They will help guide the effort’s decisions about how to improve our county’s ability to meet the needs of youth and families. The survey should take about 30 minutes to complete. You may also be contacted at a later date to provide additional feedback about your work in Saginaw through a brief interview. There are no known risks associated with participating in this study, and completing the survey may allow you to reflect on the work you do with youth and families as well as contribute to the understanding of how to improve system of care efforts. As a way of thanking you for your participation, if you choose to complete this survey you will receive a $15 gift certificate. All of your answers will be kept confidential and will be seen only by the research team at Michigan State University (MSU). Information you provide in the survey will be kept on the campus of MSU in a locked file cabinet and/or password protected computer for 5 years after the study, and only the MSU research team and the Institutional Review Board at MSU will have access to the research data. Participation in this survey is voluntary. You may choose not to participate at all, or you may refuse to answer certain questions or discontinue your participation at any time without consequence. We appreciate your efforts in filling out the survey as completely and accurately as possible. Your confidentiality will be protected to the maximum extent allowable by law. Your name will never be associated with the information you provide in this study, and no identifying information will ever be released to anyone. If you have any concerns or questions about this study, or would like assistance completing this survey, please contact Erin Watson (email: droegeer@msu.edu, toll free number: 517-355-3825, address: 238 Psychology Building, East 179 Lansing, MI 48824) or the survey project director, Dr. Pennie Foster-Fishman, (phone: 517-3535015) at Michigan State University. If you have any questions regarding your role and rights as a study participant, or would like to register a complaint about this study, you may contact, anonymously, if you wish, the Michigan State University Human Research Protection at (517) 355-2180, FAX: (517) 432-4503, email: irb@msu.edu, or regular mail: HRPP, 207 Olds Hall, East Lansing, MI 48824. We hope you will share your thoughts and experiences about your important work with Saginaw youth and families. Thank you for your help! Consent: By signing this form I indicate that I voluntarily agree to participate in this survey. ______________________________ Printed name ____________________________ ____________ Signature Date 180 Appendix E: Survey Measures Saginaw System of Care Survey The Saginaw System of Care effort has been working since 2007 to strengthen and improve the service delivery system in Saginaw for children with severe emotional disturbances (SED) and their families. (For a definition of SED see last page of survey). The Saginaw System of Care’s main goal is to help youth and families be safe, healthy, at home, and fully involved and included in their community. Two of the key strategies for reaching this goal are: 1) Coordinating services through interagency collaboration: working across agencies to provide families with integrated and accessible support 2) Engaging youth and family members as valued system partners: providing youth and families opportunities to fully participate in planning and developing their services, and to engage in leadership roles within the system of care effort. This survey is designed to learn more about the service delivery system in Saginaw. Your answers can help guide the effort’s decisions about how to make our service delivery system better at meeting the needs of youth and families. Your name will never be associated with the information you provide in this survey. Findings will be shared with your agency as well as other key stakeholders in the county. The survey contains a variety of items and questions and will take about 20 minutes to complete. Throughout the survey you will see that some questions offer a “Don’t Know” response. Please ONLY choose this option if you do not have enough information to accurately answer the question. Overuse of the “Don’t Know” response will reduce our ability to provide Saginaw county with useful feedback We thank you for completing this survey. 181 1. Which of the following agencies do you work for?        Saginaw Community Mental Health Authority Family Division of the Circuit Court Department of Human Services Saginaw Intermediate School District Saginaw Public School District Birchrun Area School District Buena Vista School District PERCEPTIONS OF YOUR WORK WITHIN THE SERVICE DELIVERY SYSTEM We would like to know about some of your perceptions of your work within the service delivery system in Saginaw. 2. Thinking about the practice of interagency collaboration, to what extent do you agree with the following? a. I believe collaborating with service providers from other agencies will have a favorable effect on my practice. b. When I think about collaborating with service providers from other agencies, I realize it is appropriate for my situation. c. Utilizing the unique expertise of providers from different agencies through collaboration will improve my ability to meet the unique needs of youth and families. d. I desire to increase my collaboration with service providers in other agencies. e. Increasing my collaboration with service providers in other agencies will benefit me in my job. f. I see increasing my collaboration with service providers in other agencies as making my work harder. 182 Not At All A Little Somewhat Mostly Quite A Bit A Great Deal                                     2. Thinking about the practice of interagency collaboration, to what extent do you agree with the following? g. I feel it is worth it to increase my collaboration with service providers in other agencies around the needs of youth and families. h. Increasing my collaboration with service providers in other agencies would make it difficult for me to meet my agency’s expectations for my job performance. i. I can better meet the needs of youth and families on my caseload by working with other service providers in Saginaw than working alone. j. Differences in attitudes and philosophies between myself and service providers from outside agencies make it difficult for us to work together. k. My understanding of the needs of SED youth in our community differs from that of service providers from outside agencies l. The goals and activities related to my work with youth and families are similar to the goals and activities of service providers from outside agencies. m. I am willing to work through differences with service providers from outside agencies to arrive at win-win solutions. Not At All A Little Somewhat Mostly Quite A Bit A Great Deal                                           KNOWLEDGE AND SKILLS The following questions will ask about your current knowledge and skills related to collaborating with individuals from outside agencies. These outside agencies could be public (e.g., Saginaw County Community Mental Health Authority, Family Division of the Circuit Court, Department of Human Services, Saginaw Intermediate School District, Saginaw Public School District, Birchrun Area School District, Buena Vista School District) or private/non-profit. Several questions will also ask about your current knowledge and skills related to partnering with youth and families 183 3. If needed, I have the ability to locate and initiate contact with Not At All the following individuals from outside agencies: A Little Somewhat Mostly Quite A Bit A Great Deal a. Relevant staff who could generally assist me in coordinating services for the youth and families on my case load.       b. Specific providers who are currently providing services to some of the same youth and families on my case load.       c. Various providers who work with youth and families at different points in the time from when they first enter services to when they finish services.       d. Providers in different departments who could assist me in addressing the needs of the youth and families on my caseload.       4. To what degree are you aware of the following related to Not At All outside agencies: A Little Somewhat Mostly Quite A Bit A Great Deal a. The full range of services offered.       b. The services eligibility requirements for youth and families.       c. The process families must go through to access services       5. To what extent do you feel confident in your ability to effectively carry out the following with providers from outside agencies? Not At All a. Communicate suggestions or ideas on how to improve services for youth and families. 184  A Little  Somewhat  Mostly Quite A Bit A Great Deal    5. To what extent do you feel confident in your ability to effectively carry out the following with providers from outside agencies? Not At All A Little Somewhat Mostly Quite A Bit A Great Deal b. Articulate the reasoning behind your ideas or views.       c. Directly work through differences in opinion.       d. Integrate diverse perspectives and expertise when problemsolving around a case.       e. Facilitate open and honest communication.       f. Elicit others’ ideas about service delivery.       g. Evaluate other’ ideas in order to improve effectiveness.       h. Communicate in ways that encourage others to critique your ideas on service delivery.       COLLABORATION The following questions will ask about your collaboration with individuals from outside agencies. These outside agencies could be public (e.g., Saginaw County Community Mental Health Authority, Family Division of the Circuit Court, Department of Human Services, Saginaw Intermediate School District, Saginaw Public School District, Birchrun Area School District, Buena Vista School District) or private/non-profit. 6. How often do you initiate the following with providers from outside agencies who are relevant to your work with youth and families? a. Exchange information (when possible) related to a specific youth/family to guide decision-making about the case. Less than quarterly  185 Quarterly  Monthly  Biweekly Weekly   Daily  6. How often do you initiate the following with providers from outside agencies who are relevant to your work with youth and families? Less than quarterly Quarterly Monthly Biweekly Weekly Daily b. Share ideas on how to improve outcomes for specific case.       c. Suggest ways to integrate your different perspectives or ideas for the sake of problem-solving around a case.       d. Talk about how to collaborate in delivering service to a youth or family.       e. Evaluate each others’ ideas in order to improve effectiveness.       f. Share reflections on the way services are being delivered to a case.       7. How often do you initiate the following with providers Less than quarterly Quarterly Monthly Biweekly Weekly from any agency, including your own? Daily a. Discuss ways to overcome barriers to collaboration.       b. Share information as a means to promote collaboration between providers from different agencies.       c. Introduce them to providers who are relevant to their work with youth and families.       d. Exchange ideas about how to better tap into the unique expertise of providers from different agencies.       186 8. To what extent do the following occur as a result of your collaboration with providers from outside agencies? Not At All A Little Somewhat a. New practices related to working with youth occur as a result of the diversity of ideas among collaborating providers.       b. Working with providers who have multiple perspectives results in new strategies for helping youth and families.       c. The roles and/or responsibilities of providers change as a result of collaboration.       d. As a result of working together, services/supports for youth are delivered in new ways.       e. Service plans are jointly created by providers who share cases.       f. Joint service plans are collectively revised by providers who share cases in order to respond to the changing needs of a case.       g. Providers work together to provide ongoing follow-up to a shared case.       Mostly Quite A Bit A Great Deal Mostly Quite A Bit 9. To what extent are you familiar with the following characteristics related to the providers you collaborate with from outside agencies? A Great Deal Not At All A Little Somewhat             a. The types of information (related to available services, specific cases, etc.) they have access to in their agency. b. The full range of their skills and expertise related to serving youth and families. 187 9. To what extent are you familiar with the following characteristics related to the providers you collaborate with from outside agencies? Mostly Quite A Bit A Great Deal                                        Not At All c. Their connections to other providers or organizations in the service delivery system. d. Their unique ideas or perspectives for addressing the needs of youth and families. e. Their general experiences with the youth and families on their caseload. f. The technical language or jargon they use related to serving youth and families. g. The service approach or philosophy they take in addressing the needs of youth and families. h. The limitations (legal, financial, policy) affecting their ability to deliver services.    i. Their educational background or previous work experiences. 188 A Little Somewhat Appendix F: Description of Study Used During Recruitment Presentation. Saginaw Partnerships Project   Purpose is to provide opportunities for providers to: o Learn how our different agencies work o Explore ways to utilize our collective resources and strengths to improve outcomes for youth o Problem-solve barriers to working together in the real world Project involves 2 different groups of diverse providers: o First group will attend 6 bi-monthly discussion meetings, 2 hours each – lunch provided at each meeting o Second group will provide feedback in the spring on how collaboration is working in Saginaw and ideas for moving forward – lunch also provided at this feedback session 189 Appendix G: Protocol for Validation Case Study Interviews Introduction: This is Erin Watson from Michigan State University. I am calling to see if you would be willing to do a 10 minute interview with me today as a follow-up to the system of care survey. As a way of thanking you for your time, I will send you a $5 gift certificate for doing the interview with me. If you have time, we can do the interview right now – or at a later time today or this week. This will be a very brief interview today, probably about 10-15 minutes. I’m going to ask you a few questions about your experiences working with other service providers in the Saginaw, as well as your ideas for making it easier for providers in Saginaw to work together. CONSENTING I’m going to be taking notes as we talk today. To make sure I get everything you say, would it be alright if I record our conversation? I will destroy the recording after I fill in my notes. You can also ask me to turn off the recorder at any time. To start out with, I want you to think about your most recent SED case that was successfully completed or closed (as opposed to the youth dropping out of services) where the youth was involved with at least one additional public service agency outside your own. By public I mean agencies like Saginaw County Community Mental Health Authority, Family Division of the Circuit Court, Department of Human Services, or the schools. Great. Now I’m going to ask you a few questions and I would like you to answer based on this case. 1. Did you ever contact another agency within the system of care for assistance with the case (e.g., to help coordinate services, access services, or address a problem for the case)? YES NO IF YES Which agencies did you contact? (mark below). Great. Let’s take each of these agencies one at a time. 190 ___ CMH ___ DHS ___ Courts ___ Schools a. Who did you first contact for assistance with the case at CMH? Agency’s general number. General number for a particular department within the agency. Someone I knew in the agency who could refer me to the right person. Someone I knew in the agency who could directly assist me with the case. Other: (please describe below) a. Who did you first contact for assistance with the case at DHS? Agency’s general number. General number for a particular department within the agency. Someone I knew in the agency who could refer me to the right person. Someone I knew in the agency who could directly assist me with the case. Other: (please describe below) a. Who did you first contact for assistance with the case at the courts? Agency’s general number. General number for a particular department within the agency. Someone I knew in the agency who could refer me to the right person. Someone I knew in the agency who could directly assist me with the case. Other: (please describe below) a. Who did you first contact for assistance with the case at the schools? Agency’s general number. General number for a particular department within the agency. Someone I knew in the agency who could refer me to the right person. Someone I knew in the agency who could directly assist me with the case. Other: (please describe below) b. Did you eventually reach someone who could assist you with the case? b. Did you eventually reach someone who could assist you with the case? b. Did you eventually reach someone who could assist you with the case? b. Did you eventually reach someone who could assist you with the case? YES NO (BOLD one) If NO, describe what you did next? YES YES NO (BOLD one) If NO, describe what you did next? 191 NO (BOLD one) If NO, describe what you did next? YES NO (BOLD one) If NO, describe what you did next? 2. Do you know of any outside providers who were assigned to the case (e.g., probation officers, school social workers, child protective services workers, therapists, foster care workers, etc)? YES NO (BOLD one) If YES: What are the names of these providers? What agencies did they come from? (write below). Great. Let’s take each of these individuals one at a time. Name: __________________ Name: __________________ Name: __________________ Name: __________________ Agency:_________________ Agency:_________________ Agency:_________________ Agency:_________________ a. how did you find out this person was on the case? a. how did you find out this person was on the case? a. how did you find out this person was on the case? a. how did you find out this person was on the case? b. In general, how often did you talk with this provider? Daily A few times a week Once a week About every other week Monthly About every 2-3 months Less than every 4 months Never b. In general, how often did you talk with this provider? Daily A few times a week Once a week About every other week Monthly About every 2-3 months Less than every 4 months Never b. In general, how often did you talk with this provider? Daily A few times a week Once a week About every other week Monthly About every 2-3 months Less than every 4 months Never b. In general, how often did you talk with this provider? Daily A few times a week Once a week About every other week Monthly About every 2-3 months Less than every 4 months Never 192 IF MORE THAN NEVER: IF MORE THAN NEVER: IF MORE THAN NEVER: IF MORE THAN NEVER: What did you two typically discussed related to the case? What did you two typically discussed related to the case? What did you two typically discussed related to the case? What did you two typically discussed related to the case? c. Did this person have any information (for instance about the case or about available services in Saginaw) you did not have that was useful for improving outcomes for the case? c. Did this person have any information (for instance about the case or about available services in Saginaw) you did not have that was useful for improving outcomes for the case? c. Did this person have any information (for instance about the case or about available services in Saginaw) you did not have that was useful for improving outcomes for the case? c. Did this person have any information (for instance about the case or about available services in Saginaw) you did not have that was useful for improving outcomes for the case? YES NO YES NO YES NO YES NO If YES, describe this information. If YES, describe this information. If YES, describe this information. If YES, describe this information. d. Did this person have any connections in Saginaw you did not have that were useful for improving outcomes for the case? d. Did this person have any connections in Saginaw you did not have that were useful for improving outcomes for the case? d. Did this person have any connections in Saginaw you did not have that were useful for improving outcomes for the case? d. Did this person have any connections in Saginaw you did not have that were useful for improving outcomes for the case? YES NO YES NO YES 193 NO YES NO If YES, describe these connections. e. Did this person suggest any service ideas you alone did not come up with that were useful for improving outcomes for the case? YES NO If YES, describe these service ideas. If YES, describe these connections. e. Did this person suggest any service ideas you alone did not come up with that were useful for improving outcomes for the case? YES If YES, describe these connections. e. Did this person suggest any service ideas you alone did not come up with that were useful for improving outcomes for the case? NO YES If YES, describe these service ideas. NO If YES, describe these service ideas. If YES, describe these connections. e. Did this person suggest any service ideas you alone did not come up with that were useful for improving outcomes for the case? YES NO If YES, describe these service ideas. IF EVER COMMUNICATED WITH ANY OF THE ABOVE PROVIDERS: 3. Did you and this/these other provider(s) implement any new strategies or approaches for improving outcomes for the case as a result of your interactions? For example a specific new approach for how to help a youth with an issue in their life. YES NO If YES: a. Can you describe these new service strategies or approaches? 194 b. How did these new strategies or approaches come about? In other words, what about your interactions led to these strategies/approaches? 4. Did you and the other provider(s) develop new ways of working together to improve outcomes for the case? For example, following up with a case together instead of doing it separately. YES NO If YES: a. Can you describe these new ways of working together? b. How did these new ways of working together come about? What about your interactions led to these new ways of working together? 195 REFERENCES 196 REFERENCES Akhavain, P., Amaral, D., Murphy, M., & Uehlinger, K. (1999). Collaborative practice: A nursing perspective of the psychiatric interdisciplinary treatment team. Holistic Nursing Practice, 13, 1–11. Alter, C., & Hage, J. (1993). Organizations working together. Newbury Park, CA: Sage. Allred, C.A., Burns, B.J., & Phillips, S.D. (2005). The assertive community treatment team as a complex dynamic system of care. Administration and Policy in Mental Health, 32(3), 211-220. Ancona, D., & Caldwell, D. F. 1992. Demography and design: Predictors of new product team performance. Organization Science, 3: 321–341. Anderson, P. (1999). Complexity theory and organization science. Organization Science, 10, 3, 216-232. Anderson, R.A., Crabtree, B., Steele, D., & McDaniel, R. (2005). Case study research: the view from complexity science. Qualitative Health Research, 15(5), 669–85. Anderson, R. A., Issel, L. M., & McDaniel, R. R., Jr. (2003). Nursing homes as complex adaptive systems: Relationship between management practice and resident outcomes. Nursing Research, 52(1), 12-21. Annie E. Casey Foundation. (1999). Myths and Opportunities: An Examination of the Impact Of Discipline-Specific Accreditation on Inter-professional Education. Alexandria, VA: Council on Social Work Education. Available: http://www.cswe.org. Argote, L., Gruenfeld, D., & Naquin, C. 1999. Group learning in organizations. In M. E. Turner (Ed.), Groups at work: Advances in theory and research: 369–413. New York: Erlbaum. Argyris, C. (1991). Teaching smart people how to learn. Harvard Business Review, 69, 3, 99109. San Francisco: Jossey-Bass Publishers. Argyris, C. (1993). Knowledge for action: A guide to overcoming barriers to organizational change. Argyris, C. and Schön, D. (1996) Organizational learning II: Theory, method and practice, Reading, Mass: Addison Wesley. Armenakis, A.A., Bernerth, J.B., Pitts, J.P., & Walker, H.J. (2007). Organizational change recipients’ beliefs scale: Development of an assessment instrument. The Journal Of Applied Behavioral Science, 43(4), 481-505. 197 Arrow, H., & Henry, K.B. (2010). Using complexity to promote group learning in health care. Journal of Evaluation in Clinical Practice, 16, 861-866. Baldwin, T.T. (1992) Effects of alternative modeling strategies on outcomes of interpersonalskills training. Journal of Applied Psychology, 77(2), 147–54. Baldwin, T.T., Ford, K.J., & Blume, B.D. (2009). Transfer of training 1988-2008: An updated review and agenda for future research. International Review of Industrial and Organizational Psychology, 24, 41-71. Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice Hall. Bapuji, H., & Crossan, M. (2004). From questions to answers: Reviewing organizational learning literature. Management Learning, 35(4), 397-417. Barr, H., Koppel, I., Reeves, S., Hammick, M., Freeth, D. (2005). Effective Inter-professional Education: Assumption, Argument and Evidence. London: Blackwell. Bartunek, J., Rousseau, D., Rudolph, J., & DePalma, J. (2006). On the receiving end: Sensemaking, emotion, and assessments of an organizational change initiated by others. The Journal of Applied Behavioral Science, 42, 182-206. Basadur, M. (2004). Leading others to think innovatively together: Creative leadership. Leadership quarterly, 15, 103-121. Basadur, M. S. (1994). Managing the creative process in organizations. In M. J. Runco (Ed.), Problem finding, problem solving, and creativity (pp. 237–268). New York: Ablex. Bonabeau, E. & Meyer, C. (2001). Swarm intelligence: A whole new way to think about business. Harvard Business Review, p. 107-114. Braithwaite, J., Westbrook, J., Foxwell, R., Boyce, R., Devinney, T., Budge, M., Murphy, K…Clay-Williams, R. (2007). An action research protocol to strengthen system-wide inter-professional learning and practice. BMC Health Services Research, 7, 144-154. Brewer, M. (1995). Managing diversity: The role of social identities. In S. Jackson & M. Ruderman (Eds.), Diversity in workteams (pp. 131–159). Washington, DC: APA Books. Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and design. Cambridge, MA: Harvard University Press. Bronstein, L. R. (2003). A Model for Interdisciplinary Collaboration. Social Work, 48(3), 297306. Brown, B., & Crawford, P. (2003). The clinical governance of the soul: ‘Deep management’ and the self regulating subject in integrated community mental health teams. Social Science & Medicine, 56, 67–81. 198 Bruns, E., Suter, J., Burchard, J. D., & Leverentz-Brady, K. (2004). A national portrait of wraparound implementation: Findings from the Wraparound Fidelity Index. In C. C. Newman, C. J. Liberton, K. Kutash, & R.M. Friedman (Eds.), The 16th annual research conference proceedings: A System of Care for Children’s Mental Health: Expanding the Research Base. (pp. 281–286). Tampa, FL: University of South Florida, The Louis de la Parte Florida Mental Health Institute, The Research and Training Center on Children’s Mental Health. Bruns, E.J., Suter, J.C., Force, M.M., & Burchard, J.D. (2005). Adherence to wraparound principles and association with outcomes. Journal of Child and Family Studies, 14, 521– 34. Bullock, R., & Little, M. (1999). The interface between social and health services for children and adolescent persons. Current Opinion in Psychiatry, 12, 421–424. Bunderson, J.S. & Sutcliffe, K.M. (2002). Why some teams emphasize learning more than others: Evidence from business unit management teams. In M. Neale, E. Mannix, & H. Sondak (Eds.), Research on Managing Groups and Teams, 4: 49‐84. Oxford: Elsevier Science Ltd. Burchard, J.D., Bruns, E.J., Burchard, S.N. (2002). The wraparound approach. In: Burns BJ, Hoagwood K, editors. Community treatment for youth: evidence-based interventions for severe emotional and behavioral disorders. (pp.69-90). New York: Oxford University Press. Burke, C.S., Salas, E., Diaz, D. (2007). The role of team learning in facilitating team adaptation within complex environments: Tools and strategies. In V. Sessa and M. London (Eds.) Work group learning (pp. 217-243). New York: Lawrence Erlbaum Associates. Burns, B. J., & Goldman, S. K. (Eds.). (1999). Systems of care: Promising practices in children's mental health, 1998 series: Volume IV. Promising practices in wraparound for children with severe emotional disorders and their families. Washington, DC: Center for Effective Collaboration and Practice, American Institutes for Research. Bushnell, M. S. & Dean, J. M. (1993). Managing the intensive care unit: nurse/physician collaboration. Critical Care Medicine, 21(9), S389. Butt, G., Markle-Reid, M., & Browne, G. (2008). Inter-professional partnerships in chronic illness care: a conceptual model for measuring partnership effectiveness. International Journal of Integrated Care, 8(15), 1-14. Caplan, G., & Caplan, R. (1993). Mental health consultation and collaboration. San Fransisco: Jossey-Bass Publishers. Carr, E., Brockbank, K., Barrett, R. (2003). Improving pain management through interprofessional education: evaluation of a pilot project. Learning in Health and Social Care 2, 6–17. 199 Center for Mental Health Services. (2008). Bringing systems of care to scale: Report of the systems change workgroup. Rockville, MD: Author. Challis, L., Fuller, S., Henwood, M., Klein, R., Plowden, W., Webb, A., Whittingham, P., & Wistow, G. (1988). Joint approaches to social policy: Rationality and practice. Cambridge: Cambridge University Press. Churchill, G. A. (1979). A paradigm for developing better measures of marketing constructs. Journal of Marketing Research, 16, 64-73. Cilliers P. (2001). Boundaries, hierarchies and networks in complex systems. International Journal of Innovation Management, 5,135–47. Clancy, T.R. (2009). In search of elegance: Making the complex simple. Journal of Nursing Administration, 39(12), 507-510. Clark, H.B., Lee, B., Prange, M.E., & McDonal, B.A. (1996). Children lost within the foster care system: Can wraparound service strategies improve placement outcomes? Journal of Child and Family Studies, 5(1), 39–54. Clark, P. G. (1994). Social, professional and educational values on the interdisciplinary team: implications for gerontological and geriatric education. Educational Gerontology, 20(1), 35-52. Comfort, L.K., (1994). Self-organization in complex systems. Journal of Public Administration, Research & Theory, 3, 393-410. Cook, J.R., & Kilmer, R.P. (2010a). Defining the scope of system of care: An ecological perspective. Evaluation and Program Planning, 33, 18-20. Cook, J.R., & Kilmer, R.P. (2010b). The Importance of Context in Fostering Responsive Community Systems: Supports for Families in Systems of Care. American Journal of Orthopsychiatry, 80, 115-123. Cott, C. (1997). ‘‘We decide, you carry it out’’: a social network analysis of multidisciplinary long-term care teams. Social Science & Medicine, 45(9),1411–21. CQ Press. (2008). City crime rates. Retrieved October 17, 2010 from http://os.cqpress.com/citycrime2008/citycrime2008.htm Crabtree, B. F. (2003). Primary care practices are full of surprises! Health Care Management Review, 28(3), 279-283. Crossan, M.M., Lane, H.W. and White, R.E. (1999). An organizational learning framework: from intuition to institution. Academy of Management Review, 24, 522–537. Cutcher-Gershenfeld, J. & Ford, J.K. (2005). Beyond disconnected learning: Bold visions and harsh realities of organizational learning. New York: Oxford University Press. 200 D'Amour D., Ferrada-Videla, M., Rodriguez, L.S.M., & Beaulieu, M. D. (2005). The conceptual basis for inter-professional collaboration: Core concepts and theoretical frameworks. Journal of Inter-professional Care, 19, 116-131. D'Amour, D., Sicotte, C., & Levy, R. (1999). The collective action within inter-professional equips in the health services. Social sciences and health, 17, 68- 94. Darlington, Y., Feeney, J.A., Rixon, K. (2005). Interagency collaboration between child protection and mental health services: Practices, attitudes and barriers. Child Abuse and Neglect, 29, 1085-1098. Davidson, L., & White, W. (2007). Recovery as an organizing principle for integrating mental health & addiction services. The Journal of Behavioral Health Services and Research, 34(2), 109-120. Dedrick, R.R., & Greenbaum, P.E. (2010). Multilevel confirmatory factor analysis of a scale measuring interagency collaboration of children’s mental health agencies. Journal of Emotional and Behavioral Disorders, XX(X), 1-14. DeVellis, R.F. (2012). Scale Development: Theory and Applications. Los Angeles: Sage Publications, Inc. Drach-Zahavy, A., & Somech, A. (2001). Understanding team innovation: The role of team processes and structures. Group Dynamics: Theory, Research, and Practice, 5(2) 111– 123. Drinka, T. J. K., & Clark, P. G. (2000). Health care teamwork: Interdisciplinary practice and teaching. Westport, CT: Auburn House/Greenwood. Durlak, J. A., and DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327-350. Easen, P., Atkins, M., & Dyson, A. (2000). Inter-professional collaboration and conceptualisations of practice. Children and Society, 14, 355–367. Easterby-Smith, M., & Arujo, L. (1999). Organizational learning: Current debates and opportunities. In M. Easterby-Smith, J. Burgoyne, and L. Araujo (Eds.) Organizational learning and the learning organization, (pp 1-22). London: Sage Publications. Edgren, L. (2008). The meaning of integrated care: A systems approach. International Journal of Integrated Care, 8(23), 1-6. Edmondson, A. (1999). Psychological safety and learning behavior in work teams. Administrative Science Quarterly, 44, 350–383. Edmondson, A. (2002). The local and variegated nature of learning in organizations: A group – level perspective. Organization Science, 13, 128-146. 201 Edmondson, A. (2003). "Speaking up in the Operating Room: How Team Leaders Promote Learning in Interdisciplinary Action Teams." Journal of Management Studies, 40(6), 1419-1452. Edmondson, A.C., Bohmer, R. & Pisano, G.P. (2001). "Speeding Up Team Learning." Harvard Business Review, 79(9), 125-134. Elwyn G, Greenhalgh T, Macfarlane F. (2000). Groups: a hands-on guide to small group work in health care, education, management and research. Oxford: Rad-cliffe Medical Press. Eoyang, G.H. (2001). Conditions For Self-Organizing In Human Systems. (Unpublished doctoral disseration). The Union Institute and University, Ohio. Epstein, M.H., Nordness, P.D., Kutash, K., Duchnowski, A., Schrepf, S., Benner, G. & Nelson, J.R. (2003). Assessing the wraparound process during family planning meetings. Journal of Behavior Health Services & Research, 30, 352-362. Espevik, R., Johnsen, B. H., Eid, J., & Thayer, J. F. (2006). Shared mental models and operational effectiveness: Effects on performance and team processes in submarine attack teams. Military Psychology, 18, 23–36. Evans, J. A. (1994). The role of the nurse manager in creating an environment for collaborative practice. Holistic Nursing Practice, 8(3), 22-31. Farmer, E.M.Z. (2000). Issues confronting effective services in systems of care. Children and Youth Services Review, 22(8), 627–650. Farmer, E.M.Z., Dorsey, S., & Mustillo, S. A. (2004). Intensive home and community interventions. Child and Adolescent Psychiatric Clinics of North America, 13, 857–884. Faw, L. (1999). The state wraparound survey. In B. J. Burns & S. K. Goldman (Eds.), Systems of care: Promising practices in children’s mental health, 1998 series: Volume IV. Promising practices in wraparound for children with severe emotional disorders and their families (pp. 79– 83). Washington, DC: Center for Effective Collaboration and Practice, American Institutes for Research. Felker, B., Barnes, P., Greenberg, D., Chancy, B., Shores, M., Gillespie-Gatey, L, Buike, K., & Morton, C.E. (2004). Preliminary outcomes from an integrated mental health primary care team. Psychiatric Services, 55(4), 442-444. Flaspohler, P.D., Anderson-Butcher, D.A., Bean, J., Burke, R.W., & Paternite, C.E. (2008). Readiness and school improvement: Strategies for enhancing dissemination and implementation of expanded school mental health practices. Advances in School Mental Health Promotion, 1(1), 16-27. Foster-Fishman, P. G., & Droege, E. (2010). Locating the system in a system of care. Evaluation and Program Planning, 33, 11–13. 202 Foster-Fishman, P.G., Watson, E.R. (2010a). Action research as systems change. In H.E. Fitzgerald, D.L. Zimmerman, C. Burack, & S. Seifer (eds.) Handbook of engaged scholarship: the contemporary landscape. volume two: Community-campus partnerships. East Lansing, Michigan State University Press. Foster-Fishman, P.G., Watson, E.R. (2010b). The ABLe change framework: A conceptual and methodological tool for promoting systems change. Manuscript in progress. Fraser, S.W., & Greenhalgh, T. (2001). Coping with complexity: educating for capability. British Medical Journal, 323, 799-803. Friedman, R.M. (2010). Real change in the real world: An achievable goal. Administrative and Policy in Mental Health, 37, 1-2, 154-159. Friedman, R. M., Katz-Leavy, J. W., Manderscheid, R. W., & Sondheimer, D. L. (1999). Prevalence of serious emotional disturbance: An update. In R. W. Manderscheid & M. J. Henderson (Eds.), Mental health, United States, 1998 (pp. 110-112). Rockville, MD: U.S. Department of Health and Human Services. Friedman, V., Lipshitz, R., & Overmeer, W. (2003). Creating conditions for organizational learning. In M. Dierkes, A.B. Antal, J. Child, and I. Nonaka (Eds.) Handbook of Organizational Learning and Knowledge (pp. 757-774). London: Oxford University Press. Friedman, S.R., Reynolds, J., Quan, M.A., Call, S., Crusto, C.A., & Kaufman, J.S.. (2007). Measuring changes in interagency collaboration: An examination of the Bridgeport Safe Start Initiative". Evaluation and Program Planning, 30, 294-306. Gage, M. (1998). From independence to interdependence. Journal of Nursing Administration, 28(4), 17-26. Gibson, C. B., & Vermeulen, F. (2003). A healthy divide: Subgroups as a stimulus for team learning. Administrative Science Quarterly, 48: 202–239. Gist, M.E., Bavetta, A.G. and Stevens, C.K. (1990) Transfer training method – its influence on skill generalization, skill repetition, and performance-level. Personnel Psychology, 43 (3), 501–23. Glisson, C. (2002). The organizational context of children’s mental health services. Clinical Child and Family Psychology Review, 5, 233-253. Glisson, C. (2007). Assessing and changing organizational culture and climate for effective services. Research on Social Work Practice, 17, 736-747. Glisson, C., Dukes, D., & Green, P. (2006). The effects of the ARC organizational intervention on caseworker turnover, climate, and culture in children’s service systems. Child Abuse and Neglect, 30, 855-880. 203 Glisson, C., & Hemmelgarn, A. L. (1998). The effects of organizational climate and interorganizational coordination on the quality and outcomes of children’s service systems. Child Abuse & Neglect, 22, 401–421. Glisson, C., & James, L. R. (2002). The cross-level effects of culture and climate in human service teams. Journal of Organizational Behavior, 23, 767–794. Goldman, J., Zwarenstein, M., Bhattacharyya, O., & Reeves, S. (2009). Improving the clarity of the inter-professional field: implications for research and continuing inter-professional education. Journal of Continuing Education in the Health Professions, 29(3), 151-156. Gruenfeld, D. H & Hollingshead, A. B. (1993) Sociocognition in work groups: the evolution of group integrative complexity and its relation to task performance. Small Group Research, 24(3), 383–405 Haley, W.E., McDaniel, S.H., Bray, J.H., Frank, R.G., Heldring, M., Johnson, S.B, Lu, E.G., Reed, G.M., & Wiggins, J. (2004). Psychological practice in primary care settings: Practical tips for clinicians. In R.G., Frank & S.H. McDaniel (Eds.) Primary care psychology (pp. 95-112). Washington, D.C.: American Psychological Association. Hallett C, Birchall E. 1992. Co-ordination and Child Protection: A Review of the Literature. London: HMSO. Hanley, J. H. &Wright, H. H. (1995). Child mental health professionals: The missing link in child mental health reform. Journal of Child and Family Studies, 4, 383-388. Hastings, G., & Saren, M. (2003). The critical contribution of social marketing: Theory and application. Marketing Theory 3(3), 305-322. Henneman, E. A., Lee, J. L., & Cohen, J. I. (1995). Collaboration: a concept analysis. Journal of Advanced Nursing, 21, 103-109. Hernandez, M., & Hodges, S. (2003). Building upon the theory of change for systems of care. Journal of Emotional and Behavioral Disorders, 11(1), 19–26. Herzog, S. M., & Hertwig, R. (2009). The wisdom of many in one mind: Improving individual judgments with dialectical bootstrapping. Psychological Science, 20(2), 231–237. Himmelman, A. T. (2004). Collaboration for a change: Definitions, decision-making models, roles, and collaboration process guide. Minneapolis, MN: Himmelman Consulting. Hinkin, T. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods, 1, 104-121. Hoagwood, K., Burns, B. J., Kiser, L., Ringeisen, H., & Schoenwald, S. (2001). Evidence-based practice in child and adolescent mental health services. Psychiatric Services, 52, 1179– 1189. 204 Hodges, S., Ferreira, K., Israel, N., Mazza, J. (2006). Strategies for system of care development: making change in complex systems. Tampa, FL: Research and Training Center for Children’s Mental Health. Hodges, S., Hernandez, M., & Nesman, T. (2003). A developmental framework for collaboration in child-serving agencies. Journal of Child and Family Studies, 12, 3, 291-305. Hodges, S., Nesman, T., and Hernandez, M. (1999). Promising practices: Building collaboration in systems of care. Systems of Care: Promising Practices in Children’s Mental Health, 1998 Series, Volume VI. Washington, D.C.: Center for Effective Collaboration and Practice, American Institutes for Research. Hohmann, A. A., & Shear, M. K. (2002). Community-based intervention research: Coping with the noise of real life in study design. American Journal of Psychiatry, 159, 201–207. Holbrook, M. B. (2003). “Adventures in complexity: An essay on dynamic open complex adaptive systems, butterfly effects, self-organizing order, coevolution, the ecological perspective, fitness landscapes, market spaces, emergent beauty at the edge of chaos, and all that jazz.” Academy of Marketing Science Review, 6. Holman, C., Jackson, S., 2001. A team-based education project: an evaluation of a collaborative education and practice development in a continuing care unit for older people. Nurse Education Today, 21, 97–103. Huo, Y. (2003). Procedural justice and social regulation across group boundaries: Does subgroup identification undermine relationship based governance? Personality and Social Psychology Bulletin, 29, 336–348. Hyde, K. L., Burchard, J. D. & Woodworth, K. (1996) Wrapping services in an urban setting, Journal of Child and Family Studies, 5(1), 67-82. Institute of Medicine. (2001). Crossing the Quality Chasm: A New Health System for the 21st Century. Washington DC: National Academy Press. Institute of Medicine. (2003). Health professions education: a bridge to quality. Washington: National Academy Press.Jensen, P. S. (2003). Commentary: The next generation is overdue. Journal of the American Academy of Child and Adolescent Psychiatry, 42, 527– 530. Institute of Medicine. (2005). Quality Treatment for People with Mental and Substance-use Disorders. Rockville, MD: Institute of Medicine Academies Press Jensen, P. S. (2003). Commentary: The next generation is overdue. Journal of the American Academy of Child and Adolescent Psychiatry, 42, 527–530. Johnson, N. D. (1992). Collaboration. An environment for optimal outcome. Critical Care Nursing Quarterly, 15(3), 37-43. 205 Karl, K.A. and Ungsrithong, D. (1992) Effects of optimistic versus realistic previews of training programs on self-reported transfer of training. Human Resource Development Quarterly, 3, 373–84. Kerr, N. L., & Tindale, R. S. (2004). Group performance and decision making. Annual Review of Psychology, 55, 623–655. Kim, J., & Mueller, C. (1978). Introduction to factor analysis: What it is and how to do it. Beverly Hills, CA: Sage. Kilduff, M., Tsai, W., Hanke, R. (2006). A paradigm too far? A dynamic stability reconsideration of the social network research program. Academy of Management Review, 31, 1031-1048. Kilmer, R.P., Cook, J.R., Palamaro Munsell, E. (2010). Moving from principles to practice: recommended policy changes to promote family-centered care. American Journal of Community Psychology, 13, 332-341. Kilminster, S., Hale, C., Lascelles, M., Morris, P., Roberts, T., Stark, P., Sowter, J., Thistlethwaite, J. (2004). Learning for real life: patient-focus inter-professional workshops offer added value. Medical Education 38, 717–726. Klein, K. J., & Knight, A. P. (2005). Innovation implementation: Overcoming the challenge. Current Directions in Psychological Science 14(5), 243-46. Klein, K.J., & Sorra, J.S. (1996). The challenge of innovation implementation. Academy of Management Review, 21(4), 1055-1080. Knitzer, J. (1982). Unclaimed children: The failure of public responsibility to children and adolescents in need of mental health services. Washington, DC: The Children’s Defense Fund. Kotter, J. (1995). Leading change: Why transformation efforts fail. Harvard Business Review, 73(2), 59-67. Krackhardt, D. (1987). Cognitive social structures. Social Networks, 9, 109– 134. Krauss, R., & Morsella, E. (2000). Communication and conflict. In M. Deutsch, & P. T. Coleman (Eds.), The handbook of conflict resolution (pp. 131-143). San Francisco, CA: Jossey Bass. Kutash, K., & Duchnowski, A.J. (1997). Create comprehensive and collaborative systems. Journal of Emotional and Behavioural Disabilities, 5, 66–75. Lemieux-Charles, L., & McGuire, W. (2006). What do we know about health care team effectiveness: A review of literature. Medical Care Research and Review, 63(3), 1-38. 206 Lin, A.C. 2000. Reform in the making: The implementation of social policy in prison. Princeton, N.J.: Princeton University Press. Lippitt, R., & Van Til, J. (1981). Can we achieve a collaborative community? Issues, imperatives, potentials. Journal of Voluntary Action Research, 10(3-4), 7-17. Lipsky, M. (1980). Street-level bureaucracy: Dilemmas of the individual in public services. New York: Russell Sage. Madhavan, R., & Grover, R. (1998). From embedded knowledge to embodied knowledge: New product development as knowledge management. Journal of Marketing, 62(4), 1–12. March, J.G. (1991). Exploration and exploitation in organizational learning. Organizational Science, 2, 71–87. Martin-Rodrigues, L.S., Beaulieu M., D’Amour, D., & Ferrada-Videla, M. (2005). The determinants of successful collaboration: A review of theoretical and empirical studies. Journal of Inter-professional Care, 1, 132-147. Malekoff, A. (2000). Bureaucratic barriers to service delivery, administrative advocacy, and Mother Goose. Families in Society: The Journal of Contemporary Human Services, 81(3), 304–314. Manikam, R. (2002). Mental health of children and adolescents. In N. N. Singh, T. H. Ollendick, & A. N. Singh (Eds.), International perspectives on child & adolescent mental health care, Volume 2. Amsterdam: Elsevier. Mannix, E., & Neale, M.A. (2005). What differences make a difference?: The promise and reality of diverse teams in organizations. Psychological Science In The Public Interest, 6(2), 31-55. Mariano, C. (1989). The case for the interdisciplinay collaboration. Nursing Outlook, 37(6), 285288. Mathieu, J.E. and Martineau, J.W. (1997) Individual and situational influences on training motivation, in Improving Training Effectiveness in Work Organizations (eds J.K. Ford, S.W.J. Kozlowski, K. Kraiger et al.), Erlbaum, Mahwah, NJ, pp. 193–221. McDaniel, R. R., & Driebe, D. J. (2001). Complexity science and health care management. In J. D. Blair, M. D. Fottler, & G. T. Savage (Eds.), Advances in health care management (Vol. 2, pp. 11-36). Stamford, CT: JAI. McPherson, M., Smith-Lovin, L., & Cook, K. M. (2001). Birds of a feather: Homophily in social networks. In K. S. Cook & J. Hagan (Eds.), Annual review of sociology, 27, 415–444. Palo Alto, CA: Annual Reviews. 207 McGinty, K., McCammon, S.L., & Koeppen, V.P. (2000). The complexities of implementing the wraparound approach to service provision: a view from the field. Journal of Family Social Work, 5(3), 95–110. McGuire, D., & Hutchins, K. (2006). A Machiavellian analysis of organizational change. Journal of Organizational Change Management, 19, 199-209. Mellin, E.A., Bronstein, L., Anderson-Butcher, D., Amorose, A.J., Ball, A., & Green, J. (2010). Measuring inter-professional team collaboration in expanded school mental health: Model refinement and scale development. Journal of Inter-professional Care, 14(5), 514523. Michigan State Police-Criminal Justice Information Center. (2009). Crime in Michigan-2007 th uniform crime report (46 edition). Retrieved December 10, 2009, from http://www.michigan.gov/msp/0,1607,7-123-1645_3501_4621-243369--,00.html Miller, C., & Ahmad, Y. (2000). Collaboration and partnership: An effective response to complexity and fragmentation or solution built on sand? International Journal of Sociology and Social Policy, 20, 1–39. Miller, C., Freeman, M., & Ross, N. (2001). Inter-professional Practice in Health and Social Care. London: Arnold. Milliken, J. F., & Martins, L. L. 1996. Searching for common threads: Understanding the multiple effects of diversity in organizational groups. Academy of Management Review, 21: 402–433. Morgan, S. (2003). Case Management and Assertive Outreach, In B. Hannigan & M. Coffey (eds), The Handbook of Community Mental Health Nursing, London: Routledge. Myaard, M. J., Crawford, C., Jackson, M., & Alessi, G. (2000). Applying behavior analysis within the wraparound process: A multiple baseline study. Journal of Emotional and Behavioral Disorders, 8, 216–229. Nation, M., Crusto, C., Wandersman, A., Kumpfer, K.L., Seybolt, D., Morrissey-Kane, E., & Davis, K. (2003). What works in prevention: Principles of effective prevention programs. American Psychologist, 58(6/7), 449-456. National Advisory Mental Health Council. (2001). Blueprint for change: Research on child and adolescent mental health (NIH Publication No. 01–4985). Washington, DC: Author. National Dissemination Center for Children with Disabilities. Retrieved November 16, 2010 from http://www.nichcy.org/Disabilities/Specific/Pages/EmotionalDisturbance.aspx. Nelson, E.C., Batalden, P.B., Huber, T.P., Mohr, J.J., Godfrey, M.M., Headrick, L.A., Wasson, J.H. (2002). Microsystems in health care: Part 2. Creating a rich information environment. Joint Commission Journal on Quality and Safety 29, 5–15. 208 New Freedom Commission on Mental Health. (2003). Achieving the promise: Transforming mental health care in America. Final report (DHHS Pub. No. SMA-03–3832). Rockville, MD: Author. Nichols, L.O., DeFriese, A.M., & Malone, C.C. (2002). Team process. In G.D. Heinemann & A.M. Zeiss (Eds.) Team performance in health care: Assesssment and development (pp.71-88). New York: Kluwer Academic/Plenum Publishers. Nordness, P. D., & Epstein, M. H. (2003). Reliability of the Wraparound Observation Form— Second Version: An instrument designed to assess the fidelity of the Wraparound approach. Mental Health Services Research, 5, 89-96. O’Connor, K. M. (1998). Experiential diversity in groups: Conceptualizing and measuring variation among teammates. In D. H. Gruenfeld (Ed.), Research on managing groups and teams (Vol. 1, pp. 167–182). Greenwich, CT: JAI Press. Odegard, A. (2007). Time used on inter-professional collaboration in child mental health care. Journal of Inter-professional Care, 21(1), 45 – 54. Oetting, E.R., Donnermeyer, J.F., Plested, B.A., Edwards, R.W., Kellly, K., & Beauvais, F. (1995). Assessing community readiness for prevention. The International Journal of the Addictions, 30(6), 659-683. Olson, E. & Eoyang, G. (2001). Facilitating organization change: Lessons from complexity science. San Francisco: Jossey-Bass/Pfeiffer. Okamoto, S.K. (2001) Interagency collaboration with high-risk gang youth. Child and Adolescent Social Work Journal, 18, 5–19. Paul, S., & Peterson, C. Q. (2001). Inter-professional collaboration: Issues for practice and research. Occupational Therapy in Health Care, 15, 1 – 12. Peck, J.S., Sheinberg, M., & Akamatsu, N.M. (1995) Forming a consortium: A design for interagency collaboration in the delivery of service following the disclosure of incest. Family Process, 34, 287-302. Pike, A. W., McHugh, M., Canney, K. C., Miller, N. E., Reiley, P., & Seibert, C. P. (1993). A new architecture for quality assurance: nurse-physician collaboration. Journal of Nursing Care Quality, 7(3), 1-8. Plsek, P. E., &Wilson, T. (2001). Complexity, leadership, and management in health care organisations. British Medical Journal, 323, 746-749. Polivka, B.J., Dresbach, S.H., Heimlich, J.E., & Elliot, M. (2008). Interagency relationships among rural early intervention collaboratives. Public Health Nursing, 18(5), 340-349. Power, A. K. (2009). Focus on transformation: A public health model of mental health for the 21st century. Psychiatric Services, 60(5), 580–584. 209 Preskill, H., & Torres, R. (1999). The role of evaluative enquiry in creating learning organizations. In M. Easterby-Smith, J. Burgoyne, and L. Araujo (Eds.) Organizational learning and the learning organization, (pp 92-114). London: Sage Publications. Quinn, K., & Cumblad, C. (1994). Service providers’ perceptions of interagency collaboration in their communities. Journal of Emotional and Behavioral Disorders, 2, 109–116. Ranade, W. (1998). Making sense of multi-agency working. Sustainable Cities Research Institute. Reeves, S., Freeth, D., Glen, S., Leiba, T., Berridge, E., Herzberg, J. (2006) Delivering practicebased inter-professional education to community mental health teams: understanding some key issues. Nurse Education in Practice, 6, 246-253. Reeves, S., Rice, K., Gotlib, L., Lee-Miller, K., Kenaszchuk, C., & Zwarenstein, M. (2009a). Inter-professional interaction, negotiation and non-negotiation on general internal medicine wards: An ethnographic. Journal of Inter-professional Care, 23, 1-13. Reeves S, Zwarenstein M, Goldman J, Barr H, Freeth D, Hammick M, Koppel I. (2009b). Interprofessional education: effects on professional practice and health care outcomes (Review). The Cochrane Library, Issue 4. Retrieved November 16, 2010 from http://www.thecochranelibrary.com. Rice K, Zwarenstein M, Conn LG, Kenaszchuk C, Russell A, Reeves S. (2010). An intervention to improve inter-professional collaboration and communications: a comparative qualitative study. Journal of Inter-professional Care, 24(4), 350-61. Rivard, J., Johnsen, M., Morrissey, J., & Starrett, B. (1999). The dynamics of interagency collaboration: How linkages develop for child welfare and juvenile justice sectors in a system of care demonstration. Journal of Social Service Research, 25(3), 61-82. Robinson, L.M. (2005) Promoting multidisciplinary relationships: A pragmatic framework for helping service providers to work collaboratively. Canadian Journal of Community Mental Health, 24, 155-127. th Rogers, E. (2003). Diffusion of innovations (5 ed.). New York: Free Press. Rossen, E.K., Bartlett, R., Herrick, C.A. (2008). Interdisciplinary collaboration: The need to revisit. Issues in Mental Health Nursing, 29, 387-396. Salmon, G. (2004). Multi-agency collaboration: The challenges for CAMHS. Child and Adolescent Mental Health, 9(4), 156-161. Sandfort, J. (1999). The structural impediments to human service collaboration: Examining welfare reform at the front lines. Social Service Review, 73(3), 314-339. Sargeant, J., Loney, E., & Murphy, G. (2008). Effective inter-professional teams: “Contact is not enough” to build a team. Journal of Continuing Education in the Health Professions, 28(4), 228–234. 210 Satin, D. G. (1994). A conceptual framework for working relationships among disciplines and the place of interdisciplinary education and practice: clarifying muddy waters. Gerontology & Geriatrics Education, 14(3), 3-24. Schoenwald, S. K., & Hoagwood, K. (2001). Effectiveness, transportability, and dissemination of interventions: What matters when? Psychiatric Services, 52, 1190–1197. Schon, D. A. (1983). The Reflective Practitioner. New York: Basic Books. Senge, P. M. (2006). The fifth discipline. New York: Doubleday Currency. Sessa, V., & London, M. Work group learning: Understanding, improving and assessing how groups learn in organizations. New York: Lawrence Erlbaum Associates. Siegler, E. L. & Whitney, F. W. (1994). Nurse-Physician collaboration. Care of adults and the elderly. New York: Stringer Publishing Company. Simmons, S., Coid, J., Joseph, P., Marriot, S., & Tyrer, P. (2001). Community mental health team management in severe mental illness: A systematic review. British Journal of Psychiatry, 178, 497-502. Smith, B.D. & Mogro-Wilson, C. (2008) Inter-agency collaboration. Administration in Social Work, 32(2), 5 — 24 Stelk, W., & Slaton, E. (2010). The role of infrastructure in the transformation of a childadolescent mental health system. Administration and Policy in Mental Health, 37, 100110. Sterman, J.D. Learning from evidence in a complex world. (2006). American Journal of Public Health, 96(3), 505-514. Stringer, E. (2007). Action research (3rd ed.). Los Angeles: Sage. Stroul, B., & Friedman, R. M. (1986). A system of care for children and youth with severe emotional disturbances. Washington, DC: Georgetown University Child Development Center, CASSP Technical Assistance Center. Suter, J. C., & Bruns, E. J. (2008). Narrative review of wraparound outcome studies. In E. J. Bruns & J. S. Walker (Eds.), The Resource Guide to Wraparound. Portland, OR: National Wraparound Initiative, Research and Training Center for Family Support and Children’s Mental Health, Portland State University. Suter, J. C., & Bruns, E. J. (2009). Effectiveness of the wraparound process for children with emotional and behavioral disorders: A meta-analysis. Clinical Child and Family Psychology Review, 12, 336-351. Swan, J., Newell, S., Scarbrough, H., & Hislop, D. (1999) Knowledge management and innovation: networks and networking. Journal of Knowledge Management 3(4), 262–275. 211 Taylor, P.J., Russ-Eft, D.F. & Chan, D.W.L. (2005) A meta-analytic review of behavior modeling training. Journal of Applied Psychology, 90(4), 692–709. Thomas, D., & Ely, R. (1996). Making differences matter: A new paradigm for managing diversity. Harvard Business Review, 74, 79–90. Thompson, A.M., Perry, J.L., Miller, T.K. (2007). Conceptualizing and measuring collaboration. Journal of Public Administration Research and Theory, 19(1), 23-56. Tajfel, H., & Turner, J. C. (1979). An integrative theory of intergroup conflict. In W. G. Austin & S. Worchel (Eds.), The social psychology of intergroup relations (pp 33–47). Montery, CA: Brooks/Cole. Tsui, A. S., Egan, T. D., & Xin, K. R. 1995. Diversity in organizations: Lessons from demography research. In M. M. Chemers & S. Oskamp (Eds.), Claremont symposium on applied social psychology— Diversity in organizations: New perspectives for a changing workplace, vol. 8: 191–219. Thousand Oaks, CA: Sage. Power, A. K. (2009). Focus on transformation: A public health model of mental health for the 21st century. Psychiatric Services, 60(5), 580–584. van Dam, K. (2005). Employee attitudes toward job changes: An application and extension of Rusbult and Farrell’s investment model. Journal of Occupational and Organizational Psychology, 78, 253-272. Van der Vegt, G.S., & Bunderson, J.S. (2005). Learning and performance in multidisciplinary teams: The importance of collective team identification. Academy of Management Journal, 48, 532-547. Van Offenbeek, M., & Koopman, P. (1996). Interaction and decision making in project teams. In M. A. West (Ed.), Handbook of work group psychology (pp. 159-187). London: Wiley. Uhl-Bien M, Marion R, McKelvey B. (2007). Complexity leadership theory: Shifting leadership from the industrial age to the knowledge era. The Leadership Quarterly, 18, 298–318. U.S. Census Bureau. (2009). American fact finder, data profile-Saginaw City, Michigan Retrieved December 10, 2009, from http://quickfacts.census.gov/qfd/staes/26/26145.html. U.S. Department of Health and Human Services. (1999). Mental Health: A Report of the Surgeon General—Executive Summary. Rockville, MD: U.S. Department of Health and Human Services, Substance Abuse and Mental Health Services Administration, Center for Mental Health Services, National Institutes of Health, National Institute of Mental Health, U.S. Department of Health and Human Services. (2003). The President’s New Freedom Commission on Mental Health. .Achieving the promise: Transforming mental health care in America. Rockville, MD: The U.S. Department of Health and Human Services. 212 Van Den Berg, J., & Grealish, E. M. (1996). Individualized services and supports through the wraparound process: Philosophy and procedures. Journal of Child and Family Studies, 5, 1, 7-21. Van der Vegt, G.S. & Bunderson, J.S. (2005). Learning and performance in multidisciplinary teams: The importance of collective team identification. Academy of Management Journal, 43(3), 535-547. Walker, J. S., & Koroloff, N. (2007). Grounded theory and backward mapping: Exploring the implementation context for wraparound. Journal of Behavioral Health Services & Research, 34, 443-458. Walker, J. S., Koroloff, N., & Schutte, K. (2003). Implementing high-quality collaborative Individualized Service/Support Planning: Necessary conditions. Portland OR: Research and Training Center on Family Support and Children’s Mental Health. Walker, J. S., & Schutte, K. M. (2004). Practice and process in wraparound teamwork. Journal of Emotional and Behavioral Disorders, 12, 182–192. Walker, J.S., & Schutte, K.M. (2005). Quality and individualization in wraparound team planning. Journal of Child and Family Studies, 14(2), 251-267. Wandersman, A., Duffy, J., Flaspohler, P., Noonan, R., Lubell, K., Stillman, L…& Saul, J. (2008). Bridging the gap between prevention research and practice: The interactive systems framework for dissemination and implementation. American Journal of Community Psychology, 41, 171-181. Way, D. O., Busing, N., & Jones, L. (2000). Implementation strategies: “Collaboration in primary care-family doctors & nurse practitioners delivering shared care”. The Ontario College of Family Physicians. Way, D. O. & Jones, L. (1994). The family physician-nurse practitioner dyad: indications and guidelines. Canadian Medical Association, 151(1), 29-34. Weick, K.E. (1995) Sensemaking in Organizations. New York: Sage Publications Inc. Wenger, E. (2000). Communities of Practice and Social Learning Systems, Organisation, 7(2), 225 - 246. West, M. A., Borrill, C. S., & Unsworth, K. L. (1998). Team effectiveness in organizations. In C. L. Cooper&I. T. Robertson (Eds.), International review of industrial and organizational psychology (Vol. 13, pp. 1–48). New York: John Wiley & Sons. Wiersema, M. F., & Bantel, K. A. (1992). Top management team demography and corporate strategic change. Academy of Management Journal, 35: 91–121. Williams, B., & Hummelbrunner, R. (2010). Systems Concepts in Action. Stanford, California, Stanford University Press. 213 Williams, K., & O’Reilly, C. 1998. The complexity of diversity: A review of forty years of research. In R. I. Sutton & B. M. Staw (Eds.), Research in organizational behavior (pp. 77–140). Greenwich, CT: JAI Press. Winters, N.C., & Metz, W.P. (2009). The wraparound approach in systems of care. Psychiatric Clinics of North America, 32, 135–151. Wittenbaum, G., Hubbell, A., & Zuckerman, C. (1999). Mutual enhancement: Toward an understanding of the collective preference for shared information. Journal of Personality and Social Psychology, 77, 967–978. Xyrichis, A., & Lowton, K. (2008). What fosters or prevents inter-professional teamworking in primary and community care? A literature review. International Journal of Nursing Studies, 45, 140-153. Zahner, G. E., Pawelkiewicz, W., DiFrancesco, J. J., & Adnopoz, J. (1992). Children’s mental health service needs and utilization patterns in an urban community: An epidemiological assessment. Journal of the American Academy of Child and Adolescent Psychiatry, 31, 951–960. Zimmerman, B., Lindberg, C., & Plsek, P. E. (1998). Edgeware: Insights from complexity science for health care leaders. Irving, TX: VHA. Zwarenstein, M., & Reeves, S. (2006). Knowledge translation and inter-professional collaboration: Where the rubber of evidence-based care hits the road of teamwork. Journal of Continuing Education for Health Professionals, 26, 46-54. Yelon, S.L. and Ford, J.K. (1999) Pursuing a multidimensional view of transfer. Performance Improvement Quarterly, 12, 58–7. Yelon, S., Sheppard, L., Sleight, D. and Ford, J.K. (2004) Intentions to transfer: how do autonomous professional become motivated to use new ideas? Performance Improvement Quarterly, 17, 82–103. Zwarenstein, M., & Reeves, S., & Perrier, L. (2004). Effectiveness of pre-licensure interdisciplinary education and post-licensure collaborative interventions. In I. Oandasan, I. et al. Interdisciplinary education for collaborative, patient-centred practice: Research and findings report. Retreived October 15, 2010 from http://www.ferasi.umontreal.ca/eng/07_info/IECPCP_Final_Report.pdf Zweben, J. E. (1993). Dual diagnosis: Key issues for the 1990's. Psychology of Addictive Behaviors, 7, 168-172. 214