I DO NOT THINK IT MEANS WHAT YOU THINK IT MEANS: PROBLEM DEFINITIONS AND COLLABORATIVE RELATIONSHIPS IN COALITIONS By Jennifer Lawlor A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Psychology--Doctor of Philosophy 2019 ABSTRACT I DO NOT THINK IT MEANS WHAT YOU THINK IT MEANS: PROBLEM DEFINITIONS AND COLLABORATIVE RELATIONSHIPS IN COALITIONS By Jennifer Lawlor Community psychologists frequently engage with coalitions in the study of community life. There is still little agreement on the way these organizations should be defined within the field and how they can support change. In my second chapter, I systematically review the literature within community psychology to define coalitions. I identify three types of coordination that they primarily engage in: knowledge coordination, negotiated coordination, and action coordination. Problem definition is one issue that arises in knowledge coordination among coalition members. Problem definitions can be understood as mental models and captured through using fuzzy cognitive maps. The way each individual defines the problem the group works on is often tied to collaborative behavior among coalition members. This brought me to two research questions: (1) In what ways are mental models similar or different within a coalition? (2) To what extent does mental model structure and content predict collaboration within a coalition? To address these questions, I interviewed members of a coalition to capture their mental models and surveyed them to capture their collaborative ties and demographics. To answer my first question, I assessed participants’ mental models in terms of their content, structure, and function. Participants varied across each of these, but converged on a few key concepts. These findings suggest that mental modeling processes can identify differences among participants that might be used to support further dialogue among coalition members about the problem they work on. To answer my second research question, I employed an exponential random graph model using mental model similarity to predict collaborative network ties. Mental model similarity did not predict collaboration, but length of time participants have been in the coalition did emerge as a significant predictor of collaboration. These findings suggest a need for future research to assess predictors of collaboration in greater depth. I conclude with a summative discussion of the findings from each of my research questions, discussing implications for coalition practice, methods for studying them, and theories regarding coalitions. For Aires, Joe, Rini, & Mary. Thank you for walking the road with me. iv ACKNOWLEDGMENTS Writing a dissertation has stretched me in ways I could not have imagined. I owe my successful completion of this journey to so many people who walked the road with me. My friends and colleagues were an invaluable sounding board through the ups and downs of research. In particular, my cohort, Fairview housemates, and the Bro-Snyder- Metta-Singer family deserve special thanks. I love you all and I am so thankful to have found you during my time in Lansing. My parents have been tremendous cheerleaders throughout my K-PhD education. They taught me to be curious and to love learning and tolerated every up and down of graduate school. They have truly seen everything and supported me anyway. Without them, I never could have made it to this point. Finally, I am grateful to the university and community groups that made this possible. The Fairweather Fund, the MSU graduate school’s Research Enhancement Award, and the MSU Dissertation Completion Fellowship provided essential funding to complete this work. My committee members provided many valuable critiques of this document and showed me what the work could be. My coalition partners kindly welcomed me into their community and generously gave their time and support to help me complete this project. v TABLE OF CONTENTS LIST OF TABLES ................................................................................................................ ix LIST OF FIGURES .............................................................................................................. x CHAPTER 1 .......................................................................................................................... 1 REFERENCES ..................................................................................................................... 3 CHAPTER 2 ......................................................................................................................... 5 Methods .................................................................................................................... 9 Search process ........................................................................................................ 10 Data Extraction ....................................................................................................... 11 Results & Discussion ............................................................................................... 13 Coalition Characteristics ......................................................................................... 13 Thematic Map ......................................................................................................... 15 Limitations ............................................................................................................. 22 Future Directions ................................................................................................... 23 Conclusions ............................................................................................................ 25 APPENDIX ........................................................................................................................ 28 REFERENCES .................................................................................................................... 31 CHAPTER 3 ....................................................................................................................... 39 Introduction ........................................................................................................... 39 Problem definition in coalitions ............................................................................ 39 Problem definition and collaboration .................................................................... 44 Problem definitions as mental models .................................................................. 47 Conclusions & hypotheses ...................................................................................... 56 REFERENCES ................................................................................................................... 59 CHAPTER 4 ....................................................................................................................... 64 Introduction ........................................................................................................... 64 Site .......................................................................................................................... 64 Midwest County demographics .................................................................. 65 Participants ............................................................................................................ 66 Sampling...................................................................................................... 66 Recruitment procedures ............................................................................. 66 Participant demographics ........................................................................... 67 Procedure ............................................................................................................... 67 vi Informed consent ........................................................................................ 67 Interviews .................................................................................................... 68 Pre-defined concepts ................................................................................... 70 Surveys ........................................................................................................ 72 Standardizing concepts .......................................................................................... 72 Analyses .................................................................................................................. 74 Summary of concepts .................................................................................. 76 Model structure ........................................................................................... 76 Model function ............................................................................................ 78 Results .................................................................................................................... 80 Content ................................................................................................................... 80 Mental model structure .......................................................................................... 85 In-degree centrality ..................................................................................... 86 Out-degree centrality .................................................................................. 87 Mental model function ........................................................................................... 91 Sector ...................................................................................................................... 92 Content ........................................................................................................ 92 Structure ...................................................................................................... 93 In-degree centrality ..................................................................................... 93 Out-degree centrality .................................................................................. 94 Function ...................................................................................................... 95 Discussion .............................................................................................................. 98 Sector analysis ....................................................................................................... 101 Limitations ........................................................................................................... 103 Future directions .................................................................................................. 104 Conclusions .......................................................................................................... 106 APPENDICES .................................................................................................................. 107 APPENDIX A: RECRUITMENT MATERIALS .................................................... 108 APPENDIX B: IRB APPROVAL AND EXEMPT DESIGNATION ....................... 113 APPENDIX C: CONSENT DOCUMENT............................................................... 115 APPENDIX D: MENTAL MODEL INTERVIEW PROTOCOL ............................. 117 APPENDIX E: SURVEY QUESTIONNAIRE & CODEBOOK ............................... 121 APPENDIX F: SENSITIVITY ANALYSIS ............................................................ 129 APPENDIX G: EXAMPLE FUNCTIONAL ANALYSIS OF A FUZZY COGNITIVE MAP ...................................................................................................................... 132 REFERENCES ................................................................................................................. 134 CHAPTER 5 ..................................................................................................................... 138 Introduction ......................................................................................................... 138 Variables ............................................................................................................... 138 Collaboration ............................................................................................. 138 vii Model similarity ........................................................................................ 139 Control variables .................................................................................................. 142 Length of time in the coalition .................................................................. 142 Sector ......................................................................................................... 143 Race & gender ........................................................................................... 143 Social network data collection ............................................................................. 144 Analysis ................................................................................................................. 145 Exponential random graph modeling ........................................................ 145 Results .................................................................................................................. 146 Network descriptives................................................................................. 146 Null model .................................................................................................. 147 Covariate model ......................................................................................... 147 Full model ................................................................................................. 148 Goodness of fit........................................................................................... 148 Discussion ............................................................................................................. 151 Limitations ............................................................................................................ 155 Future directions ................................................................................................... 156 Conclusions .......................................................................................................... 158 REFERENCES ................................................................................................................. 160 CHAPTER 6 ..................................................................................................................... 164 viii LIST OF TABLES Table 1. List of journals and article counts ....................................................................... 11 Table 2. Thematic map of coalition definitions; coalition function ................................. 15 Table 3. Code book .......................................................................................................... 29 Table 4. Summary of analyses to describe mental models (table structure taken from Yoon & Jetter, 2016).......................................................................................................... 75 Table 5. Comparison of graph metrics for social network analysis and fuzzy cognitive maps ........................................................................................................................................ 77 Table 6. Structural differences among sectors ................................................................. 88 Table 7. Liberal coding sensitivity analysis .................................................................... 130 Table 8. Missingness sensitivity analysis ........................................................................ 131 Table 9. Benefits and drawbacks of model comparison options .................................... 141 Table 10. Model estimates, maximum likelihood estimate (standard error)................. 148 Table 11. Goodness of fit for model statistics ................................................................. 150 Table 12. Goodness of fit for minimum geodesic distance ............................................ 150 Table 13. Goodness of fit for edgewise shared partner .................................................. 150 Table 14. Goodness of fit for degree ................................................................................ 151 Table 15. Goodness of fit for triad census ........................................................................ 151 ix LIST OF FIGURES Figure 1. Example fuzzy cognitive map ............................................................................ 55 Figure 2. Network of participants with ties representing shared mental model concepts and colors representing participants’ sectors ................................................................... 82 Figure 3. Network of participants with ties representing shared mental model content based on tags and colors representing participants’ sectors ........................................... 82 Figure 4. Relative impact of FAFSA completion level on postsecondary attainment ........................................................................................................................................... 91 Figure 5. Relative impact of achievement on the SAT on postsecondary attainment ........................................................................................................................................... 92 Figure 6. Relative impact of FAFSA completion level on postsecondary attainment ........................................................................................................................................... 97 Figure 7. Relative impact of achievement on the SAT on postsecondary attainment ........................................................................................................................................... 97 Figure 8. Visual representation of the coalition collaboration network ......................... 147 x CHAPTER 1 Coalitions are a vehicle frequently used for bringing together stakeholders to create change in community settings. There has been a great deal of work considering their effectiveness for addressing particular community issues, like substance abuse, HIV prevention, and youth violence (Anderson-Carpenter, Watson-Thompson, Jones, & Chaney, 2017; Ziff et al., 2010; Bess, 2015). However, there is still room to empirically understand the processes by which these groups achieve their goals. One area in particular that remains unknown in this process is the role that individual problem definition plays in coalition functioning and the collaborative relationships among coalition participants. Individual problem definition refers to the way each member of a coalition understands the problem or problems that the coalition works on. A number of theories related to coalition functioning and change processes suggest the importance of problem definition both at the individual and group level for successful collaboration and change outcomes (Lawlor & Neal, 2016; Kania & Kramer, 2011; Foster-Fishman & Watson, 2011; Foster-Fishman, Berkowitz, Lounsbury, Jacobson, & Allen, 2001), but little empirical work has explored the nature of problem definitions or how differences in individual problem definitions may relate to the collaborative relationships that coalition members engage in. Thus, the purpose of this dissertation is to explore how an individual’s understanding of the problem of interest relates to their experiences of collaboration within a coalition. I established two primary objectives for my dissertation. First, I consider the variety of mental models within a particular problem area in a coalition and the extent to which they converge or diverge. Second, I evaluate the extent to which mental model structure and content relate to collaboration patterns within coalitions. For this 1 objective, I hypothesized that individuals with more similar problem definitions would be more likely to have dyadic collaboration relationships. I present a six chapter structure to achieve these objectives, with the present chapter introducing the project, two chapters focused on reviewing the literature, two chapters presenting methods and findings, and a final chapter discussing the project and findings overall. Chapter two focuses on defining coalitions as institutions within the field of community psychology and evaluating the ways that they have been studied in the field. Chapter three reviews the literature on problem definition and collaboration in coalitions, highlighting the relationship between these two concepts. In chapter four, I describe the methods, analysis, results, and discussion regarding my first research objective. Chapter five includes methods, results, and discussion for my second research objective. Chapter six is summative discussion of the findings from the project as a whole, including implications, and future directions for this work, spanning across chapters. 2 REFERENCES 3 REFERENCES Anderson-Carpenter, K., Watson-Thompson, J., Jones, M.D., & Chaney, L. (2017). Improving community readiness for change through coalition capacity building: Evidence from a multisite intervention. Journal of Community Psychology, 45(4), 486–499. http://doi.org/10.1002/jcop.21860 Bess, K. D. (2015). Reframing Coalitions as Systems Interventions: A Network Study Exploring the Contribution of a Youth Violence Prevention Coalition to Broader System Capacity. American Journal of Community Psychology, 55(3–4), 381– 395. http://doi.org/10.1007/s10464-015-9715-1 Foster-Fishman, P. G., Berkowitz, S. L., Lounsbury, D. W., Jacobson, S., & Allen, N. A. (2001). Building Collaborative Capacity in Community Coalitions: A Review and Integrative Framework. American Journal of Community Psychology, 29(2), 241–261. http://doi.org/10.1023/A:1010378613583 Foster-Fishman, P. G., & Watson, E. R. (2011). The ABLe change framework: a conceptual and methodological tool for promoting systems change. American Journal of Community Psychology, 49(3–4), 503–16. http://doi.org/10.1007/s10464-011-9454-x Kania, B. J., & Kramer, M. (2011). Collective Impact. Stanford Social Innovation Review, 1(9), 36–41. Lawlor, J. A., & Neal, Z. P. (2016). Networked Community Change: Understanding Community Systems Change through the Lens of Social Network Analysis. American Journal of Community Psychology. http://doi.org/10.1002/ajcp.12052 Ziff, M.A., Willard, N., Harper, G.W., Bangi, A.K., Johnson, J., Ellen, J.M., the A. M. T. N. for H. I. (2010). Connect to Protect researcher-community partnerships: Assessing change in successful collaboration factors over time. Global Journal of Community Psychology Practice, 1(1), 32–39. 4 CHAPTER 2 Coalitions have a long history as part of the field of community psychology. As early as 1989, American community psychologists were suggesting that coalitions could be a critical space for collective power in communities (Heller, 1989). Since then, the field has seen a wide range of scholarship addressing many aspects of coalitions, from their development to potential impacts on the individuals participating in them and the outcomes they achieve in the communities in which they operate (Nelson, 1994; McMillan, Florin, Stevenson, Kerman, & Mitchell, 1995; Anderson-Carpenter, Watson- Thompson, Chaney, & Jones, 2016). Coalitions’ activities also fit closely with the values of community psychology as vehicles for empowerment, collective power, and promotion of wellness in communities (Rappaport, 1977; Goodman, Wandersman, Chinman, Imm, & Morrissey, 1996; McMillan, Florin, Stevenson, Kerman, & Mitchell, 1995). While community psychologists often work with coalitions, these entities engage in a wide range of activities and structures that have not been well defined within the field. The present study illuminates the nature of these entities in terms of their characteristics as well as how community psychologists define them in their work. This will provide additional clarity about their place within the field of community psychology and facilitate a discussion about future areas of exploration in the study of coalitions. Coalitions have been defined in a variety of ways in community psychology. We present several theoretical definitions here to demonstrate the ways in which coalitions are understood in the field. Chavis (2001) suggests that they include participants with diverse interests, histories, and power dynamics, they disperse resources among 5 participating institutions to achieve common goals, explicit or implicit pretext of equality, and they are inherently paradoxical. Wolff (2001) defines them according to the following set of criteria: The coalition is composed of community members; it focuses mainly on local issues rather than national issues; it addresses community needs, building on community assets; it helps resolve community problems through collaboration; it is community-wide and has representatives from multiple sectors; it works on multiple issues; it is citizen influenced if not necessarily citizen driven; and it is a long term, not ad hoc, coalition (p. 166). This definition differs from Chavis (2001) in its specification of who the participants are, the types of problems they address, and how they function. Himmelman (2001) defines them broadly, suggesting they are “an organization of organizations working together for a common purpose” (p. 277). Others describe them in terms of the type of work that they aim to achieve, suggesting “Coalitions are a commonplace approach to the pursuit of health-related structural change (Mizrahi & Rosenthal, 2001; Roussos & Fawcett,2000; Watson-Thompson, Fawcett, & Schultz, 2008), though are often used in communities as planning and coordinating bodies rather than as grassroots mobilization efforts and agents of social change (Butterfoss, 2006; Roussos & Fawcett, 2000; Valente, Chou, & Pentz, 2007). Coalitions are temporary or enduring collaborations among diverse individuals, organizations, and constituents who agree to work jointly toward a common goal (Butterfoss, 2006) (Miller, Reed, Francisco, Ellen, & the ATN 079 Protocol Team for the Adolescent Medicine Trials Network for HIV/AIDS Interventions, 2012, p. 2-3). 6 Each of these definitions differs in what they see as coalitions and all share common elements. These include participation from a diverse group of stakeholders and collective work toward achieving common goals. As research on coalitions is pervasive, there are many definitions of them as each author understands these entities somewhat differently. Coalitions address a broad spectrum of issues including youth violence, substance use, poverty reduction, and educational attainment (Bess, 2015; Evans Rosen, Kesten, & Moore, 2014; Anderson-Carpenter, Watson-Thompson, Chaney, & Jones, 2016). Many have also demonstrated successes in both shifting the way collaborative work happens in communities as well as outcomes on these key community issues. Some examples include a coalition that fostered a variety of interorganizational alliances, another that reduced the rate of low infant birth weight, and another that changed community policies and practices around youth binge drinking (Foster- Fishman, Salem, Allen, & Fahrbach, 2001; Darnell et al, 2013; Anderson-Carpenter, Watson-Thompson, Chaney, & Jones, 2016). However, there are still open questions about how coalitions can effectively act as mobilizers for change in community settings and there is room for empirical research to examine theories about how they operate (Foster-Fishman, Berkowitz, Lounsbury, Jacobson & Allen, 2001). For example, Chavis (2001) suggests that coalitions’ members must manage competing interests between organizations or groups they represent and the overall agenda of the coalition. In addition, the coalition structures can reinforce power structures, where large organizations are better able to participate than individual community members. As issues like this come up, it is important to consider how coalitions operate in 7 communities and what functions they serve in order to generate processes that can achieve goals. While there has been substantial empirical work in community psychology focused on coalitions and there have been increasing calls for communities to take new approaches to collaborative work, there has not been an update to the definition of coalitions or a systematic assessment of what coalitions look like in practice. The last time this topic was brought into critical focus was in a 2001 special issue of the American Journal of Community Psychology. Since the publication of the special issue, there has not been a coordinated effort to examine coalitions within community psychology in a systematic way. In particular, there has not been work that addresses how they can be defined and what their characteristics look like in empirical research across the field. We aim to address that gap and update the literature in this paper. Researchers and practitioners in community psychology make suggestions about what coalitions can do for communities. For example, Bess (2015) suggests that they can be conceptualized as interventions to community systems. However, a specific understanding of what they look like in practice is critical for evaluating their role in community life and separating them from other similar entities like systems of care (Suarez, Belcher, Briggs, & Titus, 2012). Further, outside of the formal coalition literature in community psychology, others are suggesting processes for collaboration and coordination among stakeholders in communities (Foster-Fishman & Watson, 2011; Kania & Kramer, 2011). Carefully considering the nature of coalitions in practice can facilitate the process of identifying whether these recent strategies are appropriate for a given coalition setting. To do this, it is important to clarify the nature of this type of 8 community entity as its role may be changing and as research about it moves in new directions. In addition, ‘coalition’ is a commonly used term in other fields, including sociology (e.g., McGimpsey, Bradbury, & Sartori, 2017), public administration (e.g., Meyfroodt, Desmidt, & Goesminee, 2017), and political science (e.g., Miller & Curtin, 2011). It is important to establish what this term means in the disciplinary context of Community Psychology in order to effectively communicate with other disciplines and to contextualize the coalitions research from other disciplines. Thus, this paper evaluates the ways in which community psychologists define coalitions in their work and describe the coalitions they work with. This will establish an understanding of coalitions that is inclusive of the overall perspective of scholars in the field. Specifically, we answer two research questions: (1) What are the characteristics of coalitions that community psychologists study? (2) What are the themes in the way authors define coalitions in their work? To answer these questions, we employ a systematic review approach, searching for articles in community psychology that include studies of coalitions, extracting data describing the coalitions under study and how the authors see coalitions. We use these data to compute descriptive statistics about the characteristics of coalitions and thematic analysis to evaluate how authors define them. Methods Our systematic review process parallels similar review papers in community psychology, like Davenish, Hooley, and Mellor (2017) and Neal and Neal (2017). The process begins with a determination of relevant sources to create a pool, an inclusion process to determine which articles are appropriate, and a data extraction process from the articles. 9 Search process We first established a pool of journals to search that specifically serve the field of community psychology. In order to find community psychology-specific journals, we searched the Society for Community Research and Action’s list of relevant journals for community psychology (SCRA, 2017). This generated a pool of seven journals: The American Journal of Community Psychology, The Journal of Community Psychology, Community Psychology in Global Perspective, The Global Journal of Community Psychology Practice, The Journal of Community and Applied Social Psychology, The Journal of Rural Community Psychology, and The Australian Community Psychologist. Next, we conducted electronic searches of each journal, looking for the term “coalition” or “collaborative” (when used as a noun, this is frequently used as a synonym for coalition in community psychology literature) in the title or abstract. We limited the keywords to the title and abstract in order to exclude articles that only make a passing reference to a coalition. We also limited the articles to those written since 2000, which narrows the pool of articles to those written since The American Journal of Community Psychology published a special issues on coalitions, which may have influenced subsequent work in this area. See Table 1 for a list of the number of articles from each of the journals included in the search process. This initial pool included 144 articles. Next, each article in the pool was evaluated for inclusion criteria (see table 1 for an overview of the inclusion process). To be included, the articles needed to (1) report that it was studying a coalition (or collaborative) and (2) be an empirical paper. The first two authors independently evaluated the title and abstract for each article for these criteria and came to consensus about any discrepancies in coding patterns. Any articles 10 for which the coders were unclear about inclusion criteria were retained for the data extraction step, at which point the full text was assessed for inclusion. This process yielded a pool of 57 articles. Four articles were excluded for not reporting an empirical study of a coalition, and an additional 83 were excluded because they did not focus their study on coalitions. Finally, each of the coders read each included article to verify its inclusion. During this process, another 6 articles were removed because they did not have a focus on coalitions and 16 articles were removed because they did not include an empirical study of a coalition. Table 1. List of journals and article counts Journal Initial After article set inclusion After Full Reading coding American Journal of Community Psychology Journal of Community Psychology Community Psychology in Global Perspective Global Journal of Community Psychology Practice Journal of Community and Applied Social Psychology Journal of Rural Community Psychology Australian Community Psychologist 82 36 3 15 0 1 7 Total: 144 Data Extraction 34 17 0 5 0 0 1 57 19 13 0 2 0 0 0 34 Next, the same two coders extracted data from each article. Data extraction focused on two areas listed in the research questions: the characteristics of the coalitions being studied and the way in which the authors defined coalitions. Data extracted to evaluate coalition characteristics included things like who participates in coalitions (i.e., 11 community members, organizational representatives, cross-sector participants), on what scale they do their work (i.e., local, state level, national, international), what kind of work do they do (i.e., program design, interventions, prevention, organizational networking). This approach follows the types of information commonly presented in coalition articles and is further informed by the types of information included in some of the common definitions included in the introduction. These data were highly structured because articles tended to present it in similar ways that fit into pre-determined categories (see the appendix for the data extraction codebook). We also extracted the definitions of coalitions presented in the articles. These were left open-ended during the extraction process, as they varied greatly in their content and we wanted to be able to evaluate the themes among them. As such, we extracted direct quotes from the articles that included the author’s definition. After training to establish a common understanding of the data points to be extracted, each coder independently read each article and recorded the relevant data points for each area. Next, the coders met to discuss discrepancies and came to consensus on all data points from extraction. To analyze the data, we applied descriptive statistics to each of the data points we extracted with the exception of the definitions of coalitions presented in the articles. To answer our second research question we applied a theoretical thematic analysis approach to the extracted data that defined coalitions and their functions. For this analysis we employed the process found in Braun & Clarke (2006). We chose a thematic approach to be able to analyze the latent content and theoretical importance of how authors were describing and defining coalitions. Our analysis process started with in depth reading and re-reading of the extracted definitions for common ideas (Miles and Huberman, 1994) and organizing these data into preliminary themes or groups 12 (Tucket, 2005). This open coding was systematic across all of the data extractions and involved tagging features of the data and collating by relevance to each code. These collated groups of codes were then read for and described as themes. Though each data extraction was coded independently, the themes were grouped across data extractions, and checked for consistency in meaning by referring back to the individual context. We then created a thematic map of the coded data to analyze the content across and between major thematic areas. The result of this mapping is displayed in Table 3 in the results and discussion section. Results & Discussion In order to assess the articles included in our pool, we present the results of our two types of analyses here. First, we present descriptive statistics to discuss the characteristics of coalitions studied in the articles we included. Second, we present the thematic map representing how coalitions are described in articles to further understand what role they serve and how they function. Coalition Characteristics To assess the first research question, we evaluate the characteristics of coalitions here by discussing who participates, the scale on which the coalitions operate, the issues they address, and the approaches they employ in their work. Using these characteristics, we establish a general understanding of what makes up a coalition. In the next section, we will further define coalitions based on what they do as entities in communities. The articles demonstrated some trends in the scale of the work coalitions conduct and the people who comprise their membership. Thirty-one of the articles reviewed included coalitions working at the local level (96.88%) and two reported working at the state level (6.25%). None of the articles reviewed included coalitions working at the 13 national or international level. The articles reported on a variety of participants including community members (18, 56.25%), organizations or organizational representatives (26, 81.25%), and cross-sector representation, meaning that they include individual or organizational representatives from multiple sectors (20, 62.5%). Although authors often did not provide the exact ages of the coalitions, ages reported ranged from less than a year to 41 years. The articles infrequently reported information about coalition governance or specific strategies employed guiding their work. Those that reported about formal strategies for approaching their work were engaging in a few approaches: Communities that Care, Connect to Protect, and the Strategic Prevention Framework. However, it is not clear from the articles reviewed what types of governance structures those may create within coalitions. Coalitions in the included articles tended to focus on several common issues. They most frequently dealt with issues relating to children, youth, and families (56.25%), including youth violence and maternal health. Seven articles (21.88%) reported about coalitions addressing issues related to general health and wellbeing and seven reported about substance use specifically (21.88%). Five articles discussed coalitions that address HIV or sexual health issues (15.63%). Violence and abuse were addressed by coalitions in five articles (15.63%). Several other issues were addressed by coalitions in single articles, including: autism, neighborhood development, food systems issues, and poverty. These issues were not mutually exclusive and coalitions sometimes addressed multiple issues (e.g., youth issues and health issues). These data suggest that coalitions in community psychology can be characterized by a focus on local issues. They bring together a variety of stakeholders, often 14 representing organizations, but also frequently including community members and representatives of multiple sectors. Thematic Map To answer our second research question, we identified three major themes from our thematic analysis of coalition definitions included in the review articles: knowledge coordination, negotiated coordination, and action coordination. Definitions of coalitions described them as dealing with these types of coordination in terms of the problems they deal with, the techniques they employ, and the outcomes they work toward. These types of coordination are not independent of each other and have identified them within a hierarchical structure in which action coordination relies on negotiated coordination, which in turn relies on knowledge coordination. Themes for each type of coordination are summarized in Table 3 below. We will unpack each of the types of coordination and offer examples from the articles we reviewed. We have italicized key components of the quotes we offer as examples to highlight essential concepts for each type of coordination. Table 2. Thematic map of coalition definitions; coalition function Coordinated knowledge Coordinated negotiation Coordinated action Each stakeholder has knowledge of different parts of the system Stakeholders have disagreement about who is responsible for parts of the problem Stakeholders are acting in ways that are not congruous and want to move toward congruous action Types of problems 15 Table 2. (cont’d) Techniques Outcomes Shared measurement or data collection processes, facilitated processes for sharing knowledge Ongoing knowledge exchange, shared definitions of the problems the coalition works on Generate a governance structure capable of facilitating consensus processes Provide services that cannot be implemented by individual members, generate policy agendas Exercise shared voice, collective power, and collective action Come to consensus about shared resources and reducing redundancies in the coalition shared resources and reducing redundancy The first theme, knowledge coordination (KC), refers to the bringing together of stakeholder knowledge in a coalition. Knowledge coordination problems relate to the coalition participants each having unique knowledge of the system that they are trying to change. Coalitions may set outcomes around KC by trying to achieve shared information systems for stakeholders that allow for ongoing knowledge exchange and common definitions of the problem the coalition addresses. To achieve KC outcomes, they can employ techniques to facilitate shared knowledge, like shared measurement or facilitated data collection processes to capture information from all stakeholders and present it in ways that are accessible to all. Watson-Thompson, Fawcett, & Schultz (2008) describe KC at a high level, suggesting that it is a precursor to solving problems and changing systems: 16 Community coalitions provide enhanced resources for community members to define and solve problems with the potential to be powerful enabling systems for community change (p. 25). Miller et al. (2012) offer specific examples of processes for collecting and evaluating knowledge among coalition stakeholders and the community they work with, including: Environmental scan, analyzing root causes of risk for adolescents in a defined geographic area, developing a logic model depicting local causes of risk, and formulating strategic plans and structural change objectives that are linked to the locally identified root causes of risk (p. 380). While Miller et al. (2012) provide a number of strategies for bringing together knowledge, KC can only go as far as the stakeholders whose knowledge is ultimately included and valued. Research with similar types of groups suggests that barriers can arise that inhibit KC. Stakeholders with critical knowledge may be left out of the coalition entirely or some stakeholders who are members of the coalition may be excluded from the group’s KC activities. When stakeholders with key knowledge of the problem do not participate in KC, coalitions may coordinate the knowledge of all their participants, but may still not have the necessary knowledge to fully understand the problems the coalition will address. This exclusion can come from personal choice by stakeholders or by group norms that prevent some participants from engaging fully (Watson & Foster-Fishman, 2012; Gone, 2006). A coalition consisting of only individuals with the same limited knowledge of the problem may not find the most benefit from engaging in KC efforts. Coalitions and researchers working with them must consider effective strategies for ensuring that knowledgeable stakeholders are invited to 17 participate and may consider which compositions of stakeholders lead to effective knowledge exchange to support other types of coordination. The second theme, negotiated coordination (NC), refers to a coalition’s structure and ability to achieve consensus. NC problems deal with the disagreements among stakeholders about ownership of various aspects of the problem and its solution. NC outcomes focus on coming to consensus about shared resources and reducing redundancies in the coalition. Techniques for achieving NC involve building governance structures that make it possible for stakeholders to deliberate about problems and make decisions for further action (described further in the coordinated action section below). This infrastructure makes it possible for stakeholders to work through problems to help participants find common ground and distributing responsibility, which may stem from an initial lack of shared norms or reciprocity (Ostrom, 2005). NC is an essential component for efficient governance, especially in overcoming collective action problems. In NC, coalition members may find they need to make decisions that are discouraged by the current system, but necessary for system level change. For example, stakeholders in a coalition may write shared funding proposals where each organization gets a smaller piece of the pie than if they were to individually compete for funding, but it may fund a larger initiative than any one organization would be capable of managing on their own. This can also be challenging when the cost of coordination is high. For example, when coordination requires intensive participation from staff members, it may take them away from their other responsibilities. Outcomes related to negotiated coordination include making decisions about generating shared resources, reducing redundancy in service provision, and generating more efficient service systems. To make these decisions, coalitions employ techniques 18 that facilitate consensus among stakeholders. They also require some level of knowledge coordination for stakeholders to be able to evaluate the current state of the system. For example, Miller et al. (2012) defines coalitions by their efforts toward establishing a common goal: “Coalitions are temporary or enduring collaborations among diverse individuals, organizations, and constituents who agree to work jointly toward a common goal" (Miller et al., 2012). In order to establish common goals, stakeholders must have awareness of each other’s understanding of how the problem operates within their context as well as the structure within which to agree about a goal. Thus, knowledge coordination is a necessary component of negotiated coordination. Wells, Ward, Feinberg, & Alexander (2008) describe this process in the coalitions they study: Each community’s leaders form a ‘‘prevention board’’ that undergoes training and then systematically assesses local risk and protective factors related to youth. They are then supposed to prioritize problems, select one or more empirically based prevention programs, and evaluate impact over time (p. 97). The coalition boards employ the knowledge acquired from assessing risk and protective factors (KC) in order to set their priorities for action (NC). This example also demonstrates the importance of the individuals who participate in sharing knowledge, as that information dictates the way the group selects priorities. With the appropriate knowledge of the problem and a process for agreeing on an action plan, the board can come to consensus about where they want to place their efforts. The theme of NC is surprising given the lack of discussion about coalition governance when authors described the coalitions they were studying empirically. While some authors described basic coalition structure and indicated that they use strategies 19 like Communities that Care and Connect to Protect, they did not frequently discuss implications of these structures or how they were decided upon. The lack of information about governance in practice may suggest that this is an area in which we do not have very much institutionalized knowledge yet. This may present an opportunity to learn from other fields with rich research about governance, like institutional economics and public administration (Milward & Provan, 2000; Ostrom, 1990). Achieving NC may also generate unintended consequences for coalitions. As members generate consensus about shared resources and responsibilities, they may create structures that make the coalition vulnerable over time. Reducing redundancies among coalition stakeholders can create a space where each stakeholder has an agreed- upon area of a problem that they address. This can increase efficiency avoiding duplication. However, this means that losing a single stakeholder may be a greater threat to coalition functioning than it would be in situations where there is redundancy among stakeholders and suggests that this type of coordination may require intentionality about sustainability. The third theme, action coordination (AC), refers to the process of taking collective action as a coalition. To do this, the coalition often needs to first have KC and NC efforts in place. Problems related to action coordination focus on stakeholders within a coalition acting in ways that are incongruous and wanting to move toward congruous action, where stakeholders establish and carry out actions as a group. Evans, Rosen, Kesten, & Moore (2014) discuss this in their definition of a coalition: Networks, coalitions, alliances and other forms of interorganizational collaboration are seen as more effective strategies for building power to affect the broader systems and policy change needed to reduce the causes of poverty. The 20 basic assumption is that an interorganizational coalition can mobilize and have a greater impact on change processes than could be achieved by organizations acting alone (p. 358). The authors describe how the action of a whole coalition can make change beyond the reach of any individual actor. This coordinated action often includes speaking with a shared voice and exercising collective power. To achieve these goals, they employ techniques like providing services as a group that cannot be managed by any individual stakeholder or generating policy briefs that each member of the group endorses. This type of coordination builds on the first two themes. To coordinate action, stakeholders must have some degree of coordinated knowledge to be able to establish actions that are relevant to the group members and the community. They also need to have negotiated coordination to make decisions about which actions to take, how to take them, and when to take them. An in-practice example from Flewelling et al. (2005) describes the coalition they studied: Coalition coordinators worked with coalition members and community-based organizations to facilitate acceptance and implementation of these practices and to increase overall attention and commitment to substance use prevention efforts in their communities (p. 336). In this example, the coalition uses their power as a group to increase efforts in their community. By using a collective voice, multiple stakeholders can reinforce a message throughout the community to enact changes that no individual stakeholder could create on their own. In situations where coalitions do not approach AC with some level of KC and NC, they may risk stakeholders taking actions that undermine each other. For example, 21 outside of the community psychology coalitions literature, Burns (2007) describes a situation in which multiple stakeholder groups were interested in work around the issue of female genital mutilation. Many stakeholders working NGOs focused on ways to stop female genital mutilation from happening by bringing girls to refuge centers, while medical professionals focused on providing a safe way to participate in the practice at hospitals. These two approaches both dealt with the same issue in the same place, but acted in opposition to each other. Coordinating the knowledge of the issue among all relevant stakeholders, including local women and girls, may have illuminated the intricacies of the problem further. Applying that knowledge in a group consensus process could help establish the roles appropriate for each stakeholder to take on and generate a plan for action that made it possible to act collectively in a coherent way. Limitations These findings should be interpreted in light of some limitations stemming from article inclusion procedures. We limited the scope of articles for inclusion based on year and keywords. This is a matter of scoping as well as practicality in terms of assessing available information. However, there may be some articles that were not included because they use different terms to refer to coalitions. Additionally, some literature published by community psychologists may be located in journals that are not specific to the field. Similarly, we did not include unpublished work in this area, which means this review may exclude some perspectives from practitioners whose work may be located outside scholarly journals, for example in evaluation reports. Although practitioner journals were included in the pool, they may not provide a comprehensive view of in- practice work with coalitions within the field. Finally, we selected the list of relevant journals from the Society for Community Research and Action’s list of journals for 22 community psychologists. While three of the seven included journal are global or non- United States focused, they were selected from a list provided by a US-based organization and may under represent the common international journals. Future studies can build on my work by examining other potential terminology for referring to coalitions in community psychology, considering additional international journals, and expanding the literature pool to include the gray literature. Future Directions In addition to addressing these limitations, these findings suggest several areas for future research relating to each type of coordination. To further explore KC, empirical coalitions research can explore how coalition composition influences the ability to coordinate knowledge, specifically to evaluate strategies and composition types that are most effective for generating comprehensive knowledge of the problem the coalition is assessing. Our findings about coalition characteristics indicate that coalitions currently involve multiple types of stakeholders including community members, organizational representatives, and cross-sector representation. Researchers working with individual coalitions can consider who is represented in the coalitions they work with and the extent to which individuals with relevant knowledge are included in processes for coordinating knowledge. Modeling could be one technique for building knowledge coordination in order to understand the types of problems coalitions address and how diverse members of coalitions think about these problems and their solutions. Mental models are individuals’ cognitive structures that help them to understand the world (Jones, Ross, Lynam, Perez, & leitch, 2011; Johnson-Laird, 2005). Making these models explicit through processes like fuzzy cognitive mapping could make it easier to assess 23 differences in the way participants understand the problem of interest (Kosko, 1986). Combining this with other types of modeling, like agent-based modeling or system dynamics can help to generate a comprehensive representation of the problem incorporating individual group members’ mental models. Coalitions can also use models to assess where various actions may have an impact and use them to inform consensus processes that lead to action. Future research should consider the feasibility of these types of techniques for eliciting knowledge within the coalition and evaluate their utility for informing NC and AC. Research on diversity in groups suggests that collective models produced by diverse groups generate more effective solutions to problems than models produced by homogenous groups (Hong & Page, 2004; Page, 2007). This may suggest that coalitions can improve their KC by ensuring that they bring together stakeholders who hold diverse knowledge of the problem of interest in order to best understand it. Future research involving modeling may explore how the composition of coalition members influences KC. To further explore NC, future research should explore different types of governance in coalitions, including what these structures look like and how they relate to coalition processes and outcomes. This was rarely reported in the papers we reviewed, but it emerged as a key theme in the way community psychologists define coalitions. Reporting this information in empirical articles can make coalition research more interpretable and relatable for individuals working with similarly structured coalitions. Researchers can also draw upon literature from fields like institutional economics and public administration to better understand how groups like coalitions come to consensus and generate and manage collective resources, particularly for problems requiring collective action (Ostrom, 1990). In addition, coalition researchers 24 can explore the relationship between NC and coalition sustainability. Future research may examine how reducing redundancies and coming to agreement about shared resources may influence a coalition’s functioning over time. Reducing redundancies may support coalition efficiency, but could be challenging in situations with high stakeholder turnover. Considering the relationships among these issues may help to develop strategies for NC that support coalition success. Future research regarding AC can explore which of types of actions coalitions are effective at taking. In addition, researchers can explore how knowledge coordination and negotiated coordination conditions lead to the most successful actions. Future research may also examine each of these over time and consider the interplay between each of these themes over time within coalitions. This can demonstrate how different types of coordination ebb and flow over time and how they work together to achieve coalition outcomes. While we suggest that action coordination requires some knowledge and negotiated coordination, over time, these types of coordination may be happening in tandem or cyclically. Conclusions The purpose of this paper was to assess the characteristics of coalitions being studied in community psychology and evaluate how they are currently being defined in the field. Through our systematic review, we have established that coalitions in community psychology tend to engage a variety of community stakeholders to focus on issues at the local level relevant to the prevention and promoting wellness. They also engage in three types of coordination in doing their work: knowledge coordination, negotiated coordination, and action coordination. The types of coordination presented here are the primary functions of coalitions. They require a recognition of the 25 interconnectedness of the coalition participants and the necessity of engaging them in order to achieve coalition goals. Future research should consider how these types of coordination operate in practice and the ways in which community psychologists can further support coalitions in achieving their goals. In addition, future research involving the empirical study of coalitions can elaborate on the details of how the coalition under study operates. 26 APPENDIX 28 Appendix. Code book Table 3. Code book Variable Name Description Values Reference APA formatted reference Open ended Coalition Definition Who participates? Community members Who participates? Organizations Who participates? Cross-sector What issue do they work on? What kind of work do they do? On what scale do they do the work? Local On what scale do they do the work? State On what scale do they do the work? National How do the authors define coalitions in their literature review? Open ended 0 = no 1 = yes 0 = no 1 = yes 0 = no 1 = yes Open ended Open ended Who is involved in the coalition? Are they community members? Who is involved in the coalition? Are they organizations? Who is involved in the coalition? Are they cross-sector? What is the topical focus of their work? E.g., substance abuse, youth violence, community development What do they do in regards to the topic they work on? E.g., prevention, program development/administrati on Are they working at the local level? 0 = no 1 = yes Are they working at the state level? 0 = no 1 = yes Are they working at the national level? 0 = no 1 = yes 29 Table 3 (cont’d) On what scale do they do the work? International How old was/were the coalitions at the time of publication? How is it/are they governed? Do they subscribe to a particular model of coalition functioning? Are they working at the international level? 0 = no 1 = yes How long has the coalition been around? Number in years Open ended Open ended What do they report on coalition governance? E.g., hub organization that coordinates coalition work What model do they follow, if any? Eg., Strategic Prevention Framework, Communities That Care, Collective Impact 30 REFERENCES 31 REFERENCES Anderson-Carpenter, K. D., Watson-Thompson, J., Chaney, L., & Jones, M. (2016). Reducing binge drinking in adolescents through implementation of the strategic prevention framework. American journal of community psychology, 57(1-2), 36- 46. *Anderson-Carpenter, K., Watson-Thompson, J., Jones, M.D., Chaney, L. (2017). Improving community readiness for change through coalition capacity building: Evidence from a multisite intervention. Journal of Community Psychology, 45(4), 486–499. http://doi.org/10.1002/jcop.21860 *Barile, J. P., Darnell, A. J., Erickson, S. W., & Weaver, S. R. (2012). Multilevel Measurement of Dimensions of Collaborative Functioning in a Network of Collaboratives that Promote Child and Family Well-Being. American Journal of Community Psychology, 49(1–2), 270–282. http://doi.org/10.1007/s10464-011- 9455-9 *Bess, K. D. (2015). Reframing Coalitions as Systems Interventions: A Network Study Exploring the Contribution of a Youth Violence Prevention Coalition to Broader System Capacity. American Journal of Community Psychology, 55(3–4), 381– 395. http://doi.org/10.1007/s10464-015-9715-1 *Boydell, K. M., & Volpe, T. (2004). A qualitative examination of the implementation of a community-academic coalition. Journal of Community Psychology, 32(4), 357– 374. http://doi.org/10.1002/jcop.20010 *Brookman-Frazee, L., Stahmer, A.C., Lewis, K., Feder, J., Reed, S. (2012). Building a research-community collaborative to improve community care for infants and toddlers at-risk for autism spectrum disorders. Journal of Community Psychology, 40(6), 715–734. http://doi.org/10.1002/jcop.21501.BUILDING Burns, D. (2007). Systemic action research: A strategy for whole system change. Policy Press. *Chutuape, K.S., Muyeed, A., Z., Willard, N., Greenberg, L., Ellen, J. M. (2014). Adding to the HIV prevention portfolio - the achievement of structural changes by 13 Connect to Protect coalitions. Global Journal of Psychology Practice, 5(2), 1–8. Cox, P. J. (2009). Brief report of community ownership of local coalitions: Community members’ perspectives. Journal of Community Psychology, 37(6), 789–794. http://doi.org/10.1002/jcop Chavis, D. M. (2001). The paradoxes and promise of community coalitions. American Journal of Community Psychology, 29(2), 309–320. 32 Darnell, A. J., Barile, J. P., Weaver, S. R., Harper, C. R., Kuperminc, G. P., & Emshoff, J. G. (2013). Testing effects of community collaboration on rates of low infant birthweight at the county level. American journal of community psychology, 51(3-4), 398-406. Devenish, B., Hooley, M., & Mellor, D. (2017). The Pathways Between Socioeconomic Status and Adolescent Outcomes: A Systematic Review. American Journal of Community Psychology, 59(1–2), 219–238. http://doi.org/10.1002/ajcp.12115 *Emshoff, J.G., Darnell, A.J., Darnell, D.A., Erickson, S.W., Schnieder, S. & Hudgins, R. (2007). Systems change as an outcome and a process in the work of community collaboratives for health. American Journal of Community Psychology, 39, 255- 267. *Evans, S. D., Rosen, A. D., Kesten, S. M., & Moore, W. (2014). Miami thrives: weaving a poverty reduction coalition. American Journal of Community Psychology, 53(3– 4), 357–68. http://doi.org/10.1007/s10464-014-9657-z *Flewelling, R. L., Austin, D., Hale, K., Laplante, M., Liebig, M., Piasecki, L., & Uerz, L. (2005). Implementing research-based substance abuse prevention in communities: Effects of a coalition-based prevention initiative in Vermont. Journal of Community Psychology, 33(3), 333–353. http://doi.org/10.1002/jcop.20052 *Folayemi, B. (2001). Case story # 1: Building the grassroots coalition. American Journal of Community Psychology, 29(2), 193–197. Foster-Fishman, P. G., Berkowitz, S. L., Lounsbury, D. W., Jacobson, S., & Allen, N. A. (2001). Building Collaborative Capacity in Community Coalitions: A Review and Integrative Framework. American Journal of Community Psychology, 29(2), 241–261. http://doi.org/10.1023/A:1010378613583 Foster-Fishman, P. G., Salem, D. a, Allen, N. a, & Fahrbach, K. (2001). Facilitating interorganizational collaboration: The contribution of interorganizational alliances. American Journal of Community Psychology, 29(6), 875–905. http://doi.org/10.1023/A:101291563195 Foster-Fishman, P. G., & Watson, E. R. (2011). The ABLe change framework: a conceptual and methodological tool for promoting systems change. American Journal of Community Psychology, 49(3–4), 503–16. http://doi.org/10.1007/s10464-011-9454-x *Freedman, D. & Bess, K. (2011). Food systems change and the environment: Local and global connections. American Journal of Community Psychology, 47(3-4), 397- 409. 33 *Fujimoto, K., Valente, T.W., Pentz, M. A. (2009). Network structural influences on the adoption of evidence-based prevention in communities. Journal of Community Psychology, 37(7), 830–845. http://doi.org/10.1002/jcop.20333.NETWORK Gone, J. P. (2006). Research reservations: Response and responsibility in an American Indian community. American Journal of Community Psychology, 37(3–4), 333– 340. http://doi.org/10.1007/s10464-006-9047-2 Goodman, R.M., Wandersman, A., Chinman, M., Imm,P., Morrissey, E. (1996) An ecological assessment of community-based interventions for prevention and health promotion, American Journal of Community psychology, 24(1), 33-61. Gray, S. A., Gray, S., Cox, L. J., & Henly-shepard, S. (2013). Mental Modeler: A Fuzzy- Logic Cognitive Mapping Modeling Tool for Adaptive Environmental Management. Proceedings of the 46th International Conference on Complex Systems, 963–973. http://doi.org/10.1109/HICSS.2013.399 *Hathaway, B. L. (2001). Case Story #2: Growing a Healthy Community: A Practical Guide. American Journal of Community Psychology, 29(2), 199–203. Himmelman, A.T. (2001). On coalitions and the transformation of power relations: Collaborative betterment and collaborative empowerment, American Journal of Community Psychology, 29(2), 277-284. Hirsch, G. B., Levine, R., & Miller, R. L. (2007). Using system dynamics modeling to understand the impact of social change initiatives. American Journal of Community Psychology, 39(3–4), 239–53. http://doi.org/10.1007/s10464-007- 9114-3 Hong, L., & Page, S. E. (2004). Groups of diverse problem solvers can outperform groups of high-ability problem solvers, 101(46), 16385–16389. *Humphreys, K., Macus, S., Stewart, E., & Oliva, E. (2004). Expanding self-help group participation in culturally diverse urban areas: Media approaches to leveraging referent power. Journal of Community Psychology, 32(4), 413–424. http://doi.org/10.1002/jcop.20009 Johnson-Laird, P.N. (2005). Mental models and thought. In K.J. Holyoak and R.G. Morrison (Eds.), The Cambridge Handbook of Thinking and Reasoning (185- 208). Cambridge, UK: Cambridge University Press. Jones, N. A., Ross, H., Lynam, T., Perez, P., & Leitch, A. (2011). Mental Models and Interdisciplinary Synthesis of Theory and Methods. Ecology and Society, 16(1), 46–46. Kania, B. J., & Kramer, M. (2011). Collective Impact. Stanford Social Innovation Review, 1(9), 36–41. 34 Kosko, B. 1986. Fuzzy cognitive maps. International Journal of Man-Machine Studies 24:65-75. http://dx.doi.org/10.1016/ S0020-7373(86)80040-2 *Kovach, A. C., Becker, J., & Worley, H. (2004). The impact of community health workers on the self-determination, self-sufficiency, and decision-making ability of low-income women and mothers of young children. Journal of Community Psychology, 32(3), 343–356. http://doi.org/10.1002/jcop.20006 *Lenk, K. M., Toomey, T. L., Wagenaar, A. C., Bosma, L. M., & Vessey, J. (2002). Can neighborhood associations be allies in health policy efforts? Political activity among neighborhood associations. Journal of Community Psychology, 30(1), 57– 68. McGimpsey, I., Bradbury, A., & Santori, D. (2017). Revisions to rationality: the translation of “new knowledges” into policy under the Coalition Government. British Journal of Sociology of Education, 38(6), 908–925. http://doi.org/10.1080/01425692.2016.1202747 McMillan, B., Florin, P., Stevenson, J., Kerman, B., & Mitchell, R.E. (1995). Empowerment praxis in community coalitions, American Journal of Community Psychology, 23(5), 699-727. Meyfroodt, K., Desmidt, S., & Goeminne, S. (2017). Is the governing coalition a strategic coalition? Analysing the relationship between political group characteristics, strategic plan quality and strategic consensus. In EGPA 2017-European Group for Public Administration. Miller, R., & Curtin, J. (2011). Counting the costs of coalition: The case of New Zealand’s small parties. Political Science, 63(1), 106–125. http://doi.org/10.1177/0032318711407294 *Miller, R.L., Reed, S.J., Francisco, V.T., Ellen, J.M., & the ATN 079 Protocol Team for the Adolescent Medicine Trials Network for HIV/AIDS Interventions (2012). Conflict transformation, stigma, and HIV-preventive structural change. American Journal of Community Psychology, 49(3-4), 378-392. Milward, H.B., Provan, K. G. (2000). Governing the Hollow State. Journal of Public Administration Research and Theory, 10, 359–379. *Nargiso, J.E., Friend, K.B., Egan, C., Florin, P., Stevenson, J., Amodei, B., Barovier, L. (2013). Coalitional capacities and environmental strategies to prevent underage drinking. American Journal of Community Psychology, 51(1–2), 222–231. http://doi.org/10.1146/annurev-immunol-032713-120240.Microglia 35 Neal, Z.P. & Lawlor, J.A. (2016). Agent-based models. In L.A. Jason & D.S. Glenwick (Eds.), Handbook of Methodological Approaches to Community-Based Research, (197-206). New York, NY: Oxford University Press. Neal, Z. P., & Neal, J. W. (2017). Network Analysis in Community Psychology: Looking Back, Looking Forward. American Journal of Community Psychology, 60(1–2), 279–295. http://doi.org/10.1002/ajcp.12158 Nelson, G. (1994). The development of a mental health coalition: A case study, American Journal of Community psychology, 22(2), 229-255. *Nowell, B., & Boyd, N. M. (2014). Sense of Community Responsibility in Community Collaboratives: Advancing a Theory of Community as Resource and Responsibility. American Journal of Community Psychology, 54(3–4), 229–242. http://doi.org/10.1007/s10464-014-9667-x Ostrom, E. (2005). Understanding institutional diversity (Vol. 41). Princeton: Princeton University Press. Ostrom, E. (1990). Governing the commons: the evolution of institutions for collective action. Cambridge, UK: Cambridge University Press. Page, S. E. (2007). Making the difference: Applying a logic of diversity. Academy of Management Perspectives, 21(4), 6-20. *Perkins, D. D., Bess, K. D., Cooper, D. G., Jones, D. L., Armstead, T., & Speer, P. W. (2007). Community organizational learning: Case studies illustrating a three- dimensional model of levels and orders of change. Journal of Community Psychology, 35(3), 303–328. http://doi.org/10.1002/jcop Rappaport, J. (1977). Community psychology: Values, research, and action. New York: Holt, Rinehart, & Winston. *Riggs, N. R., Feinberg, M. E., & Greenberg, M. T. (2002). Community sector and gender differences in the perception of community-based prevention. Journal of Community Psychology, 30(6), 709–721. http://doi.org/10.1002/jcop.10024 Robins, G., Pattison, P., Kalish, Y., & Lusher, D. (2007). An introduction to exponential random graph (p*) models for social networks. Social networks, 29(2), 173-191. *Scaccia, B. J. P., Chávez, N. R., Hatchett, L., Byrd, K., Blanton, S., Alia, K., … Stout, S. (2017). Global Journal of Community Psychology Practice, 8(1), 1–25. *Shapiro, V.B., Oesterle, S., Hawkins, J.D. (2015). Relating coalition capacity to the adoption of science-based prevention in communities: Evidence from a randomized trial of communities that care. American Journal of Community Psychology, 55(0), 1-2. 36 *Snell-johns, J., Imm, P., Wandersman, A., & Claypoole, J. (2003). Roles assumed by a community coalition when creating environmental and policy-level changes. Journal of Community Psychology, 31(6), 661–670. http://doi.org/10.1002/jcop.10073 Society for Community Research and Action (2017). Other journals relevant to community psychology. Retrieved from: http://www.scra27.org/publications/other-journals-relevant-community- psychology/ Suarez, L. M., Belcher, H. M., Briggs, E. C., & Titus, J. C. (2012). Supporting the need for an integrated system of care for youth with co-occurring traumatic stress and substance abuse problems. American Journal of Community Psychology, 49(3- 4), 430-440. *Suarez-Balcazar, Y., Hellwig, M., Kouba, J., Redmond, L., Martinez, L., Block, D., Kohrnman, C., Peterman, W. (2006). The Making of an Interdisciplinary Partnership: The Case of the Chicago Food System Collaborative. American Journal of Community Psychology, 38, 113–123. http://doi.org/10.1007/s10464- 006-9067-y Watson, E. R., & Foster-Fishman, P. G. (2012). The Exchange Boundary Framework: Understanding the Evolution of Power within Collaborative Decision-Making Settings. American Journal of Community Psychology. http://doi.org/10.1007/s10464-012-9540-8 *Watson-Thompson, J., Fawcett, S.B. & Schultz, J.A. (2008). Differential effects of strategic planning on community change in two urban neighborhood coalitions. American Journal of Community Psychology, 42, 25-38. *Watson-thompson, J., Woods, N. K., Schober, D. J., & Schultz, J. A. (2014). Implementing the capacity building for change model with substance abuse prevention coalitions. Journal of Community Psychology, 42(6), 748–763. http://doi.org/10.1002/jcop *Wells, R., Ward, A.J., Feinberg, M. & Alexander, J.A. (2008). What motivates people to participate more in community-based coalitions? American Journal of Community Psychology, 42, 94-104. *Wolff, T. (2001). Community coalition building--contemporary practice and research: introduction. American Journal of Community Psychology, 29(2), 165-72–11. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/11446274 *Wolff, T. (2013). A Community Psychologist’s involvement in policy change at the community level: Three stories from a practitioner. Global Journal of Community Psychology Practice, 4(2), 2–12. 37 *Ziff, M.A., Willard, N., Harper, G.W., Bangi, A.K., Johnson, J., Ellen, J.M., the A. M. T. N. for H. I. (2010). Connect to Protect researcher-community partnerships: Assessing change in successful collaboration factors over time. Global Journal of Community Psychology Practice, 1(1), 32–39. 38 Introduction CHAPTER 3 In the second chapter, I established a working definition of coalitions in community psychology as well as the recent trends in coalition research in the field. In this chapter, I will address one area of the coalition process that is not well studied: problem definition. First, I will consider the challenge of problem definition in coalition settings as well as its importance to stakeholder collaboration and other coalition outcomes. Second, I will discuss mental models as a potential way of understanding diverse problem definitions and their relationship with collaboration among partners in community coalition settings. Third, I will explain the methods through which mental models can be collected. Problem definition in coalitions The processes by which coalitions achieve their outcomes has not yet been fully explored in community psychology. Coalitions report using a variety of approaches or strategies for their work, but there are few assessments of those approaches or the components that make them up. One component that stands out across a number of approaches is the concept of problem definition. Problem definition refers to establishing an understanding of the problem or problems the coalition is working on (Lawlor & Neal, 2016; Kania & Kramer, 2011). Problem definition can be collective (e.g., generating a group model of how a problem operates) as well as individual (e.g., stakeholders coming to the table with their own understanding of how a problem operates). Group definitions of problems are often considered aggregates of individual problem definitions among coalition participants, though it is not always clear whose individual problem definitions contribute to group problem definitions (Foster-Fishman 39 & Watson, 2011; Kania & Kramer, 2011). Problem definitions are often described as an important part of achieving impact on coalition goals; however, there has been little work establishing the extent to which coalition members tend to differ in their individual definitions of the problem of interest (Kania & Kramer, 2011; Foster-Fishman & Watson, 2011; Lasker & Weiss, 2003; Lasker, Weiss, & Miller, 2001). In addition, the relationship between individual problem definitions and collaborative processes has not been fully explored empirically. However, the issue of problem definition has been highlighted throughout the coalition literature both in theoretical and practical literature about approaches to coalition work. The importance of problem definition at both the individual and group level comes up in a number of theoretical approaches for conducting collaborative community change, including Collective Impact, the ABLe Change Framework, and networked community change (Kania & Kramer, 2011; Foster-Fishman & Watson, 2011; Lawlor & Neal, 2016). These approaches and their associated concepts are often applied to community coalitions and similar collaborative settings (Bess, 2015; Evans, Rosen, Kesten, & Moore, 2014; Zalkind & Wilson, 2015). Lawlor & Neal (2016) indicate that communities using a networked community change approach need to establish a common issue as part of the cyclical process of learning and acting as a collective network. In Kania & Kramer’s (2011) introduction to Collective Impact, an approach to collective community change, the authors include generating a common agenda (which includes a shared understanding of the problem) as a precondition for achieving impact. The ABLe Change Framework suggests that one part of achieving systems change in collaborative community settings like coalitions and systems of care is to come to a definition of the problem using a systems perspective. They argue that the process of 40 generating a problem definition must include a diversity of stakeholder perspectives. While each of these approaches highlights problem definition, few offer suggestions for how to effectively collect data on individual definitions of the problem of interest or how to synthesize them in order to generate shared understanding or engage in comparative analysis of problem definitions. Chavis (2001) also suggests a possibility for challenges around problem definition in his discussion of the paradoxes inherent in community coalitions. He suggests that there is a paradox of unity and diversity where a variety of stakeholders come together in a coalition, but are expected to act as one. Although each of these stakeholders has their own perspectives and interests, the pressure to act as a singular entity can make it difficult to achieve outcomes. Empirical literature on coalitions in community psychology indicates that problem definition is an issue that also arises in practice. Reininger, Dinh-Zarr, Sinicrope, & Martin (1999) further suggest that that defining scope is an essential dimension in building an effective community-based coalition: “Without clearly defining how and at what level the problem will be addressed by the coalition, members may become frustrated with the leadership and with one another and may think the coalition is not accomplishing its goals. Straightforward discussions about a coalition's intended scope and whether or not activities are consistent with the predetermined course of action are essential” (p. 73). This suggests that coalitions without a common definition and with significant variation among participants’ individual definitions may struggle with setting and achieving goals. Discussions about problem definition are not limited to theory. Scholars also discuss this as part of their practice with coalitions. In many cases, establishing individual and group definitions of the problem of interest are challenging for coalitions 41 and can influence their success. Coalitions can encounter situations where diverse stakeholder groups may work in opposition to each other. They may also be influenced by their own background or sector with regard to the problem of interest, making it difficult to generate a collective definition to a problem when each individual’s understanding is unique to their experiences. In an example of the challenges associated with differing problem definitions, Burns (2007) reports on a group of stakeholders working on the issue of female genital mutilation. A group of doctors identified the problem as being one of physical safety and thus focused their efforts on generating a safe, judgment free, medical environment where individuals wanting to participate in the practice could do so. Conversely, a group of individuals working for non-governmental organizations in the area identified the practice itself as problematic and embarked on a campaign to stop individuals from engaging in it entirely. These are just two ways that stakeholders understood a problem of interest that acted in conflict with each other. In situations with many stakeholders, the differences in perspectives can be much more complicated and the possibility for doing work that may inadvertently be contradictory is high. When coalitions are working on common problems like this with collaboration or coordination as an objective, unclear or misunderstood problem definitions can inhibit individual and collective progress. Literature also suggests that the sector in which coalition stakeholders work may be related to the way they define problem of interest as well as the role of the coalition in addressing the problem. For example, in a study of a coalition, Riggs, Feinberg, and Greenberg (2002) found that individuals working in the human services sector perceived prevention activities as more important than other coalition functions in 42 order to address the issues in which the group was interested (i.e., they believed that some actions would resolve the problem of interest more effectively than others). They also tended to perceive more benefits from participating in the coalition than others who did not work in a sector so closely related with prevention activities, like law enforcement or justice. Thus, those individuals who thought about the how the problem works (i.e., defined it) and its related solution in a particular way were more likely to see benefits from participating. This suggests that the way individuals within a coalition think about the problem at hand may be informed by their position and that the extent to which they derive benefit from participating may also be related to their individual definitions of a problem. Similarly, Mizrahi and Rosenthal (1993) discuss the way collective problem definition may influence individual participation in a coalition with an example of a coalition working on improving quality of life in a neighborhood. The group problem definition focused more on things like low income housing than overall economic development. After this decision, the coalition’s composition changed as stakeholders whose understanding of the problem no longer aligned with the collective definition exited and others who found a match with the coalition’s definition became more involved. This situation demonstrates how an individual’s personal definition of a problem can influence whether they see a coalition’s work as meaningful and worthwhile to participate in. These decisions about how to define problems are dynamic and occur throughout the life of the coalition, which means that both individual and group definitions may change as new information becomes available or the context surrounding the problem changes. For example, Mizrahi and Rosenthal (1993) discuss several coalitions dealing with political advocacy. Changes in elected officials or public policies over time may 43 require group members to shift their problem definitions. This is particularly true for the kinds of problems that coalitions deal with. They tend to be complex or wicked problems, which are characterized by circumstances that persistently shift when any kind of intervention on the problem occurs (Rittel & Weber, 1973). This suggests that coalitions doing work related to a problem are going to change the nature of the problem with their advocacy efforts. These changing circumstances may also require that definitions also change over time. These are just a few examples of situations in which problem definition has influenced coalition functioning. They can generate work that operates in opposition, which directly contradicts the purposes of building a coalition, as discussed in chapter 2. In addition, stakeholder experiences and work background, particularly with regard to the sector in which they work, may influence their definition of the problem of interest and create challenges for conducting work that is inclusive of all participating stakeholders. This can be further compounded by the changing nature of problem definitions over time. Many of these examples demonstrate the individual impacts of collective problem definition, with stakeholders whose individual definitions are not represented reporting that they get less benefit from participating and individuals making decisions about continuing to participate based on how well collective problem definitions match their individual definitions. Problem definition in coalition settings also has the potential to create substantial challenges for collaboration among stakeholders. Problem definition and collaboration Problem definition has also been proposed as a particular issue for building collaboration among stakeholders in settings like coalitions and collaboratives. 44 Mattessich, Murray-Close, and Monsey (2001) define collaboration as “a mutually beneficial and well-defined relationship entered into by two or more organizations [or individuals] to achieve common goals. The relationship includes a commitment to: a definition of mutual relationships and goals; a jointly developed structure and shared responsibility; mutual authority and accountability for success and sharing of resources and rewards” (p. 11). I will use this definition of collaboration throughout the study outlined here. This definition means that for two stakeholders to participate in a collaboration they must engage in work that serves common goals that they create together and must both benefit from the engagement. These types of relationships naturally pair well with coalition settings, where participants are often working towards shared goals that cannot be addressed by any individual stakeholder. Literature regarding collaboration in coalitions and similar settings suggests that problem definition is a critical component of generating successful collaboration among coalition stakeholders. Foster-Fishman, Berkowitz, Lounsbury, Jacobson & Allen (2001) suggest a framework for building collaborative capacity among stakeholders participating in coalitions and similar groups. The framework describes a number of factors that coalitions should plan to consider in order to develop collaborative relationships. One area of focus within this framework is the development of a common understanding of the problem as well as the engagement of multiple stakeholder perspectives. In order for collaborative relationships to form, participants must have a sense for how others understand the problem as well as some level of consensus about what it means. Although their focus is on problem definition as a collective activity, this requires individual problem definitions to be consistent and for the group to compare them in 45 order to generate a collective understanding. Their framework is supported by further work by Lasker & Weiss (2003), which suggests that collaborations, like those in coalitions, must include an understanding of how a problem works in a particular place. They indicate expert opinions on the issue are insufficient and that broad participation from community members is essential to establishing a group understanding of the problem and that these problem definitions influence buy-in for sustainable collaborative relationships. This indicates that it is insufficient for a coalition to simply have a definition of the problem, but that the individual definitions of participating members each need to be valued in order to establish a useful problem definition for the group. Again, this model for collective success requires that individual problem definitions be included in collective decision-making about the work to be done. Gray (2004) provides another example of the importance of individual problem definitions as a part of collaborative work. The study examined ongoing stakeholder conflict over the use and management of public land in Minnesota. Findings indicated that the lack of compatibility in the way individual stakeholders defined the problem in their minds acted as a factor prohibiting collaboration. This case demonstrates the importance of individual perspectives on the problem and that it is insufficient to attend only to some stakeholders’ perspectives when collaboration is a goal. Literature on the formation of social networks suggests that similar ways of thinking about or defining a problem may lead to a higher likelihood that stakeholders in a coalition will work together. The concept of homophily indicates that similar individuals tend to build relationships with each other (McPherson, Smith-Lovin, & Cook, 2001). In this case, similarity in problem definitions may be associated with forming collaborative relationships in a coalition setting. Homophily has emerged as the 46 focus of a number of studies of coalitions and collaboratives. These suggest that homophily is related to network efficiency in sharing information (Lawlor & Neal, 2016), the goals of organizations participating in the network (Provan & Kenis, 2007), and the shared values of network participants (DeGregorio, 2012). Each of these may be related to an individual’s personal problem definition, as a unique understanding of the problem may guide information sharing practices as well as goals, goal-setting, and values around the problem and its solutions. However, individual problem definition within these networks has not yet been fully explored in the context of coalitions in community psychology. It is important to note that a common definition of the problem is not the only factor influencing the development of collaborative relationships. Other factors like having quality plans for action, skilled staff, and monitoring systems are also important for stakeholders to have the capacity to collaborate (Foster-Fishman et al, 2001, Mattessich, Murray-Close, and Monsey, 2001); however, it is outside the scope of this paper to address all of them and here I focus specifically on the issue of problem definition. Problem definitions as mental models Mental models are one way to understand how individuals engage in problem definition. Mental models provide an internal heuristic for understanding, interpreting, and making decisions about problems in the external world (Jones, Ross, Lynam, Perez, & Leitch, 2011; Johnson-Laird, 2005). They have been employed in a variety of settings including education (Cole & Persichitte, 2000), medical diagnosis (Mago, Mehta, Woolrych, & Papageorgiou, 2012), military intervention (Jones et al., 2010), and natural resource management (Gray, Chan, Clark & Jordan, 2012; Giordano, Passarella, 47 Uricchio & Vurro, 2005; Gray, Gray, De Kok, Helfgott, O’Dwyer, Jordan & Nyaki, 2015). Mental modeling has also been used to assess causes and potential solutions to community problems (Singer, et al., 2017). They often function like computer simulation models in that they can help individuals work through scenarios for a problem of interest (Johnson-Laird, 2005; Hegarty, 1992; Schwartz & Black, 1996). These models are also built upon both knowledge of the world and beliefs about it, which means that mental models may be different depending on an individual’s experience of the world and that they may be dynamic as new knowledge is acquired or new beliefs form. This indicates that these models may inaccurately represent phenomena because they are a function of beliefs about things like probability or an event occurring or a narrow set of evidence (Johnson-Laird, 1994). This matches well with the concept of problem definition because it captures the concepts in an individual’s mind relevant to a problem and can also be made tangible in a way that it can be compared with others for the purposes of individual action generating a collective problem definition or for comparing individual problem definitions. Traffic patterns are an example of a phenomenon that individuals create mental models to understand. While sitting in heavy traffic, an individual may construct a model to explain the traffic that includes beliefs like the way drivers behave when there is an accident, the role of driver speed in slowing rush hour traffic, and knowledge about how various weather conditions may make the roads slippery. These factors, acting in relation to each other can comprise an individual’s mental model. The individual may then consider how traffic flow would be different if one of those things changed, like changing weather conditions or teaching drivers to avoid slowing down to look at accidents. 48 Although mental models are present in many aspects of life, they are rarely made explicit by moving them from inside an individual’s mind to a concrete, operationalized model that can be observed by others and tested or refined (Epstein, 2008). However, going through a process of making mental models explicit can be beneficial for groups of stakeholders working on shared problems in communities. They can facilitate things like understanding differences in patterns among stakeholder groups’ perspectives on the problem and for generating a group plan to address problems of interest. Individual mental models can be aggregated and/or assessed to determine which factors stakeholders typically see as primary drivers of the problem of interest. This kind of information can then be used as a way of facilitating conversations among stakeholders and making decisions about how to manage the problem. I will explore a couple of examples in detail below. Here, I will use ‘mental modeling’ to refer to the process of making mental models explicit. While explicit mental modeling has not been employed in coalition settings in community psychology, it has been used in a number of collaborative settings. Often, they are examined in natural resource management settings as well as the study of socio-ecological systems (Gray et al., 2012; Giordano et al, 2005; Gray et al., 2015). I focus on its use in natural resources and socio-ecological systems here because these applications are most similar to the situations in which coalition members might employ mental models. For example, Gray, et al. (2012) used mental modeling to learn about how multiple stakeholder groups conceptualize the management of a fishery. Each of them (managers, scientists, harvesters, pre and post harvest sectors, and environmental NGOs) had unique mental models of how management of this common resource should happen and the process of collecting these models allowed for aggregation to 49 understand their differences and similarities as well as generate a community map that included a collective understanding of the problem. In another example, Giordano et al. (2005) used mental modeling as part of a process for defining water resource management problems among a group of diverse stakeholders. They gathered mental models from farmers, water agency employees, and environmentalists in order to establish the concepts each of them saw as most important in driving water management problems, to assess how concepts were related to each other, and to assess model similarity among stakeholders. The authors used this modeling process as part of a larger decision support system to facilitate stakeholders engaging in collective problem definition in order to establish collaboration and understanding among them. As demonstrated in the above examples, these kinds of problems and the processes of engaging stakeholders to deal with them look similar to the problems and settings described in my exploration of coalitions outlined in chapter two. They all feature a broad group of stakeholders across sectors, often working on problems at the local level. The groups frequently work on managing natural resource issues, which sets them apart from the coalitions in community psychology described in Chapter 2. While the focus of their work is often somewhat different from the typical issues addressed in our field, there is much to learn from the approach they take with mental modeling. In particular, this approach could be useful in unpacking problem definitions in order to establish the role they play in coalitions in community psychology contexts. Additionally, implementing the processes for comparing mental models used in these contexts can be applied to coalitions in community psychology to establish differences and similarities in problem definitions among coalition members. Then, these 50 differences can be examined along other dimensions of coalition work, like collaboration in order to begin to unpack how problem definition functions as a part of coalition settings. There are several approaches through which this can be done, including the 3CM approach (Kearney & Kaplan, 1997), the mental model approach (Morgan, Bostrom, Fishhoff & Atman, 2002), and the fuzzy cognitive mapping (Ozesmi & Ozesmi, 2004) (see Hammerback, 2017 for an in-depth assessment of approaches to mental modeling). I will briefly describe these approaches, with specific focus on fuzzy cognitive mapping and my reasons for selecting it over other available options. The 3CM (Conceptual Content Cognitive Map) approach to capturing mental models draws from the cognitive mapping literature focusing on maps of physical landscapes. Kearney and Kaplan (1997) describe it as a process that captures mental objects ‘owned’ by a participant. This means that it specifically elicits knowledge that already exists in a participants’ mind without adding in any external concepts and it captures the relationships among those objects. The authors also suggest that it facilitates participant learning about how their knowledge structures function. Participants using this approach generate a list of the mental objects they own with regard to a particular problem and then organize them by which are associated with each other. This is similar to a couple of other approaches to generating mental models (Davies, 2011). Mind mapping and concept mapping also build on participant knowledge structures. Mind mapping starts with a central idea and links radiate out from it with associated ideas, but the links do not necessarily represent any type of relationship. These are highly visual and make use of a variety of colors to represent the central and radial concepts. Concept mapping is hierarchical, with the most important 51 concepts appearing at the top of the map. Additional concepts are added with links that have specific relationships. These relationships do not need to be consistent throughout the map and can represent things like one concept containing another or contributing to another. Morgan et al.’s (2002) mental model approach was generated as part of a larger process specifically for establishing public knowledge in situations where risk communication is necessary. Thus, the process is based on comparing a model of the combined knowledge of experts against public knowledge of a problem. It starts with collecting the knowledge structures of a set of experts using decision analysis, then uses those models to inform a set of open-ended interviews with those who would receive the risk communication regarding how they perceive the problem of interest. This is used to generate an influence diagram, mapping each concept and its relation to others. Next, participants are probed about the major components of their influence diagrams. Finally, participants are asked to engage directly with the beliefs they share in the interview process by doing tasks like sorting photographs based on whether their content is relevant to the issue, defining key terms relevant to the issue, or solving problems that require the use of their mental model of the problem. Fuzzy cognitive maps (FCMs) are an adaptation of cognitive mapping and were first used as a way to represent expert knowledge (Kosko, 1986; Axelrod, 1976). Axelrod’s (1976) cognitive maps consisted of signed digraphs in which the nodes represented relevant variables for a problem of interest and the ties represented causal relationships among the variables. Kosko (1986) tied cognitive maps with fuzzy logic, which loosens the requirements for components to fit exact causal and truth requirements that arise in classical logic. This approach is more similar to the way that 52 people form mental models than other modeling approaches that may require more specificity in modeling, like agent-based models and system dynamics models. FCMs can accept values like small, medium, and large when talking about the strength of a relationship, rather than a strict quantitative treatment of relationship strength. They also do not require equations that establish probabilities or agent behavior like those that might be necessary for an agent-based model. These values can then be used as fuzzy parameters for a semi-quantitative model. The causal relationships among all relevant variables can be an effective way to represent a system as well as the feedback structures within it and they can be used to do basic simulations, like those that an individual might do in their mind when using a mental model to consider solutions to a problem (Ozesmi & Ozesmi, 2004). In addition, FCMs can be compared to each other in order to understand the variation in stakeholders’ mental models in a coalition setting. Their semiquantitative nature facilitates these comparisons by making it easy for stakeholders to generate them (as they capture information that is similar to the way individuals think about the phenomena of interest, rather than requiring them to establish concrete values and equations to represent it). From there, models can be compared based on the concepts included in them, how those concepts relate to each other, how the models behave when implemented, or a combination of these (Lavin, Giabanelli, Stefanik, Gray & Arlinghaus, 2018; Yoon & Jetter, 2016). Thus, they are a promising way to begin unpacking what mental models in coalitions look like as well as how they relate with collaboration behaviors among participating stakeholders. To demonstrate how fuzzy cognitive maps operate, I will unpack an example that represents an issue that a coalition in community psychology may deal with: preschool 53 participation. (An image of the model is provided in Figure 1.) This model starts with preschool participation, the general problem of interest and becomes specific with the addition of relevant community factors that influence it, like the number of slots available for students to enroll and the extent to which transit options are accessible to families. For example, preschool participation is influenced by the availability of slots in early childhood programs like Head Start or Great Start, which are influenced by the availability of teachers to fill positions. The number of teachers available is influenced by teacher pay rate and available training programs for individuals who want to work in early childhood education. Transportation is related to both preschool participation and accessing health resources. When accessible transit options are available, children can get to preschool and medical resources to get required care in order to be able to enroll in preschool. In the map, each factor is connected via arrows indicating the direction of causal relationships among them and the width of the arrow represents the weight of the relationship between them. The weights are equidistant from each other and characterized by their strength (e.g., strong, moderate, weak) and direction (e.g., positive or negative). 54 Figure 1. Example fuzzy cognitive map I am choosing to use the Fuzzy Cognitive Mapping approach over other the other options for several reasons. The 3CM approach differs from fuzzy cognitive modeling in the types of relationships among concepts that participants generate and it does not involve asking participants to consider causal relationships among concepts. Similar qualitative approaches make it difficult to make comparisons using things like quantitative model behavior or causal relationships. Establishing participant perspectives on causal relationships is important for understanding how people working in action-oriented settings, like coalitions, because the belief that one concept causes another can inform decisions about where to the participant may believe collective action should be targeted. The mental model approach also has a couple of drawbacks relative to FCMs. It requires that there be individuals with expert knowledge about the concept of interest. In a coalition setting where participants tend to work on local problems, the participants 55 are likely to have the most expertise on the topic in their community. Thus, there would not be an easy expert comparison group for coalition members. This process would also require the interviewer to have predetermined expectations about things that would be relevant to the problem of interest, like photographs for assessment or problems that participants could solve using their mental models, which may not be appropriate for a coalition setting, where the researcher is an outsider. Conclusion & hypotheses In community psychology, problem definition has been posed as a central issue in the work of coalitions, specifically with regard to collaboration (Foster-Fishman, et al, 2001). However, the diversity of problem definitions within a coalition has not been investigated and there are no prevailing approaches for evaluating the diversity of problem definitions or their impact on coalition work. This issue has implications for the development of collaborative relationships among coalition participants and can hinder the process of addressing the issues that the group wants to focus on. Generating concrete mental models is one way to understand multiple perspectives on a complex problem that has not yet been fully explored in the context of coalitions in community psychology. In subsequent chapters, I will explore this issue with two research questions: (1) In what ways are mental models similar or different within a coalition? (2) To what extent does mental model structure and content predict collaboration within a coalition? My first question is motivated by the idea that shared understandings of problems are significant for achieving success in coalition settings, particularly with regard to collaboration (Foster-Fishman, Berkowitz, Lounsbury, and Allen, 2001; 56 Lawlor & Neal, 2016). The sector in which individuals work has been identified as significant to the way they think about the problems they work on in coalitions. There are several reasons why stakeholder problem definitions might be similar within the same sector. Riggs, Feinberg, & Greenberg (2002) examined the role of stakeholder philosophy about the work they engaged in and found that this clustered around the sectors in which they worked. While philosophy is not the same as problem definition, there may be similarities between them as they both deal with the way in which individuals think about an issue of interest. Additionally, literature regarding community change efforts, like coalitions suggests that stakeholders from the same sector (e.g., K-12 education, non-profit organizations) tend to see similar parts of a system and have similar understandings of the work to be done within coalition settings (Foster-Fishman & Watson, 2011). Having similar experiences that may expose individuals to similar knowledge of a problem could inform their problem definitions. For my second research question, I hypothesize that individuals with more similar problem definitions will be more likely to collaborate with each other. This hypothesis is supported by theories regarding collaboration within coalitions that suggest that coalitions where participants have more similar problem definitions have a higher capacity for collaboration (Foster-Fishman, Berkowitz, Lounsbury, Jacobson, & Allen). Homophily theory also supports this hypothesis by suggestion that members of social networks have a tendency to form ties with more similar to others (McPherson, Smith-lovin, & Cook, 2001). In this case, the dimension of similarity would be in the way that participants think about the problem of interest. In the chapters that follow, I will establish the methods through which I will test each of these hypotheses and my findings for each. Chapter four deals with the methods 57 for collecting mental models and the findings from my first hypothesis. Chapter five deals with the methods for collecting social network data and the findings from my second research question. 58 REFERENCES 59 REFERENCES Axelrod, R. 1976. Structure of decision: the cognitive map of political elites. Princeton University Press, Princeton, New Jersey, USA. Bess, K. D. (2015). Reframing Coalitions as Systems Interventions: A Network Study Exploring the Contribution of a Youth Violence Prevention Coalition to Broader System Capacity. American Journal of Community Psychology, 55(3–4), 381–395. http://doi.org/10.1007/s10464-015-9715-1 Burns, D. (2007). Systemic Action Research: A strategy for whole system change. Policy Press. Chavis, D. M. (2001). The paradoxes and promise of community coalitions. American Journal of Community Psychology, 29(2), 309–320. Cole, J. R., & Persichitte, K. A. (2000). Fuzzy cognitive mapping: Applications in education. International Journal of Intelligent Systems, 15(1), 1–25. http://doi.org/10.1002/(SICI)1098-111X(200001)15:1<1::AID-INT1>3.0.CO;2-V Davies, M. (2011). Concept mapping, mind mapping and argument mapping: What are the differences and do they matter? Higher Education, 62(3), 279–301. http://doi.org/10.1007/s10734-010-9387-6 Di Gregorio, M. (2012). Networking in environmental movement organisation coalitions: interest, values or discourse? Environmental Politics, 21(1), 1–25. Epstein, J. M. (2008). Why Model? Journal of Artificial Societies and Social Simulation, 11(4), 10–15. Evans, S. D., Rosen, A. D., Kesten, S. M., & Moore, W. (2014). Miami thrives: weaving a poverty reduction coalition. American Journal of Community Psychology, 53(3–4), 357–68. http://doi.org/10.1007/s10464-014-9657-z Foster-Fishman, P. G., Berkowitz, S. L., Lounsbury, D. W., Jacobson, S., & Allen, N. A. (2001). Building Collaborative Capacity in Community Coalitions: A Review and Integrative Framework. American Journal of Community Psychology, 29(2), 241– 261. http://doi.org/10.1023/A:1010378613583 Foster-Fishman, P. G., & Watson, E. R. (2011). The ABLe change framework: a conceptual and methodological tool for promoting systems change. American Journal of Community Psychology, 49(3–4), 503–16. http://doi.org/10.1007/s10464-011-9454-x 60 Giordano, R., Passarella, G., Uricchio, M., Vurro, M. (2005). Fuzzy cognitive maps for issue identification in a water resource conflict resolution system. Physics and Chemistry of the Earth, 30(6), 463–469. http://doi.org/10.1016/j.pce.2005.06.012 Gray, B. (2004). Strong Opposition: Frame based Resistance to Collaboration. Journal of Community and Applied Social Psychology, 14(1), 166–176. Retrieved from www.interscience.wiley.com Gray, S., Chan, A., Clark, D., & Jordan, R. (2012). Modeling the integration of stakeholder knowledge in social – ecological decision-making: Benefits and limitations to knowledge diversity. Ecological Modelling, 229, 88–96. http://doi.org/10.1016/j.ecolmodel.2011.09.011 Gray, S. A., Gray, S., Cox, L. J., & Henly-shepard, S. (2013). Mental Modeler: A Fuzzy- Logic Cognitive Mapping Modeling Tool for Adaptive Environmental Management. Proceedings of the 46th International Conference on Complex Systems, 963–973. http://doi.org/10.1109/HICSS.2013.399 Gray, S. A., Gray, S., de Kok, J. L., Helfgott, A. E. R., O’Dwyer, B., Jordan, R., & Nyaki, A. (2015). Using fuzzy cognitive mapping as a participatory approach to analyze change, preferred states, and perceived resilience of social-ecological systems. Ecology and Society, 20(2). http://doi.org/10.5751/ES-07396-200211 Hammerback, J. (2017). Finding paths or getting lost: Examining the mental model construct and mental model methodology (Unpublished master’s thesis). Hegarty, M. (1992). Mental Animation: Inferring Motion From Static Displays of Mechanical Systems. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18(5), 1084–1102. http://doi.org/10.1037/0278-7393.18.5.1084 Johnson-Laird, P.N. (2005). Mental models and thought. In K.J. Holyoak and R.G. Morrison (Eds.), The Cambridge Handbook of Thinking and Reasoning (185-208). Cambridge, UK: Cambridge University Press. Johnson-Laird, P. N. (1994). Mental models and probabilistic thinking. Cognition, 50(1–3), 189–209. http://doi.org/10.1016/0010-0277(94)90028-0 Jones, R. E., Connors, E. S., Mossey, M. E., Hyatt, J. R., Hansen, N. J., & Endsley, M. R. (2010, March). Modeling situation awareness for Army infantry platoon leaders using fuzzy cognitive mapping techniques. In Proceedings of the Behavior Representation in Modeling and Simulation (BRIMS) Conference (pp. 216-223). Jones, N. a., Ross, H., Lynam, T., Perez, P., & Leitch, A. (2011). Mental Models and Interdisciplinary Synthesis of Theory and Methods. Ecology and Society, 16(1), 46– 46. http://doi.org/46 61 Kania, B. J., & Kramer, M. (2011). Collective Impact. Stanford Social Innovation Review, 1(9), 36–41. Kearney, A.R., Kaplan, S. (1997). Toward a methodology for the measurement of knowledge structures of ordinary people: The conceptual content cognitive map (3CM). Environment and Behavior, 29(5), 579–617. Kosko, B. 1986. Fuzzy cognitive maps. International Journal of Man-Machine Studies 24:65-75. http://dx.doi.org/10.1016/ S0020-7373(86)80040-2 Lasker, R.D. & Weiss, E.S. (2003). Broadening participation in community problem solving: A multidisciplinary model to support collaborative practice and research. Journal of Urban Health: Bulletin of the New York Academy of Medicine, 80(1), 14-47. Lasker, R.D., Weiss, E.S. & Miller, R. (2001). Partnership synergy: A practical framework for studying and strengthening the collaborative advantage. The Milbank Quarterly, 79(2), 179-205. Lavin, E.A., Giabanelli, P.J., Stefanik, A.T., Gray, S.A., Arlinghaus, R. (2018). Should we simulate mental models to assess whether they agree? Proceedings of the Annual Simulation Symposium, Society for Computer Simulation International. Lawlor, J. A., & Neal, Z. P. (2016). Networked Community Change: Understanding Community Systems Change through the Lens of Social Network Analysis. American Journal of Community Psychology. http://doi.org/10.1002/ajcp.12052 Mago, V. K., Mehta, R., Woolrych, R., & Papageorgiou, E. I. (2012). Supporting meningitis diagnosis amongst infants and children through the use of fuzzy cognitive mapping. BMC medical informatics and decision making, 12(1), 98. Mattessich, P. W., Murray-Close, M. & Monsey, B. R. (2001). Collaboration: what makes it work. A review of research literature on factors influencing successful collaboration. Amherst H. Wilder Foundation, 919 Lafond, St. Paul, MN 55104. McPherson, M., Smith-lovin, L., & Cook, J. M. (2001). Birds of a feather: Homophily in social networks. Annual Review of Sociology, 27, 415–444. Mizrahi, T., & Rosenthal, B. B. (1992). Managing dynamic tensions in social change coalitions. In T. Mizrahi & J. Morrison (Eds.), Community organization and social administration: Advances, trends, and emerging principles. New York: Haworth. Morgan, M. G., Fischhoff, B., Bostrom, A. and Atman, C. J. (2002), Risk communication: A mental models approach, Cambridge University Press, Cambridge, UK 62 Özesmi, U., & Özesmi, S. L. (2004). Ecological models based on people’s know//ledge: a multi-step fuzzy cognitive mapping approach, 176, 43–64. http://doi.org/10.1016/j.ecolmodel.2003.10.027 Provan, K. G., & Kenis, P. (2007). Modes of Network Governance: Structure, Management, and Effectiveness. Journal of Public Administration Research and Theory, 18(2), 229–252. http://doi.org/10.1093/jopart/mum015 Reininger, B., Sinicrope, Dinh-Zarr, T., Sinicrope, P.S. & Martin, D.W. (1999). Dimensions of participation and leadership: Implications for community-based health promotion for youth. Family & Community Health, 22(2), 72-82. Riggs, N. R., Feinberg, M. E., & Greenberg, M. T. (2002). Community sector and gender differences in the perception of community-based prevention. Journal of Community Psychology, 30(6), 709–721. http://doi.org/10.1002/jcop.10024 Rittel, H.W.J., Webber, M. M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4(2), 155–169. Schwartz, D. L., & Black, J. B. (1996). Analog imagery in mental model reasoning: Depictive models. Cognitive Psychology, 30(2), 154–219. http://doi.org/10.1006/cogp.1996.0006 Singer, A., Gray, S., Sadler, A., Schmitt Olabisi, L., Metta, K., Wallace, R., … Henderson, J. (2017). Translating community narratives into semi-quantitative models to understand the dynamics of socio-environmental crises. Environmental Modelling and Software, 97(November), 46–55. http://doi.org/10.1016/j.envsoft.2017.07.010 Yoon, B. S., & Jetter, A. J. (2016). Comparative analysis for Fuzzy Cognitive Mapping. 2016 Portland International Conference on Management of Engineering and Technology (PICMET), 1897–1908. http://doi.org/10.1109/PICMET.2016.7806755 Zalkind, H., & Wilson, J. (2015). Down East Partnership for Children Is Committed to Launching Healthy Learners. North Carolina Medical Journal, 76(1), 41–45. http://doi.org/10.18043/ncm.76.1.41 63 Introduction CHAPTER 4 In chapter 3, I established the need for further exploration of individual problem definitions in coalitions, both to understand the nature of problems that coalitions examine and to understand the role that problem definition plays in stakeholder collaboration. In this chapter, I report on my first research question: (1a) In what ways are mental models similar or different within a coalition? (1b) In what ways are mental models similar or different by the sectors represented within a coalition? First, I describe the data collection site, the process by which recruited participants, the data collection process, and sampling strategy. Second, I report on the findings from this process, including themes in mental model concepts, structure, and function. I conclude with a discussion of the similarities and differences among mental models. Site Educational attainment coalitions are present throughout the state of Michigan and they generally work on local circumstances affecting post-secondary educational attainment often at the county or city level. Post-secondary educational attainment in these contexts tends to be defined broadly as any kind of credential achieved after graduating from high school (e.g., technical certifications, associate’s degrees, bachelor’s degrees, apprenticeship programs). Because of statewide infrastructure, many of them engage in similar work and approach it with similar strategies, including a push for using a systems thinking to address the problem. In order to do this, many coalitions (with the encouragement of state-level staff) have adopted a collective impact approach to their work. This means that they tend to focus on bringing together leaders from throughout the community who have a stake in a problem and can engage in systems 64 change on complex problems, like educational attainment (Kania & Kramer, 2011). For this study, I collected data from a single coalition representing a mid-sized county in Michigan (which I will refer to as Midwest County Coalition). I focused specifically on one of their action teams for my study, the team focused on postsecondary attainment. I chose to focus specifically on an action team as a subset of the larger coalition because action teams deal with specific sub-issues within the coalition and actively engage in change efforts around these issues. Thus, participants have a particular problem that all of them have opted into working on as members of the team. This streamlines the process of understanding problem definition as the participants have all agreed that there is a common problem they are working on. This is frequently an early step in the collective impact approach (Kania & Kramer, 2011). This can facilitate the collection of mental models in a smaller, more focused setting than the larger coalition. Action teams also have defined membership lists, which facilitated my collection of social network data as well. Midwest County demographics. As of the 2016 American Community Survey, Midwest County had a population of approximately 84,000 people. 93.5% of residents identified as White, 2.3% identified as Asian, 1.5% identified as Black or African-American, and .1% identified as Asian or Pacific Islander. 2.5% of the residents identify as Hispanic or Latino. The median income for Midwest County in 2016 was $54,852. About 6,000 residents live below the poverty line. Members of the community varied in their level of education. Of the members of the population over the age of 25, 19.6% have completed a bachelor's degree as their highest level of education, 10.3% have completed an associate’s degree, and 21.3% have completed some college, but did not receive a degree. 29.6% report a high school diploma or GED as their highest level of 65 education and 5.9% report having less than a high school diploma (American Community Survey, 2016). Participants Sampling. Rather than collecting data from a sample of coalition members, I conducted a census, aiming to collect data from all members of one of the coalition’s action teams. This approach was most appropriate for this study because social network analysis is sensitive to missingness at the population level (Prell, 2012). A census approach to sampling can be difficult to achieve; thus, I took several actions to ensure this approach will be successful. First, I set a concrete boundary around who is a member of the population and who is not. The boundary was determined based on membership in an action team within my coalition of interest, which included 23 members at the time of data collection. This meant that the population was a manageable size for data collection and contacting members individually was possible. I also fostered buy-in with the coalition’s leader over the course of several meetings to describe the study and its value. He indicated support for the project to me as well as members of the action team and has offered to provide support in promoting the work with the action team while I conducted the project. As a key opinion leader, his support was critical for gaining buy-in from participants. Recruitment procedures. I recruited participants in several ways. First, I recruited them directly through email communications, which described the nature of the study, the benefits and risks of participating, and the time it will take to participate. Participants were able to reply to the email or to fill out a Qualtrics form with their preferred dates and times to set up an opportunity to participate. My partner in the coalition also reminded participants about the study during meetings and sent a couple 66 of reminder emails during the course of the study. Recruitment materials are included in Appendix A for reference. As an incentive for recruitment and a thank you for their time, I offered participants a $15.00 gift card to Amazon.com. These combined approaches helped me to meet my goals for participation. Participant demographics. Using the strategies listed above, I recruited 18 of the 23 members in the action team to participate. This yielded a 78.26% response rate. The majority of participants identified as female (61.11%) and 38.89% of participants identified as male. Participants ranged in age from 23 to 68, with an average age of 40.44 years and a standard deviation of 12.38 years. Participants primarily identified as White (n = 15, 83.33%), while one identified as biracial (5.56%) and two did not report on their race (11.11%). All participants reported having some type of postsecondary degree. Half of the participants reported having a bachelor’s degree (n = 9, 50%), eight participants reported having a master’s degree (44.44%), and one participant reported having a doctoral degree (5.56%). On average, participants had been involved in the coalition for 2.67 years, with a standard deviation of 1.46 years and a range of 0 to five years. Participants worked in several sectors: K-12 education (n = 8, 44.44%), non-profit (n = 6, 33.33%), postsecondary education (n = 3, 16.66%), and philanthropy or foundation (n = 1, 5.55%). Procedure Informed consent. The data collection process started with a conversation about informed consent. I informed participants of how long it would take to participate, the risks and benefits, and explained that their information would be kept confidential. I also reminded them that participation in the study was voluntary, that they could stop participating at any time, and that they could choose not to answer any 67 questions for any reason. Participants had the opportunity to ask questions during this period as well. After establishing informed consent, participants engaged in the interviewing and mental modeling process. Then, we concluded with collecting a survey that containing network data and demographic information. A copy of the IRB approval and exempt designation is included in Appendix B and a copy of the consent document is in Appendix C. Interviews. I treated the data collection process as a facilitated interview and modeling session with individual participants. This is a typical method for capturing mental models (Ozesmi & Ozesmi, 2004; Mouratiadou & Moran, 2007). The process was audio recorded so I could return to participant descriptions of the components of their models at the data analysis stage. Interviews started by asking participants two open-ended questions to facilitate thinking about the topic we would be modeling about and to get some more information about the participant’s experiences: 1. Can you tell me a little bit about yourself? 2. In your experience, what influences postsecondary attainment in [Midwest County] County, MI? While conducting the interviews, I worked with the participant to enter their responses into the Mental Modeler software (Gray, Gray, Cox, & Henly-Shepard, 2013). This program allows participants to input concepts of interest in their mental models, establish causal relationships among them, and input the strength of the relationships. The program itself is not required for the process of building a model, however it simplifies data analysis and reduces possible data entry errors later in the process. Participants had the option of a paper and pencil version of modeling, but none of them were interested in creating a model that way. The modeling process for collecting this data was informed by Gray’s (2015) video, which demonstrates how to collect a mental model. I first oriented participants to the process 68 using an example of a basic mental model, walking them through the basic steps of what the concepts are, what constitutes a causal relationship among them, and examples of how relationship strength can be defined. Participants discussed the topic of their action team as the problem of interest: post-secondary attainment. They were be asked to consider the concepts that were relevant to this issue. As they listed these factors, I entered them into the software, and asked probing questions to establish additional information as necessary. Probing included things like asking participants to define the terms that they suggest as factors and to unpack situations where there may be more than one factor included in a single term. I also asked participants if they can think of other related factors to the ones that they offer. Additionally, I probed to understand the differences between factors that seem similar or appear to capture the same information. At the end of the process of generating concepts for the models, I asked participants to review them for any changes or additional concepts that may have arisen for them. Next, I asked participants to establish relationships among the variables of interest. We systematically went through the list of concepts the participant generated and discussed how they are related to each other. When a relationship between the concepts is established, I asked the participant about the strength and the direction of the relationship. The strength of each relationship in a given mental model is relative to the strength of each other relationship in that model. Participants were asked to quantify them in terms of weak, moderate, and strong. Throughout the interview, I continued to probe about whether the participant wanted to add additional factors or relationships among them as needed. I also asked them to review the final mental model in mental modeler in order to make any changes before we completed the interview. A 69 copy of the protocol for the interviews is included in Appendix D. On average, interviews lasted 35.5 minutes, with a standard deviation of 15 minutes. The longest interview was one hour and 23 minutes and the shortest interview was 19 minutes and 48 seconds1. Pre-defined concepts. In some settings, researchers provide participants with a list of pre-defined factors that they are instructed to use in developing their models. This approach to data collection was not appropriate for this study because the process of generating a set concepts could compromise authentic participation in the study for several reasons. The factors needed to be specific to the setting in which the data was being collected and needed to be representative of possible cognitive structures of the group of people who would participate. Conducting pre-interviews with participants could have primed them to be thinking more about this topic in different ways than they normally would, affecting their ultimate responses to the study. A focus group approach could have allowed participants to share ideas, introducing information they wouldn’t otherwise have in their mental models. Because these models capture cognitive structures, their validity depends on the participant describing their own cognitive structures and outside information can bias participants toward describing structures that do not belong to them (Kearney & Kaplan, 1997; Rouse & Morris, 1986). Thus, priming is a threat to collecting the most accurate data if participants engage in conversations about their mental models before they are collected for analysis. Having participants generate their own factors and establish their relationships to each other helps protect against potentially priming them with concepts that may not exist in their own cognitive structures. 1 This interview was shortened because the interviewee had a scheduling change. The participant was able to submit additional comments via email after the interview ended due to the shortened participation time. 70 However, in some situations, participants are given a small set of factors that they are asked to include in their models which are used for scenario modeling in order to assess functional model behavior. These factors are generally not included in assessments of content as they are provided a priori to the participant, but they allow for the researcher to compare model behavior across all models when a common component has been increased, decreased or removed entirely. These factors can be included in measures of model structure as the participant generates the relational structure between concepts given to them and concepts they generate themselves. In this case, I asked participants to include three factors in their model. Two of them come from the coalition’s dashboard: FAFSA completion level and achievement on the SAT (Midland College and Career Access Network, 2017). These are factors that the group is regularly tracking on their organizational dashboard and are likely to come up frequently in their work. Thus, these factors have already been established as valuable to the group. FAFSA completion level refers to the number of students submitting the FAFSA form each year. Achievement on the SAT refers to the scores students receive from the SAT exam, which is a statewide requirement for all high school students. The final factor is the specific outcome their action team is working on, postsecondary attainment. Postsecondary attainment refers to any type of credentialing that students receive after secondary education. This approach alleviated potential priming issues that come from asking participants to include a large set of concepts and participants will be asked to generate all other factors on their own. It also allowed for analyses that involve manipulating the factors to see how the model changes with regard to the outcome of interest for the group. 71 Surveys. Participants were also asked to complete surveys. These included several questions related to participant demographics. The questions covered race, gender, and age as well as characteristics of their relationship with the coalition including length of time participating in the coalition, other roles they have played in the coalition, and length of time in their current work position. A copy of the survey questions is included in Appendix E. Surveys also included a social network component to establish each participant’s collaborative relationships within the coalition action team they participate in. I will describe this part of the survey in detail in chapter five where I present the methods and findings for my second research question. Standardizing concepts To compare models and evaluate the key themes among them, I first standardized the concept names across the models. Standardizing concepts in mental models to facilitate comparison among them is a common practice and is often completed after data has been collected (Yoon & Jetter, 2016; Mouratiadou & Moran, 2007; Banini & Bearman, 1998; Khan & Quaddas, 2004; Radonic, 2018). Following each interview, I compiled a list of each factor described and the definition of the factors, using the participants’ own description of the factor transcribed directly from the interview. I employed content analysis, assigning tags to each concept’s definition that briefly described the content of the definition. A second coder and myself reviewed each interview transcript in full, then reviewed the list of factors and their definitions as the participant described them and assigned tags to each concept. The second coder and myself reviewed our tags together and came to consensus about which were appropriate to apply to each concept. Next, the coder and I evaluated the definitions from each transcript within the tags in order to evaluate which concepts discussed the same 72 content (the tags served as a way to organize each concept by its general content area). We first independently reviewed the definitions and then compared our responses to come to consensus about which definitions were the same. In following standards for trustworthiness in qualitative research, throughout the process, I kept an audit trail to track the decisions my second coder and I made as well as overall project decisions (Lincoln & Guba, 1985). I also conducted a sensitivity analysis of this process and examined how my inferential analyses would have looked different if I had used a more liberal standardizing strategy. This is outlined in Appendix F. Access to transportation is an example of a concept that we standardized. There were several concepts that all dealt with this issue, but did not use exactly the same name to refer to it. For example, one respondent referred to it as “Access to transportation to get to postsecondary options” and another referred to it as “access to transportation.” While these names were somewhat different, we determined their definitions of the concept to be sufficiently similar to standardize them as “access to transportation.” One respondent said: Yeah, because even the teens that come to my program, that may be junior or seniors, 16, 17, 18 years old but they all get dropped off. They don’t have cars to drive. So if they can’t come five miles down the road to a teen program, very rarely are they going to have access to a car in the middle of the day to go to college, to go to class or something like that. Only because it’s where they all live. It’s a 20+ minute drive to get to [community college], so it’s not like it’s right around the corner to go into postsecondary education for sure. The other respondent said: “I think resources, like a lot of my students don’t have cars. So even their ability to get to a community college which may be only 15 minutes away, isn’t totally-“ Both of these participants dealt with the issue of having access to necessary transportation resources in order to get to postsecondary education, which indicated that they could be standardized as a single concept. 73 To the extent possible, I retained the original names of any factors that did not appear across models. In cases where concept names overlapped with standardized names or names were too similar to identify differences, I changed the concept names. Standardizing concepts aids in the assessment of both content and structure in the models as described below. I started the process with 158 raw concepts and after standardizing this list expanded to 164 total concepts, with 138 unique concepts. I also examined the models based on the tags each concept received and conducted a sensitivity analysis for my inferential research question in chapter 5. This approach to standardizing yielded 48 concepts based on the tags applied. Analyses To address my first research objective, exploring the similarities and differences in individual problem definition within a coalition, I conducted several analyses: content, structural, and functional. The content of the models addresses the concepts that participants most frequently include as part of their mental model and how similar or different they are to other participants’ models. These are described in terms of the specific concepts that come up most frequently as well as the tags we most frequently used in the standardizing process to give a sense for the general content areas that arose most frequently. Similarly, the structural component evaluates the model’s overall structure, including which concepts are most influential, and compares them across models. Finally, I examine functional similarity by running scenarios with the models to see how they behave given a change in a common concept. These three types of comparisons are commonly used in the mental modeling literature as ways to describe models and evaluate how similar or different groups of models are (Jetter, Gray, 74 Ellswort, Zhang, Singer, & Laraichi, 2017; Yoon & Jetter, 2016). I present an overview of these analyses in Table 4 below. Table 4. Summary of analyses to describe mental models (table structure taken from Yoon & Jetter, 2016) Mental Model Evaluation Type Metrics Purpose Example References Content Concept frequency To establish which concepts appear most frequently in mental models Clarke & Mackaness (2001) Structure In-degree centrality (Receivers) To establish which concepts receive the most direct influence Ozesmi & Ozesmi (2004) Out-degree centrality To establish which Gray, Hilsberg, McFall, concepts have the most & Arlinghaus (2015) (Transmitters) direct influence Model Function Change in central outcome To establish how model structure influences outcome Wang (1996) Summary of concepts. I describe the concepts in the mental models in terms of both structure and content. I evaluated content by reporting on the factors that 75 participants most frequently endorsed and how those factors were defined by participants. Model constructs can be characterized across participants as common (endorsed by everyone), partially common (endorsed by a subset of participants, or individual (endorsed by an individual) (Clark & Mackaness, 2001). I focus on concepts that are partially common in order to capture the places where there is most similarity among the model content (there were no concepts that were common to all mental models included in the study). I also report on the tags most frequently used in the standardizing process to provide information about the content areas that were most frequently included in the mental models. Model structure. To evaluate structure, I employed several analyses based on graph theory. Factors in mental models are categorized in three ways: as drivers, receivers, and ordinary factors, which help describe their level of influence. Drivers are factors that exclusively influence other factors. Receivers are factors that are influenced by other factors, but do not exert influence on other factors. Ordinary factors both receive influence and drive influence toward other model factors (Gray, Zanre, & Gray, 2014; Ozesmi & Ozesmi, 2004; Yoon & Jetter, 2016). These values come from metrics used in graph theory, using each factor’s in-degree and out-degree centrality. Those with only out-degree connections are drivers, those with only in-degree connections are receivers, and those with both in-degree and out-degree connections are ordinary (Ozesmi & Ozesmi, 2004; Yoon & Jetter, 2016). Each of these links has a numerical weight associated with it. When participants discuss the relationships among concepts in their mental models, they are asked to weigh them as weak, moderate, or strong. These values go into the model as positive or negative values between -1 and 1 at set intervals matching the values. The absolute value of the weights of each in- or out-link 76 are added in establishing the in-degree and out-degree values for each concept. These establish the weight of influence overall being exerted from one concept to another. Each of these provide a measure of direct influence, assessing how many factors each gets or receives influence from directly. This is similar to the use of these concepts in social networks as a means of assessing social influence, information control, or popularity (Freeman, 1978). In my results, I descriptively report which concepts came up most frequently as drivers and receivers based on their centrality scores across the models and discuss the extent to which participants’ models are similar with regard to the way concepts exert and receive direct influence in the model. The structural analyses presented here are similar to those used for social network analysis as they both draw on graph theory. In order to differentiate the purposes for these analyses, I present a comparison between social network analysis and structural analyses for fuzzy cognitive maps in Table 5. Table 5. Comparison of graph metrics for social network analysis and fuzzy cognitive maps Metric In-degree centrality Out-degree centrality Social network analysis meaning Fuzzy cognitive map meaning Popularity Extent to which a concept receives direct influence Popularity Extent to which a concept directly influences other concepts Model function. To evaluate how the models function, I manipulated the specified factors that each participant was asked to include in their model (FAFSA 77 completion level and achievement on the SAT) and assess how it impacts the outcome variable for the model. Scenario modeling is a process that allows for the manipulation of model components (increasing or decreasing them) in order to see the extent to which they elicit a relative increase or decrease in other model components. The values that emerge from scenario modeling only indicate that an increase or decrease of a particular magnitude may occur, not an absolute value of how much something would increase in the real world. When we change a variable in the model, scenario modeling first transforms the matrix into a vector, then applies an activation equation to establish changes in values in the concepts based on their weights and relational structure (Ozesmi & Ozesmi, 2004). There are several activation functions that researchers frequently employ to choose from when running scenarios on mental models. For an example of how activation functions operate in mental model scenarios, see Appendix G. In this case, I am employed the hyperbolic tangent function for several reasons. It is one of the most commonly implemented functions for scenario modeling and can generate values between -1 and 1 (denoting the degree of positive or negative increase in a concept relative to the other concepts in the mental model). Other activation functions limit the outputs to binary values of 0 and 1 or limit them to positive values between 0 and 1, which are more restrictive (Felix, Nápoles, Falcon, Froelich, Vanhoof, & Bello, 2017; Tsadiras, 2008). Because I am interested in comparing across models, the hyperbolic function allows for more variation in values, which makes similarities and differences in model function easier to see. The models then run iteratively until each concept converges on a value. These final values reflect the relative increase or decrease that can be expected from each concept, given a change in the model (Yoon & Jetter, 2016). 78 In this case, I am interested in seeing how the two specified components may influence the other concept I am providing to participants: postsecondary attainment. As this is the specific issue that their action team is working on, the way that these concepts influence it can give some indication of how the unique structure of their model will change it. This provides one additional way to understand values in a model. Essentially, this gets at the idea that we may choose to have an intervention on a variable, like FAFSA completion level, in order to increase postsecondary attainment in a community. FAFSA completion level can be related to a number of other concepts in a mental model and can thus have a non-linear relationship with the postsecondary attainment. Evaluating the resultant changes in postsecondary attainment that arise with a change in a concept like this provides insight into how the model’s content and structure work together to establish outcomes. This process provides some insight into what might happen under various conditions and it may help us understand the cognitive processes associated with an individual’s mental model. The unique structure of the model influences the outcomes of these manipulations and can be used to identify themes in how the models change when concepts are manipulated. For example, this can demonstrate if multiple models converge around similar outcomes when concepts are manipulated. This approach has been used to compare stakeholders’ mental models in other settings and represents a viable alternative to priming participants with a full list of concepts to include in their models (Yoon & Jetter, 2016; Wang, 1996). To better understand the mathematical process through which functional analysis of mental models is conducted, see Appendix G for a simple, worked out example. 79 Results To address my first research objective, exploring the similarities and differences in individual problem definition within a coalition, I conducted several analyses: content, structural, and functional. Content analyses address the concepts that each participant endorses as part of their mental model and how similar or different they are to other participants’ models. Similarly, the structural component evaluate the model’s overall structure, including which concepts are most influential, and compare them across models. Finally, I examined functional similarity by running scenarios with the models to see how they behave given a change in a common concept. These three types of comparisons are commonly used in the mental modeling literature as ways to describe models and evaluate how similar or different groups of models are (Jetter, Gray, Ellswort, Zhang, Singer, & Laraichi, 2017; Yoon & Jetter, 2016, summarized in table 4). I describe each of these types of analyses for the sample overall and by participant sector to explore the similarities and differences within the sectors participants work in. Content Though there was a great deal of variance in concepts between participants, all of them shared at least one concept of their mental model with at least one other member of their action team. Figure 2 demonstrates the extent to which participants shared concepts in their mental models as a network in which participants are connected if they have overlapping concepts in their models (excluding the three concepts that all participants were given to include in their models). Thus, if participants shared a concept, they are share a network tie. This approach to visualization is similar to bibliographic coupling, which ties two pieces of scholarship if they cite the same piece of 80 literature (Batagelj & Cerinsek, 2013). The network has a density of .32 and all participants are connected to at least one other member of the action team in their concepts. This indicates that no member of the coalition is entirely unique in their problem definition and that overall participants share aspects of their problem definitions with a great deal of their colleagues. Though I proposed to only discuss the concepts that arose most frequently, there were some tags that came up often in the standardization process and discussing the tag frequencies may provide some additional information about the similarities in the mental models. Figure 3 demonstrates the extent to which participants overlapped in the tags assigned to the concepts they included in their mental models. Here, participants share a tie if the concepts in their models received the same tags during the coding process. These tags provide a general sense for the type of content participants discussed in their mental models. This network has a density of .92, indicating that participants share some content overlap in their mental models (based on the tags assigned to the concepts) with the vast majority of their colleagues, even if the exact concepts of their models do not match. 81 Figure 2. Network of participants with ties representing shared mental model concepts and colors representing participants’ sectors Note: Ties are thicker and darker in color based on the level of overlap between models Figure 3. Network of participants with ties representing shared mental model content based on tags and colors representing participants’ sectors Note: Ties are thicker and darker in color based on the level of overlap between models 82 Overall, every mental model shared some similarities in terms of the concepts they covered or the general content of their concepts, but they differed substantially in the particular concepts they used. Only a few concepts arose in more than three mental models: family experience with college, access to transportation, and ability to pay for postsecondary options. I briefly define each of these with demonstrative quotes from interviews. In addition to exact concepts that appeared across mental models, many content areas occurred across models, as determined through the tagging process. I will discuss also discuss the categories that appeared frequently to demonstrate the general content areas of concepts that appeared frequently and the types of concepts that fell within them. Family experience with college appeared in seven mental models (38.89%). This concept referred to the extent to which students’ family members had previous experience in postsecondary institutions. Family in this case was not limited to parents and could include any member of the immediate family. One participant’s definition of this concept exemplifies it: “family history can definitely be a part of that. For any reason, family not having experience with higher education” (119). Access to transportation appeared in six mental models (33.33%). This concept referred to the extent to which students have access to transportation in order to get to postsecondary institutions. This could include a personal vehicle or public transportation. This concept can be exemplified with the following example: That being said, the first thing that came to my mind was transportation, and it probably seems weird, but I know for being in a city-ish, but no public transportation and then having a lot of rural communities in [Midwest County], I see that as a barrier for teens to, like I mentioned, being able to go and see where you could study I think is important. But that can be challenging, and 83 that's one of the things that I think our local group has done well with, but it is something that is difficult for this population, or even, you might have to go and do some things at the school if you pick your university and college and they want you to go and do some things before classes start, that can be a challenge, figuring out "How am I going to get there?", or if it's something where they're staying, maybe they're gonna stay at home and do a community college, figuring out how that looks (117). Ability to pay for postsecondary options appeared in four mental models (22.22%). This concept refers on the student’s overall ability to pay for postsecondary options. It is more comprehensive than an individual metric like scholarship funds that they receive or their family’s financial status and instead considers how feasible it is for them to cover the costs associated with postsecondary. It includes both the cost of tuition and also incidental expenses, for example, books and housing. Two mental models exemplify this with their definitions of the concept, “It's not even just necessarily, like, the college fee, right? Like, I think about kids that want to go away to school, like, there's like the housing, there's food” (102) and “ I would say that's the number one thing that they will say will stop them is that they can't afford it. Because they have no idea that financial aid exists, what that means. The difference between a scholarship and a loan. They don't, those words just aren't in the conversations at home because it's not there. Yes, I think that's huge for them” (120). As many of the concepts appeared in only a few of the models, I also evaluated the tags that were used most frequently in order to assess what general content areas arose most frequently in the mental models. There were four tags that were applied in fifty percent or more of the mental models. They included: finances, family, knowledge about postsecondary options, and soft skills. I will briefly describe each of these and provide demonstrative quotes. 84 Finances and funding appeared in 14 of the mental models (77.77%) and included individual finances for supporting postsecondary, the accessibility of resources for paying for it (both from family members and from outside sources like FAFSA and scholarships). This additionally covered finances that support programming to help students get training, for example the level of government funding available for pro- college programming. Concepts relating to families appeared in 12 of the mental models (66.66%) and included concepts related to families including experience (described above) and many more. It includes how much emotional and tangible support family members provide related to postsecondary education, their expectations for completing or not completing postsecondary education, and the extent to which students experience home stability. Concepts relating to knowledge appeared in 9 of the mental models (50%) and included concepts relating to knowledge of postsecondary options that are available to students in terms of academic programs, colleges that are nearby, and types of postsecondary education available to them. It also includes knowledge about resources necessary to obtain a postsecondary degree, for example, knowledge about the financial supports available to them and how to access them. Concepts relating to soft skills appeared in 9 of the models (50%) and included concepts relating to the non-academic skills that students need upon graduation in order to be prepared for postsecondary education. It included things like level of motivation, level of self-confidence, and ability to engage in self-care and avoid burnout. Mental model structure In order to assess mental model structure, I evaluated in-degree and out-degree centrality for each model and identified several key concepts. I evaluated degree 85 centrality in two ways: (1) the degree centrality normalized by model size and averaged based on how frequently it appeared and (1) the raw degree centrality for each concept, summed across models. I refer to these as raw centrality and normalized centrality respectively. Because there are size differences in mental models concepts in larger models can potentially reach a higher degree centrality. In order to make these more comparable, I divided in-degree and out-degree by the number of concepts minus one. This evaluates the actual centrality out of the highest possible centrality a concept could achieve in a mental model. When concepts appeared in multiple models, their centrality was computed as an average of the centrality scores from each of the models where they appeared. I report the three highest centrality concepts for each type of centrality that appeared in at least two models. I required the concepts to appear in at least two models in order to avoid single models with high centrality from sawing the results. Overall centrality values and centrality values by sector are reported in Table 6. I originally proposed to compute Katz centrality in addition to degree centrality. This was not possible because of the structure of the mental model networks. Katz centrality requires an attenuation factor of 1/λ, where lambda is the largest eigenvalue for the network matrix. In this case, many of the networks had eigvenvalues of zero, creating an undefined attenuation factor. Thus, I removed this analysis from the study. In-degree centrality. The three highest in-degree centrality concepts using the normalized centrality approach that appeared in at least two mental models are: postsecondary attainment (.48), high school GPA (.17), and awareness of nearby college options (.15). Postsecondary attainment is the outcome concept all participants were given referring to the level of postsecondary attainment among students in the county. High school GPA refers to the level of GPA students achieve in high school. Awareness 86 of nearby college options refers to how much students know about postsecondary options that are within driving distance. When using the raw centrality approach, the highest in-degree concepts are postsecondary attainment (89.4), FAFSA completion level (18.01), and achievement on the SAT (12.64). These were the three concepts given to participants at the outset of the mental modeling activity and were thus included in each model, though some participants did not link them to anything. Out-degree centrality. The three highest out-degree centrality concepts using the normalized centrality approach that appeared in at least two mental models are: access to college advisors (.28), ability to set short and long-term goals (.18), and family experience with college (.18). Access to college advisors refers to the extent to which students have access to advisors who specifically provide support around college access. In this community, the schools participated in a program where AmeriCorps members were trained and served as advisors in one or more schools2. Ability to set short and long term goals refers to how capable students are of setting goals for their future both in the short and long term. As described above, family experience with college refers to the extent to which students’ family members had previous experience in postsecondary institutions. The highest out-degree centrality concepts using the raw centrality approach were Family experience with college (14.02), FAFSA completion level (11.02), and achievement on the SAT (10.36). 2 There is a statewide program in Michigan intended to bring recent college graduates to high schools around the state as college advisors (Michigan College Advising Corps). 87 Table 6. Structural differences among sectors In-degree (average based on number of models where it appears) Out-degree (average based on models where it appears) Overall Raw in-degree Raw out-degree • Postsecondary attainment • High school GPA • Awareness of nearby college options • Access to college advisors • Postsecondary attainment • Family experience with college • Ability to set short • FAFSA completion • FAFSA completion and long-term goals • Family experience with college level • Achievement on the SAT level • Achievement on the SAT K-12 education • Postsecondary attainment • Awareness of nearby college options • Level of career planning during postsecondary • Poverty level • Access to college advisors • Postsecondary attainment • Family experience with college • FAFSA completion • FAFSA completion • How much support level level students get from high school • Achievement on the • Poverty level SAT 88 Table 6 (cont’d) Non-profit organizations • Postsecondary attainment • High school GPA • Availability of scholarships • Parent involvement • Presence of positive adult supports • Level of mentorship during high school • Postsecondary attainment • Achievement on the SAT • FAFSA completion • Family experience level • Achievement on the SAT with college • Level of mentorship during high school • Support from MCAN advisors, counselors, and recruiters • Postsecondary attainment • Extent of eligibility for funds Postsecondary education • Increasing awareness of that college is doable • Family education • Postsecondary attainment • FAFSA completion level • Achievement on the SAT about career opportunities that don’t require a 4 year degree • Parental and student awareness of educational pathways and available support • Increasing awareness that college is doable • Family experience with college • Friends and family attitudes toward college 89 Table 6 (cont’d) Philanthropy • Programming • Availability of workforce development/program development in K-12 • Presence of an independent convener to understand and explain talent gap • Government funding • Foundation funding • Presence of an independent convener to understand and explain talent gaps • Programming • Presence of an independent convener to understand and explain talent gaps • Availability of workforce development in K-12 • Government funding • Foundation funding • Presence of an independent convener to understand and explain talent gaps 90 Mental model function I assessed mental model function using scenario modeling in the Mental Model software for both of the concepts that were given to each participant. The relative impact of a maximal shift in FAFSA completion level had impacts on postsecondary attainment levels ranging from 0, indicating that no change occurred at all, to .85, indicating a large change in postsecondary attainment. The relative change in postsecondary attainment when maximally shifting achievement on the SAT ranged from 0 to .83. These results are summarized in Figures 4 and 5. Figure 4. Relative impact of FAFSA completion level on postsecondary attainment 91 Figure 5. Relative impact of achievement on the SAT on postsecondary attainment Sector Content. I examined the concepts commonly discussed within each sector to assess the similarities and differences between them. For these analyses, I focused on the tags assigned to the concepts in order to assess content area, rather than the frequencies of individual concepts. I chose this approach because each sector included a small subset of the participants and thus there was not a high level of overlap among the concepts. There were several content areas in each sector that appeared in at least half of the members of the sector. Responses from the philanthropy sector were not included in this assessment because only one respondent from philanthropy participated in the study. Figures 2 and 3 demonstrate how mental models across sectors related to each other in terms of overlapping concepts and the overlapping assignment of tags to their concepts. K-12 educators discussed concepts from five content areas in at least half of their mental models. These content areas included finances and funding (7, 87.5%), counselors (5, 63.5%), family (5, 62.5%), knowledge (4, 50%), and academic 92 achievement (4, 50%). Participants from non-profit organizations also discussed family (4, 66.66%) and finances and funding (3, 50%) in their mental models. They additionally discussed soft skills (6, 100%), mentorship (4, 66.66%), and transportation (4, 66.66%). Participants working in postsecondary education also discussed Family (3, 100%), finances (3, 100%), and knowledge (3, 100%). Structure. In order to evaluate mental model structure within each of the participants’ sectors, I report the same degree centrality metrics from the overall analysis, using both a normalized approach to centrality and a raw centrality score approach. In-degree centrality. The three highest in-degree centrality concepts among participants working in K-12 education were postsecondary attainment using the normalized centrality approach were: (.48), awareness of nearby college options (.3), and level of career planning during postsecondary (.21). The highest in-degree centrality concepts using the raw centrality approach were: postsecondary attainment (34.7), FAFSA completion level (5.34), and Achievement on the SAT (4.34). The three highest in-degree centrality concepts using the normalized centrality approach among participants working in non-profit organizations were postsecondary attainment (.50), high school GPA (.24), and availability of scholarships (.22). The highest in-degree centrality concepts using the raw centrality approach were: postsecondary attainment, FAFSA completion level, and achievement on the SAT. The three highest in-degree centrality concepts among participants working in postsecondary education were: support from MCAN advisors, counselors, and recruiters (.56), postsecondary attainment (.52), and extent of eligibility for funds (.18). The highest in-degree centrality concepts using the raw centrality approach were: 93 postsecondary attainment (21.01), FAFSA completion level (3.67), and achievement on the SAT (2.33). The three highest in-degree centrality concepts among participants working in philanthropy were: programming (.43), availability of workforce development/program development in K-12 (.25), and the presence of an independent convener to understand and explain the talent gap (.25). The highest in-degree centrality concepts using the raw centrality approach were: programming (7), presence of an independent convener to understand and explain talent gaps (4), and availability of workforce development in K- 12 (4). Out-degree centrality. The three highest out-degree centrality concepts among participants working in K-12 education were: poverty level (.67), access to college advisors (.4), and how much support students get from high school (.30). The highest out-degree centrality concepts using the raw centrality approach were: family experience with college (8.01), FAFSA completion level (6.02), and poverty level (4.67). The three highest out-degree centrality concepts among participants working in non-profit organizations were: parent involvement (.41), presence of positive adult supports (.41), and level of mentorship during high school (.33). The highest out-degree concepts using the raw centrality approach were: achievement on the SAT (4.34), family experience with college (4.01), and level of mentorship during high school (3.67). The three highest out-degree centrality concepts among participants working in postsecondary education were: increasing awareness that college is doable (.21), family education about career opportunities that don’t require a 4 year degree (.14), and parental and student awareness of educational pathways and available support (.14). The highest out-degree concepts using the raw centrality approach were: increasing 94 awareness that college is doable (3), family experience with college (2), and friends and family attitudes toward college (2). The three highest out-degree centrality concepts from the participant working in philanthropy were government funding (.38), foundation funding (.31), and presence of independent convener to understand and explain talent gaps (.29). The highest out- degree centrality concepts using the raw centrality approach were: government funding (6), foundation funding (5), and presence of an independent convener to understand and explain talent gaps (4.68). Function. To evaluate the function of each mental model, I employed the scenario analysis tool in Mental Modeler. An overall summary of model function by sector is included in Figures 6 and 7. Among participants working in K-12 education, maximally manipulating the FAFSA completion level lead to a relative change in postsecondary attainment ranging from 0 to .85. Similarly, a maximal shift in SAT score levels in models from participants working in K-12 education lead to a relative change in postsecondary attainment that ranged from 0 to .76. Among participants working in postsecondary education, a maximal shift in FAFSA completion level lead to a shift in postsecondary attainment ranging from 0 to .76. A maximal shift in SAT score levels in models from participants working in K-12 education lead to a relative change in postsecondary attainment that ranged from .01 to .53. Among participants working in non-profit organizations, a maximal shift in FAFSA completion level lead to a shift in postsecondary attainment ranging from 0 to .76. A maximal shift in SAT score levels in models from participants working in non- profit organizations lead to a shift in postsecondary attainment ranging from 0 to .83. 95 The single participant working in a philanthropic organization or foundation did not include either of the concepts in their model. Thus, manipulating the concepts yielded no shift in postsecondary attainment. 96 Figure 6. Relative impact of FAFSA completion level on postsecondary attainment Figure 7. Relative impact of achievement on the SAT on postsecondary attainment 97 Discussion The purpose of this chapter was (1) to evaluate the extent to which mental models are similar and different within a single coalition action team; and (2) to assess similarities and differences by the sector participants work within. These results illuminate key areas for understanding variation within this particular coalition as well as for applying this approach to evaluating problem definitions across coalitions. The overall content of the models suggested that participants vary greatly in the way they think about the concepts relevant to postsecondary attainment in their county. There were no concepts or content areas that appeared in all of the participants’ mental models and only a few content areas that appeared in a majority of mental models, besides the concepts that they were given a priori. This suggests that in terms of content, the participants did not closely converge, but there seemed to be some content areas that held shared importance across coalition members. Because of this lack of content convergence, participants’ models would also not converge in their structural characteristics, aside from the concepts that each participant had at the outset, because they shared very few concepts which could have similar centrality scores. The centrality scores overall further demonstrated divergence in the importance of various concepts in their models. The most central concepts did not closely align with those that came up the most frequently in the content analysis, suggesting that the most common elements of participants’ mental models were not the most influential elements of the model. The outcome concept, postsecondary attainment held the highest in- degree centrality, which is promising as concepts with high in-degree centrality are most susceptible to influence within the mental models and the group’s primary focus is on influencing this concept. This is also an artifact of the mental modeling question, which 98 established postsecondary attainment as the outcome of interest, setting it up to be a receiver of influence from other concepts. Among the concepts with the highest centrality values, the difference between the normalized centrality approach and the raw centrality approach point to important differences in what can be considered the most influential concepts in the models. In the normalized centrality approach, the three concepts participants were given at the beginning of the modeling process did not rise to the top. In the raw centrality analysis, these concepts were among the most central overall. To understand this difference, we can consider how these two approaches establish important concepts using degree centrality. The raw approach is additive, meaning that a concept can achieve high centrality status based if it appears in many of the mental models. This values the knowledge of the crowd overall, but can raise up concepts that participants do not see as highly central at the individual level. Indeed, for both in-degree and out-degree centrality, very few participants assigned the highest centrality values through their assignment of weighted ties to FAFSA completion level or achievement on the SAT. Thus, two concepts that were not individually considered very central for many participants appeared as highly central. Using the normalized centrality approach, the concepts that rose to the top more frequently represented the concepts participants weighted as most central in their mental model ties, but these concepts were included in fewer mental models. Together, these both tell us important information about the problem of interest. The raw centrality scores suggest some level of agreement that FAFSA completion level and achievement on the SAT are relevant both as concepts that exert and receive influence. The normalized centrality scores suggest that there may be other concepts relevant to postsecondary attainment that small groups of participant 99 have in mind. Follow-up inquiry into these concepts may help to identify the role these concepts play in the problem of interest. In addition, the high raw in-degree and out-degree centralities of FAFSA completion level and achievement on the SAT may indicate the need for more inquiry surrounding these concepts. These concepts were selected for inclusion in the models because they appear on the group’s dashboard as targets for change. Their high raw in- and out-degree centrality indicate that these concepts are considered ‘ordinary’ in mental modeling terms, which means they receive and exert influence. Thus, as targets for change, participants have indicated that they can be influential, but their in-degree centrality scores would suggest that there are other concepts exerting influence over them. To fully understand how these concepts can be drivers of change, coalition members may want to take into consideration the concepts that exert influence over FAFSA completion and achievement on the SAT. Including these into the process of planning efforts to increase the level of FAFSA completion and achievement on the SAT can make their work easier by helping to identify additional points of intervention that will support these goals. The results of functional analyses with fuzzy cognitive maps can be understood through two lenses: the direction of the changes in the concepts and the magnitude of the changes. The direction indicates whether a change in a single concept or subset of concepts would yield positive or negative changes in other model concepts. Here, I focused on three concepts that participants were given to include in their models: postsecondary attainment, FAFSA completion, and achievement on the SAT. I wanted to assess how changes in FAFSA completion rates and achievement on the SAT changed postsecondary attainment. 100 Functional analysis demonstrated that all participants evaluated the common concepts in their models as having no effect or having a positive effect on postsecondary attainment. While this does not demonstrate consensus among participants, the lack of negative values for postsecondary attainment suggests that none of the participants viewed the group’s efforts as having a negative effect on postsecondary attainment. Though, there were still many participants who did not think these concepts had any influence over postsecondary attainment. Among those whose models yielded a positive change in postsecondary attainment the magnitude of that change varied greatly from one model to another. The results of the functional analysis invite further inquiry to uncover the knowledge and beliefs that generated these mental models and the data required to assess whether these indicators are impacting postsecondary attainment for the county the coalition serves. This is important to consider because choosing to consistently track a metric often leads to optimization for that metric, which means that those trying to make change will focus their efforts on changing the outcomes that are measured, even when they do not achieve the desired overall impact (Elton, 2004). If these indicators are not the most appropriate for increasing postsecondary attainment, the coalition may be optimizing metrics that are not likely to lead to success. Sector analysis The content of the models across each sector reflected many of the content areas that appeared most frequently in the data set as a whole. However, there were specific content areas within each sector that appeared to reflect some of the unique concepts that may be relevant within each sector. For example, K-12 educators were the only group that included concepts related to academic achievement and counselors in more 101 than half of their models. Similarly, individuals working in non-profit organizations disproportionately discussed soft skills, mentorship, and transportation in their mental models. These differences suggest that it may be important for multisectoral representation among participants in the coalition in order to capture the full spectrum of concepts relevant for understanding the problem of interest. This is consistent with research on diversity in groups, which suggests that models of problems produced by diverse groups tend to be better than models produced by homogenous groups (Hong & Page, 2004; Page, 2007). Sector-specific concepts also emerged as important in my assessment of centrality by sector using the normalized centrality approach. For example, the participant from the philanthropic sector had foundation funding and government funding as highly central in their mental model. Similarly, the highest centrality concepts in postsecondary education focused on issues related to paying for postsecondary education, something that is likely to be very salient for individuals working in colleges and trades schools. These included things like the extent to which students are eligible for funds, how much funding they receive from FAFSA, and level of knowledge about getting financial aid approved. There is a great deal of variance in the concepts that appeared as most central between sectors, but postsecondary attainment was an anomaly in that it appeared among the highest in-degree centrality concepts in four of the five sectors. This makes sense because it is the concept the participants are actively working on changing in their community and it is expected that so many of them see it as highly influenceable. The raw centrality scores again pointed at FAFSA completion level and achievement on the SAT as influential concepts across all sectors, with the exception of 102 philanthropy. This is a function of the way raw centrality is computed, as there was only one participant in this category, so the raw centrality values were the same as the values listed as that participant’s most central concepts. This points to the challenges with using raw centrality values to determine the importance of concepts in a model, as the concepts that were most central for participants do not always rise to the top when they are assessed in aggregate. Functional analyses of the models did not show consistency within sectors. Each sector with more than one participant included both models with no change in postsecondary attainment as well as models that showed a relatively high change in postsecondary attainment as a result of manipulating the two concepts of interest. This suggests that sectoral groupings may not be indicators for mental model functional behavior. One explanation for this may be that individuals within sectors focus on different types of postsecondary education. For example, within K-12 education, some counselors may focus on supporting students interested in attending trade schools, while others are focused on supporting students interested in four year universities. For the counselor that focuses on trade school, FAFSA completion level and achievement on the SAT may not seem very salient as those are often not part of the process of enrolling. Conversely, those may be very salient for a counselor supporting students interested in four year universities. This distinction may be important for members of each sector and follow up studies can explore whether it is a meaningful way of grouping participants to better understand their perspectives on this particular problem. Limitations These findings should be interpreted in light of a few limitations. The analyses presented here are based on a small sample. This is particularly limiting for analyses by 103 sector as patterns can be difficult to discern with so few observations. With a 78% response rate, I am able to represent the mental models of the majority of coalition members. This indicates that it would not be appropriate to generalize widely to all coalition settings or to all sectoral settings, but the findings do represent the coalition from which I collected data. In addition, fuzzy cognitive mapping is one way to capture mental models, and it has drawbacks. It is an unusual task for participants and may not be as straightforward as answering interview questions, but it allows for analyses that take into account content, structure, and function of participants’ mental models. This approach meant that I was able to engage in more computational analyses, but a more typical unstructured or semi-structured interview may have provided a greater depth of information overall. Future directions Future research can continue to build on this work by applying this approach to assessing similarities and differences in more coalition contexts. Researchers can work with larger coalitions to collect larger sample size data and discern further patterns among subgroups working on coalition problems. Doing this can help to establish the way that participation in subgroups may relate to problem definitions. Collecting data from a variety of coalitions can illuminate the role that different types of issues or contexts may play in the degree of variation that exists in problem definitions among coalition members. In addition, researchers can conduct longitudinal studies to examine how mental models change over time and what fs may relate to changes in mental models. Future studies may also apply different approaches to collecting mental models beyond fuzzy cognitive maps. This can help to unpack how different approaches 104 facilitate the process of making mental models explicit and the nuances of when it is most appropriate to use each. Researchers should also explore how differences and similarities in problem definitions influence a coalition’s ability to function. Theoretical perspectives on coalitions suggest that a shared problem definition is central to success for coalitions (Kania & Kramer, 2011; Foster-Fishman & Watson, 2011; Lawlor & Neal, 2016, Chavis, 2001). In chapter 5, I will explore how similarities in problem definition relate to collaboration patterns, but additional future research can explore the extent to which these differences are problematic in practice and how mental model diversity can best be leveraged to support coalitions in taking action on their desired outcomes. Future research can also explore the lack of convergence in mental model functioning. As the concepts for the functional analysis were selected from the group’s dashboard, it may be helpful to revisit how group members selected these concepts for continuous measurement and the mechanisms by which the group expected those concepts could create desired changes in their targeted outcome, postsecondary attainment. A facilitated group process may help to surface the particular circumstances that lead to these indicators being chosen. A group model building process can facilitate knowledge coordination among coalition members to create a model that represents the contributions or lack of contribution that these concepts make in increasing postsecondary attainment in their community. A group modeling approach would also allow participants to iteratively create a model and test assumptions they may have about how the system of interest operates. A similar approach may be worth exploring for other coalitions or community groups as they revisit and make decisions about what indicators would demonstrate success. 105 Conclusions The purpose of this chapter was to describe the extent to which mental models of a central problem within a coalition vary. I found that they varied significantly across content, structure, and functional analyses. Participants in similar sectors shared more similarity in the concepts they endorsed in their models, but did not show strong convergence in terms of the function of their models. Overall, this suggests that participants do not subscribe to a single problem definition for postsecondary attainment and that each participant may have their own perspective and knowledge of the problem. Future research should consider how to further bring stakeholder perspectives together to define problems and identify points of action. Further, research should consider how these differences in problem definition relate to other aspects of coalition functioning and whether this inhibits or facilitates the work that the group is trying to do. 106 APPENDICES 107 APPENDIX A RECRUITMENT MATERIALS 108 Appendix A. Recruitment materials =======INFORMATIONAL FLYER========= Michigan State University College Access Network Study Volunteers Wanted for a Research Study Purpose: The study is about how participants in action teams in college access networks think about the problems they work on and how they collaborate with each other. Eligibility: To be eligible, participants must be members of the Postsecondary Success or Adult Learner action team in the [Midwest County] County Career and College Access Network. Research Activities: Participants will engage in a short interview about how they think about postsecondary attainment in their community and complete a survey about how they collaborate with others in the community and answer several demographic questions. All questions are optional and responses will remain confidential. The process will take about 45 minutes of your time. Participants will receive a $15.00 Amazon.com gift card as a thank you for participating. Interested? You can sign up to participate here: [LINK TO QUALTRICS FORM TO SIGN UP] or you can email the researcher at lawlorje@msu.edu with your availability. Questions? To contact the researcher: Jennifer Lawlor lawlorje@msu.edu Michigan State University Department of Psychology, Room 262 316 Physics Rd. East Lansing, MI 48824 Subject line: =======GENERAL EMAIL SCRIPT========= 109 Opportunity to participate in MSU research study Email body: Greetings! I hope this message reaches you well! You are being invited to participate in a study about college access networks run by Jennifer Lawlor at Michigan State University. The study is about how participants in action teams in college access networks think about the problems they work on and how they collaborate with each other. It involves a short interview and a survey. If you agree to participate, all questions are optional and your responses will be confidential. You will also receive a $15.00 Amazon.com gift card as a thank you for participating. If you are willing to participate in a ~45 minute interview and survey, you can sign up here: [Link to Qualtrics form to sign up] or contact Jennifer Lawlor directly at lawlorje@msu.edu. Please reply with a few days/times that would work best for you to participate. Thank you in advance for your help with my research. Sincerely, [Name of sender] =======EMAIL SCRIPT FROM JENNIFER========= Subject line: Opportunity to participate in MSU research study Email body: Greetings! I hope this message reaches you well! I am a graduate student researcher at Michigan State University. I am contacting you because I am doing a study about 110 college access networks. Your name was given to me by Doug Wright as someone I should talk to. I am trying to learn more about how participants in action teams in college access networks think about the problems they work on and how they collaborate with each other. I would love to learn about your experiences with a short interview and survey. If you agree to participate, all questions are optional and your responses will be confidential. You will also receive a $15.00 Amazon.com card as a thank you for participating. Please reply to this email or sign up here [LINK TO QUALTRICS FORM] to let me know if you are willing to participate in a ~45 minute interview and survey. If so, what dates and times might work best for you? Thank you in advance for your help with my research. Sincerely, Jennifer Lawlor =======GENERAL ANNOUNCEMENT SCRIPT========= Hi everyone! Jennifer Lawlor, a graduate student at Michigan State is conducting a study about college access networks. She is specifically studying how participants in action teams in college access networks think about the problems they work on and how they collaborate with each other. You are all invited to participate and share your experiences with a short interview and survey lasting about 45 minutes. If you agree to participate, all questions are optional and your responses will be confidential. You will also receive a $15.00 Amazon.com gift card as a thank you for participating. If you are interested in participating, please email Jennifer directly at lawlorje@msu.edu with several dates/times that would work best for you to participate or sign up at [INSERT LINK TO QUALTRICS FORM]. 111 =======ANNOUNCEMENT SCRIPT FROM JENNIFER========= Hi everyone! My name is Jennifer Lawlor and I am a graduate student researcher at Michigan State University. I am here because I am doing a study about college access networks. I am trying to learn more about how participants in action teams in college access networks think about the problems they work on and how they collaborate with each other. I would love to learn about your experiences with a short interview and survey, lasting about 45 minutes. If you agree to participate, all questions are optional and your responses will be confidential. You will also receive a $XX Amazon.com gift card as a thank you for participating. Please email me at lawlorje@msu.edu to let me know if you are willing to participate in a 45 minute interview and survey. If you are interested in scheduling an interview now, I am happy to set up a time. You can also sign up at [INSERT LINK TO QUALTRICS FORM]. Thank you in advance for your help with my research. 112 APPENDIX B IRB APPROVAL AND EXEMPT DESIGNATION 113 Appendix B. IRB approval and exempt designation 114 APPENDIX C CONSENT DOCUMENT 115 Appendix C. Consent document College Access Network Study Research Participant Information and Consent Document 1. EXPLANATION OF THE RESEARCH and WHAT YOU WILL DO: You are being asked to participate in a research study to learn more about how you think about the work you do as a member of an action team in the [Midwest County] Career and College Access Network. You will be asked interview questions about how you think about the problems that you work on. You will also be asked to complete a short survey about your experiences and your collaborators in the network. 2. YOUR RIGHTS TO PARTICIPATE, SAY NO, OR WITHDRAW: Participation in this research project is completely voluntary. You have the right to say no. You may change your mind at any time and withdraw. You may choose not to answer specific questions or to stop participating at any time. 3. COSTS AND COMPENSATION FOR BEING IN THE STUDY: You will receive a $XX.00 gift card for Amazon.com as a thank you for participating. There is no cost to participate, but it will take about an hour of your time. 4. CONTACT INFORMATION FOR QUESTIONS AND CONCERNS: If you have concerns or questions about this study, such as scientific issues, how to do any part of it, or to report an injury, please contact the researcher: Jennifer Lawlor lawlorje@msu.edu Michigan State University Department of Psychology, Room 262 316 Physics Rd. East Lansing, MI 48824 If you have questions or concerns about your role and rights as a research participant, would like to obtain information or offer input, or would like to register a complaint about this study, you may contact, anonymously if you wish, the Michigan State University’s Human Research Protection Program at 517-355-2180, Fax 517-432-4503, or e-mail irb@msu.edu or regular mail at 4000 Collins Rd, Suite 136, Lansing, MI 48910. 116 APPENDIX D MENTAL MODEL INTERVIEW PROTOCOL 117 Appendix D. Mental model interview protocol Interview materials: - Computer with Mental Modeler - Example model loaded up on the computer - Copy of the survey - Back up materials in case there is a tech failure: paper, pencils, paper copy of example - Audiorecorder Introduction: Thanks for meeting with me today! I’m so excited to get started, but first because this is a research project, we need to discuss your rights as a research participant. Consent process: - Read through the consent document with the participant, point out the key areas for them to know about - Make it clear they can contact me with any questions or concerns about the study or contact MSU with any questions or concerns about participating in research Do you have any questions before we move on? General Instructions: Today we’ll be doing an interview to learn more about how you think about the work your action team engages in. As part of the interview, we’ll be making something called a ‘mental model.’ This is just a picture of the kinds of things you think about when you think about the work you do. There are no right or wrong answers and this isn’t a test. As we talk, I’ll be entering some of your comments into this program to capture what your mental model looks like. As we go through the interview, we can make any changes or add anything that you want to the model. Before we get started, is it okay if I record our conversation? This will just make it easier for me to keep track of what we say if my notes don’t cover everything. The recording will not be shared with anyone outside the research team. Warm-up questions: 1. Can you tell me a little bit about yourself? 2. In your experience, what influences postsecondary attainment in [Midwest County]? Walk through a brief example: 118 Let’s walk through a brief example of a mental model to show you how they work and what the process will look like when we do it together. We’ll use preschool education as an example. The question we want to answer is what influences children’s participation in preschool? If we wanted to capture a mental model that demonstrates the factors influencing children participating in preschool education, we would start with a central factor. We’ll call this factor children participating in preschool education. Next, we can brainstorm as many factors as we can think of that might influence preschool participation. Here are a few potential factors: accessible transportation to preschool, number of spots available in preschools for students to enroll, accessibility of necessary health screenings, number of teachers available to staff preschools. (Note that each of these factors are things that can increase or decrease. This is one part of generating a mental model that can be a little bit confusing. It’s typically pretty easy to frame ideas as something that can increase or decrease, but if it’s challenging at any point, we can talk through it). Next, we can start with these factors and ask how they influence each other and how they influence the number of children participating in preschool. As we make each connection, we can assign it a strength. The strengths are relative values, so you can think about how strong something is within the context of the strength of other connections. For this activity, we can refer to a relationship as weak, moderate, or strong. [With participant, have them select some relevant concepts and discuss how those things might relate to each other and to the outcome of interest] Do you have any questions before we move on? Mental Model Questions: Just like in our example, we’ll start with a central concept that we can build on to understand the problem of interest: What influences postsecondary attainment in [Midwest County]? We’ll call this factor postsecondary attainment. This refers to the number of students completing a postsecondary program, including an associates degree, a bachelor's degree, or a technical training program. For this interview, we’re going to follow that same process, but it will be different in a couple of ways. First, there are two concepts that I’m going to ask you to include in your map. You do not have to connect these concepts to anything else if you don’t feel like they influence the other concepts in your map, but they will be available as part of your list of concepts. Second, I am going to ask questions as we go along to understand what each of your concepts refers to and to make sure I understand its relationship to the other concepts in the map. Do you have any questions about that? 119 Now, we will brainstorm a list of concepts that is relevant in your opinion to understanding how this problem works in your community. But first, I will offer two concepts to include in this list. The first is FAFSA completion. This refers to the number of students submitting a FAFSA form each year. The second is achievement on the SAT. This refers to the level of SAT scores that students achieve. Now, let’s talk about the concepts or factors you think are relevant to 1. postsecondary attainment in your community. What factors or concepts do you think are relevant to postsecondary attainment in your community? a. PROBE: [As they list these items, ask for definitions of each] Can you tell me how you define [X]? In what ways is [X] different from other terms, like [X]? b. PROBE: [If something seems like it might be two concepts instead of one] Can we separate that out in to [X] and [X]? Is this two ideas or one? c. PROBE: Can you think of any other relevant concepts? Remember, we can always add more later if you think of anything else. 2. Now, let’s talk about how these concepts or factors are related to each other. Starting with the two factors that we started with, how do they relate to each other? a. How do these factors relate to the postsecondary attainment? 3. Now, let’s discuss the other concepts you brainstormed. How do these factors relate to each other? a. How do these factors relate to postsecondary attainment? b. [FOR EACH]: How strong is the relationship? Is it a weak [.33], moderate [.66], or strong [1] relationship? 4. Would you like to add any additional concepts now that we’ve got a start to the map? a. PROBE: Can you tell me how you define [X]? b. PROBE: In what ways is [X] different from other terms, like [X]? 5. How do these concepts or factors relate to the other factors listed here or postsecondary attainment? 6. That’s all the questions I have for our interview. Is there anything else you’d like for me to know while we’re discussing this? 120 APPENDIX E SURVEY QUESTIONNAIRE & CODEBOOK 121 Appendix E. Survey Questionnaire & codebook Survey Questionnaire 122 123 124 125 Survey Codebook (Note that this codebook does not include values for the network data. These responses will be structured in a matrix with values of 1 for the presence of a connection and 0 for the absence of a connection.) Variable Name Description Values Data Type ID Participant ID number 1-30 Nominal Job_title What is your current job String title? Years_position How many years have >0 Ratio you worked in your current position? Sector How would you describe the sector you primarily work in? Nominal - K-12 education - Postsecondary education - Non-profit organization - For-profit organization - Government - Other: ____ Years_coalition How many years have you been involved in the > 0 Ratio [Midwest County] Career and College Access Network? 126 Highest_ed What is the highest level of school you have completed or the highest degree you have received? Nominal - Less than high school degree - High school graduate (high school diploma or equivalent including GED) - Some college but no degree - Associate degree in college (2-year) - Trade school or apprenticeship - Bachelor's degree in college (4-year) - Master's degree - Doctoral degree Professional degree (JD, MD) Birth_year In what year were you > ~1900 Year born? White How would you describe your race: White Black How would you describe your race: Black or African American 0 = No 1 = Yes 0 = No 1 = Yes Nominal Nominal 127 Am_In How would you describe your race: American Indian or Alaskan Native Asian How would you describe your race: Asian Native_HI How would you describe your race: Native Hawaiian or Pacific Islander 0 = No 1 = Yes 0 = No 1 = Yes 0 = No 1 = Yes Nominal Nominal Nominal Race_other How would you describe your race: Other 0 = No 1 = Yes Nominal Race_specify If participant selects other, how do they identify? String Latino Would you describe yourself as Hispanic or Latino/a? 0 = No 1 = Yes Nominal Gender How would you describe String your gender? 128 APPENDIX F SENSITIVITY ANALYSIS 129 Appendix F. Sensitivity analysis In order to test how differing decisions about standardizing the concepts may change the outcomes of the ERGM, I also ran an additional analysis on a lower resolution set of standardized mental models. The low resolution mental models use the tags to determine whether concepts are the same. In this case, if concepts have been assigned the same tag (i.e., discuss the same topic), they are considered to be the same. This means that more concepts are considered to be the same and the minute difference between them are not factored into the assessment of which concepts are the same. The lower resolution model yielded similar results to the original ERGM analysis, indicating that the findings were not sensitive to changes in the standardizing process used to identify similarities among the models (results are summarized in Table 7). Adding mental model similarity to the full model in the low resolution analysis generated a maximum likelihood estimate of .58 with a standard error of .82 and a p- value of .47. Goodness of fit tests also indicated that these models also had good fit with the original network. They did not indicate better fit on any of the components tested than on the original ERGM. Table 7. Liberal coding sensitivity analysis Parameter Null Covariate Full Edges Gender homophily Sector homophily Length of time in the coalition Mental model similarity -1.14 (.15)* -4.30 (.66)* -4.38 (.67)* .56 (.34) .34 (.40) .52 (.10)* .53 (.35) .36 (.40) .51 (.10)* .58 (.82) I also conducted a sensitivity analysis to determine whether removing the non- participants from the network listwise would change the outcomes of the ERGM analysis. This could increase the amount of variance in the data and potentially generate 130 a different result. However, re-running the ERGM without the missing participants did not change the outcomes. The ERGM model for this sensitivity analysis is summarized in Table 8 below. The full model with mental model similarity included a maximum likelihood estimate of .75 with a standard error of .88 and a p-value of .39. Table 8. Missingness sensitivity analysis Parameter Edges Gender homophily Sector homophily Length of time in the coalition Mental model similarity Null Covariate Full -.88 (.18)* -3.48(.72)* -3.57(.73)* .24(.38) .13(.41) .43(.11)* .24(.38) .14(.41) .43(.11)* .75(.88) 131 APPENDIX G EXAMPLE FUNCTIONAL ANALYSIS OF A FUZZY COGNITIVE MAP 132 Appendix G. Example functional analysis of a fuzzy cognitive map To demonstrate the process by which a fuzzy cognitive map can be assessed for functional behavior, I present a basic example given in Jetter and Schweinfort (2011). The authors demonstrate a basic model with four concepts. They also selected a simple activation function in which a concept is activated if its value is greater than one and not activated if its value is lower than one. I will walk through the example: First, the authors generated a matrix to represent the model: C1 C2 C3 C4 C1 0 0 1 0 C2 0 0 -1 0 C3 0 0 C4 0 0 0 0 1 0 Second, they selected a concept of interest to test the function of the model. This means that they selected a concept to manipulate in order to see how other model concepts would behave in response. In this case, it was concept C1. Third, they multiplied the matrix by a vector representing their manipulation by the original matrix: Manipulation: [1, 0, 0, 0] (note that the first entry in the matrix is marked with a ‘1.’ This is the concept they are changing.) The resultant vector is: Step 1: [1, 0, 0, 0] They continue to multiply the resultant vectors by the manipulation until the model reaches a steady state (this occurs after four steps): Step 2: [1, 0, 1, 0] Step 3: [1, 0, 1, 1] Step 4: [1, 0, 1, 1] Thus, they can assess that increasing the concept C1 will lead to a resultant increase in C3 and C4. 133 REFERENCES 134 REFERENCES American Community Survey (2016). “Demographic and housing estimates,” “Selected economic characteristic,” “Educational attainment.” Retrieved from: https://factfinder.census.gov/faces/nav/jsf/pages/community_facts.xhtml Banini, R. & Bearman, G.A. (1998). Application of fuzzy cognitive maps to factors affecting slurry rheology. International Journal of Mineral Processing, 52, 233- 244. Batagelj, V., & Cerinšek, M. (2013). On bibliographic networks. Scientometrics, 96(3), 845–864. http://doi.org/10.1007/s11192-012-0940-1 Chavis, D. M. (2001). The paradoxes and promise of community coalitions. American Journal of Community Psychology, 29(2), 309–320. Clarke, I., & Mackaness, W. (2001). Management “Intuition”: an Interpretative Account of Structure and Content of Decision Schemas Using Cognitive Maps. Journal of Management Studies, 38(2), 147–172. http://doi.org/Article Elton, L. (2009). Goodhart’s Law and Performance Indicators in Higher Education. Evaluation & Research in Education, 18(1–2), 120–128. http://doi.org/10.1080/09500790408668312 Freeman, L. C. (1978). Centrality in Social Networks. Social Networks, 1(1968), 215– 239. http://doi.org/10.1016/0378-8733(78)90021-7 Felix, G., Nápoles, G., Falcon, R., Froelich, W., Vanhoof, K., & Bello, R. (2017). A review on methods and software for fuzzy cognitive maps. Artificial Intelligence Review, 1–31. http://doi.org/10.1007/s10462-017-9575-1 Foster-Fishman, P. G., & Watson, E. R. (2011). The ABLe change framework: a conceptual and methodological tool for promoting systems change. American Journal of Community Psychology, 49(3–4), 503–16. http://doi.org/10.1007/s10464-011-9454-x Gray, S. (2015). Introduction to mental modeler. [Video] Retrieved from: https://www.youtube.com/watch?time_continue=2&v=UbKzyDctkrY Gray, S. Gray, S., Cox, L., and Henly-Shepard, S. (2013). Mental modeler: A fuzzy-logic cognitive mapping modeling tool for adaptive environmental management. Proceedings of the 46th International Conference on Complex Systems. 963-973. Gray, S. A., Zanre, E., & Gray, S. R. J. (2014). Fuzzy cognitive maps as representations of mental models and group beliefs. In Fuzzy cognitive maps for applied sciences and engineering(pp. 29-48). Springer, Berlin, Heidelberg. 135 Hong, L., & Page, S. E. (2004). Groups of diverse problem solvers can outperform groups of high-ability problem solvers, 101(46), 16385–16389. Jetter, A.J., Gray, S.A., Ellsworth, L.M., Zhang, P., Singer, A. & Laraichi, M.O. (2017). Policy scenarios for fire-adapted communities: Understanding stakeholder risk- perceptions, using fuzzy cognitive maps. Report for the Joint Fire Science Program. Jetter, A. & Schweinfort, W. (2011). Building scenarios with fuzzy cognitive maps: An exploratory study of solar energy. Futures, 43(1), 52-56. DOI: https://doi.org/10.1016/j.futures.2010.05.002 Kania, B. J., & Kramer, M. (2011). Collective Impact. Stanford Social Innovation Review, 1(9), 36–41. Kearney, A.R., Kaplan, S. (1997). Toward a methodology for the measurement of knowledge structures of ordinary people: The conceptual content cognitive map (3CM). Environment and Behavior, 29(5), 579–617. Khan, M. & Quaddus, S.M. (2004). Group decision support using fuzzy cognitive maps for causal reasoning. Group Decision and Negotiation, 13, 463-480. Kosko, B. (1988). Hidden patterns in combined and adaptive knowledge networks. International Journal of Approximate Reasoning, 2, 377–393. Lawlor, J. A., & Neal, Z. P. (2016). Networked Community Change: Understanding Community Systems Change through the Lens of Social Network Analysis. American Journal of Community Psychology. http://doi.org/10.1002/ajcp.12052 Lincoln, Y.S. & Guba, E.G. (1985). Naturalistic inquiry. Newbury Park, CA: Sage Publications. Mago, V. K., Morden, H. K., Fritz, C., Wu, T., Namazi, S., Geranmayeh, P., ... & Dabbaghian, V. (2013). Analyzing the impact of social factors on homelessness: a Fuzzy Cognitive Map approach. BMC medical informatics and decision making, 13(1), 94. Midland College and Career Access Network (2017). Dashboard. Retrieved from: https://static1.squarespace.com/static/53a43313e4b04dade8f3b406/t/596cd57c 1b631b29937d5f4a/1500304764685/LCAN+Dashboard+2017.pdf Mouratiadou, I. & Moran, D. (2007). Mapping public participation in the water framework directive: A case study of the Pinios River Basin, Greece. Ecological Economics, 62, 66-76. Doi: 10.1016/j.ecolecon.2007.01.009 136 Newman, M. (2010). Networks: an introduction. Oxford university press. Özesmi, U., & Özesmi, S. L. (2004). Ecological models based on people’s knowledge: a multi-step fuzzy cognitive mapping approach, 176, 43–64. http://doi.org/10.1016/j.ecolmodel.2003.10.027 Page, S. E. (2007). Making the difference: Applying a logic of diversity. Academy of Management Perspectives, 21(4), 6-20. Prell, C. (2012). Social network analysis: History, theory, & methodology. London: Sage Publications. Rochat, Y. (2009). Closeness centrality extended to unconnected graphs: The harmonic centrality index. In ASNA(No. EPFL-CONF-200525). Rouse, W.B. & Morris, N.M. (1986). On looking into the black box: Prospects and limits in the search for mental models. Psychological Bulletin, 100(3), 349-363. Tsadiras, A. K. (2008). Comparing the inference capabilities of binary, trivalent and sigmoid fuzzy cognitive maps. Information Sciences, 178(20), 3880–3894. http://doi.org/10.1016/j.ins.2008.05.015 Wang, S. (1996). A Dynamic Perspective of Differences Between Cognitive Maps. Journal of the Operational Research Society, 47(4), 538–549. Wei, W., Pfeffer, J., Reminga, J., & Carley, K. (2011). Handling weighted, asymmetric, self-looped, and disconnected networks in ORA. Yoon, B.S. & Jetter, A.J. (2016) Comparative Analysis for Fuzzy Cognitive Mapping. Engineering and Technology Management Faculty Publications and Presentations. 112. 1897-1908. Retrieved from: http://pdxscholar.library.pdx.edu/etm_fac/112 137 Introduction CHAPTER 5 In this chapter, I evaluate my second research question: To what extent does similarity in mental models predict collaboration among coalition members? To understand the relationship between mental models and collaboration, I employed an exponential random graph model (ERGM) analysis. ERGMs are a type of regression model used to understand what independent variables predict the presence or absence of network ties (Robins, Pattison, Kalish, & Lusher, 2007). In this case, I apply a measure of mental model similarity to predict whether collaborative ties exist between actors, controlling for demographic and structural covariates. Model similarity was computed using Pearson correlations of the matrix representations of the mental models. Here, I describe the variables included in the ERGM and my plans for analyzing the data. Variables Collaboration. Collaboration is a measure of the individuals within the same action team who engage in collaborative behaviors as defined by Mattessich, Murray- Close, and Monsey (2001): “a mutually beneficial and well-defined relationship entered into by two or more organizations [or individuals] to achieve common goals. The relationship includes a commitment to: a definition of mutual relationships and goals; a jointly developed structure and shared responsibility; mutual authority and accountability for success and sharing of resources and rewards.” These types of ties are non-directed as collaboration actively engages both parties; however, that does not mean that the ties need to be confirmed. For the purposes of this study, a collaboration between two respondents is treated as present if either or both report collaborating. 138 Model similarity. Model similarity was evaluated based on the degree of overlap in the concepts and relationships in the participants’ mental models. There are many possible ways to compare mental models, and I considered three of them as options for this analysis: Pearson correlations, jaccard coefficients, and model functional similarity. Pearson correlations and jaccard coefficients are both common approaches to comparing networks (Leydesdorff, 2008; Nobi, Lee, Kim & Lee 2014; Wang, 1996). I selected the Pearson correlation in order to capture the most information from each model for the comparison. As a reminder, the relationships between concepts are weighted values between -1 and 1. The benefits and drawbacks of each of these approaches are outlined in Table 9 below. Pearson correlations can be computed with the following equation: (cid:1) = (cid:4)Σ(cid:6)(cid:7) − (Σ(cid:6))(Σ(cid:7)) (cid:11)(cid:12)(cid:4)Σ(cid:6)(cid:13) − (Σ(cid:6))(cid:13)(cid:14)(cid:12)(cid:4)Σ(cid:7)(cid:13) − (Σ(cid:7))(cid:13)(cid:14) Where r = the degree of similarity of two mental models N = the number of pairs of scores (i.e., the number of pairs of concepts) Σxy = the sum of the products of paired scores Σx = the sum of x scores Σx = the sum of y scores Σ(cid:6)(cid:13)= the sum of squared x scores Σ(cid:7)(cid:13) = the sum of squared y scores We can compare two graph matrices of mental models using this equation by first transforming the matrices into vectors (this reshapes the data into columns that can be compared rather than matrices). Then, the data can be input into the equation like any other correlation. Thus, it evaluates similarity in the relationships (or ties), the strength 139 of the relationships, and the concepts in the graphs. Another approach to comparing two mental models is the jaccard coefficient. It can be computed with the following equation: (cid:17)((cid:6),(cid:7)) = |(cid:6) ∩ (cid:7)| |(cid:6) ∪ (cid:7)| Where (cid:17)((cid:6),(cid:7)) = the degree of similarity of two mental models |(cid:6) ∩ (cid:7)| = the number of relationships between concepts shared between both models |(cid:6) ∪ (cid:7)| = the total number of relationships in both models, shared and unshared This equation is similar to the Pearson correlation in that it includes both relationships and concepts by considering which relationships among concepts are the same in both models. They differ in that jaccard coefficients cannot include the value of the relationship, thus removing a layer of relevant information for comparing the models. There is also a third option for comparing mental models: functional similarity of models. This approach is outlined in chapter four as a means of describing how mental models behave. To use this approach, I would employ scenario modeling to compare models against their highest level of one of the variables I asked them to include in their model. The relative change in the outcome variable, postsecondary attainment, would be used to identify the level of similarity among models. This approach incorporates model structure and concepts in a less direct way. The components of the model and their relationships are driving its behavior, but it is possible that two models can yield similar outcomes without having similar structure or concepts. However, it can be used to compare models without standardizing the names of the concepts between them. Each of these approaches offers a potential way to compare mental models and each has been used in the comparison of mental models (Yoon & Jetter, 2016) or for comparing network graphs (Szell, Lambiotte & Thurner, 2010; Borgatti & Everett, 140 2000). For this research question, I wanted to incorporate as much of the mental model data as possible into the quantitative comparison. Thus, Pearson3 was the most appropriate approach to comparison as it compares the relationships among concepts present in both mental models and it includes the strength of the relationship, which is not possible with a jaccard coefficient. In cases where there was missing data for these values, I imputed the mean Pearson correlation. This follows guidance from Koskinen and Snijders (2013) about appropriate strategies for imputing attribute values in ERGM analyses. Table 9. Benefits and drawbacks of model comparison options Approach Benefits Drawbacks Pearson correlation Incorporates tie strength and similar concepts Does not incorporate model function Jaccard coefficient Incorporates similar concepts and similar ties Does not incorporate tie strength or model function 3 During analysis, I also computed correlations between mental models using a spearman correlation. This did not generate meaningfully different results than a pearson correlation in the ERGM. 141 Table 9 (cont’d) Model scenario output comparison Control variables Resilient to issues with standardizing concepts in the models Demonstrates how models behave with regard to concepts that the group has already deemed to be important Does not directly measure overlap in concepts or their relationship to each other (i.e., two very different models may yield similar outcomes) Length of time in the coalition. Participants were asked to indicate how long they had participated in the coalition. Coalition participation may influence individual problem definition due to the interactions or lack of interactions among participants in the group and it may influence the network ties an individual has. For example, if a member has just moved to the area and started working with the group, they may have few connections overall regardless of model similarity. This question was framed in the survey as follows: “How long have you participated in the [Midwest County] College and Career Access Network?” and participants were asked to respond in years. I asked them how long they had participated, rather than how long they had been members of the coalition because it is possible that participants were engaged before officially becoming members of the group and that engagement would still be relevant as an influence on their mental models and network ties. The number of years of participation were included as a continuous node covariate in the model. In cases where data was missing from this variable, I imputed the mean value for length of time in the coalition, following guidance from Koskinen and Snijders (2013). 142 Sector. Participants were asked to select the sector in which they work. I chose this question because theoretical approaches collaborative work in coalitions suggest that individuals who work in a similar area may think about the problems they work on in a similar fashion (Foster-Fishman & Watson, 2011; Kania & Kramer, 2011). For example, individuals working in the K-12 education sector on postsecondary attainment issues may be more focused on the role that the school system plays in preparing students for postsecondary experiences. Conversely, individuals working in the social services sector may be more inclined to see the problem as having to do with particular issues that families deal with in supporting students’ postsecondary attainment. Thus, participants were asked to select one of several options for a sector on the survey (e.g., K-12 education, non-profit, postsecondary education) and had the opportunity to input their own sector if they did not feel the options matched the sector in which they work. During analysis, this is treated as node-level categorical variable, assessing match between participants on its value. Moving forward, I will refer to this variable as sector homophily to indicate that it evaluates the match between participants. Race & gender. Participants were also asked to describe how they identify their race and gender. Race and gender are characteristics that often predict network ties in these types of networks (McPherson, Smith-Lovin, & Cook, 2001). These types of ties tend to form in accordance with homophily (i.e., people who share a characteristic are more likely to share a tie). Participants were homogenous with respect to race, and therefore there was limited opportunity to observe racial homophily. However, participants were diverse with respect to gender, for which homophily could be observed. I treated gender as a node-level categorical variable, evaluation match between participants. 143 Social network data collection To collect collaboration data, each participant was presented with the definition of collaboration outlined in Mattessich, Murray-Close, and Monsey (2001). Participants were then asked to select the names of other individuals from their coalition’s action team with whom they engage in collaboration. They were provided with a roster with each participant’s name in order to facilitate their recall of individuals with whom they may have collaborative relationships. Using rosters in social network analysis facilitates participant recall and can generate more accurate results than asking participants to generate names from memory (Borgatti, Everett, & Johnson, 2013). Participants were instructed that they could select as many or as few individuals in their as collaborators as they would like. Limiting the number of individuals a participant can select from the roster can lead to challenges in data interpretation, as it places a ceiling on the variation that is observable and can encourage participants to indicate relationships with individuals they do not have relationships with in order to meet the limit (Prell, 2012). Thus, the survey was formatted with check boxes for participants to endorse all individuals with whom they have relationships. Participants also had the opportunity to name any additional individuals who were missing from the roster of members of the group. A draft of the network survey is included in Chapter 4, Appendix F. It includes a list of pseudonyms for the purposes of review. They were replaced with action team members’ names during data collection. In the analysis stage, relationships only needed to be reported by one party in order to be counted as a collaborative relationship. This helped to reduce missingness in the network data as one participant can report about a relationship with an individual who did not participate, providing information about their collaborative relationships, even though I was unable to talk to both parties. 144 Analysis Exponential random graph modeling. Exponential Random Graph Modeling (ERGM) is a tool for making inferences with network data (Robins, Pattison, Kalish, & Lusher, 2007). Because the observations in network datasets are not independent, traditional regression approaches are not appropriate. For this project, I employ a specific type of ERGM, a p* model. P* models make it possible to estimate a logit model (i.e., the presence of a network dyadic relationship can be the outcome variable in the model) using network data (Knoke & Yang, 2008). To do this, they employ maximum likelihood estimation and control for network-level interdependencies (Knoke & Yang, 2008). ERGMs have the general form as described in Robins, Pattison, Kalish, & Lusher (2006): Pr((cid:24) = (cid:7)) = (cid:25)1 (cid:27)(cid:28) (cid:29) exp {! "#$#((cid:7)) } % This can be interpreted as the probability of a particular network arrangement being predicted by a set of specified network structures as well as other specified variables representing attributes of individuals either at the individual or structural level. In this case, the outcome variable is collaboration and there are several predictors. These include mental model similarity, length of time in the coalition, sector homophily, gender homophily, and two additional network-level dependencies (described below). The user must determine which specific network-level interdependencies are relevant for the particular type of network they are evaluating. ERGMs allow for the specification of a number of network-level dependencies. I have chosen one particular type to control for: ties. Network ties are generally used as a control variable in analyses using ERGM (Lusher & Robins, 2013). This constrains simulated models to the density 145 of the original model. Results Below I present the results of the ERGM analysis to evaluate the extent to which mental model similarity predicts collaborative relationships. I first present some network descriptives, then describe the model’s output as I added variables into it. Network descriptives. The collaboration network included 23 individuals with 59 undirected collaborative ties. The network has a density of .23 and it has two main components, with one isolate. The correlations among mental models ranged from .0005 to .21, with a mean of .04 and a standard deviation of .04. The network is visualized in Figure 8 with nodes representing each participant and ties representing the collaborative relationships among them. Node colors represent the sectors participants reported working in. Black nodes represent individuals who did not participate in the study. 146 Figure 8. Visual representation of the coalition collaboration network Null model. I first created a null model, which included the network control variable edges4. The null model including only ties had a maximum likelihood estimate of -1.14, SE = .15, z = -7.60, p < .001. Covariate model. Next, I added the control variables in to the model: gender homophily, sector homophily, and length of time participating in the coalition. Estimates, standard errors, and p-values for each parameter are listed in Table 10. Length of time in the coalition was the only significant variable of this group with a maximum likelihood estimate of .51, SE = .10, z = 4.9, p < .001. 4 A triangle term lead to non-convergence for each of the models I tested. This parameter could not be included in the analysis. I ran a similar analysis using geometrically weighted edgewise shared partners in place of triangles, which also did not yield models that converged. 147 Full model. I next added mental model similarity to the covariate model to assess the full model. The estimates, standard errors and p-values for each of these is listed in Table 8. Mental model similarity was not a significant variable in this model, with a maximum likelihood estimate of .79, SE = .79, z = 1.00, p = .32. Length of time in the coalition remained significant in this model. I also ran a sensitivity analysis using a more liberal similarity strategy, which did not change the outcomes of the model. This indicates that the results are not sensitive to my approach to standardizing concepts. These results are reported in Appendix F. Table 10. Model estimates, maximum likelihood estimate (standard error) Parameter Edges Covariate -4.30 (.66)* -4.38 (.66)* Null -1.14 (.15)* Full Gender homophily Sector homophily Length of time in the coalition Mental model similarity .56 (.34) .52 (.35) .34 (.40) .37 (.40) .52 (.10)* .51 (.10)* .79 (.32) Note: * indicates significance at the .001 level Goodness of fit. Goodness of fit statistics for ERGM models can be computed using Monte Carlo Markov Chains to simulate models using the parameters included in the ERGM. To assess how well the proposed model describes the observed network, the Monte Carlo empirical p-values should be larger than 0.05, as small p values indicate significant differences between the observed network and the simulated networks. Additionally, the observed values should fall within the range of values in the simulated networks for each parameter (Luke, 2015). To assess goodness of fit for my proposed 148 model, I first examined fit for each of the parameters included in the model, second, I examined the goodness of fit across several network structures: geodesic distance, edgewise shared partners, degree distribution, and the triad census. The results of the goodness of fit test are summarized in Tables 11, 12, 13, 14, and 15. The goodness of fit statistics for the model parameters suggest good fit. Each observed parameter falls within the range of values that occurred in the simulated networks and included large p values. Minimum geodesic distance refers to the shortest pathway between two nodes. For geodesic distance values between 1 and 5, the simulated networks demonstrated fit with the observed network, indicating that they were able to generate networks with the same number of node pairs with each value for geodesic distance. Edgewise shared partners are those partners shared by members of connected dyads in the network. The simulated networks also demonstrated goodness of fit with the observed network with the exception of those with 10 edgewise shared partnerships. Degree refers to the number of ties a node has. The simulated models fit well with the observed model for all degree values. Triads refer to the types of triangle shapes that can appear in a network. For this type of network, there are four types of possible triads, and the model demonstrated fit with three of them. Overall, the model demonstrates goodness of fit for the all but two of the values for the parameters included in the goodness of fit test. 149 Table 11. Goodness of fit for model statistics Note: significant values in the context of goodness of fit statistics indicates poor model fit. Variable Observed Minimum 59 23 15.00 51 16.00 9 Mean 61.90 23.39 15.28 Maximum 80 36 20 377.33 321.66 394.03 487.33 7.55 4.64 7.63 10.79 P .66 1.00 1.00 .66 .94 P .88 .82 .68 .64 .10 .58 P .32 .06 .76 .06 .62 .14 .88 .16 1.00 1.00 .02 Edges Gender homophily Sector homophily Time in the coalition Mental model similarity Table 12. Goodness of fit for minimum geodesic distance 1 2 3 4 5 Inf Observed Minimum 5960.81 138.59 40.23 3.83 .08 9.46 51 109 18 0 0 0 Mean 60.90 134.26 45.70 3.20 .10 7.84 Observed Minimum Table 13. Goodness of fit for edgewise shared partner Mean 10.47 16.90 14.77 9.84 5.47 2.68 1.20 .40 .11 .05 .01 6.67 8.87 15.40 17.34 3.32 6.21 0 1 2 3 4 5 6 7 8 9 10 3 5 7 2 0 0 0 0 0 0 0 0 2 0 0 1 Maximum 80 157 69 22 4 63 Maximum 20 31 26 18 12 10 7 4 2 1 1 150 Table 14. Goodness of fit for degree 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Observed Minimum Mean Maximum .43 3.46 1.08 1.92 3.68 3.10 3.29 2.04 2 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 .36 1.09 2.16 3.33 3.20 3.10 2.59 1.89 1.58 1.25 .92 .79 .36 .19 .12 .04 .03 3 4 5 7 8 8 8 5 5 4 4 3 2 2 1 1 1 Table 15. Goodness of fit for triad census 0 1 2 3 Observed Minimum 859.76 598.81 259.09 53.34 584 585 172 21 Mean 802.49 678.01 249.61 40.89 Maximum 926 763 355 74 Discussion P .58 .04 .64 .32 .72 .74 .46 1.00 1.00 .50 .76 .78 1.00 1.00 .24 1.00 .06 P .44 .02 .74 .34 The purpose of this chapter was the answer the following question: To what extent does mental model structure and content predict collaboration within a coalition? I hypothesized that greater similarity in mental models would predict collaboration between coalition participants. I applied an exponential random graph model approach to analyzing collaboration data in order to answer this question. The results indicate that my hypothesis is not supported by the data, although the proposed model does have a good fit with the observed network data. Results also suggested that gender and sector 151 homophily were not significant predictors of collaboration, but length of time in the coalition was a significant predictor. Mental model similarity was not a significant predictor of network ties. Literature about how coalition functioning suggests that similarity in problem definitions is key for successful collaboration (Foster-Fishman, Berkowitz, Lounsbury, Jacobson, & Allen, 2001; Kania & Kramer, 2011; Foster-Fishman & Watson, 2011; Lawlor & Neal, 2016; Reininger, Sinicrope, Dinh-Zarr, Sinicrope, & Martin, 1999). This idea is supported by the concept of homophily, suggesting that similar individuals are more likely to have relationships with each other (McPherson, Smith-Lovin, & Cook, 2001). Thus, these findings may indicate a need to consider a more complex approach to thinking about problem definition and collaboration and invites consideration of literature about collaboration outside of that which focuses on coalitions specifically. Studies about cognitive diversity in teams, indicate that more diverse teams are able to make better models of problems than homogenous teams (Hong & Page, 2004; Page, 2007). This is consistent with the findings in my systematic review in chapter two regarding coalitions as a space for knowledge coordination. It suggests that knowledge coordination may occur over the lifespan of the coalition and that individual differences in knowledge or perception of the problem of interest may contribute to collective understanding. Thus, collaboration may be more complex than what the coalitions literature suggests and may require diverse types of connections to generate diverse collaborative groups. Others have pointed to the garbage can model as a way of understanding collective approaches to understanding and solving problems (Cohen, March, & Olsen, 1972). This model postulates that organizations approach opportunities to make 152 decisions by placing problems and solutions into ‘garbage cans,’ which can hold disparate problems and solutions and organizational culture can create settings in which these garbage cans are more or less problematic. For example, in organizations with high turnover, individuals may not work on a single problem or its solution for very long and may leave little institutional knowledge of the problems they worked on. Though this model focuses specifically on decision-making opportunities, coalitions can consider this when they work on establishing common problem definitions overall and can use it to approach collective definitions of problems with healthy skepticism. It can facilitate consideration of the extent to which the mental models of coalition members are sufficient for identifying problem definitions and what additional information may be necessary to determine the actions they plan to take. Participants also reported only on who they currently have collaborative ties with and not the quality of those ties. It is possible that some of them may not be fruitful collaborations and that there may be more complexity to the nature of the relationships that could not be captured within the scope of this project. This could be improved upon in the future by asking participants questions about issues like how satisfied they are with their collaborative relationships and how much they feel that those relationships help them to achieve shared goals. My approach to measuring problem definition similarity directly may also explain why it did not emerge as a significant predictor in the ERGM. Coalition stakeholders may not have direct knowledge of the way their colleagues define the problems they work on, but may have perceptions of the way they define them. In this study, I compared each participant’s problem definition against the other members of their coalition action team, rather than considering the extent to which stakeholders 153 consider themselves to be similar or different from their colleagues in the coalition. Future research may ask participants to comment directly on their perceptions of others with a question like “Do you believe [they] think about this problem in the same way you do?” This may also provide further information about why stakeholders build relationships with each other in coalitions. The exponential random graph model did identify length of time in the coalition as a significant predictor of relationships between participants. This indicates that, as expected, participants who have been in the coalition for a longer period of time are more likely to have relationships with others in the coalition. Given that these participants have had more chances to interact with others in the coalition, it makes sense that they would have more connectivity in the network. Surprisingly, the sector homophily and gender homophily were not significant predictors of relationships among participants. This counteracts the relationships we would expect based on literature regarding homophily and community change efforts (Watson & Foster-Fishman, 2011; Kania & Kramer, 2011; Lawlor & Neal, 2016; McPherson, Smith-Lovin, & Cook, 2001) and it invites future investigation of the types of features among participants that are important for collaboration as well as questions about the ways in which I measured homophily, particularly with regard to sector. In conducting mental model interviews with participants, they often discussed which aspect of postsecondary attainment they were involved with specifically. This included things like trade school, community college, or four year universities. These conceptualizations of their engagement with the problem of interest were different from the sectors I asked about in the survey and may be important for understanding why participants even within a sector would not have collaborative ties. For example, 154 counselors working in K-12 education that support students in applying to trade school for college may not collaborate directly with counselors supporting students in applying to four year universities. For these individuals, it may be more likely that they would connect with recruiters from trade schools or four year universities in the postsecondary education sector than with each other. In sum, the ERGM analysis indicates that stakeholder networks emerge over time as stakeholders have an opportunity to interact with one another. It may suggest that collaborations simply take time to form and that similarity in problem definition, sector, or gender do not increase it. This may suggest that coalitions can focus on engagement among participants and facilitate opportunities for new members to become acquainted with those who have been engaged in the coalition over time in order to encourage collaboration among stakeholders, rather than placing effort on generating shared problem definitions among stakeholders as a vehicle for increasing collaboration. Limitations This research should be interpreted in light of several limitations. The data collected here is cross-sectional and represented only one coalition. While studying single network cases are common in studies employing social network analysis (For example: Bess, 2015; Evans, Rosen, Kesten, & Moore, 2014; Luque et al., 2013), collecting data from many coalitions may uncover patterns in problem definitions and collaboration that did not appear here. Similarly, collecting data at a single time point only provides a single snapshot of problem definition and collaborative relationships, which are constantly changing in these types of networks. My data collection instruments also have limitations. Asking participants to self- report about relationships in a network requires participants to recall their relationships 155 clearly. I did not require the network ties to be confirmed in order to address this issue, but it is possible that some ties were left out because participants simply did not recall the relationship during the time of data collection. Relatedly, the study had a 78% response rate, which is appropriate for a network analysis, but does limit the findings of the study as some relationships among members of the coalition were not accounted for in the analysis. Because I did not require collaboration ties to be confirmed, I was able to include some data about collaborative ties among coalition members and the five members of the action team who did not participate in the study. I do not have any data about the relationships among the five people who did not participate, this means that there was completely missing data for 10 ties out of 253 potential ties. Missingness is more complex for comparing model similarity values. Because model similarity is a dyad level variable that requires data from both members of the dyad, I am missing similarity scores for each dyad in which a non-participant could have engaged (totaling 90 dyads with missing data). Being able to include data for the full population of the coalition action team may have illuminated some additional patterns not observed here. In the analysis, this missing data also had to be addressed through the use of imputation for mental model similarity and for length of time in the coalition in order for the model to run. For these, I chose to impute the average value for each, following advice in Koskinen & Snijders (2013). These provided an estimate for the missing participants, but is ultimately less accurate than the getting direct data from the individuals who did not participate. Future directions These findings and limitations give rise to several future directions for research in this area. In particular, future analyses can explore the evolution of collaborative 156 networks over time as well as the changes in problem definitions over time to get a better sense for how these change in relation to each other. Social learning theory would suggest that as participants in a coalition interact with each other, their problem definitions would converge over time (Prell, 2012). This could provide a basis for a follow-up study within coalitions. In addition, research with a large number of coalitions could provide a better sense for whether the patterns I found here are universal or if unique coalition contexts facilitate the relationship between problem definition and collaboration. Methodologically, future research might apply an approach that uses archival data or observational methods in order to avoid issues of missingness. This might include assessing network relationships based on coalition archives of who co-attended activities, or co-participated in leading groups, for example. A cognitive social structure approach could be employed to capture full network data, but this could get burdensome for participants in larger networks (Neal, 2008). Finally, future research can explore how differences in participant problem definitions may be useful or create contributions to coalition functioning. Considering coalitions as a space for knowledge coordination, identifying different understandings of a problem and applying tools like participatory modeling can help to uncover the assumptions that individuals have about how the problem functions and generate more collective knowledge about the issue of interest. Researchers can further explore how the application of these tools leads to participant interaction and the formation of collaborative relationships. Future research can also explore other approaches to capturing and comparing problem definitions among members of coalitions. Researchers can compare a fully qualitative, interview-based approach against a mental modeling approach or another 157 simulation modeling approach to identify the different ways these approaches can illuminate problem definitions. Researchers can also dig deeper into comparisons among mental models as they were identified here. Pearson correlations between mental models are one way to compare them, but there may be other options that provide additional insights into the degree of similarity between two models. For example, machine learning is a growing area of data science that can identify patterns that may be missed by some traditional statistical approaches to data analysis. Specifically, the unsupervised learning approach to machine learning can cluster similar data together, which may be useful for identifying similar mental models using computing rather than statistics. The algorithms used in machine learning can engage data in more complex ways, making them powerful for making comparisons among mental models, but also require more consideration to find an appropriate one and to evaluate its successful implementation. While it was outside the scope of this study to employ this type of approach, researchers should consider the role machine learning tools can play in assessing the degree of similarity among mental models. For example, K means clustering can be explored as a way to group similar mental models in order to use those groups as predictors in inferential models, like ERGM (Shi, Xumin, Yong, 2010; Aminpour, 2018). Conclusions My hypothesis that mental model similarity would predict collaborative ties was not supported. This suggests a need for a deeper dive into theories about coalition functioning to explore the role of problem definition and the extent to which agreement on problem definition contributes to coalition success. There are a number of limitations to this study, including the use of a single case with a few missing 158 observations and a limited analysis approach. Future research should further explore this phenomenon to improve the method for studying this problem, the way networks and problem definitions change over time, and the presence of patterns in problem definitions and collaborative relationships across many coalitions. 159 REFERENCES 160 REFERENCES Aminpour, P. (2018). PyFCM. Retrieved from: https://github.com/payamaminpour/PyFCM Borgatti, S.P., Everett, M.G. & Johnson, J.C. (2013). Analyzing social networks. London: Sage Publications. Borgatti, S. P., & Everett, M. G. (2000). Models of core/periphery structures. Social networks, 21(4), 375-395. Fortunato, S. (2010). Community detection in graphs. Physics reports, 486(3-5), 75-174. Foster-Fishman, P. G., Berkowitz, S. L., Lounsbury, D. W., Jacobson, S., & Allen, N. A. (2001). Building Collaborative Capacity in Community Coalitions: A Review and Integrative Framework. American Journal of Community Psychology, 29(2), 241– 261. http://doi.org/10.1023/A:1010378613583 Foster-Fishman, P. G., & Watson, E. R. (2012). The ABLe change framework: A conceptual and methodological tool for promoting systems change. American Journal of Community Psychology, 49(3-4), 503-516. Handcock, M.S. & Gile, K.J. (2010). Modeling social networks from sampled data. Annals of Applied Statistics, 4(2), 5-25. Kania, B. J., & Kramer, M. (2011). Collective Impact. Stanford Social Innovation Review, 1(9), 36–41. Knoke, D. & Yang, S. (2008). Social network analysis. Thousand Oaks, CA: Sage Publications. Koskinen, J. & Snijders, T. (2013). Simulation, estimation, and goodness of fit. In D. Lusher, J. Koskinen, & G. Robins (Eds.), Exponential Random Graph Models for Social Networks: Theory, Methods, and Applications (141-166). New York, NY: Cambridge University Press. Lawlor, J. A., & Neal, Z. P. (2016). Networked Community Change: Understanding Community Systems Change through the Lens of Social Network Analysis. American Journal of Community Psychology, 426–436. http://doi.org/10.1002/ajcp.12052 Leydesdorff, L. (2008). On the normalization and visualization of author co‐citation data: Salton's Cosine versus the Jaccard index. Journal of the Association for Information Science and Technology, 59(1), 77-85. 161 Luke, D.A. (2015). Statistical network models. A User’s Guide to Network Analysis in R (pp. 163-187). Cham, Switzerland: Springer. Luke, D. A., Harris, J. K., Shelton, S., Carothers, B. J., Mueller, N. B., & Allen, P. (2010). Systems analysis of collaboration in 5 national tobacco control networks. American Journal of Public Health, 100(7), 1290–1297. http://doi.org/10.2105/AJPH.2009.184358 Lusher, D. Robins, G. (2013). Formation of social network structure. In D. Lusher, J. Koskinen, & G. Robins (Eds.), Exponential Random Graph Models for Social Networks: Theory, Methods, and Applications (17-28). New York, NY: Cambridge University Press. Luque, J., Martinez Tyson, D., Lee, J. H., Gwede, C., Vadaparampil, S., Noel‐Thomas, S., & Meade, C. (2010). Using social network analysis to evaluate community capacity building of a regional community cancer network. Journal of Community Psychology, 38(5), 656-668. Mcpherson, M., Smith-lovin, L., & Cook, J. M. (2001). Birds of a feather: Homophily in social networks. Annual Review of Sociology, 27(2001), 415–444. Neal, J. W. (2008). “Kracking” the missing data problem: Applying Krackhardt’s cognitive social structures to school-based social networks. Sociology of Education, 81(2), 140–162. Retrieved from https://search-proquest- com.proxy1.cl.msu.edu/docview/216488262?accountid=12598 Nobi, A., Lee, S., Kim, D. H., & Lee, J. W. (2014). Correlation and network topologies in global and local stock indices. Physics Letters A, 378(34), 2482-2489. Prell, C. (2012). Social network analysis: History, theory, & methodology. London: Sage Publications. Reininger, B., Sinicrope, Dinh-Zarr, T., Sinicrope, P.S. & Martin, D.W. (1999). Dimensions of participation and leadership: Implications for community-based health promotion for youth. Family & Community Health, 22(2), 72-82. Robins, G., Pattison, P., Kalish, Y., & Lusher, D. (2007). An introduction to exponential random graph (p*) models for social networks. Social networks, 29(2), 173-191. Robins, G.L., Pattison, P., & Woolcock, J. (2004). Missing data in networks: Exponential random graph (p*) models for networks with non-respondents. Social Networks, 26, 257-283. Shi, N., Liu, X., & Guan, Y. (2010). Research on k-means clustering algorithm: An improved k-means clustering algorithm. 3rd International Symposium on 162 Intelligent Information Technology and Security Informatics, IITSI 2010, 63–67. http://doi.org/10.1109/IITSI.2010.74 Szell, M., Lambiotte, R., & Thurner, S. (2010). Multirelational organization of large- scale social networks in an online world. Proceedings of the National Academy of Sciences, 107(31), 13636-13641. Wang, S. (1996). A Dynamic Perspective of Differences Between Cognitive Maps. Journal of the Operational Research Society, 47(4), 538–549. 163 My dissertation evaluated two objectives. My first objective focused on the variety CHAPTER 6 of mental models within a particular problem area in a coalition and the extent to which they converge or diverge. My second objective focused on the extent to which model structure and content relate to collaboration patterns within coalitions. Each of the chapters in my dissertation speak to these objectives in unique ways. In chapter two, I conducted a systematic review to establish a definition of coalitions within the field of community psychology and reviewed the current state of research about coalitions. This created a common language and frame of reference for appropriate literature for this project. In this chapter I also identified three types of coordination coalitions engage in: (1) knowledge coordination, (2) negotiated coordination, and (3) action coordination. These types of coordination and the associated problems, techniques, and outcomes involved in each establish new directions for inquiry into coalition functioning. In chapter three, I reviewed the relationship between problem definition in coalitions and collaborative relationships and established mental models as a way of understanding individual problem definitions. This informed the data collection process and analyses presented in chapters four and five. This approach may be useful for future research to understand problem definitions in this context. As this project developed and I learned that some participants’ focus was directed toward particular types of postsecondary education. Others using this approach may find it useful to consider ways to change their modeling process as information becomes available about the problem of interest and how 164 In chapter four, I addressed my first research objective by describing the mental models I collected and evaluating similarities and differences among them across all members of the coalition action team and across sectoral groups in the action team. In this chapter, I noted that members of the action team varied greatly in their thinking about the problem of interest, with a few areas of overlap. When exploring these patterns by sector, I noted that sectors tended to included concepts that related specifically to the type of work that emerges within those sectors, but there were some commonalities across them. This suggests future directions for considering how individual problem definitions may fit together to understand a problem through the lens of the wisdom of the crowd. This can be important for identifying how each individual’s understanding of the problem contributes to a collective model of it. A collective model can then be tested against data the group collects to about the problem to determine how well it fits and to refine their understanding in order to identify inaccurate assumptions and directions for future intervention. In chapter five, I addressed my second research objective, evaluating the relationship between similarity in problem definitions and dyadic collaboration. The findings from the ERGM analysis suggested that similarity in problem definitions among participants were not significant predictors of collaboration among participants. These findings operate contrary to the literature reviewed in chapter 3 and raise questions about whether collaborative relationships require shared understandings of problems. These findings collectively have implications for empirical research on coalitions, methods for researching coalitions, and for coalition stakeholders and the organizations that support them. In chapter two, my systematic review of coalitions defined them in 165 terms of three types of coordination. This created a space for future empirical work to determine whether coalitions operate using the three types of coordination and what strategies they can use to maximize them. This review also identified shared problem definitions as a part of knowledge coordination, which I further elaborated in my literature review in chapter three, tying collaboration patterns to shared problem definitions. In the context of my findings from chapter four and chapter five about the diversity of problem definitions and their insignificant role in collaboration, this suggests a need for further inquiry. Specifically, researchers can identify whether such a diversity of problem definitions is common across coalitions or whether it is unique to this one. Additionally, this presents an opportunity to consider whether more diversity in stakeholders’ problem definitions predicts how successful the coalition is in achieving goals or engaging in action and negotiated coordination as suggested in the systematic review in chapter two. Researchers can also capture problem definitions over time in order to observe how they change and in what ways those changes relate to coalition activities. Relatedly, the purpose of this project focused on the similarities and differences in problem definitions, but not about empirically observing the problem of interest, postsecondary attainment. This means that this project demonstrates the patterns in how participants think about the problem and the extent to which they converge in their thinking, but it does not tie these problem definitions to direct observations of the data. However, the level of divergence between participants does suggest that future research may benefit from considering how individual problem definitions inform a more complete understanding of the problem and move the coalition towards better identifying areas for intervention. As I have discussed throughout, research regarding 166 the wisdom of crowds, in particular research about cognitive diversity in collaborative settings, may help to create group models that can recreate observed data about problems more effectively than individual problems and can serve as tools to guide decision-making and action. An extension of this project would be to identify how an aggregate model of the problem definitions fits with the data the group collects about their efforts to change postsecondary attainment in their county. Coalition researchers should also consider implications for the methods they use to research coalitions. This project applied a new approach to capturing problem definitions among coalition stakeholders using fuzzy cognitive maps. These proved useful for capturing multiple ways of understanding problem definitions, in terms of their content, structure and function. By employing longitudinal designs, researchers can get a better sense for how persistent mental models of problem definitions are over time. Using this can better guide researchers in understanding the malleability of problem definitions and how useful they can be for understanding perspectives on a problem within a particular time scale. Researchers can also explore how mental modeling compares to other approaches to collecting problem definitions from coalition stakeholders in order to further understand the benefits and drawbacks of various ways of capturing problem definitions. Coalition stakeholders and the organizations that support them can also consider these findings in their own work. The three types of coordination identified in chapter two may serve as areas for evaluation to consider how they function in each of these areas and to engage in scoping to determine which of them are a focus of their work. Coalitions can also consider identifying problem definitions among stakeholders directly as a starting point for addressing problems. This can facilitate identification of shared 167 understanding as well as assumptions and differences. Stakeholders can use this information to assess their current actions relative to the way stakeholders understand the problem working in order to determine places where they may need to collect data or generate group models to identify the best places to place their efforts. Coalitions can also use the findings from chapter five to consider the ways in which stakeholders work together. As these findings suggest that length of time in the coalition is significant for collaboration among stakeholders. Coalition leadership (e.g., backbone organizations), can facilitate opportunities for interaction and consider ways to engage new members that help them to become integrated in order to build fruitful collaborations with other stakeholders. The findings from chapter five also suggest that having a shared problem definition may not be critical for generating collaborative relationships among stakeholders. Coalition leaders may consider placing efforts towards stakeholder engagement that do not emphasize the generation of shared problem definitions. These findings should be interpreted in light of some limitations that spanned the project as a whole. These include the single case nature of a network analysis study, the response rate, and the cross-sectional nature of the study. Using a single case to study this problem limits the generalizability of findings. This study focuses in on a single action team within a coalition. While this coalition structure and its approach are common throughout the state of Michigan and follow a model that has gained popularity nationally, generalizability across all coalitions may be limited and future research should replicate this study with other types of coalitions to better understand how these problem definitions operate in these types of groups. The response rate for the study was high at 78%, but it did limit my ability to assess collaboration as network analysis is sensitive to missing data. Finally, the cross-sectional nature of the study 168 limits my knowledge of how the mental models formed, the ways in which they are changing, and the ways in which collaborative relationships formed and are changing. This means that there was no context for who had collaborated in the past and how participants’ mental models may have changed over time as they have engaged with other members of the coalition or other aspects of working on increasing postsecondary attainment. Future research can address these limitations and extend the work I have started here by extending the sample to multiple coalitions addressing other types of problems and exploring these questions over time. Conducting research with multiple coalitions will increase generalizability and can make it possible to identify additional contextual factors that may be relevant to understanding how problem definitions and collaboration operate in coalition contexts. Exploring these questions over time will demonstrate how problem definitions change over time, the factors that influence their change, and the interactions between collaboration and problem definitions. 169