DESIGN, COMMUNITY, CHANGE: EVALUATING APPROACHES TO INNOVATIVE INTERDISCIPLINARY PROBLEM SOLVING By Kathryn McAlindon A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Psychology – Doctor of Philosophy 2019 ABSTRACT DESIGN, COMMUNITY, CHANGE: EVALUATING APPROACHES TO INNOVATIVE INTERDISCIPLINARY PROBLEM SOLVING By Kathryn McAlindon The purpose of this study was to evaluate the preconditions and products of interdisciplinary collaboration between community-based organizations and graphic designers. The study was conducted within the context of design charrettes, or rapid collaborative design events, to facilitate interdisciplinary collaboration between graphic designers and community organizations. Two research questions were addressed: First, to what extent do graphic designers’ and community organizations’ disciplinary values, objectives, and methodological approaches to community problem solving align? Second, is higher alignment between graphic designers’ and community organizations’ respective problem solving approaches associated with more effective collaboration, or more specifically, the creation of more integrated products? These questions were explored via a qualitative multiple-case study and comparative case analysis of four community design charrettes in Michigan. Across the four charrettes, alignment between the designers’ and community organizations’ problem solving approaches ranged from low to high, with two cases of moderate alignment. The charrettes with higher alignment created more integrated, collaborative products than those with lower alignment. The results suggest that the problem solving approaches of these two respective disciplines may not be constants that are easily defined; thus, alignment can vary. However, themes did emerge suggesting that certain aspects of these approaches (e.g., processes for defining the problem; experience working with or knowledge of the other discipline) may be leveraged given their association with the production of highly collaborative, interdisciplinary products. This speaks to a need for more training and practice in interdisciplinary approaches, like design for community change, as they appear to be more effective than later attempting to facilitate collaboration between two distinct disciplines. These findings contribute to a better understanding of how strategies like design charrettes can be used to integrate design and community change, and the implications of design-community collaboration and interdisciplinary practice. ACKNOWLEDGEMENTS To my dissertation chair, Dr. Jennifer Watling Neal, I cannot begin to express how grateful I am for your support and mentorship. Thank you for embracing my ideas, having the patience to help me articulate them, and giving me the skills to make them a reality. A sincere thank you to my committee, Drs. Rebecca Campbell, Zachary Neal, and Miles McNall. Your insight and openness have been invaluable to this project and my growth as a researcher. To those involved with AIGA Design for Good and the IMPACT! Design for Social Change program, your work is endlessly inspiring and important. Thank you for allowing me to be a part of it. This is for you, dad. iv TABLE OF CONTENTS LIST OF TABLES LIST OF FIGURES INTRODUCTION CHAPTER 1: LITERATURE REVIEW Interdisciplinary Collaboration in Community Problem Solving The Integration of Design Into Interdisciplinary Community Problem Solving Facilitating Interdisciplinary Community Problem Solving: Community Design Charrettes Design-Community Partnerships: Different Approaches to Community Problem Solving Effective Interdisciplinary Collaboration: Integrated Products of Design-Community Partnerships Aligned Approaches as Predictors of Effective Interdisciplinary Collaboration Values Objectives Community organization values Designer values Community organization objectives Designer objectives Methodological approach Community organization method Designer method Community organization contributions Designer contributions Aligned values Aligned objectives Aligned methodological approaches Current Study CHAPTER 2: METHOD Setting Sample Cases Research Design Measures Data Collection Pilot testing Study procedures Interviews Semi-structured open-ended interviews Researcher observations Archival documents v viii ix 1 6 6 9 16 20 21 21 22 23 23 24 25 25 26 28 29 30 30 33 35 38 41 44 45 48 50 51 53 56 58 59 59 62 62 Researcher observations Document retrieval CHAPTER 3: ANALYSIS Data Preparation Analytic Approach Within-case analysis Cross-case analysis Validity and Trustworthiness of Data and Analysis CHAPTER 4: RESULTS Research Question 1 Charrette 1 problem solving approach alignment: low Charrette 2 problem solving approach alignment: low-moderate Charrette 3 problem solving approach alignment: moderate-high C1 value alignment: moderate C1 objective alignment: low C1 methods alignment: low C2 value alignment: moderate C2 objective alignment: low C2 methods alignment: moderate C3 value alignment: moderate C3 objective alignment: moderate C3 methods alignment: high Charrette 4 problem solving approach alignment: high C4 value alignment: high C4 objective alignment: high C4 methods alignment: high Research Question 2 Charrette 1 integrated product: low C1 organization contribution: low C1 designer contribution: high Charrette 2 integrated product: moderate C2 organization contribution: moderate C2 designer contribution: high Charrette 3 integrated product: moderate-high C3 organization contribution: moderate C3 designer contribution: high Charrette 4 integrated product: high C4 organization contribution: high C4 designer contribution: high Cross-Case Summary CHAPTER 5: DISCUSSION Limitations Further Research on Collaborative Context to Benefit Current Practice vi 64 66 67 67 67 68 69 70 72 72 72 72 74 75 76 76 78 79 81 82 83 85 86 86 88 89 89 90 90 92 93 94 94 96 97 98 99 99 101 102 104 106 108 111 113 114 116 117 118 138 160 162 163 169 171 176 Understanding Outcomes to Promote Design-Community Collaboration Practical Implications Conclusion APPENDICES Appendix A IRB Determination Letter Appendix B Data Collection Materials Appendix C Pilot Testing Materials Appendix D Interview Recruitment Emails Appendix E Informed Consent Appendix F Analysis Codebook Appendix G Example Data Condensation, Visualization, and Analysis Tools Appendix H Comparative Case Analysis Procedure REFERENCES vii LIST OF TABLES Table 1. Charrette Participants and Community Issues Table 2. Analysis Codebook 50 163 viii LIST OF FIGURES Figure 1. Alternative Approaches to Collaborative Work (Obrien et al., 2003) Figure 2. Community Design Charrette Study Model Figure 3. Data Collection Timeline Figure 4. Charrette 1 Case Summary Figure 5. Charrette 2 Case Summary Figure 6. Charrette 3 Case Summary Figure 7. Charrette 4 Case Summary Figure 8. Cross-Case Summary Figure 9. IRB Determination Letter Figure 10. Interview Protocol – Designers Figure 11. Interview Protocol – Organizations Figure 12. Researcher Observation Protocol Figure 13. Pilot Interview Recruitment Email Figure 14. Pilot Interview Protocol - Designers Figure 15. Pilot Interview Protocol – Organizations Figure 16. Pilot Observation Protocol Figure 17. Designer Interview Recruitment Email Figure 18. Organization Interview Recruitment Email Figure 19. Informed Consent Figure 20. Example Coding Spreadsheet Figure 21. Example Themeing Spreadsheet ix 40 42 59 93 96 99 102 103 117 118 127 136 138 139 148 156 160 161 162 169 169 Figure 22. Example Cross-Case Comparison Matrix 170 x INTRODUCTION The appeal of an interdisciplinary approach stems from the idea that each contributing discipline offers “unique knowledge, methodological approaches, conceptual frameworks, and theories” to new innovations or information that could not be created by one discipline alone (Hall et al., 2012, p. 416). In both research and practice, there is a growing acknowledgement that the world’s most complex problems (e.g., global warming, animal extinction, public health inequities) require multi-dimensional solutions that can best be addressed via interdisciplinary collaboration (Gibson & Owens, 2014; Jacobs & Frickel, 2009; Stokols et al., 2005; Stokols et al., 2008). Localized community-based problems, such as high unemployment in a certain neighborhood or minimal access to fresh food in an area, are similarly complex. Therefore, efforts to address these complex community problems could also benefit from interdisciplinary collaboration, which can simultaneously address diverse cultural perspectives while attending to critical factors like physical environments, social dynamics, policy constraints, and organizational contexts (Maton et al., 2006). One way to incorporate interdisciplinary collaboration into community problem solving is through partnerships between community organizations and designers. Community change agents, particularly staff in human service organizations or nonprofits, are often plagued by complex problems and have minimal resources to solve them. Although these staff members have critical experiences, skills, and problem solving perspectives within their own system, they may ultimately be boxed in by demanding workloads, limited experience with systems-based problem solving, or inadequate resources (Sutton & Kemp, 2006). On the other hand, the problems commonly addressed by professional designers are ill-defined and complex by nature 1 (e.g., efficiently communicating new ideas to entire populations or optimizing the use of functional space in a building plan; Cross, 2004; Stempfle & Badke-Schaub, 2002). There is an important intersection between designers and community change agents that Crane (2011) explains elegantly when she describes their distinct individual ecosystems, but highlights their fundamentally shared identity of “world-builders” who “are a part of the same larger ecosystem and interact, overlap, and ultimately strengthen each other’s agendas and outcomes” (p. 2). With the understanding that the expertise of designers is dedicated to solving a vast array of complex problems, and that their work is fundamentally entrenched and instrumental in communities of all kinds, there is a growing realization that design and community change are critically interrelated disciplines. Thus, there is promising potential in expanding the study and practice of interdisciplinary collaboration between designers and community organizations. To date, the design discipline has not been able to fully embed itself in the planning and practice of community organizations. This is due not only to limited time and resources in these settings, and limited research to legitimize the role of designers, but also to the difficult process of systematically integrating two very distinct disciplines (Gibson & Owens, 2014; Margolin & Margolin, 2002). In particular, best practices for collaborative efforts that integrate the work of graphic designers and community organizations remain largely elusive. Graphic design is “the activity that organizes visual communication in society”, with concern for “the efficacy of communication, the technology used for its implementation, and the social impact it effects” (Frascara, 1988, p. 20). Graphic designers are positioned to guide the design (e.g., color, typography, content organization), communication (e.g., message delivery, brand reinforcement), implementation (e.g., selecting mediums like print or the web), and desired outcomes (e.g., increasing awareness, shifting attitudes or behaviors) of information 2 (Frascara, 1988). This type of design is commissioned by clients to organize and deliver information in order to influence either the perceptions or the behaviors of targeted users (Cornish et al., 2015; Frascara, 1988). Graphic designers work in many areas including film, advertising, wayfinding, print, and web media, with a primary focus on effective visual communication (Cornish et al., 2015). Visual communication in graphic design is considered effective to the extent that it influences the user (Frascara, 1988). This may include getting individuals to buy a new product, directing them to the right place in an airport, informing them of an upcoming local event, or changing their opinion on a social cause. These outcomes, translated into the human service sector, mean graphic designers can help community organizations be more effective at getting individuals to participate in programs, directing them to service centers or place-making spaces, notifying them of events or opportunities, or informing their views on community issues. Graphic designers have a distinct set of skills that can help community organizations effectively communicate their identity and purpose or promote the dissemination and use of programs and services. Even so, defining an interdisciplinary process for problem solving that values both graphic designers’ and community organizations’ disciplinary approaches and that promotes their equitable integration has been a notable challenge (Cornish et al., 2015). Design charrettes are rapid work sessions in which partners, like community organizations and designers, are brought together to collaboratively and quickly ideate a targeted solution to a specific problem (Sanoff, 2000; Sutton & Kemp, 2006; Sutton & Kemp, 2002;). Although design charrettes are primarily implemented and studied in other areas of design like architecture or land planning, they represent one possible approach to systematically fostering the integration of community services and graphic design. 3 The current study was conducted within the context of design charrettes to facilitate interdisciplinary collaboration between graphic designers and community organizations. I begin by first reviewing the relevant literature outlining interdisciplinary collaboration in community problem solving, the integration of the design discipline into this process, and the facilitation of collaboration via community design charrettes. Next, I turn to literature on the preconditions (e.g. an aligned approach to problem solving including shared or complementary objectives, values, and methods) that contribute to effective collaboration (i.e., more integrated community design charrette products). This literature was used to inform this study’s two research questions. First, to what extent do graphic designers’ and community organizations’ disciplinary objectives, values, and methodological approaches to community problem solving align? Second, as is supported in various interdisciplinary and collaborative literatures, is higher alignment associated with more effective collaboration (i.e., the creation of integrated products)? These questions were explored via a qualitative multiple-case study and comparative case analysis of four community design charrettes, which partnered community organizations with graphic designers, in Michigan. Across the four charrettes, alignment between the designers’ and community organizations’ problem solving approaches ranged from low to high, with two cases of moderate alignment. The charrettes with higher alignment created more integrated, collaborative products than those with lower alignment. The results suggest that the problem solving approaches of these two respective disciplines may not be constants that are easily defined; thus, alignment can vary. However, themes did emerge suggesting that certain aspects of these approaches (e.g., processes for defining the problem; experience working with or knowledge of the other discipline) may be leveraged given their association with the production of highly collaborative, interdisciplinary products. This speaks to a need for more training and practice in 4 interdisciplinary approaches, like design for community change, as they appear to be more effective than later attempting to facilitate collaboration between two distinct disciplines. These findings can be used to better understand and optimize the use of charrettes in community-based graphic design, and to explore implications for the institutionalized expansion of design- community collaboration. 5 CHAPTER 1: LITERATURE REVIEW Interdisciplinary Collaboration in Community Problem Solving Interdisciplinary collaboration is briefly defined as “an effective interpersonal process that facilitates the achievement of goals that cannot be reached when individual professionals act on their own” (Bronstein, 2003, p. 299). Following this definition, ‘interdisciplinary’ is often used as a generic umbrella term in collaborative literature to represent two or more disciplinary perspectives working jointly on a common problem to produce a product that integrates each one (e.g., new knowledge, plans, programs, tools, theories, or disciplines) (Hall et al., 2012; Maton et al., 2006). Some researchers distinguish between ‘multidisciplinary’ (i.e., contributors operate strictly within their own methodological or theoretical frameworks to solve a common problem), ‘transdisciplinary’ (i.e., contributors combine theoretical and methodological aspects of their disciplines to create new shared frameworks to solve a common problem), and ‘interdisciplinary’ (i.e., contributors combine theoretical and methodological aspects of their disciplines but do not seek to generate new shared frameworks to solve a common problem) collaboration (Hall et al., 2012; Jacobs & Frickel, 2009; Masse et al., 2008; Maton et al., 2006; Stokols et al., 2008). However, as is common in other work, ‘interdisciplinary’, which falls in the middle of this spectrum, will be used in this study as a general term to describe all three of these types of collaboration (Maton et al., 2006). Problem solving via an interdisciplinary approach is guided by a systems perspective - that multiple causes to complex problems are interrelated and thus require equally complex solutions. The concept of a wicked problem is described in the context of multidisciplinary design as “breakdowns in the social and natural fabric that resist clear definition, [and] for which 6 there is no ultimate ‘good solution’” (Gibson & Owens, 2014, p. 387). Yet, in the search to uncover “good solutions” to such complex and multi-dimensional problems, there is a growing consensus that single independent “disciplines represent disconnected silos that inhibit innovation and stifle inquiry on topics outside of the narrow conflicts of each discipline” (Jacobs & Frickel, 2009, p. 48). As explained by Stokols et al. (2008), the commitment of various disciplines to addressing these problems collaboratively “stems from the inherent complexity of contemporary public health, environmental, political, and policy changes…and the realization that an integration of multiple disciplinary perspectives is required to better understand and ameliorate these problems” (p. 96). From this realization, an initial push for interdisciplinary research and practice quickly burgeoned into a demand from major funding entities, research initiatives, NGOs, local and national agencies, and the world’s largest private foundations (Campbell, 2005; Masse et al., 2008; Russos & Fawcett, 2000; Stokols et al., 2005). Moreover, the idea of interdisciplinary problem solving in community settings is not new, as is evidenced by the widespread study and implementation of cross-sector social work and social services (Bronstein, 2003; Butt et al., 2008), community coalitions (Butterfoss et al., 1993; Luque et al., 2010; Roussos & Fawcett, 2000; Zakocs & Edwards, 2006), participatory research (Andrews et al., 2010; Cornwall & Jewkes, 1995), organizational collaboration (Israel et al., 1998; Spath et al., 2008), and community engagement in problem solving (Lasker & Weiss, 2003; Strier, 2011; Suarez-Balcazar et al., 2006). Specifically, the study and practice of interdisciplinary collaboration is highly relevant in the field of community psychology given the values and skills of community researchers, and the benefits of interdisciplinary collaboration for community settings (Tebes et al., 2014). Ecological inquiry is a core tenet of community psychology (Maton et al., 2006). Addressing community 7 problems that are often embedded in complicated, interconnected systems requires an approach similar to that of interdisciplinary efforts and one that community psychologists have long practiced and refined (Bennett, 1965). Community psychologists are situated in environments in which they aim to understand diverse contexts and cultures, promote collaboration and community engagement, and champion “theoretical and methodological pluralism” in the solving of community problems (Tebes et al., 2014, p. 482; Maton et al., 2006). Community psychologists are well suited for interdisciplinary study given competencies including knowledge of group processes, organizational development, coalition development, and program implementation (Tebes et al, 2014). Reciprocally, community psychology can contribute knowledge and skills to the study and support of interdisciplinary science while also making significant advancements in the field’s own understanding of group processes and diverse integrative solutions (Maton et al., 2006). Further, the contexts and problem spaces within which community psychologists seek to solve problems stand to benefit tremendously from interdisciplinary collaboration. Interdisciplinary collaboration has been associated with a more accurate analysis of social problems (Suarez-Balcazar et al., 2006), a higher likelihood of implementation of solutions (Sutton & Kemp, 2006), greater sustainability (Valencia-Sandoval et al., 2009; Zhang et al., 2015), and increased potential for innovation (Maton et al., 2006; Nie, 2016; Stokols et al., 2005). In a discussion of community interdisciplinary collaboration specifically, Maton et al. (2006) explain that “the more complex and multifaceted the problems and settings being addressed, the more likely involvement in an interdisciplinary effort will yield a sufficiently complex, sophisticated and useful intervention effort” (p. 11). Examples of this can be drawn from the vast literature describing the success and utility of community coalitions or 8 participatory research, both efforts of interdisciplinary collaboration, in holistically addressing multi-dimensional community problems like child welfare (Horwath & Morrison, 2007; Lewandowski & GlenMaye, 2002; Spath et al., 2008), community health (Alexander et al., 2000; Butt et al., 2008; Butterfoss et al., 1993; Israel et al., 1998), domestic violence (Allen, 2005; Nowell, 2009), research-community partnerships (Andrews et al., 2010; Strier, 2011), and equitable food systems (Suarez-Balcazar et al., 2006). As is the case in these examples, interdisciplinary collaboration is widely championed for its potential to generate holistic solutions for closely related fields with common problems like health, social work, sociology, and urban planning. A lesser known strength of interdisciplinary collaboration is that this potential also exists when bringing together more distant disciplines. In fact, some scholars argue that seemingly unexpected partnerships may be the key to creativity and innovation (Baer, 2010; Nie, 2014). Justesen (2004) refers to the crucial diversity of skills and knowledge in collaboration as “innoversity” because it is so essential to innovation (Nie, 2016). In speaking to the creative and innovative potential of interdisciplinary work, some scholars reference Granovetter’s (1973) “strength of weak ties” concept to support the idea that connections between more distant disciplines introduce knowledge with a range of diversity necessary for the most innovative ideas and help combat group-think (Baer, 2010; Nie, 2016). In other words, in interdisciplinary collaborations, more diversity can mean more creative and innovative outcomes (Nie, 2016). This is one reason behind the growing support for an unlikely, and seemingly distant, partner in community and social problem solving: the design discipline. The Integration of Design Into Interdisciplinary Community Problem Solving The word ‘design’ is perhaps one of the most difficult words to define in a succinct and universal manner (Buchanan, 1992). For example, in a single paper on design theory, Friedman 9 (2003) describes design as a process, a profession, a discipline, and a field. Although there are universal understandings of design and its varying uses within the design field itself, common rhetoric has often related this term with art and craft (Friedman, 2003). Deeper thinking may remind one of common design practices like architecture or land planning, but a likely initial association would conjure thoughts of aesthetically pleasing logos, websites, clothing, or sculptured furniture. Despite its ambiguity in common dialogue, and the tendency to relate it to the aesthetics, design is equally art and science by principle and as an entire discipline it is a crucial contributor to the contemporary social world (Friedman, 2003; Sutton & Kemp, 2006). Think of the signs that direct daily life by guiding movement, thoughts, and responses; or the buildings that make those acts safe, familiar, or more efficient. In modern environments, design exercises constant influence by mediating human interaction with most of the physical and cognitive world (Friedman, 2003). Everything you could reach out and touch or look up and read right now was designed not only to look nice but, more importantly, to solve a problem (Frascara, 1988). The products of design include an “array of communicative, dialogic, and action-fueled tools” that disseminate information, guide the processing of that information, and instruct efficient response or action (Gibson & Owens, 2014, p. 387; Smith, 2012). In general, there are four areas of design, of which the understanding will help solidify the massive reach of design: symbols and visual communications; physical objects; activities and organized services; and systems for living, working, playing, and learning (Buchannan, 1992). The work of various types of designers - including graphic designers but also architectural, industrial, human-computer interaction, user experience, apparel, interior, product, or landscape designers - falls within these 10 four areas. Essentially, across the discipline there is an underlying approach to problem solving that aims to produce innovative and approachable solutions to complex problems. The impact of design reaches beyond aesthetic appeal in that “inadequate or inferior” design “can affect the safety, social opportunity, stress level, sense of belonging, self-esteem, or even physical health…in a community” (Margolin & Margolin, 2002, p. 26). The community impact of design is promoted by organizations like The Center for Urban Pedagogy, which “employs graphic design to elucidate complex public policy in ways that teach, engage, and entertain” (Catherwood, 2012, p.19). Many similar efforts have following suit, based on the power of design to “raise public awareness and affect decision making about community health issues, sustainability, poverty, proposed planning ‘scenarios’, and complex policy details” (Catherwood, 2012, p. 19). The potential role of designers in the work of community organizations can be illustrated through a simple description of design’s role in creating something new. Within the process of creating a typically designed product (e.g., buildings, websites, furniture, etc.) there are “agents”, often including clients and designers (Galle, 1999). A client provides the specifications, or “design brief”, for the product; that is, the problem it needs to solve, who is going to use it, personal preferences, and relevant contextual information. From there, this brief goes to a designer whose job is to ideate the most efficient and effective product while balancing all of the specifications and being aware of the feasibility given the capabilities of whomever will construct the product. This means, essentially, a designer creates a “representation” of a product (e.g., a plan, blueprint, solution). This representation is then either passed on to a maker to be constructed, or the designer or client may construct the final product themselves. 11 Galle (1999) uses examples of small construction projects to illustrate the critical role of designers in this process. Similar to Galle’s examples, say a homeowner (client) specifies that their porch is too small for entertaining; then an architect addresses this problem in terms of client preferences, physical requirements, and feasibility to create a representation of a deck; finally, a carpenter interprets this representation and builds the deck. Although rudimentary, this example captures the basic essence of design’s important transformative and translational role in generating applicable, innovative, and feasible solutions. If the client went straight to the carpenter, they likely would not be able to articulate their needs in actionable terms for the carpenter (e.g., specific dimensions, type and quantity of building materials) and translate those needs into a building plan. This circumventing approach is prone to miscommunication, unsystematic ambiguous design, and, likely, an inadequate solution to the problem. Of course, an even more troubling approach would be if the client decided to do it all alone (assuming they are not a master architect and carpenter). That is, they could gather the specifications based on their expert knowledge of the problem and environment, but then attempt, with much more limited expertise, to translate that knowledge into an integrative and feasible design, and finally construct the final product. Likely, the deck would end up with structural integrity similar to that of the Little Rascals’ clubhouse. The example above illustrates one of Galle’s (1999) points that, in many contexts, the process of generating new products is often condensed; the client produces something without a systematic design as a guide. This results in products that are infeasible, unsustainable, weak, and based only on things that already exist in the client’s environment. Now, think about the similar processes to which community-based organizations are confined for creating new programs, interventions, services, or products. For example, a youth shelter might discover that 12 the majority of its inhabitants cannot sufficiently read and write. Due to lack of time, staff, money, and the demands of the children’s already expansive needs, the shelter is on its own to generate a solution to quickly provide a service that addresses the problem while fitting their unique needs. Alternatively, a graphic designer is trained in guiding and managing systematic conceptualization and ideation to produce innovative, customized tools for strategic communication, behavior initiation, or imaginative stimulation. Essentially, design is the process in which ideas and custom solutions are cultivated, and graphic designers are trained to systematically execute that process in ways to maximize innovation and creativity. For the shelter, a graphic designer could take their needs and capacities, extract and present them via visual tools like concept maps or infographics, and then use those tools to arrange and ideate unique solutions together. For example, the designer and shelter might work on adapting an evidence-based math game that the shelter already uses, based on its locally-effective teaching style, into a tablet application for reading. Although this is a hypothetical example, imagine if graphic designers were more readily available to help community organizations in this way by working together to visually and systematically process some of the complex problems they face and transform them into creative and practical solutions. The idea of designers partnering with community-based organizations is not necessarily new. Land use designers have partnered with environmental organizations, architects have partnered with neighborhood organizations, and product designers have partnered with international development efforts. These are only a few broad examples, as the value of design has garnered increased recognition in various fields in recent years. Major funding entities like the National Science Foundation have issued requests for proposals that call for the inclusion of designers and design researchers in various types of efforts (Gibson & Owens, 2014). Margolin 13 and Margolin (2002) support the reason for this by describing some of the products that can be created when designers collaborate with organizations to help meet a community or social need: “teaching aids of all kinds including aids to transfer knowledge and skills to those with learning difficulties and physical disabilities; training aids for poor people who are trying to move into the work force; medical diagnostic devices, hospital equipment, and dental tools; equipment and furnishings for mental hospitals; safety devices for home and work; and devices that address pollution problems” (p. 28 via Papanek & Fuller, 1972). Unfortunately, designer-community collaborations often do not occur organically and there is limited research “to demonstrate what a designer can contribute to human welfare” (Margolin & Margolin, 2002, p. 28). Constraints from both sides make interdisciplinary collaboration in this arena difficult and sometimes even undesirable (Lee et al., 2009). From a design perspective, “incorporating community knowledge is not new to design, but design has not entirely or successfully come to grips with this reality” (Gibson & Owens, 2014, p. 386; Lee et al., 2009). Good design by necessity is participatory, and in some areas of practice even interdisciplinary, but for the most part “the inherently narrow focus of design disciplines makes it hard to address situations that involve diverse communities and their wildly varying expectations” (Gibson & Owens, 2014, p. 387). From the perspective of community organizations, time is a precious resource and funding is limited. Further, unfamiliar outsiders lack the cultural and contextual knowledge and experience that communities value as central to their problem solving processes (Rappaport, 2000; Sutton & Kemp, 2006). Even so, given that these partnerships presently do not occur naturally and are difficult to execute effectively, they must be facilitated with “purposeful action” in order to reap the many benefits of designer- community collaboration and to continue to legitimize its worth (Gibson & Owens, 2014). Bringing graphic designers, in particular, into community problem solving is a way to break free from the status quo by collaborating with a discipline whose expertise is based on 14 innovation and reimaging the realm of possibility within an ambiguous problem space (Frascara, 1988). The ability to solve complex community problems can be aided by graphic designers with the expertise to craft elegant visual representations of identities, feelings, messages, or the problems themselves (Frascara, 1988; Sutton & Kemp, 2006). The graphic design approach is driven by the need for practicality and simplicity, which can help clear the noise in complex systems to isolate the most direct and useful components of potential solutions (Buchanan, 1992; Cross, 2004; Galle, 1999). Graphic design can guide systems change by using visual and manageable tools that hold information and transformation in place as the work evolves (Galle, 1999; Herbert, 1993). Lastly, graphic designers are trained to design with and for the users, prototype and innovate for their needs, and inspire simple and direct change with elegant tools (Girling et al., 2006; Howard & Somerville, 2014; Roggema, 2014). These skills allow graphic designers to learn from the knowledge and skills of community change agents and their constituents and help apply them in new and powerful ways (Sutton & Kemp, 2006). Despite this, there is limited research on collaboration between community organizations and graphic designers. But, the literature describes processes for design-community collaboration similarly across other types of design that have more substantial records of community collaboration (e.g., architecture or landscape design; Smith, 2012; Webber, 2016; Zhang et al., 2015). Thus, it stands to reason that these processes can also be implemented with graphic design, given the shared problem solving approach and objectives across the design discipline as a whole. One particular approach to design-community collaboration that could be utilized to bring the benefits of graphic design to community organizations, and has already demonstrated feasibility throughout the discipline, is community design charrettes. 15 Facilitating Interdisciplinary Community Problem Solving: Community Design Charrettes The design discipline has not yet been able to embed itself in the planning and practice of community organizations despite the potential to collaboratively generate more innovative and practical solutions. This may be due in part to limited time and funding or limited research to support the inclusion of designers, but is also due to the difficulty of integrating two very distinct disciplines (Gibson & Owens, 2014; Margolin & Margolin, 2002). Graphic designers have the skills to improve community organizations’ ability to effectively communicate their identity and purpose, for example, or to promote the dissemination and use of programs and services. Even so, defining an interdisciplinary process for problem solving that values both graphic designers’ and community organizations’ disciplinary approaches and promotes their equitable integration has been a notable challenge (Cornish et al., 2015). Design charrettes are rapid work sessions in which the multi-disciplinary approaches of designers, typically architectural or land planning, and community organizations are brought together to collaboratively and quickly ideate a unique solution to a targeted problem (Sanoff, 2000; Sutton & Kemp, 2006; Sutton & Kemp, 2002). The word “charrette” means “cart” in French and the concept came from art students in 1800’s France who loaded their final projects on carts that would take them to be submitted. The final moments leading up the cart’s arrival, and even after with some students jumping on the carts to complete final details, were known to be high energy and incite last minute bursts of creativity (Sutton & Kemp, 2006). Thus, the term “en charrette” came to be synonymous with rapid creation spurred by a quickly approaching deadline (Lennertz, 2003; Sanoff, 2000). Later, the term also took on a collaborative meaning with accounts alleging that students could be found coming together on the carts to collaboratively solve their most challenging problems in the final moments (Sutton & Kemp, 16 2002). Whether or not those tales are true, collaboration and consensus became guiding principles of design charrettes (McLaughlin, 2013; Sutton, 2000). The concept was adopted by architecture students and lived on primarily in design training. However, more recently, design charrettes have been used most often to involve clients and community members in urban planning, architecture, and landscape design (Smith, 2012; Webber, 2016; Zhang et al., 2015). Contemporary definitions of the charrette process often describe it as “a multi-day planning process during which an interdisciplinary professional design team creates a complete and feasible plan that reflects the input of all interested parties by engaging them in a series of feedback loops” (Lennertz, 2003, p. 2). An example in a community context might be a group of volunteer designers teaming up with the director of a homeless shelter to spend two days collaborating on innovative solutions to a pressing problem, like a lack of beds or limited use of a new service. Modern design charrettes can involve a mix of participants from various disciplines like architecture, community research, environmental design, urban development, and education and have been found to have great utility and success in many contexts including participatory action research (Howard & Somerville, 2014), urban planning (Smith, 2012), post- disaster recovery (Goedert, 2008; Tanaka et al., 2009; Zhang et al., 2015), design education (Kowaltowski et al., 2015; Walker & Seymour, 2007; Webber, 2016), building safety (McLaughlin, 2013), land use and landscape planning (Dhar & Khirfan, 2016; Girling et al., 2006; Maryman & Maggio, 2004; Valencia-Sandoval et al., 2009), and youth and community engagement (Lessard & Torres, 2007; Onyango & Noguchi, 2009; Rottle & Johnson, 2007; Sutton & Kemp, 2002). Studies of design charrettes in these contexts have revealed various positive outcomes including sustainable community development plans (Valencia-Sandoval et al., 2009), locally appropriate responses to climate change (Dhar & Khirfan, 2016), fresh and 17 innovative plans for disaster recovery (Tanaka et al., 2009), buy-in and competence in implementing new innovations (Rottle & Johnson, 2007), and even improved attitudes toward design (Onyango & Noguchi, 2009). The success and “effectiveness of charrettes can be linked to the fact that interdisciplinary teamwork can be practiced” (Kowaltowski et al., 2015, p. 54). This interdisciplinary collaboration increases the potential for innovation and integrative knowledge generation (McLaughlin, 2013; Webber, 2016). In a community context, the collaborative nature of design charrettes may present an incredible tool for “advancing proactive adaptation through ecological design” (Dhar & Khirfan, 2016). A design charrette involving designers and community organizations provides a space to integrate the organization’s contextual knowledge and deep understanding of the problem with the designer’s practical, innovative, and systematic problem solving expertise (Dhar & Khirfan, 2016). Community design charrettes are said to promote “lasting, transformative community change” via key strategies including joint work sessions, structured feedback loops, and a focus on creating both holistic solutions and feasible plans for action (Lennertz, 2003, p. 1). Recently, the charrette approach has appeared in community psychology literature as a tool for interdisciplinary problem solving between social scientists, designers, and community members. However, it has yet to develop a substantial hold in the field despite its complementary focus on interdisciplinary collaboration, participation, and capacity building (Sutton & Kemp, 2006). Sutton and Kemp’s (2006) case studies of three charrettes involving grade school students, design students, social science students, and community partners (a school district, city planners, and a neighborhood agency) document positive outcomes including increased ecological awareness, highly integrated products, collaborative publications, and immediate 18 solution implementation. Charrettes have the potential to be a promising and innovative problem solving tool in community psychology, as they have been in other fields, given their stylized, collaborative, and goal-oriented nature that creates a “business-not-as-usual space” within which to explore new ways of thinking and challenge default thought processes (Sutton & Kemp, 2002, p.172; Webber, 2016). In addition to reflecting on the positive outcomes of the community design charrettes in their case studies, Sutton and Kemp (2006) give equal attention to the drawbacks and barriers present in each case. Unfortunately, given the extent to which these charrettes bring together such a starkly contrasted mix of approaches, experiences, and practices, there is a need to understand and build a process to facilitate the most integrative and collaborative community design charrettes. They discovered, as with any collaborative venture, that there are certain preconditions that support the efficacy and utility of design charrettes (Sutton & Kemp, 2006). Notably, in terms of process, charrettes serve as a “successful participatory design strategy when applied to specific goal-oriented objectives of a clearly defined problem” (Sanoff, 2000, p. 50). More specifically, an effective collaborative approach to solving a shared problem must be supported by common values, objectives, and methods among all contributing members. From a design perspective, the literature agrees that successful charrettes depend on this consensus and designers approaching charrettes with a willingness to use more aligned approaches (Girling et al., 2006; McLaughlin, 2013). The process of facilitating these charrettes, especially in the pursuit of visual solutions to complex community problems, is rarely studied and infrequently practiced. Currently, there is not a systematic understanding of or approach to facilitating effective charrettes between designers and community organizations that produce the most integrated and useful solutions 19 (Sutton & Kemp, 2006). To begin to address this, I turn to literature on various forms of collaborative problem solving efforts among diverse participants (i.e., participatory and interdisciplinary research, design charrettes, and interorganizational collaboration) to better understand the preconditions (e.g. an aligned approach to problem solving including shared or complementary objectives, values, and methods) that contribute to effective collaboration (i.e., more integrated community design charrette products). Design-Community Partnerships: Different Approaches to Community Problem Solving The literature suggests that there are certain factors related to how individuals or organizations frame problems and approach solving problems that influence the likelihood of effectively collaborating to produce interdisciplinary products (Barron, 2000; Butterfoss et al., 1993; Gray, 2004; Israel et al., 1998; Nie, 2016; Sanoff, 2000; Stempfle & Badke-Schaub, 2002; Stokols et al., 2008; Suarez-Balcazar et al., 2006; Sutton & Kemp, 2006; Zhang et al., 2015). There is agreement across many fields of collaborative study that aligned problem solving approaches or, more specifically, shared or complementary values, objectives, and methodological approaches contribute to effective collaboration (Howard & Somerville, 2014; Horwath & Morrison, 2007; Maton et al., 2006; Nowell, 2010; Sanoff, 2000; Stokols et al., 2005; Sutton & Kemp, 2002). In order to understand how this alignment influences collaboration between designers and community organizations, it is helpful to first define their respective approaches to problem solving. Sutton and Kemp (2006) used their case studies of community design charrettes and related literature to construct a community problem solving framework outlining the approaches of community partners, designers, and social scientists. Specifically, the framework defines each group’s approach based on conceptual aspects of problem solving (i.e., values, objectives, and 20 methods) and the results of their respective approaches (i.e., advantages, drawbacks and outcomes; Sutton & Kemp, 2006). In the present study, this framework will be used as a guide for distinguishing the unique approaches to problem solving present in design charrettes involving graphic designers and community partners. This section focuses on the procedural aspects of problem solving: values, objectives, and methods and, in line with the topic of this study, more narrowly on those of only community organizations and designers. Values. The unique values of each partner are an important factor in interdisciplinary collaboration (Andrews et al., 2010; Butt et al., 2008; Long, 2001; Maton et al., 2006; Nie, 2016; Nowell, 2009; Stokols et al., 2005; Stokols et al., 2008; Sutton & Kemp, 2006). Each discipline within a community design charrette has different standards for competence, meaning their primary considerations (i.e., values) during work and planning are often different (Sutton & Kemp, 2006). For example, consider a community organization partnering with an environmentally focused graphic design firm to clean up a local river. This example demonstrates that an interest in similar issues does not always mean disciplines value the same things or approach problems in the same way. While the community organization’s foremost value is its residents’ health and quality of life, the graphic designers may value the greater impact of wasteful behavior on water quality or have more abstract values like influencing public knowledge or attitudes. More generally, community organizations tend to value tested, familiar approaches while designers value creativity and innovation (Sutton & Kemp, 2006). The following sections use Sutton and Kemp’s (2006) framework to further detail the values of community organizations and designers related to community problem solving. Community organization values. Within their framework, Sutton and Kemp (2006) explain that community organizations “value technical solutions that reflect the complexity of 21 their everyday realities” (p. 53). Because their objectives are primarily focused on rapid, local solutions, inherently these objectives are driven by values of personal experience and local history. Rappaport (2000) offers an example of the value of local history in communities in a discussion of the importance of local narratives. Even communities trying to address incredibly complex problems like homophobia or racism place value in the power of positive historical narratives to influence change (Rappaport, 2000). Further, community organizations’ values are built on local knowledge and an evolving collective approach to problem solving, often making them partial to the status quo and reliable, tested solutions (Cornwall & Jewkes, 1995). Community organizations also value efficiency and resource conservation (Sutton & Kemp, 2006). Solutions may be particularly desirable if they are affordable, non resource-intensive, familiar, or already effective in similar contexts. Designer values. Designers’ core values are typically based on intuition, action, and the theoretical elements of good design rather than their own surroundings or experiences (Cross, 2004; Friedman, 2003). The community problem solving framework summarizes that designers “value originality and artistic expression that is practical and uplifts the human spirit” (Sutton & Kemp, 2006, p. 53). For a designer, the guide for making things uplifting is intuition. Designers are taught early in their training to value the role of intuition in creativity and decision-making (Cross, 2004; Friedman, 2003). Designers also place great value on originality and creativity (Dorst & Cross, 2001), as is evidenced by their value of innovative solutions. In terms of innovation, designers aim to transform problem spaces into “preferred situations” by creating something new or reimagining something that already exists (Friedman, 2003; Sutton & Kemp, 2006). This value often receives the most resistance in community settings due to community 22 organizations placing strong value in what is already shown to work and not taking risks with valuable resources (Sutton & Kemp, 2006). Objectives. Stakeholder objectives are a commonly discussed factor in interdisciplinary collaboration (Andrews et al., 2010; Bronstein, 2003; Horwath & Morrison, 2007; Howard & Somerville, 2014; Israel et al., 1998; Suarez-Balcazar et al., 2006; Sutton & Kemp, 2002; Sutton & Kemp, 2006). Bringing together diverse perspectives often involves similarly diverse needs, wants, agendas, goals, and targeted beneficiaries (Horwath & Morrison, 2007; Maton et al., 2006; Spath et al., 2008). Returning to the partners in the river cleanup example, although their methods may be complementary and beneficial to one another, their objectives are likely different because they aim to serve different beneficiaries, have different agendas, or operate based on different training and disciplinary practices. The organization has a local agenda to create posters that encourage volunteers to help clean up a single river in their particular community. The graphic design firm on the other hand, has a regional, or even national, agenda to develop visual communication tools or strategies that can be applied across many communities to promote more environmentally clean behaviors (e.g., creating posters to promote recycling, improved waste management, or littering policies). The following is a description of the general themes guiding community organization’s and designer’s respective objectives in problem solving efforts as described in Sutton & Kemp’s (2006) framework and supported in the literature. Community organization objectives. When attempting to solve problems, the objectives of community organizations are driven primarily by one, obvious theme: their local community. These organizations are rooted in their unique environments. They have local knowledge that inspires objectives highly tailored to the collective context of the organization and its 23 surrounding community (Strier, 2011; Sutton & Kemp, 2006). Further, community organizations are often limited in their range of options for solutions based on political context. Limited funding, time, or staff, and restrictive policy or leadership often push community organizations to adopt objectives that are as practical, applicable, and non-controversial as possible. As summarized in the community problem solving framework, the objective of community organizations is “to achieve proactive or reactive goals that reflect their varying backgrounds and motivations” (Sutton & Kemp, 2006, p. 53). This applies to the backgrounds and motivations that influence individuals acting in representation of a larger group as well, as is often the case with community organizations (Sutton & Kemp, 2006). In these settings, the backgrounds and experiences of the staff inform the organizational objectives as these individuals are dealing with the problems, witnessing them at home, or interacting with those affected in their day-to-day lives (Cornwall & Jewkes, 1995; Lappe & Du Bois, 1994). These detailed local perspectives, along with organizations’ collective history and limitations, are the building blocks of objectives in community problem solving. Designer objectives. In the design discipline, objectives tend to be aimed at more universal solutions that emphasize simplicity, functionality, and beauty (Cross, 2004; Friedman, 2003). Sutton & Kemp (2006) summarize designers’ objectives in community problem solving as “[providing] a specific solution that beautifies and responds to functional and symbolic needs” (p. 53). Designers try to identify simple leverage points within complex problems and imagine solutions that will address a specific audience and a targeted aspect of the problem (Friedman, 2003). In order to design for and/or with that audience, their needs, preferences, or capabilities need to be defined as accurately and as universally as possible for maximum usability and collective impact (Stempfle & Badke-Schaub, 2002). Therefore, objectives are often focused on 24 serving a large, albeit highly detailed, user profile over varied individuals contexts. Individual context is far less useful, especially if it represents outliers, when considering optimal functionality for the largest possible group. Thus, the objective of functionality calls for a balance between person-centered and universally useable designs. Finally, good design is said to require the presence of beauty (Sutton & Kemp, 2006). ‘Beauty’ can of course take on different meanings to a furniture designer, architect, land planner, or graphic designer, for example, but a primary goal of design is to inspire the user in some way, be it intellectually or emotionally, visually or spatially. Methodological approach. Perhaps the most influential aspect of collaborative problem solving, above and beyond partners’ objectives and values, are their respective methodological approaches. Interdisciplinary collaboration can bring together partners with starkly different training, skills, and methods even when they are trying to solve the same problem (Maton et al., 2006; Sutton & Kemp, 2006). For example, the river cleanup organization may be compelled to promptly call a local meeting to elicit volunteers so they can get out and clean as they always do but with greater numbers, while the design firm’s approach might be to build models or visual representations of unconventional cleanup efforts in order to synthesize them into one new, innovative strategy. Variation in problem solving methods is noted as an important consideration across the interdisciplinary, collaborative, and participatory literatures (Barron, 2000; Cornwall & Jewkes, 1995; Howard & Somerville, 2014; Maton et al., 2006; Obrien et al., 2002; Stokols et al., 2008). The following sections outline the respective methods of community organizations and designers according to Sutton and Kemp’s (2006) community problem solving framework. Community organization method. Across the literature, and in practice, there are many diverse examples of community organizations taking different methodological approaches to 25 solving problems (e.g. Rappaport, 2000; Suarez-Blacazar et al., 2006; Sutton & Kemp, 2006). Similarly, there is limited consensus on any one, clearly defined approach (Sutton & Kemp, 2006). As descried by Sutton and Kemp (2006), community organizations “utilize the skills of their outside lives; bring relationships, preconceptions, and agendas; [and] lack a normative methodology (p. 53). As noted previously, lived experience and local familiarity are essential for community organizations (Rappaport, 2000), so these organizations call upon them to inform familiar and applicable methods. Further, external influences like funding or policy on organizational preconceptions and agendas create a great deal of variation in methods based on community organizations’ own unique circumstances and restrictions. There is one recurring theme present across the literature, communities’ use of networks and personal relationships as a problem solving method (Suarez-Blacazar et al., 2006; Sutton & Kemp, 2006). Still, it is likely that a single convergent method does not exist across all community organizations. In fact, the overall “method” may be that community organizations do not have a common approach, or even individually defined processes, but instead may take different actions for different problems that are guided by influences like local context and previously attempted solutions. Designer method. Design methods for problem solving are more clearly defined, as solving problems is at the root of a designer’s training. Although seemingly chaotic to an outsider, designers’ methods are highly intentional and systematic. When solving a specific problem, these methods include facilitating the navigation of the problem space with the client (Dhar & Khirfan, 2016; Sutton & Kemp, 2002) via modeling or visualizations (Cornwall & Jewkes, 1995; Friedman, 2003; Lennertz, 2003), followed by the rapid generation of multiple possible solutions, and then iterative decision-making (Stempfle & Badke-Schaub, 2002; Sutton & Kemp, 2002) through prototypes, feedback loops, and exploration of newly framed problem or 26 solution spaces (Dorst & Cross, 2001). Sutton and Kemp (2006) summarize these methodological approaches within the community problem solving framework, stating that designers “co-evolve problem and solution; use an inquiry mode that involves simplification; [and] derive a concept (the big move) that guides future decisions” (p. 53). Also, in contrast to the network-based approaches of community organizations, designers’ problem solving approach is often primarily an independent process (Stempfle & Badke-Schaub, 2002; Sutton & Kemp, 2006). Efforts to promote the training and practice of participatory design are growing in areas like architecture and land planning, but there are still plenty of challenges to implementing it effectively (Lee et al., 2009; Obrien et al., 2002). Taken altogether, the approaches outlined in Sutton and Kemp’s (2006) community problem solving framework are supported in the respective community and design literatures, but further validation is necessary to consider it as a guide for defining the problem solving approaches of designers and organizations in interdisciplinary collaboration efforts like community design charrettes. This framework may be particularly helpful for studying long- standing disciplinary approaches, which influence interdisciplinary problem solving between designers and community organizations. Specifically, in charrettes where community organizations and designers are brought together to solve a common problem, the following research question is raised: Q1. Within community design charrettes, in what areas and to what degree do the community problem solving approaches of designers align with those of community organizations in terms of their objectives, values, and methodological approaches? 27 Effective Interdisciplinary Collaboration: Integrated Products of Design-Community Partnerships Understanding the alignment of problem solving approaches among partners is critical to the study of community design charrettes because this alignment is potentially a key contributor to effective interdisciplinary collaboration (i.e., highly integrated products; Barron, 2000; Butterfoss et al., 1993; Gray, 2004; Israel et al., 1998; Nie, 2016; Sanoff, 2000; Stempfle & Badke-Schaub, 2002; Stokols et al., 2008; Suarez-Balcazar et al., 2006; Zhang et al., 2015). Studies of interdisciplinary collaboration from various fields have defined effectiveness as both positive long-term outcomes and the development of critical proximal outcomes (Masse et al., 2008; Roussos & Fawcett, 2000; Suarez-Balcazar et al., 2006; Zakocs & Edwards, 2006). The ultimate, overarching goal of these collaborations is often appropriately lofty: to improve public health (Israel et al., 1998), create new scientific fields of study (Stokols et al., 2008; Tebes et al., 2014), or respond to climate change (Dhar & Khirfan, 2016) for example. Of course, because these goals are both complex and time-intensive, it is necessary to identify and study intermediate indicators and outcomes that ultimately will contribute to the desired high-level impact (Masse et al., 2008; Mattessich & Monsey, 1997; Roussos & Fawcett 2000; Stokols et al., 2008). In interdisciplinary efforts, the most immediate indicator of collaborative effectiveness is a tangible, integrated product that reflects a clear integration of two or more disciplines (Bronstein, 2003; Hall et al., 2012; Masse et al., 2008; Maton et al., 2006; Stokols et al., 2008). These integrated products are often wide-ranging (e.g., plans, buildings, academic curricula) depending on the type of collaboration and reflect the requests and experiences of all contributors (Bronstein, 2003; Lennertz, 2003; Zhang et al., 2015). The products present evidence of co-creation or co-authorship of the product (i.e., participants’ disciplinary knowledge 28 and skills have been synthesized or dually represented; Horwath & Morrison, 2007; Howard & Somerville, 2014; Lennertz, 2003; Nie, 2016; Stokols et al., 2008; Stokols et al., 2005; Suarez- Balcazar et al., 2006), and, in the case of a viable solution, consensus around the chosen approach (Gray, 2004; Sanoff, 2000). In contrast, products of ineffective collaboration are unbalanced in disciplinary contribution (i.e., one discipline’s knowledge or skills is unfavorably dominant; Sutton & Kemp, 2006b), are highly likely to not be understood by all partners right away (Stempfle & Badke-Schaub, 2002), and even risk never being used at all (Howard & Somerville, 2014). In the case of designers and community organizations, effective collaboration results in integrated products that reflect the knowledgeable, experienced, and community-embedded organizations as well as the creative, intuitive, and practical designers (Dhar & Khirfan, 2016; Sutton & Kemp, 2006). Each discipline, design and community organizational practice, offers unique knowledge and skills that must both be reflected in a product in order to establish effective interdisciplinary collaboration. Community organization contributions. The indicators of community organizations’ contributions in collaborative products understandably reflect much of their values, objectives, and methods described in the community problem solving framework (Sutton & Kemp, 2006). The main themes of a community organization’s contributions involve local specifics, tradition, and a nuanced contextual understanding. Therefore, if community organizations’ contributions have been integrated into the product, it will be localized, embedded in the organizational context (Lee et al., 2009), culturally competent (Suarez-Balcazar et al., 2006; Sutton & Kemp, 2002), responsive to organizational and local priorities (Cornwall & Jewkes, 1995; Dhar & 29 Khirfan, 2016; Sutton & Kemp, 2006; Zhang et al., 2015), and of course appropriate for and desirable to the organization (Girling et al., 2006). Designer Contributions. Again, the indicators of design contributions in collaborative products mirror it’s disciplinary problem solving approach described via Sutton & Kemp’s (2006) framework. A designer’s job, in the most general sense, is to make something new or reimagine something that already exists in order to create a preferred situation (Friedman, 2003). Thus, in a design-community partnership, a designer will often take the contributions of the organization and transform them into a communicative, inspiring, or action-oriented tool to practically solve the problem at hand. Therefore, if designers’ contributions have been integrated into the product, it will have a practical purpose and is often marked by creativity (Sutton & Kemp, 2002), systematic attention to technical detail (Friedman, 2003; Sutton & Kemp, 2006), and ergonomic or aesthetic quality (Dorst & Cross, 2001). Another key indicator of design contribution is innovation (Nie, 2016), such that the organization or community would likely perceive the products as new or “fresh”, even if the content, or local contributions, that shaped them is not (Tanaka et al., 2009, p. 315). Aligned Approaches as Predictors of Effective Interdisciplinary Collaboration The literature suggests that effective interdisciplinary collaboration is influenced by the alignment of problem solving approaches among the collaborators in various contexts (Barron, 2000; Gray, 2004; Butterfoss et al., 1993; Israel et al., 1998; Nie, 2016; Sanoff, 2000; Stempfle & Badke-Schaub, 2002; Stokols et al., 2008; Suarez-Balcazar et al., 2006; Zhang 2015). To explain this phenomenon in the design discipline, Nie (2016) calls upon the metaphor of music performers (from Sawyer, 2006) to describe the importance of group ‘flow’ in communities of creative innovation. Imagine the precise alignment of approaches that must be necessary for 90 30 musicians to play the music of a great orchestra. In order to create such a beautifully integrated product (i.e., the music), there must be a common objective, high consensus on what they value in the work, and shared techniques among all the players. Nie (2016) says that within a creative collaborative context, the potential for both creativity and innovation is optimized when a group achieves this flow, or has a common objective, shared values, and complementary techniques. Likewise, Sutton and Kemp’s (2006) framework specifies values, objectives, and methods as primary elements of a discipline’s problem solving approach to consider in collaborative partnerships like design charrettes. Across the collaborative literature, shared or complementary values, objectives, and methodological approaches are all both theoretically and empirically linked to effective collaboration within a diverse array of partnership contexts including design charrettes (e.g., Howard & Somerville, 2014; Sanoff, 2000; Sutton & Kemp, 2002), student groups (e.g., Barron, 2000), community coalitions (e.g., Horwath & Morrison, 2007; Nowell, 2010), and interdisciplinary research (e.g., Maton et al., 2006; Stokols et al., 2008). So, although Sutton & Kemp’s (2006) framework may suggest low alignment between the disciplinary problem solving approaches of designers and community organizations, the literature states that participants in successful collaborative efforts must have perspectives that defy their disciplinary boundaries and align with their partners’ approaches (Stokols et al., 2008). These differences are certainly not unique to designers and community organizations. The inevitability of multiple diverse perspectives is intuitively clear in interdisciplinary work (Butterfoss et al., 1993; Maton et al., 2006; Strier, 2011), and this diversity can be difficult to navigate (Long, 2001; Suarez-Balcazar et al., 2006), but scholars agree that aligned problem solving approaches “integrate and transcend the multiple disciplinary perspectives” and are necessary for effective collaboration (Stokols et al., 2008, p.97). 31 On the other hand, Gray (2004) explains the there are also risks involved with limited or no alignment across problem solving approaches in the context of community collaboration and calls for greater attention to these factors. She states that “[while] many studies of collaboration have relied on [procedural] factors to explain outcomes…the failure of collaboration can also be explained by the divergence of stakeholders’ frames about the issues” (p. 166). Testing this claim, Strier (2011) conducted a case study of a partnership between a university and a community organization, within which a lack of alignment between their approaches to problem solving led to a minimally integrated product and incommensurate assessments of the product’s impacts. Thus, the importance of aligned problem solving approaches is further supported by literature that suggests that the absence of aligned values, objectives, and methods has a negative influence on effective collaboration. Stokols and colleagues (2008) summarize the need for alignment and the consequences of its absence in a quote that draws on prominent interdisciplinary collaboration literature on community coalitions: “Coalitions whose members endorse competing goals and outcomes; hold different views of science and society; and use dissimilar terminology, language, and decision making styles are likely to experience conflicts that undermine the team’s performance. Coalitions that identify clear goals and objectives perceived to be attainable, agree on shared research-principles, and reach consensus on major areas of concern face fewer collaborative challenges” (Stokols et al., 2008, p. 104; Butterfoss et al., 1993; Israel et al., 1998; Lantz et al., 2001; Stokols et al., 2005). In the design charrette literature specifically, there are both theoretical calls (e.g., Girling et al., 2006; Lennertz et al., 2008; Sanoff, 2000) and empirical support (e.g., Zhang et al., 2015) for aligned approaches to problem solving. For example, Zhang et al. (2015) observed a 10-day design charrette of 14 experts (e.g., in landscape design, community resilience, farming, environmental management) brought together for post-disaster reconstruction planning following the Fukushima nuclear disaster in Japan. The findings suggested that an aligned approach 32 contributed to holistic consideration of the reconstruction and an integrated final product. In addition, this alignment was considered essential in the charrette problem solving process, as it meant participants were not making decisions based only on their own individual approaches (Zhang et al., 2015). Because of the limited time frames and unique problem solving conditions, approach alignment is so critical in design charrettes that the “quality and integrity” of the products depends on mutual understanding and consensus “among the multiple people and perspectives involved” (Girling et al., 2006, p. 114). Thus, there is a significant need to explore problem solving approach alignment in design charrette contexts in order to help them be a more effective collaborative tool in communities. The following three sections will review various collaborative literatures (e.g., design charrettes, community coalitions, interdisciplinary research, and participatory action research) that suggest that shared or complementary values, objectives, and methodological approaches are important for promoting effective interdisciplinary collaboration. Aligned values. Before collaborative efforts even begin, each discipline brings a problem solving approach driven by a unique set of values (Gibson & Owens, 2014; Maton et al., 2006; Sutton & Kemp, 2006). Even so, throughout the collaborative literature, scholars agree that aligned values among partners are crucial for effective collaboration (Andrews et al., 2010; Butt et al., 2008; Long, 2001; Maton et al., 2006; Nie, 2016; Nowell, 2009; Stokols et al., 2005; Stokols et al., 2008) and their absence can seriously hinder collaboration (Horwath & Morrison, 2007; Howard & Somerville, 2014; Jacobs & Frickel, 2009; Stokols et al., 2008; Strier, 2011; Suarez-Balcazar et al., 2006). In the literature, including that on design teams, coalitions, interdisciplinary science, and participatory research, the presence of shared or complementary values is linked to effective interdisciplinary collaboration (Butt et al., 2008; Long, 2001; Nie, 33 2016; Stokols et al., 2005; Stokols et al., 2008;) and considered a factor helpful in overcoming common barriers to effective interdisciplinary processes (Maton et al., 2006). On the other hand, the absence of shared values, which can be observed in studies of very diverse collaborators, increases the likelihood of ineffectively integrated products (Sutton & Kemp, 2002). Aligned values influence collaborative problem solving because cultural, organizational, or disciplinary values shape the way one defines, and subsequently chooses to approach, a problem (Maton et al., 2006; Nie, 2016; Nowell, 2009). As observed by Nowell (2009) following a comparative study of domestic violence coordinating councils, shared values “directly [relate] to how stakeholders think about the targeted issue…and what beliefs and assumptions they hold about the most effective means for addressing it” (p. 107). One example of the powerful influence of aligned values on collaboration is a qualitative study of The Haifa Partnership for the Eradication of Poverty in Israel (Strier, 2011). The partnership observed in the study brought together social work professionals and students with poor families to address the presence of poverty and social inequality in their community. Via interviews, archival records, and reflection essays, Strier (2011) concluded that shared core values were minimal and participants were driven by one of three principle value sets: professional/educational, instrumental, or political. The social workers fell into the first category, meaning they valued building their understanding of poverty and professional development. The families fell into the latter two categories. Those with primarily instrumental values were concerned with their own welfare and building personal relationships with the social workers. The rest of the families were classified as having primarily political values, meaning they believed social awareness, activism, and social action should drive the work. These differing values led to dissatisfaction with others in the partnership. For example, the social workers, 34 oriented around a value for knowledge, thought the families with instrumental values were using the program for what they believed to be the wrong purpose. On the other hand, families with instrumental values considered the values of social workers and families’ with political values to be remote and utopian. Finally, families with political values ended up expressing dissatisfaction with the partnership, specifically the social workers for their lack of value in political activism. Strier (2011) concluded that a lack of shared values impacted successful, integrative collaboration such that many participants reported that the other groups did not adequately contribute to the final product. Although there has not been extensive research on aligned values in the community design charrette context (see Howard & Somerville et al., 2014 for an exception), there are examples in the charrette literature that emphasize the importance of attending to different values in community settings (e.g., Lennertz, 2003; Roggema, 2014; Sutton & Kemp, 2006). Supporters suggest that charrettes are a good venue for bringing together diverse values, given that they are designed to be transformative spaces that strive for innovation by overcoming the constraints of dated, intra-disciplinary values (Roggema, 2014). Designers having an understanding that complements working in alignment with local values is important for charrettes focused on community improvement (Lennertz, 2003). Holding community charrettes on site at organizations or in the neighborhoods, or setting up studio workspaces in the local setting may encourage this alignment (Lennertz, 2003), thus making charrettes a good potential venue to support interdisciplinary collaboration between designers and community organizations. Aligned objectives. “The critical importance of clear and specific shared goals is stressed across the literature on collaboration” (Horwath & Morrison, 2007, p. 62). Variations of this same statement and empirical support for aligned objectives in effective collaboration are 35 prevalent throughout the literature on community coalitions and interagency partnerships (e.g., Alexander et al., 2000; Butterfoss et al., 1993; Horwath & Morrison, 2007; Israel et al., 1998; Lewandowski & GlenMaye, 2002; Roussos & Fawcett, 2000; Spath et al., 2008); as well as, interdisciplinary research (e.g., Bronstein, 2003; Long, 2001; Maton et al., 2006; Stokols et al., 2008), participatory research (e.g., Andrews et al., 2010; Strier, 2011; Suarez-Balcazar et al., 2006), and design charrettes (e.g., Girling et al., 2006; Howard & Somerville, 2014; Lennertz, 2003; Sutton & Kemp, 2006; Sutton & Kemp, 2002). In an extensive literature review on community health collaborations, Roussos and Fawcett (2007) summarize their findings on shared objectives, stating that "[a] clear vision and mission may help generate support and awareness for the partnership, reduce conflicting agendas and opposition, help identify allies, and minimize time costs and distractions from appropriate action" (p. 384). Thus, further collaborative literature both theoretically and empirically suggests that a clear, shared objective promotes effective interdisciplinary collaboration (Butterfoss et al., 1993; Bronstein, 2003; Horwath & Morrison, 2007; Long, 2001; Maton et al., 2006; Nie, 2016; Spath et al., 2008; Strier, 2011; Suarez-Balcazar et al., 2006). Entering into an interdisciplinary partnership with an aligned objective is understandably difficult (Suarez-Balcazar et al., 2006; Israel et al., 1998) given that different disciplines, and individuals for that matter, have different priorities (Maton et al., 2006; Spath et al., 2008), motivations, languages, and organizational cultures (Horwath & Morrison, 2007). Even so, the literature stresses that aligned objectives are critical from the onset of the project (Andrews et al., 2010; Horwath & Morrison, 2007) and without them, there are detrimental consequences including nonintegrated, unfavorable products (i.e., ineffective collaboration; Howard & Somerville, 2014; Sutton & Kemp, 2002). 36 Sutton and Kemp (2002) provide an example of the influence of shared objectives on collaboration in one of their earlier studies on two charrettes involving designers and community organizations (three schools and a neighborhood planning organization, respectively). The charrette participants included architects, the organizations, community constituents, and young students who sought to transform the neighborhood and school environments to improve children’s experiences (Sutton & Kemp, 2002). In the end, one charrette was far more successful at collaborating, creating integrated products that were useful, desirable to the community, and actually implemented in some cases. The second charrette, on the other hand, not only created products that were never used but also received unfavorable feedback about the process itself. Findings from a formative evaluation of the charrettes revealed that a major factor leading to these outcomes was the alignment of objectives (Sutton & Kemp, 2002). In the first successful charrette, both the designers and community partners shared a community-focused objective, and also had complementary goals that involved interdisciplinary benefits like gaining new skills and learning from one another (Sutton & Kemp, 2002). In the second unsuccessful charrette, the design partners, the architects, set their primary objective as a high quality design. Possibly due to the fact that their organization was a paying client, the designers had a personal stake in the project and focused on a good design that would ensure the products reflected well on their abilities. The community partners, the organization and youth, were far more focused on problems in their neighborhoods and their driving objective was actionable solutions (Sutton & Kemp, 2002). Regardless of the context, Sutton and Kemp (2002) reached the conclusion, common across collaborative literature, that aligned objectives are crucial for effective collaboration in design charrettes. 37 Aligned methodological approaches. The third and final element of an aligned problem solving approach in interdisciplinary collaboration is a aligned set of methods among all partners. Although it is evident that different disciplines are trained in and practice different skills and methods (Maton et al., 2006; Sutton & Kemp, 2006), interdisciplinary problem solving is unique in that it requires “methodologic flexibility” to promote effective interdisciplinary collaboration (Stokols et al., 2008, p. 104; Israel et al., 1998; Suarez-Balcazar et al., 2006). Suarez-Balcazar and colleagues (2006) theorized that in a research setting “understanding a social problem through interdisciplinary research comes from the melding of different systematic ways of organizing and studying phenomena” (p. 119). The melding of methodological approaches promotes effective collaborative and co-created outcomes (Barron, 2000; Howard & Somerville, 2014; Maton et al., 2006). Conversely, when disciplines remain rigid and do not allow for complementary methods, it can result in diverging priorities, limited interaction, and stymied development of collective norms and mutual learning (Howard & Somerville, 2014; Obrien et al., 2002; Stokols et al., 2008), all of which are precursors of non-integrated interdisciplinary products. Thus, although it may take some adjustment, aligned methods are critical for effective interdisciplinary collaboration. One instance of a shared methodological approach promoting a unique and fruitful collaboration comes from an example in the participatory research literature involving Zimbabwean women and researchers (Cornwall, 1992). The partners faced obstacles of different language, terminology, and starkly contrasting cultures when trying to discuss health issues. Fortunately, they discovered that visual representations were mutually understood and a skill possessed by both partners. In turn, body maps were drawn by the women that helped articulate critical differences between their and westerners’ conceptions of the body (Cornwall, 1992; 38 Cornwall & Jewkes, 1995). Thus, the shared method of visual representation helped create products that effectively incorporated both the researchers’ and the women’s expertise. As stated by Cornwall and Jewkes (1995) in a later account of the research, “this served as a medium for sharing ideas and locating explanations” that advanced the work, concluding that “[visualizations] reveal much that is masked by verbal communication” (p. 1671). Although relatively dated, this example is a simple and intriguing case of a shared method, visualization, being the principle contributor to effective collaboration. In the design charrette context, again there is limited research on aligned methodological approaches as they relate to effective interdisciplinary collaboration (see Howard & Somerville, 2014 and Sutton & Kemp, 2002 for exceptions). There is though, a widely acknowledged desire to better understand how to make charrette methods more integrated and fulfill one of their core values of designing ‘with’ rather than ‘for’ the community (Condon, 2012; Cornish et al., 2015; Dhar & Khirfan, 2016; Howard & Somerville, 2014; Roggema, 2014). Typically, there are three types of workflow strategies in design: serial, concurrent, and integrative (Obrien et al., 2003). Obrien et al. (2003) describe these three strategies in the context interdisciplinary design teams and collaborative processes (see Figure 1). In a serial strategy, the least collaborative, work flows from one discipline group to the next with minimal exchanges. This workflow, also referred to as “over-the-wall”, is common in design and represents what designing “for” the community might look like. With the final two strategies, work is either occurring at the same time, so challenges can be collaboratively discussed, or multiple feedback loops are established across all discipline groups. In both of these cases, a designer may be much more likely to achieve designing “with” the community (Obrien et al., 2003). In either case, this would still require the discipline groups 39 in a community-design charrette to have shared or overlapping methodological approaches in order to create products “with” one another and achieve effective collaboration. Figure 1. Alternative Approaches to Collaborative Work (Obrien et al., 2003) As is evident in the interdisciplinary, collaborative, and participatory literatures, alignment of problem solving approaches (i.e., shared or complementary objectives, goals, and methodological approaches) between partners contributes to more effective interdisciplinary collaboration (Barron, 2000; Butterfoss et al., 1993; Gray, 2004; Israel et al., 1998; Nie, 2016; Sanoff, 2000; Stempfle & Badke-Schaub, 2002; Stokols et al., 2008; Suarez-Balcazar et al., 2006; Zhang 2015). In the context of community design charrettes specifically, this relationship has serious implications, as it is suggested that the disciplines involved may have markedly different approaches (Sutton & Kemp, 2006). Therefore, it is important to understand how alignment of these approaches impacts effective collaboration by addressing the following research question: 40 Q2: Are design charrettes including designers and community-based organizations with more aligned community problem solving approaches associated with more effective interdisciplinary collaboration, defined as more integrated products that more effectively incorporate and utilize the disciplinary strengths of both designers (design contribution) and community organizations (community contribution), than those with less aligned approaches? Current Study In order to better understand the use of charrettes as a tool for interdisciplinary collaboration between designers and community organizations, it is first necessary to study the preconditions and products associated with their use. The current study aimed to do so by addressing two research questions: Q1. In what areas and to what degree do the community problem solving approaches of designers align with those of community-based organizations in terms of their values, objectives, and methodological approaches? Q2. Are design charrettes including designers and community-based organizations with more aligned community problem solving approaches associated with more effective interdisciplinary collaboration, defined as more integrated products that more effectively incorporate and utilize the disciplinary strengths of both designers (design contribution) and community organizations (community contribution), than those with less aligned approaches? In an attempt to better understand and optimize the use of charrettes, and perhaps build support for the institutionalized expansion of design-community collaboration, these research questions were explored via qualitative inquiry (See Figure 2 for full study model). This study 41 used Sutton & Kemp’s (2006) approaches to problem solving within community design charrettes framework along with the breadth of literature on community and interdisciplinary collaboration to investigate the components of effective interdisciplinary collaboration between designers and community-based organizations. The problem solving framework focuses on Figure 2. Community Design Charrette Study Model the differences in designer and community member approaches based on their values, objectives, and methods, in addition to the advantages and drawbacks of each approach and their respective associated products. The research questions and construct definitions were drawn from this framework in conjunction with theories of and research on interdisciplinary partnerships and 42 community collaboration (e.g. Kegler & Wyatt, 2003; Stokols et al., 2005; Suarez-Balcazar et al., 2006) to investigate the preconditions and products of community design charrettes. The aim was to first assess designers’ and community-based organizations’ approaches to community problem solving based on Sutton & Kemp’s (2006) framework and determine their degree of alignment. Then, guided by studies of similar phenomena in coalitions and interdisciplinary collaboration (e.g., Gray, 2004; Howard & Somerville, 2014; Nowell, 2010; Sutton & Kemp, 2002), the primary purpose was to examine the relationship, or lack there of, between alignment and effective collaboration. 43 CHAPTER 2: METHOD This study utilized a qualitative methodology to explore the research questions (Creswell, 2009; Patton, 2002; Merriam, 2009; Miles, Huberman, & Saldana, 2014). A qualitative approach was chosen over a quantitative or mixed-methods approach based on the community design charrette context under study and the nature of the research questions (Dyson & Genishi, 2005; Kim, 2013; Patton, 2002). Currently, there are no established quantitative instruments that could comprehensively assess all of the proposed research questions in this unique setting. Moreover, at this time, there is still a need for a more holistic understanding of how and why community design charrettes work before embracing more expansive studies of if they work, the latter often being associated with more quantitative approaches (Yin, 2003). For this reason, the naturalistic and emergent nature of qualitative inquiry is most appropriate for an in-depth investigation of approach alignment and integrated products (Creswell, 2007; Merriam, 2009; Kim, 2013; Patton, 2002), which are minimally understood in this unique context. A significant benefit of qualitative approaches is the ability to examine real phenomena as they naturally unfold in their settings with a holistic lens, which provides a “strong potential for revealing complexity” (Miles et al., 2014, p. 30; Stake, 2006; Yin, 2013). Given the complexity of the approaches and products explored throughout the research questions, a qualitative approach was well suited for this study. Further, both community-based and design disciplines often utilize qualitative inquiry due to its descriptive attentiveness to local context, personal experience, individual beliefs, and unique approaches to problems (Condon, 2012; Maton et al., 2006; Miles et al., 2014; Smith, 2012). Much like community practice, “the design process is…necessarily and inherently qualitative, depending on intuition and judgment to select 44 from alternatives” (Condon, 2012, p. 58; Smith, 2012). Also, Patton (2002) describes qualitative inquiry as both science and art, critical and creative, which is not only descriptive of the charrette process, but is also engrained in the fundamental purpose and function of design charrettes and the partnerships they support (Dhar & Khirfan, 2016; Sutton & Kemp, 2006). Finally, qualitative research is often described as being dependent on the researcher’s skills, experiences, and perspectives (Patton, 2002). The researcher is uniquely suited to this study, as she strives to personally balance roles of designer and community researcher, artist and scientist, creative and critic; while professionally exploring, throughout her academic and practical experiences, intersections between the two realms in pursuit of community change. This qualitative study utilized a comparative multiple-case study design with data collected via interviews, researcher observations, and archival documents (Miles et al., 2014; Patton, 2002; Stake, 2006; Yin, 2003). The project sought to help build a more detailed understanding of design-community partnerships, particularly community design charrettes, as vehicles for interdisciplinary collaboration and innovative community change. Specifically, as outlined by the research questions, do the disciplinary approaches of designers and community organizations fundamentally align, and is this alignment or lack thereof associated with the degree to which these partnerships result in effectively integrated products? To assess these questions, four community design charrettes, held over the course of a three-day Design for Good event in 2017, served as the cases under study. Setting Designing for social change, or ‘design for good’, is an effort championed by the largest professional association for graphic design, AIGA (originally the American Institute of Graphic Arts but no longer associated with the full title; AIGA.org). It is described as a movement, or a 45 place for design to evolve, that supports the role of graphic design in social change. As of 2016, an official division of AIGA called Design for Good had several chapters all over the US that aimed to uphold this vision for over a decade. Unfortunately, the vision’s vague nature and a limited understanding of the many roles of graphic design in social change left Design for Good with a fuzzy reach and function in the eyes of the public, which is a reflection of the need inspiring the present study - for a better understanding of how graphic design and communities can connect to inspire innovative change. Design for Good was perhaps one of the most visible and widespread venues for exploring this, yet its efforts were misunderstood and a bit disorganized to the dismay of the leadership. As a result, the organization is currently re-defining and creating new divisions, like Design for Democracy (aimed at addressing community change via the political influence of graphic design), to hopefully offer more directed support to the design for social change movement. AIGA leaders attribute the struggle to define and establish salient efforts primarily to a lack of visible evidence demonstrating either how design- community partnerships have an impact, or that they even produce positive, sustainable impacts in the first place. Design for Good and its fledgling existence is an embodiment of the unfortunate lack of understanding of, and resulting interest in, design-community collaboration. Fortunately, one regional AIGA chapter, the partner for this study, has managed to maintain some legitimacy and traction in the design for good realm. For one, there is a large and very active design community in the area as it is home to a prestigious college for art and design and an internationally renowned art competition. The area is also a hub for design talent in the Midwest. In addition, the regional AIGA chapter has seen continued interest in, and success with, volunteer-spirited efforts like their annual Weekend Blitz event. Weekend Blitz is a three- day design-community partnership event that was created to offer volunteer graphic design 46 services and a collaborative problem solving space to local community organizations in need. Every year, AIGA essentially organizes several community design charrettes for Weekend Blitz, each focused on one local community organization and the challenges it brings to the table. Volunteer graphic designers are recruited to partner with the community organizations via AIGA’s connections to the vast designer networks in the area. For the past six years, Weekend Blitz has hosted up to ten charrettes per year, occurring over the course of three days in a shared collaborative workspace in Michigan. As of 2016, community design charrettes had been organized to serve 31 community organizations with the help of more than 175 designers. In 2017, Weekend Blitz expanded by merging with a similar annual volunteer event called Give Camp. Give Camp utilized the same 3-day, rapid collaboration process as Weekend Blitz but instead of graphic designers, software developers and web programmers partner with community organizations to help build the back-end technology for websites, mobile applications, or databases. The merge resulted in an event called Weekend for Good 2017, which served as the setting for this research. In 2017, Weekend for Good (WFG) was held November 3rd through November 5th and hosted 15 charrettes, which partnered graphic designers and developers with community organizations. Of those 15 charrettes, four included a graphic design team and a community organization working on design-focused visual communication products like logos, branding, and websites. The selection of this setting was the result of partnerships with both the local WFG organizers and AIGA’s national leadership. After many conversations with members of both parties about the struggle for sustainability and understanding the impact of Design for Good, the researcher devoted a great deal of time to participating in and observing Design for Good efforts, including two years at Weekend Blitz and a residency in design for social change under the 47 direction of one of Design for Good’s national founders. The research questions emerged from these experiences, the requests of the partners, and extensive reviews of related literature. WFG 2017 was chosen as the study setting due to AIGA’s expressed desire for assessment and its established partnership with the researcher, as well as the setting’s ability to supply a uniquely large, diverse sample in a manageable setting. A comparative multiple-case study of numerous charrettes addressing different problems in an identical setting provided a unique opportunity to add to current knowledge of design-community collaboration and charrettes. Sample Cases Sampling in a qualitative case study or multiple-case study is dependent on the desired unit of analysis, or the definition of a “case” (Miles et al., 2014; Stake, 2006; Yin, 2003). In the current study, for all four proposed research questions, the unit of analysis is a community design charrette (e.g., to what degree are charrette participants’ problem solving approaches aligned, how supportive is the collaborative context, what did it produce, etc.). Thus, the primary sample of cases included community design charrettes were selected using purposive criterion sampling (Patton, 2002; Palinkas et al., 2015). The study aimed to examine alignment of disciplinary approaches, collaborative contexts, and outcomes within and across design charrettes while holding as many other procedural and contextual charrette factors constant (e.g., duration, organizing entity, physical setting, size). For this reason, cases were chosen if they met two criteria based on a maximum-variation sampling strategy (Patton, 2002; Sandhu et al., 2007): (1) Each case must be a community design charrette hosted by the three-day WFG 2017 event, and (2) the charrette included a design team and a community organization that were partnered together to solve a problem proposed by the organization and to create a design-focused product. The maximum-variation sampling strategy employed in this study is often employed in multiple 48 case studies to produce a sample that will provide the most possible variation for cross-case analysis and subsequent conclusions about the larger group (Miles et al., 2014). The sampling criteria for this study were presented to the organizers of WFG two weeks prior to the event, who then provided an initial list of nine charrettes that featured a design- focused product and would include graphic design teams and a community organization. At the start of the event, these nine charrettes were reviewed by the researcher based on the sampling criteria to make sure that they were in fact proceeding with a design focus and included a graphic design team. Four were deemed ineligible, as the review revealed that they had changed course to focus on development instead of graphic design (i.e., although initially charged with designing a website, they decided to focus on programming an online database instead). Thus, after reviewing these revisions with the event organizer responsible for graphic design projects, five cases were eligible for inclusion in the study. Of these five cases, one community organization did not respond to requests to participate in an interview, thus the final sample included four cases (80% response rate). According to Stake (2006) and Miles et al. (2014), the ideal sample size for a comparative multiple case study is roughly four to ten cases. Thus, this study included a manageable sample size that allowed for both the unique analysis of each case, as well as a more broad and diverse analysis of the phenomena in question (i.e., collaboration in community design charrettes). The four cases spanned a diverse array of both graphic design specializations and community issues. The four design teams consisted of five to eight members, with specializations including web design, print design, branding, illustration, and user- interaction/experience design. One designer was assigned to be the project leader in each charrette. A development advocate was also paired with each design team to answer any 49 questions regarding back-end development, although many teams included designers already somewhat familiar with this. Two of the four community organizations provided one staff member (one organization sent two and one sent four) to participate in the charrette and to report back to the organization throughout the process. Each organization’s work focused on a specific community issue that involved social justice, local narratives, the arts, and/or supporting vulnerable populations (see Table 1 for a summary of the charrettes). Table 1. Charrette Participants and Community Issues Charrette C1 C2 C3 C4 Research Design Organization’s General Focus Cooperative Sustainability Local History Violence Prevention Art and Design Number of Designers Number of Organization Members 6 8 5 8 1 4 2 1 Case study designs, like the comparative multiple-case study design applied here, are popular for studies of interdisciplinary research (e.g., Stokols et al., 2005; Suarez-Balcazar et al., 2006), community-based organizations (e.g., Evans et al., 2015; Goodman, 2009; Vohra, 2014), designer approaches (e.g., Anderson et al., 2010), and design charrettes (e.g., Dhar & Khirfan, 2016). Further, there are calls to do more multiple case studies in all of these collaborative settings (e.g., Howard & Somerville, 2014; Kegler & Wyatt, 2003; Stokols et al., 2005; Suarez- Balcazar et al., 2006). Multiple-case study designs, considered inherently comparative by Stake (2006), are often deemed the most appropriate research designs within these settings for several reasons. 50 First, true multiple case studies focus first, and most scrupulously, on understanding the rich context and complexity of each single case. Comparisons are then made across these unique, detailed findings to make assumptions, but again, with the single case as the phenomenon of interest (Miles et al., 2014; Stake, 2006). Also, because this study sought to test assumptions related to pre-determined research questions, a multiple case study was the most appropriate design. Miles et al. (2014) emphasize that multiple-case studies “offer the researcher an even deeper understanding of the processes and outcomes of cases, the chance to test (not just develop) hypotheses, and a good picture of locally grounded causation” (p. 45). This study did not seek nor necessarily claim the ability to determine causation, but this benefit of a multiple case design was certainly instrumentally aligned with the aims of the project (Stake, 2006). In the same sense, “multiple-case sampling adds confidence to findings, By looking at a range of similar and contrasting cases, we can understand a single case finding, grounding it by specifying how and where and, if possible, why it carries on as it does. We can strengthen the precision, validity, stability, and trustworthiness of the findings. In other words, we are following a replication strategy (Yin, 2009). If a finding holds in one setting and, given its profile, also holds in a comparable setting but does not in a contrasting case, the finding is more robust… Nevertheless, the multiple-case sampling gives us confidence that our emerging theory is generic, because we have seen it work out—and not work out—in predictable ways” (Miles et al., 2014, p. 48). Thus, a multiple-case study design was deemed the best choice for the setting, questions, and goals of this study. Measures. Researchers often suggest that a good case study requires multiple sources of data (Stake, 2006). Multiple data sources increase the accuracy, richness, and trustworthiness of data and the resulting findings (Patton, 2002; Yin, 2003). A primary benefit of case studies is the opportunity for multiple, diverse sources of evidence (Yin, 2003). This study drew data from semi-structured expert interviews, researcher field observations, and archival documents, all 51 common data collection methods in case study designs (Evans et al., 2015; Stake, 2006; Vohra, 2014). The primary source of data was the semi-structured interviews with the WFG design leaders and lead organization representatives from each charrette. This source provided data that represents the direct experiences and perspectives of the designers and organization representatives who were most involved in each charrette. The questions asked about interview respondents’ disciplines, in which they are considered experts, as well as their first-hand accounts of the collaborative process and products of their WFG charrette. Researcher observations served as a secondary source of data used to gain another perspective of the collaborative process in the charrette by allowing trained outsiders to assess the study constructs within each case during WFG. The observations, conducted simultaneously by two trained observers (the researcher and a second observer) throughout the weekend, captured larger group dynamics that the interview respondents may not have been aware of, and data that could be used to help balance response biases like social desirability or tendencies to report overly positive accounts of one’s personal contribution. Archival and procedural documents were collected as a tertiary source to supplement the interviews and observations with an account of the collaborative process and product that was not influenced, for example, by interview questions or observation protocols, which focused solely on the study constructs (e.g., the project briefs were created prior to the event, independent from the research process). Together, the three data sources were used to form a more comprehensive and multi- perspective assessment of the constructs than could be formed from one source alone. Numerous case studies, including those examining interdisciplinary research (e.g., Stokols et al., 2005), community coalitions (e.g., Chaskin, 2001; Kegler & Wyatt, 2003), community-based partnerships (e.g., Andrews et al., 2010) and design charrettes (e.g., Sutton & Kemp, 2006) have 52 utilized all of these sources in concert to strengthen their case study data. The benefits of drawing from multiple sources include a richer picture of the phenomena and context by accessing multiple perspectives, and the opportunity to bolster confidence in the findings (e.g., convergence across sources related to identified relationships strengthens internal validity) via data-source triangulation (Denzin & Lincoln, 2000; Patton, 2002; Stake, 2006; Yin, 2013; Yin, 2003). For this study, each data source described below had an accompanying protocol developed using Sutton & Kemp’s (2006) problem solving framework, elements of established measures in related studies, and relevant empirical and theoretical literature. The protocol designs were further refined via pilot testing. All revisions were thoroughly documented via a developmental audit trail, deemed unnecessary to submit to the MSU Institutional Review Board as they did not alter the exempt status (see Appendix A for exempt decision letter received July 7, 2017), and approved by the researcher’s graduate advisor or dissertation committee via brief memos or meetings when necessary. Semi-structured open-ended interviews. In qualitative multiple-case studies, interviews are perhaps the most common and effective way for gathering rich, detailed information about individuals’ experiences, beliefs, and perceptions of the world around them (Stake, 1995; Yin, 2003). Designing case study interviews typically involves using open-ended, unassuming questions that elicit respondents’ rich explanations that inform the processes and contexts addressed in the research questions (Spradley, 1980; Stake, 1995; Yin, 2003). In a multiple case study, interviews must be structured enough to remain the same across cases and maintain the ability for later comparison of the responses (Vohra, 2014). Thus, this study used a semi- structured interview guide (Patton, 2002; Vohra, 2014) containing open-ended questions, as well 53 as additional probing questions to elicit further details related to the research questions when necessary (Spradley, 1980). The charrette participant interviews were utilized to gather in-depth data representing the perspectives and experiences of both designers and community organizations related to the charrettes. Two interview protocols were used, one for designers and one for community organizations (see Appendix B for protocols). Following the same structure for the designer and organization interviews, the first section of the protocol contained four main questions, with additional probing questions, assessing the disciplinary problem solving approaches of either the design team or the community organization. In this section, designer respondents were asked questions about “as a designer, your” values, objectives, and methods; while the community organization respondents were asked the same questions about “your organization’s” values, objectives, and methods. The language for these questions and the corresponding probes was drawn from both the Sutton & Kemp (2006) problem solving approach framework and from various widely used measures in community collaboration research and practice, from which variable-specific items were adapted when available (Center for the Advancement of Collaborative Strategies in Health, 2006; Goldstein, 1997; Gottleib et al., 1993; Hays et al., 2000; University of Illinois, 1999). The questions also included prompts to orient respondents within the specific time or context under study. Because the problem-solving approach questions in this section were meant to address more long-standing, engrained disciplinary perspectives, these questions were oriented within a more general context than the charrette event (e.g., Can you describe the process for how [your organization/you as a designer] typically…). The final question in section one assessed the interviewees’ perspectives on the alignment between the designers’ and community organization’s problem solving approaches during the charrette; thus, the orientation was shifted 54 via guiding language to the charrette itself (e.g., Now, think about [the organization/designers] on your Weekend for Good team. In what ways, if any, did they share your approach to solving the problem?). Although the approach alignment variable was primarily assessed during analysis, these responses were designed to help corroborate or challenge the initial findings. The language for this question was also derived from measures in seminal studies investigating community collaboratives’ orienting approaches (Borden & Perkins, 1999; Kegler et al., 1998b). The second section of the interview protocol relevant to the current study1, contained questions adapted from published and widely cited survey questions and interview protocols from community coalition and interdisciplinary research, as well as other transdisciplinary, collaboration, and team science measures assessing collaboration and the associated outcomes (e.g., Center for the Advancement of Collaborative Strategies in Health, 2006; Borden & Perkins, 1999; Goldstein, 1997; Goodman et al., 1993; Mansilla, 2006; Misra et al., 2015; Wageman et al., 2005; Kegler et al., 1998b; University of Illinois, 1999; Vogel et al., 2011; Vogel et al., 2014). This section focused on research question two and included three main questions with additional probing questions; one asking the respondent to describe the product, and two asking about the designers’ and the organization’s contributions to the product, respectively. The first question probed for extent to which the product was a collaborative creation, and the following two questions provide both the designers’ and the community organization’s opinion of who contributed and how. These questions were designed to determine the degree of collaboration within the charrette, based on how collaborative the resulting 1 Additional sections of the interview protocols included questions related to the designers’ and organizations’ perceptions of the collaborative climate during WFG and potential resulting outcomes within the organization. These questions were not the focus of the current study but will be examined in subsequent studies to further build an understanding of the processes and positive outcomes associated with interdisciplinary collaboration in community design charrettes. 55 products were deemed. Additional questions were also included in the interview protocols to gather basic supplemental data for both the study and the partners including: previous collaborative experience with the other discipline, years of experience participating in charrettes associated with WFG, the nature of the problem addressed during in the charrette, and reactions to the WFG experience. Researcher observations. In qualitative multiple-case studies, observations are typically used to provide yet another layer of depth to the case profiles by assessing the settings, behaviors, and processes of the cases as they naturally occur. Moreover, in the case of ethnographic work, observation is described as “motivated looking. That is, the local scene…is seen and surveyed as a constellation of organized activities. It is the patterns and patterning which the ethnographer is looking for and not a realistic, behaviouralised description or natural history” (Anderson, 1992, p. 162; Millen, 2000). In the spirit of conducting “motivated looking” observations, an observation protocol was created for the study based on the research questions and the phenomena of interest. Because of the incredibly short window of opportunity, and a relatively large sample of stimuli considering, the collection of unbiased and accurate observations by the researcher presented numerous challenges and required a more structured protocol than typical qualitative observation. To help overcome these challenges, the observation protocol design drew on strategies in the literature related to systematically conducting rapid and useful observations including rapid ethnography (Millen, 2000), and focused observations (Spradley, 1980). Rapid ethnography is utilized most often for researching human-computer interaction and human centered product design (Millen, 2000; Sandhu et al., 2007). The pressures of technological markets changing at an exponential rate create unique circumstances for gathering detailed data, 56 and observing many diverse experiences, quickly enough to keep up with constant innovation (Millen, 2000). Similarly, rapid epidemiologic assessment uses techniques closely related to RE to quickly evaluate many sites across a disaster stricken area based on key indicators of health and safety (Bradt & Drummond, 2002). Strategies to help improve the quality of fast, repetitive, and focused observations in these settings include “limiting or constraining the research focus and scope, using key informants, [and] capturing rich field data by using multiple observers and interactive observation techniques” (Millen, 2000, p. 280). Following the suggestions in the literature, the initial observation protocols were designed to keep the researcher strictly focused on the variables under study (Millen, 2000; Spradley, 1980). Thus, during pilot testing, observations were recorded on worksheet-style protocols as this format is suggested in the literature to help focus the observations on the relevant questions and allow for an isolated observation of each new case by guiding the researcher through each one based on its unique association with the criteria in the worksheet (Bradt & Drummond, 2002; Bentley et al., 1998). Unfortunately, these worksheets proved to be unfeasible and ineffective during piloting, given their inability to capture important details of observed activities. Thus, the final observation protocols (see Appendix B Figure 12) were semi-structured, containing simple worksheet-style questions as well as ample space for real-time jottings and fieldnotes. In order to maintain alignment with rapid assessment/ethnographic principles, the protocol included a small set of questions that could be answered very quickly (e.g., check boxes, fill in a time, etc.). This section allowed for key constructs to be addressed in the same way across all of the charrette observations and insured that the observers consistently attended to these constructs. The second section of each protocol was used for jotting, and later fieldnotes, 57 as outlined by Emerson, Fritz, & Shaw (1995). The observers were given space to recorded key phrases and words to freely denote important events or interactions. Archival documents. If accessible and relevant, archival documents are also often used in qualitative case studies to help corroborate and support the findings from other data sources (e.g., Kegler & Wyatt, 2003; Stokols et al., 2005; Sutton & Kemp, 2006). These documents can sometimes extend beyond the perceptions of participants and researchers, offering yet another unique data source (Stake, 1995). Because WFG presented three sources of archived data that were consistently available across all charrettes (initial project briefs, group notes and schematics, and the products themselves), these materials were intended to supplement participant accounts and observations by offering documentation of the processes and products of each charrette. Thus, the archival documents were utilized to further detail specific aspects of problem solving approaches (values, objectives, methods) and effective collaboration (designer contribution, community organization contribution) The inclusion of these materials was also intended to help corroborate the assignment of codes or themes drawn from the other data sources during analysis. The first source of archival data was each charrette’s initial project brief, which provided insight into the values, objectives, and methods of the designers and the organization. Project briefs are commonly used in design to outline details of a forthcoming project including objectives, plans, or relevant client information. The WFG organizers created a project brief for each charrette and provided it to the design leaders and the organization prior to the weekend. Each brief outlined the organization’s mission, the kind of work they do, and their needs or goals for the project, as well as the design team member’s skills, methods, and areas of expertise. The second archival source was a collection of process notes from each charrette, which primarily 58 provided insight on effective collaboration. Throughout the weekend, each charrette was provided whiteboards, wall space, and sticky notes to help with planning and documenting their processes. Thus, these spaces emerged as a primary venue for communication, collaboration, and design or organizational contributions. The final source of archival data was each charrette’s final product. Each charrette under study created a new website for the organization, which documented all of the products ultimately created by the group (e.g., logos, branding, the website itself). Reviewing the websites provided data on the contributions from both the designers and the organization (e.g., were the pictures used provided by the organization or all stock photos chosen by designers?). Of course, the extent to which each source targeted each of the intended variables was dependent upon the content and nature of each charrette’s materials (see Figure 3 for data collection timeline). Figure 3. Data Collection Timeline Data Collection Pilot testing. As recommended by both Yin (2003) and Stake (1995), piloting was utilized to both test the measures and to refine the content and procedures outlined for data collection. Each data collection method was tested following the initially proposed procedures, with particular focus dedicated to further developing the research design and gaining insight 59 from charrette participants and experts. For example, in order for case study interviews to be successful, testing the specific questions and protocols was required (Stake, 1995). Yet in addition, cognitive pretesting and exploration of respondents’ reactions to the language and interview experience is also essential for eliciting the richest data (Woolley et al., 2006; Yin, 2003). Thus, the interview protocol was tested formally with one designer and one community organization leader from a previous community design charrette, but also via cognitive interviewing procedures (see Appendix C for pilot protocols and example pilot recruitment script) with a different designer and organization representative (e.g., asking if pilot respondents understood certain words or how they interpreted specific questions). Following the interview pilots, minor shifts in language and question flow resulted in the final protocols described above and the procedure outlined below. In kind, pilot observations and document retrieval were conducted according to the proposed study procedures, with additional insight into their content and execution gained via discussions with the pilot charrette participants and leaders. Following the evolutionary nature of naturalistic inquiry, pilot testing was treated primarily as a formative exercise to clarify and refine the research design (Yin, 2003). Thus, piloting was conducted during the three months leading up to WFG data collection. The cases under study for the piloting of interview and archival measures included one organization from Weekend Blitz 2016, as well as one organization and two graphic designers from 2016’s IMPACT! Design for Social Impact: Strategies for Community Engagement residency at the School of Visual Arts in New York City. The four participants were members of three different charrettes (the IMPACT! organization and one IMPACT! designer were members of the same charrette). IMPACT! charrettes are nearly identical to WFG charrettes in objective, structure, and group size. The major differences, length and setting, only added to the case’s appeal as an 60 appropriate pilot case, as it was selected based on several criteria recommended in the literature including extended duration and complexity, as well as accessibility and support for research development (Yin, 2003). For example, because IMPACT! charrettes are held in a much larger and more diverse setting, they tend to be more complex than WFG charrettes. This presented more opportunities to address unforeseen challenges and to develop adaptive and comprehensive measures (Yin, 2003). Next, piloting measures with IMPACT! offered extensive accessibility to the researcher, given her prior participation in IMPACT! and continued work with the organizers. Finally, the IMPACT! organizers are affiliated with academic institutions, including The School of Visual Arts and Parsons School of Design, and national design entities focused on social change (one being a founder of AIGA Design for Good at the national level), which made them strong supporters and contributors to this research from the start. The observational protocol was piloted in two different settings. First, the observation protocol pilot testing included one community charrette in Northern Michigan ran by the interim director of The National Charrette Institute at Michigan State University. The one-day charrette event provided an ideal setting for adapting the protocol to the fast-paced charrette environment and identifying the best formatting for capturing the variables under study. The second observation pilot involved all of the charrettes to be observed at WFG, and was conducted during an orientation and planning event one week prior to WFG. This allowed for further testing of the protocol, while also providing an opportunity to test key elements of the procedure like timing, case selection, and capturing unique observations in succession. All piloting partners including the IMPACT! participants, AIGA leadership, and The National Charrette Institute contributed to a climate of learning and research development. It is important to be responsive to the qualitative development process in order to execute case studies 61 using the most authentic and successful approaches (Yin, 2003). Thus, the procedures outlined below were the product of an adaptive process (involving collaboration with the partner and the pilot testing) wherein all adjustments were thoroughly documented via developmental audit trails. Study procedures. All data collection for the primary study occurred during a time frame beginning with the first day of WFG on November 3 2017, and ending on December 9 2017, five weeks after the charrettes took place. The majority of observation and archival data collection occurred during the Friday (November 3, 2017) through Sunday (November 5, 2017) when the charrettes were held, and the interview data was collected between November 9 and December 9, 2017. Each data collection approach followed carefully planned procedures both during and following WFG. These procedures are outlined below. Interviews. Case studies often require that the researcher outline strategies for sampling on multiple levels (i.e., sampling the cases and then sampling who or what will be observed within them and when; Kim, 2013; Merriam, 1998; Miles et al., 2014; Stake, 2006). Thus, a secondary sample must be drawn to select individual respondents from each case for the interviews. A slightly different sampling strategy, purposive elite sampling, was used to identify the respondents capable of providing the most rich and representative information. Elite sampling (although not always specifically addressed by this name) is used in organizational research, for example, to identify individuals who are most knowledgeable and able to best address questions at the organization level (Palinkas et al., 2015; Spath et al., 2008). For this study, the purpose of the interviews was to obtain a self-description of the problem solving approach of both the designers and the community organizations from the respective parties. In addition, the interviews were intended to collect accounts of the perceived collaborative 62 processes and products of each of the charrettes. For this reason, respondents had to be the most involved members of both the design team and community organization throughout the charrette, as well as have adequate disciplinary experience to represent their group accurately. Thus, the lead designer and the lead community organization representative from each charrette were recruited during WFG to participate in an interview following the event (i.e., two participant interviews per case). Three days after WFG, these representatives were sent an email describing the interview process and requesting to schedule an in person interview (see Appendix D). In this email, individuals were also offered compensation for their time in the form of a $25 Visa giftcard if they chose to participate, which would be dispersed in person following the interview. Of the ten individuals (from five charrettes) approached for an interview, nine responded, and eight interviews were scheduled following the exclusion of one charrette due to non-response from the organization (response rate = 80%). All eight interviews were conducted within the five weeks following WFG, each lasting roughly 60 minutes, and were recorded with permission. Each interview occurred either at the organization’s office or at a local coffee shop based on the participant’s request. Following the protocols, each interview began with the researcher reading the informed consent (see Appendix E) and receiving verbal consent for both participation and audio recording. Upon consent, the audio recorder was started and the interview commenced. In addition to the recording, the researcher took brief written notes on the protocol. Notes were minimal and served only to guide the conversation, as the focus remained primarily on the respondent. Following each interview, the participant was thanked and given a giftcard. Immediately after the interview, the researcher spent 30 to 60 minutes reflecting, revisiting her notes, and elaborating on her written cues, as is suggested in the literature (Merriam, 2009; Stake, 2005). 63 Researcher observations. Although researcher observations required a semi-structured protocol in this study, they were subject to the most flexible and emergent process during planning. As a result, this procedure required the most refinement following the each of the two pilots and discussions with the partner, which offered opportunities to test and develop the following procedure. First, in order to systematically acquire the most accurate and comprehensive picture of each of the charrettes within the incredibly short timeframe, secondary sampling involved deciding what charrette to observe, when, and for how long (Patton, 2002; Stake, 2006). After the selection of the participating charrettes was confirmed Friday evening, a tentative observation schedule was created for the rest of the weekend. For Saturday (i.e., the only full day of the event), each charrette was randomly assigned to a one-hour timeslot to be observed simultaneously by both observers. As initially proposed, if a charrette was not active during their observation time, or if a significant event was happening within another charrette, the observers moved to the next charrette on the schedule and returned during the following timeslot. There were only two instances when a charrette was not active during its scheduled observation time (e.g., members of the charrette were using that time for silent individual work or the members were absent from their workspace). Those conditions were recorded in the observation log and the observers moved on to the next charrette on the schedule. The layout of the workspace featured a central common area, surrounded by open workspaces and rooms with glass walls within which the charrettes worked. This allowed the observers to be mindful of all activity and to follow applicable action at any point if necessary. Yet, no significant events occurred which caused the observers to deviate from the schedule. After each one-hour observation, 30 minutes was allotted for the observers to read over their fieldnotes, elaborate on cues, and debrief. During debriefing, each observer explained the 64 overarching themes that emerged in his or her notes from the one-hour session. Each of these themes was discussed and a final set of agreed upon themes was recorded to document the shared narrative that was confirmed by both observers. After debriefing, all notes and materials from that session were filed away and the observers started labeling fresh observation protocols to help clear their minds and orient themselves to the next charrette. The observers could not return to or view any completed protocols once they were filed away until the weekend was complete. This minimized the likelihood of observation data being contaminated or muddled with other cases’ data. The observers also read through the protocol in full before each new observation and filled in the charrette number and starting time to help orient themselves to a fresh setting and the full set of variables under study. After finishing all the observations on Saturday, the observers debriefed and purposively sampled a smaller set of charrettes (three, one of which was later dropped from the study due to lack of response for an interview) to observe again on Sunday before the final presentations. The chosen charrettes were those deemed the most likely to have the organization on site with the designers and actively working Sunday morning. This strategy, described as “theory-based” sampling by Palinkas et al. (2015), allowed for a deeper focus on manifestations of targeted constructs (collaboration and collaborative context) by identifying and observing charrettes that were actively working together. This strategy also utilized intensity sampling, as it selected only the cases with the most concentrated probable display of the constructs under study (Palinkas et al., 2015). Thus, three charrettes were randomly assigned to another one-hour timeslot to be observed on Sunday. During this time, brief notes were also be taken on the charrettes that were working less collaboratively but the main focus of observation on Sunday was the three selected cases, which were engaging in the most observable interactions between the designers and the 65 organization. At the end of the event on Sunday afternoon, all charrettes presented to the larger group one at a time. Thus, all cases were again observed as they presented their final products. Document retrieval. All archival documents were accessed and collected with the consent of the WFG organizers. First, the WFG organizers provided each charrette with a project brief, outlining the goals for the project and information about the organization and the participating designers. All project briefs were provided to the researcher by the WFG organizers on the first day of the event. The second archival source was pictures of each charrette’s collaborative workspace. Every charrette was provided a whiteboard and wall space to display plans, ideas, or processes. At the end of each day, the researcher took pictures of each charrette’s workspace, which provided information on the content and quantity of their collaborative work. The final archival source was each organization’s website. The researcher visited the websites immediately after the event and five weeks later. At each time point, the researcher recorded whether or not the organizations had implemented the new sites that were created during WFG. Screenshots were also taken of the sites to offer a visual supplement to the interview respondents’ descriptions of contributions from the designers and the organizations. 66 CHAPTER 3: ANALYSIS Data Preparation Immediately following WFG, the researcher observation logs and corresponding fieldnotes were electronically transcribed by either the researcher or a trained undergraduate assistant. The interview recordings were transcribed after all interviews were completed by a secure transcription service. Notes taken during the interviews and any physical archival documents that could not be retrieved digitally were scanned or photographed. A digital file was created for each case that included its observation logs, fieldnotes, interview transcripts, and archival materials (e.g., projects briefs, whiteboard photos, website screenshots). Analytic Approach The analytic plan for this study was aligned with Miles et al.’s (2014) approach to cross- case analysis, moving “from one inductive inference to another by selectively collecting data, comparing and contrasting this material in the quest for patterns or regularities, seeking out more data to support or qualify these emerging clusters, and then gradually drawing inferences from the links between other new data segments and the cumulative set of conceptualizations” (p. 29). In short, analysis began with a unique case orientation using inductive analysis and creative synthesis (Patton, 2002; Stake, 2006), followed by cross-case comparisons for patterns related to the research questions (Miles et al., 2014). This approach, while less emergent and exploratory than analytic induction (Hammersley & Cooper, 2012), allows for the analysis to be guided by a framework or research questions established a priori, as well as being conducive to the manageable comparison of data from up to ten cases. In related research, Miles et al.’s (2014) cross-case analytic approach is considered the gold standard and is widely applied (e.g., 67 Anderson et al., 2010; Goodman, 2009; Kegler & Wyatt, 2003; Vohra, 2014). Thus, both within and cross-case data analysis occurred via the three iterative activities outlined in their approach: 1) data condensation, 2) data display, and 3) conclusion drawing/verification. Within-case analysis. Data condensation began by organizing each case’s interview responses by variable. In most cases, this was relatively straightforward, as the interview questions focused on each variable distinctly. Even so, respondents did sometimes address multiple variables of interest in response to a single question (e.g., while discussing objectives, they also touched on their methods or values). In other instances, the key variable of interest in a question may not have been addressed by respondents in their initial answers (e.g., before being probed to shift the focus to their methods, a respondent returned to the previous question topic to elaborate on their objectives when asked to describe their methods). Thus, a system was used to code each interview response for the variable(s) addressed. For each interview, the transcription was broken into segments (i.e., each question or interviewer remark was one segment, each response was one segment, with responses punctuated by pauses or shifts in conversation broken into two or more segments). Transcriptions of the observation fieldnotes were also added to the coding spreadsheet in segments, as well as the whiteboard photos, relevant sections from the project brief (i.e., organization overview, mission, objectives, audience, how will this project help, design lead skills and areas of expertise, and WFG organizer notes), and screenshots of the final products. Based on a codebook (Appendix F Table 2) containing the original variable definitions and example responses, each data segment was independently coded by both the principal researcher and a trained undergraduate assistant (see Appendix G Figure 20 for example coding spreadsheet). The codebook was designed based on an adaptation of a widely sourced rubric developed by The Center for Prevention Research & Development at the 68 University of Illinois for evaluating community coalition effectiveness (Mid-South Prevention Department, 2010; University of Illinois, 1999). It included definitions of each variable, derived from the literature, and corresponding check boxes to categorize each segment from the interview data, observation fieldnotes, and archival documents. After each case was coded, both coders met to compare their codes and resolve any disagreements. These discussions also contributed to ongoing revisions to and clarifications of the codebook definitions. From there, each data segment that addressed a specific variable was moved to the appropriate column in the theming spreadsheet (see Appendix G Figure 21 for an example of a case’s theming spreadsheet). The theming spreadsheets served as the initial data display in the analysis process, allowing for within-case comparisons that led to further data condensation. In each case’s spreadsheet, using the subcoding strategy outlined by Miles et al. (2014) subcodes were derived from each data source for each variable in each case. These subcodes were then compared across all data sources to begin to draw within-case conclusions by summarizing charrette-level themes for each variable. Cross-case analysis. The next iteration of the comparative case analysis process involved data condensation of the variable themes for all cases, data display in a single matrix displaying all cases by variable, and conclusion drawing, which involved searching for patterns or relationships across all compared cases and assigning levels for each variable (e.g., low, moderate, or high value alignment) relative to the other three cases (Chaskin, 2001; Kegler & Wyatt, 2003; Miles et al., 2014; Vohra, 2014; see Appendix G Figure 22 for example cross-case analysis excerpt). Although member checking with the partner was planned at this stage in order to verify the conclusions to the best of their ability and share their feedback, it was not possible due to turnover in the event staff as well as changes within Design for Good and AIGA. As an 69 alternative, the second independent data coder from the research team was brought back in to review the themes against the original data, after which they debriefed with the PI on the levels assigned to each variable within each case. As Miles et al. (2014) note, the analysis process is iterative; thus, throughout codebook development, data condensation, display, and conclusion drawing, verification was employed after each step using multiple-coders, secondary verification via multiple data sources, and adaptive analysis tools. It is important to note that although this analysis plan was relatively rigid for qualitative analysis, it was meant to serve only as a tentative guide; the inductive nature of the coding and iterative analysis allowed for emergent findings while still maintaining alignment with the specific research questions (Miles et al., 2014). All decisions and steps throughout this process were logged with a corresponding rationale in an analysis audit trail (see Appendix H for an outline of the final within and cross-case analysis steps). Validity and Trustworthiness of Data and Analysis Within qualitative studies, there is a need to establish systems to assure validity and trustworthiness of the data and findings given the heavy involvement of the researcher’s judgment in the design, collection, and analysis processes (Lincoln & Guba, 1985). For this study, several steps were taken to increase validity and trustworthiness throughout data collection and analysis. For instance, the use of multiple cases is itself an approach to increasing the robustness of findings via case corroboration or disconfirmation, and applying the logic of replication to overcome unfounded generalizations (Vohra, 2014; Yin, 1984). In other words, with four different cases, the researcher was able to seek multiple sources of support for the findings and assure those findings hold true across multiple unique charrettes. The use of multiple data collection methods also helped to assure agreement across various scenarios or 70 perspectives (Evans et al., 2015; Stokols et al., 2005). Further, all data collection procedures and corresponding decisions were documented via an audit trail. An audit trail document was also kept for the analysis steps and corresponding decisions made by the researcher. This allowed for complete transparency of the researcher’s internal decision-making processes during collection and analysis. In addition to providing transparency, these audit trails will allow for replication, or the ability for another researcher to carry out the study in the exact same way and know explicitly how all procedures were conducted. Finally, strategies including using multiple coders and establishing research team consensus were used to ensure the findings are the product of careful, systematic analysis and not simply the judgment of the principle investigator (Chaskin, 2001). For example, as is illustrated by the methods of Kegler and Wyatt (2003), two members of the research team independently coded the raw data; after which, discrepancies were noted and resolved to establish a valid coding scheme. In sum, Patton (2002) suggests that validity and trustworthiness of qualitative findings can be increased via multiple perspectives, multiple data source corroboration, and multiple investigators. Thus, each of these elements was carefully incorporated into this study. 71 CHAPTER 4: RESULTS Research Question 1 Q1: In what areas and to what degree do the community problem solving approaches of designers align with those of community-based organizations in terms of their objectives, values, and methodological approaches? The results for question one are presented below for each case, beginning with a summary of the charrette’s problem solving approach alignment and a classification of the level of alignment (low, moderate, or high) relative to the other cases. Next, within-case comparisons of the charrette’s designers and organization are outlined for each of the three variables (objectives, values, and methodological approaches), detailing the charrette-level themes and supporting data points that contributed to the classification of low, moderate, or high problem solving approach alignment. Charrette 1 problem solving approach alignment: low. Overall, charrette 1 (C1) displayed the lowest alignment between the organization and the design team when it came to their respective approaches to problem solving. Despite sharing some fundamental values, which they were both particularly passionate about, the designers and the organization had other distinctly different values, as well as conflicting objectives and methods. C1 value alignment: moderate. When asked directly, both the design lead and the organization thought their values were highly aligned. In their interview, for example, the organization representative expressed their belief that “value alignment was especially present within our team”, and “aligned values helped us collaborate.” The organization representative also described that the design team embraced the organization’s mission and expressed desires to 72 get involved or make monetary donations after WFG. Likewise, the design lead noted that the organization and the designers shared “socialist, down with whatever” ideals. The observation data also supported that the designers and the organization shared laid-back, progressive personalities and attitudes. However, both the interview and observation data also suggested that some values that the designer and organization perceived as shared were actually defined or carried out differently by the two parties. For example, although both parties valued understanding their user or audience, the organization wanted to understand the user in order to communicate, reach out, and form connections while the design lead wanted to understand the “user” in order to appeal to as many individuals as possible and make the organization - their client - competitive. Similarly, the organization and designers also differed in how they interpreted the values of collaboration and cooperative work. While the organization expressed in their interview that the designers shared these values, the design lead defined collaboration as intermittently getting “input” from a client, whereas the organization’s definition was much more involved and requires “connecting through joint work.” This divergence was also evident during observations, as the designers worked alone for the majority of the weekend and asked for feedback when needed, and the organization representative seemed eager, yet unable, to introduce ideas and work through solutions with the designers. The most apparent, and ultimately detrimental, lack of aligned values involved the organization’s strong ties to tradition against the designers’ drive for innovative ideas that would make them stand out. It was clear, in the interviews, observations, and archival data, that the organization strongly values its history and maintaining tradition. In the interview, the design lead expressed that they “tried to align” with these values, but the observations and the review of the final product suggested that the designers did not make history and tradition a priority. For example, during observations the 73 organization could be heard talking proudly about the symbolism in their logo and describing how they had recently resurrected a 100 year old newsletter and planned to digitize the font to use moving forward. At the same time, the design lead worked throughout the entire weekend trying to create a new, more elaborate logo for the organization, hoping to make them look more current. In the end, the final product that the organization requested, a new website, did not include any logo at all and displayed very little visual connection to their history. C1 objective alignment: low. Like many of the participating charrettes, the organization and designers in C1 entered into WFG with different objectives in their day-to-day work, and this resulted in conflicting objectives for the project. The lead designer, for example, stated that one of their core objectives in their work was trying to “meet all the client’s expectations, even if they are unrealistic.” The organization, on the other hand, described far more lofty day-to-day goals in their application for WFG like illustrating, communicating, and advancing their mission, or “[reaching] full attendance for its events.” While these objectives may not seem to be in direct contest with one another, when the designer aims to please, and the organization aims to move mountains, things can become unmanageable. Such was the case during WFG, when the organization tried to achieve its goal of creating a “collaborative design.” The organization was constantly throwing out new ideas and grand ambitions for improving their work. While they were simply trying to “spitball” and engage the designers in a collaborative process, the designers took much of it literally and the list of things they felt they had to accomplish over the weekend became infeasible. The organization’s approach also conflicted with the design lead’s objective to “define a clear and comprehensive solution” before starting work on a project. Many of the organization’s lofty and ambiguous goals did not present clear-cut solutions. The design lead described this disconnect as something that caused issues even before the weekend began, 74 with the organization’s objectives described in the design brief, like communicating the mission or increasing engagement, being initially interpreted through the disciplinary lenses of designers, who presumed that meant rebranding the organization and creating a new logo. This was not what the organization had in mind but the “designers very much leaned toward brand stuff, because it’s their area of expertise.” So while the organization later explained in the interview that they actually needed better user interaction design on their website to manage information and allow people to engage online, the designers tended to this request but stuck to what they knew and primarily aimed to make the organization look more “competitive” and “increase their value in the eyes of the users.” C1 methods alignment: low. It was clear across the data sources that the designers and organization from C1 had different ways of getting work done. Much like their differing definitions of collaboration as something they both value, the two parties also operationalized the process of collaborative work differently. The design lead described preferring to work alone, and to show the client the progress for feedback intermittently. They explained that they find it difficult to do design work with someone looking on or offering suggestions as one works. The other members of the design team displayed similar tendencies during observations and C1 spent the majority of the weekend quietly working independently. The organization claimed in the interview that there was a “spirit of collaboration” and they “shared a cooperative decision- making process,” but when asked to describe what the collaboration looked like, they noted that “the collaborative work was actually more talking and fleshing out ideas and less hands on work” for the organization. Upon further reflection, the organization representative concluded that they were not involved at all in the actual building of the website, and had not even seen the site when the weekend was over. The organization representative appeared visibly conflicted 75 during the observations, often looking on at the designers and trying to spark conversations as they worked with little success. They later confirmed this observation saying they were torn between wanting the designers to have their freedom and wanting to be a part of the process. The organization also noted that the designers initially told them it was not necessary to be present during the weekend, and that they could come see the final product at the end. This seemed incredibly odd to the organization and they ultimately decided to be there each day, but also noted that even though they wanted a collaborative process, the designers did not anticipate that. Thus, the organization decided to “trust the designers expertise and [go] along with what they said.” Charrette 2 problem solving approach alignment: low-moderate. Overall, the problem solving approaches of the designers and the organization in charrette 2 (C2) were more aligned than in C1, resulting in low to moderate alignment across the three contributing variables. Similar to C1, C2 displayed moderate alignment on values. Specifically, while the designers and organizations shared some fundamental values, other values seemed to work in opposition. The same was true for C2’s minimally aligned objectives. However, because the organization in this charrette was actually a coalition made up of several community organizations, their methods for solving problems were not necessarily typical of their counterparts at WFG and resulted in moderate alignment with the design team. C2 value alignment: moderate. The alignment between the designers’ and the organization’s values in C2 was deemed moderate due to evidence across the data sources that supported some highly aligned and some minimally aligned values. Speaking to the former, both the interviews and the observations highlighted that the designers’ and organization held a shared value of communicating an accurate understanding of prevention and the nuances associated 76 with addressing the organization’s cause with a trauma-informed approach. In the interview with the organization lead, they noted that all of the designers in their charrette identified as members of populations affected by the issues the organization addressed. The organization also discussed how the design lead “really seemed to get it” and clearly did their homework by researching what prevention efforts typically look like and how various types of coalitions promote it. This was also evident in the observations, as the design lead was seen diligently searching for and discussing statistics, research, and news articles that helped to illustrate the current social conversation on the issues. In terms of day-to-day problem solving however, the design lead and organization described two different core values that motivated their current work. When asked what they believed was the most important thing to consider when solving problems, the design lead answered simply, “the end user.” They went on to describe the value of simplicity and functionality, and that above all else their designs need to “actually do something.” The design lead reiterated this point several times, ultimately saying that a designer can create something that looks cool or is visually appreciated by audiences but, “if [the] user needs to access something very quickly and [it is] too convoluted, we've failed.” When asked the same question, the organization’s primary concern was more about their own image and projecting legitimacy, importance, and familiarity. The organization lead explained that they are “all social workers who want to be doing things in the community, but can’t if people, [legislators, and businesses] don’t know…and respect who you are.” Of course, considering that they are a coalition, this organization does not actually provide services like they do within their own individual human service agencies. So, as a coalition, their focus shifts from engaging a typical “user” to communicating a universal message to the community that promotes awareness of and respect for anyone working in service to their cause. These values were echoed in the project brief, 77 where the organization listed their mission statement as: “eliminating racism; empowering women and girls; promoting peace, justice, freedom, and dignity for all.” Thus, the values of the designers and the organization were very minimally aligned in this area, as establishing simplicity or defining functionality when it comes to issues like “eliminating misconceptions about violence prevention” proved to be much less straightforward than the designers were accustomed to. Another point of contrast that contributed to overall low alignment, was the designers’ and organization’s respective attitudes toward design and marketing. The design lead emphasized a strong desire to work only with clients who were “super competent,” invested, and excited to continue actively working on marketing and engagement after the design project is finished. The organization on the other hand, explained that “when it comes to marketing or development…people working in social justice don't really want to do that, they want to go do the work.” They also noted that the coalition only meets once per month and when it comes down to selecting the few projects they are able to focus on, marketing and development are not a priority. C2 objective alignment: low. The designers and organization in C2 displayed the lowest alignment across the three variables when it came to their respective objectives. For the design lead, one of the most prominent themes that emerged throughout the interview was an objective to define and measure quantifiable benchmarks for success. For a website, that might include the number of page views or a site’s ranking on search engines. While these are certainly valid objectives in web development, the design lead ended up paired with the organization that seemed to be the least interested in counting clicks or becoming highly visible on search engines. In fact, since the coalition does not provide direct services related to the issues they address, and individuals searching for services may be in immediate need or high-risk situations, they 78 struggled with the idea of appearing as a top result when someone performs a search on the topic. The organization’s most salient objectives were less about numbers and more about creating some sort of space in the community for the coalition, which had no physical location, public events, or even its own staff. They ultimately decided that a website could serve as a hub to establish a legitimate, respectable identity and to provide information and education on the issues they address, similar efforts in the community, and their purpose as a coalition. Needless to say, objectives like educating the community and shifting attitudes toward policy are not easily defined or quantifiable, and the design lead believed that “just going from no site to [having] a site” is not a good benchmark for success. Another point of misalignment that proved challenging for C2 involved the design lead’s objective in their day-to-day practice to “make people think ‘oh man, I want to check out [a client or service]’”. During the observations, for example, there were multiple instances when the designers pushed to include mainstream news articles on highly-politicized and sensitive topics (e.g., violence prevention) that the coalition addressed. Despite visible hesitation from the organization, the design lead continuously stressed the need to “add to the larger conversation” and market the coalition. Unfortunately, this typical approach to increasing interest and web traffic did not align with the subject matter on the organization’s website. After WFG, the design lead reflected on this challenge and attributed it to the fact that “no one wants to be the face [of this issue]” and no one wants to seek it out like they might do with a product or service. Thus, the designers struggled by maintaining their typical marketing objectives because they did not “know how to make people want something that they don’t even know they want.” C2 methods alignment: moderate. Some of the methods of the designers and organization in C2 were misaligned for similar reasons as their objectives. Yet, there were also 79 unique circumstances in both groups that resulted in some of their respective methods being highly aligned. Like their objectives, some of the designers’ methods were informed by marketing principles, which proved to be more difficult to apply in the context of this organization. For example, the design lead explained that in an effort to help clients market and promote themselves, they always ask “who is your competition?” When they asked the organization at WFG this question, the answer was much different than the designers were accustomed to: “Our competition is the fact that nobody really wants to do articles on us. Nobody wants to talk about this as being important. Our competition is universities who don't want to change, it's hospitals that don't want to change, it's all these people that are resistant to changing the conversation.” However, despite having to navigate these challenges related to marketing, C2 was perhaps the most efficient charrette because the designers’ and organization’s work styles lined up in a way that allowed them to get an immense amount of work done in a very short period of time. First, the design lead explained that the other designers on the team had no experience working with clients because they typically had a project manager acting as the go-between. Thus, they were accustomed to working alone, without noise or interruptions. This was certainly evident in the observations of C2, as the designers were often scattered across the building throughout the day, usually wearing headphones, and sometimes even tucked away alone behind closed doors. Unlike other charrettes however, this ended up working well for C2 because, as a coalition, members of the organization typically work alone as well, with only one meeting per month to connect on every member’s individual tasks. The organization lead explained that their process involves a facilitator leading the group in brainstorming ideas, dividing up action steps, and then everyone taking “their little step back to their organization to complete it between 80 meetings.” This is almost exactly how C2 operated throughout the weekend, with the design lead serving as the facilitator. While the other designers lacked experience with clients, the project brief listed the design lead’s skillset as: “mainly project [managing] a group of freelancers to design and develop Wordpress sites.” This was certainly clear during the observations, as the design lead was the only one seen interacting with the organization, or writing plans and strategizing on the whiteboards. Another interesting theme from the observations involved the design lead’s choice to almost exclusively use “I” statements when discussing the project (e.g., “Maybe I’ll also have statistics here” or “What I want to say here is…”). The organization also noted in their interview that they “worked with design lead great, but it almost felt more like brainstorming with her, than sometimes the whole team.” Again, for this charrette though, these circumstances (e.g., individual work coordinated by a facilitator) were all fairly familiar to both the designers and the organization, as they were much more aligned when it came to this aspect of their problem solving approaches. Charrette 3 problem solving approach alignment: moderate-high. Relative to the other charrettes, the designers’ and the organization’s respective approaches to solving problems were moderately to highly aligned in charrette 3 (C3). Assessing alignment in this charrette proved to be considerably more complex than in the other cases. Specifically, the organization was in a state of transition and had a major objective of changing their approach to solving problems as part of their experience at WFG. Further, while there were some clear differences between the designers’ and organization’s approaches (e.g., independent and streamlined work styles vs. very collaborative and vocal), C3 presented a variation of alignment with approaches that complemented one another like interlocking puzzle pieces (e.g., one provides a constant 81 stream of information and one listens intently and uses that information to inform their work), as opposed to being identical. C3 value alignment: moderate. Although no explicitly similar values were identified, it was clear from both the interviews and the observations that the designers’ and organization’s priorities complemented one another when it came to working together. In their interview, the design lead repeatedly emphasized the importance of asking questions and vigilantly seeking clarity until you are confident that you accurately understand what a client really wants. They explained that in their work as a designer, clients often do not know what they want in terms of a physical design. While a client may have a sense of how they want something to look or sound, a designer could put in a great deal of work up front and deliver just that, only to later learn that particular look or sound does not actually function in service of the client’s real goal (e.g., attracting new members or modernizing their image). Thus, the design lead believed that is a designer’s job to work with the client to ensure that above all else, the designs or content they create accurately captures what the client actually needs the product to accomplish. This commitment to understanding the client and patience for establishing clarity up front aligned well with the detail-oriented organization whose primary values involved accuracy and “being involved in the creation of everything [they] put out” so things get done just as they want them done. As an organization focused on historical research, the members had an exacting attention to details because historical accuracy and telling others’ stories authentically was incredibly important to them. These values expressed by the organization’s leader were evident throughout the observations as well, with one example being an instance when a member of the organization became flustered because a picture the designers chose did not depict a historical figure how she “actually would have looked”. The designers however, appreciated that type of feedback and 82 took time to listen to the organization’s rationales – all of which amounted to an unanticipated crash course in local history that the designers ultimately used to inform their work throughout the weekend. The organization leader, as well as the project brief, also highlighted the importance of presenting themselves in a way that inspired other communities to do similar work, motivated local women to get involved in their efforts, and championed the inclusion of their subjects in the conversation about the area’s history. In kind, the design lead felt that the most important thing to consider in any project is an understanding of who the project is targeting and building personas to systematically assess how to meet the goals for each audience. Thus, this value for identifying and speaking to each target audience was also complementary to the organization’s dedication to reaching other communities, getting local women involved, and influencing the local conversation. C3 objective alignment: moderate. Much like C1, the designers and organization in C3 assessed the problem through different disciplinary lenses before they even began working together; and thus, approached the project with different objectives. The organization saw WFG as an opportunity to address their longstanding goal of updating their website. Thus, they entered into the project with a detailed and directive list of objectives, summarized in the project brief as “[updating and expanding] upon our current website, …fully [redesigning] it, and [moving] it to a platform that is accessible, offers analytics, is compatible with social media, and is easy to use and update.” However, the designers, who noted that a primary objective in their work was getting to the root of what the client “really wants”, took their cues from more nuanced commentary in the project brief like, “ultimately, having a website that serves as a portal of women’s history for our community and connects our research and presentations with our social media efforts will go a long way in expanding our audience.” From this, the designers derived 83 their primary objective of improving the organization’s brand, which included creating a more consistent image across their communications (e.g., to connect research, presentation, and social media) and modernizing their overall image (e.g., to appeal to a younger audience). Both the design lead and the organization lead acknowledged that this was in no way a part of the organization’s plan, nor was it in the brief for the project. The organization also said that they wished they had known what the designers were going to do beforehand, and how it ultimately tied into what they wanted to achieve, so they could have been more actively involved in and prepared for the branding and logo updates. Compared to C1 and C2, C3 was different in that alignment did occur related to more fundamental objectives that the organization had been working toward in their day-to-day practice. In their interview, the organization lead outlined goals to become more open to change and include more members in the work, both of which ended up becoming driving themes throughout C3’s project. Prior to WFG, the new board president (organization lead) was struggling to build a desire to try new things and a willingness to make change within the organization. When the designers presented the unexpected branding changes, the organization lead quickly recognized that, despite their personal hesitations, it was an opportunity to facilitate a group effort that embraced rapid change and built confidence in a new direction for the organization. This, coupled with the designers’ commitment to maintaining that what the client wants comes first, allowed for both of their objectives to be pursued concurrently and culminate in products that addressed similar problems (e.g., users interacting with a difficult to use site and getting users engaged with the information) in different ways (e.g., the website and the branding). 84 C3 methods alignment: high. The design lead and organization lead both described two completely opposite ways of getting work done that actually served as a great example of the kind of methodological alignment where distinct activities complement each another like interlocking puzzle pieces. The design team was self-described and observed as quiet, independent problem solvers who process solutions internally and press on steadily toward checkpoints when an intermediary then shares their work with the client and gathers feedback. In their day-to-day practice, the design lead has no reason to establish methods for interacting with a client in real time because, while they do exchange and merge information or ideas, it is the role of a designated broker to systematically facilitate collaboration. The organization lead, on the other hand, explained that while they have no formal processes in place for problem solving and strategizing, it typically involves recurring lengthy discussions and they struggle to actually put things into action. When the organization does move forward with a project or task they either take it on themselves or prefer to be incredibly involved every step of the way. When collaborating, the members ask a lot of questions, like to talk through things as they happen, and are not afraid to interject with honest input in order to get an outcome that reflects their goals and values. Getting these two approaches to run smoothly side-by-side took some effort, and there were points that felt awkward or difficult to navigate for both parties throughout WFG. For example, the design lead’s plan was to have the organization lead come in and meet with the designers an hour each day to give feedback and check in, and then return to see the final presentation of the product. Despite initially agreeing to that plan, both the design and organization leads laughed in their interviews as they explained that their organization was the first to arrive at the event the next morning, with four members in tow (the most of any organization), and was among the last to leave each night. Both sides proceeded through the 85 weekend sticking to their typical work styles and, although it felt awkward and unbalanced at times, it became clear during the analysis that their overall method did align: the designer takes in information and visualizes solutions, and the organization provides information, reviews, and approves solutions. Essentially, to their surprise, the designers’ internal and steadfast processes became even more streamlined, accurate, and informed with the client quite literally over their shoulders providing real-time critiques and answering questions as soon as they arose. On the other end, the organization seemed to thrive when their process of endless discussion was not reciprocated. The designers did not engage in the discussion in a way that allowed the organization to get stuck ruminating on different options and contrasting hypotheticals. By intently listening and continuing to work, the designers could test options or incorporate feedback on the spot, allowing the organization to see which ideas worked, make quick decisions, and refine more details - to the extent that they left with a product that actually addressed their problem more comprehensively than their initial ideal. Charrette 4 problem solving approach alignment: high. Of the four charrettes, charrette 4 (C4) displayed the highest alignment between the designers’ and organization’s problem solving approaches. What the organization described in their interview as “perfect alignment” could be attributed to the fact that they are an arts-based organization, but their training in design-thinking and previous experience working with graphic designers also contributed to the high alignment across all three variables. C4 value alignment: high. Given their joint focus on the arts, the organization and design team in C4 aligned on fundamental values like creativity and expression. The organization’s mission statement, outlined in C4’s project brief, included the core value of “[building] creative community,” and this was supported throughout their interview as well. Even in relation to 86 designers specifically, the organization lead emphasized that “we're an art and design organization so the value of design is very obvious to us, we both appreciate and champion design work. That's just part of our mission.” Because of this, the organization lead went on to say that they are always happy to welcome opportunities to collaborate with independent designers. Additionally, this respect and appreciation was reciprocated by the designers in C4, as they were familiar with the organization’s work prior to WFG. In their interview, the design lead described the organization as one that “definitely [already had] a lot of visual sense” with a “really strong visual brand” known throughout the community. Before the project even began, the designers were confident that the organization “[knows] who they are, and we [didn’t] want to do anything to change that.” In kind, the organization strongly valued maintaining their long- standing image, and noted that it was “reaffirming” and felt good to know the designers came in with equal respect for their visual brand. This alignment was further validated with the review of the final products, as C4 was the only charrette to not make a single change to the organization’s visual identity. It made sense then that, outside of visuals and design, the designers’ and organization’s values were ultimately highly aligned because their most prominent shared value actually involved working efficiently to produce practical solutions. The design lead was succinct and to the point in naming “information” as the most important consideration when problem solving. More specifically, you have to gather information on the problem up front and know “who your user is, the right context, and why they're using whatever you're trying to design for them” in order to make the best use of time. The design lead explained that this information is critical for efficient progress because if the time is not taken up front to define and redefine the problem until it is correct, designers can end up dedicating huge amounts of time to work that will 87 ultimately have to be redone. In the same sense, the organization leader made several mentions of their value for efficiency since they have limited resources and only a few staff members. They make it a priority to make the best possible use of their time and money by ensuring that solutions are practical and useful in their everyday work. C4 objective alignment: high. Similar to their values, C4 was the only charrette in the study with highly aligned objectives between the designers and the organization. The vast majority of all the data gathered on both the designers’ and organization’s objectives centered around one key theme: the user. According to the design lead, the greatest strength of their charrette was the fact that “everyone was approaching [the project] with a user-centric lens; the organization had the same user focus.” One of the most important objectives the design lead described in their day-to-day practice was “designing for the right user.” This was incredibly instrumental in the success of C4 because the organization wanted to drastically change how their website operated, but only for one specific user group: their staff. Essentially, a former employee created the original website and the back-end technology was far too complicated for anyone else to update or edit. So, the organization’s primary goal was to simplify the user experience for themselves without changing much for front-end users who visit the site. The designers were so dedicated to the same goal that instead of fixing the original site and teaching the current staff how to navigate their workarounds for old problems, they built an entirely new site. This new site looked and functioned exactly the same on the front-end, but used technology on the back-end that was so simple and intuitive, even future staff would be able to jump in and make updates with little or no training. This broader attention to a more universal back-end user stemmed from the designers’ objective to design sustainable solutions that account for future 88 users – which also aligned with the organization’s desire for “something that could be passed on easier than the old site given high turnover” within their staff. C4 methods alignment: high. The designers and organization in C4 displayed the highest alignment when it came to their methods. The organization credited their various aligned methods to a shared understanding of the “design thinking rubric,” explaining that “we're an arts and design organization and all of [the designers] are trained in design thinking. We really approached it from a design thinking perspective and framework, and that worked really well.” The organization lead also noted that they had worked with designers before so, perhaps more so than the other organizations, they knew to expect long processes with “different iterations and feedback and testing.” True to these expectations and design-thinking strategies, the design lead discussed, and was observed facilitating, iterative reassessments of the problem, prototyping, and feedback loops. The walls and the whiteboards in C4’s workspace were covered in sticky notes and process maps that resembled typical design-thinking tools, and the designers and organization actively contributed to these visual strategizing methods equally throughout the weekend. Also, in line with the designers’ values and goals related to the user, the design lead described some of their typical methods for user experience design in their interview including conducting user interviews and creating personas. Somewhat to the designers’ surprise, the organization was also familiar with these methods and described the process of creating personas together during WFG as “user experience 101; everyone was on board with that and it worked really well.” Research Question 2 Q2: Are design charrettes including designers and community-based organizations with more aligned community problem solving approaches associated with more effective 89 interdisciplinary collaboration, defined as more integrated products that more effectively incorporate and utilize the disciplinary strengths of both designers (design contribution) and community organizations (community contribution), than those with less aligned approaches? The results for question two are presented similarly to question one, first with a summary of the degree to which the charrette’s products reflect the contribution of both parties and a classification of the level of integration (low, moderate, or high) relative to the other cases. Next, within-case comparisons of the charrette’s organization contributions and designer contributions are outlined, detailing the charrette-level themes and supporting data points that contributed to the classification of low, moderate, or high integration. A cross-case summary is then presented to review themes and associations observed across all of the charrettes in the study. Charrette 1 integrated product: low. In C1, no final products were completed or delivered to the organization by the time data collection concluded (approximately 1.5 months after the WFG event). However, the interviews, observations, and review of partially completed products including branding materials and a website layout suggested minimal integration in the materials that were created, with designers contributing almost entirely to these materials and the organization contributing little. C1 organization contribution: low. When asked who they believed contributed most to their charrette’s final product, the organization lead responded: “I would say we probably had the most input. [The designers] did what I was asking them to do.” This response, however, was given in the weeks following WFG when the organization was still waiting to receive and had not seen the products on which the designers had been working. In fact, still unbeknownst to the organization at that time, the designers were able to do very little of what they had originally asked. Namely, the organization’s primary request was for a new website with the ability to book 90 appointments, record visitors’ needs, and essentially automate much of their information management that was becoming too tedious to do by hand. While the organization did contribute many ideas and specifications related to these tools, the design lead ultimately revealed in their interview that none of them could be created. Due to what both parties described as miscommunications, as well as mismatched skills and objectives for the project, the website effort resulted in a framework for a Wordpress site, meaning a basic organizational scheme for the site layout and navigation, but lacking most of the content. While the organization did contribute to the organization of the sitemap via an interview about their typical users, the design lead noted that the organization spent most of their time working with another designer on content, the “most collaborative part” of the project, which ultimately did not make it into the unfinished website. The design lead went on to sum up the collaborative effort similarly to what was observed by the researchers; “the organization was there to act as a consultant, we presented solutions and [they] just okayed them.” Even with the branding and logo design, both parties acknowledged the organization’s contribution of an expertise related to their history and aesthetic that only they would have, but little of that came through in the products. For example, the logo created by the designers did not include what the organization described as the most important visual component “[representing] the international symbol for cooperation,” and they ultimately never used it. So while the organization did initially make the statement about contributing more than the designers, they ultimately later reflected on the fact that they had not yet seen any of the components they were hoping to get from WFG noting that the “collaboration was a lot of talking and fleshing out ideas but maybe not as hands on…[we] weren’t really involved in the actual building that much.” 91 C1 designer contribution: high. The designers contributed to the materials made during WFG considerably more than the organization. According to both interviews, the designers did basic user research, mapped the user flow through the website, created a wireframe and navigation, and organized two of the organization’s existing sites by merging them together. Despite all of this resulting in an incomplete framework for a website, the designer and organization both described it as an important “design solution” delivered by the designers, that the organization would later fill in. Although, a review of the organization’s websites six months after WFG showed that the two sites had not been integrated and none of the designers’ solutions had been implemented. The designers also made significant contributions to another thing the organization ultimately would not use: new branding. Despite not wanting to update their branding, the organization revealed that they “just let [the designers] do it because it was part of [their] skillset and what [they] wanted [their] role to be” in the project. Thus the designers went through processes to figure out the organization’s competitors, establish their visual identity (e.g., modern vs. traditional), and identify new imagery that thematically represented the organization. The design lead then went on to make two new logos, one for the organization and one for the annual festival they host. The design lead also handled planning, delegated tasks, and even spent additional time after the weekend trying to figure out if the specifications the organization had asked for were even possible in an online information management system, concluding that nothing similar even exists. Ultimately, the design lead noted that things would have been very different had the organization been making the all the decisions, and that “[they] trusted our expertise, so [they] went with what we said most of the time.” 92 Figure 4. Charrette 1 Case Summary Charrette 2 integrated product: moderate. By the end of the weekend, C2 had produced a Wordpress website and all its content; a brand identity with a logo, color scheme, photographs, and social media materials; and a host of accompanying communication tools including a Facebook page and email account. Not only did they complete all of these products, but they were also able to make them all live by the end of WFG, and C2 was the only charrette in the study that had to start completely from scratch. While C2 was perhaps the most productive, their products only moderately integrated the contributions of both the designers and 93 the organization relative to the other charrettes, as the design lead alone made the majority of the decisions. C2 organization contribution: moderate. Like the charrette as a whole, the organization in C2 was steadily productive throughout the weekend. This was the only charrette where there was not an existing website or branding elements to draw from, so the organization spent the majority of the weekend writing all of the content for the various pages that would be on their new site. While the design lead cited this as the biggest hurdle for the project and wished the content would have been prepared prior to the weekend, they also noted that the process of the organization “just dumping content on every page” while the designers built out the structure, actually allowed the organization to have some say in the layout and flow of the website. Another major contribution by the organization, observed across all of the data sources, was the education provided to the designers on the nuances of their work, the intentions of the website, and the important considerations required to communicate such topics. The design lead noted that this was incredibly helpful and their “collaboration really brought up different types of confusion or problems that we wouldn’t have thought about had we been making it without their input.” The organization also worked in close proximity to the design lead throughout the weekend and actively answered questions and granted approval on design decisions “on the spot.” Similar to C1, the organization believed they had more input on the final products because it was their idea to create the website and branding, and it would not have looked the same without their directions, approvals, and the content they created. C2 designer contribution: high. While the design lead enthusiastically acknowledged that the products reflected the contributions of both the designers and the organization, when asked if one had more input they replied frankly, “it was mostly me.” And while the organization 94 did make key contributions, it was clear that the design lead in C2 was driving the project while the other charrette participants did the legwork. As a whole, the designers designed and built a website, set up an email account, a members only portal, a Facebook page, and search engine optimization for the organization, while also teaching them the basics of posting to social media, making a blog post, sharing links from other sites, crediting sources for their content, and making edits in Wordpress. The designers also took on branding the organization and had to recruit an additional member to help with a logo while the design lead chose fonts, colors, and corresponding design elements. In line with the observations, the design lead made the majority of the design and planning decisions on the project and, although noting that they did seek approval for most things, the organization’s feedback throughout the process was little more than “oh, we love it. It’s all great…great, great, great.” The design lead also ultimately decided not to place any decisions on the other members of the design team because they were “just [going to have] to divide and conquer to get [the work] done…so it wasn’t like I was going to have somebody design a layout. I did the layout, I chose the colors.” The design lead felt that delegating open-ended tasks would create “stumbling blocks” and cost them time. Beyond the physical products, the organization also noted that the design lead specifically, expanded their idea of what visibility and engagement meant, offered an outside perspective that helped them understand how the community views the coalition, and walked them through basic communication strategies like how to make a blog successful and setting up timed reminders to regularly post on social media. While the website was implemented and remained live at the close of the study, reviews of the blog and social media related products suggested that the additional tools outside of the website were not yet being used by the organization. Overall, the review of the final products and archival data supported that there was moderate integration with 95 a heavy influence from the design lead, as the final website was essentially a replication of the lead’s previous website builds, with the content and visual elements tailored to the organization. Figure 5. Charrette 2 Case Summary Charrette 3 integrated product: moderate-high. C3 also produced products that moderately integrated the input of the designers and the organization, although slightly more so than C2. The products included a new website that made it easier to navigate the vast amount of research that the organization provides online, as well as and update to their branding that “took what they had and made it more informed and modern.” 96 C3 organization contribution: moderate. As noted in the description of their problem solving approach, the organization in C3 was incredibly particular and vocal, which led to a level of contribution above and beyond what the designers expected. Again, this organization brought the most members (four) to participate in WFG and this helped them contribute in several ways. First, the organization did not bring along the large group for numbers alone. Given their diverse membership and a large variation in age and time in organization, the organization lead wanted to assemble representative group that could provide a voice for all members. The distinct perspectives were noted during the observations, for example, with one member offering consistent feedback on the organization’s past and historical accuracy, and another speaking primarily to their future directions. Also, because the designers were mostly working alone on unique tasks throughout the weekend, the four members were able to engage in more aspects of the work than other charrettes. Both the design lead and the organization explained that it was ultimately helpful to have the members “just keeping an eye on things” throughout the project because it created many opportunities for them to ask questions, learn how things were done, make suggestions, and communicate who they are and what they do in a way that made the it exciting to the designers. According to the organization lead, this was also instrumental in creating a product that even the less tech-savvy members of the organization were able to engage with, because they were able to comment on their capacities and contribute to making sure the products me their specific needs. Beyond he live feedback, the organization in C3 also came into WFG with web content ready for the new site, and a logo, colors, images, and a strong sense of the ideals behind their brand and why they needed to be preserved. In turn, the designers relied on heavily on the visual elements and “made it more modern and clean, and gave them something that they can be more consistent with.” 97 C3 designer contribution: high. The organization in C3 was observed being much more engaged in the designers’ work, so unlike C1 and C2, they may have been able to better account for the contributions of the designers that could otherwise go unseen. Thus, also unlike C1 and C2, the organization lead felt that “the designers probably had more input in the end, but we’re totally happy with it!” Specifically, the designers produced a new Wordpress site that was ready to go live by the end of the weekend and a handbook for editing the site. Within the site, they reorganized muddled content from the old site, updated the navigation, and made the user flow more simple and appealing. The designers also updated the organization’s branding by making a cleaner version of their logo that could be resized and used throughout their communication materials, a set of images sized for different social media platforms and their blog, branded email templates for the email service the organization was already using, and a brand guide including color codes, for example, that could be used to keep their color scheme consistent across everything produced in the future. According to the organization lead, one of the most important contributions the designers made during the weekend was the fact that they were able to leverage their expertise and visualize possible changes in a way that inspired members to expand their idea of communication strategies to include branding. They explained in the interview that: “For our organization, that’s a huge win, and for me it was like, ‘Yay! We got a got a website, but also, yay! We got this project done in a way where everyone came together to make changes.’ I think that, to me, was a big outcome.” The design lead felt that the contributions from the designers and the organization were “pretty equal” and that there was a good balance between the modernity that a designer would want and maintaining what the organization found important. Overall, the design lead thought that the designers “pushed the organization a little bit, to be open to change and updates, but they accepted it and just made sure it wasn’t too drastically different.” 98 Figure 6. Charrette 3 Case Summary Charrette 4 integrated product: high. The final products from C4 displayed the most equitable integration of the designers’ and organization’s contributions across the four cases. The main product in this charrette was a new website, which was actually a visual “replication of [the organization’s] old site with updates and enhancements” to the user experience design. C4 organization contribution: high. When asked if the final products reflected the contribution of both the designers’ and organization’s knowledge, skills, and expertise, the organization replied “yes, in all ways.” Similar to C3, the organization in C4 was described and observed as particular, direct, and “always willing to jump in and say ‘that doesn’t work for us.’” 99 A notable difference is that this organization was more familiar with the design process and, more specifically, the capacities of graphic designers. In their interview, the organization explained that they felt “fortunate to have had the experience of being a client in a website build before,” and credited that for the fact that they “knew going in what the steps were going to be and what various special skills were going to be brought to it.” This allowed them to enter into the project with a more informed plan for working with the designers and ultimately “guide how the problem would be solved.” Before WFG even began, the organization sat down with the design lead with a very systematic approach to outlining a strategy for the weekend: “[We] were able to say, here is the problem, here are the parts of the problem that we are not able to solve, here is the outcome we would like. Then, let’s talk about how that is actually going to happen. Who is going to do what? What are the stages going to be?” This set the tone for the weekend as well, where the organization was observed actively working with designers more than any other charrette and acting as the “final decision maker.” Beyond planning and strategizing, the organization was also able to contribute a great deal of substantive materials to the new website. First, they wanted it to look as visually similar to their current website as possible and include the same content. Again, this organization brought a strong visual sense and confidence in their brand to the table, providing the designers with a formal brand standards guide outlining the use of colors, fonts, and visual assets required for the site. The goals they presented were more user-focused, which resonated well with the designers, and included fixing a host of technological issues and making the site easier to navigate, organize, edit, and pass on to future staff. Finally, the design lead believed the most helpful input from the organization was when they “would say ‘that’s great but my boss wouldn’t want this,’ or ‘we have to use this type of technology instead of what you’re suggesting.” The design lead described these as “business and tech constraints…blah, blah, blah,” that they would not have known 100 enough about to incorporate into the design on their own, but were critical for making a site the organization could actually use. C4 designer contribution: high. The designers also made significant contributions to the product, despite their impact being “less visually obvious,” according to the design lead. First, the organization’s initial plan involved fixing and updating their current website, which was highly customized by a former employee. While the designers could have done this, they realized that fixing broken code would not make the site any easier for the organization to manage in the future because the underlying technology would still be too complicated for them to fix, edit, or add to on their own. The designers “helped them to understand why their site was so complicated, and to arrive at the solution to make a new one.” In doing so, the designers demonstrated the importance of keeping things simple because “the person implementing it won’t be the one updating it.” The organization noted that this was an important lesson for them given high staff and volunteer turnover. It also helped them “understand the limits of how custom and fancy” they can make things if they want the technology to be sustainable, and in a more global sense how to “work in a less innovative space, but a more stable space.” From there, the organization lead and the design lead both credited the designers for the “heavy lifting” and “grunt work” that was required not only to build a website, but also to replicate an existing site using completely different, but more simple, technology. The design lead was adamant throughout the project that they “make it as close to what they were used to as possible,” and if certain elements could not be the same, they needed to be easier. The designers in C4 were also the most engaged with their organization. The observations noted that it was sometimes hard to tell who was who since the whole charrette worked so well as a unit. The designers involved the organization in every aspect of the user experience design, from reorganizing content to mapping 101 user paths and labeling navigation, which both parties acknowledged could not have been done without one another. Overall, the design lead believed that the both contributed to the project equally, and that “the whole thing was really collaborative.” Figure 7. Charrette 4 Case Summary Cross-Case Summary Across the four cases, the alignment of the designers’ and organization’s problem solving approaches ranged from low (C1) to high (C4), with two charrettes exhibiting slightly varying degrees of moderate alignment (C2 and C3; see Figure 8). C1, which had the lowest alignment, created products that were deemed to have the lowest integration of both organization and 102 designer contributions relative to the other charrettes. Similarly, the moderately aligned charrettes, C2 and C3, created moderately and moderately to highly integrated products respectively. Lastly, C4 displayed high alignment across their values, objectives, and methods, and ended the weekend with products that were highly representative of both the designers’ and the organization’s contributions. Thus, these trends suggested that the degree to which the two groups’ problem solving approaches align upon entering into a collaborative project was positively related to the degree of interdisciplinary collaboration, as evidence by the integration of both disciplinary perspectives in the resulting products. Figure 8. Cross-Case Summary 103 CHAPTER 5: DISCUSSION The first research question in this study focused on the alignment between graphic designers’ and community organizations’ problem solving approaches. Prior literature in this area has attempted to define a standard problem solving approach for each discipline when exploring collaborative efforts between them (Sutton & Kemp, 2006). However, the analysis of the cross-case matrix in this study revealed that, contrary to the literature, the respective problem solving approaches of the designers and organizations were not standard across the four cases and alignment between the designers and organization in each charrette differed based on variations in these approaches. While the designers did have more common themes across their approaches (e.g., branding, innovation, visual mapping and planning, user experience design, or working primarily alone) than the organizations (e.g., avoiding radical changes or prioritizing their mission over their image), both exhibited within-discipline variation in their approaches to solving problems. Further cross-case analysis of the charrette level themes that contributed to each case’s alignment identified common factors that appeared to be associated with differences in alignment. In other words, the more descriptive findings for research question one were related to the ways in which alignment varied. For example, across the four charrettes, the degree of methods alignment tended to be related to the amount of experience each partner had working with the other discipline in the past. On the one hand, the organization and designers in C4 had the most alignment between their methods. The organization in C4, not only had experience working with designers on similar projects, but also regularly partnered with designers to promote their work via the organization’s platform. Likewise, the designers in C4 developed 104 their skills in similar settings and had long been professionally associated with similar organizations. On the other hand, the organization and designers in C1 had the least alignment between their methods. The organization had never had a designer involved in their work and the designers in C1 exclusively worked for corporate clients or small businesses, and had little experience working collaboratively with these clients. Prior experience working with the other discipline also emerged as themes in C2 and C3 and was associated with their methodological alignment. Another example involves trends across the charrettes’ themes related to objective alignment, which varied the most across the four charrettes. One factor that appeared to be strongly associated with objective alignment was differing definitions of the problem within each charrette, or more broadly, differences in how each discipline framed the role of graphic design in the context of community organizations (e.g., competitive branding versus creating a more approachable or aesthetically pleasing website). Again, the organization and designers in C4 defined the problem through a distinctly similar “user-centric” lens and both approached visual communications from the perspective of the users. Across the other three charrettes, issues emerged because the organizations and designers defined problems differently. Specifically, the organizations often defined their communication problems in terms of information organization or appearing more legitimate, while the designers relied heavily on competitive branding or user experience to frame the problem. Ultimately, issues with problem definition were associated with aligned objectives across all of the cases. Given that this study also suggested that higher alignment was related to more integrated, collaborative products (see research question two), these findings suggest that there may be certain capacities, which could be leveraged to strengthen alignment; and subsequently, interdisciplinary collaboration. 105 The second research question in this study asked if problem solving approach alignment was related to the degree to which both disciplines equally contributed to the product of the collaborative effort. A positive association between alignment and collaboration was observed across the four charrettes, where higher alignment was associated with higher collaboration. Further, those with more collaborative products had more experience with or knowledge of the other discipline and tended to define the problem in more similar terms than those who created less collaborative products. This association, and the malleable nature of these capacities in particular (e.g., there is more potential to increase each discipline’s experience working with or knowledge of the other than to shift their values), speaks to the benefit of interdisciplinary training and the expansion of interdisciplinary practice with an integrated understanding of visual communication design and community change work. Limitations While these findings can be used to inform future directions in research and practice, there are important limitations to consider in the interpretation and application of these results. First, given limited time and resources, the approach for the current study was to sample relatively short timeslots and select certain cases to be observed in more depth. Because the charrettes were observed at different points in the collaborative process and only for one to two hours each, the depth and breadth of the observation data was limited, meaning the results were primarily informed by the perspectives of the participants themselves as reported in their interviews. Future research in similar settings could benefit from more extensive observations. For instance, in this study, ideally two independent observers would have been present in each charrette for the entire weekend in order to holistically capture an outside account of variables like organization and designer contributions. Alternatively, participant observers could be 106 assigned in each charrette and given basic structured protocols to record key indicators or general observations. Another aspect of the study affected by time constraints involved the short timeframe for data collection and the timing of the data collection relative to WFG. For example, all interview data had to be collected in a single interview after the charrette had occurred, and as soon as possible so the participant could recall the event as accurately as possible. Based on best practices, questions related to problem solving approach were worded to orient the participant to speak on their values, objectives, or methods outside of or prior to WFG, but it may still be possible that aspects they described were a result of the collaboration during WFG and not their independent approach entering into WFG. Although this did not appear to be the case based on the interview responses, alignment may have been inflated in some cases due the recency of working directly with the other discipline. Given that the results to question one suggest that alignment may increase with exposure to the other discipline, an important next step could be to assess problem solving approaches before and after design charrettes to see if these events may be able to be used specifically to build interdisciplinary approaches to problem solving. Finally, unforeseen changes to AIGA, Design for Good, and WFG, coupled with a lengthy analysis by a small research team, led to the inability to employ member checking or participatory approaches during analysis. In the spirit of participatory design and interdisciplinary research, AIGA leadership helped guide and build the theoretical model used for the study and WFG organizers informed the research design and data collection methods. These perspectives contributed significantly to the quality of this research, and while their contributions were kept in mind during analysis, including these stakeholders as active participants in the analysis process would have been ideal. Overall, the principal researcher went 107 to great lengths to consider these limitations and adjust for potential biases (e.g., time and case sampling during observations or scenario-oriented interview questions). Even so, acknowledging their implications presents opportunities to incorporate more accommodating approaches into subsequent research designs. Further, addressing these limitations with more comprehensive or inclusive methods would allow for exploration outside of the variables presented in this model, and the consideration of additional factors stressed in the literature like collaborative context and organizational change. Further Research on Collaborative Context to Benefit Current Practice Despite the identification of possible leverage points for increasing alignment between the problem solving approaches of designers and community organizations, it is important to remember that disciplinary approaches are often difficult to change and embedded as effective practices within designers’ and community organizations’ individual work. Further, activating those leverage points via interdisciplinary training or extensive experience working with the other discipline, for example, are potential solutions that require significant time and effort. Thus, this study was limited because it did not address how designers and community organizations can start to effectively collaborate within their current contexts given the possibility that their approaches may not align. Literature discussing interdisciplinary research, coalitions, design charrettes, and participatory research suggests that a future direction may be to expand beyond a focus on the more engrained, independent characteristics of the individual disciplines to one that also accounts for processes and relational dynamics during collaboration (i.e., the collaborative context; Gray, 1985; Gray, 2004; Lee et al., 2009; Lakhani et al., 2012; Wallerstein et al., 2008). Collaborative context is distinct from individual, disciplinary problem solving approaches. 108 Specifically, individual problem solving approaches are unique to each discipline’s long- standing work style outside of the collaboration, meaning they can be observed, and their alignment assessed, before (or independent of) the act of collaborating. In contrast, the collaborative context refers specifically to the joint work environment and is to be assessed during the collaboration. In other words, different disciplines may enter into partnerships with contrasting approaches that make it difficult to collaborate (Gray, 2004; Hall et al., 2012; Stokols et al., 2008; Suarez-Balcazar et al., 2006), but there are specific relational factors within the collaborative context that may establish a climate capable of mitigating that difficulty while maintaining the contributions and integrity of the unique disciplines (Lakhani et al., 2012; LeDantec, 2010; McDonnel, 2009; Stempfle & Badke-Schaub, 2002; Sutton & Kemp, 2006; Wallerstein et al., 2008). Specifically, according to collaboration literature, the key relational factors necessary to establish a supportive collaborative context include: positive attitudes toward collaboration, supportive leadership, and communication (Butt et al., 2008; Granner & Sharpe, 2004; Kegler et al., 1998; Lakhani et al., 2012; Spath et al., 2008; Suarez-Balcazar et al., 2006; Zakocs & Edwards, 2006). Data was collected on each of these variables during WFG for future analysis, and preliminary findings suggest that collaborative context supported more integrated products. For example, in C3 even though the designers’ standard methods did not involve working collaboratively, the organization and the observations both noted their patience and positive attitudes toward accepting and implementing the organization’s input. Further C4’s design lead was observed engaging the designers and the organization with more equal frequency than the other charrettes, and they were described as a leader who mirrored the organization’s “flat and collaborative leadership style.” Finally, C3 and C4 both exhibited and spoke to the frequent and bi-directional communication between the designers and the 109 organization via various mediums throughout WFG. Despite these early observations however, the marked importance of the collaborative processes detailed here highlights a potential benefit of a research design with more substantial observation data. For the current study, it remains to be seen if the interview and archival data will provide enough corroborating evidence to draw conclusions related to these context-specific processes that would be best assessed with more in- depth observations. There are currently few studies that explore the relationship between individual disciplinary characteristics, particularly approaches to problem solving, and effective collaboration while comparing the influence of collaborative context across multiple partnerships (Allen, 2005; Gray, 1985). Relational factors like context are infrequently studied as dynamic, procedural components of collaboration (Lasker & Weiss, 2003) and the primary focus of research in this area is “oriented toward individual actors at the expense of larger system dynamics” (Gray, 1985, p. 913). Especially in design, there is ample room for increasing the understanding of collaborative potential in charrettes by embracing collaborative context as a boundary-spanning, malleable, and leveraging force within a diverse system of distinct disciplines. Thus, it would be valuable to explore whether a supportive collaborative context can promote effective collaboration within community design charrettes. In other words, using data from the current study or further observational research in a similar setting, one could ask if design charrettes with more supportive collaborative contexts, including positive attitudes toward collaboration, supportive leadership, and communication, are also associated with more effective interdisciplinary collaboration than those with less supportive collaborative climates. 110 Understanding Outcomes to Promote Design-Community Collaboration Even if a charrette is able to bring together designers and a community organization with aligned problem solving approaches, and effectively collaborate to produce an integrated interdisciplinary product, that is only the beginning. Charrettes are short by nature and when they end, the resulting tool or solution is in the hands of the implementer (e.g. the community organization in the present context). Unfortunately, in the case of both designers and community organizations, there is currently not an interdisciplinary infrastructure that supports the time and funding necessary to work together on a continual basis (Gibson & Owens, 2014; Sutton & Kemp, 2006). Because of this, the charrette must produce more than an integrated product; the collaborative process must equip the organization with the tools to understand and use the product effectively. Changes within the organization represent intermediate outcomes that can be studied to help assess the greater impact of the collaboration within the charrette. In the broader collaborative literature, these intermediate outcomes of collaboration are considered key indicators of both positive impact as well as more distal outcomes like community change and continued interdisciplinary efforts (Mattesich & Monsey, 1997; Masse et al., 2008; Sutton & Kemp, 2002). In the case study of the Chicago Food Systems Collaborative, Suarez-Balcazar and colleagues (2006) used a modified version of the three elements of Senge and Sharmer’s (2001) “knowledge generating system” to classify and analyze the intermediate outcomes of interdisciplinary collaboration. According to this system, and supported by research drawn from various bodies of collaboration literature, effectively integrated collaborative knowledge or products may result in positive organizational outcomes including: increased understanding of the problem, increased capacity for action, and increased likelihood of implementation or related shifts in practice (Hacker et al., 2012; Masse et al., 2008; McLaughlin, 111 2013; Rottle & Johnson, 2007; Suarez-Balcazar et al., 2006; Sutton & Kemp, 2002; Sutton & Kemp, 2006; Stokols et al., 2008; Valencia-Sandoval et al., 2009; Zakocs & Edwards, 2006; Zhang et al., 2015). Research suggests that intermediate outcomes like capacity and implementation are crucial to assessing collaborative processes and predicting future impacts (Mattessich & Monsey, 1997; Masse et al., 2008; Roussos & Fawcett, 2000). More specifically, effective interdisciplinary collaboration promotes increased understanding of the problem and context, increased product capacity, and changes in practice including product implementation and further innovation. Additional data gathered during WFG suggests this may be true, yet nuanced as in the case with problem solving approaches. For example, increased capacity within the organizations was positively related to the degree of collaboration. The interviews and observations suggested that the organization in C4 left WFG with the highest capacity to use, edit, and maintain their website, while the organization in C1, having never seen their website or how it was being built, stated there was virtually no change in their skills or capacities as a result of WFG. However, whether or not any skills gained by the organizations were put to use, or if changes were made to their practices as a result of WFG could be better assessed using a longitudinal approach with one or more follow-up interviews with the organizations to assess implementation and sustainability outside of the immediate window of the event and without the help of the designers. A longitudinal study of organizational outcomes could also provide more insight into increases in understanding of the problem, as one could observe how both the organizations and the designers approach problem definition in a future design-community partnership. Because the ultimate intended impact of a charrette product will likely take an unidentifiable length of time to come to fruition (e.g. community level shifts like awareness or 112 improvements in health), additional research is needed on more intermediate outcomes like knowledge, capacity, and implementation in order to understand their use for problem solving in communities. Thus, the qualitative data gathered during WFG, or additional longitudinal research, can help researchers and practitioners better understand if and how effective interdisciplinary collaboration in a community design charrette is associated with better intermediate outcomes. More specifically, does more effective interdisciplinary collaboration in community design charrettes, defined as highly integrated products influenced by the contributions of both designers and organizations, promote more positive interdisciplinary outcomes within the community organization (i.e. increased problem understanding, product capacity, and shifts in practice) than less effective collaboration? Practical Implications As this research develops and continues to expand the understanding of the preconditions, processes, products, and outcomes of design-community collaboration, the findings from this study can be used by practitioners to inform potential points of improvement in the interim. Particularly for WFG, or similar community design charrettes, the findings suggest that certain information on the other discipline could be provided to participants up front to potentially guide their understandings of the problem and expectations for solving it. For example, event organizers could provide the organizations with a list of skills that graphic designers are able to contribute to their work, along with examples of the types of visual communication problems that are best addressed by each skill (e.g., branding can be used to create or change a recognizable identity for your organization, or user experience design can be used to simplify complex activities like navigating your website or completing intake forms). This would allow the organization to make a more informed contribution to the project brief by 113 outlining all of the ways deign may be able to help them, as well helping them decide what kind of products they need based on their underlying needs. The designers could also benefit from a quick introduction to working within the context of community organizations. Event organizers could provide a tip sheet outlining the key considerations when doing so, or a list of questions to ask the organization that would identify unique considerations (e.g., ‘are there sensitive topics that need to be communicated or visually represented in a certain way?’ or ‘tell us about the history of your logo/brand in the community and how it represents what you’re trying to do.’). Questions like this would help draw out the valuable context the organizations can provide, while also positioning it within a context the designers find familiar. Regardless of what format the organizers choose to use, the key takeaway is that strategies like this can ensure key information is exchanged related to certain competencies or knowledge that may ultimately help the designers and the organization enter into the project with more aligned approaches to solving the problem. Conclusion Specific to the present study, the findings contribute to a better understanding of how strategies like design charrettes can be used to integrate design and community change, and the implications of design-community partnerships and interdisciplinary practice on community problem solving. In a more global sense, the aim of the study was to employ community psychology research and evaluation approaches to advance a movement involving integrating design, and the innovative problem solving approaches involved with design, into community- based practice. By continuing research in this area, and by partnering with professional design organizations and experts in design for social change, the findings can contribute to the development of frameworks for effectively bridging design and community change in an 114 interdisciplinary and participatory way. Ultimately, research in this area will be instrumental in establishing impact measures and publishing evidence that can support the institutionalization of design and design thinking into community systems change. 115 APPENDICES 116 Appendix A Figure 9. IRB Determination Letter IRB Determination Letter Initial IRB Application Determination *Exempt* July 6, 2017 To: Re: Jennifer Neal Department of Psychology 127A Psychology Building East Lansing, Mi 48824-1116 IRB# x17-923e Category: Exempt 2 Approval Date: July 6, 2017 Title: Problem Solving Community Design Charrette Processes and Products: An Approach to Interdisciplinary The Institutional Review Board has completed their review of your project. I am pleased to advise you that your project has been deemed as exempt in accordance with federal regulations. The IRB has found that your research project meets the criteria for exempt status and the criteria for the protection of human subjects in exempt research. Under our exempt policy the Principal Investigator assumes the responsibilities for the protection of human subjects in this project as outlined in the assurance letter and exempt educational material. The IRB office has received your signed assurance for exempt research. A copy of this signed agreement is appended for your information and records. Renewals: Exempt protocols do not need to be renewed. If the project is completed, please submit an Application for Permanent Closure. Revisions: Exempt protocols do not require revisions. However, if changes are made to a protocol that may no longer meet the exempt criteria, a new initial application will be required. If the project is modified to add additional sites for the research, please note that you may not begin your research at those sites until you receive the appropriate approvals/permissions from the sites. Problems: If issues should arise during the conduct of the research, such as unanticipated problems, adverse events, or any problem that may increase the risk to the human subjects and change the category of review, notify the IRB office promptly. Any complaints from participants regarding the risk and benefits of the project must be reported to the IRB. Follow-up: If your exempt project is not completed and closed after three years, the IRB office will contact you regarding the status of the project and to verify that no changes have occurred that may affect exempt status. Please use the IRB number listed above on any forms submitted which relate to this project, or on any correspondence with the IRB office. If we can be of further assistance, please contact us at 517-355-2180 or via email at IRB@msu.edu. Thank you for your cooperation. c: Kathryn McAlindon 117 Office of Regulatory Affairs Human Research Protection Programs Biomedical & Health Institutional Review Board (BIRB) Community Research Institutional Review Board (CRIRB) Social Science Behavioral/Education Institutional Review Board (SIRB) 4000 Collins Road Suite 136 Lansing, MI, 48910 (517) 355-2180 Fax: (517) 432-4503 Email: irb@msu.edu www.hrpp.msu.edu MSU is an affirmative-action, equal-opportunity employer. Appendix B Figure 10. Interview Protocol – Designers Data Collection Materials 118 Figure 10. (cont’d) 119 Figure 10. (cont’d) 120 Figure 10. (cont’d) 121 Figure 10. (cont’d) 122 Figure 10. (cont’d) 123 Figure 10. (cont’d) 124 Figure 10. (cont’d) 125 Figure 10. (cont’d) 126 Figure 11. Interview Protocol – Organizations 127 Figure 11. (cont’d) 128 Figure 11. (cont’d) 129 Figure 11. (cont’d) 130 Figure 11. (cont’d) 131 Figure 11. (cont’d) 132 Figure 11. (cont’d) 133 Figure 11. (cont’d) 134 Figure 11. (cont’d) 135 Figure 12. Researcher Observation Protocol 136 Figure 12. (cont’d) 137 Appendix C Pilot Testing Materials Figure 13. Pilot Interview Recruitment Email Dear NAME, My name is Katie McAlindon and I am a community researcher at Michigan State University. As part of an upcoming study, that will serve as my doctoral dissertation research, I am interviewing designers and organizations about their experiences collaborating on community projects during events like IMPACT! Design for Social Change. I am a former participant from SVA's IMPACT! program and I was recently given your name by Mark Randall, who said you would be a good person to talk with. I know the NYPD in Brownsville participated in IMPACT! last summer, as I was a participant in the other group (in Stapleton). I would love to get a chance to talk with you about the project and how it contributed to the NYPD's work. Your interview would serve as a pilot test for a larger study of a similar designer/community collaboration event later this year. If you agree to participate in this phone interview, your answers will be kept strictly confidential and will not be able to be linked back to you. You will also receive a $25 VISA gift card as a thank you for participating in the study. Please reply to let me know if you are willing to participate in a 45-60 minute interview. If so, what dates and times might work best for you? Thank you so much in advance for your help with this research! Sincerely, Katie McAlindon ___________ M.A. | Michigan State University Doctoral Candidate | Ecological/Community Psychology Community Design Consultant | Michigan School Program Information Project mcalindo@msu.edu | www.msu.edu/~mcalindo 138 Figure 14. Pilot Interview Protocol - Designers (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) Date [ ] Charrette # [ ] PILOT Interview Guide – Designer (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) INTRODUCTION First, thank you again for taking the time to do this interview! As I mentioned, the interview today is about your experience at [Weekend Blitz/IMPACT!] and collaborating with your team to get to your final product. We’ll also talk about your work as [a designer] and how you typically approach problems like the one your team was focusing on throughout the event. I also noted that the interview will be recorded with your permission, is that ok? Do you have any questions before we start? [ Y | N ] [ Y | N ] [Start recorder if permission granted] [ PROBE ] How many years have [you] participated? BACKGROUND/ICE BREAKER To start out, could you tell me about how [you] became involved with [AIGA’s Weekend Blitz/the IMPACT!] event? Describe the problem addressed by your [Weekend Blitz/IMPACT!] team this year: 139 Figure 14. (cont’d) APPROACH TO PROBLEM SOLVING Now we are going to talk about how [designers] typically think about, approach, or solve problems, like the one you worked on during [Weekend Blitz/IMPACT!], but in your everyday practice. Can you tell me a little about [your] mission or general philosophy for how [you as a designer] approach problems like this? [ PROBE ] Does [Weekend Blitz/IMPACT!] have an official mission statement? Can you tell me what you think it might be? What are [you as a designer] trying to accomplish in your work right now? [ PROBE ] When trying to solve problems like this, what are [a designer’s] typical objectives or goals? (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2)MCALINDON | 2017(cid:1) (cid:2) 140 Figure 14. (cont’d) What do [you as a designer] value most when trying to solve a problem? [ PROBE ] What do [you as a designer] feel are the most important things to consider in order to arrive at the best solution to a problem? Can you describe how [you as a designer] typically do your work? [ PROBE ] Can you walk me through what [a designer] would do to gather information, make decisions, and produce the tools or solutions for a problem like this? Now, think about [the organization] on your [Weekend Blitz/IMPACT!] team. In what ways, if any, did they share [your] approach [as a designer] to the problem? Meaning, the same goals, values, or methods for doing work that you just talked about? [ PROBE ] Can you describe any differences in your goals, values, or ways of getting work done? (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2)MCALINDON | 2017(cid:1) (cid:2) 141 Figure 14. (cont’d) COLLABORATIVE CLIMATE As you probably know, one of the reasons for [Weekend Blitz/IMPACT!] is to help designers and community organizations collaborate. Collaborating, specifically in this context, means using the knowledge, skills, or expertise of both the designers and the organizations to produce a new solution or idea that they both helped create. What do [you as a designer] feel are the benefits, if any, of collaborating with [community organizations]? [ PROBE ] Do [you as a designer] feel the benefits outweigh any inconveniences or costs of such work? [ PROBE ] Have [you as a designer] collaborated with [community organizations] on projects like this before? Did you find it worthwhile? [ PROBE ] Did [the organization] on your [Weekend Blitz/IMPACT!] team demonstrate a positive attitude toward collaborating with [the designers]? For the rest of the interview we are going to shift to talking about [Weekend Blitz/IMPACT!] specifically and [your] experiences with the event and your team. First, we’ll talk about collaboration within your team throughout the event, then we’ll discuss the solutions or tools your team came up with and how those might help [the] organization moving forward. (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2)MCALINDON | 2017(cid:1) (cid:2) 142 Figure 14. (cont’d) How, if at all, were [you as a team leader] effective in promoting collaboration between the designers and the organization? [ PROBE ] How, if at all, did [you as a team leader] encourage the voicing of all points of view? [ PROBE ] How, if at all, did [you as a team leader] promote participation by the whole team in key decisions? How did the designers and organization on your team communicate throughout Weekend Blitz? (For example, was it face-to-face or electronic? Did you hold formally scheduled meetings? Did the whole team complete the work activities together through the weekend?) [ PROBE ] How frequently did the designers and organization on your team communicate throughout the event? [ PROBE ] Do you think the designers and organization on your team communicated effectively? Thinking about [Weekend Blitz/IMPACT!], and the things we just discussed, what, if anything, do you think helped support collaboration between the designers and organization on your team? [ PROBE ] What, if anything, do you think made collaborating more difficult? (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2)MCALINDON | 2017(cid:1) (cid:2) 143 Figure 14. (cont’d) CHARRETTE PRODUCTS Now we are going to talk about the final product or idea that your team presented at the end of [Weekend Blitz/IMPACT!]. Can you describe the product and how it is intended to address the problem? Do you think the product reflects the contribution of both the designers’ and the organization’s knowledge, skills, or expertise? [ PROBE ] In what ways did [the designers] contribute to this product? [ PROBE ] In what ways did [the organization] contribute to the product? [ PROBE ] Did either the designers or the organization on your team have more input on the final product? [ PROBE ] If anything, explain what you think the group was able to do by collaborating that you could not do using only the designers’ or organization’s approach? (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2)MCALINDON | 2017(cid:1) (cid:2) 144 Figure 14. (cont’d) ORGANIZATION OUTCOMES For the last section of the interview, we’ll talk about the changes or improvements, if any, that [Weekend Blitz/IMPACT!] may have helped promote within [the organization you worked with]. How, if at all, did collaborating to create your team’s final product change [the organization’s] understanding of the initial problem? [ PROBE ] By collaborating with the designers on a new solution, what did [the organization] learn about the problem or approaches to solving it that it did not know before [Weekend Blitz/IMPACT!]? [ PROBE ] Did [the organization] acquire any new knowledge of services, programs, or people in the community that can help address the problem? How, if at all, did collaborating with the designers help [the organization] better understand how to use the product or implement the new idea? [ PROBE ] By collaborating with the designers, did [the organization] learn any new skills related to the product or how it addresses the problem? [ PROBE ] Do you feel confident in [the organization’s] abilities to use the product or implement the new idea? (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2)MCALINDON | 2017(cid:1) (cid:2) 145 Figure 14. (cont’d) After collaborating with designers on a product or idea, how well do you think [the organization] will be able to implement changes that are likely to work in the community? [ PROBE ] Was there a sufficient plan developed for the use and implementation of the product or idea? [ PROBE ] How feasible and sustainable do you think the product or idea is within [the organization]? [ PROBE ] To your knowledge - has [the organization] made any changes to its practices or programs as a result of the product or idea from [Weekend Blitz/IMPACT!]? CLOSING REFLECTIONS Overall, how satisfied were you with the [Weekend Blitz/IMPACT!] experience and the resulting outcomes? [ PROBE – IF + ] What contributed most to your positive experience? [ PROBE – IF - ] What could have been done differently to improve this? [ PROBE ] Will [you] consider participating again? Is there anything else that you think is important to talk about before we end the interview? (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2)MCALINDON | 2017(cid:1) (cid:2) 146 Figure 14. (cont’d) PILOT FOLLOW-UP QUESTIONS Thank you for participating in the interview today! I would like to finish up by asking you a few questions about the interview and your understanding of some of the questions. This will help improve the interview process and make the upcoming study more accurate. [INTERVIEWER: Return to questions that were answered with hesitation or presented a problem and review terms or issues] First, were there any questions, or particular words that stood out, that you did not understand or were confused by? In one question I asked: “Can you tell me a little about [your] mission or general philosophy for how [you as a designer] approach problems like this?” How would you define the terms “mission or general philosophy” as stated in this question? In another question I asked: “What do [you as a designer] value most when trying to solve a problem?” How would you define the word “value” as stated in this question? How would you restate the question “What are [you as a designer] trying to accomplish in your work right now?” in your own words? Finally, how would you restate the question “Can you describe how [you as a designer] typically do your work?” in your own words? (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2)MCALINDON | 2017(cid:1) (cid:2) 147 Figure 15. Pilot Interview Protocol – Organizations (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) Date [ ] Charrette # [ ] PILOT Interview Guide – Community Organizations (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) INTRODUCTION First, thank you again for taking the time to do this interview! As I mentioned, the interview today is about your experience at [Weekend Blitz/IMPACT!] and collaborating with your team to get to your final product. We’ll also talk about your work as [an organization] and how you typically approach problems like the one your team was focusing on throughout the event. I also noted that the interview will be recorded with your permission, is that ok? Do you have any questions before we start? [ Y | N ] [ Y | N ] [Start recorder if permission granted] [ PROBE ] How many years has [your organization] participated? BACKGROUND/ICE BREAKER To start out, could you tell me about how [your organization] became involved with [AIGA’s Weekend Blitz/the IMPACT!] event? Describe the problem addressed by your [Weekend Blitz/IMPACT!] team this year: 148 Figure 15. (cont’d) APPROACH TO PROBLEM SOLVING Now we are going to talk about how [your organization] typically thinks about, approaches, or solves problems, like the one you worked on during [Weekend Blitz/IMPACT!], but in your everyday practice. [ PROBE ] Does [your organization] have an official mission statement? Can you tell me a little about [your organization’s] mission or general philosophy for how it approaches problems like this? What is [your organization] trying to accomplish in its work right now? [ PROBE ] When trying to solve problems like this, what are [your organization’s] typical objectives or goals? (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2)MCALINDON | 2017(cid:1) (cid:2) 149 Figure 15. (cont’d) What does [your organization] value most when trying to solve a problem? [ PROBE ] What does [your organization] feel are the most important things to consider in order to arrive at the best solution to a problem? Can you describe how [your organization] typically does its work? [ PROBE ] Can you walk me through what [your organization] would do to gather information, make decisions, and produce the tools or solutions for a problem like this? Now, think about [the designers] on your [Weekend Blitz/IMPACT!] team. In what ways, if any, did they share [your organization’s] approach to the problem? Meaning, the same goals, values, or methods for doing work that you just talked about? [ PROBE ] Can you describe any differences in your goals, values, or ways of getting work done? (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2)MCALINDON | 2017(cid:1) (cid:2) 150 Figure 15. (cont’d) COLLABORATIVE CLIMATE As you probably know, one of the reasons for [Weekend Blitz/IMPACT!] is to help designers and community organizations collaborate. Collaborating, specifically in this context, means using the knowledge, skills, or expertise of both the designers and the organizations to produce a new solution or idea that they both helped create. What does [your organization] feel are the benefits, if any, of collaborating with [designers]? [ PROBE ] Does your organization feel the benefits outweigh any inconveniences or costs of such work? [ PROBE ] Has [your organization] collaborated with [designers] on projects like this before? Did you find it worthwhile? [ PROBE ] Did [the designers] on your [Weekend Blitz/IMPACT!] team demonstrate a positive attitude toward collaborating with [your organization]? For the rest of the interview we are going to shift to talking about [Weekend Blitz/IMPACT!] specifically and [your organization’s] experiences with the event and your team. First, we’ll talk about collaboration within your team throughout the event, then we’ll discuss the solutions or tools your team came up with and how those might help [the] organization moving forward. (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2)MCALINDON | 2017(cid:1) (cid:2) 151 Figure 15. (cont’d) How, if at all, was your team leader effective in promoting collaboration between the designers and the organization? [ PROBE ] How, if at all, did your team leader encourage the voicing of all points of view? [ PROBE ] How, if at all, did your team leader promote participation by the whole team in key decisions? How did the designers and organization on your team communicate throughout [Weekend Blitz/IMPACT!]? (For example, was it face-to-face or electronic? Did you hold formally scheduled meetings? Did the whole team complete the work activities together through the weekend?) [ PROBE ] How frequently did the designers and organization on your team communicate throughout the event? [ PROBE ] Do you think the designers and organization on your team communicated effectively? Thinking about [Weekend Blitz/IMPACT!], and the things we just discussed, what, if anything, do you think helped support collaboration between the designers and organization on your team? [ PROBE ] What, if anything, do you think made collaborating more difficult? (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2)MCALINDON | 2017(cid:1) (cid:2) 152 Figure 15. (cont’d) ORGANIZATION OUTCOMES For the last section of the interview, we’ll talk about the changes or improvements, if any, that [Weekend Blitz/IMPACT!] may have helped promote within [your organization]. How, if at all, did collaborating to create your team’s final product change your understanding of the initial problem? [ PROBE ] By collaborating with the designers on a new solution, what did you learn about the problem or approaches to solving it that you did not know before [Weekend Blitz/IMPACT!]? [ PROBE ] Did you acquire any new knowledge of services, programs, or people in the community that can help address the problem? How, if at all, did collaborating with [the designers] help you better understand how to use the product or implement the new idea? [ PROBE ] By collaborating with the designers, did you learn any new skills related to the product or how it addresses the problem? [ PROBE ] Do you feel confident in your organization’s abilities to use the product or implement the new idea? (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2)MCALINDON | 2017(cid:1) (cid:2) 153 Figure 15. (cont’d) After collaborating with designers on a product or idea, how well do you think your organization will be able to implement changes that are likely to work in the community? [ PROBE ] Was there a sufficient plan developed for the use and implementation of the product or idea? [ PROBE ] How feasible and sustainable do you think the product or idea is within [your organization]? [ PROBE ] Has [your organization] made any changes to its practices or programs as a result of the product or idea from [Weekend Blitz/IMPACT!]? CLOSING REFLECTIONS Overall, how satisfied were you with the [Weekend Blitz/IMPACT!] experience and the resulting outcomes? [ PROBE – IF + ] What contributed most to your positive experience? [ PROBE – IF - ] What could have been done differently to improve this? [ PROBE ] Will [your organization] consider participating again? Is there anything else that you think is important to talk about before we end the interview? (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2)MCALINDON | 2017(cid:1) (cid:2) 154 Figure 15. (cont’d) PILOT FOLLOW-UP QUESTIONS Thank you for participating in the interview today! I would like to finish up by asking you a few questions about the interview and your understanding of some of the questions. This will help improve the interview process and make the upcoming study more accurate. [INTERVIEWER: Return to questions that were answered with hesitation or presented a problem and review terms or issues] First, were there any questions, or particular words that stood out, that you did not understand or were confused by? In one question I asked: “Can you tell me a little about [your organization’s] mission or general philosophy for how it approaches problems like this?” How would you define the terms “mission or general philosophy” as stated in this question? In another question I asked: “What does [your organization] value most when trying to solve a problem?” How would you define the word “value” as stated in this question? How would you restate the question “What is [your organization] trying to accomplish in its work right now? in your own words? Finally, how would you restate the question “Can you describe how [your organization] typically does its work?” in your own words? (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2)MCALINDON | 2017(cid:1) (cid:2) 155 Figure 16. Pilot Observation Protocol (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) Weekend Blitz Researcher Observation Guide Date [ ] Charrette # [ ] Start Time [ : ] End Time [ : ] (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) Aligned Approach ✓ if observed ⏏ Aligned or Different Objectives, Values, Methods O ⏏ Misc. APPROACHES TO PROBLEM SOLVING Notes 156 COLLABORATIVE CLIMATE Figure 16. (cont’d) Collaborative Climate ✓ if observed ⏏ Attitudes, Leadership, Communication O ⏏ Misc. Notes (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2)MCALINDON | 2017(cid:1) (cid:2) 157 CHARRETTE PRODUCTS Figure 16. (cont’d) Effective Collaboration ✓ if observed ⏏ Designer/Org. Contributions to Product O ⏏ Misc. Notes (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2)MCALINDON | 2017(cid:1) (cid:2) 158 ORGANIZATION OUTCOMES Notes Figure 16. (cont’d) Organization Outcomes ✓ if observed ⏏ Understanding, Capacity, Changes in Practice O ⏏ Misc. (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2) (cid:1) (cid:2)MCALINDON | 2017(cid:1) (cid:2) 159 Appendix D Interview Recruitment Emails Figure 17. Designer Interview Recruitment Email SUBJECT: Invitation to participate in MSU Weekend for Good Study Follow-Up Interview Hello NAME, My name is Katie McAlindon and I am the community researcher from Michigan State University who observed your team during Weekend for Good. As part of the Weekend for Good study, I am interviewing team leads and organizations I observed about their experiences collaborating during the event. The Weekend for Good organizers identified you as a good person to talk with for the research. I would love to get a chance to come sit down with you and talk about your project and how it may have helped the organization. If you agree to participate in this interview, your answers will be kept strictly confidential and will not be able to be linked back to you. You will also receive a $25 VISA gift card as a thank you for participating in the study. Please reply to let me know if you are willing to participate in a 45-60 minute interview. If so, what dates and times might work best for you? Thank you so much in advance for your help with this research! Sincerely, Katie McAlindon ___________ M.A. | Michigan State University Doctoral Candidate | Ecological/Community Psychology Community Design Consultant | Michigan School Program Information Project mcalindo@msu.edu | www.msu.edu/~mcalindo 160 Figure 18. Organization Interview Recruitment Email SUBJECT: Invitation to participate in MSU Weekend for Good Study Follow-Up Interview Hello NAME, My name is Katie McAlindon and I am the community researcher from Michigan State University who observed your team during Weekend for Good. As part of the Weekend for Good study, I am interviewing team leads and organizations I observed about their experiences collaborating during the event. The Weekend for Good organizers identified you as a good person to talk with for the research. I would love to get a chance to come visit your organization and talk with you about your project and how it may have helped your efforts. If you agree to participate in this interview, your answers will be kept strictly confidential and will not be able to be linked back to you. You will also receive a $25 VISA gift card as a thank you for participating in the study. Please reply to let me know if you are willing to participate in a 45-60 minute interview. If so, what dates and times might work best for you? Thank you so much in advance for your help with this research! Sincerely, Katie McAlindon ___________ M.A. | Michigan State University Doctoral Candidate | Ecological/Community Psychology Community Design Consultant | Michigan School Program Information Project mcalindo@msu.edu | www.msu.edu/~mcalindo 161 Appendix E Figure 19. Informed Consent Informed Consent 162 Analysis Codebook Appendix F Table 2. Analysis Codebook Variable Code Definition When to use… "Why?" (why they do the work OR important factors for doing it well) Discussion of motivations, reasons, considerations, principles, standards (OR the lack of - e.g., "I don't think beauty is a factor in how well something works") This might include discussions of…helping the community, organizational history, community context, user experience, aesthetics or beauty, considering their partner's opinions or expertise, etc. Important considerations, principles, or standards that drive the organization's or designers' work. Values Example Quotes "I think the most important thing is properly defining the problem, yeah. I don't remember where I heard it, but there's an example I heard once where there was a room of people and they were all there to discuss the same problem, and the people working on the project, the first thing they asked was everyone to write down what the problem was. Then they looked at all the answers, and they all had different answers. Actually knowing what the problem is. It's pretty important." NOTE Values vs. Objectives vs. Methods Some terms could be used for describing a value, objective, OR method. Look at the question they were asked and if they're talking about why (or important factors), what, or how... EXAMPLE: Feedback Value: "It's important to consider community feedback when working with these orgs because they are ultimately the ones impacted" 163 Table 2 (cont’d) "What?" Discussion of needs, wants, aims, goals, agendas, end results (OR the lack of - e.g., "I don't typically work with an end goal in mind") This might include discussions of…what an ideal solution looks like or wanting to create products that are simple, easy to use, user friendly, right for the organization, sustainable, etc. "How?" Discussion of procedures, processes, approaches, practices (OR the lack of - e.g., "We don't follow any specific procedure, we tend to make it up as we go") This might include discussions The desired results of the organization's or designers' work. The process or approach used by the organization or designers to achieve their objectives. Objectives Methods WHEN TRYING TO SOLVE A PROBLEM LIKE THIS, WHAT ARE YOU AS A DESIGNER, WHAT ARE YOUR TYPICAL OBJECTIVES OR GOALS? "To have a solution, one that makes sense for the organization and for the customers that are using it. Something that, in this specific situation, something that's not cumbersome to use, or something that's going to break in three months, which is another thing I heard coming into the weekend was that, that happens." "Talk to the client, interview the client about what they think is going on. Often research competition. For this project, because there was a team there that knew how to do it, I also did user research, but in a really quick way where we didn't actually go Objective: "My main goal is to create products that accurately incorporate all the feedback we get from the client" Method: "We usually gather feedback by surveying the user at different stages of the project" ALL THREE: "Feedback is crucial for getting to the best 164 Table 2 (cont’d) The degree to which the organization and the designers solve problems in the same ways. Aligned Problem Solving Approach of…types of research they conduct, approaches to planning, creating user profiles or personas, if they work in groups or alone, how they gather feedback or opinions, tools they use (e.g., whiteboards, team work apps, client briefs, wordpress), etc. This will include the discussion of similarities OR differences between the organization and the designers, including their…values, reasons for doing the project, definitions of the problem, priorities, objectives, ideas for what the final product should be, methods, feedback styles (e.g., constant exchange vs. work alone and exchange later), tools (e.g., computers vs. paper, email vs. team work app). etc. Remember: The discussion must interview customers, but we created personas and created user flows through the website." solution, so we repeatedly survey users because we want the end result to address their concerns" "I think [org] was pretty open to our suggestions. I didn't feel as though there was any conflict with that approach. Seemed like he was pretty open to the whole thing. It's like a socialist hippy organization, so he was pretty down with whatever, and I shared those similar ideals. It did seem like a good match as far as personalities and approaches go." 165 Table 2 (cont’d) The use of the organization's skills, knowledge, or expertise in creating the final product. Organization Contribution be about the alignment between the designers and the organization, NOT just among the designers/developers (e.g., NOT "only one designer really valued feedback while the other designers wanted to skip that" OR "the developers liked to work alone but the designers were much more collaborative") Discussion of the use of the organization's skills, knowledge, expertise, ideas, input, feedback, history, context, materials, tools, suggestions, opinions, approval (OR the lack of - e.g., "the organization never had much feedback so we had to make most of the decisions based on what we liked") This might include discussions of...the involvement of the organization in the decision making process, pamphlets or information 166 IN THE SAME WAY, IN WHAT WAY DID THE ORGANIZATION CONTRIBUTE TO THE PRODUCT? "Kind of more of like a consultant's role a little bit, and content creation, also, and just general feedback, and kind of the expertise of the organization and the brand that only he would have, obviously." Table 2 (cont’d) about the organization used to inform the designers, pre-existing website content or images provided by the organization used on the new site, information about users provided by the organization to create user profiles, solutions or ideas brought forth by the organization (e.g., choosing their The use of the designers' skills, knowledge, or expertise in creating the final product. Designer Contribution own wordpress theme, changing layouts, including new pages or content, using their preferred technology like CiviCRM or wordpress), etc. Discussion of the use of the designers' skills, knowledge, expertise, ideas, input, feedback, history, context, materials, tools, suggestions, opinions, approval (OR the lack of - e.g., "we didn't know anything about the services the organization provides, so the 167 IN WHAT WAYS DID THE DESIGNERS CONTRIBUTE TO THE PRODUCT? I MEAN THAT'S A BIG QUESTION, BUT OVERALL? "Okay. Branding knowledge, UX design, WordPress development, general content creation, copywriting." Table 2 (cont’d) designers didn't write those sections") This might include discussions of...the involvement of the designers in the decision making process, content or images generated by the designers, knowledge of design, knowledge of user research/UX/UI, understanding of technology like wordpress or plugins, manuals or training guides created for the organization, solutions or ideas brought forth by the designers (e.g., reorganizing web content or pages, updating branding, creating new logo, using new email client, providing additional content for facebook or blog posts) 168 Appendix G Figure 20. Example Coding Spreadsheet Figure 21. Example Themeing Spreadsheet Example Data Condensation, Visualization, and Analysis Tools 169 Figure 22. Example Cross-Case Comparison Matrix 170 Appendix H Comparative Case Analysis Procedure Codebook Creation The codebook (Appendix G Figure 23) was designed based on an adaptation of a widely sourced rubric developed by The Center for Prevention Research & Development at the University of Illinois for evaluating community coalition effectiveness (Mid-South Prevention Department, 2010; University of Illinois, 1999). The following elements are included in the codebook: • Each variable to be identified in initial coding to answer Research Questions 1 and 2: o Values o Objectives o Methods o Aligned Problem Solving Approach o Designer Contribution to Final Product o Organization Contribution to Final Product • A definition of each variable based on the literature (outlined in Chapter 1) • Guidelines for when to assign codes • Example data segments for each code • Notes based on clarifications established during coding Overview of Analysis The analytic plan for this study was aligned with Miles et al.’s (2014) approach to cross-case analysis, moving “from one inductive inference to another by selectively collecting data, comparing and contrasting this material in the quest for patterns or regularities, seeking out more data to support or qualify these emerging clusters, and then gradually drawing inferences from the links between other new data segments and the cumulative set of conceptualizations” (p. 29). In short, analysis began with a unique case orientation using inductive analysis and creative synthesis (Patton, 2002; Stake, 2006), followed by cross-case comparisons for patterns related to the research questions (Miles et al., 2014). This approach, while less emergent and exploratory than analytic induction (Hammersley & Cooper, 2012), allows for the analysis to be guided by a 171 framework or research questions established a priori, as well as being conducive to the manageable comparison of data from up to ten cases. In related research, Miles et al.’s (2014) cross-case analytic approach is considered the gold standard and is widely applied (e.g., Anderson et al., 2010; Goodman, 2009; Kegler & Wyatt, 2003; Vohra, 2014). Thus, both within and cross-case data analysis occurred via the three iterative activities outlined in their approach: 1) data condensation, 2) data display, and 3) conclusion drawing/verification. Within-Case Analysis Data Condensation Step 1: Each case’s data from all sources was organized in a spreadsheet for that case: 1.a Interview transcriptions were divided into segments. Specifically, segments were created for each response to a particular interview question in the protocol. 1.b The observation debriefing notes were organized by bullet point at the time of the debriefings and every bullet point became a data segment. 1.c Archival data was divided into segments based on the headings in the project brief (e.g., project description, organization’s mission, design lead’s skills, etc.) and each photo from both the collaborative workspaces and the final products became separate data segments. 1.d All data segments were placed into a single coding spreadsheet for each of the four cases (see Appendix G Figure 20 for example). Step 2: All data segments were coded: 2.a Each segment was coded for the presence (1) or absence (0) of each variable defined in the codebook. 2.b The resulting codes from two independent coders were compared and the coders met to discuss discrepancies. Discrepancies ranged from 0-12% for all of the 366 - 481 data segments coded for each variable. During the meetings to discuss discrepancies, coders either: • Came to a consensus based on the existing codebook definition, or • Refined the codebook definition and came to a consensus (this occurred in less than 10 instances and refinements were documented in the “Notes” column of the codebook). 172 Data Display Step 3: Coded data segments were reorganized and displayed in a Data Segments: Variable X Data Source matrix for each charrette (see Appendix G Figure 21 for example). 3a. Variables were listed in the column headers of the matrix and data sources in the row headers. 3b. For each variable and each source, any data segments coded as present (1) were entered into the appropriate cell of the matrix. Conclusion Drawing Step 4: Subcodes were created for each data segment based on each variable within each data source. “A subcode is a second-order tag assigned after a primary code to detail or enrich the entry. The method is appropriate for virtually all qualitative studies, but particularly for ethnographies and content analyses, studies with multiple participants and sites, and studies with a wide variety of data forms. Subcoding is also appropriate when general code entries will later require more extensive indexing, categorizing, and subcategorizing into hierarchies or taxonomies, or for nuanced qualitative data analysis” (Miles, Huberman, & Saldana, 2014, p. 85). Step 5: Similar subcodes were condensed to create a list of distinct subcodes for each variable within each data source (e.g., within the organization interview data related to values, the subcodes efficiency with time, making good use of resources, efficiency with money, being efficient with limited resources, were combined to become org values efficiency with limited resources like time and money). Step 6: In the same way, similar subcodes were then compared and combined across all data sources for each variable to establish themes for each variable. 6.a At this point, the variable subcodes for values, objectives, and methods were compared and combined across data sources (designer interview, organization interview, observations, and archival) to create themes for aligned values, aligned objectives, and aligned methods. 6.b When subcodes were present in both the designer interview and organization interview data (and supported by an observation or archival subcode) suggesting an aligned or misaligned value, objective or method, a theme was created (e.g., 173 across a case’s four data sources related to methods, the subcodes designers typically work under a project director (designer interview), org has facilitator guide problem solving and workflow (organization interview), charrette observed welcoming design lead’s tendency to make the decisions and delegate (observations), and design lead skills include project management (archival) were combined to create the case-level theme for aligned methods designers and org shared method of working under facilitator, which aligned with the design lead’s methods of project management). Cross-Case Analysis Data Condensation Step 1: Each case’s data was condensed down to key themes for each of the following variables (from within-case analysis Step 6): a. Aligned Values b. Aligned Objectives c. Aligned Methods d. Aligned Problem Solving Approaches e. Designer Contribution f. Organization Contribution Data Display Step 2: Themes were displayed in a Data Themes: Variable X Case matrix for cross- case analysis (see Appendix G Figure 22 for example). 2a. Variables were listed in the column headers of the matrix and cases in the row headers. 2b. For each variable and each case, themes were entered into the appropriate cell of the matrix. Conclusion Drawing Step 3: The themes for each variable were compared across cases to assign each case a level for that variable (low, medium, high). 3.a The levels were assigned to variables a through d using magnitude coding (Miles, Huberman, & Saldana, 2014) based on the number of themes a case had 174 representing alignment and the number of themes a case had representing misalignment relative to the other cases (e.g., if one case had 1 theme representing aligned values and 4 representing misaligned values, it would be considered to have lower alignment relative to a case with 5 themes representing alignment and 1 representing misalignment). 3.a Levels were assigned to variables e and f using magnitude coding based on the number of themes a case had representing contribution and lack of contribution relative to the other cases. Step 4: Variable levels were compared across cases and combined within each case to establish the magnitudes (low, moderate, high) of the following within each case: • Overall Aligned Problem Solving Approach (determined by comparing and combining the levels of variables a through d) • Integrated Product (determined by comparing and combing the levels of variables e and f) Step 5: The resulting magnitudes for Aligned Problem Solving Approach and Integrated Product were reviewed by another independent member of the research team against the Data Themes: Variable X Case matrix (from cross-case analysis Step 2), and based on that team member’s notes from observing during WFG and completing the initial data coding. The team met to debrief and collectively review cross-case analysis Steps 1-4 to ensure consensus on the final magnitudes displayed in the cross-case matrix (see Figure 8). 175 REFERENCES 176 REFERENCES Alexander, J. A., Weiner, B. J., Metzger, M. E., Shortell, S. M., Bazzoli, G. J., Hasnain-Wynia, R., Sofaer, S., & Conrad, D. A. (2003). Sustainability of collaborative capacity in community health partnerships. Medical Care Research and Review, 60(4_suppl), 130S- 160S. Allen, N. E. (2005). A multi-level analysis of community coordinating councils. American Anderson, R. J. (1994). Representations and requirements: the value of ethnography in system Journal of Community Psychology, 35(1-2), 49-63. design. Human-Computer Interaction, 9(3), 151-182. Anderson, K. J. B., Courter, S. S., McGlamery, T., Nathans-Kelly, T. M., & Nicometo, C. G. (2010). Understanding engineering work and identity: A cross-case analysis of engineers within six firms. Engineering Studies, 2(3), 153-174. Andrews, J. O., Newman, S. D., Meadows, O., Cox, M. J., & Bunting, S. (2010). Partnership readiness for community-based participatory research. Health Education Research, 27(4), 555-571. Baer, M. (2010). The strength-of-weak-ties perspective on creativity: A comprehensive examination and extension. Journal of Applied Psychology, 95(3), 592. Barron, B. (2000). Achieving coordination in collaborative problem-solving groups. The Journal of the Learning Sciences, 9(4), 403-436. Bennett, C. C. (1965). Community psychology: Impressions of the Boston Conference on the Education of Psychologists for Community Mental Health. American Psychologist, 20(10), 832. Bentley, M. E., Pelto, G. H., Straus, W. L., Schumann, D. A., Adegbola, C., de la Pena, E., Oni, G., Brown, K., & Huffman, S. L. (1988). Rapid ethnographic assessment: Applications in a diarrhea management program. Social Science & Medicine, 27(1), 107-116. Borden, L. M., & Perkins, D. F. (1999). Assessing your collaboration: A self evaluation tool. Journal of Extension, 37(2), n2. Bradt, D. A., & Drummond, C. M. (2002). Rapid epidemiological assessment of health status in displaced populations—An evolution toward standardized minimum essential data sets. Prehospital and Disaster Medicine, 17(4), 178-185. Bronstein, L. R. (2003). A model for interdisciplinary collaboration. Social Work, 48(3), 297- 306. 177 Buchanan, R. (1992). Wicked problems in design thinking. Design Issues, 8(2), 5-21. Butt, G., Markle-Reid, M., & Browne, G. (2008). Interprofessional partnerships in chronic illness care: A conceptual model for measuring partnership effectiveness. International Journal of Integrated Care, 8(2). Butterfoss, F. D., Goodman, R. M., & Wandersman, A. (1993). Community coalitions for prevention and health promotion. Health Education Research, 8(3), 315-330. Campbell, L. M. (2005). Overcoming obstacles to interdisciplinary research. Conservation Biology, 19(2), 574-577. Catherwood-Ginn, J., & Leonard, B. (2013). Playing for the public good: The arts in planning and government. Animated Democracy, A working guide to the landscape of arts for change. Washington, DC: Americans for the Arts. Center for the Advancement of Collaborative Strategies in Health (2006). Partnership Self- Assessment Tool. https://atrium.lib.uoguelph.ca/xmlui/bitstream/handle/10214/3129/ Partnership_Self-Assessment_Tool-Questionnaire_complete.pdf Chaskin, R. J. (Ed.). (2001). Building community capacity. Transaction Publishers. Condon, P. M. (2012). Design charrettes for sustainable communities. Island Press. Cornish, K., Goodman-Deane, J., Ruggeri, K., & Clarkson, P. J. (2015). Visual accessibility in graphic design: A client–designer communication failure. Design Studies, 40, 176-195. Cornwall, A. (1992). Body mapping in health RRA/PRA. RRA Notes, 16, 69-76. Cornwall, A., & Jewkes, R. (1995). What is participatory research?. Social Science & Medicine, 41(12), 1667-1676. Crane, L. (2011). Building worlds together: The many functions and forms of arts and community development. Animated Democracy, A working guide to the landscape of arts for change. Washington, DC: Americans for the Arts. Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods approaches. Sage publications. Cross, N. (2004). Expertise in design: An overview. Design Studies, 25(5), 427-441. Denzin, N. K., & Lincoln, Y. S. (Eds.). (2011). The Sage handbook of qualitative research. Sage. Dhar, T. K., & Khirfan, L. (2016). Community-based adaptation through ecological design: Lessons from Negril, Jamaica. Journal of Urban Design, 21(2), 234-255. 178 Dorst, K., & Cross, N. (2001). Creativity in the design process: Co-evolution of problem– solution. Design Studies, 22(5), 425-437. Dyson, A., & Genishi, C. (2005). On the case. New York, NY: Teachers College Press. Evans, M. P., Newman, A., & Winton, S. (2015, July). Not your mother's PTA: Hybridity in community-based organizations working for educational reform. The Educational Forum 79(3), 263-279. Frascara, J. (1988). Graphic design: Fine art or social science?. Design Issues, 5(1), 18-29. Friedman, K. (2003). Theory construction in design research: criteria: Approaches, and methods. Design Studies, 24(6), 507-522. Galle, P. (1999). Design as intentional action: A conceptual analysis. Design Studies, 20(1), 57-81. Gibson, M. R., & Owens, K. M. (2014). Strategies for bridging gaps in understanding between researchers who possess design knowledge and those working in disciplines outside design. The Routledge Companion to Design Research, 386. Girling, C. L., Kellett, R., Johnstone, S. (2006). Informing design charrettes: Tools for participation in neighbourhood-scale planning. Integrated Assessment, 6(4). Goedert, J. (2008). Rebuilding New Orleans with affordable, hurricane-resistant residential Goldstein, S. M. (1997). Community coalitions: A self-assessment tool. American Journal of construction. Cityscape 10(3), 167-183. Health Promotion, 11(6), 430-435. Goodman, R. M. (2009). A construct for building the capacity of community‐based initiatives in racial and ethnic communities: A qualitative cross‐case analysis. Journal of Public Health Management and Practice, 15(2), E1-E8. Granner, M. L., & Sharpe, P. A. (2004). Evaluating community coalition characteristics and functioning: a summary of measurement tools. Health Education Research, 19(5), 514-532. Granovetter, M. S. (1973). The strength of weak ties. American Journal of Sociology, 78(6), 1360-1380. Relations, 38(10), 911-936. Gray, B. (1985). Conditions facilitating interorganizational collaboration. Human Gray, B. (2004). Strong opposition: Frame‐based resistance to collaboration. Journal of 179 Community & Applied Social Psychology, 14(3), 166-176. Hacker, K., Tendulkar, S. A., Rideout, C., Bhuiya, N., Trinh-Shevrin, C., Savage, C. P., Grullon, M., Strelnick, H., Leung, C., & DiGirolamo, A. (2012). Community capacity building and sustainability: Outcomes of community-based participatory research. Progress in Community Health Partnerships: Research, Education, and Action, 6(3), 349. Hall, K. L., Vogel, A. L., Stipelman, B. A., Stokols, D., Morgan, G., & Gehlert, S. (2012). A four-phase model of transdisciplinary team-based research: Goals, team processes, and strategies. Translational Behavioral Medicine, 2(4), 415-430. Hammersley, M., Cooper, B., (2012). Analytic induction versus qualitative comparative analysis. In B. Cooper, J. Glaesser, R. Gomm, & M. Hammersley, (Eds.). Challenging the qualitative-quantitative divide: explorations in case-focused causal analysis, 129- 169. Hays, C. E., Hays, S. P., Deville, J. O., & Mulhall, P. F. (2000). Capacity for effectiveness: The relationship between coalition structure and community impact. Evaluation and Program Planning, 23(3), 373-379. Herbert, D. M. (1993). Architectural study drawings. John Wiley & Sons. Horwath, J., & Morrison, T. (2007). Collaboration, integration and change in children's services: critical issues and key ingredients. Child Abuse & Neglect, 31(1), 55-69. Howard, Z., & Somerville, M. M. (2014). A comparative study of two design charrettes: Implications for codesign and participatory action research. CoDesign, 10(1), 46-62. Israel, B. A., Schulz, A. J., Parker, E. A., & Becker, A. B. (1998). Review of community-based research: Assessing partnership approaches to improve public health. Annual Review of Public Health, 19(1), 173-202. Jacobs, J. A., & Frickel, S. (2009). Interdisciplinarity: A critical assessment. Annual Review of 180 Justesen, S. (2004). Innoversity in communities of practice. Knowledge Networks: Innovation Sociology, 35, 43-65. Through Communities of Practice, 79-95. Kegler, M. C., Steckler, A., Mcleroy, K., & Malek, S. H. (1998). Factors that contribute to effective community health promotion coalitions: A study of 10 Project ASSIST coalitions in North Carolina. Health Education & Behavior, 25(3), 338-353. Kegler, M. C., Steckler, A., Malek, S. H., & McLeroy, K. (1998b). A multiple case study of implementation in 10 local Project ASSIST coalitions in North Carolina. Health Education Research, 13(2), 225-238. Kegler, M. C., & Wyatt, V. H. (2003). A multiple case study of neighborhood partnerships for positive youth development. American Journal of Health Behavior, 27(2), 156-169. Kim, Y. S. (2013) Ethnic senior schools, religion, and psychological well-being among older Korean immigrants in the United States: A qualitative study, Educational Gerontology, 39(5), 342-354. Kowaltowski, D. C., Bernardi, N., & Martin, C. M. (2015). Universal design pedagogy through a charrette to increase professional sensitivity. Journal of Accessibility and Design For All, 5(1), 47-76. Lakhani, J., Benzies, K., & Hayden, K. A. (2012). Attributes of interdisciplinary research teams: A comprehensive review of the literature. Clinical & Investigative Medicine, 35(5), 260-265. Lantz, P. M., Viruell-Fuentes, E., Israel, B. A., Softley, D., & Guzman, R. (2001). Can communities and academia work together on public health research? Evaluation results from a community-based participatory research partnership in Detroit. Journal of Urban Health, 78(3), 495-507. Lappe, F. M., & Du Bois, P. M. (1994). The Quickening of America: Rebuilding Our Nation, Remaking Our Lives. San Francisco: Jossey-Bass, Inc. Lasker, R. D., & Weiss, E. S. (2003). Broadening participation in community problem solving: A multidisciplinary model to support collaborative practice and research. Journal of Urban Health, 80(1), 14-47. Lee, J., Popovic, V., Blackler, A. L., & Lee, K. (2009). User-designer collaboration during the early stage of the design process. IASDR 2009 Proceedings, 1-12. Lennertz, B. (2003). The Charrette as an Agent for Change. In In New Urbanism: Comprehensive Report JI Jackson and Best Practice Guide. Ithaca: New Urban Publications. Lennertz, B., Lutzenhiser, A., & Failor, T. (2008). An introduction to charrettes. Planning Commissioners Journal, 71, 1-3. Lessard, M., & Torres, J. (2007). Children and design students as partners in community design: The Growing Up in Cities projects in Montreal and Guadalajara. Children Youth and Environments, 17(2), 527-540. Lewandowski, C., & GlenMaye, L. (2002). Teams in child welfare settings: Interprofessional and collaborative processes. Families in Society: The Journal of Contemporary Social Services, 83(3), 245-256. Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic Inquiry. Sage. 181 Long, K. A. (2001). A reality-oriented approach to interdisciplinary work. Journal of Professional Nursing, 17(6), 278-282. Luque, J., Martinez Tyson, D., Lee, J. H., Gwede, C., Vadaparampil, S., Noel‐Thomas, S., & Meade, C. (2010). Using social network analysis to evaluate community capacity building of a regional community cancer network. Journal of Community Psychology, 38(5), 656-668. Mansilla, V. B. (2006). Assessing expert interdisciplinary work at the frontier: An empirical Margolin, V., & Margolin, S. (2002). A “social model” of design: Issues of practice and exploration. Research Evaluation, 15(1), 17-29. research. Design Issues, 18(4), 24-30. Maryman, B., & Maggio, C. (2004). In Seattle, the mother of all charrettes: Three hundred designers convene to craft new visions of Seattle’s waterfront. Landscape Architecture, 94(8). 64, 66–70, 72–75. Maton, K. I., Perkins, D. D., & Saegert, S. (2006). Community psychology at the crossroads: Prospects for interdisciplinary research. American Journal of Community Psychology, 38(1-2), 35-49. Mattessich, P. W., & Monsey, B. R. (1992). Collaboration: What Makes it Work. A Review of Research Literature on Factors Influencing Successful Collaboration. St. Paul, MN: Amherst H. Wilder Foundation. Mâsse, L. C., Moser, R. P., Stokols, D., Taylor, B. K., Marcus, S. E., Morgan, G. D., Hall, K. L., Croylr, R. T., & Trochim, W. M. (2008). Measuring collaboration and transdisciplinary integration in team science. American Journal of Preventive Medicine, 35(2), S151-S160. McDonnell, J. (2009). Feedback to the design: A study of the negotiation of contributions in design conversation between architects and building users. In McDonnell, J. & Lloyd, P. (Eds.). About Designing – Analysing Design Meetings. Abingdon: Taylor & Francis. McLaughlin, E. A. (2013). Design charrette as methodology for student learning assessment relative to building safety and security. Journal of Interior Design, 38(2), 35-46. Merriam, S. B. (2009). Qualitative research: A guide to design and implementation. San Francisco, CA: Jossey-Bass. Mid-South Prevention Department (2010). Mid-South Prevention Department’s Coalition Effectiveness Evaluation Rubric. https://www.yumpu.com/en/document/view/37480846/ mid-south-coalition-effectiveness-evaluation-rubric-mi-pte Miles, M. B., Huberman, A. M., & Saldaña, J. (2014). Qualitative data analysis: A methods 182 sourcebook. Sage Publications : California. Millen, D. R. (2000). Rapid ethnography: Time deepening strategies for HCI field research. In Proceedings of the 3rd conference on Designing interactive systems: processes, practices, methods, and techniques (pp. 280-286). ACM. Misra, S., Stokols, D., & Cheng, L. (2015). The transdisciplinary orientation scale: Factor structure and relation to the integrative quality and scope of scientific publications. Journal of Collaborative Healthcare and Translational Medicine, 3(2), 1042. Morrison, T. (1996). Partnership and collaboration: Rhetoric and reality. Child Abuse & Neglect, 20(2), 127–140. Nie, X. (2016). Communities of innovation: Individual, group, and organizational characteristics leading to greater potential for innovation. International Journal of Science, 3(10), 105-112. Nowell, B. (2009). Profiling capacity for coordination and systems change: The relative contribution of stakeholder relationships in interorganizational collaboratives. American Journal of Community Psychology, 44(3-4), 196-212. Nowell, B. (2010). Out of sync and unaware? Exploring the effects of problem frame alignment and discordance in community collaboratives. Journal of Public Administration Research and Theory, 20(1), 91-116. Onyango, J., & Noguchi, M. (2009). Changing attitudes of community through the design charrette process. The International Journal of Neighbourhood Renewal, 1(3), 19-30. Palinkas, L. A., Horwitz, S. M., Green, C. A., Wisdom, J. P., Duan, N., & Hoagwood, K. (2015). Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Administration and Policy in Mental Health and Mental Health Services Research, 42(5), 533-544. Papanek, V., & Fuller, R. B. (1972). Design for the real world. London: Thames and Hudson. Patton, M. Q. (2002). Qualitative evaluation and research methods. Sage Publications. Rappaport, J. (2000). Community narratives: Tales of terror and joy. American Journal of Community Psychology, 28(1), 1-24. Roggema, R. (2014). The Design Charrette. Springer Netherlands. Rottle, N. D., & Johnson, J. M. (2007). Youth design participation to support ecological literacy: Reflections on charrettes for an outdoor learning laboratory. Children Youth and Environments, 17(2), 484-502. 183 Roussos, S. T., & Fawcett, S. B. (2000). A review of collaborative partnerships as a strategy for improving community health. Annual Review of Public Health, 21(1), 369-402. Sandhu, J. S., Altankhuyag, P., & Amarsaikhan, D. (2007). Serial hanging out: Rapid ethnographic needs assessment in rural settings. In International Conference on Human- Computer Interaction (pp. 614-623). Springer, Berlin, Heidelberg. Sanoff, H. (2000). Community participation methods in design and planning. John Wiley & Sawyer, R. K. (2006). Group creativity: Musical performance and collaboration. Psychology of Music, 34(2), 148-165. Senge, P., & Sharmer, O. (2001). Community action research: Learning as a community of practitioners, consultants, and researchers. In P. Reason & H. Bradbury (Eds.), Handbook of Action Research: Participative Inquiry and Practice. London: Sage. Smith, N.D. (2012). Design Charrette: A Vehicle For Consultation Or Collaboration? In Proceedings from The Participatory Innovation Conference. Melbourne. Spath, R., Werrbach, G. B., & Pine, B. A. (2008). Sharing the baton, not passing it: Collaboration between public and private child welfare agencies to reunify families. Journal of Community Practice, 16(4), 481-507. Spradley, J. P. (1980). Participant observation. Waveland Press. Stake, R. E. (1995). The art of case study research. Sage. Stake, R. E. (2006). Multiple case study analysis. Guilford Press. Stempfle, J., & Badke-Schaub, P. (2002). Thinking in design teams-an analysis of team communication. Design Studies, 23(5), 473-496. Stokols, D., Harvey, R., Gress, J., Fuqua, J., & Phillips, K. (2005). In vivo studies of transdisciplinary scientific collaboration: Lessons learned and implications for active living research. American Journal of Preventive Medicine, 28(2), 202-213. Stokols, D., Misra, S., Moser, R. P., Hall, K. L., & Taylor, B. K. (2008). The ecology of team science: Understanding contextual influences on transdisciplinary collaboration. American Journal of Preventive Medicine, 35(2), S96-S115. Strier, R. (2011). The construction of university-community partnerships: Entangled perspectives. Higher Education, 62(1), 81-97. Suarez-Balcazar, Y., Hellwig, M., Kouba, J., Redmond, L., Martinez, L., Block, D., Kohrman, C. Sons. 184 & Peterman, W. (2006). The making of an interdisciplinary partnership: The case of the Chicago Food System Collaborative. American Journal of Community Psychology, 38(1- 2), 113-123. Sutton, S. E. (2000). Reinventing professional privilege as inclusivity: A proposal for an enriched mission of architecture. The Discipline of Architecture, 173-207. Sutton, S. E., & Kemp, S. P. (2002). Children as partners in neighborhood placemaking: Lessons from intergenerational design charrettes. Journal of Environmental Psychology, 22(1), 171-189. Sutton, S. E., & Kemp, S. P. (2006). Integrating social science and design inquiry through interdisciplinary design charrettes: An approach to participatory community problem solving. American Journal of Community Psychology, 38(1-2), 125-139. Sutton, S. E., & Kemp, S. P. (2006b). Children's participation in constructing a socially just public sphere. In M. Blades & C. Spencer (Eds.), Children and their environments: learning, using, and designing spaces (pp. 256–276). Cambridge, UK: Cambridge University Press. Tanaka, T., Abramson, D. B., & Yamazaki, Y. (2009). Using GIS in community design charrettes: Lessons from a Japan–US collaboration in earthquake recovery and mitigation planning for Kobe. Habitat International, 33(4), 310-318. Tebes, J. K., Thai, N. D., & Matlin, S. L. (2014). Twenty‐first century science as a relational process: From eureka! to team science and a place for community psychology. American Journal of Community Psychology, 53(3-4), 475-490. University of Illinois, Center for Prevention Research and Development (1999). Building coalition effectiveness for sustainability: An evaluation rubric, http://www.cprd.illinois.edu/files/coalition%20rubric_0.pdf Valencia-Sandoval, C., Flanders, D. N., & Kozak, R. A. (2010). Participatory landscape planning and sustainable community development: Methodological observations from a case study in rural Mexico. Landscape and Urban Planning, 94(1), 63-70. Vogel, A.L., Stipelman, B.A., Feng, A., Stokols, D., Hall, K.L., & Nebeling, L. (2011). Strategies for facilitating and supporting cross-disciplinary team science on cancer: Lessons from the National Cancer Institute's TREC initiative. Poster Presentation at the 2011 American Public Health Association Conference. Washington, DC. Vohra, V. (2014). Using the multiple case study design to decipher contextual leadership behaviors in Indian organizations. Electronic Journal of Business Research Methods, 12(1). Wageman, R., Hackman, J. R., & Lehman, E. (2005). Team diagnostic survey: Development of 185 an instrument. The Journal of Applied Behavioral Science, 41(4), 373-398. Walker, J. B., & Seymour, M. W. (2008). Utilizing the design charrette for teaching sustainability. International Journal of Sustainability in Higher Education, 9(2), 157-169. Wallerstein, N., Oetzel, J., Duran, B., Tafoya, G., Belone, L., & Rae, R. (2008). What predicts outcomes in CBPR. In Minkler, M., & Wallerstein, N. (Eds.). Community-Based Participatory Research for Health: From Process to Outcomes. John Wiley & Sons. Webber, S. B. The charrette design model provides a means to promote collaborative design in higher education. Systemics, Cybernetics and Informatics 14, 84-91. Woolley, M. E., Bowen, G. L., & Bowen, N. K. (2006). The development and evaluation of procedures to assess child self-report item validity educational and psychological measurement. Educational and Psychological Measurement, 66(4), 687-700. Yin, R. K. (2003). Case Study Research: Design and Methods. Thousand Oaks, CA: Sage Publications. Zakocs, R. C., & Edwards, E. M. (2006). What explains community coalition effectiveness?: A review of the literature. American Journal of Preventive Medicine, 30(4), 351-361. Zhang, H., Mao, Z., & Zhang, W. (2015). Design charrette as methodology for post-disaster participatory reconstruction: Observations from a case study in Fukushima, Japan. Sustainability, 7(6), 6593-6609. 186