SCIENCING THE STORY: METHODOLOGICAL APPROACHES TO STUDYING THE USE OF NARRATIVE IN SCIENCE COMMUNICATION By Alison Singer A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Community Sustainability - Doctor of Philosophy 2019 ABSTRACT SCIENCING THE STORY: METHODOLOGICAL APPROACHES TO STUDYING THE USE OF NARRATIVE IN SCIENCE COMMUNICATION By Alison Singer As scientists increase their engagement with the public, it has become clear that traditional modes of engagement are no longer sufficient to help us solve the complex scientific and environmental issues of the day. The deficit model, in which there is a one-way transmission of scientific knowledge from scientists to the public, has been critiqued, and scientists are now being urged to be more engaged with their audiences and participate in two-way communication. Narrative is one approach that scholars are advocating for to foster this dialogue. While narrative approaches to science communication appear promising, there is limited empirical research into the impacts of narrative modes of communication about scientific issues. What narrative is, how to measure it, and how to measure its impacts are all relatively undeveloped in the field of science communication. In this dissertation I describe the current state of the science regarding narrative in science communication, and then develop and test approaches to studying the integration of narrative in the dialogue of science communication. In the first paper I conduct systematic reviews to understand how narrative is being defined and measured across scientific fields. This helps to create a foundational understanding of what scientists think narrative means and how it can be measured. The results suggest that many scientists do not explicitly or implicitly define narrative. Additionally, scholars primarily measure narrative in one of three ways: stylistically, structurally, or intuitively. I argue that for the field of science communication to have a more robust understanding of the function of narrative, it must take a systematic approach to defining, identifying, and measuring narrative. In the second paper I develop and test a novel methodological approach to testing the impacts of narrative on comprehension and recall of scientific information. The results from this study suggest that narrative may in fact be distracting if the communication goal is to increase consumers’ recall and comprehension of scientific information. In the third chapter I explore how a community can use narrative to relate their research and lived experiences to scientists in the context of a dialogue approach to communication. This research demonstrates how participatory modeling can give communities a way to structure their thoughts, develop recovery actions, and communicate with those in charge of crisis recovery efforts. By providing a synthesis of the field and methodological recommendations, this dissertation helps develop a theoretical and empirical foundation for continued research into the uses of narrative in science communication. TABLE OF CONTENTS LIST OF TABLES ............................................................................................................................................. vi LIST OF FIGURES .......................................................................................................................................... vii Introduction .................................................................................................................................................. 1 1. Dissertation Outline .............................................................................................................................. 4 WORKS CITED ................................................................................................................................................ 8 Chapter 1. A Narrative of Narrative in Science Communication ................................................................ 11 1. Introduction ........................................................................................................................................ 11 2. Methods .............................................................................................................................................. 14 2.1. Defining Narrative ........................................................................................................................ 14 2.2. Measuring Narrative .................................................................................................................... 15 3. Results ................................................................................................................................................. 15 3.1. Defining Narrative ........................................................................................................................ 15 3.2. Measuring Narrative .................................................................................................................... 16 4. Discussion ............................................................................................................................................ 17 4.1. Structural Measurements ............................................................................................................ 19 4.2. Stylistic Measurements ................................................................................................................ 23 4.3. Intuitive Measurements ............................................................................................................... 25 5. Conclusions ......................................................................................................................................... 27 APPENDIX .................................................................................................................................................... 29 WORKS CITED .............................................................................................................................................. 31 Chapter 2. Rethinking Integrating Narrative into Science Communication................................................ 37 1. Introduction ........................................................................................................................................ 37 1.1 Narrative Treatments .................................................................................................................... 39 2. METHODS ............................................................................................................................................ 41 2.1. Design ........................................................................................................................................... 42 2.2. Research Question ....................................................................................................................... 42 2.3. Survey Instrument ........................................................................................................................ 43 2.4. Survey Administration .................................................................................................................. 45 2.5. Analysis ........................................................................................................................................ 46 3. Results ................................................................................................................................................. 49 3.1. Descriptive Statistics .................................................................................................................... 49 3.2. Comparative Analysis ................................................................................................................... 50 4. Discussion ............................................................................................................................................ 55 4.1. Limitations .................................................................................................................................... 57 4.2. Future Research ........................................................................................................................... 59 5. Conclusions ......................................................................................................................................... 60 APPENDICES ................................................................................................................................................ 61 APPENDIX A Survey ................................................................................................................................. 62 APPENDIX B Abstracts ............................................................................................................................. 72 WORKS CITED .............................................................................................................................................. 78 iv CHAPTER 3. Translating Community Narratives into Semi-quantitative Models to Understand the Dynamics of Socio-environmental Crises .................................................................................................... 84 1. Introduction ........................................................................................................................................ 84 2. The Flint Water Crisis .......................................................................................................................... 85 3. Methods .............................................................................................................................................. 89 4. Results ................................................................................................................................................. 95 5. Discussion .......................................................................................................................................... 100 WORKS CITED ............................................................................................................................................ 102 Conclusions ............................................................................................................................................... 105 WORKS CITED ............................................................................................................................................ 108 v LIST OF TABLES Table 1. Classifications of the 34 articles that measure narrative and link it to learning, engagement, or other outcomes. .......................................................................................................................................... 30 Table 2. The sample size for each narrative treatment. ............................................................................. 42 Table 3. Demographic characteristics of the sample population. .............................................................. 46 Table 4. Mean and standard deviation of scientific recall, narrative engagement, and perception of scientists for each abstract treatment within each context. ...................................................................... 49 Table 5. Predictors of scientific recall for each context. ............................................................................. 50 Table 6. Translating quotes from workshop participants into concepts and relationships into a fuzzy cognitive map in the Mental Modeler software. ........................................................................................ 91 vi LIST OF FIGURES Figure 1. Dialogue model of science communication. Adapted from Faller 2019........................................ 3 Figure 2. Frequencies of the most commonly used words in definitions of “narrative” based on the review of 31 articles. Common English words were removed prior to analysis. ........................................ 16 Figure 3. Frequency of the three categories of narrative measurement, from the review of 34 articles in the field of science communication. The frequencies add to 35 because one article uses both intuitive and stylistic measurements approximately equally. .................................................................................. 17 Figure 4. Story grammar model depicting the elements and structure of a simple story. Adapted from Stein et al. (1979). ....................................................................................................................................... 20 Figure 5. Causal chain (a) and causal network (b) examples. Events within the causal chain or network are indicated with circles, and “dead-end” events are numbers with no circle. The direction of the causal relationship is indicated with an arrow. Adapted from Trabasso and van den Broek (1985) (a) and Trabasso and Sperry (1985) (b). .................................................................................................................. 22 Figure 6. Mean scientific recall and summary word count in the biofilm context. .................................... 51 Figure 7. Mean scientific recall and summary word count in the sea level context. ................................. 52 Figure 8. Mean scientific recall and summary word count in the blue mind context. ............................... 52 Figure 9. Co-occurrence networks for each abstract in the biofilm context. Nodes represent discrete pieces of scientific information, as described in section 2.3.1. Edge thickness is determined by the number of times the information of the connected nodes are included in a summary; in other words, the thicker the edge, the more often the two pieces of information co-occur in respondent summaries. .... 53 Figure 10. Co-occurrence networks for each abstract in the sea level context. Nodes represent discrete pieces of scientific information, as described in section 2.3.1. Edge thickness is determined by the number of times the information of the connected nodes are included in a summary; in other words, the thicker the edge, the more often the two pieces of information co-occur in respondent summaries. .... 54 Figure 11. Co-occurrence networks for each abstract in the blue mind context. Nodes represent discrete pieces of scientific information, as described in section2.3.1. Edge thickness is determined by the number of times the information of the connected nodes are included in a summary; in other words, the thicker the edge, the more often the two pieces of information co-occur in respondent summaries. .... 54 Figure 12. Timeline of the major events in the Flint water crisis. .............................................................. 87 vii Figure 13. Simplified model-building process using participant quotes. The quotes are translated into concepts and relationships between concepts and placed into a fuzzy cognitive map using Mental Modeler software. ...................................................................................................................................... 93 Figure 14. Venn diagram showing the causes of the FWC identified by each of the four workshops. Three of the four workshops identified the Governor’s pro-business administration and a loss of local agency in decision-making as causes of the crisis. ...................................................................................................... 95 Figure 15. Venn diagram showing the consequences of the FWC identified by each of the four workshops. All workshops identified increased household labor, health complications, and community health as consequences. Financial costs, uncertainty, and stress were identified by three of the four workshops. .................................................................................................................................................. 96 Figure 16. Venn diagram showing the potential solutions to the FWC. All workshops believed that reparations should be provided, and three workshops also suggested replacing household pipes and installing whole house filters. ..................................................................................................................... 96 Figure 17. Results of implementing the scenario of replacing all the household pipes. “Lead pipe replacement” was the concept turned on. Lead exposure, emotional stress, and uncertainty are decreased relative to other concepts. Quality of life and outside businesses are increased relative to other concepts. ........................................................................................................................................... 97 Figure 18. Results of implementing the scenario of training and using a local workforce to replace household pipes and install filters. “Local workforce” and “workforce training” were the concepts turned on. Flint sees gains to infrastructure health, return on investment, educational outcomes, and the local economy. Marginalization by race and by socio-economic status are both decreased. ............................ 98 Figure 19. Results of implementing the scenario of reducing marginalization by race. Flint sees gains to trust, the local economy, educational outcomes, community health, and quality of life. Lead exposure is reduced, and emotional stress and daily household labor are also decreased. ........................................ 99 viii Introduction As scientists increase their engagement with the public (Parks and Takahashi 2016, Dahlstrom 2014, Nisbet and Scheufele 2009), it has become clear that traditional modes of science communication are no longer sufficient to help us collaborate to solve the complex scientific and environmental issues of the day. For some time, the deficit model, in which there is a one-way transmission of scientific knowledge from scientists to the public, has been critiqued (Sturgis and Allum 2004, Trench 2006, Nielsen et al. 2007, Nisbet and Scheufele 2009, Dudo and Besley 2016), and scientists are urged to be more engaged with their audiences and participate in two-way communication (Davies 2008, Dahlstrom 2014, Dudo and Besley 2016). Studies have shown that participation in dialogues between scientists and the public can not only increase scientific knowledge, but can also lead to a more robust understanding of the scientific process itself, and the concomitant ethical, social, economic, and political implications of science (Nisbet and Scheufele 2009). Communicators now acknowledge that scientific information is necessary but not sufficient to provoke public participation and engagement in the scientific process and problem-solving (Davis et al. 2018). Individuals and groups each have their own unique mental models with which they perceive any complex problem, which in turn influences how they interpret scientific information (Longnecker 2016). Scientific education then, is not accomplished simply through dissemination of information, as learning is an iterative, personal process in which learners integrate new information with their previous knowledge, values, and beliefs (Deans for Impact 2015, Fraser 1998). This is in direct contrast with the earlier deficit model of science communication, and raises new challenges about how to stimulate a dialogue between scientists and stakeholders. However, despite movement away from the deficit model, science communication scholars still largely talk about science communication as a way that scientists can better bring their research to the public (Lorono-Leturiondo et al. 2018, Metcalfe 2019). While this is one important goal of science 1 communication, it continues to neglect the interpretive process of learning. Additionally, by only focusing on one-way communication, scientists are unable to capture expert knowledge that individual and community stakeholders may hold; this local knowledge is of particular importance as we strive to address wicked socio-environmental problems such as climate change, resource management, and human health (Lorono-Leturiondo et al. 2018, Otto-Banaszak et al. 2011, Singer et al. 2017). In light of this, it’s important to take an engaged approach to science communication; in addition to studying how scientists can improve their communication with the public, we must also study how the public can bring their research, in terms of lived experience and local knowledge, to scientists. The dialogue model is a proposed framework of science communication that recognizes the necessity of continued, multi-way stakeholder engagement (Metcalfe 2019, Trench 2008). Figure 1 depicts the dialogue model of science communication in which there are conversations between the various stakeholders. I have positioned myself in the space between the experts (scientists) and society. Specifically, I introduce methods to study how communities can use narrative to transmit their local knowledge to scientists, and how scientists can use narrative to transmit their knowledge to others. While this research does not engage in a dialogue model of science communication, it presents methods that can be used in such a model. 2 Figure 1. Dialogue model of science communication. Adapted from Faller 2019. In addition to shifting the conceptual framework that we apply to science communication, scholars are investigating how different modes of communication impact the interpretation of that communication. Narrative is one communication approach that is currently popular amongst scientists and communicators, but even as it is being advocated for (Dahlstrom 2014, Hinyard and Kreuter 2007, National Academies of Science, Engineering, and Mathematics 2017, Reddy 2009, Cronin 2010, Dudo and Besley 2016, AGU100 2018), there is limited empirical research into the impacts of narrative modes of communication about scientific issues (Kahan 2013). Additionally, results from the few empirical studies that have been done have been somewhat mixed, suggesting that it is still unclear how, when, and with whom narrative may be a useful form of science communication. Finally, what narrative means to both scientists and science communicators is not yet clear; nor is how narrative can be empirically measured and used to test hypotheses. 3 The objective of this dissertation is to first, describe the current state of the science regarding narrative in science communication, and second, develop and test approaches to studying the integration of narrative in the dialogue of science communication. I propose that scholars both continue to empirically test the impacts of narrative when communicating to the public, and suggest a systematic method to do so. Additionally, I elucidate an approach to translating individual or community narratives in semi-quantitative models to help people articulate their own experiences and perceptions. This is particularly useful in times of socio-environmental crises, when communities may feel that their voices are not being heard by those with decision-making power. Overall, this dissertation contributes to the young and growing body of research investigating narrative in science communication by providing a foundation of the field, and methodological approaches to studying it. 1. Dissertation Outline In this dissertation I disentangle the use of narrative in science communication to provide a more robust foundation for methodological approaches to studying it. I situate my work in the space between experts and society, and use empirical and case study approaches to investigate how narrative can be used to help scientists better communicate with communities, and communities to better communicate with scientists and others with decision-making power. This research highlights the need for systematic, empirical approaches to understanding and using narrative in science communication. The dissertation is split into three chapters, each of which explores the use of narrative in science communication. The chapters are intended to be published as separate manuscripts. In Chapter 1, I synthesize the current state of narrative research and how it fits into the field of science communication. I conduct a systematic review to understand how narrative is being defined across scientific fields. This helps to create a foundational understanding of what scientists think narrative means, which varies from individual to individual. The results suggest that many scientists do not explicitly or implicitly define narrative, and therefore rely on their audience’s interpretation of what 4 narrative means, which may be quite different. From those scientists who do define narrative, I was able to develop a simple definition that accounts for the most common elements found in their definitions. Next, I conduct a systematic review to understand how science communication scholars are measuring narrative. The results suggest that scholars primarily measure narrative in one of three ways: stylistically, structurally, or intuitively. Based on these findings, I suggest that researchers clearly define narrative and define criteria for how they are measuring narrative before conducting any empirical study. In Chapter 2, I develop and test a novel methodological approach to testing the impacts of narrative on comprehension and recall of scientific information, engagement with the text, and perceptions of scientists. Before we can assess impacts of narrative, we must first develop ways to measure the narrativity of a text. While effects of narrative communication have been studied (e.g., Golding et al. 1992, Gross 2008, Mattila 2000, Negrete and Lartique 2010), many empirical studies have tended towards simply calling a text either narrative or non-narrative, or more or less narrative, without any quantification or qualification of what that actually means. Here I isolate specific aspects of narrative – simile; what I call simple narrative, which introduces a setting, research motivation, and potential impacts or future research; and characterization – and test the impacts of these aspects on narrative engagement, perceptions of scientists, and recall of scientific information. The results from this study suggest that narrative may in fact be distracting if the communication goal is to increase consumers’ recall and comprehension of scientific information. I also found no significant impacts of narrative on narrative engagement or interest in the topic, or on people’s perceptions of scientists. Indeed, when asked to summarize the information they were provided, respondents who read the control abstract with no narrative generally included significantly more scientific information than did the respondents who read the more narrative abstracts. This may be due to the lack of narrative in the abstracts, or it may be due to the bullet point format utilized in the 5 control abstract; either way, it indicates a need for additional research. This research develops and tests one approach to empirically researching the impacts of narrative in a science communication format, and I suggest that this type of approach be refined and applied to additional contexts and types of communication. These results can help scientists better know when and how to employ narrative techniques when communicating with the public. In Chapter 3, I explore how a community can use narrative to relate their research and lived experiences to scientists and people with decision-making power. For science communication to engage in dialogue, as opposed to one-way transmission, it is imperative that we develop strategies for fostering communication between scientists and communities. Participatory modeling is one approach that can facilitate a dialogue between communities or individuals and scientists. Although participatory approaches to crisis recovery often use environmental modeling, explicit ways in which stakeholders’ narratives and experiences can be translated into computer-based models for scenario analysis are not readily available to modelers or decision-makers (Singer et al. 2017). In this chapter I use a case study approach to understand how participatory modeling can harness narrative. Specifically, I work with members of the Flint, Michigan community to understand how they perceive the causes, consequences, and solutions to the crisis of having lead in their drinking water. I present an approach to translating community narratives about crisis events using fuzzy cognitive mapping in a participatory approach. This research demonstrates how participatory modeling can give communities a way to structure their thoughts, develop recovery actions, and communicate with those in charge of crisis recovery efforts. Results suggest that translating narrative knowledge and experiences into semi-quantitative models offered a way for Flint residents to identify the causes, consequences, and solutions to the water crisis, and to broaden and articulate their understanding of the issue as recovery efforts were organized. 6 By drawing upon research from diverse disciplines, such as rhetoric, political science, health communication, computer science, education, and others, I demonstrate the need for new methods to aid in our understanding of narrative in science communication. It’s imperative that we understand the various ways in which narrative may be helpful or may be distracting depending on how and we it is used. By providing a synthesis of the field and methodological recommendations, this dissertation helps develop a theoretical and empirical foundation for continued research into the uses of narrative in science communication. 7 WORKS CITED 8 WORKS CITED AGU100. (2018). AGU Narratives Webinar: Telling the Story of our Science. https://connect.agu.org/events/event-description?CalendarEventKey=844b705d-100b-4d6d- b2ad-ca54285b3186&Home=%2Fhome. Cronin, K. (2010). "The “citizen scientist”: reflections on the public role of scientists in response to emerging biotechnologies in New Zealand." East Asian Science, Technology and Society 4(4): 503-519. Davies, S. R. (2008). Constructing communication: Talking to scientists about talking to the public. Science Communication, 29(4), 413-434. Dahlstrom, M. (2014). Using narratives and storytelling to communicate science with nonexpert audience. Proceedings of the National Academy of Sciences, 111(4): 13614-13620. Davis, L., Fähnrich, B., Nepote, A. C., Riedlinger, M., & Trench, B. (2018). Environmental communication and science communication—conversations, connections and collaborations. Environmental Communication, 12(4), 431-437. Deans for Impact. (2015). The Science of Learning. Austin, TX: Deans for Impact. Dudo, A., & Besley, J. C. (2016). Scientists’ prioritization of communication objectives for public engagement. PloS one, 11(2), e0148867. Faller, A. (2019). Dialogue vs. Deficit: Strategies for Science Communication. https://www.colorado.edu/earthlab/2019/05/13/dialogue-vs-deficit-strategies-science- communication-0. Fraser, B. J. (1998). 5.1 Science learning environments: Assessment, effects and determinants. International handbook of science education, 527-564. Golding, D., Krimsky, S., & Plough, A. (1992). Evaluating risk communication: Narrative vs. technical presentations of information about radon. Risk Analysis, 12(1), 27-35. Gross, K. (2008). Framing persuasive appeals: Episodic and thematic framing, emotional response, and policy opinion. Political Psychology, 29(2), 169-192. Hinyard, L. J., & Kreuter, M. W. (2007). Using narrative communication as a tool for health behavior change: a conceptual, theoretical, and empirical overview. Health Education & Behavior, 34(5), 777-792. Kahan, D. M. (2013). Making climate-science communication evidence-based—all the way down. www.ssrn.com. Longnecker, N. (2016). An integrated model of science communication—More than providing evidence. Journal of Science Communication, 15(5), Y01. 9 Lorono-Leturiondo, M., O'Hare, P., Cook, S., Hoon, S. R., & Illingworth, S. (2018). Give me five!–reasons for two-way communication between experts and citizens in relation to air pollution risk. Advances in Science and Research, 15, 45-50. Mattila, A. S. (2000). The role of narratives in the advertising of experiential services. Journal of Service Research, 3(1), 35-45. Metcalfe, J. (2019). Comparing science communication theory with practice: An assessment and critique using Australian data. Public Understanding of Science, 0963662518821022. National Academies of Sciences, Engineering, and Medicine. 2017. Communicating Science Effectively: A Research Agenda. Washington, DC: The National Academies Press. https://doi.org/10.17226/23674. Negrete, A., & Lartigue, C. (2010). The science of telling stories: Evaluating science communication via narratives (RIRC method). Journal of Media and communication studies, 2(4), 98. Nielsen, KH, Kjaer, CR and Dahlgaard, J (2007) Scientists and science communication: A Danish survey. Journal of Science Communication 6(1): 1-12. Nisbet, M. C., & Markowitz, E. M. (2015). Expertise in an age of polarization: Evaluating scientists’ political awareness and communication behaviors. The ANNALS of the American Academy of Political and Social Science, 658(1), 136-154. Otto-Banaszak, I., Matczak, P., Wesseler, J., & Wechsung, F. (2011). Different perceptions of adaptation to climate change: a mental model approach applied to the evidence from expert interviews. Regional environmental change, 11(2), 217-228. Parks, P., & Takahashi, B. (2016). From Apes to Whistleblowers: How Scientists Inform, Defend, and Excite in Newspaper Op-Eds. Science Communication, 38(3), 275-302. Reddy, C. (2009). Scientist citizens. Science, 323(5920), 1405-1405. Singer, A., Gray, S., Sadler, A., Olabisi, L. S., Metta, K., Wallace, R., Lopez, M. C., Introne, J., Gorman, M., & Henderson, J. (2017). Translating community narratives into semi-quantitative models to understand the dynamics of socio-environmental crises. Environmental Modelling & Software, 97, 46-55. Sturgis, P and Allum, N (2004) Science in society: re-evaluating the deficit model of public attitudes. Public understanding of science 13(1): 55-74. Trench, B (2006) Science communication and citizen science: how dead is the deficit model? In IX International Conference on Public Communication of Science and Technology (PCST), Seoul, Korea. 10 Chapter 1. A Narrative of Narrative in Science Communication 1. Introduction Studies have shown that communication between scientists and the public can not only increase scientific knowledge, but can also lead to a more robust understanding of the scientific process itself, including an understanding of the concomitant ethical, social, economic, and political implications of science (Dudo and Besley, 2016; Nisbet and Scheufele, 2009). Historically, the dominant approach to science communication has relied upon the deficit model, which is predicated on the assumption that a one-way dissemination of science will lead to increased scientific understanding (Nisbet and Scheufele 2009; Dickson, 2005). In this model, it is assumed that everyone who is told the same facts will interpret them in similar ways. Over time, however, this model has been widely critiqued (e.g., Dudo and Besley, 2016; Nisbet and Scheufele, 2009; Nielsen et al., 2007; Trench, 2006; Sturgis and Allum, 2004). In particular, critics of the deficit model have demonstrated that an individual’s interpretation of science is based upon the ways science is embedded in her own political, cultural, and social contexts, and therefore each individual’s interpretation of the same “facts” varies widely (Sturgis and Allum, 2004). More recently, communications scholars have called for more dialogue between scientists and the public, relying upon a back-and-forth mode of communication instead of a one-way transmission (Entradas et al., 2019; Dudo and Besley, 2016; Nisbet and Scheufele, 2009). The scientific community has responded by diversifying their modes of communication to include texts such as science blogs, podcasts, opinion pieces, art, social, and other modes of communication beyond more traditional dissemination of research through scientific journals (see e.g., Langan et al., 2019; MacKenzie, 2019; Nicholson et al., 2019). In addition to educating the public, these newer forms of communication are used to further a variety of communication goals: career advancement (Nisbet and Markowitz, 2015), increasing scientific credibility and building trust with the public (Goodwin and Dahlstrom, 2014; Dietz, 2013), and influencing policy (Parks and Takahashi, 2016; Sommer and Maycroft, 2008). As scientists 11 increase and diversify their public engagement, it is important that we understand how to best communicate with the public depending on the communication goals. Some scholars have proposed narrative as a useful way to guide dialogues with the public (AGU100, 2018; National Academies of Science, Engineering, and Mathematics, 2017; Dahlstrom, 2014; Hinyard and Kreuter, 2007), but research about how exactly to integrate narrative is lacking (National Academies of Sciences, Engineering, and Medicine, 2017). Indeed, what narrative means, and how it can be best utilized, remains under-studied (Baldwin, 2016; Dahlstrom, 2014). In the field of science communication, narrative has often been treated as a monolithic entity that is either present or absent (e.g., Negrete and Lartigue, 2010; Busselle and Bilandzic, 2009; De Wit et al., 2008; Golding et al., 1992), and how narrative is defined and measured has been both vague and diverse (Rudrum 2005). To make science communication evidence-based, and to ensure that science communicators are utilizing narrative in the best ways for their particular communication goals, we must first understand what narrative is and how to measure it. In this article we investigate two research questions: • RQ1: How is narrative defined in scientific disciplines? Amongst the calls for using more narrative in scientific communication (AGU100, 2018; National Academies of Science, Engineering, and Mathematics, 2016; Dahlstrom, 2014; Hinyard and Kreuter, 2007), it is first and foremost imperative that scholars and communicators are clear about what narrative means. This is particularly important because, unless they have studied narrative, the majority of scientists have likely developed their own ideas of what a narrative is based on their particular experiences with narrative. While scientists seem to open to embracing narrative as a form of communication, as evidenced by the increasing number of publications1, it is sometimes unclear how 1 A Web of Science search for the term “narrative” in the title of “social science” and “science technology” research domains produces around 500 results in the year 2000, and almost 3,000 results in 2018, and demonstrates a consistent upward trend since around 1975. 12 they understand narrative and how they choose to measure narrative, which leads to our second research question: • RQ2: How is narrative measured in the field of science communication? Measuring narrative provides a starting point for understanding how and why narrative might be useful in science communication. The understanding that we are predisposed to cognitively process narratives (e.g., Young and Saver, 2001; Schank, 1995; Brooks, 1992) has contributed to an increase in research that explores the use and impacts of narrative in science communications. However, much of this research relies on inconsistent or sometimes missing explanations of what and what is not narrative. Often, texts are labeled narrative if they either follow some sort of intuited narrative arc (e.g., Negrete and Lartigue 2010, Mattila, 2000; Shanahan et al., 1999; Golding, 1992), or if they use language that we generally associate more with stories than with science (e.g., Rakedzon, 2017; Baram-Tsabari and Lewenstein, 2013). Without a systematic approach to identifying and measuring narrative, we are left to simply intuit whether or not something is narrative, which does not help us identify which aspects of narrative may be more or less effective for different communication goals. To that end, it is useful to explore how narrative has been represented and measured in the past, and how it might be represented and measured in the future. To address these research questions we performed systematic reviews of the literature to better understand how scientists are defining and measuring narrative. This helps us understand how narrative is being understood and researched in the field of science communication, and allows us to propose new avenues of research to maximize the potential of narrative as a tool for scientists and science communicators. 13 2. Methods 2.1. Defining Narrative We performed a Web of Science search for articles with “narrative” in the title and limited the search to articles labeled as “highly cited in field,” which resulted in 61 articles. We used Web of Science’s algorithm to categorize the articles into the following disciplines: behavioral sciences, psychology, health care sciences and services, communication, business economics, environmental sciences and ecology, sociology, general internal medicine, information science, and neuroscience. We eliminated all articles that incorporated narrative as part of a “narrative review.” We then took the top four cited articles from each discipline, and were left with 19 articles (repeated citations were removed). To create a more robust dataset, and one with an emphasis on science communication, we next performed a Web of Science search for articles with narrative in the title and in the subject of science communication. This yielded 18 additional articles. Six of these articles were removed due to a lack of relevance (e.g., narrative does not refer to a text) or redundancy with the Web of Science search. That left us with a final dataset of 31 articles that included narrative in the title and had a focus on investigating the nature of narrative as interpreted in the sciences. To extract definitions of narrative, we searched for the term “defin” (to account for both “define” and “definition”), and if that was not present “narrative,” to account for any attempts to define or describe a definition of narrative. To analyze the specific definitions, we removed common English words (e.g., and, the, or) and combined singular and plural versions of the same word to avoid redundancy. We then performed a frequency count of the specific words used in the stated definition of narrative. In cases where there was no specific definition of narrative, we examined the text to see if there were implied or suggested definitions of narrative, and analyzed these for patterns and themes. 14 2.2. Measuring Narrative We began our search using the results found through the previous Web of Science search using “narrative” in the title and “science communication,” which gave us a starting point of four articles that specifically measured narrative and empirically tested it against impacts. We then used the bibliographies of these four articles to develop a larger set of articles in the science communication field. We reached a saturation point (i.e., we stopped finding unique articles in the bibliographies) when we had 34 articles that empirically link narrative to impacts (e.g., retention, comprehension, engagement, persuasion). These articles approached science communication from a range of scientific disciplines. To understand how the authors measured narrative, we looked for the criteria they reported when discussing the narrativity of their texts. We listed the criteria for each article, and then combined all the criteria to look for patterns. Based on the patterns that emerged, we were able to group the measurements into three broad categories, and then classify each paper based on which category it belongs to. 3. Results 3.1. Defining Narrative Of the 31 articles, 18 did not explicitly define narrative, relying instead on the assumption that the reader would know what narrative means. Of these 18 articles, five spent enough time discussing narrative that readers would have an idea of what the authors meant by the term. For example, Dahlstrom (2010) focused on the internal causal structure of narratives, suggesting that causation is an integral aspect of narrative. Rose et al. (2018) suggested that narratives are similar to frames, or the broader societal contexts in which something can be expressed. The remaining three articles (Kelly and Russo, 2018; Bessi et al., 2015; Eldson-Baker, 2015) treated narratives even more broadly by suggesting that narratives are systems of many stories that transcribe negotiated social identities. This treatment is reminiscent of sociologist Margaret Somer’s argument that “all of us come to be who we are… by being 15 located or locating ourselves (usually unconsciously) in social narratives rarely of our own making” (Somers, 1994). Thirteen of the articles provided a concrete definition of narrative. Figure 2 shows the frequencies of the nine most common words used in the definitions. Based on this review, we developed a synthetic definition of narrative that captures the major elements contained across these 13 articles: ‘A particular story in a place that consists of events that effect characters over time and involves conflict,’ or less stringently, as ‘a sequence of events that occur to and with a particular person (or people) in a particular place (or places).’ Now that a definition of narrative has been established, we can being to measure narrative based on specific criteria. Frequency of Words used in "Narrative" Definition 8 7 6 5 4 3 2 1 0 Figure 2. Frequencies of the most commonly used words in definitions of “narrative” based on the review of 31 articles. Common English words were removed prior to analysis. 3.2. Measuring Narrative Three broad categories of narrative measurement emerged from the survey of the literature: structural, stylistic, and intuitive. Each research article generally measured narrative in a particular way. Structural measurements are concerned with the causal connections between episodes in a narrative. Stylistic measurements focus on the way in which a narrative is told, identifying and measuring 16 elements like perspective, voice, sentence connectivity, settings, and more. Intuitive measurements are more precisely a lack of measurement, in which texts are interpreted as either narrative or not narrative (or as more or less narrative) without any specific support, but based on the researchers’ own assumptions about what constitutes narrative. The majority of the 34 articles measured narrative in a single broad way, either structurally, stylistically, or intuitively, though a few of them used more than one type of measure. As shown in Figure 3, 13 studies used a primarily structural approach to measuring narrative, 15 used a primarily stylistic approach, and seven used a primarily intuitive approach. The specific categorizations for each article can be seen in the Appendix. Below we discuss the three measurement categories in more detail and review strengths and weaknesses of these measures. Frequency of Narrative Measurement Categories 16 14 12 10 8 6 4 2 0 Intuition Structure Style Figure 3. Frequency of the three categories of narrative measurement, from the review of 34 articles in the field of science communication. The frequencies add to 35 because one article uses both intuitive and stylistic measurements approximately equally. 4. Discussion Our review of the literature highlights that many researchers assume that their audience already knows what a narrative is, and they therefore do not define it. However, it is likely that individuals have at least slightly different perceptions of narrative, with some focusing on meta-narratives or archetypal 17 narratives, others focusing on causal structure, still others focusing on stylistic elements, and a final group assuming narrative is equivalent to story. If not clearly defined, it is possible that none of these assumptions are what the author is intending, and that his or her own assumptions about what narrative means are entirely different than those of the reader. Indeed, Medin and Bang (2014) argue that science communication relies on artifacts (e.g., text, visualizations, images) that reflect the communicator’s own cultural orientations and assumptions. Orientations and assumptions that hold true in one culture may not hold true in other cultures (Kim and Nehm, 2011). Studies have shown that cultural associations affect what and how people think (e.g., Wu and Keysar, 2007; Masuda and Nisbett, 2001). These differences are apparent also in how people communicate about the world. For example, members of the Menominee Indian Tribe have said that when they tell a story, the goal is to put a picture in the listener’s head so that she can experience a first-person perspective (Medin and Bang 2014). We argue that as science communicators and scholars begin to both use and study narrative more, they must, at the very least, grapple with the definition of narrative and be clear how they are using the term narrative. As mentioned above, our review of the literature suggests that narrative be defined as: “a sequence of events that occur to and with a particular person (or people) in a particular place (or places).” While this may not be the definition of narrative, it is a good place to start, integrating the concepts of time, place, and characters. Without a concrete definition of narrative, we lack an opportunity for shared understanding and empirical investigation. Only by clearly defining narrative can we avoid misrepresentations and miscommunication, and learn how to best harness narrative as a powerful tool in science communication. Most importantly, we argue that researchers explicitly define narrative in their own studies to avoid misinterpretation and confusion. This will ensure that a reader does not assume narrative includes, for example, a setting and conflict, when the researcher is assuming that those elements are not necessary for a text to be narrative. 18 Once narrative has been defined, we can approach measuring it in order to link narrative with specific communication goals. Measuring narrative can provide a measurement of how narrative something is, or in what ways something is narrative. This allows for comparisons between different levels and types of narrativity. There have been several attempts to measure narrative that provide fruitful starting points for how measuring narrative may be accomplished, but each measurement method relies on different definitions of narrative and is interested in different elements of narrative, creating a lack of cohesion in how to measure narrative. Three general ways to measure narrative emerged out of our literature review: stylistic, structural, and intuitive. Below, we provide a detailed overview about how narrative has been measured in each of these categories and review their strengths and weaknesses. 4.1. Structural Measurements Much of the research on the structural measurement of narrative aims to understand how we cognitively process and understand narrative. One assumption is that our memory is organized around our experiences, or episodes, as opposed to around semantic categories. These episodes consist of propositions linked within a particular event or time span (Rubin, 2006; Young and Saver, 2001; Schank and Abelson, 1977). Structural measurements of narrative often seek to understand how the structure of a story relates to how we cognitively encode the story (e.g., Trabasso and van den Broek, 1985; Nezworksi et al., 1982; Trabasso et al., 1982; Stein et al., 1979). Two common, and related, ways to measure narrative structure are story grammars and causal networks or chains. Story grammars measure narrative based on internal structures known as schemata (van Dijk and Kintsch, 1983; Schank and Abelson, 1977; Bartlett, 1932). These schemata are organized in episodic ways, and can be analyzed according to a set of rules used to produce stories. There are multiple story grammar models (e.g., Stein and Glenn, 1979; Mandler and Johnson, 1977; Thorndyke, 1977; Rumelhart, 19 1975), and while each one is slightly different, the general concepts are similar: a story grammar includes specific types or categories of story components, which are then linked together in episodes. The story grammar models include the following components, and each story generally includes most of or all of the components : 1) setting (time, location), 2) initiating event (obstacle, moral dilemma, 3) internal response (how a character responds to the event), 4) attempt (an attempt to solve the problem, 5) direct consequence (the success or failure of the attempt), and 6) reaction (the character’s emotional response at the outcome) (Merritt and Liles, 1987; Trabasso and van den Broek, 1985). An episode is comprised of, at a minimum, an initiating event or internal response, an attempt, and a direct consequence (Stein et al., 1979). Figure 4 shows the story grammar model, in which the story itself is depicted as the setting + an episode, which consists of the story components. Figure 4. Story grammar model depicting the elements and structure of a simple story. Adapted from Stein et al. (1979). Story grammars have been used to study story comprehension and recall, particularly among children, as well as to test how children create unique stories (e.g., Alves et al., 2015: Merritt and Liles, 1987; Miller et al., 2018; Stein and Glenn, 1975). In empirical studies, children as young as six were able to recall and comprehend stories with a clear story grammar, and children as young as nine are able to 20 develop stories that include the generic story grammar components (Merritt and Liles, 1987). Gordon and Braun (1982) (as cited in Gordon and Braun, 1983) demonstrated that fifth-grade schoolchildren were better able to recall stories after being trained in story grammar. More specifically, certain grammar components have been shown to consistently be better recalled in memory tests: settings, initiating events, goals, and consequences (Trabasso and van den Broek, 1985). While the story grammar model has been shown to be useful, particularly in youth education, it does have limitations in its ability to comprehensively measure narrative. Story grammars only apply to stories with a single protagonist with a single overarching goal (Black and Wilensky, 1979), and it is difficult to apply them to stories with many actors and complex episodes (Mani, 2012). While story grammar does allow for minor episodes, its simplicity is somewhat limiting when applied to complex narratives. Additionally, Black and Wilensky (1979) argue that story grammars are limited in their ability to help with story understanding and comprehension, as they leave out the context needed to understand a story and the semantic processes necessary for both comprehension and understanding. Finally, few studies offered convincing analyses of intercoder reliability, and examples of anyone other than the original researchers using these grammars are hard to identify (Mani, 2012). One way that structural models have expanded upon the story grammar model is through causal chain models. The causal chain functions on the assumption that the primary relations in a story are causal, and are linked by causal inferences (Trabasso and Sperry, 1985; Stein and Glenn, 1979; Mandler and Johnson, 1977; Rumelhart, 1977, among others). A narrative, then, can be parsed into its individual events, often based on story grammar categories, and a chain of events is developed, linked through causal relationships (Dahlstrom, 2010; Trabasso and Sperry, 1985; Trabasso and van den Broek, 1985; Trabasso et al., 1982). The causal relationships are identified through “logical criterion of necessity” (Mackie, 1974), such that there is a causal relationship if Element B could not occur without Element A (Dahlstrom, 2010). Causal chains are then created assuming that the chain begins with the introduction 21 of the setting, character, and initiating event, and ends with the outcome of the protagonist’s goal. The causal path is then traced between the beginning and end, and any events that do not have logical causes or that do not lead to the closing event are considered “dead-ends.” The causal chain is generally the path with the longest chain of events (Trabasso and van den Broek, 1985). An example of how to visually represent a causal chain is shown in Figure 5a. Figure 5. Causal chain (a) and causal network (b) examples. Events within the causal chain or network are indicated with circles, and “dead-end” events are numbers with no circle. The direction of the causal relationship is indicated with an arrow. Adapted from Trabasso and van den Broek (1985) (a) and Trabasso and Sperry (1985) (b). Trabasso et al. (1982) extended the causal chain model into a causal network model, in which separate causal strands form networks, as opposed to a linear chain. Figure 5b shows an example of what a causal network of a story looks like. It is less linear than the causal chain models, though there is often still a main causal chain, but with a more networked structure. Both causal chain and causal network models have been used in memory experiments. Studies have shown correlations between the causal structure of a story and narrative understanding and memory (e.g., Schwartz and Trabasso, 1988; Stein and Trabasso 1982). Using a causal structure to examine narrative, Dahlstrom (2010) showed that assertions placed at causal locations in a narrative 22 were seen as more “truthful” than assertions placed in non-causal locations. He also demonstrated that there was significantly better recall of information placed in causal locations than in non-causal locations. Trabasso and van den Broek (1985) showed that the centrality of an event, or the number of connections it has to other events in the story, is correlated with recall, with the more closely connected events leading to higher recall, both immediately and after a time lapse. While both story grammars and causal models have been used to assess comprehension and recall in empirical studies, they may sometimes suffer from being overly precise and difficult to evaluate (Sebesta et al., 1982; Black and Wilensky, 1979). Although these early narrative grammars identified important aspects of stories, it is very hard to apply them to stories with many actors and complex episode graphs, and they have suffered from a lack of intercoder reliability (Mani, 2012). In addition, though measuring narrative based on its structure has provided contributions to the field of narrative and discourse research, it neglects the stylistic components of narrative, which we explore in the next section. 4.2. Stylistic Measurements Narrative style includes the non-structural elements of storytelling, such as voice, tone, and perspective. While style is recognized as an important aspect of narrative (e.g., Dawson, 2013; Lavelle, 1997; Banfield, 1973), it is difficult to define and measure. For example, Bucchi (2013) identifies important elements of style: lightness, exactitude, visibility, multiplicity, quickness, and consistency, but what these terms mean or how they can be useful for science communication remain uncertain. It is still unclear how important style elements like voice, description, and metaphor are to science communication. There are, however, measurements of style that are easily quantifiable, such as sentence length, word length, and voice (active or passive), and these elements have been used as measurements in several studies (e.g., Hartley et al., 2002; Bostian, 1983; DeVito, 1969). These measurements generally 23 assume that shorter words and sentences, and active voice sentences, are easier to understand. Other studies have incorporated these simple measures of style as parts of more nuanced measurements of narrative style (e.g., Hillier et al., 2016; Sharon and Baram-Tsabari, 2014; Baram-Tsabari and Lewenstein, 2013; Petersen et al., 2008). Additionally, other easily-quantifiable elements of style are commonly used as aspects of a broader stylistic analysis. These include jargon (Rakedzon et al., 2017; Sharon and Baram- Tsabari, 2014; Baram-Tsabari and Lewenstein, 2013) and the use of conjunctions (Hillier et al., 2016). Hillier et al. (2016) for example, in their analysis of abstracts dealing with climate change, included measurements of setting, sensory language, and appeal, along with the more easily measured conjunctions, connectivity (referential linking between sentences), and narrative perspective (first- or third-person). The setting was measured simply by its presence or absence, though what the definition of a setting is has grown more complex and nuanced over time (Hones, 2011), and it may sometimes be difficult to claim with certainty that there is or is not a setting. Similarly difficult to define is sensory language, defined by Hillier et al. (2016) as that which expresses “emotions, attitudes, beliefs, and interpretations,” and appeal, which is defined as the moral orientation of a narrative. While Hillier et al. (2016) used crowdsourcing as a novel form of defining and measuring these ideas, they warn that these narrative elements, particularly appeal, are not necessarily well understood. Indeed, this is the difficulty for measurements of narrative style. Either the specific elements are not easily defined and measured, or what constitutes “good” or “bad” use of an element is controversial. Baram-Tsabari and Lewenstein (2013) face these difficulties in their assessment instrument for science writing. They include humor and metaphors, among other attributes. While, humor and metaphor can likely be assessed for presence or absence, the “goodness” of a metaphor or an instance of humor is not so easily assessed. In addition, when narrative is measured only through style and not through structure, it neglects some important aspects of narrative. 24 4.3. Intuitive Measurements The third broad type of narrative measurement is what we are calling intuitive measurement, which is based not on specific elements of narrative, whether structural or stylistic, but on the intuition that one piece of communication is more or less narrative than another. These types of measurement have commonly been used in empirical studies that investigate the impact of narrative on audiences, generally in terms of comprehension or interest in the topic (Goldman, 2012; Lartigue, 2010; Mattila, 2000; Negrete and Shanahan et al., 1999). This intuitive measure of narrative is predicated on the assumption that we share a cultural or societal understanding of what narrative is. However, while there is likely some societal consensus on what makes a text more or less narrative, the lack of specific criteria for identifying and measuring narrative makes it difficult to assess which aspects of communication are effective in terms of the messaging goals. For example, Golding et al. (1992) tested the impacts of a narrative newspaper series about radon versus a technical series. The narrative piece included specific characters and dialogues between them, while the technical piece used the passive voice and was primarily drawn from existing Environmental Protection Agency documents. The results showed that the narrative piece was able to hold people’s attention longer, but the non-narrative piece was more successful at relaying the scientific information. While it is clear that one of these pieces was more narrative than the other, the authors did not specify criteria for why one piece was more narrative than the other, and that makes our interpretation of the results less meaningful than they could be. Similarly, Negrete and Lartigue (2010) explored the effects of narrative communication, in the shape of excerpts from published pieces of fiction, compared to a bulleted list of facts drawn from the narrative. They measured the impacts of the communication using the RIRC (Retell, Identify, Remember, Contextualize) method, which measures learning on four dimensions. Their findings indicate that narrative communication may lead to increased retention, but not increased understanding. Again, however, this study was designed based on the assumption that a published work of fiction is narrative 25 and a bulleted fact sheet is not. While this assumption is likely valid, it makes testing for the effects of narrative difficult, as we cannot parse out which structural or stylistic elements contributed to the potential increases in retention, and which ones may have not had any impact. Other studies have carried out similar sorts of research, in which they analyze comprehension, perception, or recall of information within a “narrative” text and a “non-narrative” text (e.g., Saenz and Fuchs, 2002; Mattila, 2000; Shanahan et al., 1999). What these studies have in common is that their criteria for what makes something narrative is never stated. It is certainly useful to test how narrative impacts learning outcomes, and using texts that we call narrative based on our individual understanding of what that means is one way to approach this. However, we argue that by including specific criteria for what makes something more or less narrative, we can then better link specific aspects of narrative to learning impacts or achievement of communication goals. It is clear that each of these methods to measure narrative has their strengths and weaknesses, and is more or less useful to use based on the research questions and methods employed. While both more structural and more stylistic ways to represent and measure narrative have been developed and studied, the diversity of measurement methods indicates that the field of narrative research in the sciences is somewhat murky. Our results suggest that while many scholars are measuring narrative either structurally or stylistically, many others are relying on the assumption that there are narratives that are culturally accepted as being more or less narrative than others. Furthermore, very few scholars are measuring narrative both stylistically and structurally. There are surely reasons for this, as measuring narrative at all is difficult, and measuring based on both stylistic and structural elements integrates even more variables. However, we believe this is a worthwhile endeavor for science communication scholars so that we can better understand how specific elements of narrative work individually and together. At the least, criteria for measuring narrative should be clearly stated and linked to the researcher’s definition of narrative. 26 5. Conclusions Science communicators and scientific organizations are calling for more narrative in science communication (AGU100, 2018; National Academies of Science, Engineering, and Mathematics, 2017; Dahlstrom, 2014; Hinyard and Kreuter, 2007). However, with this increased focus on narrative comes the recognition that it is a contested field of study. What exactly narrative is, and what it means for science communication, is still unclear. To that end, we have provided systematic reviews to better understand how scientists are defining and measuring narrative. Definitions of narrative are often vague and sometimes in conflict, but it is important to explore what exactly narrative consists of so that we can better understand how to best use narrative in science communication. While differing definitions and differences in what elements of narrative are important are to be expected, it is imperative that science communication scholars articulate how they are defining and measuring narrative when performing empirical research. We have categorized narrative measurements into three broad categories: structural, stylistic, and intuitive. While each of these modes has its merits, a comprehensive way of measuring narrative would include both structural and stylistic elements and be very clear about what these measures are. Measuring narrative should be grounded in empirical research, and before narrative is accepted as a norm in science communication, its impacts on communication should be further studied. Specifically, we suggest that science communication scholars take a systematic approach to identifying, defining, and measuring narrative. This will provide science communicators with evidence that can help them best utilize narrative to target certain communication goals, whether that may be education, engagement, or trust. Indeed, it may be that narrative can at times be distracting or detrimental to achieving particular communication goals. Results of empirical studies have often been mixed, and there is some evidence that narrative communication is sometimes less effective than more traditional forms of science communication (e.g., Kromka and Goodboy, 2019; Negrete and Lartigue, 27 2010; Golding et al., 1992). It may be that narrative is useful in certain situations, but should be avoided in others. The only way to determine this is with a scientific approach to the use and impacts of narrative in science communication, with particular care to address linkages between specific elements of narrative and specific outcomes (i.e., scientific comprehension, interest, perceptions about science and scientists, engagement, and science literacy, among others). In particular, we recommend that the following research questions are addressed by the science communication community in more detail: • How is narrative defined within and across scientific disciplines? • How can narrative be qualitatively and quantitatively measured? • What are the impacts of different aspects of narrative on specific learning impacts or communication goals? Because the study of narrative in science communication is still relatively new and spans scientific disciplines, we are seeing calls for increased narrative without a robust, evidence-based foundation. If we are able to isolate certain aspects of narrative, whether that is using metaphors, or characters, or structuring stories in a certain way, science communicators will be better placed to successfully use narrative and achieve their communication goals. Early evidence suggests that narrative can influence how science communication is received and interpreted (e.g., Kromka and Goodboy, 2019; Dahlstrom, 2010; Negrete and Lartigue, 2010; Golding et al., 1992). However, these early studies must be regarded as preliminary until we can establish more rigorous approaches. 28 APPENDIX 29 Table 1. Classifications of the 34 articles that measure narrative and link it to learning, engagement, or other outcomes. Article Aaronson 1977 Ash 2007 Aavrominou and Osborne 2009 Baram-Tsabari and Lewenstein 2013 Black et al. 1979 Bostian 1983 Busselle and Bilandzic 2009 Dahlstrom 2010 Dahlstrom and Rosenthal 2018 De Wit et al. 2008 Downs 2014 Fraser 2004 Golding et al. 1992 Hartley 2003 Hartley et al. 2002 Hartley et al. 2004 Hillier et al. 2016 Jones 2014 Jones and McBeth 2010 Mattila 2000 McCance et al. 2001 Negrete and Lartigue 2010 Nezworski et al. 1982 Norris et al. 2005 Pennington and Hastie 1992 Petersen et al. 2008 Rakedzon et al. 2017 Rakedzon and Baram-Tsabari 2017 Shanahan et al. 1999 Sharon and Baram-Tsabari 2014 Stern 1991 Stewart and Nield 2013 Trabasso and ven den Broek 1985 Trabasso et al. 1982 Turner and Bower 1979 Total Narrative Measurement Type Structural x x x x x x x x x x x x x 13 Intuitive x x x x x x x 7 Stylistic x x x x x x x x x x x x x x x x 15 30 WORKS CITED 31 WORKS CITED AGU100 (2018) AGU Narratives Webinar: Telling the Story of our Science. Available at https://connect.agu.org/events/event-description?CalendarEventKey=844b705d-100b-4d6d- b2ad-ca54285b3186&Home=%2Fhome (accessed 13 March 2019). Alves, KD, Kennedy, MJ, Brown, TS, and Solis, M (2015) Story Grammar Instruction with Third and Fifth Grade Students with Learning Disabilities and Other Struggling Readers. Learning Disabilities--A Contemporary Journal, 13(1). Baldwin, C (2016) Introduction to special issue: narrative across disciplines. Narrative Works 6(1). Banfield, A (1973) Narrative style and the grammar of direct and indirect speech. Foundations of language 10(1): 1-39. Baram-Tsabari, A and Lewenstein B (2013) An Instrument for Assessing Scientists’ Written Skills in Public Communication of Science. Science Communication 35(1): 56-85. Bartlett, FC (1932) Remembering: An experimental and social study. Cambridge: Cambridge University. Bessi, A Coletto, M, Davidescu, GA, Scala, A, Caldarelli, G and Quattrociocchi, W (2015) Science vs conspiracy: Collective narratives in the age of misinformation. PloS one 10(2): e0118093. Black, JB and Wilensky, R (1979) An evaluation of story grammars. Cognitive science 3(3): 213-229. Bostian, LR (1983) How active, passive and nominal styles affect readability of science writing. Journalism quarterly 60(4): 635-670. Brooks, P (1992) Reading for the plot: Design and intention in narrative. Harvard University Press. Bucchi, M (2013) Style in science communication. Public Understanding of Science 22(8): 904-915. Busselle, R and Bilandzic, H (2009) Measuring narrative engagement. Media Psychology 12(4): 321-347. Dahlstrom, MF (2010) The role of causality in information acceptance in narratives: An example from science communication. Communication Research 37(6): 857-875. Dahlstrom, MF (2014) Using narratives and storytelling to communicate science with nonexpert audience. Proceedings of the National Academy of Sciences 111(4): 13614-13620. Dawson, P (2013) Style, the Narrating Instance, and the “Trace” of Writing. Style 47(4): 466-489. De Wit, JB, Das, E and Vet, R (2008) What works best: objective statistics or a personal testimonial? An assessment of the persuasive effects of different types of message evidence on risk perception. Health Psychology 27(1): 110. DeVito, JA (1969) Some psycholinguistic aspects of active and passive sentences. Quarterly Journal of Speech 55(4): 401-406. 32 Dickson, D (2005) The case for a ‘deficit model’ of science communication. SciDev.net: 27. Dietz, T (2013) Bringing values and deliberation to science communication. Proceedings of the National Academy of Sciences 110(Supplement 3): 14081-14087. Dudo, A, and Besley, JC (2016) Scientists’ prioritization of communication objectives for public engagement. PloS one 11(2): e0148867. Elsdon-Baker, F (2015) Creating creationists: The influence of ‘issues framing’on our understanding of public perceptions of clash narratives between evolutionary science and belief. Public Understanding of Science 24(4): 422-439. Entradas, M, Marcelino, J, Bauer, MW and Lewenstein, B, (2019) Public communication by climate scientists: what, with whom and why? Climatic Change: 1-17. Golding, D, Krimsky, S, and Plough, A (1992) Evaluating risk communication: Narrative vs. technical presentations of information about radon. Risk Analysis 12(1): 27-35. Goldman, C (2012) This is your brain on Jane Austen, and Stanford researchers are taking notes. Stanford Report. Goodwin, J and Dahlstrom, MF (2014) Communication strategies for earning trust in climate change debates. Wiley Interdisciplinary Reviews: Climate Change 5(1): 151-160. Gordon, CJ and Braun, C (1983) Using story schema as an aid to reading and writing. The Reading Teacher: 116-121. Hartley, J, Sotto, E and Pennebaker, J (2002) Style and substance in psychology: Are influential articles more readable than less influential ones? Social Studies of Science 32(2): 321-334. Hillier, A, Kelly, RP and Klinger, T (2016) Narrative style influences citation frequency in climate change science. PloS one 11(12): e0167983. Hinyard, LJ and Kreuter, MW (2007) Using narrative communication as a tool for health behavior change: a conceptual, theoretical, and empirical overview. Health Education & Behavior 34(5): 777-792. Hones, S (2011) Literary geography: setting and narrative space. Social and Cultural Geography 12(7): 685-699. Kelly, MP and Russo, F (2018) Causal narratives in public health: the difference between mechanisms of aetiology and mechanisms of prevention in non-communicable diseases. Sociology of health & illness 40(1): 82-99. Kim, SY and Nehm, RH (2011) A cross-cultural comparison of Korean and American science teachers’ views of evolution and the nature of science. International Journal of Science Education 33(2): 197-227. 33 Kromka, SM and Goodboy, AK (2019) Classroom storytelling: using instructor narratives to increase student recall, affect, and attention. Communication Education 68(1): 20-43. Langan, LM, Cheng, Y, and Hunka, AD (2019) Empirically supported out-of-the-box strategies for science communication by environmental scientists. Integrated environmental assessment and management. Lavelle, E (1997) Writing style and the narrative essay. British Journal of Educational Psychology 67(4): 475-482. MacKenzie, LE (2019) Science podcasts: analysis of global production and output from 2004 to 2018. Royal Society Open Science 6(1): 180932. Mackie, JL (1974) The cement of the universe: A study of causation. Oxford: Clarendon Press. Mandler, JM and Johnson, NS (1977) Remembrance of things parsed: Story structure and recall. Cognitive psychology 9(1): 111-151. Mani, I (2012) Computational modeling of narrative. Synthesis Lectures on Human Language Technologies 5(3): 1-142. Masuda, T and Nisbett, RE (2001) Attending holistically versus analytically: Comparing the context sensitivity of Japanese and Americans. Journal of personality and social psychology 81(5): 922- 934. Mattila, AS (2000) The role of narratives in the advertising of experiential services. Journal of Service Research 3(1): 35-45. Medin, DL and Bang, M (2014) The cultural side of science communication. Proceedings of the National Academy of Sciences 111(Supplement 4): 13621-13626. Merritt, DD and Liles, BZ (1987) Story grammar ability in children with and without language disorder: Story generation, story retelling, and story comprehension. Journal of Speech, Language, and Hearing Research 30(4): 539-552. Miller, RD, Correa, VI, and Katsiyannis, A (2018). Effects of a story grammar intervention with repeated retells for english learners with language impairments. Communication Disorders Quarterly, 40(1), 15-27. National Academies of Sciences, Engineering, and Medicine (2017) Communicating Science Effectively: A Research Agenda. Washington, DC: The National Academies Press. Negrete, A and Lartigue, C (2010) The science of telling stories: Evaluating science communication via narratives (RIRC method). Journal of Media and communication studies 2(4): 98. Nezworski, T, Stein, NL and Trabasso, T (1982) Story structure versus content in children's recall. Journal of Verbal Learning and Verbal Behavior 21(2): 196-206. 34 Nicholson, PJ, Dixon, D, Pullanikkatil, D, Moyo, B, Long, H and Barrett, B (2019) Malawi Stories: mapping an art-science collaborative process. Journal of Maps: 1-9. Nielsen, KH, Kjaer, CR and Dahlgaard, J (2007) Scientists and science communication: A Danish survey. Journal of Science Communication 6(1): 1-12. Nisbet, MC and Markowitz, EM (2015) Expertise in an age of polarization: Evaluating scientists’ political awareness and communication behaviors. The ANNALS of the American Academy of Political and Social Science 658(1): 136-154. Nisbet, MC and Scheufele, DA (2009) What’s next for science communication? Promising directions and lingering distractions. American journal of botany 96(10): 1767-1778. Parks, P and Takahashi, B (2016) From Apes to Whistleblowers: How Scientists Inform, Defend, and Excite in Newspaper Op-Eds. Science Communication 38(3): 275-302. Petersen, DB, Gillam, SL and Gillam, RB (2008) Emerging procedures in narrative assessment: The Index of Narrative Complexity. Topics in language disorders 28(2): 115-130. Rakedzon, T, Segev, E, Chapnik, N, Yosef, R and Baram-Tsabari, A (2017) Automatic jargon identifier for scientists engaging with the public and science communication educators. PloS one 12(8): e0181742. Rose, DC, Brotherton, PM, Owens, S and Pryke, T (2018) Honest advocacy for nature: presenting a persuasive narrative for conservation. Biodiversity and Conservation 27(7): 1703-1723. Rubin, DC (2006) The basic-systems model of episodic memory. Perspectives on Psychological Science 1(4): 277-311. Rudrum, DC (2005) From narrative representation to narrative use: Towards the limits of definition. Narrative 13(2): 195-204. Rumelhart, DE (1975) Notes on a schema for stories. In Representation and understanding. Morgan Kaufman, pp.211-236. Ryan, ML (2007) Toward a definition of narrative. In The Cambridge companion to narrative. Pp.22-35. Saenz, LM and Fuchs, LS (2002) Examining the reading difficulty of secondary students with learning disabilities: Expository versus narrative text. Remedial and Special Education 23(1): 31-41. Schank, RC (1995) Tell me a story: Narrative and intelligence. Chicago: Northwestern University Press. Schank, RC and Abelson (1977) Scripts, Plans, Goals, and Understanding: An Inquiry into Human Knowledge. Schwartz, RM and Trabasso, T (1988) Children's understanding of emotions. Emotions, cognition, and behavior: Psychology Press. 35 Sebesta, SL, Calder, JW and Cleland, LN (1982) A story grammar for the classroom. The reading teacher 36(2): 180-184. Shanahan, J, Pelstring, L and McComas, K (1999) Using narratives to think about environmental attitude and behavior: An exploratory study. Society & Natural Resources 12(5): 405-419. Sharon, AJ and Baram-Tsabari, A. (2014) Measuring mumbo jumbo: A preliminary quantification of the use of jargon in science communication. Public Understanding of Science 23(5): 528-546. Somers, MR (1994) The narrative constitution of identity: A relational and network approach. Theory and society 23(5): 605–649. Sommer, B and Maycroft, JR (2008) Influencing Public Policy: An Analysis of Published Op-Eds by Academics. Politics & Policy 36(4): 586-613. Stein, NL and Glenn, CG (1975) An Analysis of Story Comprehension in Elementary School Children: A Test of a Schema. Stein, NL, Glenn, CG and Freedle, R (1979) New directions in discourse processing. New directions in discourse processing. Sturgis, P and Allum, N (2004) Science in society: re-evaluating the deficit model of public attitudes. Public understanding of science 13(1): 55-74. Thorndyke, PW (1977) Cognitive structures in comprehension and memory of narrative discourse. Cognitive psychology 9(1): 77-110. Trabasso, T, Secco, T and van den Broek, P (1982) Causal cohesion and story coherence. In H. Mandl, Stein, N and van den Broek, P (eds) Learning and comprehension of text (83–112). Hillsdale, NJ, USA: Lawrence Erlbaum Associates. Trabasso, T and Sperry, LL (1985) Causal relatedness and importance of story events. Journal of Memory and language 24(5): 595-611. Trabasso, T and van Den Broek, P (1985) Causal thinking and the representation of narrative events. Journal of memory and language 24(5): 612-630. Trench, B (2006) Science communication and citizen science: how dead is the deficit model? In IX International Conference on Public Communication of Science and Technology (PCST), Seoul, Korea. Van Dijk, TA, Kintsch, W and van Dijk, TA (1983) Strategies of discourse comprehension. Wu, S and Keysar, B (2007) The effect of culture on perspective taking. Psychological science 18(7): 600- 606. Young, K and Saver, JL (2001) The neurology of narrative. SubStance 30(1): 72-84. 36 Chapter 2. Rethinking Integrating Narrative into Science Communication 1. Introduction In the past several years, communication scholars and scientific organizations have been advocating for the use of narrative in science communication (AGU100 2018, Cronin 2010, Dahlstrom 2014, Dudo and Besley 2016, Hinyard and Kreuter 2007, National Academies of Science, Engineering, and Mathematics 2017, Reddy 2009). While narrative approaches to science communication appear promising, there is limited empirical research into the impacts of using narrative modes of communication about scientific issues, which limits out understanding of how narrative can further communication goals (Kahan 2013). To that end, we propose that science communication scholars take a more systematic approach to measuring narrative and testing how narrative may affect the desired outcomes of communication goals. In the field of science communication, narrative has often been treated as a monolithic entity that is either present or absent (e.g., Busselle and Bilandzic 2009, De Wit et al. 2008, Golding et al. 1992, Negrete and Lartigue 2010); how narrative is defined and measured in this literature has been both vague and diverse (Singer et al. under review). The ways in which the outcomes of narrative usage are measured have been similarly broad and not always well-defined. For example, Avraamidou and Osborne (2009) point out that the value of narrative in science rests on the assumptions that narrative enhances memory, interest, and understanding (Norris et al. 2005), as well as student motivation and enjoyment (Solomon 2002). Dahlstrom (2014) suggests that narrative can increase comprehension, interest, and engagement. However, to date there is a lack of empirical testing into measuring narrative and linking it to measured learning impacts, which contributes to a lack of cohesion and consistency in the field of science communication. This, in turn, limits our ability to know when and how narrative should be used and what outcomes can be reasonably expected when it is employed. Rarely have specific narrative elements been tested against specific outcomes, with some notable exceptions. Kahan (2013) argues that “science communicators should now use valid empirical 37 methods to identify which plausible real-world strategies for counteracting [public conflict] actually work” (pg. 1). This research begins to fill that gap by using an experiment to test how specific narrative elements influence recall and comprehension, perception of scientists, and narrative engagement. To provide empirical evidence about the effectiveness of narrative in science communication, we first need a way to measure narrative, as opposed to simply claiming that a text is either narrative or non- narrative, as many previous studies have done (e.g., Golding 1992, Mattila 2000, Negrete and Lartigue 2010, Shanahan et al. 1999). To accomplish this we need to be able to differentiate between specific elements of narrative, so that these elements can be empirically tested to understand which narrative elements may help scientists and scientific institutions better accomplish their communication goals. Narrative has generally been measured either structurally, stylistically, or intuitively (Singer et al. in review). This experiment focuses on the stylistic elements of narrative, but it offers a methodological approach that can be adapted to test many aspects of narrative, including structural elements. We have chosen to focus primarily on stylistic elements for two reasons: 1) it is likely easier for people to integrate style elements, such as humor, metaphor, descriptive language, point-of-view, etc. into communication than it is to integrate structural changes based on causal structures, and 2) based on Singer et al.’s (under review) review, narrative is most often measured stylistically. Here we isolate specific elements of narrative style and tests these elements against learning impacts. We chose to test the addition of a character, the use of simile, and the use of simple narrative, all of which will be described in more detail below. These elements have all been linked to learning impacts, though results have generally been mixed (e.g., Cunningham 1976, Harris and Mosier 1999, Stern 1991). This research also isolates specific outcomes that narrative may be associated with, including scientific comprehension and recall, engagement and interest with the science, and perceptions of scientists, which are all important communication goals (Busselle and Bilandzic 1999, Golding et al. 1992, Shanahan et al. 1999, Trabasso 1982). Below, we briefly describe the experimental 38 design, and then the justification for our narrative measurement and learning outcomes. We then define our particular research questions and hypotheses before detailing our experimental design. To isolate elements of narrative we drew upon literature across diverse disciplines, and to test for learning impacts we developed a survey instrument. Specifically, we developed brief abstracts from different scientific fields (i.e., medical research, psychology, and human-environment interactions) followed by a survey that tests recall and comprehension, engagement and interest in the content, and perceptions of scientists. Abstracts have frequently been used as texts for analysis in the field of science communication and writing analysis (e.g., Freeling et al. 2019, Hartley 2003, Hillier et al. 2016, Stotesbury 2003). We used abstracts because they are a common way that scientists use to summarize and present their research to other scientists, and because they generally include the most salient information and are the first sections of a paper read (Hillier et al. 2016). Abstracts are designed to present the main points of a research study as concisely and objectively as possible (Stotesbury 2003). Additionally, abstracts are short communications, and therefore relatively easy for people to read and internalize, and can be manipulated for experimental design. 1.1 Narrative Treatments Here we define narrative as “a sequence of events that occur to and with a particular person (or people) in a particular place (or places)” (Singer et al., under review). Based on previous studies, we chose to test what we call simple narrative, simile, and the use of character, all described in more detail below. 1.1.1. Simple Narrative For the simple narrative abstracts, we included small amounts of narrative information to provide both a setting and a simple narrative arc: motivation for the research or importance of the issue, the research or issue itself, and the potential outcomes or implications of the research or issue. This simple narrative included the elements of a story grammar, which includes: 1) a setting (time, location), 2) an initiating event (obstacle, moral dilemma, 3) internal response (how a character 39 responds to the event), 4) an attempt (an attempt to solve the problem, 5) a direct consequence (the success or failure of the attempt), and 6) a reaction (the character’s emotional response at the outcome) (Merritt and Liles 1987, Trabasso and van den Broek 1985). These pieces form a narrative arc. As we manipulated the bullet point abstracts into the simple narratives, we kept the sentences from the bullet point abstracts the same to the extent possible, and simply provided additional information. 1.1.2. Simile Previous studies have looked into the influence of figurative language on comprehension (Harris and Mosier 1999), though results of this research have been mixed. Some studies suggest that metaphors and similes increase reading comprehension or memory (e.g., Harris and Mosier 1999), while others suggest that figurative language is more difficult to comprehend (e.g., Cunningham 1976), or that results are dependent on the surrounding text and context (e.g., Inhoff et al. 1984). In our pilot study, the abstract that incorporated similes in the description of the research was correlated with significantly higher levels of scientific recall than the other abstracts. To test the influence of similes, we introduced two similes that described scientific process into each abstract, building upon the simple narrative treatments. For example, we introduced a Trojan horse simile to represent the process by which clay “tricks” a bacteria into opening its cell wall and allowing iron into the cell, which then kills the cell. 1.1.3. Character Scientific research in the past several decades has generally been communicated through an objective, third-person, passive lens (Aaronson 1977, Hartley et al. 2002). As narrative approaches spread throughout scientific communication, however, incorporating characters is one method often used to connect with the audience. For example, first-person narrators elicit trust from the audience because they appear to reveal confidences, personal values, and intimacies in a way that a third-person narrator does not (Eliot 1957, Martin 1986, Stern 1991). A third-person narrator, on the other hand, gains credibility based on its assumed omniscience (Kenney 1988, Stern 1991). Incorporating characters, whether fictional or real, into science communication may help audiences develop an emotional 40 connection with the science or scientist (Jones 2014). Spoel et al. (2008) point out that characterization can be used to develop trust in a narrator or scientist, and inspire participation in scientific discourse. Using hero characters in particular has the potential to impact individuals’ perceptions of risk and policy preferences when information is presented through a narrative (Jones 2014). Characters can either be in the form of the scientist doing the research, or as a protagonist of a story. Characters have particularly been incorporated into narrative policy research, which argues that every policy narrative must have characters, who act as either heroes or villains (Jones 2014, McBeth et al. 2005). Because our pilot study suggested no significant differences when scientists as characters were introduced, we focused on non-scientist characters in this study, developing fictional characters with a vested interest in the research. The abstracts with characters were identical to the simple narrative abstracts except for the addition of a character and his or her motivations for being a part of the research process. 1.1.4. Character + Simile The final narrative treatment incorporates both the character and similes. We took the character abstract and inserted both similes from the simile abstract into the same locations. 2. METHODS We developed a control abstract that included none of these elements and was modeled after traditional scientific abstracts. We then integrated the test narrative treatments into each the control abstract. For this study we incorporated this definition by including a setting and generic scientist characters. We then specifically chose to test more stylistic elements of narrative beyond this simple framework because of the ease with which scientists could integrate such elements into their own writing. To provide a robust sample, we ran this experiment in three different context areas, each with its own set of control and treatment abstracts. One context dealt with using clay to treat bacterial biofilms (hereafter referred to as the “Biofilm” context). Another context dealt with the threat of sea 41 level rise and climate change on a coastal community in Alaska (hereafter referred to as the “Sea level” context). The final context dealt with how being in or near water affects mental well-being (hereafter referred to as the “Blue mind” context). All treatments can be found in Appendix B. The control abstract for each context was written in a bullet point format in a traditional scientific mode, using passive voice and objectivity (Aaronson 1977, Chang 2011, Hartley et al. 2002). The jargon was kept to a minimum to ensure it can be understood by the largest possible audience. Bullet points have been used in several studies that compare non-narrative and narrative writing about science (e.g., Carriger 2011, Negrete and Lartique 2010, Maurer and Gierl 2014), suggesting that bullet points are a traditional mode for communicating scientific information. Supporting this assumption is that fact that bullet points are often used on fact sheets disseminated by scientific agencies such as the Environmental Protection Agency (EPA), National Oceanic and Atmospheric Administration (NOAA), and the Ecological Society of America (ESA), among others. 2.1. Design Each context has either three or five treatments. Table 1 shows these treatments. The biofilm context was not given a character treatment (and therefore no simile + character treatment) because of the focus on the pure science aspect of the research, as opposed to the human interaction aspect of it. All other contexts were tested across all five treatments. This design allows for comparisons both within and across contexts. Table 2. The sample size for each narrative treatment. Treatment Bullet points Simple narrative Simile Character Simile + character Biofilm n=103 n=102 n=101 Sea Level n=103 n=99 n=102 n=100 n=96 Blue Mind n=100 n=98 n=103 n=98 n=96 2.2. Research Question Our driving research question is: What specific narrative elements are associated with higher or lower rates of scientific recall, interest in the subject, and perceptions of scientists? 42 Based on the previous literature and the results of our pilot study, we propose the following hypotheses: • H1: The introduction of narrative into an abstract will be correlated with increases in participants’ comprehension and recall of scientific information. • H2: The introduction of narrative into an abstract will be correlated with higher levels of participants’ engagement with the text. • H3: The introduction of narrative into an abstract will be correlated with more positive perceptions of scientists amongst participants. 2.3. Survey Instrument We designed the survey instrument to test the relationships hypothesized above regarding the impact of narrative on scientific recall and comprehension, engagement with the content of the abstracts, and perception of scientists. The survey began with a series of demographic questions to collect information on education, age, ethnicity, race, and gender. Next, the survey presented the abstract with instructions to carefully read it. Similar to other recall experiments, respondents then were asked to solve three simple arithmetic problems that served as a distractor (e.g., Barriga et al. 2010, Dahlstrom 2010, Shapiro 1986). There were then three multiple choice questions about the abstract that served as attention checks, and respondents who incorrectly answered all three were removed from the sample. The respondents were then asked to summarize the abstract in their own words, being sure to include everything they could remember. Next, they answered a series of questions designed to test their engagement and perception of scientists. The full survey can be seen in Appendix A. 2.3.1. Recall and Comprehension The open-ended summary question was designed to tests respondents’ comprehension and recall of the abstract. We are primarily concerned with the lower-order cognitive skills required for knowledge and comprehension (Anderson et al. 2001, Bloom 1956, Crowe et al. 2008). Knowledge proficiency is the ability to identify and recall information, and comprehension proficiency is the ability 43 to describe or explain in your own words the information (Crowe et al. 2008). Recall tasks in which participants are asked to write down everything they remember, similar to what we asked in our summary question, emphasize comprehension because respondents must reconstruct their understanding of the passage. These types of summary tasks can also elucidate if something has been misunderstood (Shohamy 1984, Wolf 1993). Respondents may exhibit recall by including particular pieces of information, or even exact wording, from the abstract. For example, when similes were included many respondents included the precise simile in their summary response. Individuals may exhibit comprehension by using different terms to describe a phenomena from the abstract, or to extrapolate beyond the abstract itself. For example, in the sea level abstract, the albedo effect was described, but not called the albedo effect. Some respondents referred to it as the albedo effect in their summary, indicating comprehension of that particular phenomenon. Shohamy (1984) argues that open-ended summary questions require respondents to exhibit both comprehension and response construction. Similarly, Yonelinas (2002) claims that open-ended questions require that an individual be able to recollect information, as opposed to simply being familiar with it. This type of recollection task requires that respondents use their mental models to construct a sensical response (McCarthy et al. 2018). 2.3.2. Narrative Engagement We measured narrative engagement using a revised version of Appel et al.’s (2015) narrative transportation scale to measure how “lost in the story” a participants felt when reading the abstract (Gerrig 1993, Green and Brock 2000, Nell 1988). Narrative engagement occurs when “all mental systems and capacities become focused on events occurring in the narrative” (Green and Brock 2000, pg. 701). Participants were asked a series of three questions that measure aspects of engagement on a 5-point scale from “strongly disagree” to “strongly agree.” 44 2.3.3. Perception of Scientists To measure people’s perceptions of scientists, we adapted Hendriks et al.’s (2015) Muenster Epistemic Trustworthiness Inventory (METI) to include those items relevant to scientists. The METI was designed to be used in an online context with little information about the source’s credibility. Adapting this scale to be used to assess perceptions of a researcher from only an abstract is therefore appropriate. We provided respondents a 5-point scale to rate scientists on multiple characteristics associated with expertise, integrity, and benevolence. The scale ranged from, for example, “dishonest” to “honest” and “unethical” to “ethical.” 2.4. Survey Administration The survey was administered using Qualtrics via Amazon’s Mechanical Turk (MTurk), a popular website for crowdsourcing tasks that require human intelligence, known as Human Intelligence Tasks (HITs) (Buhrmester et al. 2011, Paolacci et al. 2010). MTurk allows researchers to recruit human workers to complete a task, commonly something such as transcribing audio, identifying images, or completing surveys. MTurk has become popular with social scientists, as it allows them to recruit samples with reduced costs and increased participant diversity and flexibility compared to more traditional recruitment methods, without any loss in data quality (Goodman and Paolacci 2017). For this study, we recruited approximately 100 respondents for each treatment, for a total of 1301 participants. Table 2 shows the breakdown of demographic characteristics of the sample. This sample size is similar to or larger than other studies that empirically investigate the impacts of narrative (e.g., Dahlstrom and Rosenthal 2018, Negrete and Lartigue 2010). Data were collected from January to March of 2019. Throughout the data collection process we followed best practices for MTurk recommended by Goodman and Paolacci (2017), including paying fair wages, clearly identifying ourselves, allowing a reasonable time to complete work, maintaining responsive lines of communication with participants, and approving work quickly. To ensure quality participation and data, we used filters to only allow workers with a HIT approval rating of > 98%, at least 500 completed HITs, who live in the 45 United States, and who had not previously taken the survey. We compensated workers $1.50 for the task, which took an average of 6 minutes, for an hourly rate of $15.00. Table 3. Demographic characteristics of the sample population. Total (n=1301) Percent Gender 725 Male 571 Female 5 Other/Rather not say Age 406 18-29 496 30-39 222 40-49 122 50-59 55 60 or older Education 6 Less than high school 181 High school or equivalent 287 Some college 167 Associate degree 519 Bachelor degree 141 Graduate degree Ethnicity 989 White 103 Black or African American 4 American Indian or Alaska Native 93 Asian or Pacific Islander Hispanic or Latino 91 Native Hawaiian or Pacific Islander 1 20 Other 56 44 0 32 38 17 9 4 1 14 22 13 40 11 76 8 0 7 7 0 2 2.5. Analysis 2.5.1. Content Analysis To measure recall of scientific and narrative information, we used a variation of directed content analysis (Hsieh and Shannon 2005). We developed individual codes for each piece of information included in the abstract, and categorized these codes into broader categories. For example, in the abstract about bacterial biofilms, we had the following broad categories: biofilm definition, clay characteristics, study motivation, mechanism, methods, and results. Each of these categories included one or more specific codes. As the abstracts became more narrative, we expanded the codebook to include narrative categories. 46 The original codes were deductively derived from the information within the abstracts themselves. We iteratively parsed out discrete pieces of information any time a respondent included something new or different. For example, the first respondent may have said “Blue clay from Oregon has been tested,” and we would have coded that as a single piece of information. If the next respondent said “Clay from Oregon has been tested,” we then went back and included “clay is blue” as a separate code, and “clay is from Oregon” as a separate code. This process resulted in a codebook with pieces of information reduced into the smallest size that still provides meaningful information. Additionally, similar to Humble (2009), we inductively derived additional codes throughout the process to include information not able to be captured by our directive codebook, and iteratively revising and refining the coding scheme as new information arose. These additional codes emerged when respondents included extrapolations about the research or new narrative details that were not included in the abstracts. In particular, we developed codes for incorrect or partially correct information, if, for example if a respondent stated that the blue clay was from Idaho, as opposed to the correct location of Oregon. To quantify the proportion of scientific information presented in each respondent’s summary, we developed a weighted scheme to apply to the codes. The weights ranged from -1 to 2, depending on the accuracy of the information. Incorrect information was given a weight of -1, a 0 was given when no information was provided, partially correct, yet incomplete, information was given a 1, and fully correct information was given a 2. This allowed us to differentiate between, for example, responses that stated that the village of Shaktoolik needs money to address rising sea levels (partially correct=1) and those that precisely stated that the village needs $100 million to address rising sea levels (correct=2). We then summed the weights for each respondent and divided that number by the total possible score (and multiplied by 100) to give an overall scientific information scale, referred to as “scientific recall.” 47 Because once narrative was introduced into the abstracts many respondents began incorporating that narrative into their summaries, we developed a variable called “narrative presence.” Once narrative was integrated into the abstracts, respondents began incorporating aspects of the narrative into their summaries. For example, they may have included the character, or the simile, or aspects of the simple narrative. This is a binary variable; if the respondent included any narrative information this variable was 1, and if they did not include any narrative information it was 0. 2.5.2. Descriptive Statistics We calculated the mean scientific recall, narrative engagement, and perception of scientists for each abstract treatment, within each context. 2.5.3. Comparative Analysis To determine which independent variables impacted the scientific recall, narrative engagement, and perception of scientists, we ran stepwise linear regressions for each context. Our independent variables included the narrative treatments, the narrative presence, the word count of the abstracts, the word count of the summaries, and the demographic variables. The summary word count was the most important predictor in each stepwise regression; we predicted that to an extent, participants were also including more narrative information as their summaries grew longer. Using a box and whisker plot, we visually compared mean summary word count with mean scientific recall across all treatments in all contexts. This provides us with a more robust understanding of how summary word count interacts with the narrative treatments. To account for the fact that individuals who wrote longer summaries exhibited higher levels of scientific recall, we removed the summary word count variable and ran the stepwise regression again to understand what roles the other variables may play. Because the word counts of the abstracts increased as narrative was introduced, there was multicollinearity between abstract word count and the narrative treatments. To account for this, we ran the stepwise regression once with abstract word count as a variable and once without, and then compared the model fitness based on the adjusted R2 values. 48 To better understand how discrete pieces of scientific information were connected in the respondents’ summaries, we developed co-occurrence networks. To do this, we counted the number of times each piece of scientific information was included with each other piece of information in each summary. We then plotted these co-occurrence frequencies in a histogram and visually determined thresholds based on where the long tail of the histogram began. For each treatment this visual thresholding corresponded to a threshold of approximately 30 percent, below which we removed the co-occurrences form the network (i.e., 30 percent of the co-occurrence relationships were eliminated due to their low frequencies). Using Gephi version 0.9.2, we projected the co-occurrence networks of the abstracts. This allowed us to see how certain pieces of information and co-occurrence relationships between information decayed as narrative was introduced. 3. Results 3.1. Descriptive Statistics Table 3 shows the mean scientific recall, engagement, and perception of scientists for each abstract, based on narrative treatment. Table 4. Mean and standard deviation of scientific recall, narrative engagement, and perception of scientists for each abstract treatment within each context. Biofilm Bullet point Simple narrative Simile Sea level Bullet point Simple narrative Simile Character Character + simile Blue mind Bullet point Simple narrative Simile Character Character + simile Narrative Engagement Mean (SD) 8.39 (2.20) 8.15 (2.22) 8.00 (2.12) 7.04 (2.81) 6.94 (2.42) 7.07 (2.58) 6.90 (2.38) 7.02 (2.55) 7.73 (2.47) 7.10 (2.25) 7.06 (2.28) 7.31 (2.22) 6.68 (2.46) Perception of Scientists Mean (SD) 13.89 (4.98) 13.43 (5.10) 13.27 (4.13) 13.23 (6.11) 14.34 (5.81) 14.55 (5.88) 13.00 (4.39) 14.31 (5.19) 13.95 (5.02) 14.05 (4.99) 13.20 (4.04) 13.50 (4.77) 13.41 (4.50) Scientific Recall Mean (SD) 33.56 (19.19) 23.84 (15.20) 25.11 (14.69) 28.85 (15.55) 25.09 (14.46) 24.30 (13.45) 23.94 (12.49) 23.47 (11.73) 49.21 (21.53) 42.57 (19.51) 40.29 (20.69) 34.99 (17.63) 33.78 (19.07) 49 3.2. Comparative Analysis 3.2.1. Scientific Recall and Comprehension After comparing the model fitness with and without abstract word count as an independent variable, we chose the model with the highest adjusted R2 for each context. For the biofilm context this was the model without abstract word count, and for the sea level and blue mind contexts it was the models that included abstract word count. The differences between the adjusted R2s were minimal, suggesting that we cannot confidently distinguish between the influences of word count compared to narrative treatment. B 32.26 -10.19 *** -9.46 *** 5.82 ** 36.43 -0.05 *** Table 5. Predictors of scientific recall for each context. Biofilm SE Constant 2.29 Simple narrative Simile 2.32 2.05 Over 39 Sea level Constant Abstract word 0.01 count Narrative presence Blue mind Constant Abstract word count Female Narrative presence Over 39 * p-value<0.05, ** p-value<0.01, *** p-value<0.001 58.03 -0.11 *** 1.79 2.12 1.94 1.39 0.02 5.31 *** 3.56 * 4.50 * 4.13 * Beta -0.28 -0.26 0.16 -0.20 0.18 -0.31 0.09 0.10 0.09 Adjusted R2 0.020 0.059 0.080 0.016 0.042 0.072 0.081 0.087 0.093 Table 4 shows the results of the stepwise regression. The adjusted R2 for each model are low, indicating that these models only predict a small part of the scientific recall. Abstract word count and narrative treatment (in the sea level and blue mind, and biofilm contexts, respectively) consistently negatively influence the scientific recall, suggesting that as the abstracts get longer and have narrative introduced, the scientific recall decreases. These results do not support H1, that narrative will positively influence scientific recall. 50 However, narrative presence is correlated with higher levels of scientific recall in the sea level and blue mind abstracts, which indicates that when respondents included narrative elements in their summaries, they also included more scientific information, suggesting a partial confirmation of H1. Finally, in both the biofilm and blue mind contexts, those participants who were over the age of 39 had significantly higher levels of scientific recall. In the blue mind context females exhibited higher levels of scientific recall. We found no significant impacts from characteristics or race/ethnicity or education. Figures 6-8 illustrate the mean scientific recall and the mean summary word count for each treatment in each context. Word count is the most important predictor of scientific recall in each regression model, but figures 6-8 demonstrate that despite that, as word count increases from treatment to treatment, scientific recall decreases. Biofilm l l a c e R c i f i t n e i c S t n e c r e P 100 90 80 70 60 50 40 30 20 10 0 46 44 42 40 38 36 34 t n u o C d r o W y r a m m u S Bullet point Simple narrative Simile Scientific recall Word count Figure 6. Mean scientific recall and summary word count in the biofilm context. 51 Sea level l l a c e R c i f i t n e i c S t n e c r e P 100 90 80 70 60 50 40 30 20 10 0 Bullet point Simple narrative Simile Character Simile and character Scientific recall Word count Figure 7. Mean scientific recall and summary word count in the sea level context. Blue mind l l a c e R c i f i t n e i c S t n e c r e P 100 90 80 70 60 50 40 30 20 10 0 Bullet point Simple narrative Simile Character Simile and character Scientific recall Word count Figure 8. Mean scientific recall and summary word count in the blue mind context. 65 63 61 59 57 55 53 51 49 47 45 49 47 45 43 41 39 37 35 t n u o C d r o W y r a m m u S t n u o C d r o W y r a m m u S Co-occurrence projections for each context are shown in figures 9-11. The projections show the strengths of the relationships between each piece of information (node) and each other piece of information. Each node represents one piece of discrete scientific information. The thickness of the connecting line represents the number of times both pieces of information were included in a respondent’s summary. In other words, the thicker the line is between nodes, the more times both 52 pieces of information were mentioned in individual respondent summaries. Each figure shows the trajectory of co-occurrences as narrative is introduced. Figure 9. Co-occurrence networks for each abstract in the biofilm context. Nodes represent discrete pieces of scientific information, as described in section 2.3.1. Edge thickness is determined by the number of times the information of the connected nodes are included in a summary; in other words, the thicker the edge, the more often the two pieces of information co-occur in respondent summaries. As seen in figure 9, the information decays as narrative is introduced into the abstracts. The information in nodes L and K, which disappeared from the bullet point abstract to the simile abstract has to do with specific results of the research, namely that clay was tested in a suspension and that it is effective against Staphylococcus aureus. Respondents did continue to recall that the clay was effective in general, just not against the specific strain. We also see that the information contained in nodes G, E, and F is strongly retained and co-occurs across treatments, and is even strengthened in the simile treatment. This information deals with the mechanism of how the clay works – it opens the cell wall, lets iron in, and kills the cell. 53 Figure 10. Co-occurrence networks for each abstract in the sea level context. Nodes represent discrete pieces of scientific information, as described in section 2.3.1. Edge thickness is determined by the number of times the information of the connected nodes are included in a summary; in other words, the thicker the edge, the more often the two pieces of information co-occur in respondent summaries. In figure 10, we see that the co-occurrences rapidly decrease from the bullet point treatment to the simple narrative, and then decrease slightly more when simile and character are introduced together. The links between nodes H, A, and M stay fairly strong throughout all the abstracts; this information encapsulates the heart of the narrative arc: that temperatures are rising, leading to a threat to Shaktoolik, which can potentially be mitigated with money. Figure 11. Co-occurrence networks for each abstract in the blue mind context. Nodes represent discrete pieces of scientific information, as described in section2.3.1. Edge thickness is determined by the number of times the information of the connected nodes are included in a summary; in other words, the thicker the edge, the more often the two pieces of information co-occur in respondent summaries. In the blue mind context, information is once again lost as narrative is introduced, as exhibited in figure 11. The co-occurrence of information represented by nodes G and A stays relatively high throughout all the treatments. This information includes the general main points of the abstract: that being around water is helpful (A), and that aquatic therapy can help people with PTSD (G). 3.2.2. Narrative Engagement Based on the stepwise regression, we found no significant differences in the level of narrative engagement respondents expressed based on either demographic variables or the narrative treatment 54 they were exposed to. We also tested specifically for interest in the topic and found no significant differences based on narrative treatment or demographics. These results do not confirm H2, that the introduction of narrative contributes to more narrative engagement with the text. 3.2.3. Perception of Scientists Based on the stepwise regression, we found no significant differences in how respondents perceived scientists based on either demographic variables or the narrative treatment they were exposed to. These results do not confirm H3, that the introduction of narrative will contribute to more positive perceptions of scientists. 4. Discussion This study suggests that narrative may in fact be distracting if the goal of science communication is to help people recall and comprehend scientific information. The comparative analysis suggests that narrative may contribute to a reduced focus on the scientific information. Across all three contexts, there were significant differences in the mean scientific information recalled, which decreased as narrative was introduced and as word count increased (see table 4). Similar results have been found in other studies. For example, Negrete and Lartigue (2010) explored the learning impacts of narrative communication compared to a bulleted list of facts drawn from the narrative. They found that initially the group of participants given the fact sheet performed better, but that the difference diminished over time, suggesting that narrative communication may lead to increased retention, but not increased recall. There are indications, however, that narrative may focus attention at times. For example, it is likely that there are more- and less-effective analogies, and one difficulty for science communicators is to choose more useful analogies. As seen in figure 9, the simile used in the biofilm context appears to have been successful at highlighting how the mechanism of the clay works, as respondents retained this information more in the simile treatment (as seen in the relationships between nodes G, E, and F). These mixed results suggest the need for further research into how narrative can be productively harnessed to achieve specific communication goals. 55 The loss in recall and comprehension observed in this study could be attributed to the fact that the abstracts increased slightly in length as the narrative was introduced; however, the respondents actually wrote longer summaries as the length of the abstracts increased, as seen in figures 6-8. We can assume then that the respondents were focusing less on the scientific information and more on the narrative information as the narrativity of the abstracts increased. Kromka and Goodboy (2019) found that students exposed to a narrative lesson expressed more interest and did slightly better on a recall test, but also retained more extraneous information, suggesting that narrative may contribute to overall cognitive load. Our results also suggest that narrative presence in the respondents’ summaries is significantly correlated with higher levels of scientific recall. It may be that respondents who included narrative information in their summaries simply included more information in general, or it may be that recalling the narrative caused those individuals to also recall the science. We would need to perform more research before assigning any causation to these trends. The co-occurrence analysis provides a broader picture of the comprehension and recall of the scientific information, which is in line with the regression results. As seen in figures 9-11, the bullet point abstract is more thickly networked than any of the other treatment abstracts, across all three contexts. Certain connections remain strong across all treatments (e.g., A and G in the blue mind context), but in the bullet point abstracts there are more co-occurrences overall. This demonstrates that the introduction of narrative may distract from certain types of information, but not others, though additional research is needed to understand which types of information are likely to quickly be lost and which types are more likely to be retained, or even strengthened. We did find that age was a significant predictor of scientific recall in two of the contexts, with those participants over the age of 39 demonstrating higher levels of recall, and females showing higher levels of recall in one context. There is little literature that investigates how different groups of people might interpret narrative in science communication, and our results here indicate that this would be 56 useful research to conduct to see if these trends are upheld in other experiments. It may be that older adults are simply more invested in online research like this, or there may be environmental or neural reasons for these results. Though we predicted that introducing narrative into the abstracts would have significant effects on how engaged or interested in the subject respondents were (H2), we found no significant differences in any of the treatment groups. We hypothesized that introducing narrative, particularly characters, would personalize the abstracts and therefore cause people to become more engaged in them, as personal narrative has been extensively used in message design, particularly in public health fields (e.g., Hecht and Miller-Day 2009, Hinyard and Kreuter 2007, Miller-Day and Hecht 2013). Our results, however, did not support this hypothesis. While literature suggests that in some cases using personal stories can promote engagement and interest in science (e.g., Miller et al. 2015, Shelton et al. 2016, Spoel 2008), it is likely dependent on the mode of communication, the science itself, and the audience. H3 was also not confirmed, in that individuals showed no differences in their perception of scientists when narrative was introduced. Characters can be used to increase an audience’s trust in a scientist or in science (Spoel 2008), and previous research has shown that particular character tropes, such as heroes and villains, can shape how people perceive scientific uncertainty and risk (Jones 2014). However, we saw no significant differences in how people perceived scientists based on the narrative elements in an abstract, which indicates that narrative may only shift people’s perceptions in certain cases or when used more appropriately. 4.1. Limitations While this study offers a simple empirical approach to testing the impacts of narrative in science communication, it should be taken as a starting point to this type of research. In this study, we explored only stylistic narrative approaches, and as Singer et al. (under review) point out, narrative can be 57 measured stylistically, structurally, and intuitively. Additionally, we explored only a few elements of narrative, and there are myriad stylistic elements to narrative that should be tested. Similarly, this research was constrained by the scientific topics explored in the three context areas. While we were careful to choose diverse topic areas, it may be that narrative is more useful for communicating certain types of science. For example, perhaps narrative is useful in communicating science that deals with human-environment interactions, but is less useful when communicating about pure physics, or vice versa. Additionally, the complexity of the science may affect how useful narrative is. For example, in this study, the sea level rise abstract was the most complex based on the number of discrete pieces of information included, and it was also the context with the least diversity in scientific recall; respondents generally included proportionally less information in their summaries than in the other two contexts, but they also exhibited less loss of information across the narrative treatments. Using empirical approaches to test more diverse types of science will help identify when narrative is more or less useful. While Mechanical Turks offers a relatively inexpensive and accessible pool of respondents, it does have some limitations. MTurk participants are more likely to be white, technologically savvy, educated, and secular when compared with the general population (McDuffie 2019). Additionally, Turkers self-select into any HIT, choosing to participate in certain ones and not in others based on their interests, their pay expectations, their time, and any other criteria that go into an individual’s reason for participation. This self-selection means that researchers must be careful when making generalizations based on MTurk research. However, many of the limitations to Mechanical Turk data, such as people clicking responses quickly just to get through, or lying, are limitations to survey research in general (Stritch et al. 2017). We were able to mitigate at least some of these limitations by only accepting responses from people who provided summaries that at the very minimum were on the topic of the 58 abstract they had read. However, it is important to perform this type of empirical research with audiences beyond Mechanical Turks. 4.2. Future Research This study demonstrates a simple experimental design to test specific elements of narrative against learning, engagement, and perception. The results suggest that narrative may not lead to increases in scientific understanding, or increased interest in science, or better perceptions of scientists. However, it is only one experiment using a specific form of communication – abstracts – which may not be as conducive to narrative interventions as other forms of communication. We believe that similar experiments should be run using different measures of narrative, and different elements of narrative (e.g., descriptive language, use of settings, archetypal characters, etc.). This study tested only immediate recall and comprehension of scientific information, but there is some evidence that narrative may increase retention even in cases when it does not increase understanding (Negrete and Lartigue 2010). Future research should test not only the immediate impacts of using narrative as we have done here, but also the long-term impacts. This is of particular importance in settings where a communication goal is to foster long-term interest in science, such as educational settings. While we were careful to isolate and test very small, specific elements of narrative, there are ways to perform even more robust tests of these specific elements. For example, while we found that bulleted, non-narrative abstracts were the most effective at imparting scientific information in this study, we did not test it against bulleted, narrative abstracts. It may be that the bullet point style, rather than the lack of narrative, is what contributed to higher rates of recall and comprehension. Future research should focus not only on the content of communication, but also on the presentation. In addition, this study focused only on the use of narrative in written communication. While writing is an important aspect of science communication, scientists and communicators also rely heavily 59 on verbal and visual communication, and these methods should also be tested for narrative impacts. Empirical studies similar to this one should be applied across communication forms. 5. Conclusions This study adds to the growing body of empirical studies on the impacts of using narrative in science communication. Much of the previous research has approached these studies from a holistic perspective, in which texts are either narrative or non-narrative, without parsing out the specific elements that make a text more or less narrative (Singer et al. under review). There have been some studies that look at particular elements of narrative, such as causal structure (Dahlstrom 2010) or passive voice (Hartley et al. 2002), and this study builds on those by reducing narrative to discrete stylistic elements of simile, characterization, and simple narrative arcs. We tested the impacts of these narrative elements on individuals’ recall and comprehension of scientific information, their perception of scientists, and their engagement the subject matter. Contrary to our expectations, we found that the narrative had no significant impacts on perceptions of scientists or engagement, but that introducing narrative significantly reduced people’s recall and comprehension of scientific information, based on their summaries of the treatment abstracts. It is unclear, however, if it is the introduction of narrative elements or concomitant increases in word count that led to the reduction in scientific recall. Additionally, in our pilot study, we found that the use of similes was correlated with significant increases in recall and comprehension of scientific information. These mixed results suggest 1) that further research needs to be done, and 2) that narrative may in fact distract people from focusing on scientific information, and should therefore be used judiciously and depending on what the communication goals are. 60 APPENDICES 61 APPENDIX A Survey 62 Example Survey Q16 Thank you for participating in this study. This research project investigates how people understand scientific concepts through reading texts. You are being asked to participate in a brief reading exercise, followed by a series of simple math questions, followed by a brief survey. This should take around 8 minutes total. Your participation in the survey indicates your consent to this project. Your participation in this survey is voluntary, and you can choose to withdraw your participation at any time. You will not receive payment unless you complete the survey. There will be no identifying data collected, and all responses will be aggregated prior to analysis. Thank you again for your cooperation. For questions regarding this project, please contact: Steven Gray grayste1@msu.edu 646-915-2915 End of Block: Block 10 Start of Block: Block 13 Q2 What is your gender? o Male (1) o Female (2) o Other (please specify) (3) ________________________________________________ o Rather not say (4) 63 Q3 Which category below includes your age? o 18-20 (1) o 21-29 (2) o 30-39 (3) o 40-49 (4) o 50-59 (5) o 60 or older (6) Q4 What is the highest level of school you have completed or the highest degree you have received? o Less than high school degree (1) o High school degree or equivalent (e.g., GED) (2) o Some college but no degree (3) o Associate degree (4) o Bachelor degree (5) o Graduate degree (6) 64 Q12 Please specify the ethnicity you most identify with. o White (1) o Black or African American (2) o American Indian or Alaska Native (3) o Asian or Pacific Islander (4) o Hispanic or Latino (5) o Native Hawaiian or Pacific Islander (7) o Other (6) End of Block: Block 13 Start of Block: Block 12 Temperatures in the Arctic are rising faster than in other regions of the globe. Sea ice is white, Q1 Please Read the Following Scientific Passage The following passage describes the potential impacts of sea level rise. Please read it carefully. and as it melts there is less surface for the sun’s energy to reflect off. This causes more energy to be absorbed, leading to warmer waters and increased rates of ice melt. sea levels threaten the existence of coastal towns. sea ice and slush that protects the land from storm surges and powerful waves is decreasing. The Alaskan village of Shaktoolik faces the very real possibility of disappearing in the next decades. As of right now, residents have chosen to stay and try to develop ways to deal with the inevitable rising waters and land erosion. To safely stay, the village needs at least $100 million to spend on improvements, like an evacuation road, improvements to the water system, and a fortified shelter in case of a storm. In remote areas in Alaska, rising On Alaska’s western coast, the layer of winter End of Block: Block 12 Start of Block: Block 11 Q16 What is 4+7 ________________________________________________________________ 65 End of Block: Block 11 Start of Block: Block 11 Q16 What is 47-18 ________________________________________________________________ End of Block: Block 11 Start of Block: Block 12 Q18 What is 37+16 ________________________________________________________________ End of Block: Block 12 Start of Block: Block 5 Q8 Please think back to the scientific passage you read and summarize it to the best of your ability. Be sure to include everything that you can remember. ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ End of Block: Block 5 Start of Block: Block 2 66 Q17 Temperatures in the Arctic are rising than temperatures in other regions of the globe. o Faster (1) o Slower (2) o At the same rate (3) End of Block: Block 2 Start of Block: Block 13 Q19 Which of the following improvements does Shaktoolik need? Check all that apply. ▢ School buses (1) ▢ Evacuation road (2) ▢ Sea wall (3) ▢ Airport (4) ▢ Water system (5) ▢ Fortified shelter (6) End of Block: Block 13 Start of Block: Block 6 67 Q21 What currently protects Alaska's western coast against storm surges? o Levees (1) o Sea ice (2) o Nothing (3) o Sea walls (4) End of Block: Block 6 Start of Block: Block 8 Q15 Please choose the response that you most closely agree with. Strongly disagree (1) Disagree (2) Neither agree nor disagree (3) Agree (4) Strongly agree (5) The passage affected me emotionally. (1) I was mentally involved in the passage while reading it. (2) I wanted to learn more after reading the passage. (3) End of Block: Block 8 Start of Block: Block 13 o o o o o o o o o o o o o o o 68 Q34 In general, do you believe that scientists are.... 1 (1) 2 (2) 3 (3) (4) (5) o o o o o o o o o o o o o o o o o o Honest Competent Intelligent Well- educated Responsible Professional Experienced Ethical Important to society End of Block: Block 13 Start of Block: Block 10 o o o o o o o o o o o o o o o o o o o o o o o o o o o Dishonest Incompetent Unintelligent Not educated Irresponsible Unprofessional Inexperienced Unethical Not important to society Q22 Please choose the response that you most closely agree with. Strongly disagree (1) Disagree (2) Neither agree or disagree (3) Agree (4) Strongly agree (5) 69 I am politically more liberal than conservative. (1) o o o o o In any election, given a choice between a Republican over he Democrat. (2) Republican and a Democratic candidate, I will select the o o o o o Communism has een proven to be a failed political ideology. (3) o cannot see myself ver voting to elect conservative candidates. (4) o (5) he major national edia are too left- wing for my taste. ocialism has many advantages over o capitalism. (6) o right. (7) o On balance, I lean olitically more to e left than to the End of Block: Block 10 Start of Block: Block 7 o o o o o o o o o o o o o o o o o o o o 70 Q13 The research described is taken from: Goode, E., & Haner, J. (2016). A wrenching choice for Alaska towns in the path of climate change. New York Times, November, 29. End of Block: Block 7 71 APPENDIX B Abstracts 72 Treatment Abstracts Context 1: Biofilm Bullet points • Bacterial biofilms occur when bacteria attach to surfaces and develop a protective coating, making them resistant to antibiotics. • Biofilms appear in two-thirds of the infections seen by health care providers. • New research has shown that certain types of blue clay from Oregon can kill some strains of bacteria in laboratory conditions. • Elements in the clay work together to prop open the cell wall and let iron enter the cell, killing it. • The researchers have so far tested one concentration of the clay in a suspension, and it has been effective against common pathogens, including Staphylococcus aureus, which causes skin and respiratory infections Simple narrative Bacterial biofilms occur when bacteria attach to surfaces and develop a protective coating, making them resistant to antibiotics. These biofilms appear in two-thirds of infections seen by health care providers. Scientists at Arizona State University are researching ways to combat these dangerous biofilms, and have turned their focus to clay, which has been used in medicinal practices since prehistoric time, and is still often used by indigenous people. Historically, clay has been used for skin treatments, as an antiseptic, and for intestinal issues. The scientists have found that certain types of blue clay from Oregon can kill some strains of biofilm bacteria in laboratory conditions. One element in the clay causes the cell to open its wall, and the another element props the cell wall open, allowing iron to enter the cell and kill it. The researchers have so far tested one concentration of the clay in a suspension, and it has been effective against common pathogens, including Staphylococcus aureus, which causes skin and respiratory infections. These results will hopefully help scientists develop new drug designs incorporating natural clays, though much more research remains to be done. Simile Bacterial biofilms occur when bacteria attach to surfaces and develop a protective coating, making them resistant to antibiotics. Biofilm is a sort of protective living cocoon that prevents antibiotics from working. These biofilms appear in two-thirds of infections seen by health care providers. Scientists at Arizona State University are researching ways to combat these dangerous biofilms, and have turned their focus to clay, which has been used in medicinal practices since prehistoric time, and is still often used by indigenous people. Historically, clay has been used for skin treatments, as an antiseptic, and for intestinal issues. The scientists have found that certain types of blue clay from Oregon can kill some strains of biofilm bacteria in laboratory conditions. The clay works like the Trojan horse attack in ancient Greece, with the enemy lurking inside the innocent exterior: one element in the clay causes the cell to open its wall, and the another element props the cell wall open, allowing iron to enter the cell and kill it. The researchers have so far tested one concentration of the clay in a suspension, and it has been effective against common pathogens, including Staphylococcus aureus, which causes skin and respiratory infections. These results will hopefully help scientists develop new drug designs incorporating natural clays, though much more research remains to be done. 73 Context 2: Sea level Bullet points • Temperatures in the Arctic are rising faster than in other regions of the globe. • Sea ice is white, and as it melts there is less surface for the sun’s energy to reflect off. This causes more energy to be absorbed, leading to warmer waters and increased rates of ice melt. In remote areas in Alaska, rising sea levels threaten the existence of coastal towns. • • On Alaska’s western coast, the layer of winter sea ice and slush that protects the land from storm surges and powerful waves is decreasing. • The Alaskan village of Shaktoolik faces the very real possibility of disappearing in the next decades. • As of right now, residents have chosen to stay and try to develop ways to deal with the inevitable rising waters and land erosion. • To safely stay, the village needs at least $100 million to spend on improvements, like an evacuation road, improvements to the water system, and a fortified shelter in case of a storm. Simple narrative On Alaska’s remote Western coast lies Shaktoolik, a village of about 250 people. Temperatures in the Arctic are rising faster than in other regions of the globe. Sea ice is white, and as it melts there is less surface for the sun’s energy to reflect off. This causes more energy to be absorbed, leading to warmer waters and increased rates of ice melt. These rapid temperature increases are leading to rising sea levels that threaten the very existence of coastal towns and villages like Shaktoolik. Historically, winter sea ice and slush has protected Shaktoolik from storm surges and powerful waves, diminishing the force of the water. Now, however, that protective barrier is shrinking, and even disappearing, and Shaktoolik faces the very real possibility of disappearing in the next decades. Residents are faced with the difficult choice of staying and trying to survive on a shrinking land mass, or relocating their entire village to safer ground. Collectively, the village has chosen to stay and try to develop ways to deal with the inevitable rising waters and land erosion. The village needs at least $100 million to spend on improvements like an evacuation road, improvements to the water system, and a fortified shelter in case of a storm. If Shaktoolik is able to acquire this funding, it may be able to weather the coming storms, but its future is precarious. Simile On Alaska’s remote Western coast lies Shaktoolik, a village of about 250 people. Temperatures in the Arctic are rising faster than in other regions of the globe. Sea ice is white, and as it melts there is less surface for the sun’s energy to reflect off. Like lounging on a dark blanket in the sunshine, this causes more energy to be absorbed, leading to warmer waters and increased rates of ice melt. These rapid temperature increases are leading to rising sea levels that threaten the very existence of coastal towns and villages like Shaktoolik. Historically, winter sea ice and slush has protected Shaktoolik from storm surges and powerful waves, working like siege fortifications to diminish the force of the water. Now, however, that protective barrier is shrinking, and even disappearing, and Shaktoolik faces the very real possibility of disappearing in the next decades. Residents are faced with the difficult choice of staying and trying to survive on a shrinking land mass, or relocating their entire village to safer ground. Collectively, the village has chosen to stay and try to develop ways to deal with the inevitable rising waters and land erosion. The village needs at least $100 million to spend on improvements like an evacuation road, improvements to the water system, and a fortified shelter in case of a storm. If 74 Shaktoolik is able to acquire this funding, it may be able to weather the coming storms, but its future is precarious. Character On Alaska’s remote Western coast lies Shaktoolik, a village of about 250 people, where people like Edna Maktevik have lived all their lives. Temperatures in the Arctic are rising faster than in other regions of the globe. Sea ice is white, and as it melts there is less surface for the sun’s energy to reflect off. This causes more energy to be absorbed, leading to warmer waters and increased rates of ice melt. These rapid temperature increases are leading to rising sea levels that threaten the very existence of coastal towns and villages like Shaktoolik. Historically, winter sea ice and slush has protected Shaktoolik from storm surges and powerful waves, diminishing the force of the water. Now, however, that protective barrier is shrinking, and even disappearing, and Shaktoolik faces the very real possibility of disappearing in the next decades. Residents are faced with the difficult choice of staying and trying to survive on a shrinking land mass, or relocating their entire village to safer ground. Edna Maktevik helps support her family by gathering cranberries and ice fishing, as well as helping to care for her eight grandchildren. While she worries about the future, she feels invested in her village, and so do the rest of Shaktoolik’s residents. Collectively, the village has chosen to stay and try to develop ways to deal with the inevitable rising waters and land erosion. The village needs at least $100 million to spend on improvements like an evacuation road, improvements to the water system, and a fortified shelter in case of a storm. Edna worries about where the money will come from, as federal funding is uncertain. If Shaktoolik is able to raise enough money, it may be able to weather the coming storms, but its future is precarious. Character + simile On Alaska’s remote Western coast lies Shaktoolik, a village of about 250 people, where people like Edna Maktevik have lived all their lives. Temperatures in the Arctic are rising faster than in other regions of the globe. Sea ice is white, and as it melts there is less surface for the sun’s energy to reflect off. Like lounging on a dark blanket in the sunshine, this causes more energy to be absorbed, leading to warmer waters and increased rates of ice melt. These rapid temperature increases are leading to rising sea levels that threaten the very existence of coastal towns and villages like Shaktoolik. Historically, winter sea ice and slush has protected Shaktoolik from storm surges and powerful waves, working like siege fortifications to diminish the force of the water. Now, however, that protective barrier is shrinking, and even disappearing, and Shaktoolik faces the very real possibility of disappearing in the next decades. Residents are faced with the difficult choice of staying and trying to survive on a shrinking land mass, or relocating their entire village to safer ground. Edna Maktevik helps support her family by gathering cranberries and ice fishing, as well as helping to care for her eight grandchildren. While she worries about the future, she feels invested in her village, and so do the rest of Shaktoolik’s residents. Collectively, the village has chosen to stay and try to develop ways to deal with the inevitable rising waters and land erosion. The village needs at least $100 million to spend on improvements like an evacuation road, improvements to the water system, and a fortified shelter in case of a storm. Edna worries about where the money will come from, as federal funding is uncertain. If Shaktoolik is able to enough money, it may be able to weather the coming storms, but its future is precarious. 75 Context 3: Blue mind Bullet points and healthier. without water. main benefits they get. • So-called “blue mind” science argues that being near, in, or under water can make you happier • There are several organizations using aquatic therapy to help people with Post Traumatic Stress Disorder (PTSD) by providing therapeutic integration of the senses. • Views of water from one’s home is associated with lower psychological stress than views • People who visit places with water features identify social and psychological benefits as the • Studies show that blue and gray are colors people most commonly associate with calmness. • Spa bathing in a jacuzzi or hot tub has been associated with reduced stress levels. Simple narrative So-called “blue mind” science argues that being near, in, or under water can make you happier and healthier. Research shows several benefits to being around water features, whether outside or inside one’s home. For example, people who visit places with water features identify social and psychological benefits as the main benefits they get. Spa bathing in a jacuzzi or hot tub has been associated with reduced stress levels. Similarly, views of water from one’s home are associated with lower psychological stress than views without water, and studies have shown that blue and gray are colors people most commonly associate with calmness. Because being close to water or in water seems to reduce anxiety and stress, there are several organizations using aquatic therapy to help people with Post Traumatic Stress Disorder (PTSD) by providing therapeutic integration of the senses. This therapy can help with mobility and confidence, and also promote serenity and calm. Understanding our blue mind mentality may lead to further development of therapeutic uses of water, as well as an awareness of our own relationship to water. Simile So-called “blue mind” science argues that being near, in, or under water can make you happier and healthier. Research shows several benefits to being around water. For example, people who visit places with water features identify social and psychological benefits as the main benefits they get. Spa bathing in a jacuzzi or hot tub, where the water snuggles you like a warm blanket, has been associated with reduced stress levels. Similarly, views of water from one’s home are associated with lower psychological stress than views without water, and studies have shown that blue and gray are colors people most commonly associate with calmness. Because being close to water or in water seems to reduce anxiety and stress, like a tonic for one’s soul, there are several organizations using aquatic therapy to help people with Post Traumatic Stress Disorder (PTSD) by providing therapeutic integration of the senses. This therapy can help with mobility and confidence, and also promote serenity and calm. Understanding our blue mind mentality may lead to further development of therapeutic uses of water, as well as an awareness of our own relationship to water. Character So-called “blue mind” science argues that being near, in, or under water can make you happier and healthier. Brian Ward, who struggles with Post Traumatic Stress Disorder (PTSD), was on vacation at the beach when he recognized that being near the ocean was helping with his own mental health, and he began to investigate the science behind it. Research shows several benefits to being around water. For 76 example, people who visit places with water features identify social and psychological benefits as the main benefits they get. Spa bathing in a jacuzzi or hot tub has been associated with reduced stress levels. Similarly, views of water from one’s home are associated with lower psychological stress than views without water, and studies have shown that blue and gray are colors people most commonly associate with calmness. Because being close to water or in water seems to reduce anxiety and stress, there are several organizations using aquatic therapy to help people with PTSD by providing therapeutic integration of the senses. This therapy can help with mobility and confidence, and also promote serenity and calm. Ward has been using aquatic therapy for two years now, and it helps him deal with his PTSD. He also started a surfing group with other PTSD sufferers, and the members agree that closeness to water seems to reduce anxiety and stress. Understanding our blue mind mentality may lead to further development of therapeutic uses of water, as well as an awareness of our own relationship to water. Character + simile So-called “blue mind” science argues that being near, in, or under water can make you happier and healthier. Brian Ward, who struggles with Post Traumatic Stress Disorder (PTSD), was on vacation at the beach when he recognized that being near the ocean was helping with his own mental health, and he began to investigate the science behind it. Research shows several benefits to being around water. For example, people who visit places with water features identify social and psychological benefits as the main benefits they get. Spa bathing in a jacuzzi or hot tub, where the water snuggles you like a warm blanket, has been associated with reduced stress levels. Similarly, views of water from one’s home are associated with lower psychological stress than views without water, and studies have shown that blue and gray are colors people most commonly associate with calmness. Because being close to water or in water seems to reduce anxiety and stress, like a tonic for one’s soul, there are several organizations using aquatic therapy to help people with PTSD by providing therapeutic integration of the senses. This therapy can help with mobility and confidence, and also promote serenity and calm. Ward has been using aquatic therapy for two years now, and it helps him deal with his PTSD. He also started a surfing group with other PTSD sufferers, and the members agree that closeness to water seems to reduce anxiety and stress Understanding our blue mind mentality may lead to further development of therapeutic uses of water, as well as an awareness of our own relationship to water. 77 WORKS CITED 78 WORKS CITED Aaronson, S. (1977). Style in scientific writing. Current Contents, 2, 6-15. AGU100. (2018). AGU Narratives Webinar: Telling the Story of our Science. https://connect.agu.org/events/event-description?CalendarEventKey=844b705d-100b-4d6d- b2ad-ca54285b3186&Home=%2Fhome. Anderson, L. W., Krathwohl, D. R., Airiasian, W., Cruikshank, K. A., Mayer, R. E., & Pintrich, P. R. (2001). A taxonomy for learning, teaching and assessing: A revision of Bloom's Taxonomy of educational outcomes: Complete edition. NY: Longman. Avraamidou, L., & Osborne, J. (2009). The role of narrative in communicating science. International Journal of Science Education, 31(12), 1683-1707. Barriga, C. A., Shapiro, M. A., & Fernandez, M. L. (2010). Science information in fictional movies: Effects of context and gender. Science Communication, 32(1), 3-24. Bloom, B. S. (1956). Taxonomy of educational objectives. Vol. 1: Cognitive domain. New York: McKay, 20-24. Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon's Mechanical Turk: A new source of inexpensive, yet high-quality, data?. Perspectives on psychological science, 6(1), 3-5. Busselle, R., & Bilandzic, H. (2009). Measuring narrative engagement. Media Psychology, 12(4), 321-347. Carriger, M. (2011). Narrative approach to corporate strategy: empirical foundations. Journal of Strategy and Management, 4(4), 304-324. Chang, H. (2011). The philosophical grammar of scientific practice. International Studies in the Philosophy of Science, 25(3), 205-221. Cronin, K. (2010). "The “citizen scientist”: reflections on the public role of scientists in response to emerging biotechnologies in New Zealand." East Asian Science, Technology and Society 4(4): 503-519. Crowe, A., Dirks, C., & Wenderoth, M. P. (2008). Biology in bloom: implementing Bloom's taxonomy to enhance student learning in biology. CBE-Life Sciences Education, 7(4), 368-381. Cunningham, J. W. (1976). Metaphor and reading comprehension. Journal of Reading Behavior, 8(4), 363-368. Dahlstrom, M. F. (2010). The role of causality in information acceptance in narratives: An example from science communication. Communication Research, 37(6), 857-875. Dahlstrom, M. (2014). Using narratives and storytelling to communicate science with nonexpert audience. Proceedings of the National Academy of Sciences, 111(4): 13614-13620. 79 Dahlstrom, M. F., & Rosenthal, S. (2018). Third-Person Perception of Science Narratives: The Case of Climate Change Denial. Science Communication, 40(3), 340-365. De Wit, J. B., Das, E., & Vet, R. (2008). What works best: objective statistics or a personal testimonial? An assessment of the persuasive effects of different types of message evidence on risk perception. Health Psychology, 27(1), 110. Dudo, A., & Besley, J. C. (2016). Scientists’ prioritization of communication objectives for public engagement. PloS one, 11(2), e0148867. Eliot, T. S. (1957). On poetry and poets. New York, Farrar, Straus and Cudahy. Freeling, B., Doubleday, Z. A., & Connell, S. D. (2019). Opinion: How can we boost the impact of publications? Try better writing. Proceedings of the National Academy of Sciences, 116(2), 341- 343. Golding, D., Krimsky, S., & Plough, A. (1992). Evaluating risk communication: Narrative vs. technical presentations of information about radon. Risk Analysis, 12(1), 27-35. Goodman, J. K., & Paolacci, G. (2017). Crowdsourcing consumer research. Journal of Consumer Research, 44(1), 196-210. Harris, R. J., & Mosier, N. J. (1999). Memory for metaphors and similes in discourse. Discourse Processes, 28(3), 257-270. Hartley, J., Sotto, E., & Pennebaker, J. (2002). Style and substance in psychology: Are influential articles more readable than less influential ones? Social Studies of Science, 32(2), 321-334. Hecht, M.L. and Miller-Day, M. 2009. “The Drug Resistance Strategies Project: Using narrative theory to enhance adolescents' communication competence”. In Routledge handbook of applied communication, Edited by: Frey, L. and Cissna, K. 535–557. New York, NY: Routledge. Hendriks, F., Kienhues, D., & Bromme, R. (2015). Measuring laypeople’s trust in experts in a digital age: The Muenster Epistemic Trustworthiness Inventory (METI). PloS one, 10(10), e0139309. Hillier, A., Kelly, R. P., & Klinger, T. (2016). Narrative style influences citation frequency in climate change science. PloS one, 11(12), e0167983. Hinyard, L. J. and Kreuter, M. W. 2007. Using narrative communication as a tool for health behavior change: A conceptual, theoretical, and empirical overview. Health Education & Behavior, 34: 777–792. Hsieh, H. F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative health research, 15(9), 1277-1288. Humble, Á. M. (2009). Technique triangulation for validation in directed content analysis. International Journal of Qualitative Methods, 8(3), 34-51. 80 Inhoff, A. W., Lima, S. D., & Carroll, P. J. (1984). Contextual effects on metaphor comprehension in reading. Memory & Cognition, 12(6), 558-567. Jones, M. D. (2014). Communicating climate change: Are stories better than “just the facts”? Policy Studies Journal, 42(4), 644-673. Kahan, D. M. (2013). Making climate-science communication evidence-based—all the way down. www.ssrn.com. Kenney, W. P. (1988). How to read and write about fiction. Arco. Kromka, S. M., & Goodboy, A. K. (2019). Classroom storytelling: using instructor narratives to increase student recall, affect, and attention. Communication Education, 68(1), 20-43. Martin, W. (1986). Recent theories of narrative. Cornell University Press. Mattila, A. S. (2000). The role of narratives in the advertising of experiential services. Journal of Service Research, 3(1), 35-45. Maurer, M., & Gierl, H. (2015). Listen to my story: a comparison of the narrative to the factual format of information presentation in advertisements. In Advances in Advertising Research (Vol. V) (pp. 47-60). Springer Gabler, Wiesbaden. McBeth, M. K., Shanahan, E. A., & Jones, M. D. (2005). The science of storytelling: Measuring policy beliefs in Greater Yellowstone. Society and Natural Resources, 18(5), 413-429. McCarthy, K. S., Kopp, K. J., Allen, L. K., & McNamara, D. S. (2018). Memory, Comprehension, and Learning. Handbook of Research Methods in Human Memory. McDuffie, D. (2019). Using Amazon’s Mechanical Turk: Benefits, Drawbacks, and Suggestions. APS Observer, 32(2). Merritt, D. D., & Liles, B. Z. (1987). Story grammar ability in children with and without language disorder: Story generation, story retelling, and story comprehension. Journal of Speech, Language, and Hearing Research, 30(4), 539-552. Miller, Clark A., Jason O’Leary, Elisabeth Graffy, Ellen B. Stechel, and Gary Dirks. "Narrative futures and the governance of energy transitions." Futures 70 (2015): 65-74. Miller-Day, M., & Hecht, M. L. (2013). Narrative means to preventative ends: A narrative engagement framework for designing prevention interventions. Health communication, 28(7), 657-670. National Research Council. (2007). Taking science to school: Learning and teaching science in grades K-8. National Academies Press. National Academies of Sciences, Engineering, and Medicine. (2017). Communicating Science Effectively: A Research Agenda. Washington, DC: The National Academies Press. https://doi.org/10.17226/23674. 81 Negrete, A., & Lartigue, C. (2010). The science of telling stories: Evaluating science communication via narratives (RIRC method). Journal of Media and communication studies, 2(4), 98. Omanson, R. C. (1982). An analysis of narratives: Identifying central, supportive, and distracting content. Discourse processes, 5(3-4), 195-224. Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010). Running experiments on amazon mechanical turk. Judgment and Decision making, 5(5), 411-419. Reddy, C. (2009). Scientist citizens. Science, 323(5920), 1405-1405. Shanahan, J., Pelstring, L., & McComas, K. (1999). Using narratives to think about environmental attitude and behavior: An exploratory study. Society & Natural Resources, 12(5), 405-419. Shapiro, M. A. (1986). Analogies, visualization, and mental processing of science stories. Annals of the International Communication Association, 9(1), 339-355. Shelton, C. C., Warren, A. E., & Archambault, L. M. (2016). Exploring the use of interactive digital storytelling video: Promoting student engagement and learning in a university hybrid course. TechTrends, 60(5), 465-474. Shohamy, E. (1984). Does the testing method make a difference? The case of reading comprehension. Language testing, 1(2), 147-170. Singer, A., Goralnik, L., Gray, S., Introne, J., and Jordan, R. A Narrative of Narrative in Science Communication. Public Understanding of Science (in review). Solomon, J. (2002). Science stories and science texts: What can they do for our students? Spoel, Philippa, David Goforth, Hoi Cheu, and David Pearson. "Public communication of climate change science: Engaging citizens through apocalyptic narrative explanation." Technical Communication Quarterly 18, no. 1 (2008): 49-81. Stern, B. B. (1991). Who talks advertising? Literary theory and narrative “point of view”. Journal of Advertising, 20(3), 9-22. Stotesbury, H. (2003). Evaluation in research article abstracts in the narrative and hard sciences. Journal of English for Academic Purposes, 2(4), 327-341. Stritch, J. M., Pedersen, M. J., & Taggart, G. (2017). The opportunities and limitations of using Mechanical Turk (Mturk) in public administration and management scholarship. International Public Management Journal, 20(3), 489-511. Trabasso, T. (1982). Causal cohesion and story coherence. Trabasso, T., & Van Den Broek, P. (1985). Causal thinking and the representation of narrative events. Journal of memory and language, 24(5), 612-630. 82 Wolf, D. F. (1993). Issues in reading comprehension assessment: Implications for the development of research instruments and classroom tests. Foreign Language Annals, 26(3), 322-331. Yonelinas, A. P. (2002). The nature of recollection and familiarity: A review of 30 years of research. Journal of memory and language, 46(3), 441-517. 83 CHAPTER 3. Translating Community Narratives into Semi-quantitative Models to Understand the Dynamics of Socio-environmental Crises 1. Introduction In times of socio-environmental crisis, breakdowns in trust and communication can drive the development of insular, rigid, and balkanized knowledge structures (Brown and Duguid 2001) that may result in developing factions that severely hinder collective policy development, making rapid recovery difficult. Such factions are represented by multiple, closed social networks (e.g., separate networks of government responders and community residents impacted by the event) with limited knowledge sharing between them. The insular nature of these closed knowledge networks may result in a series of isolated groups, each with their own explanations for a crisis’ causes, consequences, and solutions, based on differences in perception, logic, and interpretation of the environmental problem (Otto- Banaszak et al. 2011) and how it is being handled (Kamo et al. 2015). Recovery efforts to the many “wicked problems” that emerge from acute socio-environmental crises require a collective understanding of the dynamics of the issue from heterogeneous stakeholder perspectives, including those impacted by the event and those in charge of recovery efforts (Gray et al. 2012). However, gaining this type of understanding within and across social networks about a crisis as events unfold remains difficult (Kamo et al. 2015). We propose that the various beliefs about a crisis can be debated, analyzed, and used to improve collective learning and decision-making if this variation in understanding across groups is represented through standardized modeling techniques. Further, we suggest that such rapid participatory modeling techniques can be facilitated through easy-to-use concept mapping that translates qualitative community narratives, experiences, beliefs and stories into semi-quantitative models that test ideas about potential futures through scenario analyses. Such scenario analyses can then be used to promote community learning, thereby increasing communication capacity within and between groups and improving recovery efforts. 84 Our work builds on recent studies that have demonstrated that including narrative processes in a learning environment increases critical thinking and can help manage conflicts (Clark and Rossiter 2008, Mattingly and Lawlor 2009, Lindgren and McDaniel 2012). Thus, we propose that models, and low barrier community-based modeling approaches in particular, can serve as visual and quantitative representations of qualitative narrative descriptions (Gray et al. 2015) by making relationships between objects or events that are experienced during crisis events explicit. Additionally, we add to the emerging discussion about the potential of software-based modeling approaches to support translating qualitative narratives into modeling scenarios (Mallampalli et al. 2016) as a means to support communication through knowledge capture, knowledge-sharing, and learning (Gray et al. 2015). Although such approaches to participatory modeling in general are gaining popularity (Gray et al. in review), explicit information about the translation process that occurs as modelers and communities move from narrative descriptions of events into model parameters remain poorly understood. Using the recent socio-environmental water crisis in Flint, Michigan as a case study, we demonstrate the use of a novel and free Fuzzy Cognitive Mapping software called Mental Modeler (www.mentalmodeler.org) to capture community narratives, translate these stories into model parameters with residents, and use these models to discuss competing futures based on group scenario analysis. 2. The Flint Water Crisis “The emergency manager took our democracy away” (Flint resident, 2016). Two years after the official report was released about the increased blood lead levels in children from elevated lead levels in the water supply (Hanna-Attisha et al. 2016), residents and emergency responders in Flint, Michigan still face a number of challenges. The so-called “Flint Water Crisis” affected an already marginalized community that over the last decades has experienced a disproportionate share of the economic burdens associated with deindustrialization, racial discrimination and segregation, and municipal fragmentation (Sadler and Highsmith 2016; Sadler and Lafreniere 2016). It is now recognized 85 that the crisis was the result of political and management failures (see Figure 12 for a detailed timeline). Specifically, on top of pre-existing issues with public water infrastructure, the failure to add anti- corrosives to a new water source led to the leaching of lead from pipes (Flint Water Advisory Task Force 2016). This, in turn, resulted in a significant increase in the number of children with elevated blood lead levels (Hanna-Attisha et al. 2016). Although the short-term water quality issues are now being addressed years after the crisis was brought to public attention, these events continue to have cascading long term effects such as further declines in property values, negative impacts on the tax-base, declines in community health, and stigmatization. As a result of these events, the community has been placed into an environment of high uncertainty about the long term health, economic, and social impacts, and how these issues will be addressed. In addition, many people impacted by the leaded water felt that their concerns were not taken into account and were largely ignored by those in charge of recovery efforts (Butler et al. 2016). A lack of decision-making power is common during ‘environmental justice’ events (Kamo et al. 2015), and often leads to diminished trust in the government agencies, making collective responses to crises even more difficult (Nicholls and Picou 2013). 86 Figure 12. Timeline of the major events in the Flint water crisis. 87 Figure 12 (con’t). 88 3. Methods “Who do you hold accountable? Everybody is lying” (Flint resident, 2016). To support collective learning and decision-making during the Flint Water Crisis, we conducted a series of community-based “mental modeling” workshops to capture resident beliefs about the causes, consequences, and solutions to the water crisis in partnership with the Community Foundation of Greater Flint (CFGF), a local non-governmental organization (NGO). Additionally, during workshops, we administered short surveys to evaluate residents’ trust in, and use of, information sources about risks associated with the event. We designed a group modeling exercise using the Mental Modeler software (Gray et al. 2013, www.mentalmodeler.org) to capture, quantify, and share different perspectives from the community in workshops throughout Flint. We worked through the partnering NGO to (1) capture variation in community beliefs, and (2) aggregate and share these beliefs within and across resident and decision-making groups. We conducted four workshops, each drawing from different sections of the city. Participants were recruited by the CFGF, which is active in the local community and particularly focused on the water crisis. Each workshop had between 4 and 16 participants, with a cumulative total of 36 community members across the four workshops. Finally, one larger-scale debriefing modeling workshop was held after models were aggregated, to which all workshop participants were invited, along with other management officials. This debriefing workshop was attended by 28 residents and officials. These workshops were held in spring of 2016, while the water crisis was ongoing and residents were still dependent on bottled water. Each workshop was videotaped so that the conversations and facilitation could be analyzed after. The workshops began with a survey asking participants to rank the trustworthiness and usefulness of information sources including federal, state, and local officials, university scientists, NGOs, and fellow community members. The workshops continued with a participatory modeling process using Fuzzy Cognitive Mapping (FCM) and the software Mental Modeler (Gray et al. 2013; 89 www.mentalmodeler.com). An FCM is a type of systems modeling that helps people reflect and share knowledge about the structure and function of complex systems (Kosko 1986), including the nature of poorly understood socio-environmental problems (Nyaki et al. 2014). The Mental Modeler software allows for scenario analysis based on the FCMs. These scenarios help bridge the qualitative methods that are more accessible but offer less in the way of analytical capabilities, and the quantitative methods that allow for more analytical capabilities but also require more training for participants (Gray et al. 2017). During the workshops, a computer modeling screen (see “Model screen” in Mental Modeler) was projected in front of the participants. A facilitator then asked three broad questions throughout the modeling session, first about the consequences of the lead exposure, second about the causes that led to the water crisis, and third about solutions to the water crisis. Each question led to a discussion with workshop participants, allowing them to share their experiences and beliefs. During these discussions the conversation was captured by the facilitator using the Mental Modeler software. The software helped facilitate a transition from narrative statements into model concepts that could then be quantified and used for analysis and communication. Iteratively, participants were asked to reduce and clarify their comments into nodes and quantify the strength of the causal relationships between variables using the slider bar parametrized between -1 and +1, following common FCM techniques (Ozesmi and Ozesmi 2004; Gray et al. 2015). At several times, the facilitator asked refining questions about the names of concepts and relationships. Table 5 provides an example of how the workshop participants and the research team negotiated to translate comments into concepts and cause and effect relationships between concepts. This is a simplified example that leaves out the sometimes extended discussions associated with translating the initial statement into the final concept nodes and relationships. 90 Table 6. Translating quotes from workshop participants into concepts and relationships into a fuzzy cognitive map in the Mental Modeler software. Quote from Workshop Participation Final Model Concepts and Relationships Clarifying Questions and Responses Facilitator: Who could not fight back? What was the nature of these residents? Response: Black people and poor people Facilitator: How does this stress relate to community health? Response: Negatively Facilitator: What leads to emotional stress of caregivers? Response: Increased stress of household labor because we can’t use water Facilitator: So uncertainty about the future is increased? How does that relate to trust? Response: It reduces trust Facilitator: Would you call this “stigma”? Respondent: Yes Facilitator: In general would you say that the exposure had led to increased amounts of uncertainty about your future? Respondent: Yes “The lead problem was allowed to happen because there was a group of people who could not fight back.” “I wonder if just the pressures, the stress, that is thrown on these great- grandmothers and grandmothers who are raising these babies is not causing some of these problems.” “And to me that looks like somebody’s price gouging the water bills you know. So it's pretty much like everyone here is saying. It's hard to trust anything right now. (...) And basically, most people won't ever trust the city again. (...) They’re not telling people the whole truth about that water testing.” “I was out of the country and was asked ‘Where are you from’ “Michigan,’ ‘What part of Michigan?’ and you take a deep breath and say ‘Flint,’ and they say ‘FLINT,’ and look at me like I have cooties or something.” “We don’t know the long- term effect… 10-20 years from now kids will still be affected.” In the workshop settings, the facilitator would create a new node each time a new concept came up. The facilitator then asked clarifying questions about what the name of the concept should be, and what the definition of the concept was (see Table 5 for examples). Participants discussed these 91 clarifying questions until there was general agreement about the concept, at which point the facilitator asked how the concept related to other Relationships were defined as positive or negative, again through general participant agreement. Finally, the strengths of the relationships were discussed on a spectrum of very weak to very strong. As discussions progressed, the model slowly branched out from the initial concept of “lead exposure.” Figure 13 shows a simplified model-building process using participant quotes from Table 5. The participants added nodes and relationships as they articulated their beliefs about why the lead exposure occurred, the broader impacts of exposure, and potential ways to mitigate not only the lead exposure itself, but also the systemic issues they believe contributed to the initial crisis, such as racial and socio-economic marginalization. 92 Figure 13. Simplified model-building process using participant quotes. The quotes are translated into concepts and relationships between concepts and placed into a fuzzy cognitive map using Mental Modeler software. After the workshops, the research team revised the models for simplicity and language standardization to enable comparisons between the workshops and develop an aggregated model. These revised models were shared with the CFGF. Together, the researchers and NGO created an aggregated model that represented the perspectives from each of the workshops. Aggregation was done by merging standardized concepts from each model, and building out all the relationships identified in each model. At times, the CFGF chose to remove a concept if they believed it was superfluous to the model. Using the individual workshop models and the aggregated model, researchers 93 identified the causes, consequences, and suggested solutions of the water crisis from a community perspective. At the debriefing workshop, participants used the aggregated model to identify differences and similarities between the individual workshop model and run scenarios of suggested solutions on the aggregated model. To run a scenario, a particular node in the FCM is “turned on” (i.e., increased to 1), and the impacts that has on other nodes are mathematically derived. Scenario output is based on previous FCM applications, using parameterized model structure (components and causal relationships between components) to generate system-state responses under matrix algebra “what if” conditions (see Jetter and Kok 2014; Ozesmi and Ozesmi 2004; Gray et al. 2015). The resulting scenario states for different potential community-based solutions use histogram type output in the Mental Modeler software to represent the degree (amount) and direction (positive or negative) of change. The scenario outputs showed a visual representation of possible impacts from specific solutions that had been identified by the workshop participants. The aggregate FCM included “lead pipe replacement,” “local workforce,” and “workforce training” as the three primary solutions to focus on. We ran scenarios on these solutions at the all-hands workshop. The scenario output allowed the community members to translate the results into a personal or community narrative explaining what might happen in Flint. For each scenario, the workshop participants discussed whether they believed the predicted output was accurate or not, and if not, what was missing or incorrect about the model. They also had conversations about the feasibility and equity of each solution. This exercise completed the cycle of translating a narrative into a model, using the model to quantify relationships and predict outcomes, and translating the model output into a narrative that represents the community perspective. 94 4. Results “Sad as it may seem, I think this has brought the community together” (Flint resident, 2016). The modeling exercise offered a way for Flint residents to identify the causes, consequences, and solutions to the water crisis, and to broaden and articulate their understanding of the issue as recovery efforts were organized. Though each workshop identified different specific causes of the water crisis, common themes were extracted across workshops. These include the low tax-base of Flint, the socioeconomic and racial marginalization of the community, aged infrastructure, austerity measures put in place by the administration, and the loss of municipal control and community agency. The direct cause of lead in the water is the switch from Lake Huron water to Flint River water, but the community identified a multitude of systemic causes that allowed the water switch to take place and to continue even after problems were identified. Similarly, each workshop identified some unique consequences of and solutions to the water crisis, but there was significant overlap. Figures 14-16 show the causes, consequences, and solutions identified by each workshop, including similarities and differences between workshops. Figure 14. Venn diagram showing the causes of the FWC identified by each of the four workshops. Three of the four workshops identified the Governor’s pro-business administration and a loss of local agency in decision-making as causes of the crisis. 95 Figure 15. Venn diagram showing the consequences of the FWC identified by each of the four workshops. All workshops identified increased household labor, health complications, and community health as consequences. Financial costs, uncertainty, and stress were identified by three of the four workshops. Figure 16. Venn diagram showing the potential solutions to the FWC. All workshops believed that reparations should be provided, and three workshops also suggested replacing household pipes and installing whole house filters. 96 Solutions included addressing the specific issue of lead in the water, such as by installing filters and replacing the pipes. Participants also identified solutions that would address some of the systemic issues that allowed the water crisis to occur in the first place, and ran scenarios on these. For example, figure 17 shows the results of replacing household pipes, and Figure 18 shows the results of increasing and training the local workforce to address the water crisis, such as by training locals to replace pipes and install filters. Scenario: Replace household pipes Figure 17. Results of implementing the scenario of replacing all the household pipes. “Lead pipe replacement” was the concept turned on. Lead exposure, emotional stress, and uncertainty are decreased relative to other concepts. Quality of life and outside businesses are increased relative to other concepts. The scenario results show that replacing the household pipes reduces the lead exposure, uncertainty, and emotional stress. However, it does not address the systemic issues of marginalization and poverty that have the city trapped in a wicked cycle that allows such problems as the water crisis to occur. Participants were particularly concerned that outside businesses were being used to install filters and replace infrastructure, evidenced by the increase in outside businesses. If, however, an emphasis is 97 placed on local workforce training, some of the systemic, wicked problems are mediated, as seen in Figure 18. Scenario: Train local workforce to replace pipes and install filters Figure 18. Results of implementing the scenario of training and using a local workforce to replace household pipes and install filters. “Local workforce” and “workforce training” were the concepts turned on. Flint sees gains to infrastructure health, return on investment, educational outcomes, and the local economy. Marginalization by race and by socio-economic status are both decreased. The community perspective on relationships between nodes shows that training a local workforce results in improved infrastructure, a return on investment in the city, increased educational outcomes, and a healthier local economy (Figure 18). Additionally, marginalization by race and by socio- economic status are decreased. This solution put forth by the community addresses some of the systemic issues that have kept Flint in a wicked cycle of marginalization over the past decades. The scenario results provided a visual representation of the narratives told by the community members. The community, in turn, was able to extend their understanding of the lead problem and how 98 it relates to the wicked problems they had identified. As an example of this continued learning, a participant asked what a scenario in which racism was reduced would look like. Figure 19 shows the scenario in which marginalization by race is reduced. Flint sees gains in trust, the local economy, educational outcomes, community health, and quality of life. Lead exposure is reduced, and emotional stress and daily household labor are also decreased. This scenario output was met with clapping and laughter, as the community saw their expectations met. Scenario: Reduce marginalization by race Figure 19. Results of implementing the scenario of reducing marginalization by race. Flint sees gains to trust, the local economy, educational outcomes, community health, and quality of life. Lead exposure is reduced, and emotional stress and daily household labor are also decreased. Reducing the institutional racism addresses the vicious cycle in which the community is caught. The scenario output echoed the community’s own narrative of the cycle, and affords the community a different way to share their perspective. Of course, reducing racism requires fundamental social, economic, and political changes. Some of the suggested solutions to the water crisis, such as investing in 99 entrepreneurship and local workforce training, may help reduce institutional racism. Participants also suggested changing the cost-sharing mechanism between cities and the state, as well as eliminating the governor’s ability to appoint an emergency manager. 5. Discussion “People have been victimized and they already feel powerless” (Flint resident, 2016). The participatory modeling process, structured around the Mental Modeler software, was a way for community members to articulate their perspectives in a new way. The fuzzy cognitive mapping allowed individuals to translate their personal narratives into model representations. This coupling of qualitative story and semi-quantitative simulation maximizes the value of scenario planning (Mallampalli et al. 2016). Workshop participants brought their own personal narratives into the model-building process, and were able to run scenarios of their proposed solutions to the water crisis. The scenario output and aggregated model were shared widely in the community. We presented the results to a group of approximately 75 Flint residents, which included representatives from the State, City, and Federal governments. The audience was extremely receptive to the presented models and scenarios, and feedback from individuals suggest that the results resonated with the majority of attendees. This suggests that by holding workshops in several locations within the city helped collect the diversity of perspectives on the issue. The results were also presented to the Genessee County Community Collaborative and the Flint Recovery Group, and people were generally interested and receptive. They did indicate that the modeling workshops may have left out certain facets of the population, including home-bound seniors, non-English speakers, and the disabled, which are perspectives that should be included in further research of this type. The immediate crisis of lead exposure is currently being addressed through filters and infrastructure replacement, but the community perspective exposes the systemic problems that allowed the lead to initially infiltrate the water system and to perpetuate. The group modeling process provided 100 a space and an accessible method for community members to explore their perspectives, and others’ perspectives, as part of a process that also provided them with tangible results and stories to share. The Flint residents know that their community exists in a state of marginalization, both by race and socio- economic status, and that this marginalization is part of a wicked cycle that cannot be broken by infrastructure replacement alone. As representations of community narratives surrounding the Flint water crisis, the model output and scenarios can be used as a powerful communication method to those in charge of recovery. The translation from individual narrative into community models helps create a cohesive, representative perspective to share. The feedback from residents and government officials has been very positive, and indicates that the workshops were able to collect the diversity of perspectives represented in the Flint community, and to translate those perspectives into a semi-quantitative modeling context, and then back into a narrative that resonates with community members. The results have been disseminated through community channels to help aid in communication during the recovery. 101 WORKS CITED 102 WORKS CITED Brown, J. S., & Duguid, P. (2001). Knowledge and Organization: A Social-Practice Perspective. Organization Science, 12(2), 198:213. Butler, L. J., Scammell, M. K., & Benson, E. B. (2016). The Flint, Michigan, Water Crisis: A Case Study in Regulatory Failure and Environmental Injustice. Environmental Justice, 9(4), 93-97. Clark, M. C., & Rossiter, M. (2008). Narrative learning in adulthood. New directions for adult and continuing education, 2008(119), 61-70. Flint Water Advisory Task Force. (2016). Final Report. Commissioned by the Office of Governor Rick Snyder. State of Michigan. Gray, S., Chan, A., Clark, D., & Jordan, R. (2012). Modeling the integration of stakeholder knowledge in social–ecological decision-making: benefits and limitations to knowledge diversity. Ecological Modelling, 229, 88-96. Gray, S. A., Gray, S., Cox, L. J., & Henly-Shepard, S. (2013, January). Mental modeler: a fuzzy-logic cognitive mapping modeling tool for adaptive environmental management. In System Sciences (HICSS), 2013 46th Hawaii International Conference on (pp. 965-973). IEEE. Gray, S. A., Gray, S., De Kok, J. L., Helfgott, A. E., O'Dwyer, B., Jordan, R., ... & Analyzing, F. (2015). Using fuzzy cognitive mapping as a participatory approach to analyze change, preferred states, and perceived resilience of social-ecological systems. Ecol. Soc, 20(2), 11. Gray, S.,A. Voinov, M. Paolisso, R. Jordan, T. BenDor, P. Bommel, P. Glynn, B. Hedelin, K. Hubacek, J. Introne, N. Kolagani, B. Laurson, C. Prell, L. Schmitt-Olabisi, A. Singer, E.J. Sterling, and M. Zellner. (In press) Purpose, Processes, Partnerships, and Products: 4Ps to advance Participatory Socio-Environmental Modeling. Ecological Applications. June 2017. Vol 27, Issue 4. Hanna-Attisha, M., LaChance, J., Sadler, R. C., & Champney Schnepp, A. (2016). Elevated blood lead levels in children associated with the Flint drinking water crisis: a spatial analysis of risk and public health response. American journal of public health, 106(2), 283-290. Jetter, A. J., & Kok, K. (2014). Fuzzy Cognitive Maps for futures studies—A methodological assessment of concepts and methods. Futures, 61, 45-57. Kamo, Y., Henderson, T. L., Roberto, K. A., Peabody, K. L., & White, J. K. (2015). Perceptions of Older Adults in a Community Accepting Displaced Survivors of Hurricane Katrina. Current Psychology, 34(3), 551-563. Kosko, B. (1986). Fuzzy cognitive maps. International Journal of man-machine studies, 24(1), 65-75. Lindgren, R., & McDaniel, R. (2012). Transforming Online Learning through Narrative and Student Agency. Educational Technology & Society, 15(4), 344-355. 103 Mallampalli, V. R., Mavrommati, G., Thompson, J., Duveneck, M., Meyer, S., Ligmann-Zielinska, A., Druschke, C.G., Hychka, K., Kenney, M.A., Kok, K., & Borsuk, M. E. (2016). Methods for translating narrative scenarios into quantitative assessments of land use change. Environmental Modelling & Software, 82, 7-20. Mattingly, C., and Lawlor, M. (2000). Learning from stories: Narrative interviewing in cross-cultural research. Scandinavian journal of occupational therapy, 7(1), 4-14. Nicholls, K., & Picou, J. S. (2013). The impact of Hurricane Katrina on trust in government. Social Science Quarterly, 94(2), 344-361. Nyaki, A., Gray, S. A., Lepczyk, C. A., Skibins, J. C., & Rentsch, D. (2014). Local-scale dynamics and local drivers of bushmeat trade. Conservation biology, 28(5), 1403-1414. Otto-Banaszak, I., Matczak, P., Wesseler, J., & Wechsung, F. (2011). Different perceptions of adaptation to climate change: a mental model approach applied to the evidence from expert interviews. Regional environmental change, 11(2), 217-228. Özesmi, U., & Özesmi, S. L. (2004). Ecological models based on people’s knowledge: a multi-step fuzzy cognitive mapping approach. Ecological modelling, 176(1), 43-64. Sadler, R. C., & Highsmith, A. R. (2016). Rethinking Tiebout: The Contribution of Political Fragmentation and Racial/Economic Segregation to the Flint Water Crisis. Environmental Justice. Sadler, R. C., & Lafreniere, D. J. (2016). Racist housing practices as a precursor to uneven neighborhood change in a post-industrial city. Housing Studies, 1-23. 104 Conclusions In this dissertation I contribute both foundational frameworks and methodological approaches to the field of science communication as it continues to transition from a deficit model to a dialogue model. Narrative has been a popular approach to bridging the communication gaps between various stakeholders involved in the dialogue model of science communication, as many argue that our constructions of reality and ability to function as social beings are inextricably bound with our ability to tell and interpret stories (Bruner 1987, Bruner 1991, Schank 1995). Narrative can also help us create cultural cohesion and identity (Hasson et al. 2008, Nathanson 2006, Negrete and Lartique 2010), which is necessary as scientists engage with more diverse groups of stakeholders. As communicators, scientists, and scientific organizations around the world argue that science communication should be using more narrative approaches, we must first acknowledge that narrative is a fuzzy idea, and that very few scientists have training in the use of narrative (Brownell et al. 2013). My objective here was to describe the current state of the science regarding narrative in science communication, and then develop and test approaches to studying the integration of narrative in the dialogue of science communication. I hope that this research can help scientists and communicators: 1) better understand what narrative is, 2) understand how narrative may or may not be useful for specific communication goals, and 3) develop tools to help them use narrative with diverse stakeholders. In chapter 1 I conducted systematic reviews to synthesize the current state of narrative research and how it fits into the field of science communication. The results suggest that many scientists do not explicitly or implicitly define narrative, which may cause confusion. Additionally, I found that scientists generally measure narrative one of three ways: stylistically, structurally, or intuitively. This research helps to create a foundational understanding of what scientists think narrative means, and how scholars are measuring and testing narrative. This provides us with diverse ways to approach empirical studies moving forward. 105 In Chapter 2, I developed and tested a novel methodological approach to testing the impacts of narrative on comprehension and recall of scientific information, narrative engagement, and perceptions of scientists. The results from this study suggest that narrative may in fact be distracting if the communication goal is to increase consumers’ recall and comprehension of scientific information. Omanson (1982) categorizes narrative as central, supportive, or distracting, and I hypothesize that narrative can function in each of these ways as part of science communication as well. Much more research must be done so that we can understand in which situations narrative is central, supporting, or distracting, and apply that to our own modes of communication. In Chapter 3, I explored how a community can use narrative to relate their research and lived experiences to scientists and people with decision-making power. For science communication to engage in dialogue, as opposed to one-way transmission, it is imperative that we develop strategies for fostering communication between scientists and communities. Participatory modeling is one approach that can facilitate a dialogue between communities or individuals and scientists. In this case study, I worked with members of the Flint, Michigan community to understand how they perceive the causes, consequences, and solutions to the crisis of having lead in their drinking water. This research demonstrates how participatory modeling can give communities a way to structure their thoughts, develop recovery actions, and communicate with those in charge of crisis recovery efforts. This dissertation then first helps develop a foundational definitions and measurements of narrative to be used in future research in the field, proposes an experimental protocol for testing narrative against learning impacts, and proposes a framework for facilitating the translation of individual or community narratives into semi-quantitative models. This work can help move the field of science communication to more of a dialogue model. I argue that at this point the dialogue model is largely aspirational, as the majority of research is focused still on how scientists can better communicate with the public (Lorono-Leturiondo et al. 2018, Metcalfe 2019). This is certainly one important aspect of the 106 dialogue model, which I explore in Chapter 2. However, for the dialogue model to function, it is as important to develop ways for various stakeholders to communicate with scientists and with other stakeholder groups. Therefore, in chapter 3 I focus on the under-studied question of how stakeholders can better communicate with scientists. Together, these frameworks can help scientists, communicators, and stakeholder groups engage in dialogue to address our most pressing wicked problems. 107 WORKS CITED 108 WORKS CITED Brownell, S. E., Price, J. V., & Steinman, L. (2013). Science communication to the general public: why we need to teach undergraduate and graduate students this skill as part of their formal scientific training. Journal of Undergraduate Neuroscience Education, 12(1), E6. Bruner, J. (1987). Life as narrative. Social research, 11-32. Bruner, J. (1991). The narrative construction of reality. Critical inquiry, 18(1), 1-21. Hasson, U., Landesman, O., Knappmeyer, B., Vallines, I., Rubin, N., & Heeger, D. J. (2008). Neurocinematics: The neuroscience of film. Projections, 2(1), 1-26. Lorono-Leturiondo, M., O'Hare, P., Cook, S., Hoon, S. R., & Illingworth, S. (2018). Give me five!–reasons for two-way communication between experts and citizens in relation to air pollution risk. Advances in Science and Research, 15, 45-50. Metcalfe, J. (2019). Comparing science communication theory with practice: An assessment and critique using Australian data. Public Understanding of Science, 0963662518821022. Nathanson, S. (2006). Harnessing the power of story: Using narrative reading and writing across content areas. Reading Horizons, 47(1), 1. Negrete, A., & Lartigue, C. (2010). The science of telling stories: Evaluating science communication via narratives (RIRC method). Journal of Media and communication studies, 2(4), 98. Omanson, R. C. (1982). An analysis of narratives: Identifying central, supportive, and distracting content. Discourse processes, 5(3-4), 195-224. Schank, R. C. (1995). Tell me a story: Narrative and intelligence. Northwestern University Press. 109