INTEGRATING FOOD SYSTEMS WITH COMMUNITY-IDENTIFIED VALUES By Mahdi Zareei A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Community Sustainability – Doctor of Philosophy 2024 ABSTRACT In the effort to build sustainable and resilient local food systems, a clear understanding of community dynamics and values is essential. This dissertation addresses the critical need to align food system strategies with community-identified values. Community-identified values are fundamental principles and priorities that reflect the unique needs, desires, and characteristics of a community. These values can vary widely depending on local contexts; each community has its own cultural, economic, and environmental contexts that shape its food system needs and goals. This heterogeneity means that a one-size-fits-all approach in local food system planning is ineffective. Instead, understanding and integrating diverse community perspectives can lead to more tailored, effective solutions that respect local traditions, meet specific nutritional needs, and enhance economic opportunities. The primary objectives of this dissertation are threefold: first, to explore various ways for measuring community-identified values within local food systems; second, to analyze the variations in expert predictions regarding the impacts of specific interventions within these systems; and third, to integrate these values into community-based resilience planning, emphasizing a balanced consideration of both monetary and non-monetary factors. The second chapter of this dissertation explores various methods for measuring community-identified values within food systems. This exploration is essential as it acknowledges the diversity of perspectives that different stakeholders contribute, and highlights the challenges in quantifying values that are often subjective or culturally specific. Instead of proposing a rigid framework, this chapter delves into comparing the features of different measurement ideas to ensure they capture a comprehensive understanding of community needs and aspirations. By developing and aggregating ideas for measuring these values, this chapter directly contributes to more effective food system planning by ensuring that policies and interventions are grounded in a comprehensive understanding of community needs and aspirations. The identification and measurement of these values enable planners and policymakers to monitor and evaluate the effectiveness of their strategies more accurately, ensuring that the interventions are truly beneficial to the community. This sets the stage for the subsequent chapters by establishing the importance of integrating a wide range of community perspectives into food system planning. Building on the measurement ideas discussed in the second chapter, the third chapter delves into the mental models of food system experts, utilizing fuzzy cognitive maps to illustrate how different experts predict the impacts of known leverage points on community-identified values. This analysis not only reveals the degree of consensus or disagreement among experts but also contrasts these individual predictions with outcomes from a collective intelligence model. The significance of this chapter lies in its demonstration of how diverse expert opinions can lead to varied predicted impacts, underscoring the necessity of considering multiple perspectives in food system planning. This chapter enriches the localized food system planning process by highlighting the potential discrepancies and alignments between individual and collective predictions, thereby informing more balanced and inclusive policymaking. The insights gained here complement the previous chapter’s focus on measurements by showing how different interpretations of data and projected impacts can influence planning outcomes. The fourth chapter extends the discussions from measuring and modeling community values to applying these concepts in community-based resilience planning for local food systems. It critiques the dominant economic focus in strategy evaluation and advocates for a more comprehensive approach that includes non-monetary criteria such as community empowerment and partnership. This chapter is pivotal as it synthesizes the ideas presented in the earlier chapters into practical strategies for enhancing food system resilience against hazards and risks. By incorporating community-identified values into the evaluation process, this chapter promotes a more holistic approach to resilience planning, emphasizing the importance of stakeholder engagement and the exploration of trade-offs between different criteria. It ensures that the strategies not only are economically viable but also resonate with the community’s values. Together, these chapters create a cohesive narrative that advances the field of localized food system planning by integrating community-identified values into every phase of policymaking, from measurement and modeling to implementation and evaluation. This approach highlights the interconnectedness of various planning phases, ensuring that local food systems are designed to be sustainable, culturally sensitive, and responsive to the specific needs of the community. Suggested citation: Zareei, M. (2024), “Integrating Food Systems with Community-Identified Values”, Ph.D. Dissertation, Michigan State University, East Lansing, Michigan, United States. Copyright by MAHDI ZAREEI 2024 AKNOWLEDGEMENTS I wish to extend my sincere thanks to my academic advisor, Professor Steven Gray, for his invaluable guidance, constant support, and encouragement throughout my doctoral studies. Without his support, navigating the complexities of my academic journey would have been incredibly challenging. My gratitude also extends to my dissertation committee members, Professor Laura Schmitt Olabisi, Dr. Richard Sadler, and Dr. Chelsea Wentworth, for their insightful suggestions and constructive feedback on my dissertation work. During my Ph.D. program, I was fortunate to be part of the Flint Leverage Point Project (FLPP), which provided significant learning opportunities through collaboration with my teammates. I sincerely thank members of the Flint community who participated in data collection and in advising our team for their essential contributions to this dissertation. This dissertation progress is indebted to our collaborators at the Community Foundation of Greater Flint, and the members of our Community Consultative Panel. My gratitude also goes to the researchers whose foundational work and insights formed the basis of this study. I am grateful to Dr. Jennifer Helgeson and NIST team members for our ongoing collaboration and for being particularly inspiring, especially for chapter 4. I would like to thank my peers, Dr. Carissa Knox and Dr. Nathan Brugnone, for their engaging discussions and support. I would also like to extend my heartfelt thanks to my friends Dr. Payam Aminpour, Dr. Saeed Memari, and Dr. Claire Ugo-Ike for their invaluable support, compassion, and enduring friendship. Most importantly, my heartfelt gratitude goes to my lovely wife, Mina Akbari Moghaddam, my kindest parents, my wonderful sister, and other family members for their unconditional love, support, and encouragement. v TABLE OF CONTENTS CHAPTER 1: INTRODUCTION ................................................................................................... 1 CHAPTER 2: MEASURING WHAT MATTERS: MONITORING COMMUNITY- IDENTIFIED VALUES IN LOCAL FOOD SYSTEMS ............................................................... 9 CHAPTER 3: CONSENSUS OF DISAGREEMENT? AN EXPLORATORY ANALYSIS OF PREDICTED IMPACTS OF LEVERAGE POINTS ON COMMUNITY-IDENTIFIED VALUES USING FUZZY COGNITIVE MAPS ......................................................................... 37 CHAPTER 4: COMMUNITY-INFORMED DECISIONS FOR EQUITABLE, COST- EFFECTIVE, AND INCLUSIVE DISASTER RESILIENCE PLANNING (CO-DECIDR): MODELING APPROACH ........................................................................................................... 70 CHAPTER 5: CONCLUSION ................................................................................................... 104 BIBLIOGRAPHY ....................................................................................................................... 110 APPENDIX A: FOOD SECURITY DIMENTIONS ................................................................. 127 APPENDIX B: SCENARIO ANALYSIS FOR THE CI MODEL ............................................ 128 APPENDIX C: INPUT FOR THE BCA ANALYSIS ................................................................ 134 APPENDIX D: SENSITIVITY ANALYSIS OF DR AND PH ................................................. 146 vi 1-1-Localized Food System Planning CHAPTER 1: INTRODUCTION Localized food system planning is an intricate approach that tailors agricultural and food- related activities to meet the specific needs and characteristics of a local or regional community (Buchan et al., 2015). This planning framework evaluates and incorporates unique cultural, economic, ecological, and social factors, aiming to develop systems that are sustainable, resilient, and closely aligned with local demands and resources (Stein & Santini, 2022). By engaging in localized planning, communities can optimize the structure and functionality of their food systems, enhancing food security and local economic stability while minimizing ecological footprints. Such systems are designed to promote shorter supply chains, improve the freshness and nutritional quality of food, and support local farmers and businesses, thus fostering a stronger community bond and sense of place and prepare them against the future shocks and hazards (Béné, 2020). The goal of localized planning is to create food systems that are responsive to local resources and demands. This involves critical assessments of local agricultural capacities, consumer preferences, and logistical frameworks. Planners and stakeholders work together to identify potential bottlenecks and opportunities in the food supply chain, aiming to create a streamlined, efficient, and equitable system (Carlsson et al., 2017). By focusing on the specificities of a local food system, the planning process can yield systems that not only meet immediate food needs but also contribute to long-term environmental and social sustainability. 1-2-Community-Identified Values Community-identified values are core principles and priorities that reflect the unique needs, desires, and characteristics of a community (Bosomworth & Gaillard, 2019, Campbell- Arvai & Lindquist, 2021). These values can vary widely depending on local contexts and may include priorities such as economic equity, which ensures fair access to economic resources; cultural traditions, which preserve and celebrate local customs and practices; and the preservation of local biodiversity, which focuses on maintaining diverse biological ecosystems. Recognizing and prioritizing these values in food system planning is crucial. They serve as a guide for creating a system that not only meets the logistical needs of food production and distribution but also resonates with the social and environmental ethos of the community (Campbell et al., 2022). By embedding these values into the planning process, the resulting food system can enhance public health, promote food sovereignty (i.e. authority of local populations to govern their food systems, 1 including the control of markets, ecological resources, traditional food practices, and production methods (Wittman, 2011)), and contribute to environmental conservation, while reflecting the community's identity and aspirations (Dubbeling et al., 2017). Operationalizing these values through strategic planning involves creating actionable steps that translate community priorities into the structures and practices of local food systems. This process begins with thorough community engagement (see section 6 of this chapter for a detailed explanation of community engagement) to identify and understand these values deeply. Because these visions can vary greatly across different communities and regions, each community has unique cultural, economic, and environmental contexts that shape their food system needs and goals. This heterogeneity means that a one-size-fits-all approach in food system planning is ineffective (Dengerink et al., 2021; Ng'endo & Connor, 2022). Instead, understanding and integrating diverse community perspectives can lead to more tailored, effective solutions that respect local traditions, meet specific nutritional needs, and enhance economic opportunities. Strategic planning then focuses on aligning food system policies, resource allocation, and developmental initiatives with these identified values (Belisle-Toler et al., 2021). For instance, if economic equity is a priority, strategies might include supporting small-scale local producers or creating markets that facilitate direct consumer-producer interactions. 1-3-Monitoring and Evaluation in Food Systems Monitoring and Evaluation (M&E) play pivotal roles in localized food system planning, ensuring that interventions are effective and sustainability goals are met. Local food systems, as complex and dynamic social-ecological systems, involve intricate interactions between human societies and natural environments. Effective M&E in this context necessitates an integrated approach that captures the diverse impacts of food production, distribution, consumption, and waste management across ecological and social dimensions (Fanzo et al., 2021). Traditionally, M&E practices have emphasized quantifiable outputs such as yield rates, economic returns, and resource use reduction. However, there is a growing shift towards incorporating broader ecological indicators like biodiversity, soil health, and water quality, alongside social indicators including food security, community well-being, and equitable resource access (Garton et al., 2022). This evolution reflects a deeper, more holistic understanding of sustainability within local food systems, acknowledging that true sustainability encompasses far more than mere economic efficiency and conservation (Ng'endo & Connor, 2022). 2 Despite these advances, significant gaps remain in the M&E practices within local food systems. One major challenge is integrating qualitative indicators that accurately reflect the social dimensions of sustainability, such as community satisfaction, cultural appropriateness of food, and social cohesion (Gaviglio et al., 2016; Maynard et al., 2020). These aspects are often overlooked in traditional M&E frameworks, which tend to prioritize quantifiable metrics. Moreover, there is a pressing need for improved temporal and spatial resolution in data collection, which would allow for more detailed, continuous monitoring to track changes over time and facilitate timely adjustments in management practices. Developing adaptive management practices informed by ongoing M&E is crucial; this involves not just data collection but also using this data to refine and adjust strategies based on feedback, thereby enhancing the resilience and adaptability of local food systems to new challenges and opportunities. As such, enhancing M&E practices involves a more dynamic, responsive approach that can pivot as necessary to support sustainable, effective local food system management. 1-4-Systems Thinking for Understanding the Food Systems Systems thinking is a conceptual framework that emerged in the mid-20th century, primarily from the work of biologists, ecologists, and management theorists who recognized the limitations of traditional linear analysis in understanding complex and interconnected systems (Von Bertalanffy, 2010; Arnold & Wade, 2015). The theory posits that to fully understand the behavior of complex systems, one must consider the system as a whole rather than merely focusing on its parts. This holistic approach was significantly advanced by scholars like Ludwig von Bertalanffy with General Systems Theory (Von Bertalanffy, 2010) and Jay Forrester’s work in system dynamics (Forrester, 1994). Systems thinking encourages the examination of the connections and interactions between components of a system, recognizing that the interdependencies can lead to emergent behaviors that would not be predictable from the properties of the individual components alone. This approach has been fundamental in addressing the multifaceted nature of ecological and social systems, where numerous interdependent factors must be considered simultaneously (Arnold & Wade, 2015). When applied to food system studies, systems thinking enables a comprehensive examination of how various components of the food system interact with each other and with external factors such as climate change, global markets, or local cultural practices. By recognizing the food system as a complex adaptive system, this approach helps to identify how agricultural 3 practices affect environmental sustainability, how market demands drive land use changes, or how policy adjustments might influence nutritional outcomes (Jagustović et al., 2019). It allows for an integrated analysis that considers not just the production and distribution of food but also socio- economic, environmental, and political dimensions (Monasterolo et al., 2016). Importantly, systems thinking facilitates the prediction of the impacts that different strategies and interventions will have on community-identified values (Martín et al., 2020). For example, it can illustrate how a policy aimed at increasing food production might affect water usage practices, labor conditions, and market prices, and in turn, how these changes might influence broader community goals such as food security, employment, and health. By enabling stakeholders to foresee and evaluate the potential consequences of actions within the food system, systems thinking ensures that plans are not only sustainable and equitable but also aligned with the values and needs of the community. This predictive capability is crucial for crafting resilient food systems that can adapt to both current and future challenges while continuing to meet human needs without compromising ecological health. 1-5-Resilience Thinking for Food System Planning Resilience theory, originating from ecological studies in the 1970s, fundamentally transformed how scientists and practitioners understand the dynamics and stability of complex systems. Pioneered by ecologist C.S. Holling, resilience theory initially focused on the capacity of ecological systems to absorb disturbances and still maintain their basic structure and function. Holling introduced the concept of resilience to describe the persistence of systems in the face of change and to explain phases of growth, collapse, and regeneration in ecosystems. This perspective was a departure from earlier views that emphasized stability and equilibrium, highlighting instead the dynamic nature of ecosystems and their ability to transition between multiple stable states (Holling, 1973). Over time, resilience theory has been expanded and applied beyond ecology to encompass social and economic dimensions, evolving into a crucial framework for understanding the interplay between humans and their environments in social-ecological systems (Folke, 2006). This holistic view recognizes that the resilience of a system is not just about resisting shocks but also about adapting to and transforming in response to changing conditions. In food system studies, resilience theory is particularly applicable for addressing the challenges posed by the increasingly unpredictable global climate and market conditions. Applying resilience thinking to food systems involves analyzing how agricultural practices, supply chains, 4 and consumption patterns absorb and adapt to disturbances such as pest outbreaks, market crashes, or extreme weather events (Tendall et al., 2015; Zurek et al., 2022). This approach encourages the exploration of strategies that enhance the food system's resilience, such as diversifying crops to reduce the risk of total crop failure, creating more localized food networks to minimize reliance on global supply chains, or implementing sustainable farming practices that improve soil health and biodiversity, thereby contributing to longer-term agricultural sustainability. By understanding and enhancing the resilience of food systems, communities can better withstand and adapt to disruptions, ensuring food security and sustainability in a changing world. 1-6-Community Engagement and Participatory Modeling Community engagement and participatory modeling are pivotal strategies in the management and transformation of food systems, enabling a more inclusive and comprehensive understanding of the challenges and opportunities within local and regional contexts (Wentworth et al., 2024). Community engagement involves actively involving stakeholders, especially those directly affected by food system issues, in the decision-making process. This approach ensures that the diversity of community voices, including those of smallholder farmers, local business owners, consumers, and marginalized groups, are heard and considered. A spectrum for different levels of community engagement was done by the “International Association for Public Participation” (IAP2, 2007). This spectrum introduces the five different levels of participation for the public in decision-making and planning. The first level “inform” just provides information to the public about the pre-determined alternatives and the decisions. In this level, there is no role for the public to provide any feedback before decision-making or assist in decision-making. The next level “Consult” will gain some feedback from the public and they will be informed how their feedback has influenced the final decisions. The next two levels “Involve” and “Collaborate” will work directly with the public to make sure that the public’s concerns have been incorporated into the decision-making. For the “Collaborate” level, the public will be involved in each phase of decision-making; especially in identifying the preferred solutions. The “Empower” level is considered the highest level of public participation and they will be engaged in every step, and they will have the ability to make the final decision. Participatory modeling extends this concept by involving community members in creating and refining models of their own food systems. This process not only democratizes knowledge and empowers participants, but also enhances the relevance and accuracy of the models by 5 incorporating local knowledge and values (Olabisi et al., 2023). For example, through participatory modeling, stakeholders can collaboratively simulate potential interventions in the food system, such as the introduction of new farming technologies or changes in food policy, and visually explore their potential impacts. This collaborative approach helps to align scientific analysis with community priorities, fostering solutions that are both innovative and culturally appropriate. Fuzzy Cognitive Mapping (FCM) is another powerful tool that can be integrated into participatory modeling to further enhance our understanding of food systems. FCM is a cognitive modeling technique where stakeholders build a graphical representation of a system, mapping out various elements and their causal relationships. Each relationship is weighted by its perceived strength, capturing the nuanced views of different stakeholders about the dynamics within the system (Gray et al., 2013; Gray et al., 2015). In food system studies, FCM can be used to illustrate and analyze complex relationships, such as the impact of climate change on crop yields or the effects of policy changes on food security (Morone et al., 2019). The 'fuzzy' aspect of this method allows for handling the ambiguity and uncertainty inherent in stakeholders' perceptions, making it particularly useful in situations where data may be incomplete or highly subjective. By engaging diverse groups in creating FCMs, researchers and policymakers can gain a richer, more nuanced understanding of the system dynamics and potential leverage points for intervention. 1-7-Dissertation Chapters As previously discussed, community-identified values are the foundational step toward achieving a desirable future for the community. The primary objectives of this dissertation are threefold: first, to explore various ways for measuring community-identified values within local food systems; second, to analyze the variations in expert predictions regarding the impacts of specific interventions within these systems; and third, to integrate these values into community- based resilience planning, emphasizing a balanced consideration of both monetary and non- monetary factors. As Figure 1-1 illustrates, Chapter 2 discusses different measurement ideas for monitoring and evaluating how food systems respond to community-identified values. Chapter 3 delves into understanding and modeling the complexities of food systems through the lens of systems thinking, enhancing our comprehension of the interplay between various components. Chapter 4 addresses how to strengthen food systems against sudden shocks and stresses. Together, these chapters synergistically work toward developing a more desired food system that is responsive and resilient, aligning closely with community needs. 6 Figure 1-1. Conceptual Framework for Integrating Community-Identified Values in Local Food System Planning The second chapter of this dissertation explores various methods for measuring community-identified values within food systems. This exploration is essential as it acknowledges the diversity of perspectives that different stakeholders contribute and highlights the challenges in quantifying values that are often subjective or culturally specific. Instead of proposing a rigid framework, this chapter delves into comparing the features of different measurement ideas to ensure they capture a comprehensive understanding of community needs and aspirations. By developing and aggregating ideas for measuring these values, this chapter directly contributes to more effective food system planning by ensuring that policies and interventions are grounded in a comprehensive understanding of community needs and aspirations. The identification and measurement of these values enable planners and policymakers to monitor and evaluate the effectiveness of their strategies more accurately, ensuring that the interventions are truly beneficial to the community. This sets the stage for the subsequent chapters by establishing the importance of integrating a wide range of community perspectives into food system planning. Building on the measurement ideas discussed in the second chapter, the third chapter delves into the mental models of food system experts, utilizing fuzzy cognitive maps to illustrate how different experts predict the impacts of known leverage points on community-identified values. This analysis not only reveals the degree of consensus or disagreement among experts but also contrasts these individual predictions with outcomes from a collective intelligence model. The significance of this chapter lies in its demonstration of how diverse expert opinions can lead to varied predicted impacts, underscoring the necessity of considering multiple perspectives in food 7 system planning. This chapter enriches the localized food system planning process by highlighting the potential discrepancies and alignments between individual and collective predictions, thereby informing more balanced and inclusive policymaking. The insights gained here complement the previous chapter’s focus on measurements by showing how different interpretations of data and projected impacts can influence planning outcomes. The fourth chapter extends the discussions from measuring and modeling community values to applying these concepts in community-based resilience planning for local food systems. It criticizes the dominant economic focus in strategy evaluation and advocates for a more comprehensive approach that includes non-monetary criteria such as community empowerment and partnership. This chapter is pivotal as it synthesizes the ideas presented in the earlier chapters into practical strategies for enhancing food system resilience against hazards and risks. By incorporating community-identified values into the evaluation process, this chapter promotes a more holistic approach to resilience planning, emphasizing the importance of stakeholder engagement and the exploration of trade-offs between different criteria. It ensures that the strategies not only are economically viable but also resonate with the community’s values. Together, these chapters create a cohesive narrative and conceptual framework that advances the field of localized food system planning by integrating community-identified values into every phase of policymaking, from measurement and modeling to implementation and evaluation. This approach highlights the interconnectedness of various planning phases, ensuring that local food systems are designed to be sustainable, culturally sensitive, and responsive to the specific needs of the community. 8 CHAPTER 2: MEASURING WHAT MATTERS: MONITORING COMMUNITY- IDENTIFIED VALUES IN LOCAL FOOD SYSTEMS Abstract Although various goals and targets have been established for the improvement of local food systems, the monitoring and operationalization of these targets necessitate further research to develop more appropriate measurements. This research focuses on 15 community-identified values (CIVs) that delineate the desired food system in Flint, Michigan. A survey was conducted among researchers and practitioners involved in the food system, gathering their insights on potential measurements for these CIVs, alongside evaluations of measurability and data availability. A total of 444 valid measurement ideas were collected, reflecting a broad spectrum of the Flint food system. Through inductive qualitative analysis, 21 subthemes and 5 main themes were identified within the data on "what has been measured." These measurement ideas were further analyzed using the perceived measurability and availability scores of respondents for their defined measurements to identify areas requiring enhanced data collection or innovative standardization of measurement development. The measurement ideas were also coded according to the six dimensions of food security—availability, access, utilization, agency, stability, and sustainability—to facilitate their application in other communities. This participatory approach not only generated a diverse array of measurement ideas but also underscored the significance of aligning these measurements with community-specific values and needs, bringing together local experts and academics to capture a comprehensive view of the local food system's diverse aspects. 2-1- Introduction Understanding food systems necessitates navigating their inherent complexity, which stems from the intricate interactions between diverse stakeholders, the multifaceted nature of food production, processing, and consumption, varying regulatory environments, and the delicate balance of economic, technological, social, and environmental factors (Meter, 2019; Zhong et al., 2021; Marshall et al., 2021). Determining the desired visions for these systems is particularly challenging, as each community has unique cultural, environmental, and economic characteristics that shape what they value in their food systems—from sustainability and local sourcing to affordability and nutritional quality (Ng'endo & Connor, 2021; Fanzo et al., 2021, Ammann et al, 2023). Recognizing these distinctive needs and values is crucial, and communities must actively 9 participate in defining these priorities. These priorities, in turn, should guide the selection of strategies and their implementation (Belisle-Toler et al, 2021). Despite significant efforts in understanding the local food systems' complexity, recognizing the current state and underlying issues, and selecting suitable interventions and policies, there remains a critical need for further investigating effective ways to monitor and evaluate local food systems' dynamics (Liddy et al., 2023, Schneider et al., 2023). It is essential to assess both the progress and obstacles of implemented strategies, and how they align with global and national food security goals and the community’s most valued visions for a desirable future (Bassarab et al., 2019; Kugelberg et al. 2021). This ongoing evaluation is crucial to ensuring that the local food system evolves in a way that genuinely supports and enriches the communities it serves (Fanzo et al, 2021; Manikas & Sundarakani, 2023). Continual feedback and adaptation of policies and selected strategies, guided by these evaluations and monitoring efforts, make the local food system more adaptive and responsive to emerging needs. Monitoring the effectiveness of policies and other interventions in food systems introduces significant hurdles. The scope of these evaluations is influenced not only by how communities identify desirable outcomes in their food systems but also by the complexities of cross-scale interactions (Wentworth et al., 2022). Navigating across various spatial, temporal, and institutional scales adds layers of complexity to the process of monitoring and evaluating policy implementations (Leiter, 2015). Consider, for example, an intervention aimed at improving food security in a small community through enhanced local food production. While such initiatives may succeed locally and improve immediate food availability, they could potentially disrupt regional food distribution networks or conflict with broader agricultural policies, thereby creating complex challenges for monitoring and evaluation. Temporal scales can further complicate monitoring: immediate outcomes might seem positive while longer-term effects, which unfold over years or even decades, may reveal unintended negative consequences. Additionally, the social and cultural factors that influence food practices and preferences are crucial for understanding the full impact of interventions on community well-being and cultural sustainability (Garton et al., 2022). These underscore the necessity for a comprehensive monitoring framework that accounts for these interactions and is capable of adapting to the dynamic nature of local food systems and related supply chains, ensuring that interventions achieve their intended outcomes by minimizing the unforeseen drawbacks. 10 The need for holistic monitoring in food systems calls for a multifaceted evaluation that is both interdisciplinary and considers the interactions between different food system sectors (Wilk, 2012; Horton et al., 2017; Zurek et al., 2018; Yuan et al., 2021). For instance, economists might measure the success of food systems through economic indicators such as market prices and employment rates in agriculture, while public health professionals might focus on measurements like nutritional outcomes and rates of food-related illnesses. Environmental scientists, on the other hand, could prioritize measurements related to soil health, biodiversity, and the use of agricultural chemicals. This diversity of measurements means that a monitoring approach that focuses solely on one dimension, such as economic viability, may overlook crucial aspects like environmental sustainability or public health. Therefore, this emphasizes the importance of interdisciplinary studies, which can help bridge these gaps and lead to more comprehensive monitoring of local food systems (Horton et al., 2017). While achieving consensus on a core set of measurements can be beneficial, it is not always essential. Allowing room for diverse perspectives can reveal new, context-specific insights, address unique community needs, and foster innovation in tracking hard-to-measure values to ensure that all relevant aspects of a local food system are adequately addressed and monitored. The process of determining what to measure and how to measure it is the result of a complex set of considerations that include factors such as available time, resources, and capacities, as well as the epistemological and disciplinary background of research team members (Prosperi et al., 2015; Boyd & Charles, 2006). This decision-making process is influenced by the need to balance scientific rigor with practical applicability, ensuring that selected measurements are not only scientifically sound but also feasible to implement within the constraints of the local food system. To ensure the effectiveness of these measurements, it is crucial to evaluate them against a range of criteria. These criteria should consider factors such as the availability of data at the appropriate scale, the measurability of indicators (some indicators may be challenging to quantify), cost- effectiveness (data should be accessible with acceptable monetary input), reliability and credibility (data should be collected rigorously and consistently from reliable sources and replicable across time periods), and understandability and usability (they should be easy to understand by people who might need to explain it to others) (Prosperi et al., 2015). Although the measurability of indicators and the availability of data are essential for evaluating measurements, focusing solely on these two criteria poses a potential risk: it may lead to the exclusion of crucial aspects of the 11 local food system that are inherently difficult to quantify or lack readily available data. This prioritization can result in a biased focus on easily measurable indicators. Given the complexities identified in the evaluation and monitoring the performance of local food systems, our study introduces qualitative and quantitative methods to encapsulate and explore the diverse dimensions of this process. Our approach leverages collective insights from a range of participants, including academic members and community-based experts, to develop measurements that monitor the values defined by the community, referred to as Community- Identified Values (CIVs) in this study. This collaborative effort is crucial in localizing food policy and tailoring monitoring strategies to align closely with community needs and priorities, thereby enhancing the efficacy of policymaking (Boyd & Charles, 2006, Allen et al, 2019; Dengerink et al., 2021). Furthermore, our study illuminates the potential differences in perspectives among participants, providing a deeper understanding of how diverse viewpoints contribute to defining measurements that cover different dimensions of local food systems. This insight is vital for formulating more balanced and inclusive food policy evaluation (Olabisi et al., 2023). This comprehensive analysis helps in crafting nuanced and effective monitoring efforts by acknowledging the diversity of viewpoints and values within the community. In doing so, we identify the prevalent themes and distinctive characteristics observed in the measurements regarding what they measure and how they measure, shedding light on the practical aspects of operationalizing community values into evaluation and monitoring of local food systems. Our findings offer a procedure that may be replicated or adapted in other contexts to improve the alignment of food policy and governance with community aspirations. This can be particularly useful for local entities such as food policy councils, food-related associations, and NGOs, providing them with robust tools for ongoing monitoring of interventions and refinement of food policies. Furthermore, different themes and sub-themes emerged from the measurement ideas, highlighting what has been measured. Although these measurements have been collected for monitoring the progress toward approaching CIVs for the specific location—Flint, Michigan, US, we categorized the selected measurement ideas into categories aligned with the six food security dimensions defined by Clapp et al. (2023)—see Appendix A for definitions of each dimension. This categorization facilitates generalized usage for other communities or at different scales, allowing for broader application of our collected measurement ideas in varied contexts, 12 potentially guiding the development of monitoring frameworks, policy evaluation and refinements, and enhancing the local food systems toward a more desirable future. 2-2-Materials and Methods 2-2-1-Data Collection The Flint Leverage Point Project (FLPP) is a community engaged research project aimed at understanding the Flint food system and promoting positive change1. The workshops were collaboratively designed to address the project's research questions, ensuring deep involvement of community members and stakeholders in meaningful dialogue.. As a result, various CIVs were articulated, highlighting the diverse dimensions that are considered important by the Flint community for a desirable food system (Belisle-Toler et al, 2021). Each of these CIVs represents a specific aspect of the Flint food system that the community believes should be prioritized to reach their desired food system in the future. These CIVs and their qualitative definitions are listed in Table 2-1, providing a clear framework for targeted evaluation and ongoing monitoring of the local food system improvements. To effectively assess these qualitative CIVs, a survey was designed, featuring sections corresponding to each of the CIVs. The survey was distributed to a listserv comprising experts in food systems, all of whom either had affiliations with community organizations or universities and were actively engaged in food system projects—inside and outside of Flint. Survey participants were asked to state their job affiliation and then propose one or two measurement ideas for monitoring each CIV. They were also required to evaluate the measurability of their proposed measurement ideas and assess the availability of the necessary data using a 5-point Likert scale. A total of 31 participants completed the survey, initially providing a broader set of measurement ideas. After excluding responses where participants indicated uncertainty about definitions, offered no ideas, proposed measurements irrelevant to the 15 CIVs, or left the response blank, 444 acceptable measurement ideas remained, suitable for monitoring the local food system. These remaining measurement ideas, along with their associated perceived measurability and availability scores, were then qualitatively and quantitatively analyzed as described in the following section. 1 https://www.canr.msu.edu/flintfood/ 13 2-2-2-Data Analysis 2-2-2-1-Qualitative Data Analysis An inductive qualitative content analysis (Thomas, 2003) was conducted on the collected data to explore the measurement variables that respondents provided for monitoring the CIVs, such as the distance to grocery stores, the amount of funds allocated to a specific program, available educational resources, and community perceptions about a specific topic. These measurement variables were synthesized into themes and sub-themes. The number of codes assigned to each sub-theme enabled comparisons between the most and least frequently cited measurement variables for monitoring the food system. The frequency of codes for each main theme was compared to reveal contextual similarities or differences across CIVs and between two respondent groups—academic members and community experts. Additionally, the sub-themes of measurement variables were used to examine the diversity of perspectives in defining indicators for each CIV. The Shannon diversity index (Sarma et al., 2015) was employed to calculate and compare diversity indices across all CIVs. In the formula below, Hi represents the Shannon diversity index for the ith CIV, S is the total number of sub-themes and pij represents the proportion of the jth sub-theme for monitoring the ith CIV. This analysis led to the development of a spectrum indicating the diversity of perceptions for measuring each CIV. It highlighted that respondents have varying levels of consensus about how to measure the changes associated with each CIV. H i = - ∑ 𝑆 𝑗=1 𝑝𝑗𝑖 log(𝑝𝑗𝑖) There are various methods for procuring information and data necessary for monitoring efforts, which involve different ways to measure the measurement variables. Each method can offer unique insights into the dynamics of CIVs within a local food system. These insights are crucial as they link the practical steps of measuring to understanding different types of knowledge, as categorized by Miller et al. (2008). These types of knowledge—mechanistic, contingent, and narrative—provide distinct perspectives that are essential for effective monitoring. Mechanistic knowledge enables the analysis of empirical data, such as statistical trends, which is vital for understanding systematic changes in the food system. Contingent knowledge helps assess how factors like geographic location or economic conditions influence food-related variables. Narrative knowledge offers insights into how personal and cultural narratives shape behaviors and decisions regarding food, which is crucial for capturing the more qualitative aspects of food systems. Contingent knowledge helps assess how factors like geographic location or 14 economic conditions influence food-related outcomes. Narrative knowledge offers insights into how personal and cultural narratives shape behaviors and decisions regarding food, which is crucial for capturing the more qualitative aspects of food justice. Together, these approaches enable a more nuanced understanding of the food system, enhancing the monitoring of changes in CIVs. To further align these insights with practical applications, a deductive qualitative analysis (Proudfoot, 2023) was conducted to identify the types of knowledge respondents used when defining their measurements, with a comparison of the frequency of assigned codes across CIVs and respondent groups. Food security is a multifaceted concept traditionally encompassing availability, access, utilization, and stability (Schmidhuber & Tubiello, 2007), but recent expansions include two additional dimensions—agency and sustainability—to address broader challenges in global food systems (Clapp et al., 2022)—see Appendix A for definitions of each dimension. The agency dimension emphasizes the roles and choices of individuals and communities in securing access to adequate food, empowering them to influence food-related decisions and actions despite potential disruptions. Sustainability, on the other hand, focuses on long-term environmental, economic, and social viability to support food security for future generations. In this context, the measurement ideas that had been collected to monitor the 15 CIVs recognized by the Flint community were qualitatively analyzed and codes assigned to the measurement ideas based on these six dimensions of food security. This categorization was undertaken to explore possible overlaps between the CIVs and the six dimensions of food security, thereby enhancing our understanding of how local initiatives align with broader food security goals. Specifically, we categorized the selected measurement ideas into categories aligned with the six food security dimensions defined by Clapp et al. (2023)—see Appendix A for definitions of each dimension. This approach not only helps generalize the usage of these categories for other communities or at different scales, enhancing the application of our collected measurement ideas in varied contexts, but also potentially guides the development of monitoring frameworks, policy evaluation, and refinements, thereby enhancing the local food systems toward a more desirable future. Moreover, this approach enables a more integrated and comprehensive method for monitoring and enhancing food security at the community level, diverging from the prevailing trend where food security indicators and metrics are predominantly tailored for regional or national scales. 15 Regarding perceived measurability and availability, respondents were asked to rate their proposed measurement idea using a 5-level Likert scale to evaluate perceived measurability and availability. These ratings facilitated the statistical analysis for comparison of consistency or inconsistency in perceptions across the CIVs or participant groups. The measurement ideas were then categorized into five distinct groups: 1) low measurability and low availability scores, 2) low measurability and high availability scores, 3) high measurability and low availability scores, 4) high measurability and high availability scores, and 5) other measurements that received at least one medium-level score in either measurability or availability. The qualitative analysis primarily focused on the first four categories to explore their interconnected characteristics and to derive insights on how each category could be optimized or enhanced for more effective ongoing food system monitoring initiatives. This structured approach helps in identifying potential areas for improvement and prioritizing resource allocation in data collection and measurement development for local food system monitoring efforts. The aforementioned inductive and deductive qualitative analyses of measurement ideas were all conducted within the MAXQDA software. To ensure reliability and agreement on this assessment, specifically through intercoder reliability, the codes assigned to measurement ideas for five CIVs by two researchers were compared. A reliability threshold was set at 90%, and in instances where discrepancies or differences arose in the assigned categories, collaborative discussions were engaged in to resolve any disagreements and achieve consensus. 2-2-2-2-Statistical Tests To explore statistical associations within and across CIVs, respondent groups, and other categories identified through qualitative analysis, various statistical tests were employed. The chi- square test of independence was utilized to evaluate if there's a significant association between two categorical variables, indicating whether changes in one variable are linked to changes in another (McHugh, 2003). The Mann-Whitney U test compares distributions of two independent groups to ascertain if they differ in central tendencies, especially helpful when assumptions of normality or equal variances are violated (McKnight & Najab, 2010). Extending this analysis, the Kruskal- Wallis test allows comparison across three or more groups, assessing if there are significant differences in distributions without relying on normality assumptions (Ostertagova et al., 2014). 16 2-3-Results 2-3-1-Comparing the Number of Valid Measurement Ideas Per CIV As mentioned in previous sections, data cleaning was conducted by excluding invalid responses, including those where participants indicated uncertainty about the definition of CIV, offered no ideas, proposed measurements irrelevant to the CIV, or left the response blank. The subsequent evaluation of valid versus invalid measurement ideas across CIVs and among academic members and community experts reveals that certain aspects of the food system were identified as more challenging for the respondents to measure or quantify. Specifically, "Comfort and Safety", "Tradition", and "Economic Justice" had higher proportions of no-answers or invalid measurement ideas, suggesting these dimensions are more complex and difficult to measure. This complexity may arise from the subjective interpretations and cultural nuances inherent in these CIVs, as well as possible systemic gaps in existing research and measurements that prioritize economic and production-focused indicators over socio-cultural indicators, requiring a broader socio-economic understanding that varies significantly across different demographics. In contrast, "Urban Farming", "Healthy Food System", and "Affordability" had a higher rate of valid measurement ideas, indicating these CIVs deemed easier to measure for the respondents. This ease likely stems from the tangible impacts these CIVs have on community well-being and the food environment. Additionally, increasing public and academic engagement related to these CIVs have enhanced the ability to define more appropriate measurements. Table 2-1 provides a comprehensive overview, including detailed definitions of each CIV alongside examples of valid measurement ideas. These samples exemplify how diverse perspectives can be leveraged to monitor each of the CIVs. Additionally, Figure 2-1 depicts the percentage of valid and invalid measurement ideas for each CIV, categorized by the two groups of respondents. In comparison, academic members typically contributed a higher proportion of valid measurement ideas than community experts. 2-3-2-Themes and Subthemes for Measurement Variables As discussed in Section 2-2-1, an inductive qualitative analysis was conducted to evaluate the measurement variables—specific aspects extracted from measurement ideas. For instance, for the measurement idea of "# of grocers, producers, and restaurants offering ethnic or regional cuisine," the variables "# of grocers, producers, and restaurants" and "availability of ethnic or regional cuisine" were coded as subthemes of "food outlet density" and "food category," respectively. Through this analysis, we identified 21 sub-themes, which were then categorized into 17 five main themes: 1) Food Landscape, 2) Community, 3) Individuals and Households, 4) Food Characteristics, and 5) Supports and Services. This coding schema provided a robust set of measurement variables that can effectively monitor the complex interactions within local food systems and evaluate progress toward achieving CIVs. Table 2-2 presents a detailed explanation of each of the 21 sub-themes identified through the inductive qualitative analysis. Additionally, Figure 2-2 illustrates the frequency of assigned codes (subthemes) across the CIVs and among the participant groups and represents how often these sub-themes were utilized by participants in developing their measurement ideas. 18 Table 2-1. Definition of CIVs based on workshops with community members and example valid measurement ideas for monitoring each of the CIVs CIVs Statements Example Measures and Indicators Affordability Comfort and Safety Common Good Ensuring that food prices are within the financial reach of individuals, particularly those with lower incomes, to maintain access to nutritious options without financial hardship Ensuring a food environment where community members feel secure, welcomed, and comfortable, free from racial/gender discrimination during purchasing or accessing food. Ensuring public welfare through accessible food options and government assistance programs like SNAP and WIC, regardless of income status. Convenience Ensuring easily accessible and readily available food options to accommodate busy lifestyles and limited time constraints. Economic Justice Prioritizing fairness and equity within the food system, emphasizing community well- being over corporate profits, and striving for equitable resource distribution. Economic Opportunity Fostering local ownership and economic advancement within the food system, with a focus on supporting small businesses and empowering local communities to thrive. Price of identical or similar food products compared to state average price (in dollars) Per capital income of local population Ask people what they think is "affordable" in terms of money spent on food per month Crime stats for stores and their surrounding areas Assess perceptions of safety by community Assessment of whether stores/lots are well-lit, regularly patrolled, and frequented by a diverse range of patrons % of businesses that provide living wage jobs $value of food welfare/charitable support provided by corporate food entities Number of people experiencing food insecurity Number of takeout options in that location Average miles from home to full grocery store Household-level consumption pattern of prepared foods Prevalence of not-for-profit food stores/co-ops etc. in a locality Amount of store wages and profits recirculated into community Ratio of number of people employed at a livable wage to those that are employed at a non- livable wage throughout the food system Percentage of food businesses owned by residents of the municipality where the business is located. Dollars invested in local ownership and economic advancement per year Do local colleges/universities have related classes? Do they promote this? 19 Table 2-1. (Cont’d) CIVS Statements Example Measures and Indicators Education Feeling of Community Food Diversity Food Waste Fresh and Natural Food Offering opportunities to learn essential food skills like cooking, gardening, and canning, alongside educational programs focusing on nutrition and food literacy. Fostering trust and camaraderie within the food system, emphasizing shared experiences and opportunities for food sharing among neighbors and social circles. Providing a range of culturally relevant food options within the food system that cater to diverse needs, tastes, and preferences Minimizing the amount of discarded food within the food system, focusing on reducing waste at both organizational and individual levels Prioritizing minimally processed options with fewer additives and preservatives Number of food-related educational events held within a specific timeframe and measuring participant attendance levels % of people in the community with these skills What incentives are there to encourage participation to the educational programs? Percentage of people in population who feel like they regularly meet new people in their community over food (e.g., by where they buy their food) Over time, measure the level of community that people feel through survey Are the owners/vendors involved in the neighborhood they serve? Do they live there? # of grocers, producers and restaurants offering ethnic or regional cuisine Can households get the diversity of foods they want? Presence of educational opportunities aimed at teaching individuals diverse cooking and food preparation techniques lbs of food waste discarded by restaurants, homes and grocery stores # of food rescue programs and/or # of food composting programs percentage of pounds of food harvested that makes it to market destination # of fresh/natural food items available at highly trafficked grocery stores Availability of food without certain additives or with excessive salt, sugar, oil, etc. Comparing the price of fresh, quality food items to packaged or processed alternatives 20 Table 2-1. (Cont’d) CIVS Statements Example Measures and Indicators Health Local Food Tradition Urban Agriculture Prioritizing nutritious food options that are rich in essential nutrients and available in smaller portions Providing food options sourced from nearby producers or within the local region Honoring and participating in cultural, familial, and religious food customs, preserving traditional practices in food preparation and consumption. Promoting community and personal gardens within the food system, fostering participation in urban agricultural practices. Overall nutritional diversity (% of necessary nutrients) readily available within a store/neighborhood Access to healthy food, including cost, travel time and options. Prevalence of diet-sensitive chronic disease at population level. Access to healthy food, including cost, travel time and options. Prevalence of diet-sensitive chronic disease at population level. Farmers market hours per year per population or unit area Identify elders in the community and ask them to evaluate this on a Likert scale or similar # of food-focused cultural events people can practice or food festivals. Tracking historical and cultural practices passed down through generations within families or cultures Acreage of urban farms (current & planned) / total acreage of region The monetary amount given to farmers within an urban geographic region by grant and loan funding. Percentage of food from urban farming being stocked in stores or purchased by local restaurants 21 Figure 2-1. Comparison of valid vs invalid measurement ideas across CIVs and among respondent groups Each category of measurement variables captures unique aspects of local food systems that can help for more comprehensive monitoring and policy evaluation. The "Food Landscape" theme, accounting for 35.6% of assigned codes, is pivotal for assessing physical and logistical aspects such as access and distribution patterns. This is demonstrated by measurement variables like available food (e.g., pounds of fruits and vegetables) and food retailer distribution (e.g., number of full-service grocery stores). The "Food Characteristics" theme, representing 21.1% of the codes, focuses on the quality and diversity of food products, with food price as its dominant measurement variable. These measurement variables are critical for promoting healthier and culturally appropriate food choices. Community-related measurement variables, which constitute 18.7% of the codes, reveal aggregated social dynamics and social norms that are crucial for assessing community capacity to address food-related issues. While the previously discussed three themes dominated the measurement variables suggested by participants, the Supports and Services and Individual/Household-related measurement variables, though less frequent, are equally important for a holistic understanding of the local food system. Supports and Services measurement variables, accounting for 14.7% of the 22 assigned codes, primarily focus on the monitoring of structural supports such as financial aid. This theme aids in the development of policies that resonate with CIVs and address specific structural needs within the food system. On the other hand, Individual/Household-related measurement variables, which represent 9% of the assigned codes, are crucial for assessing interventions aimed at influencing consumer behaviors towards more responsible consumption practices and behavioral changes. Figure 2-2. Frequency of assigned codes to the measurement ideas based on the subthemes for the measurement variables 23 Table 2-2. Themes and subthemes extracted from measurement variables used by participants in developing their measurement ideas Theme Sub-Theme Educational Opportunity Supports and Services Food Aid Distribution Individuals and Households Financial Support Choice Flexibility Consumption and Purchasing Patterns Waste Management Cultural Food Dynamics Food Price Nutritional Quality Food Characteristics Food Process Level Food Categories Food Miles Definition of sub-themes It encompasses measurement variables such as frequency of educational events, variety of food-related educational services, and distribution of educational materials. This sub-theme covers the provision and scale of supplementary and emergency food services such as SNAP (Supplemental Nutrition Assistance Program), WIC (Women, Infants, and Children), food banks, and food pantries. It covers the provision of financial support including funds, grants, loans, and efforts to facilitate access to these services. It emphasizes the flexibility and autonomy that individuals or households have in accessing and choosing their food, such as the number of food procurement options available (e.g., supermarkets, local markets, online delivery services). It encompasses key aspects of individual and household food consumption and purchasing behavior. It includes the frequency of purchases, the types of food purchased, spending patterns, sources of food, dietary preferences, and nutritional intake. It includes measurement variables such as household food waste levels and composting practices, tracking the amount of food discarded and waste management strategies. it encompasses the measurement variables related to the cultural, ethnic, religious, and traditional food practices and preferences of individuals and households it includes the price of different types of foods or food price comparison through different spatial scales or trends it assesses the nutritional content, including carbohydrates, proteins, minerals, and vitamins, present in food products It assesses the degree to which food products have been altered from their original state, focusing on aspects such as the composition and preservation of prepared meals. It refers to the measurement variables that assess the variety of food categories offered within a defined area like food retailers or neighborhoods. It focuses on the distance food travels from production to consumption, and its related impacts like greenhouse gas emissions. 24 Table 2-2. (Cont’d) Theme Sub-Theme Community Perceptions Community Engagement Definition of sub-themes • It explores how communities perceive different aspects of the food system through qualitative methods like surveys or interviews. • It evaluates the level of engagement or participation of community members in events, festivals, local initiatives, or learning opportunities related to the food system. Community Public Health • It concentrates on health outcomes, particularly chronic food- related disease trends or status, at the population level Community Knowledge • It assesses the collective knowledge of the community regarding specific topics within the food system, such as cooking skills, nutritional values, or gardening expertise Community Safety Food Outlet Density • It focuses on measures related to community safety such as crime rates, infrastructure safety, and public health regulations, that could impact food accessibility or other aspects of the food system. • It encompasses measures related to food retailers, including their density or number in specific locations, as well as their specific characteristics. Food Inventory and Supply • It assesses the physical presence of various food types within specific spatial boundaries, typically by comparing quantitative amounts, often in pounds or other weight units. Distance to food sources • It involves measurement variables that typically gauge the average distance consumers must travel to access food Agricultural Lands • It includes measurement variables related to the acreage of agricultural lands in various settings such as suburban and urban areas, rooftops, and high tunnels. Food Landscape To investigate potential contextual differences in measurement variables used by two groups of respondents—academic researchers and community experts—and across the CIVs, we compared the frequency of coded themes. As illustrated in Figure 2-3, there were no significant differences in the themes assigned to measurement ideas between the two groups for each CIV. However, variations were evident across the different CIVs. For example, CIVs such as comfort and safety or tradition predominantly utilized measurement variables related to the Community theme. In contrast, for values such as food diversity and economic justice, measurements related to the Food Characteristics theme were more dominant. This indicates that while the fundamental measurement themes are consistently utilized across different respondent groups, the focus of these measurements can significantly vary depending on the specific CIV being prioritized. 25 Figure 2-3. Comparison of theme frequency across CIVs and among respondent groups The Shannon diversity indices were calculated to assess the variety of measurement variables used, based on the frequency of assigned codes (sub-themes) across each CIV and respondent group. Figure 2-4 displays the diversity index for each CIV, indicating variations in the range of ideas among participant groups. A higher diversity index presents a more varied utilization of measurement variables for monitoring the CIVs. Our findings reveal that community experts typically employed a more diverse array of measurement variables across most CIVs compared to academic members. Particularly, CIVs such as Common Good, Economic Justice, and a Healthy Food System exhibited the highest diversity indexes, illustrating their complex and multifaceted nature. These values often involve multiple stakeholders and dimensions, necessitating varied approaches to capture their changes. In contrast, values like Affordability and Comfort and Safety presented lower diversity indexes, suggesting more uniform ideas about what to measure for monitoring these aspects. This uniformity is likely due to more established or universally accepted metrics within these CIVs, which might not require as much innovation or adaptation in measurement strategies. The variation in perspectives emphasizes the critical need for developing inclusive and participatory-based monitoring strategies for the food system that 26 effectively reflect and respond to the diverse needs and values of the communities they aim to serve. Figure 2-4. Shannon diversity index for measurement variables utilized in each CIV by Participant Group 2-3-3-Perception of Measurability and Availability In assessing the differences in measurability and availability scores between two groups of respondents—academic members and community experts—academic respondents tend to report higher measurability scores and lower availability scores compared to community experts. Despite these variations in average scores, statistical analyses conducted using the Mann-Whitney U test showed no significant differences in perceptions of measurability and availability between the two groups. This suggests that both academic and community experts generally hold similar views on the measurability and availability of data for their defined measurements across CIVs. It’s important to note that these findings are based on our sample of 31 respondents. This analysis can be valuable because it highlights a shared understanding between academics and community members, which may facilitate collaborative efforts in local food system planning. By identifying 27 common ground in their perceptions of data measurability and availability, future initiatives can more effectively bridge the gap between theory and practice. Further investigation into the scores across various CIVs, regardless of respondent groups, was conducted using the Kruskal-Wallis test. For measurability, the test indicated no significant differences across the CIVs, suggesting a uniform perception of measurability among these variables. Conversely, a significant difference was observed in the availability scores across CIVs (Kruskal-Wallis test statistic = 57.36, p-value < 0.001), indicating varied perceptions of data availability across different CIVs. CIVs such as healthy food systems, convenience, and local food had higher availability scores, while tradition, feeling of community, and comfort and safety recorded lower scores. This significant variation in availability scores highlights areas where certain required data are perceived as less readily available for monitoring of CIVs. As discussed in Section 2-2-1, four groups of measurement ideas were qualitatively analyzed to investigate their distinct features. The following paragraphs provide a detailed description of each group of measurement ideas based on the measurability and availability scores. 2-3-3-1-High Measurability and High Availability This ideal category encompasses 37.4% of the measurement ideas and includes metrics and indicators that are both well-defined and supported by readily accessible data, as perceived by the participants. Measurements in this group are the most conducive to reliable monitoring, ensuring effective and efficient data collection and analysis. All of the CIVs featured at least a few measurement ideas in this category, although 'Feeling of Community' and 'Tradition' had the fewest indicators. In contrast, 'Affordability' and 'Urban Farming' were the dominant CIVs in this category, highlighting the existence of various measurement ideas that had high measurability and availability scores. Notably, there was no dominant measurement variable as all 21 sub-themes were equally represented, demonstrating a comprehensive and balanced approach to capturing diverse aspects of the food system 2-3-3-2-Low Measurability and High Availability Measurements in this category constitute 12% of all collected measurement ideas and are supported by readily available data. However, these measurements themselves are inherently difficult to accurately measure the intended CIVs regarding the measurability scores by participants. Challenges in this category often arise from a lack of standardized measurement procedures or the complexity of translating available data into meaningful insights for monitoring 28 purposes. Both groups of participants had almost equal representation in defining these measurements—community experts (56%) and academic researchers (44%). The dominant CIVs in this category include "Healthy Food System" and "Economic Opportunity". The primary measurement variable was the attributes of food retailers. Example measurement ideas in this category include "Number of farmers' market hours per area or population" and "Change in the number of licensed food businesses, with breakdowns by age, race/ethnicity, and gender." This category highlights the ongoing need to refine these indicators for better clarity and efficacy in monitoring food systems. 2-3-3-3-High Measurability and Low Availability This category, which accounts for 9.5% of all collected measurement ideas, includes measurements that are conceptually well-defined and lend themselves to precise measurement for specific CIV but face challenges due to the unavailability of requisite data, as reflected by the participants' assigned scores. The lack of accessible data for these well-structured measurements typically arises from issues such as restricted data access, cost constraints, or the specialized nature of the data, making routine monitoring difficult. A significant majority, 71%, of these indicators were defined by academic members. Dominant CIVs in this group include “Food Waste” and “Economic Justice”. Example measurement ideas include "pounds of food waste discarded by restaurants, homes, and grocery stores," and "amount of revenue (dollars) generated by food businesses that is reinvested into the community." Strategies to enhance data accessibility, potentially through partnerships with data holders or investments in data generation projects, are essential to enable more effective monitoring and implementation of food interventions. 2-3-3-4-Low Measurability and Low Availability Measurements in this category comprise 9% of all the collected measurement ideas. This category suffers from inability to effectively and accurately measure the intended CIVs based on the participants' assessments; often, they are proxies that fall short of capturing the essence of the targeted outcomes. Accompanied by a lack of accessible data, this category was perceived by participants as having lower data availability. This dual shortfall undermines the monitoring effectiveness of these measurements, indicating a need for fundamental work in developing more conceptually sound ideas and enhancing data collection strategies. Notably, 61% of these indicators were defined by community experts, predominantly focusing on CIVs such as “Tradition” and “Comfort and Safety.” The primary measurement variables for these measurement 29 ideas center on community perceptions and the quantity of food available. Example measurements in this group include "asking elders in the community to rate traditional food practices on a Likert scale," or "conducting customer surveys at various outlets to assess their safety and comfort levels." Considering these two examples, the absence of standardized tools or protocols can make it challenging to design reliable and valid measurements for this group of measurements. Collecting data from elders or customers requires direct engagement through surveys or interviews, which can also be time-consuming and resource intensive. These measurements highlight the need to promote both the conceptualization and the methodologies used for data collection, aiming to bridge the gap between theoretical objectives and practical implementation. 2-3-4-Data Acquisition for Measurements After exploring the measurement variables and assessing the perceived measurability and availability scores, we analyzed the measurement ideas to understand how respondents suggested using different categories of knowledge to inform the data collection process. The Venn diagram presented in Figure 2-5 illustrates the distribution and overlap of different knowledge types— mechanistic, contingent, and narrative—used in the measurement ideas for monitoring CIVs. Knowledge characterized as mechanistic, constituting 24.5% of the codes, primarily reflects an objective understanding of the systematic and predictable nature of monitoring efforts. This is crucial for developing standardized measurements that are consistently reliable across different local food systems. Knowledge categorized as contingent, accounting for 16.9% of the codes, emphasizes the dependency of data collection on specific conditions, highlighting the importance of contextual sensitivity in measurements. The knowledge described as narrative, representing 9% of the codes, captures the individuals' stories and experiential aspects of knowledge, crucial for incorporating the subjective and qualitative dimensions of both community and individual experiences. The significant overlap observed between mechanistic and contingent knowledge (36.9%) suggests that a large portion of measurement ideas integrate both standardized and context- sensitive approaches for data collection, indicating a comprehensive strategy in monitoring efforts. However, the relatively smaller intersections with narrative knowledge (5.2% with mechanistic and 7.4% with contingent) underscore a potential underutilization of qualitative, story-based insights in development of measurements. This implies that while current measurements are robust in quantifiable and situational analysis, there may be an opportunity to enhance monitoring 30 frameworks by incorporating more narrative approaches. Integrating narrative knowledge can provide deeper insights into community values and experiences, thereby supporting more effective policy evaluation and adaptation to local needs. Such an integration could lead to more nuanced and effective monitoring and evaluation practices that better capture the complexities of local food systems’ dynamics and outcomes. Figure 2-5. Percentage distribution of different approaches for data collection (different types of knowledge) across the collected measurement ideas 2-3-5-Food Security Dimensions Quantitative analysis of measurement ideas for monitoring our 15 CIVs reveals their overlap with the six dimensions of food security, as depicted in Figure 2-6. The bar chart depicts the proportional distribution of codes assigned to measurement ideas across the dimensions— Access, Availability, Utilization, Stability, Agency, and Sustainability. The data show a notable variance in emphasis on different dimensions, with 'Agency' receiving the highest proportion of codes at 27.35%, underscoring its significant role in the participants' approach to monitoring the food system. In contrast, 'Stability' received the fewest, only 3.50% of the total, indicating it is the least emphasized dimension among the study's participants. This disparity suggests a strong focus on enhancing and understanding the capacity and rights of individuals within the food system (as 31 reflected in 'Agency'), while there appears to be less focus on the stability of the local food system over time ('Stability'). The percentages for Access (20.99%), Availability (16.69%), Utilization (17.01%), and Sustainability (14.47%) highlight a moderate level of interest in these areas, suggesting a balanced concern for both the immediate and long-term aspects of food security beyond the extremes represented by Agency and Stability. It's important to note that the measurement ideas developed for monitoring the 15 CIVs for the Flint food system encompass all six dimensions of food security, ensuring a comprehensive approach to assessing and improving food security within the community. This inclusiveness is largely attributed to leveraging the collective intelligence of participants, fostering a co-developed measurement ideas that thoroughly covers different aspects of the local food system (Moragues-Faus & Marceau, 2018). Figure 2-6. Proportional distribution of codes assigned to measurement ideas based on the six dimensions of food security 32 2-4-Discussion Benefiting from participatory approaches for the development of measurements to monitor changes in social-ecological systems has a long history (Swain & Hollar, 2003; Hibbard & Lurie, 2012; Carnegie et al., 2019; Wanjiru & Xiaoguang, 2021). However, few participatory efforts have been designed to develop measurements specifically for food systems (Cleveland et al., 2015; Moragues-Faus & Marceau, 2018). This study demonstrated how it is feasible to leverage participatory approaches for the development of diverse ideas for understanding the status quo and changes in CIVs for the Flint food system. This yielded a rich data set containing brilliant ideas covering diverse aspects of monitoring—even those CIVs perceived as challenging to measure, such as "comfort and safety" and "tradition". The qualitative analysis of these proposed measurements, which include a variety of themes and sub-themes, suggests that no single measurement suffices. Instead, a customized selection of measurements that match CIVs is essential. Moreover, the diverse ways of acquiring knowledge embedded in these measurement ideas illustrate that participants may hold different epistemological views on how to acquire the necessary data—whether through numerical data, regional contexts, or community narratives. Acknowledging and integrating these varied approaches can provide a more comprehensive framework for monitoring, well-suited to the complexity of local food systems. This holistic understanding ensures that the progression toward a desirable future is effectively captured, reflecting the true diversity of perspectives within the community. Furthermore, the findings underscore the value of community-based participatory research and participatory science, as they demonstrate how collaborative efforts with community members can enhance the development of meaningful and context-specific measurements for complex systems like food systems. While there are numerous well-designed measurements for assessing national-level concerns such as food security (Jones et al, 2013), sustainability of food systems and agricultural production (Gustafson et al, 2016), and hunger (Gödecke et al, 2018), these metrics may not align with what the community values most and describes as essential for their ideal local food system (Dengerink et al., 2021). Although different well-established metrics and indicators exist for measuring the affordability, availability, and accessibility of food, there are no standardized measurements for "common good", "feeling of community", or "tradition"—priorities identified by the Flint community for their food system. Therefore, to effectively navigate the impacts of various interventions, strategies, and policy assessments, the development of measurements that 33 can monitor these specific values using a localized perspective is crucial. This approach can pave the way for more effective evaluations of local food system dynamics. Although the measurement ideas collected for monitoring CIVs specific to the Flint food system, as discussed in the results section, they also have potential applicability in other communities to monitor the six dimensions of food security (Clapp et al., 2022). Different criteria exist for the evaluation of measurement, as described by Prosperi et al. (2015). However, their measurability and the availability of data play a critical role. In this study, the assessment of measurability and availability scores for each measurement idea proposed by participants led to the identification of CIVs that need more available and accessible data, such as Food Waste and Economic Justice, or require more standardized measurement procedures, such as Healthy Food System and Economic Opportunity. Based on the survey data, only 34% of measurement ideas had high measurability and availability scores, while the remaining 66% faced challenges in either data accessibility or precision in the measuring process. This insight might help in monitoring local food systems, as it highlights the need for purposeful data collection and the development of more standardized evaluation methods, ensuring that CIVs are accurately tracked and integrated into local food system planning. Developing measurements for monitoring CIVs through a participatory approach can present several challenges. One potential issue is the difficulty of bringing together a sufficient number of experts and balancing the inclusion of participants with diverse backgrounds and experiences to brainstorm possible measurement ideas that cover different aspects of the local food system. In our study, we asked participants to propose measurements for all 15 CIVs, resulting in a lengthy survey that contributed to a lower completion rate, with only 31 participants finishing the survey. Although this small number of participants provided 444 measurement ideas that addressed 21 sub-themes, assigning fewer CIVs per participant could have shortened the survey and potentially increased the number of respondents and diversity of measurement ideas. Additionally, only 58% of participants had direct experience with research or projects in Flint, while the remaining participants were academics or community-based experts from outside the area. Increasing the proportion of Flint-based participants, who have a deeper understanding of the local food system dynamics and data availability, could enhance the validity of the proposed measurement ideas. Finally, developing a dataset of measurement ideas is only the first step; it is crucial that these ideas be evaluated by local NGOs or food policy councils, and the research 34 findings be disseminated. Different monitoring and intervention efforts require specific sets of measurements based on available local resources and data, making local communication and validation of measurements essential for effective monitoring and implementation. The findings of this chapter have been communicated to the Flint Food Policy Council for further evaluation and potential incorporation into their future reports and assessments. 2-5-Conclusion Our findings underscored the role of CIVs as targeted system outcomes in the monitoring of local food systems, indicating that CIVs serve as benchmarks for assessing the effectiveness and alignment of local food initiatives. By treating CIVs as targeted outcomes, monitoring efforts can directly measure the extent to which local food systems are meeting the specific priorities and needs identified by the community (Atoloye et al., 2023). The use of a participatory process in our study not only yielded a diverse array of measurement ideas but also highlighted the importance of aligning these measurements with the community's values and needs. This approach brought together a group of participants, including local experts and academics, capturing a wide range of perspectives on the different aspects of the local food system. Such an approach proved instrumental in developing measurements that are both comprehensive and culturally relevant, ensuring that monitoring efforts are deeply rooted in the community's realities and aspirations (Lam et al., 2019). The variety of measurement ideas generated through this process reflected the complexity of food systems and underlined the need for adaptable monitoring tools that can respond effectively to this complexity, crucial for the effective implementation of policies and interventions aimed at enhancing local food systems (Fanzo et al., 2021). Furthermore, the participatory approach demonstrated its efficacy in bridging theoretical models with practical applications in food system monitoring. By involving community members in the development of measurements, our study not only enriched the pool of ideas but also enhanced the legitimacy and acceptance of the monitoring processes. This inclusive method of data collection ensures that the resulting measures are multifaceted and responsive to the dynamic changes within the food system, providing a robust framework for ongoing evaluation. Thus, the participatory process aids not only in identifying what to measure but also in determining how these measurements should be implemented and adjusted over time, reflecting the evolving needs and conditions of the community. As local food systems continue to face pressures from various 35 external factors, the flexibility and responsiveness of this approach in monitoring will be indispensable in navigating future challenges and ensuring that local food systems remain resilient and aligned with the community's values (Campbell et al., 2022). 36 CHAPTER 3: CONSENSUS OF DISAGREEMENT? AN EXPLORATORY ANALYSIS OF PREDICTED IMPACTS OF LEVERAGE POINTS ON COMMUNITY-IDENTIFIED VALUES USING FUZZY COGNITIVE MAPS Abstract Understanding the inherent complexities of social-ecological systems (SES) is crucial for accurately assessing the potential outcomes of interventions within these systems. This study addresses two central questions: 1) How do various stakeholders assess the impacts of known interventions in SES, highlighting areas of consensus and divergence, and 2) How can participatory modeling enhance our understanding and management of the inherent uncertainty of potential impacts? Employing Fuzzy Cognitive Mapping (FCM), we analyzed 51 individual maps from diverse stakeholders involved in the Flint (MI, USA) food system, and a collective intelligence (CI) model developed by aggregating these individual FCMs. This method not only captures the complex interplay of interventions and system outcomes but also investigates the reliability of model results by assessing parameter uncertainty through variations in edge weights within the CI model. By systematically examining the causal pathways described by different stakeholders, the study seeks to uncover the underlying assumptions driving variations in potential impacts and evaluate the role of intermediary components in influencing these outcomes. The findings emphasize the need to incorporate a broad spectrum of stakeholder views to evaluate the effectiveness of interventions within SES. By identifying areas of consensus and disagreement among potential impacts of interventions, this study illuminates the complex dynamics at play within the Flint food system, underscoring the need for ongoing dialogue and engagement with various community members. Furthermore, the comparison between individual FCMs and the collective intelligence model highlights the added value of collective insights in capturing a broader array of perspectives for exploring the potential impacts, thereby enhancing the robustness and inclusivity of intervention evaluation and planning. 3-1-Introduction Understanding the inherent complexities of social-ecological systems (SES) presents a significant challenge in environmental and social sciences but is a vital step to accurately assessing the outcomes of interventions. SESs are comprised of dynamic interactions that are influenced by environmental variability, human behavior, economic pressures, and policy changes. Knowing these intricate interdependencies and feedback between ecological processes and societal 37 structures aids in efforts to evaluate how strategic changes will impact system functionality (May, 2022). This complexity is further compounded by difficulties in identifying essential system components and estimating their interaction strengths, which further complicate the ability to assess social and ecological outcomes with reasonable accuracy (Pearson & Clark-Wolf, 2023). Moreover, uncertainties stemming from institutional settings, historical contingencies, and individual behaviors significantly hindered the assessment of natural resources management interventions (Nuno et al., 2023). The complexity of SESs often hinders effective scenario planning and system function assessments, as evaluating potential outcomes for proposed interventions requires a comprehensive understanding of both human and natural dynamics of system. Evaluation of potential outcomes within complex systems can be discordant. Bull et al. (2021) illustrate that using different alternative scenarios by various actors results in varying perceived impacts of biodiversity conservation interventions, complicating the consensus on their effectiveness (Bull et al., 2021). Martone et al. (2017) emphasize that qualitative modeling of coastal fisheries reveals unexpected outcomes from management interventions, such as market-based incentives and ecotourism subsidies, highlighting the challenge in anticipating intervention impacts due to complex feedback among biophysical and socioeconomic components (Martone et al., 2017). These examples underscore the inherent difficulties in achieving consensus and accurately evaluating potential outcomes in SES. Participatory modeling has emerged as a useful approach to enhance our understanding of complex SES since such models are informed by both the “best available science” and the lived experiences on the ground (Sterling et al 2018). By involving various stakeholders—ranging from local community members to scientists and policymakers—participatory modeling facilitates knowledge sharing and the integration of diverse perspectives (Kenny et al., 2022). This approach is crucial for capturing the multi-dimensional aspects of SES, allowing for more effective and inclusive decision-making (Bell & Reed, 2021). Among the various participatory modeling techniques, Fuzzy Cognitive Mapping (FCM) has gained considerable traction. FCMs are a type of network used to model the relationships and interactions between various components within a system. In these maps, nodes represent concepts or system components, while edges (directed links) signify the causal influence one node exerts on another. Analyzing these networks helps us understand the dynamics within complex systems 38 (Kosko, 1986). FCM is particularly useful for capturing the mental models of individuals across different disciplines, enabling the visualization of how different components within a system are perceived to interact (Gray et al., 2017). The flexibility of FCM has facilitated transdisciplinary research, crucial for a comprehensive study of SES, by providing a structured yet adaptable method to explore and synthesize complex system dynamics from a multitude of stakeholder viewpoints (Mourhir, 2021). FCMs also offer significant utility in simulation and evaluating the potential impacts of interventions within SES studies (Nápoles & Giabbanelli, 2024). To run "what-if" scenarios in FCMs, stakeholders alter the initial values of one or more nodes, reflecting hypothetical changes in the system components. The model then computes how these changes influence other nodes, leading to increases or decreases in their values based on the causal relationships depicted by the edges (Kok, 2009; Barbrook-Johnson & Penn, 2022). This iterative process allows stakeholders to observe the cascading effects of specific changes across the network, providing insights into potential systemic impacts. By allowing stakeholders to construct and explore various scenario narratives, FCM can assist in examining the impacts of alternative strategies using systems thinking (Morone et al., 2021), providing insights into how different interventions might influence targeted system outcomes (Ameli et al., 2020). Given the subjective nature of FCMs, it is important to acknowledge that different stakeholders may perceive system interactions and potential impacts of interventions differently. Most FCM scenario analysis studies have primarily focused on assessing how each intervention might influence the system’s predefined outcomes, often overlooking the nuanced variability in model outcomes of individual FCMs—representative of each stakeholder's mental model (Bakhtavar et al., 2021). Acknowledging this subjectivity, collecting multiple individual FCMs and running a known scenario for each allows us to explore the varied range of potential impacts, highlighting areas of both consensus and significant disagreement. Building on the identified diversity of potential model outcomes, investigating the causal pathways within each individual FCM sheds light on the underlying assumptions and perceptions that drive variability in potential impact. This detailed analysis of various chains of causation helps to pinpoint the assumptions and perceptions—including those presenting knowledge gaps that some participants may not be aware of—that drive variability in potential impacts (Gray et al., 2013). Moreover, this exploration of causal pathways can reveal a unique narrative recognized by 39 only one or a few stakeholders, which is pivotal for the evaluation of intervention impacts. Furthermore, by considering the intermediary components mentioned by stakeholders, we can identify which of these have significant roles in the efficacy of interventions. This process not only clarifies the underlying factors influencing stakeholder opinions but also enables us to consider the various aspects through which intervention(s) can affect targeted system outcomes, thereby paving the way for reaching consensus by systematically evaluating the role of intermediary components in these processes. Rather than conducting scenario analysis for each individual FCM separately, aggregating multiple individual FCMs into a single collective intelligence (CI) model to evaluate the impacts of interventions offers a more comprehensive approach to FCM scenario analysis (Aminpour et al., 2020, Knox et al. 2023). Different studies indicate that CI models, built from aggregated individual FCMs, provide a more comprehensive understanding of SES functions compared to models based on a single FCM. This integration addresses the multifaceted nature of SES, where individual maps often serve complementary roles, each covering different dimensions of the system's complexity (Aminpour et al., 2020; Gray et al., 2020). Knox et al. (2023) demonstrates various methods for integrating these individual maps, depending on the intended use of the collective model—whether as a communication tool to facilitate understanding among stakeholders or as a detailed model that captures a realistic function of the system under study (Knox et al., 2023). However, the inherent uncertainties arising from varied CI model comphieraronents and the weights of edges necessitate careful consideration when using the CI model for scenario analysis, as they can substantially influence the evaluation of interventions and the reliability of potential outcomes. Expanding on the established application of FCMs in scenario analysis and evaluation of interventions, this study further explores the variations in potential impacts by employing both individual FCMs and a CI model. By comparing these FCMs, we aim to identify main causal chains and understand how discrepancies in stakeholder perceptions contribute to variability in potential impacts. Furthermore, this study will delve into the causal pathways that describe the impact of interventions on targeted system outcomes, investigating the sources of variations in potential impacts across the individual FCMs. Moreover, this exploration includes examining how these causal chains extracted from individual FCMs align or diverge from the CI model. By conducting this comparison, we aim to identify the main causal chains and intermediary 40 components in individual FCMs and see how they align with the causal pathways in the CI model. This detailed analysis will enhance our ability to integrate diverse stakeholder perspectives effectively, which could inform strategic interventions and policy evaluations within SES. Finally, in the last part of this study, the effects of weight distribution on edges will be examined to understand how varied reported weights for edges that exist in the CI model can affect its potential impacts, representing the parameter uncertainty. This study advances the participatory modeling and intervention evaluation field by providing empirical evidence that diverse stakeholders’ assessment of SES emphasize the need for inclusive participatory approaches in evaluating interventions. It introduces a robust methodological process that transitions from qualitative to quantitative analysis, enabling a detailed examination of causal pathways to understand variability in potential impacts and their linkage to interventions. By comparing structural and functional differences in how interventions impact targeted outcomes across individual Fuzzy Cognitive Maps (FCMs) and a collective intelligence (CI) model, the study highlights discrepancies and commonalities in stakeholder perceptions of the system under study. Furthermore, it assesses the reliability of the CI model by examining parameter uncertainty through varying edge weights to evaluate the model's robustness. We applied this methodology using collected FCM data from the Flint food system in Michigan, USA, to analyze the potential impacts of two known leverage points mentioned by the Flint Community on selected system outcomes. 3-2-Method 3-2-1-Data Collection Data collection for this study was conducted as part of the Flint Leverage Point Project (FLPP), a transdisciplinary initiative designed to deepen understanding of the Flint food system and guide its transformation towards a more desirable future2. This project engaged various stakeholders from the Flint community through a series of workshops to identify key system outcomes, which were termed community-identified values (CIV) in this study. These values represent the collective aspirations for a desirable food system in Flint, highlighting seven critical goals that align with the community's vision. In order to capture a comprehensive picture of these system dynamics, interviews were conducted with a range of food system experts who have served in various sectors within Flint 2 https://www.canr.msu.edu/flintfood/ 41 food system, including production, retail, emergency services, and supplementary food systems, as well as those spanning multiple sectors. These interviews facilitated the collection of 51 FCMs, each representing the mental models of participants regarding the Flint food system. Respondents also identified potential interventions, referred to as leverage points, which could significantly enhance the functionality and outcomes of the Flint food system. A Leverage Point (LP) is defined as “a place in the system where a small change could lead to a large shift in behavior” (Meadows, 2008). These individual mental models were then aggregated to construct a CI model, which integrates the diverse perspectives and insights of the community's stakeholders (Knox et al., 2023). For the purposes of this study, the seven CIVs were designated as the targeted system outcomes—Figure 3-1. Two specific LPs—LP 1: Improved Local Gardening and Urban Agriculture and LP 2: Enhanced Access to Transportation Scenario, were selected for detailed analysis. These LPs were explored through both the 51 individual FCMs and the CI model to assess their potential impacts on the seven CIVs. In the context of the Flint Food System, the Improved Local Gardening and Urban Agriculture scenario (LP1) focuses on enhancing local communities by increasing their capacity to grow their food. This initiative uses vacant urban land to establish community gardens and small-scale farms, which not only empowers residents with the skills to produce their own food but also improves urban spaces and improves social cohesion. On the other hand, the Enhanced Access to Transportation scenario (LP2) addresses the challenges of food accessibility in underserved areas of Flint. By improving transportation links between residential areas and food markets, this leverage point aims to reduce the physical and economic barriers to accessing nutritious food. Better transportation systems make it easier for residents to reach supermarkets, community stores, food banks, and the Flint farmers' market. 42 Figure 3-1. Seven Community-Identified Values for the Flint Food System 3-2-2-Determination of Eligible FCMs After determining the LPs and CIVs, the first step was to identify the individual FCMs that include these elements. Initially, the individual FCMs that had the selected LP as a driver or ordinary component (with an outdegree of 1 or more) in their described map were selected. This selection focuses on FCMs where the LP actively influences other components, indicating its impact within the model. Next, these FCMs were further analyzed to find those that contain at least one path from the selected LP to any of the CIVs. This is done by tracing possible routes in the adjacency matrix A of each FCM, ensuring there is at least one complete path from the LP to a CIV. Specifically, for each FCM, we checked for the existence of a directed path from node i (representing the LP) to node 𝑗 (representing the CIV) such that 𝐴𝑖𝑗 ≠ 0.The presence of such a path is essential to explore how each individual FCM describes the impact of the LP on the CIVs, forming the basis for subsequent analyses on the potential impacts of LPs in driving desirable outcomes in the system modeled by each FCM. 3-2-3-FCM Scenario Analysis To investigate how selected LPs will affect the CIVs based on participants' mental models, scenario analysis in FCM was conducted by activating one LP (setting the LP at +1) at a time and recording its potential impact on the CIVs. This approach enabled the systematic evaluation of how different or similar individual FCMs anticipate the impacts on CIVs. Various methods are available for FCM scenario analysis, which differ based on inference rules and squashing functions (check Kok, 2009 and Barbrook-Johnson & Penn, 2022 for a more detailed explanation of FCM scenario analysis techniques). For this study, the hyperbolic tangent was selected as the squashing function, and 'k' was chosen as the inference rule. Different platforms and software, such as Mental 43 Modeler (Gray et al., 2013), as well as programming languages like Python and R, can be utilized to conduct FCM scenario analysis. In this study, Python was employed to execute the scenarios. By recording the potential impacts of selected LPs on CIVs, a list of potential impacts by individual FCMs was generated. In this regard, statistical measures such as the range of potential impacts, upper and lower bounds, mean, median, and standard deviation could help exploring the level of consensus or disagreement among the model outcomes of individual FCMs. These statistical tools provided a comprehensive understanding of the variability and central tendencies in the potential impacts, offering valuable insights into the collective perspectives of the participants. 3-2-4-Path Analysis A path in a FCM is a sequence of nodes connected by directed edges (or casual chain), representing the flow of influence or interaction from one node to another. For example, in an FCM, a path might start from the node "Improved Local Gardening and Urban Agriculture" (representing LP1) and move through "Farmers Markets" to "Affordability" (representing a CIV), indicating how LP1 can impact affordability through provision of more local produces for farmers’ market. To investigate how individuals described the causal chain from the LP to the CIVs, all paths from the LP to the targeted CIV were extracted for each individual FCM. Then, all the unique paths, unique components, and weighted edges were aggregated to form a comprehensive dataset. The extraction of paths provided a basis for further analysis of variations across the individual FCMs, LPs, and CIVs. Path analysis was conducted in three main parts: 1) Qualitative Analysis: This approach provides detailed insights into the described influence of an LP on specific CIVs and the narratives of participants. 2) Jaccard similarity index and Shannon index: These metrics quantify the similarity and diversity of system components between LP and CIV in the paths described by individual FCMs. 3) Network Analysis: This analysis identifies critical components, edges, and paths that influence the impacts of specific LPs on CIVs Each approach provides specific valuable insights that enhance our understanding of the potential impacts and intermediary system components that mediate between the LP and CIVs. The following sections detail these parts. 44 3-2-4-1-Qualitative Path Analysis Qualitative analysis of the extracted paths provides detailed insights into the described influence of an LP on specific CIVs and the narratives of participants. Although it offers rich information, conducting qualitative analysis on all extracted paths can be time-consuming. Instead, focusing on the upper and lower bounds of potential impacts or outliers can help investigate potential perspective differences. There are two approaches for qualitative analysis of the extracted paths: 1) individual- based path analysis and 2) aggregate-based path analysis. Individual-based Path Analysis: This approach considers the described paths from the LP to the CIV in each individual map and compares them across the individual FCMs. This method is useful for exploring the assumptions and causal chains that lead to outlier potential impacts or the upper and lower bounds of the range of potential impacts. It reflects the diverse perspectives and perceptions that individual participants have in their FCMs. Aggregate-based Path Analysis: This method aggregates all the unique paths mentioned by all participants for a specific pair of LP and CIV. It then determines the frequently mentioned intermediary components and edges between the LP and CIV that contribute to the overall understanding of the system. Additionally, the comparison of unique paths, components, and weighted edges across the CIVs and LPs helps to provide insights into the commonalities and differences in how various LPs impact different CIVs. In this aggregated approach, comparing unique paths extracted from all the individual FCMs with those from the CI model can shed light on how the CI model prioritized more frequent paths and even developed unique paths that no individual described between a specific LP and a specific CIV. To better understand the indirect effects of LPs on CIVs, we performed a thematic analysis (Braun, V., & Clarke, V. 2006). Thematic analysis is a method for identifying, analyzing, and reporting patterns (themes) within data. It minimally organizes and describes your data set in (rich) detail. Thematic analysis is widely used in qualitative research and is a foundational method for qualitative analysis. It involves a process of coding and categorizing data to identify significant themes or patterns that emerge from the data. This involved: Identifying 16 themes for the intermediary components, Counting how often each theme appeared for each LP-CIV pair. Comparing these counts to see which themes were most common across different CIVs and LPs, and how they differed between individual FCMs and the CI model. This analysis helped us 45 understand the main themes in the pathways from LPs to CIVs and highlighted differences between individual FCMs and the CI model. 3-2-4-2-Jaccard Similarity Index and Shannon Index To examine the level of similarity and diversity of system components in the paths described by the individual FCMs from a specific LP to CIVs, both the Jaccard similarity index and the Shannon index were utilized. The Jaccard similarity index quantifies the similarity between finite sample sets, offering a metric to gauge the overlap in characteristics or elements between sets. In a broader context, this index can be used to measure the similarity in various types of data sets, making it useful in content analysis. The Shannon index, commonly used to assess diversity in ecological contexts. However, it enables measuring the entropy in data, providing insights into the cognitive diversity of ideas and perspectives. In this study, the Jaccard similarity index was employed for pairwise comparisons of individual FCMs to assess the degree of similarity in their utilization of intermediary components in paths for each pair of LP-CIV. This aimed to identify clusters of similar individual FCMs. The equation below represents the calculation of this index. In this equation, A and B are the sets of intermediary components mentioned by two different individual FCMs. |A∩B| represents the number of common intermediary components, and |A∪B| is the union of unique intermediary components. By evaluating the pairwise Jaccard similarity indices among individual FCMs for each pair of LP-CIV, the distribution of indices was analyzed. Higher average Jaccard indices for each pair of LP-CIV indicate a higher consensus among FCMs regarding the use of similar intermediary components to describe indirect effects, while lower values suggest more disagreement or diversity. This analysis helps identify whether there is a common understanding or agreement among participants about the mediating components and paths and can reveal distinct subgroups of FCMs with similar views for each pair of LP-CIV. 𝐽(𝐴, 𝐵) = |𝐴 ∩ 𝐵| |𝐴 ∪ 𝐵| To identify clusters of participants with similar perceptions of the indirect effects of LPs on CIVs, hierarchical clustering was employed (Ferreira & Hitchcock, 2009). This clustering was performed using Ward's method (Murtagh & Legendre, 2014), which is based on a Euclidean distance matrix calculated from the data on intermediary components mentioned by each participant. Each participant's ID was then assigned to a corresponding cluster group. By grouping 46 participants according to the similarity of their mentioned intermediary components, this method effectively highlighted distinct subsets of participants who share more homogeneous views regarding the mediation paths between LPs and CIVs. This approach enhances our understanding of consensus and diversity within the participant group, informing strategies for addressing varying perceptions in community planning and policymaking. The Shannon index was utilized to quantify the diversity level of intermediary components for each pair of LP-CIV, reflecting the variability or uncertainty in the paths described by the individual FCMs (See Hamilton et al., 2019 for how Shannon index can be used in FCM analysis). This index calculates diversity by analyzing the frequency of intermediary components across the individual FCMs. Higher Shannon index values suggest a greater diversity, indicating a wide range of intermediary components used and less consensus on how the LP affects the CIV. Conversely, lower values denote a more uniform understanding among the FCMs, implying higher consensus and a narrower range of perspectives. Additionally, the Shannon index helps assess the complexity and uncertainty in stakeholder perceptions, providing insights into the cognitive landscape of the FCMs. By considering the Jaccard and Shannon indices, a comprehensive understanding of both the consensus and diversity in the descriptions of intermediary components within the FCMs that contribute to potential impacts on CIVs was achieved. This dual approach facilitates the identification of areas of agreement, potential clusters of similar perspectives, and the overall structural uncertainty in how the system is represented by different participants. 3-2-4-3-Network Analysis Recalling that FCMs function as networks, where nodes represent concepts or variables and edges signify causal influences between these nodes, this network structure allows us to analyze the interactions and relationships within complex systems. Network analysis is essential in understanding the dynamics within complex systems by identifying critical components, edges, and paths that significantly influence the impacts of specific LPs on CIVs (Newman, 2010; Barabási, 2016). Traditional network metrics such as betweenness centrality, closeness centrality, and eigenvector centrality are commonly used to identify important nodes within networks based on their structural positions and connectivity (Freeman, 1977; Brandes, 2001; Bonacich, 1972). These measures effectively pinpoint nodes that 47 serve as vital links between other nodes or are central to the network's functionality (Wasserman & Faust, 1994. However, for analyzing our sub models that involve only directed paths from a source to a sink—specifically, all the extracted paths from one LP to a targeted CIV—more customized approaches were necessary. In this context, we calculated the path score and node participation score for each pair of LP-CIV to determine the importance of paths and intermediary components based on their role in sub models. In this study, the calculation of the path score involved determining the aggregate influence of paths between selected LPs and CIVs within our sub models. As the equation below represents, each path score was calculated by multiplying the weights of the consecutive edges along a predefined path, thus quantifying the total positive or negative impacts exerted by the LP on the CIV through that specific path. Each weight within sub models represents the average of weights determined by respondents in their individual FCMs between two components. In this equation, for a given path P consisting of nodes n1, n2, ..., nk, 𝑤(𝑛𝑖, 𝑛𝑖+1) denotes the weights of the edge from node ni to node ni+1. This score is pivotal for assessing the strength and significance of the paths within the sub models. A higher path score indicates a stronger and potentially more influential causal chain from the LP to the CIV, reflecting a path where the LP is more likely to significantly impact the CIV. Conversely, a lower path score suggests a weaker influence. 𝑘−1 𝑠𝑃 = ∏ 𝑤(𝑛𝑖, 𝑛𝑖+1) 𝑖=1 Additionally, the node participation score (Sn) was calculated to assess the contribution of individual nodes within extracted paths from LP to selected CIV. The node score, as represented in the equation below, is calculated by aggregating the absolute scores of the paths in which each node participates, reflecting the node's overall influence and centrality within the sub model network. In the equation, P(n) represents all paths that include node n |Sp| denotes the absolute score value of path p j represents the total number of paths in P(n) S total is the sum of the scores of all paths from LP to CIV, regardless of the presence or absence of node n in those paths. 48 𝑆𝑛 = ∑ |𝑆𝑝| 𝑗 𝑝∈𝑝(𝑛) 𝑆𝑡𝑜𝑡𝑎𝑙 × 100 This score quantifies how pivotal an intermediary component is across various paths, indicating its strategic importance in potentially amplifying or mediating the effects of LPs on CIVs. Nodes with higher participation scores are considered crucial as they frequently occur in significant paths, thereby playing a key role in the transmission and modification of impacts within the sub model network. Conversely, nodes with lower scores might be less central or influential but could still be significant in specific contexts or scenarios. 3-2-5-Uncertainity Analysis for CI Model In this study, we conducted uncertainty analysis for FCM scenario analysis using Monte Carlo simulations, modifying the weights of edges based on probability distributions derived from paths from LPs to CIVs (Metropolis & Ulam, 1949; Rubinstein & Kroese, 2016). A total of 7000 variations of edge weights were analyzed to evaluate four key aspects: 1) the effect of these altered weights on the directionality of potential impacts, 2) the upper and lower bounds of the 80% confidence interval, 3) the deviation of potential impacts from those of the CI model without weight modifications using Root Mean Squared Error (RMSE) (Zhou et al., 2020), and 4) the influence of edge weight modification on the comparative ranking of CIVs for each LP. The probability distributions for modifying the weights were based on the paths extracted from the CI model describing the paths from LPs to each of the CIVs. Unique edges were determined, and the weights mentioned by different individual participants were identified. A list of mentioned weights for each edge was created, with each weight assigned a uniform probability. For example, if different people mentioned weights for an edge A to B as [-0.3, 0.5, 0.7, 0.9], each weight was uniformly assigned to the edge A to B during the CI model simulations. 3-3-Results 3-3-1-Potential Impacts An initial analysis assessed the potential impact of two LPs on seven CIVs within individual FCMs that included the LPs in their maps. This analysis was concentrated on three key characteristics: 1) FCMs with non-zero potential impact, indicating the percentage of FCMs demonstrating either a positive or negative impact of the LP on the CIV; 2) FCMs with Zero Potential Impact, denoting the percentage of FCMs where the CIV and LP are both present in the map but LP does not affect the CIV; and 3) FCMs without Mentioning the CIV, reflecting the 49 percentage of FCMs where the CIV is absent despite the inclusion of the LP in the map. These characteristics are detailed in Table 3-1, summarized under each leverage point. Table 3-1. 26 Individual FCMs had LP1 and 32 Individual FCMs had LP2—PI in the table is the Potential Impacts LP1: Improved Local Gardening and Urban Agriculture Scenario LP2: Enhanced Access to Transportation Scenario FCMs with Non- Zero PI FCMs with Zero PI FCMs without Mentioning the CIV FCMs with Non- Zero PI FCMs with Zero PI FCMs without Mentioning the CIV Affordability Availability Community Empowerment Education Nutritious Foods Partnerships Quality of Life is Respected 46% 62% 54% 23% 88% 19% 42% 31% 15% 8% 69% 4% 54% 8% 23% 23% 38% 8% 8% 27% 69% 66% 31% 19% 72% 6% 6% 19% 22% 69% 19% 59% 50% 47% 6% 25% 16% 47% 13% 9% 34% 47% Regarding the above-mentioned characteristics, there was considerable variability in how different CIVs are affected by the same LP. Among the individual FCMs that included the first LP (Improved Local Gardening and Urban Agriculture), nutritious foods (88%) and availability (62%) were the most frequent CIVs with non-zero potential impacts. Conversely, 69% and 54% of FCMs anticipated zero impacts on education and partnership, respectively. Furthermore, 50% of the FCMs did not include quality of life despite the presence of the first LP in their maps. Considering the second LP (Enhanced Access to Transportation), nutritious foods (72%) was the CIV with the most frequent non-zero potential impact. Notably, affordability, which had a lower percentage of non-zero potential impacts for the first LP, showed a higher percentage of non-zero impacts (69%) with the second LP. Similar to the first LP, most individual FCMs anticipated zero impact on partnership (59%) and education (69%) regarding the second LP. Furthermore, in comparison with the first LP, a higher percentage of FCMs anticipated zero impacts on community empowerment, indicating a reduced influence of the second LP on this CIV. 50 The CIVs exhibiting high percentages of non-zero impacts could be crucial for more in- depth analysis to understand the magnitude of these impacts and the extent of consensus among individual FCMs regarding their assessment across the CIVs and LPs. To achieve this, the standard deviation (SD) and range of potential impacts were considered for further investigation of consensus level. Table 3-2 summarizes the descriptive statistics, taking into account both zero and non-zero potential impacts by individual FCMs. Table 3-2. Descriptive statistics for the distribution of potential impacts by individuals for different LPs and CIVs LP1: Improved Local Gardening and Urban Agriculture Scenario LP2: Enhanced Access to Transportation Scenario SD of PI Maximu m PI by Individua l FCMs Minimu m PI by Individua l FCMs Range of PI SD of PI Maximu m PI by Individua l FCMs Minimu m PI by Individua l FCMs Range of PI 0.242 0.249 0.784 0.731 -0.386 0.000 1.170 0.731 0.189 0.604 0.000 0.604 0.181 0.764 0.000 0.764 0.275 0.982 -0.035 1.017 0.173 0.554 0.000 0.554 0.46 0.46 0.44 0.22 0.52 0.08 1.22 0.91 0.87 0.61 0.79 0.40 -0.62 -0.64 -1.29 -0.81 -1.68 -0.06 1.84 1.55 2.16 1.42 2.47 0.46 0.295 0.848 -0.090 0.937 0.54 0.85 -1.27 2.12 Affordability Availability Community Empowerment Education Nutritious Foods Partnerships Quality of Life is Respected For the LP1, the SD across CIVs such as community empowerment, education, and partnership were relatively low, indicating a moderate consensus among individual FCMs. In contrast, CIVs such as quality of life is respected, nutritious foods, availability, and affordability exhibited relatively high SDs, indicating a lower level of consensus. Notably, affordability demonstrated a significant variation, ranging from -0.39 to 0.78, highlighting substantial discrepancies in individuals’ potential impacts (both positive and negative values) regarding the LP1’s influence on affordability. For the LP2, the SDs and Ranges of potential impacts on CIVs were considerably higher in comparison with LP1. Unless the partnership that only 6% of individual FCMs anticipated non-zero impacts regarding the LP2 (Table 3-1), the other CIVs 51 represented a wide range of positive and negative impacts by LP2. Quality of life is respected and nutritious foods had the highest SDs and Ranges based on the potential impacts of LP2 by individual FCMs. The box-and-whisker plots depicted in Figures 3-2 and 3-3 represent the range and distribution of potential impacts of LPs on seven CIVs. These figures illustrate not only the variability but also the central tendencies, represented by the mean and median, derived from individual FCMs. The central tendencies enable a clear comparison of which CIVs are most influenced by the respective LP. Additionally, the plots include the potential impacts calculated by the CI model (distinctly marked by a red star), which allows for an evaluation of how these potential impacts align with the broader spectrum of impacts forecasted by the individual FCMs. This comparative analysis highlights the CI model’s positioning within or outside the range of individual assessments. Considering the mean and median of potential impacts of LP1 by individual FCMs, both measures displayed a roughly similar ranking from highest to lowest influenced CIVs. However, the median indicated zero impacts for affordability, education, and partnership, while the mean showed non-zero but near-zero impacts for these CIVs. In contrast, the potential impacts by the CI model were positioned in the optimistic part of the range of potential impacts. Specifically, for affordability, availability, education, and partnership, the impacts anticipated by the CI model could be considered outliers when compared to the box-and-whisker plots of potential impacts by individual FCMs. Moreover, although the relative ranking of the highest to the lowest influenced CIVs by LP1 is consistent for five CIVs, for the other two CIVs the CI model had different assessment; it placed quality of life is respected in the highest rank, and community empowerment in the fourth, whereas, based on the mean and median of potential impacts by individual FCMs, quality of life is respected held the fourth rank and community empowerment the second. For LP2, while the median and mean of potential impacts by individual FCMs displayed a consistent ranking for five CIVs, disparities emerged for nutritious foods and affordability, where their rankings flipped between second and third place for highest influenced CIVs by LP2. CI model's anticipated impacts generally fell within the optimistic range of potential impacts by individual FCMs, yet notably different for quality of life is respected, which exceeded the potential range by individual FCMs. Furthermore, LP2's ranking of CIVs diverged significantly from the central tendency of individual potential impacts, particularly elevating the quality of life is 52 respected and community empowerment to the top ranks, contrary to their lower ranks based on mean and median values from individual FCMs. As seen in Figure 3-2, the expected impacts from individual FCMs showed significant differences across some CIVs, especially in the LP2 scenario. These differences stem from the various perspectives included in the described FCMs, reflecting differing interpretations and assumptions by participants. Additionally, the broader impact ranges noted in LP2—enhanced transportation scenario—highlight the complex and potentially contentious nature of assessing the impacts of infrastructural changes compared to more localized interventions like enhanced local gardening and urban agriculture (LP1). Comparing the mean and median values of potential impacts from individual FCMs to those from the CI model reveals that the CI model's outcomes often fall within the optimistic range or beyond the range of individual assessments for LP1 and LP2. This underscores the need for a thorough investigation into the reasons for these disparities and the validation of potential impacts before incorporating them into planning processes. Figure 3-2. Distribution of potential impacts of LP 1 on different CIVs regarding individual FCMs 53 Figure 3-3. Distribution of potential impacts of LP 2 on different CIVs regarding individual FCMs 3-3-2-Path Analysis To explore the reasons behind the discrepancy in the potential impacts of LPs on CIVs, all paths from LPs to CIVs were extracted from individual FCMs for each LP. By aggregating these extracted paths, a list of unique paths, components, and weighted edges between the LPs and CIVs was recorded. Quality of life is respected, with 5.2 paths per participant, and nutritious foods, with 3.6 paths per participant for both LPs, had the highest number of paths per participant, while partnership, with 1.4 paths per participant, and availability, with 2 paths per participant, had the lowest number of paths per participant. See Table 3-3 for summary of unique paths, components, and edges mentioned in paths aggregated from individual FCMs and paths extracted from CI model. 54 Table 3-3. Frequency of Unique Paths, Components, and Edges for the extracted paths from individual FCMs and CI model Affordability Availability Community Empowerment Education Nutritious Foods Partnerships Quality of Life is Respected LP1: Improved Local Gardening and Urban Agriculture Scenario (26 Individual FCMs) # of Participants Paths All FCMs Paths CI Component All FCMS Component CI Edges All FCMs Edges CI # of Participants Paths All FCMs Paths CI Component All FCMS Component CI 12 45 23 28 15 72 31 22 71 6 36 5 Edges All FCMs Edge CI 104 11 16 24 16 22 10 46 21 14 21 4 18 7 36 10 6 17 3 17 7 33 9 23 63 22 43 14 96 28 5 6 2 7 4 11 5 Enhanced Access to Transportation Scenario (32 Individual FCMs) 21 56 3 31 2 82 5 10 44 0 30 0 71 0 6 16 0 14 0 28 0 23 88 3 45 4 124 7 2 3 0 2 0 6 0 11 80 22 38 15 110 31 15 69 8 26 6 79 14 3-3-2-1-Qualitative Exploration of Paths Comparing the paths across individual FCMs could help to understand the causal chains that led to outliers or extremes in the potential impacts. For instance, considering the effects of PL1 on Education, PID 115 anticipated a considerable positive impact (+0.76) of LP1 on education, while the majority (69%) anticipated zero impact (Table 3-1). This outlier was due to PID 115 being the only one to describe a direct positive effect of LP1 on education, a connection not mentioned by others. In another case, PID 124 and PID 114 had considerably different 55 potential impacts of LP2 on affordability that represented the upper bound and lower bound of potential impacts by individual FCMs. PID 124 anticipated a considerable positive impact (+1.11) and PID 114 anticipated a considerable negative impact (-0.62) for affordability. PID 124 outlined various indirect paths from LP2 to affordability, involving intermediary components like “convenience stores” and “grocery stores”, suggesting enhanced transportation would lead people to shop at more affordable stores outside the city, like grocery stores, and shop less from convenience stores located nearby that offer higher prices. Conversely, PID 114 assumed enhanced transportation would increase purchases from farmers' markets, leading to higher spending on nutritious but less affordable foods. These illustrations highlight how qualitative analysis across individuals can explore their cognitive causal chains, providing a deeper understanding of diverse perspectives and assumptions in the potential impacts among individual participants. Qualitative analysis and comparison of unique paths extracted from all the individual FCMs with paths extracted from the CI model could provide insights on why some of the potential impacts of the CI model have a considerable difference from the mean and median of the individual impacts. This is evident in cases where the CI model's assessments are considered outliers in comparison with the individual potential impacts, such as the potential impacts of LP1 on education, partnership, and availability, or the potential impacts of LP2 on quality of life is respected. For instance, the CI model anticipated a considerable positive impact (+0.30) of LP1 on education, while the majority anticipated zero impact. The CI model identified a positive connection from LP1 to partnership and from partnership to education, a link that none of the individual FCMs, which included LP1 and education, mentioned. This highlights the value added by the CI model. However, it is worth to note that the CI model may summarize perspectives and might not consider a system component that was mentioned only by one participant, which can provide important insight. Therefore, individual FCMs should also be considered for scenario analysis as they can bring a unique story about the potential impacts of LPs on CIVs. In another example, the CI model anticipated a considerable positive impact (+0.90) of LP2 on quality of life is respected, while the median and average of individual potential impacts were +0.06 and +0.16, respectively. In the CI model, there were two unique paths that none of the individual FCMs, which included this pair of LP and CIV, had mentioned. In these two paths, the CI model described how enhanced transportation can decrease the use of chain restaurants, leading to an increased quality of life directly, and how it can enhance access to nutritious food, indirectly promoting the quality 56 of life. These examples underscore the importance of integrating diverse perspectives to develop a comprehensive understanding of the system. A) Share of different themes in intermediatory components for aggregated paths from individual B) Share of different themes in intermediatory components for extracted paths from CI Model for Figure 3-4. Theme distribution for intermediary components for individual FCMs and CI model 57 In the qualitative analysis of intermediary components that describe the indirect effects of LPs on CIVs, 16 different themes were assigned to the intermediary components based on paths aggregated from individual FCMs and paths extracted from the CI model. Subsequently, the prevalence of each theme for impacted CIVs was calculated. Figure 3-4-A illustrates the theme distribution for intermediary components mentioned by individual FCMs for LP1 and Figure 3-4- B shows the theme distribution for the CI model for LP1. For both LPs, it was evident how the CI model emphasizes a few selected themes. Additionally, in the CI model, some themes that were not dominant in the paths from individual FCMs became more dominant. For instance, for LP1, 'Retail Sector', 'Economics', and 'Built Environment/Infrastructure' were the dominant themes observed in paths aggregated from individual FCMs. However, in the CI model, 'Retail Sector', 'Emergency Sector', and 'Supplemental Sector' emerged as dominant themes, indicating a shift in thematic focus. As demonstrated in Figures 3-4-A and 3-4-B, the number of themes used to explore the impacts of LP1 on CIVs decreased from 14 themes in individual FCMs to 9 themes in the CI model. The overall reduced diversity of themes from 14 in the individual FCMs to 9 in the CI model, suggesting a more concentrated thematic representation that may contribute to differences in potential impacts between the CI model and the mean and median of potential impacts by individual FCMs. 3-3-2-2-Diversity of Intermediary Components Analysis of the causal pathways and intermediary components linking LPs to CIVs in individual FCMs revealed a heterogeneity in their mentioned components. This heterogeneity is evident from the Shannon diversity indices, which vary across different CIVs. For instance, values such as "Nutritious Foods" exhibit higher diversity indices, indicating a broader range of components influencing the indirect impacts, while some CIVs, like "Partnership," display lower diversity indices. Comparing the diversity indices between the LPs, "Affordability" ranks fourth in heterogeneity of intermediary components for LP1 but ranks second for LP2. This variation in rankings across LPs underscores the different levels of heterogeneity in component involvement in indirect effects across CIVs. Table 3-4 summarizes these indices across the CIVs and LPs. Overall, higher diversity indices not only reflect greater heterogeneity in the description of causal chains but also suggest increased structural uncertainty regarding the roles and impacts of intermediary components. 58 When comparing the diversity indices for the paths extracted from the CI model, a considerable decrease was recorded, particularly for LP2. For example, the diversity index of components in causal chains for the pair of LP2 and "Availability" decreased dramatically from 2.75 to 0.69, highlighting a significant narrowing of perspectives in the CI model. This reduction could be attributed to the CI model prioritizing components that are mentioned more frequently and hold greater significance in the network of component interactions. However, this considerable decline in diversity indices might result from oversimplifying complex interactions described by aggregated paths from all individual FCMs. It underscores the importance of conducting a more detailed investigation to avoid losing critical insights provided by less frequent but potentially influential components, especially for CIVs that exhibit a wider range and standard deviation of potential impacts, to capture the full spectrum of influences on CIVs. 3-3-2-3-Jaccard Similarity Index By calculating the pairwise Jaccard similarity index for shared intermediary components describing the indirect effects of LPs on CIVs, a diagonal matrix was created for each LP-CIV pair. Figure 3-5 demonstrates an example of the pairwise Jaccard similarity index for LP1-Quality of Life. A value of 0.45 for the participant pair of 115 and 123 indicates that 45% of their mentioned intermediary components for describing the effects of LP1 on Quality of Life were similar. For LP1, Quality of Life and Affordability had the highest average similarity index, while individual FCMs for Partnership and Community Empowerment had near-zero average similarity indices, indicating a low level of similarity in the usage of intermediary components. For LP2, Nutritious Food and Affordability had the highest average Jaccard similarity index, while Partnership and Education had the lowest similarity index. Comparing the average Jaccard similarity index between the LPs, the averages were higher for LP2 compared to LP1, indicating that participants used more similar intermediary components to describe the impacts of enhanced transportation rather than enhanced local gardening and urban agriculture on CIVs. 59 Table 3-4. Diversity Index of intermediary components across the CIVs and LPs LP1: Improved Local Gardening and Urban Agriculture LP2: Enhanced Access to Transportation CIVs Diversity Index for Aggregated Paths of Individual FCMs Diversity Index for Paths of CI Model Diversity Index for Aggregated Paths of Individual FCMs Diversity Index for Paths of CI Model Affordability Availability Community Empowerment Education Nutritious Foods Partnerships Quality of Life is Respected 2.83 2.94 2.65 2.36 3.21 1.79 3.22 2.46 2.22 1.89 1.89 2.43 1.39 2.40 3.09 2.75 3.04 2.34 3.22 1.04 2.87 1.56 0.69 0.00 0.00 1.33 0.00 1.75 Using the hierarchical clustering method, different clusters of participants with similar Jaccard indices were identified for each pair of LP-CIV. This clustering revealed distinct groups of participants who described the indirect effects of LPs on CIVs in a comparable manner. Figure 3-5-A shows the hierarchical clustering dendrogram for LP1 and Quality of Life, while Figure 3- 5-B displays the four similar groups of participants (each group with a unique color) who shared more similar intermediary components when describing the impacts of LP1 on Quality of Life. These clusters indicate that certain participants share a common way of interpreting the effects of LPs on CIVs, which is valuable for scenario analysis and investigating potential impacts. By examining these clusters, researchers can identify common patterns and variations in participants' mental models that shape their understanding of the impacts of different LPs on CIVs. 60 Figure 3-5. Pairwise Jaccard similarity index for the intermediary components in the paths of LP1 and Quality of Life and their hierarchical clustering dendrogram 3-3-2-4-Recognition of Important Paths and Intermediatory Components As detailed in Section 2.2.3, sub-models for each pair of LP and CIV were constructed by aggregating the paths from LP to CIV. This process resulted in the development of 14 sub-models derived from the individual FCMs and 11 sub-models based on the CI model. The reason for having fewer sub-models in the CI model was that it did not include any paths from LP2 to the CIVs of education, community empowerment, and partnership. These sub-models facilitated the examination of which pathways from LP to CIVs contributes the greatest impact (positively or negatively) and which intermediary components played pivotal roles in connecting LPs and CIVs, utilizing network analysis measures to enhance our understanding of these dynamics. 61 Evaluating sub-models from both FCMs and the CI model, and assessing path scores, revealed paths from LPs to CIVs that represent significant positive and negative impacts. In most cases, paths from individual FCMs included both positive and negative impacts, but the absence of negative impact paths in some LP-CIV pairs within the CI model reflects its focus on prioritizing only the most important system components and edges. Additionally, while shorter causal chains typically exhibited higher path scores, analysis indicates that there are instances where longer causal chains, particularly when considering the weights of their edges, could represent a more substantial influence on the CIVs than shorter causal chains. Moreover, paths that were descriptive of negative impacts generally had a longer average length and they tended to score lower in terms of absolute path scores. Table 3-5 showcases an example of the high-scoring positive and negative paths for the pair of LP2 and nutritious foods, incorporating paths from individual FCMs. The identification of high-scoring positive and negative paths for each pair of LP-CIV enables a deeper exploration of causal chains, providing insights into how LPs may influence CIVs through both positive and negative dynamics. Table 3-5. Examples of high-scoring paths for the pair of LP2 and nutritious foods Paths ['Access to Transportation'-->'Nutritious Foods'] ['Access to Transportation'-->'Expectation of Free/Convenient Food'-->'Convenience Stores'-->'Nutritious Foods'] ['Access to Transportation'-->'Grocery Stores'--> 'Nutritious Foods'] ['Access to Transportation'-->'Farmers Markets'-->'Prepared Foods in Market'-->'Nutritious Foods'] ['Access to Transportation'-->'Grocery Stores'-- >'National/Global Agriculture'-->'Gardening + Local Agriculture'-->'Nutritious Foods'] ['Access to Transportation'-->'Convenience'-->'Convenience Stores'-->'Nutritious Foods'] Path Score 0.70 0.33 0.33 -0.18 -0.20 -0.24 Length of Path 1 Mentioned by Participant IDs [114, 153] 3 2 3 4 3 [107] [145] [146] [149] [149] To assess the importance of intermediary components within the sub-models, node participation scores were computed for all intermediary components mentioned at least once in a path. This analysis prioritized the top five high-scoring intermediary components for further 62 investigation within each LP-CIV pair. Notably, certain high-scoring components consistently ranked among the top five across multiple LP-CIV pairs. For example, in the aggregated paths from individual FCMs that described the effects of LP1 on CIVs, "Farmers Market" and "Flint Fresh + Food Hub" were identified as crucial intermediary components, appearing in the top five for five CIVs. Conversely, in the CI model for the same LP1, "Partnership" and "Education"— both CIVs themselves—were prominent, appearing in the top rankings for six and four of the seven CIVs, respectively. For LP2, "Farmers Markets" and "Grocery Stores" were consistently high- ranking intermediaries in paths from individual FCMs, while "Grocery Stores" and "Convenience Stores" were predominant in the CI model. Recognition of these key intermediary components highlights their critical role not only in enhancing the success of targeted interventions but also in accurately assessing the impacts of such interventions on CIVs. 3-3-3-Uncertainty Analysis In this study, the uncertainty analysis of modified weights for the edges of the CI model using Monte Carlo simulations revealed a few insights regarding the potential impacts of LPs on CIVs (7 CIVs for LP1 and 4 CIVs for LP2—since the impacts of LP2 on education, partnership, and community empowerment were zero, they were excluded from uncertainty analysis for LP2). For LP1, the directionality of potential impacts for all the CIVs did not change except for Affordability, in which only 2% of the total runs showed a negative potential impact, while 98% demonstrated positive impacts. For LP2, only Quality of Life did not have any directionality change, with 100% of the scenarios indicating a positive impact of LP2 on Quality of Life. On the other hand, for Affordability and Nutritious Foods, a considerable directionality change was recorded for the impacts of LP2, as demonstrated in Table 3-6. While modification of weights did not affect the directionality of potential impacts for most LP-CIV pairs, the considerable variability under different weight modifications in the directionality of potential impacts of LP2 on Affordability and Nutritious Foods highlights the need for more detailed investigation of weights described in the paths for these specific CIVs. Given the scope of this study, the utilization of 51 FCMs for the CI model development and low number of individual FCMs for weight distribution, must be considered carefully; such a relatively small sample size may limit the robustness and generalizability of our findings, underscoring the need for cautious interpretation of the results. The uncertainty analysis provided the upper and lower bounds of the 80% confidence intervals for the potential impacts. Additionally, the difference between potential impacts when 63 modifying the weights and the actual potential impacts by the CI model was calculated through RMSE, as described in Table 3-6. Comparing the LPs, the higher range of variations (the range between lower and upper bounds of the 80% CI) and higher RMSE were observed for LP2, reflecting its higher sensitivity to the defined weights of edges in the CI model. Among the CIVs, for both LPs, Nutritious Foods had the greatest difference between the upper and lower bounds— specifically for LP2—and a higher RMSE, indicating the need for more accurate weight assignments to describe the effects of LP on this CIV. Conversely, Quality of Life had the lowest RMSE for LP2, while Partnership and Community Empowerment had the lowest RMSE for LP1. This indicates that while some CIVs are more robust to changes in edge weights, others like Nutritious Foods require precise assignment of weights to ensure accurate impact potential. Table 3-6. Uncertainty Analysis parameters across LPs and CIVs LPs CIVs Positive Impact (%) Negative Impact (%) 80% CI Lower Bound 80% CI Upper Bound RMSE Affordability Availability Community Empowerment Education Nutritious Foods Partnerships Quality of Life is Respected Affordability Availability Nutritious Foods Quality of Life is Respected 98 100 100 100 100 100 100 76 97 79 100 LP1 LP2 2 0 0 0 0 0 0 24 3 21 0 0.25 0.68 0.43 0.22 0.63 0.43 0.75 -0.19 0.38 -0.25 0.64 0.47 0.91 0.65 0.47 0.93 0.60 0.87 0.39 0.87 0.75 0.91 0.13 0.15 0.10 0.10 0.16 0.07 0.17 0.26 0.26 0.55 0.15 The modification of edge weights in the CI model resulted in non-considerable changes in the comparative ranking of CIVs, relative to the potential impacts of LPs. As illustrated in Figure 3-6, for LP1, the top three ranks were consistently held by Availability, Nutritious Food, and Quality of Life is Respected, indicating their strong and stable positive impact from LP1 across various weight modifications. Ranks 4 and 5 were more variable, occupied by Partnership and Community Empowerment, suggesting these CIVs experienced a moderate positive impact 64 relative to others. The less impacted CIVs by LP1, ranking 6th and 7th, were Affordability and Education. For LP2, Quality of Life is Respected maintained the highest rank, demonstrating its robust positive impact, unaffected by changes in edge weights. Affordability and Nutritious Food, which occupied the 3rd and 4th ranks, exhibited lower or potentially negative impacts from LP2. As discussed, the impacts of LP2 on other CIVs of Community Empowerment, Education, and Partnership were zero and they were excluded from the ranking analysis. Despite the modifications, the ranking exhibited limited variability, indicating that edge weight adjustments in the CI model do not significantly alter the comparative ranking of CIVs compared to their ranking without such modifications, as depicted in Figure 3-6. 3-4-Discussion 3-4-1-Identifying Areas of Consensus and Disagreement Running the FCM scenario analysis for each individual FCM, a varied range of predicted impacts of LPs was recorded for some of the CIVs. By doing this, areas of consensus and disagreement were identified across the CIVs and LPs, highlighting the diversity of perspectives among stakeholders. This indicates that different individuals or groups perceive the food system and the effects of LPs differently, which is crucial for understanding the complexity of the food system. This diversity can inform decision-makers about the various concerns, priorities, and knowledge gaps among stakeholders. CIVs like nutritious foods and quality of life is respected showed a wider range of predicted impacts in comparison with other CIVs, suggesting that investigating the impacts of LPs on these two CIVs may need further analysis and stakeholder engagement to build higher level of consensus. This can help prioritize these values for further discussion and research or may require more stakeholder engagement to align different perspectives. However, consensus is not always achievable as differences in stakeholder priorities and varying interpretations of values can create persistent divergences, highlighting the need to balance diverse perspectives rather than seeking complete agreement. 65 Figure 3-6. Proportion of changes in ranking of CIVs for the LP1 and LP2 regrading the random weights of edges in CI model 66 Even among the LPs, more disagreement and directionality change were observed for LP2 in comparison with LP1, demonstrating the need for more precise intervention evaluations for LP2. On the other hand, the high consensus for LP1 suggests a more common understanding or agreement on its impacts on CIVs, indicating that interventions related to LP1 might be implemented more smoothly with higher acceptance from the community. Despite the varied predicted impacts for LP1, the majority of stakeholders predicted positive impacts for CIVs related to LP1, further supporting its potential for successful implementation. Larsen et al. (2019) also emphasize the importance of achieving stakeholder consensus about policies' outcomes as it can significantly influence policy effectiveness. 3-4-2-Individual FCMs V.S. CI model The comparison between the predicted impacts of individual FCMs and the CI model revealed insights into the diversity and convergence of stakeholder perspectives. Although most of the CI predictions fell within the range of predicted impacts by individuals, some, such as the predicted impact on availability for LP1, were outside this range. This discrepancy can be attributed to the broader and more comprehensive scope of the CI model, which incorporated FCMs from all 51 participants, compared to the subset of 12 participants who directly addressed the causal path from LP1 to availability. Participants outside of this subset introduced new system components or edges that can mediate the effects of LP1 on availability that none of the 12 participants mentioned in their FCMs. This indicates the added value of the CI model in capturing a wider array of perspectives and considerations within the food system. Most of the time, the CI model predicted impacts were on the positive side for both LP1 and LP2. The optimistic skew of the CI model might stem from a collective bias or overestimation of positive outcomes when diverse perspectives are combined. While the CI model informs a more balanced and comprehensive approach, it is also beneficial to consider the upper and lower bounds of individual predictions to understand the full spectrum of possible impacts. This exploration not only reveals the variability among stakeholder perspectives but also emphasizes the necessity of tracing and understanding the causal paths to identify the sources of discrepancies and convergences. Investigating these causal paths across the individual FCMs and between individual FCMs and the CI model is crucial for better understanding the system's structure and enhancing the applicability of FCMs in intervention evaluation and informed decision-making processes. 67 3-4-3-Comparison of methods for paths analysis Regarding the qualitative approach, the path analysis conducted within this study employed a multifaceted approach, ranging from qualitative explorations to quantitative assessments, to understand the dynamics of how LPs influence CIVs. The qualitative exploration involved examining the upper and lower bounds of predicted impacts, particularly focusing on cases with both positive and negative predicted outcomes for each CIV. By tracing and narratively analyzing the LP-CIV paths, this method allowed for an understanding of the potential dual effects an LP can have on a CIV. Additionally, thematic analysis of intermediary components provided insights into which themes were more frequently involved in influencing CIVs, enhancing our understanding of the dominant narratives and mechanisms at play. This qualitative depth was crucial for contextualizing the quantitative findings, offering a narrative foundation that explains the numerical data. On the other side, through quantitative analysis, diversity analysis of intermediary components and the calculation of Jaccard similarity indices facilitated a broader understanding of consensus and variation within the stakeholder models. By examining the diversity of intermediary components, this study assessed the range of elements considered by different stakeholders in describing the indirect impacts of LPs. The Jaccard similarity indices further quantified the extent to which individual FCMs shared common intermediary components, allowing for the grouping of models based on their similarity. This analysis helped identify clusters of similar perspectives, highlighting patterns of agreement or disagreement among participants. Moreover, the calculation of path scores and component scores provided a quantitative measure to recognize and prioritize the most influential paths and intermediary components. These scores are critical for determining which components and paths are most significant in describing the indirect effects of LPs on CIVs, thereby guiding targeted interventions and further research. Each method complements the others, with qualitative insights enriching the interpretation of quantitative data, and quantitative metrics offering empirical support to qualitative assessments, together providing a comprehensive analysis of the paths influencing CIVs. 3-5-Conclusion In conclusion, the extensive analysis performed through individual and collective intelligence FCMs, can enhance policy and decision-making processes by incorporating the diverse perspectives and potential impacts associated with LPs on CIVs in the food system. This 68 multifaceted approach identifies areas of both consensus and disagreement, offering decision- makers a nuanced understanding of stakeholder perceptions and priorities, which is essential for developing inclusive and effective policies. As demonstrated in this study, implementing FCM analysis is particularly recommended in situations where the decision-making landscape is complex and involves multiple stakeholders with varying interests and perspectives. In such environments, FCMs provide a structured method to capture and analyze the subjective views of different groups, facilitating a more democratic approach to policy formulation. This is vital in ensuring that interventions are not only based on a broad consensus but also respect the diverse values and expectations within the community. Moreover, the findings from this study underscores the critical role of FCMs in enhancing the adaptability and responsiveness of policymaking in complex systems. By detailing how different LPs influence CIVs, FCMs help policymakers anticipate potential outcomes and adapt strategies in real-time, thereby increasing the likelihood of achieving desired impacts. This adaptive capacity is crucial in complex systems where initial conditions and external influences can unpredictably alter the effectiveness of policies. Overall, the application of FCM analysis (both individual FCMs and collective intelligence model of FCM) in this context not only supports more informed decision-making but also promotes a more transparent and participatory approach, thereby fostering greater trust and cooperation among stakeholders. 69 CHAPTER 4: COMMUNITY-INFORMED DECISIONS FOR EQUITABLE, COST- EFFECTIVE, AND INCLUSIVE DISASTER RESILIENCE PLANNING (CO-DECIDR): MODELING APPROACH Abstract Evaluating coping strategies for community resilience planning in the face of natural hazards and disasters presents significant challenges, including limited system clarity, diverging community priorities, resource inequalities, and inherent uncertainties. Traditional methods— often reliant on economic metrics through benefit-cost analysis—primarily assess monetary aspects, overlooking critical non-monetary factors. The increasing shift towards transdisciplinary methods underscores the importance of integrating local community values into resilience planning. This research introduces the Co-DECIDR modeling framework, designed to evaluate resilience planning alternatives by embracing community-identified values, leveraging local insights, navigating uncertainties, and balancing monetary with non-monetary considerations. By merging Fuzzy Cognitive Mapping (FCM) for qualitative depth with Benefit-Cost Analysis (BCA) for quantitative rigor, Co-DECIDR enables a comprehensive understanding of complex social- ecological systems. The framework utilizes accessible online tools—Mental Modeler for FCM and Economic Decision Guide Software (EDGe$) for BCA—to enhance user engagement and model efficacy. Demonstrated through proof-of-concept examples in Flint, Michigan, focusing on resilience to extreme weather events and pandemics like COVID-19, Co-DECIDR showcases its practicality and adaptability in real-world contexts. These examples highlight the framework's capacity to support informed, community-centric, and equitable resilience planning. Furthermore, Co-DECIDR advances good modeling practice by utilizing participatory modeling in resilience planning to bridge the gap between modelers and end-users. It systematically captures stakeholder requirements, investigates subjectivity, addresses collaborative modeling challenges, encourages the engagement of diverse groups, and adopts a comprehensive approach to assessing model performance, embracing both qualitative and quantitative criteria. 4-1-Introduction The importance of resilience planning in the context of socio-environmental systems (SES) has gained considerable attention recently, as this type of systems thinking can enhance the ability of communities to adapt to, build resilience towards, and recover from a wide range of acute challenges as well as persistent environmental and social stressors (Jaramillo et al., 2021; Lambrou 70 & Loukaitou-Sideris, 2022). Resilience planning can be defined as a process in which possible risks and hazards are pinpointed, and subsequently, strategies for adaptation, mitigation, and recovery are developed (McAllister et al, 2015). These efforts aim to create more resilient SESs that can better withstand shocks and ensure the continued well-being of community members, socio-economic functions, and ecosystem stability. Yet, resilience planning in the context of complex adaptive systems (Levin et al., 2013), such as SES, entails multiple challenges. Primarily, a notable lack of clear definitions for resilience indicators, trade-offs among different resilience goals and community-identified values, and the uncertainty and complexity of socio- environmental dynamics (Sellberg et al., 2018; Helgeson & O’Fallon, 2021; Jaramillo et al., 2021; Chollett et al., 2022; Gu et al., 2023). As a result, resilience planning for SESs requires a comprehensive and interdisciplinary approach that considers the intertwined interactions of multiple human and natural subsystems within and across various temporal and spatial scales, while addressing the substantial uncertainties inherent in these systems (Sellberg et al., 2018). Moreover, it is essential for resilience planning to foster effective collaboration among diverse stakeholder groups and community members to ensure the selection of equitable and effective interventions (Bostick et al., 2017; Neely et al., 2021). The approach presented in this paper emphasizes integrating Good Modeling Practices (GMP) in modeling to support resilience planning. It extends beyond merely recognizing challenges to actively formulating and implementing a framework that is both robust and inclusive. This shift is crucial in addressing the complex dynamics of SES and ensuring that resilience strategies are effective, equitable, and aligned with the diverse values and needs of communities. (Koliou et al., 2020; Van de Lindt et al., 2023). Several key principles have been identified as 'good practices' for modeling SES, including the use of interdisciplinary approaches and stakeholder engagement, rigorous identification, quantification, and communication of uncertainty in model development, transparency in model design and execution, and the careful comparison and justification of methodological tools selected for model development (Jakeman et al., 2006; Guillaume et al., 2017; Jakeman et al., 2018). Addressing the previously mentioned challenges in resilience planning requires a modeling approach that combines both qualitative and quantitative methods. This approach should not only consider the inherent uncertainty in model input data and output information—facilitating improved decision-making under conditions of uncertainty—but also account for the intricate interrelationships and feedback loops among the system's 71 components. Additionally, this approach must harness transdisciplinary methods to actively engage a diverse range of stakeholders with varying perspectives to ensure the model's comprehensiveness and relevance (Koliou et al., 2020). In this study, we introduce the Community-informed Decisions for Equitable, Cost- effective, and Inclusive Disaster Resilience (Co-DECIDR) modeling approach, tailored for community-based resilience planning within SES. Co-DECIDR is grounded in GMP, designed to enhance decision-making by incorporating trade-off analysis and considering the complex interplay between physical infrastructure, social institutions, and natural ecosystems. In the domain of resilience planning, choosing the suitable modeling technique is crucial. Economic modeling techniques (such as input-output models) offer simplicity and ease of use for policy assessment, but often at the expense of nuance and comprehensiveness. These methods typically rely on single-point estimates and may not fully capture non-market values, community preferences, or the uncertainties associated with input variables (Boardman et al., 2018). Moreover, they may fail to address the structural complexities of socio-environmental systems, leading to potential oversights in planning for uncertain outcomes (Helgeson & Li, 2022). In contrast, sophisticated models—including agent-based and system dynamics models—encompass these complexities and uncertainties in resilience planning and evaluation of candidate strategies. Such models, however, come with their own set of challenges, including intensive demands on time, financial resources, and data requirements (Mls et al., 2023; Bottero et al., 2020). To overcome these challenges, there is a growing need for user-friendly, accessible models that provide comprehensive and reliable results for community resilience planning. The Co- DECIDR modeling approach addresses these needs by integrating Fuzzy Cognitive Mapping (FCM) with Benefit-Cost Analysis (BCA), harnessing the strengths of both methods and allowing for the consideration of both monetary and non-monetary valuation within resilience planning and providing an avenue by which community-level values can be assessed. This integrated approach systematically combines the qualitative depth of FCM with the quantitative precision of BCA, enabling planners to navigate through complex SES with greater clarity and effectiveness. Through more inclusive and transparent models like the Co-DECIDR approach, which incorporates economic factors and the broader socio-environmental context, we can enhance resilience interventions that equitably and effectively address the complexities and uncertainties of dynamic 72 systems. Subsequent sections of this article will further demonstrate how the Co-DECIDR approach facilitates these outcomes. 4-2-Co-DECIDR The Co-DECIDR approach employs two publicly available online tools: Mental Modeler (Gray et al., 2013) for FCM and Economic Decision Guide Software (EDGe$) (Helgeson et al., 2017, 2021) for BCA. Mental Modeler facilitates collaborative system modeling with stakeholders, capturing collective knowledge for disaster response prioritization and broader community concerns. EDGe$, developed by the National Institute of Standards and Technology (NIST), offers a standardized modeling tool for the economic evaluation of resilience investments. Together, these tools streamline the planning process, allowing for a nuanced analysis of strategies against the backdrop of economic limitations and the SES’s inherent complexities. Economic models are often employed to support resilience planning in SESs by providing quantitative estimates of the costs and benefits associated with different interventions, scenarios, and policies (Gilbert & Ayyub, 2016). One widely used economic tool in this context is BCA, which helps with economic assessment of various resilience alternatives by quantifying the monetary values of net associated benefits and costs (Proag, 2021). However, as with participatory modeling techniques, relying solely on economic models for resilience planning can have several drawbacks. First, economic models may not capture the complex interactions and feedback between human and natural systems, potentially overlooking the non-market values and services. Such elements, including social cohesion, dignity, and well-being, cannot be readily converted into economic values. (Rogers et al., 2019, Rising et al., 2022). Second, economic models may reinforce structural racism as they prioritize economic efficiencies over social equity, contributing to the hindrance of achieving equity in resilience planning (Hendy et al., 2023). Third, these models may not adequately account for the uncertainty and unpredictability of SES dynamics, often assuming a stable or equilibrium state (Welsh, 2014, Berger & Marinacci, 2020, Helgeson & Li, 2022). Fourth, economic models often do not involve the active participation and collaboration of multiple stakeholders—such as local communities, governments, and researchers—each of whom may have different perspectives, values, and interests in the SES (Raciborski et al., 2022). Therefore, economic models for the resilience planning of SESs should be complemented by other modeling approaches that can ensure a more robust and equitable response to the resilience challenges faced by communities. 73 Integrating economic models with the participatory modeling process is a promising approach to ensure the meaningful engagement of various disciplines and stakeholders, a vital aspect of comprehensive modeling for resilience planning in SESs (Miles, 2018, Helgeson & Li, 2022). Such a holistic approach also assists in managing the inherent uncertainties that characterize SES. FCM is a participatory modeling technique that allows stakeholders and experts to collectively construct semi-qualitative cognitive maps that represent their mental models of the SES. These maps help to explain the complexities and interconnections within the system, facilitating better decision-making (Gray et al., 2015). However, FCMs are unitless and simulations show relative change under different scenarios without any temporal dynamics, which can be insufficient information for decision-makers. It can be vital to compare economic factors like direct costs or return on investment over different time horizons. By integrating FCMs into economic modeling tools (e.g., BCA) for resilience planning in SES, we can utilize both local expertise and systems modeling with economic evaluation of alternatives. The four steps in the Co-DECIDR process are summarized below and visualized in Figure 4-1. Step 1: Develop a thorough definition of the scope of resilience the community will address in the modeling process. This entails identifying the key components of the SES under study, clarifying the objectives and values pertinent to the community, and comprehending the interrelationships and dynamics among these system components. This foundational phase— critical for defining 'resiliency of what and for whom (Meerow & Newell, 2021)'—utilizes FCM to visually depict and assess the interactions among system components (Gray et al., 2013). This process also highlights how system components can impact non-monetary community values. The participatory aspect of this step is crucial not only for understanding the system's complexity but also for fostering inclusiveness and knowledge sharing, which are instrumental in advancing GMP. Step 2: Determine potential shocks that could challenge the system's balance to assess the system’s 'resilience to what.’ (Meerow & Newell, 2021). Anticipating various disruptions— whether they be environmental, economic, or social—and exploring both the potential monetary losses and non-monetary impacts is key to identifying shocks. Initiate with a literature review to understand the 74 Figure 4-1. The four steps of the Co-DECIDR modeling approach impacts of similar shocks on comparable systems, laying a foundational knowledge base. Then, conduct a qualitative analysis of community narratives and mental models related to these shocks, utilizing FCMs to dive deep into localized perceptions. This labor-intensive endeavor is crucial for the nuanced impact assessment of shocks. The inclusion of local knowledge decreases structural uncertainty about the system's functions, enabling a more accurate assessment of both the direct 75 and indirect effects of shocks on the system (Van Vliet et al., 2020). This is essential for enhancing resiliency planning, as it ensures that strategies are grounded in a comprehensive understanding of the community's specific needs and vulnerabilities. Step 3: Select and evaluate candidate alternatives as coping strategies for the identified shocks. The narratives gathered from local experts through FCM serve as a valuable resource for choosing these candidate alternatives. Once the alternatives are determined, their effectiveness can initially be tested through FCM scenario analysis. This ‘what-if’ scenario analysis can be conducted in two distinct ways. The first approach involves creating a single FCM through group conversation with local experts, followed by running the scenario analysis (Jetter & Schweinfort, 2011). The second approach aggregates individual FCMs—each captured separately from local experts—to form a collective intelligence (CI) model, upon which scenario analysis is then conducted (Knox et al., 2023). These analyses examine how the selected alternatives impact the community-identified values and other system components, focusing on the non-monetary performance of the alternatives. These community-identified values represent what the community most values regarding its desirable future and can be captured through workshops and interviews (See section 4.1). To complement the qualitative impact assessment of each candidate alternative from the FCM scenario analysis, conducting a BCA is essential for the quantitative assessment of the economic feasibility of each alternative. This process requires identifying the associated benefits, costs, and externalities of each candidate alternative, which is crucial for gaining insights into their economic performance (Douthat et al., 2023). The outcomes from FCM scenario analysis can shed light on the externalities of each alternative at the community level, providing instrumental insights for BCA. Step 3 promotes GMP by incorporating local priorities and valuing descriptive knowledge for quantification and extending performance assessment beyond common metrics to include qualitative methods. Step 4: Conduct a trade-off analysis to rank the alternatives based on their impacts on community-identified values and relevant economic outcomes. This step effectively merges the BCA outcomes of each alternative with the scenario analysis results from the FCM. Only those alternatives deemed viable by the community are ranked, ensuring the identification and prioritization of the most effective strategies for enhancing resilience. Given the iterative nature of the Co-DECIDR process, the initial results will be shared with community members and local experts to evaluate if the alternatives align with the community's criteria for viability. In this 76 context, 'viability' refers to the acceptability and feasibility of the alternatives for the community. If any alternative is found non-viable according to the community's criteria, a new alternative should be selected, necessitating a revisit to steps 2 and 3 of Co-DECIDR. Conversely, if the alternatives are confirmed as viable by the community, a multi-criteria decision analysis (MCDA) is applied to the outcomes for a thorough ranking process. MCDA is used in resilience planning to evaluate and compare alternative candidates, enhancing decision-making for robust preparedness strategies (Abdullah et al., 2021; Gomes et al., 2023; Rezvani et al, 2023). Depending on the level of community engagement for final assessment of the alternatives, this step can significantly enhance the legitimacy of the resilience planning process, which is a crucial aspect of GMP. In addition, for this step, conducting sensitivity analysis and uncertainty analysis on BCA and MCDA results is highly recommended (Stewart, 2005; Maliene et al., 2018; Farrow et al., 2020). These analyses are crucial for evaluating the robustness and reliability of the rankings, ensuring a comprehensive assessment of each candidate alternative. To test the application of Co-DECIDR, we developed two proof-of-concept examples. These examples were centered around the use of data collected by a community-engaged initiative, the Flint Leverage Points Project (FLPP)3. The primary goal of FLPP was to identify crucial leverage points capable of positively transforming Flint’s food system (Schmitt-Olabisi et al 2023, Hodbod & Wentworth, 2022). The objective of these proof-of-concept examples was to showcase how data obtained from such community-based projects could be efficiently integrated into the Co-DECIDR framework. This integration aims to facilitate the assessment of potential shocks and the evaluation of various candidate alternatives for coping with these shocks. 4-3-Proof-of-Concept Examples Each proof-of-concept demonstration in this study includes one specific shock and two candidate alternatives that provide adaptations or coping strategies. During the FLPP data collection, participants from diverse communities and disciplines expressed concerns regarding numerous factors that could disrupt Flint's food system. Among these factors, we selected two distinct yet representative shocks: a pandemic such as COVID-19, (representing a shock rooted in societal dynamics) and extreme weather events (originating from climate change). 3 https://www.canr.msu.edu/flintfood/ 77 4-3-1-COVID-19 The effects of COVID-19 on the food system and communities have been significant (Galanaksi, 2020; O'Hara & Toussaint, 2021) , and clearly intensified in areas where food access is difficult already, such as Flint. Some studies have investigated the impact of the pandemic on the food distribution sector—such as the temporary or permanent closure of food outlets during the pandemic—revealing that a considerable number of restaurants, fast-food establishments, bars, convenience stores, and groceries were forced to close their businesses (Yi et al., 2021; Bell & Taylor, 2023). These closures, in turn, had far-reaching consequences, resulting in the loss of revenue for their staff and food producers, while simultaneously reducing food access for the community (Yi et al. 2021). Furthermore, the pandemic caused consumers to face a substantial increase in food prices— especially for healthy, perishable food items (Lewis et al. 2023). As a result of these economic challenges, the number of food insecure individuals in Michigan surged by 18.7% during the pandemic (Michigan’s Food Security Council, 2022), underscoring the critical impact on many communities’ well-being. Beyond these socio-economic hardships, the impact on human lives cannot be overstated, as many individuals found themselves hospitalized or faced the tragic loss of family members, resulting in unimaginable damage to communities. For this study, two distinct alternatives were selected as coping strategies to mitigate and adapt to the impacts of COVID-19 on the food system: 1) establishment of a new open-air (outdoor) farmers market and 2) opening a new food hub with delivery services that acts as a food processing plant. Various studies have affirmed the pivotal role of localized food systems in ensuring food security during crises (Worstell, 2020; Thilmany et al., 2021). To strengthen local and regional food systems, different business models can be employed. Notably, farmers' markets emerged as a cornerstone of resilience during the pandemic, connecting local producers directly to consumers at a time when larger supermarkets grappled with empty shelves due to their reliance on national and international food supply chains (Wentworth et al., 2023). Additionally, the surge in consumer demand for food delivery during COVID-19 led to the consideration of local food processing hubs that offer online markets as a viable alternative (Gu et al., 2021). With individuals spending more time at home and greater inclination towards home-cooked meals, local food processing hubs offering delivery options for community members emerged as a promising solution. These strategies represent alternatives aimed at enhancing food system resilience in the face of pandemics. 78 4-3-2-Extreme Weather Events Additionally, we selected extreme weather due to its impact on food systems and the relevance to resilience planning. Extreme weather events exacerbated by climate change, such as wildfires, tornados, heatwaves, or storms currently pose significant challenges for cities and urban planning. The main forms of extreme weather in Michigan are heatwaves and severe precipitation (EPA, 2016; MDHHS, 2018). These extreme weather events impact food systems in a few ways. First, transportation and distribution systems can be disrupted. Participants in FCM interviews described how deeply connected transportation is to community members’ abilities to source the type, quality, and amount of food they want. Second, electricity outages and higher temperatures increase food spoilage and waste. Low-income, elderly, and disabled populations are particularly vulnerable to interrupted accessibility of food and various extreme weather-related health risks (White et al., 2010). In this paper, we selected two alternatives that would lessen the impacts of extreme weather on the transportation system and food accessibility. These two alternatives are 1) installing bus shelters and 2) upgrading the limited number of convenience stores to healthy food outlets. There are approximately 1,200 bus stops for the fourteen lines that the Flint Mass Transit Authority (MTA) operates (MTA, 2022). While some currently have shelters, a vast majority do not. Bus shelters have multiple benefits, including benches to rest on and roofing and walls to protect riders from rain, sun, and wind. Stover and McCormack (2012) found shelters to be an effective intervention that ameliorated ridership losses from rain, particularly at high-use stops to increase usage by riders. Additionally, this alternative can enhance food accessibility for public transportation users during heatwaves and severe precipitation, making it easier for them to reach food sources without discomfort. For the BCA, we will model the construction and maintenance of 100 new bus shelters. The next alternative proposes transforming 10 existing convenience stores in different neighborhoods with limited access to fresh markets into ‘healthy convenience stores’. This initiative is designed to enhance infrastructure and facilities within these stores to offer a wider range of fresh local products, including fruits, vegetables, meat, and dairy ensuring higher diversity of choice and affordability. By collaborating with local producers, these health convenience stores will provide consumers—especially those without personal vehicles—easier access to healthy food options in their neighborhood, promoting better dietary habits and reducing 79 food deserts. Moreover, shortening distances to food stores can enhance accessibility during extreme weather events. 4-4-Methods 4-4-1-Data Collection The data collection for our study primarily relied on the extensive efforts of the Flint Leverage Points Project (FLPP). Over a span of five years, a Michigan State University research team collaborated with a community partner, the Community Foundation of Greater Flint,) and was advised by a broader Community Consultative Panel (CCP) composed of Flint community members active in different aspects of the food system. Forging a strong partnership allowed us to gain profound insights into various aspects of the Flint food system (Schmitt-Olabisi et al 2023), work which built on many years of local, community-engaged research (Sadler et al., 2015; Masson-Minnock and Stockmann, 2010; Alaimo et al., 2008; Ober Allen et al., 2008). An array of data collection methods—including literature reviews, workshops, surveys, discussion group, and interviews—served as valuable tools to comprehensively capture the essence of the Flint food system. During workshops and interviews, participants were engaged in insightful discussions to articulate their vision of a desirable food system for Flint. These collaborative sessions revealed a rich array of values held by Flint community members (a more detailed outline of this method is described here, Belisle-Toler et al, 2021). As a result of these dialogues, a total of seven synthesized community-identified values (CIVs) emerged as pivotal to the Flint food system4: affordability, availability, nutritious foods, community empowerment, education, partnership, and quality of life is respected (Figure 4-2). The outcomes of these workshops served as a guiding framework for this study, with these seven CIVs acting as non-monetary criteria for the Co- DECIDR steps by which we assessed the potential impact of various shocks and evaluated candidate alternatives to enhance the Flint food system. 4 https://www.canr.msu.edu/flintfood/resources-and-publications/values-for-the-flint-food-system 80 Figure 4-2. Seven synthesized community-identified values (CIVs) and associated definitions, that are representative of desired Flint Food System. As a part of the work of the FLPP, we conducted 51 semi-structured FCM interviews with experts in the Flint food system. These interviews were carefully structured to concentrate on the impact of various system components on CIVs. They offered a chance to gain a deeper understanding of the Flint food system. This understanding was valuable for assessing the impacts of potential shocks on the system and how different candidate alternatives might influence it, providing useful insights for the application of the Co-DECIDR. We complemented our insights through the data collected through the FLPP with a diverse range of primary and secondary data sources. This comprehensive approach was adopted to enrich our understanding of the potential impacts that various selected shocks could have on the Flint food system, as well as to determine the input data associated with each candidate alternative for the BCA. These additional data sources encompassed a wide spectrum of information, including but not limited to statistical records, historical trends, government reports, and academic studies. Additionally, the data collection included four semi-structured interviews with experts who have extensive experience in Flint's food system. These experts have worked on the organizational/non- profit side of the food system and employed their academic training to work in settings where they can make policy changes (i.e. by serving on groups like the food policy council). The interviews served two main purposes: first, to confirm whether the suggested alternatives were practical and 81 could be implemented within the Flint food system; and second, to determine the importance of each criterion, both monetary and non-monetary, that would be used in the trade-off analysis. By integrating this multifaceted dataset with the CIVs and the insights garnered from our FCM interviews, we aimed to construct a robust analytical foundation for Co-DECIDR, one that could holistically assess the vulnerabilities and opportunities within Flint’s food system. 4-4-2-Data Analysis 4-4-2-1-FCM Analysis A rigorous inductive qualitative analysis (Bingham & Witkowsky, 2021) was employed to analyze the data from 51 semi-structured FCM interviews. The process began with the transcription of these interviews, ensuring every detail was captured accurately. Following transcription, a coding system was utilized to categorize the data, allowing for more in-depth insights about the identified key themes, patterns, and relationships within the data, which were essential for understanding the complex dynamics of Flint’s food system. This initial qualitative analysis served to determine the resilience scope (Step 1 of Co-DECIDR) to reveal how the components of Flint’s food system impact seven CIVs, to investigate how various drivers of change affect the food system, and to explore how different candidate alternatives can promote food system preparedness for future shocks. This qualitative analysis lays the foundation for our subsequent assessments. To create a holistic, parsimonious model of the system, FCMs can be aggregated to represent key concepts and relationships in a complex system. In this regard, the collected individual FCMs were aggregated through principal component analysis (PCA) to group participants with conceptually similar maps. The five group maps were created by using the mean of connection strengths, then were combined into a single CI model using the median of connection strengths (see Knox, 2023 for the full methodology). The CI Model created through FLPP was utilized as the basis for running ‘what-if’ scenarios to investigate how selected shocks and candidate alternatives as coping strategies affect Flint’s food system, specifically the seven CIVs (Step 2 & 1 of Co-DECIDR). See Appendix B for a more detailed explanation about the Flint food system CI model and running FCM scenarios representing each candidate alternative. This scenario analysis helped to understand the non-monetary effects of alternatives on Flint’s food system, especially those for which it may not be appropriate to attempt to monetize through revealed or stated economic techniques. 82 4-4-2-2-Benefit-Cost-Analysis To assess the economic feasibility of the candidate alternatives within each proof-of- concept example, we employed the NIST Economic Decision Guide Software (EDGe$) Online Tool (Helgeson et al., 2017, 2021). EDGe$ is capable of providing either point estimates or probability distributions for key financial indicators, including the benefit-cost ratio (BCR) and annual return on investment (AROI). To initiate the BCA with EDGe$, it is essential to determine the planning horizon, discount rate, and probability of occurrence for each shock. For the primary BCA assessment, we set a planning horizon of 30 years and a discount rate of 5%. The estimated probability of a pandemic akin to COVID-19 occurring is approximately once every 129 years, with a 95% confidence interval ranging from 120 to 140 years (Marani, 2021). For calculating the probability of heatwaves and severe precipitation events, we utilized 50 years of weather data for Flint. Our analysis revealed an average occurrence of three heatwaves and six severe precipitation events per year, as reported by NOAA (2023). In the subsequent phase of the BCA with EDGe$, we conducted a comprehensive assessment of costs, benefits, and externalities for each candidate alternative. Costs encompassed direct expenditures, including initial implementation and ongoing maintenance expenses. Conversely, benefits represented the positive outcomes and more tangible financial gains associated with each alternative. Additionally, we accounted for externalities—whether positive or negative—arising from the potential impacts of each alternative on third parties and the environment, relying on the outputs provided by FCMs and primary/secondary data collection. A comprehensive list of EDGe$ input values and data sources for all alternatives is available in Appendix C. To account for the associated uncertainty in input data, we adopted three distinct scenarios for conducting the BCA: a ‘best case’ with the lowest costs and highest benefits/externalities, a ‘mean case’ with average values, and a ‘worst case’ with the highest costs and lowest benefits/externalities. Table 4-1 illustrates an example input for the ‘Open Air Farmers Market’ alternative within the COVID-19 scenario. This analysis—integral to Co-DECIDR's third step—provided crucial insights into the monetized expected value of each alternative. 4-4-2-3-Trade-Off Analysis After an initial assessment of the monetary and non-monetary performance of candidate alternatives, trade-off analysis (i.e., Co-DECIDR step 4) serves as a vital tool to guide the identification of appropriate solutions, taking into account both societal priorities and financial 83 considerations. As outlined in Section 2, before moving forward with the ranking process, the community must assess the viability of the proposed alternatives. For our proof-of-concept examples, we utilized semi-structured interviews with four Flint food system experts to first thoroughly present the detailed operational aspects of each proposed alternative. Following this explanation, we asked these experts to assess the acceptability and feasibility of each option. This process ensured that the alternatives were viable, making them eligible for the subsequent ranking process. Table 4-1. The required EDGe$ input data for benefit-cost analysis of open-air farmers market alternative. The input data for all the alternatives is available in Appendix C Item Rent for the site Initial Supplies and Miscellaneous Market Management and Staffing Licenses and Permits Insurance Utility and Services Marketing and Promotions Revenue from Vendor Fees Health Benefits Spillover Effect on Nearby Businesses Reduction in Losses of Closed Food Outlets Reduction in Food Insecure People t e k r a M s r e m r a F r i A n e p O Classification Cost Cost Cost Cost Cost Cost Cost Best Case $3,000 Value Mean Case $15,000 Worst Case $24,000 $3,500 $6,000 $9,000 $80,000 $100,000 $125,000 $95 $150 $255 $1,000 $2,000 $3,000 $1,750 $4,200 $7,000 $4,500 $5,400 $6,000 Benefit $78,750 $55,500 $26,250 Externality $50,000 $35,000 $20,000 Externality $66,570 $57,500 $44,380 Externality $252,000 $210,000 $170,000 Externality $62,500 $50,000 $37,500 Fatalities Averted Human Life 10 8 6 To ensure a nuanced evaluation, the Co-DECIDR framework incorporates the Multi- Criteria Decision Analysis (MCDA) techniques within its trade-off analysis (see Kiker et al., 2005; Dodgson et al., 2009 for a variety of MCDA techniques). This approach systematically combines the outputs from the FCM scenario analysis, based on the CI model, with the BCA results from the EDGe$ to provide a clear and systematic ranking of alternatives. Figure 4-3 illustrates this process. Initially, all monetary and non-monetary results were transformed into normalized scores ranging from 1 to 100. The normalization process guarantees that scores for each criterion are proportionately adjusted within this range, thus allowing for a fair comparison of performance across varied metrics. Furthermore, through semi-structured interviews with experts in Flint’s food 84 system, using the Analytic Hierarchy Process, the evaluation process was refined by the determination of importance weights for each criterion, ensuring that the final scores accurately represent the prioritized values of the community. The Analytic Hierarchy Process (AHP), a well-structured method for organizing and analyzing complex decisions, was used to determine these importance weights (Saaty, 2003; Dodgson et al., 2009). Through interviews with four local food system experts, the AHP’s pairwise comparison method was employed to establish the relative significance of each criterion. The publicly available AHP-OS online tool was used to facilitate the calculation of these weights in a user-friendly and efficient way (Goepel, 2018). After individual responses were collected, the geometric mean method was applied to aggregate the importance weights from all respondents (Xu, 2000). With the importance weights and normalized scores for both monetary and non- monetary criteria in hand, a multi-attribute utility model (MAUM) is utilized to compute the final scores for each of the candidate alternatives. (See Benromdhane, 2021 for the detailed description of MAUM formulas). Figure 4-3. The information flow of FCM scenario analysis outputs, EDGe$ results, and importance weights for monetary and non-monetary criteria for trade-off analysis and ranking process 85 4-4-2-4-Sensitivity and Uncertainty Analysis The principles of GMP underscore the significance of sensitivity and uncertainty analysis in resilience planning (Perz et al., 2013). Adhering to these principles ensures that models employed to support decision-making processes are not only robust, but also reliable. In complex SES characterized by escalating environmental challenges and economic uncertainties, these analyses become imperative as they enable us to systematically evaluate how variations in key parameters can exert influence on the outcomes of resilience planning models. We conducted a sensitivity analysis to explore the impact of two critical parameters in BCA: the discount rate and the planning horizon. The discount rate is a fundamental factor in evaluating the present value of future costs and benefits, potentially affecting the economic feasibility of candidate alternatives (Boardman et al., 2018). To assess its influence, we tested the sensitivity of our model’s outcomes by varying the discount rates at 3%, 5%, and 7%. Furthermore, the planning horizon is another pivotal aspect of resiliency planning, determining the timeframe over which benefits, costs, and externalities are considered and directly impacting the long-term effectiveness of proposed strategies. In our analysis, we investigated planning horizons spanning 20, 30, 40, and 50 years. This allowed us to gauge how different discount rates and planning horizons could alter the cost-benefit analysis results, final scores, and ranking of alternatives in this study. In conjunction with sensitivity analysis, the influence of the uncertainty inherent in BCA input data on the final scores and the prioritization of alternatives across the spectrum of worst- case, mean-case, and best-case scenarios was meticulously evaluated. The details of the inputs utilized for each scenario and their data sources can be found in Appendix C. This approach provided a more comprehensive understanding of the uncertainties associated with resiliency planning, helping to make more informed decisions in the face of an uncertain future. 4-5-Results 4-5-1-Monetary and Non-Monetary Losses The COVID-19 pandemic and extreme weather events have significantly disrupted food systems worldwide, with considerable impact on food production, distribution, and consumption. Flint’s food system, like others, faced substantial challenges during these times. This section outlines the monetary losses and non-monetary impacts extracted from FCM narratives, focusing on key areas of disruption. 86 4-5-1-1-Monetary Losses Regarding COVID-19 The pandemic led to disruptions in both global and national food supply chains, causing a notable increase in food prices. From December 2019 to December 2020, food prices in the United States increased by approximately 3.4%, and from December 2020 to December 2021, by about 7% (U.S. Bureau of Labor Statistics, 2022). This inflation significantly burdened food consumers, impacting their economic accessibility to food products (Lewis et al., 2023). The increased food prices contributed to more than $34 million in losses for the Flint community during the first two years of COVID-19. The estimated value is based on the 33854 households in Flint and average food spending of $839 per month per family of two in Michigan (USCB, 2022; Uphomes, 2023). Moreover, health and safety guidelines during the pandemic necessitated the closure of many food outlets and restaurants, leading to the unemployment of many workers within the food system. In Flint, specifically, the unemployment rate surged by 19.5% (Jablonski et al., 2021). This increase in unemployment was notably influenced by the closure of approximately 16.5% of food outlets, which included 32 restaurants and 10 convenience stores in Flint (Bell & Taylor, 2023). These closures resulted in approximately $4.5 million in losses. Unemployment and other circumstances related to the pandemic increased the number of food-insecure individuals. Between 2018 and 2020, Flint experienced a notable increase in food insecurity, with rates climbing from 15% to 21% (Jablonski et al., 2021). By assuming a 4% increase due to the impact of COVID-19, and based on Flint's population of 79,000 (USCB, 2022), this adjustment translates to approximately 4,000 more residents facing food insecurity. Based on the estimated cost of healthcare associated with food insecurity, $250 per capita (Michigan’s Food Security Council, 2022), this led to about $1 million in losses. 4-5-1-2-Non-Monetary Losses Regarding COVID-19 Based on the narratives collected from the FCM interviews, the non-monetary losses of COVID-19 on the Flint community are multifaceted and profound, affecting various aspects of daily life and community well-being. The pandemic has significantly increased isolation and reduced community connections, which led to mental health deterioration and underscored the critical need for empathy and care in managing community relationships. Additionally, it has disrupted nutritional and educational programs, particularly affecting students' access to nutrition and educational resources due to the shift to virtual learning environments. The demand for emergency and supplemental food services surged as the community sought to navigate the 87 challenges of accessing food. Moreover, several people decreased their purchases from indoor big supermarkets because of safety concerns, highlighting the importance of food delivery services for those unable or unwilling to venture out. Furthermore, health disparities among different regions in Flint became more evident. Together, these factors illustrate the complex web of non-monetary losses that extend beyond financial metrics, deeply affecting the Flint community during the pandemic. Two example quotes from participants in FCM interviews underscore these points: “Participant ID-117: When you think about some who have lost their jobs due to COVID-19, not having access to get food, the mental piece is-is impactful. And we still are not sure how deep this is going. We're still learning as we go on this part. But I do truly feel that the mental state has suffered so much” “Participant ID-147: And that was real- it's been horribly impacted by COVID. Yeah, youth food access is really negatively affected. It changed where kids could get that. And it just changed it so, so much. And so many of those systems that we took for granted, were now changed. And the problem was that it continuously changed.” 4-5-1-3- Monetary Losses Regarding Extreme Weather Events Based on 50 years of NOAA Weather Service data (2023), Flint experiences an average of three heatwaves and six severe precipitation events annually. Yue & Kahn (2019) found that severe precipitation increases vehicle accidents by 40%. With Genesee County's average daily crashes at 29 (CJIC, 2021) and Flint accounting for 20% of the county's population, this results in an additional 8.12 crashes in Flint during severe precipitation. Using USDOT's 2023 valuations— $12,172,415 per fatal crash and $300,328 per injury crash—these precipitation events are estimated to cause economic impacts of approximately $764,033 and $529,190 per storm in Flint, respectively. During heatwaves, the incidence of heat-related illnesses (HRI), such as heat stroke and heat exhaustion, increases and can be life-threatening. These conditions particularly impact outdoor workers and those lacking access to shade, rest, and water (Anderson & Bell, 2011; Schmeltz et al., 2016). On average, Genesee County experiences 15 HRI cases per heatwave (MDHHS, 2018). For Flint, which represents 20% of the county's population, this translates to about 5 cases. Given the cost of $8,965 per HRI case (Schmeltz et al., 2016), the financial burden on Flint amounts to $44,825 per heatwave. These weather-related challenges not only pressure public health and safety systems but also disrupt the local food system, affecting both supply chains and food accessibility in Flint. 88 Electricity outages from storm damage leads to significant costs, including lost productivity, health issues, and food spoilage. The DOE (2013) estimates that annual weather- related outages cost between $81 and $157 per capita. In Flint, assuming 25% of its population is affected, this translates to annual losses between $1,632,717 and $3,164,649. Outages also increase food waste; a 5% rise in waste due to outages amounts to 80,628 lbs of additional food waste for Flint. Furthermore, food storage life decreases by half with every 2-3°C increase in temperature (Vermeulen et al., 2012). Assuming a 5% increase in food waste during heatwaves, it leads to 8,466 lbs for Flint. Considering the cost of $1.53 per pound of wasted food (Buzby & Hyman, 2012), this results in $61,680 in losses per severe precipitation event and $12,953 per heatwave. 4-5-1-4- Non-Monetary Losses Regarding Extreme Weather Events While FCM interview participants did not directly address the impact of extreme weather events on the Flint food system, their narratives shed light on residents' challenges with food accessibility. These challenges include the scarcity of adequate grocery stores in certain areas, difficulties accessing frequently visited food outlets via public transport, and an increased reliance on emergency food systems, such as food banks, due to limited food access. The importance of these insights becomes even more evident when considering the broader context of transportation's role in accessing food. Among the 51 participants, 34 highlighted "Access to Transportation" in their FCMs, underscoring transportation as a critical factor for community members to obtain the food they desire in terms of type, quality, and quantity. This emphasis on food accessibility illustrates its foundational role in addressing food security challenges in Flint. Severe precipitations and heatwaves further complicate these issues for the Flint community, highlighting the interconnectedness of weather, transportation, and food security in shaping residents' daily lives and well-being. The quote below from one of the FCM interviewees demonstrates an example challenge related to the food accessibility. “Participant ID-103: There are [a] lot of barriers to transportation and gaps to being able to get to a location. Especially when you’re talking about taking home groceries if you’re on the bus and it’s very cumbersome, especially if you’re managing small children. And safety, I think of ice and snow, right? Trying to get on and off a bus with two or three bags of groceries and a toddler and an infant? It’s near impossible.” 89 4-5-2-Evaluation of Alternative Candidates for Coping with COVID-19 By running scenarios through the CI FCM model, we investigated the non-monetary impacts of two alternative strategies for coping with COVID-19 on CIVs, as demonstrated by Figure 4-4. Both alternatives demonstrated a considerable positive impact on the CIVs of availability, nutritious food, and quality of life is respected; both strategies are capable of providing fresh, local products to consumers in a safer and more comfortable manner. When it comes to CIVs of education, community empowerment, and partnership, however, open-air farmers' markets have a more favorable impact as they enhance the direct sale of products from local producers to consumers, can offer educational training for their customers, and shorten the supply chain, which can be more resilient during pandemics. Affordability was the only CIV where both alternative strategies had a negative impact, possibly because they do not benefit from economies of scale and may offer slightly higher prices to their customers. Overall, the open-air farmers' market appears to be a more effective alternative than local food delivery in addressing the CIVs. Figure 4-4. Scores of FCM scenario analysis for the candidate alternatives against COVID-19. Scores could vary from -1 to +1. When comparing the economic feasibility of alternatives with a consideration of a 5% discount rate and a 30-year planning horizon, the food hub with delivery services demonstrates better economic performance in mean and best-case scenarios. The open-air farmers' market, however, has a slight advantage in the worst-case scenario. Table 4-2 presents some outputs from 90 the EDGe$ online tool—BCR and AROI—for these scenarios, both including and excluding externalities. Incorporating externalities accounts for the economic impacts on the community and third parties. This approach ensures that all social and environmental benefits and costs are integrated into our economic evaluation. Therefore, only outputs with externality have been considered for the evaluation of alternatives. (See Appendix C for the detailed costs, benefits, and externalities associated with each alternative). Regarding different scenarios, the BCR improves for the open-air farmers' market, moving from 0.79 in the worst case to 2.9 in the best possible situation. The same pattern is recognized in the AROI, which goes from a low end of -0.71% in the worst-case to a much better 6.34% in the best-case scenario. On the other side, the food hub with delivery services begins with a lesser BCR of 0.73 in the worst-case scenario but ultimately surpasses the open-air farmers' market in the best- case scenario with a BCR of 4.37. The AROI for the food hub, starting at -0.9% in the worst-case scenario, significantly increases to 11.25% in the best-case scenario, exceeding that of the open- air farmers' market. The broader variance in BCA and AROI across scenarios for the Food Hub indicates a higher degree of uncertainty associated with this alternative compared to the open-air farmers' market. The sensitivity of these results against different discount rates and planning horizons is further investigated in Section 5.5. Table 4-2. BCA results from EDGe$, demonstrating benefit-cost ratio (BCR) and annual return on investment (AROI) of candidate alternatives for COVID-19 Alternatives Economic Feasibility Metrics Without Externalities With Externalities Worst Case Mean Case Best Case Worst Case Mean Case Best Case Open-Air Farmers' Market BCR AROI (%) 0.4 -2 0.83 1.62 0.79 1.55 2.9 -0.58 2.06 -0.71 1.84 6.34 Food Hub with Delivery Services BCR 0.36 1.05 2.13 0.73 1.95 4.37 AROI (%) -2.12 0.18 3.77 -0.9 3.17 11.25 91 4-5-3-Evaluation of Alternative Candidates for Extreme Weather Events The non-monetary effects of bus shelter were significant on some of the CIVs as illustrated in Figure 4-5. The bus shelter initiative boosts substantially three CIVs including availability, nutritious food, and quality of life is respected, underscoring the role of enhanced transportation facilities in increasing food source accessibility and overall community well-being. This alternative, however, does not considerably advance education, partnership, or community empowerment. Conversely, healthy convenience store improvements moderately enhance availability, quality of life is respected, and nutritious food, showing that localized store enhancements can offer immediate, but limited benefits in food access. Both alternatives exert minimal effects on education, community empowerment, and partnership, suggesting the need for complementary programs to strengthen these particular CIVs. In contrast to the COVID-19 strategies, which negatively affected affordability, these alternatives positively impact affordability, allowing community members to utilize public transport and access proper food outlets or find necessities within their neighborhoods. Table 4-3, derived from EDGe$, indicates that both alternatives provide better results in the presence of externalities, with bus shelters showing a BCR increase from 0.047 to 0.244 and an AROI improvement, although remaining negative, from -3.17 to -2.51 from the worst to the best case. Healthy convenience stores exhibit a stronger economic performance, with the BCR jumping from 0.89 to an impressive 3.48 and the AROI turning positive, ranging from -0.37 to 8.27, indicating a considerable return in the best-case scenario. Healthy convenience stores hold a substantial economic advantage in both mean and best- case scenarios, suggesting that this alternative can be a proper choice in scenarios that consider the full array of externalities (i.e. strengthen local economy, health and nutrition enhancement, and saving energy). Considerable variance existed in economic performance across different scenarios, however, especially for healthy convenience stores, which indicates a higher level of uncertainty. The more conservative economic performance of bus shelters may appeal to decision-makers prioritizing stability. The sensitivity of these results to variations in discount rates and planning horizons is a critical factor and is further examined in Section 5.5. 92 Figure 4-5. Scores of FCM scenario analysis for the candidate alternatives against extreme weather events. Scores could vary from -1 to +1 Table 4-3. BCA results from EDGe$, demonstrating benefit-cost ratio (BCR) and annual return on investment (AROI) of candidate alternatives for extreme weather events Alternatives Economic Feasibility Metrics Without Externalities With Externalities Worst Mean Best Worst Mean Best Case Case Case Case Case Case Bus Shelters BCR 0.007 0.034 0.096 0.047 0.113 0.244 AROI (%) -3.3 -3.21 -3.01 -3.17 -2.95 -2.51 Healthy Convenience Store BCR 0.41 0.79 1.48 0.89 1.81 3.48 AROI (%) -1.97 -0.7 1.6 -0.37 2.71 8.27 93 4-5-4-Ranking of Criteria and Alternatives After capturing the perceptions of different experts on the importance weights for various dimensions—both monetary and non-monetary—and associated criteria, a geometric mean was calculated to aggregate these insights. Collectively, experts assigned greater significance to non- monetary aspects, which was reflected in the importance weight that was notably higher for these aspects compared to their monetary importance weight (Table 4-4). Upon normalization of weights across the nine criteria, quality of life is respected emerged with a significant weight of 0.2204, followed by community empowerment at 0.1172, signifying these as the top priorities among the seven CIVs. Conversely, nutritious food and education were considered less critical, with weights of 0.0490 and 0.0324, respectively, occupying the lower end of the spectrum. In the monetary dimension, the BCR was predominant with the highest importance weight of 0.2509. Table 4-4. Weights summary of dimensions and criteria Dimension Local Weights Criteria Local Local Global Global Weights Rank Weights Rank Non-Monetary 0.6733 Availability Affordability Education Partnership 0.1323 0.1556 0.0481 0.1399 Quality of Life is Respected 0.3273 Community Empowerment 0.1740 Monetary 0.3267 Benefit to Cost Ratio Nutritious Food 0.0728 0.7679 Annual Return on Investment 0.2321 5 3 7 4 1 2 6 1 2 0.0891 0.0711 0.0324 0.0942 0.2204 0.1172 0.0490 0.2509 0.0758 5 7 9 4 2 3 8 1 6 In examining the normalized scores for CIVs and economic criteria across different candidate alternatives, we observed nuanced trade-offs. As Table 4-5 represents, for alternatives designed to address COVID-19 challenges, the open-air farmers' market scored higher in non- monetary aspects (79.17) compared to the food hub with delivery services (70.31), yet it underperformed in monetary terms, scoring 31.02 against 38.57, respectively. This inverse relationship highlights a common trade-off where a strategy that excels in community-driven criteria may not be as strong monetarily. Moreover, strategies for extreme weather events reveal a similar pattern. The bus shelter, with a higher non-monetary score (74.39), faces a trade-off with 94 its lower monetary score (3.90), whereas healthy convenience stores maintain a more balanced profile with closer non-monetary (62.97) and monetary scores (35.93), leading to a higher normalized final score of 53.64. These findings underscore the complexity of resilience planning, where both monetary and non-monetary factors must be weighed to discern the most effective approach. The higher final score for the open-air farmers' market indicates that while monetary components are critical, incorporating non-monetary values is indispensable for holistic community-based resilience planning. Table 4-5. Normalized Final Score Calculation Normalized Evaluation Scores for COVID-19 Alternatives Normalized Evaluation Scores for Extreme Weather Alternatives Global Weights Open-Air Farmers Market Food Hub with Bus Delivery Shelter Services Healthy Convenience Stores 0.0891 0.0711 0.0324 0.0942 85.00 38.50 79.50 77.00 90.00 74.50 94.00 31.0 31.1 80.00 43.00 55.50 58.50 84.50 60.50 84.50 39.0 37.1 91.50 63.00 50.50 50.50 95.00 50.50 86.00 2.3 9.3 64.00 65.50 53.00 54.50 71.00 55.50 62.00 36.2 35.0 79.17 70.31 74.39 62.97 Criteria Availability Affordability Education Partnership Quality of Life is Respected 0.2204 Community Empowerment Nutritious Food Benefit to Cost Ratio 0.1172 0.0490 0.2509 Annual Return on Investment 0.0758 Normalized Non-Monetary Score Normalized Monetary Score Normalized Final Scores 31.02 61.62 38.57 58.36 3.90 49.50 35.93 53.64 95 4-5-5-Sensitivity Analysis and Uncertainty Analysis 4-5-5-1-Sensitivity to Discount Rate and Planning Horizon The sensitivity analysis performed on the final scores, considering variations in discount rates (DR) and planning horizons (PH), underscores the robustness in the ranking of alternative strategies. Based on this analysis, the final scores for each alternative candidate remained within unique, non-overlapping ranges regarding the 12 different DR and PH combinations (see detailed explanation of these combinations in Section 4-2-4). This demonstrates consistent stability in the rankings under the mean-case scenario, ensuring that the relative performance of each strategy remains unaffected by variations in financial forecasting for our proof-of-concept examples. Detailed results for the worst-case, mean-case, and best-case scenarios are provided in Appendix D, which reinforces the reliability of these rankings despite the alterations in DR and PH. While the variation in these BCA input parameters did not significantly affect the outcomes for our proof- of-concept examples, their consideration remains critical for comprehensively accounting for parameter uncertainty in resilience planning. 4-5-5-2-Sensitivity to Importance Weights The sensitivity analysis, as illustrated in Figure 4-6, sheds light on the diverse perspectives among four experts when assigning importance weights to monetary and non-monetary criteria, and how these variances impact the final scores and rankings of the alternative strategies. For instance, Expert 3 placed a higher emphasis on the criterion of partnership relative to other experts, and Expert 2 weighed the BCR more heavily while assigning less importance to quality of life is respected. Despite these individual differences, the aggregate effect of their varied perceptions largely aligned with the outcomes derived from the geometric mean approach, leading to a consistent ranking of candidate alternatives. According to Figure 4-7, the singular deviation was noted with Expert 3’s weights, where the healthy convenience stores, although scoring higher overall, were ranked below the Bus Shelter alternative. This variation underscores the crucial role of expert judgment in strategy evaluation and highlights how differing valuations of criteria importance can influence the final decision-making process. Such insights reinforce the necessity of incorporating diverse expert opinions to ensure a comprehensive assessment of strategies, which is fundamental for resilience planning. 96 Figure 4-6. The assigned importance weights for the monetary and non-monetary criteria by experts and their geometric mean Figure 4-7. Final scores for the candidate alternatives based on the experts’ importance weights and their geometric mean of their importance weights 97 4-5-5-3-Uncertainty Analysis—Worst-Case, Mean-Case, and Best-Case Scenarios In BCA, variations associated with input data (benefits, costs, and externalities) are another source of uncertainty that should be considered in resiliency planning. Figure 4-8 illustrates the final scores of four alternatives under three different scenarios: worst case, mean case, and best case. For each scenario, the graph depicts a range of potential final scores, indicated by the upper and lower bounds. These bounds demonstrate the variability in final scores due to the differing combinations of DR and PH, as explained in section 4.2. Inherent uncertainty in the costs, benefits, and externalities significantly affects both the final scores and the subsequent ranking of the alternatives. Regarding the COVID-19 alternatives, the open-air farmers' market holds the highest final scores, suggesting it is the preferable option under worst-case and mean-case scenarios. However, in the best-case scenario, the food hub with delivery services surpasses the open-air farmers' market, indicating its potential to yield the greatest final scores under optimal conditions. Conversely, between alternatives of extreme weather events, the healthy convenience store is the most favorable alternative for the mean-case and best-case scenarios. Yet, in the worst-case scenario, the bus shelter emerges as the better option. Among all the alternatives, the food hub with delivery services alternative exhibited the highest variation in final scores across scenarios, signifying heightened uncertainty. Conversely, the bus shelter alternative demonstrated minimal changes in final scores, indicating greater stability. This divergence may stem from the fact that the bus shelter is an infrastructure enhancement, whereas the other three alternatives are business models vulnerable to fluctuations in product supplies and customer demands. Moreover, the analysis revealed that uncertainty in costs, benefits, and externalities had a greater impact on final scores compared to DR/PH or variation in importance weights, underscoring the importance of carefully considering these factors in decision-making. 4-6-Discussion: The results elucidate the complexity of assessing resilience planning alternatives, illustrating how trade-offs between monetary benefits and non-monetary values shape decision-making. The model outputs highlight the variability in final scores based on different expert perspectives and the uncertainty inherent in input data, which can significantly influence the rankings of alternatives. 98 This complexity underscores the importance of a nuanced approach to resilience planning that accounts for both quantifiable economic impacts and qualitative aspects like CIVs. Figure 4-8. Comparison final scores for the candidate alternatives under worst-case, mean-case, and best-case scenarios As demonstrated through the proof-of-concept examples, the Co-DECIDR modeling approach significantly enhances resilience assessment frameworks by incorporating three publicly available online modeling tools, each adding unique value to understanding and planning for resilience in the SESs. Mental Modeler (Gray et al., 2013) was employed to capture the diverse mental models of stakeholders, facilitating an understanding of the complex dynamics within the SESs. EDGe$ (Helgeson et al., 2017) provided a robust modeling technique for the economic assessment of resilience planning alternatives. AHP-OS (Goepel et al., 2018) was instrumental in capturing expert perceptions to determine crucial importance weights for trade-off analysis. This integration offers a comprehensive and effective approach for community-based resilience planning across various SESs, considering both monetary and non-monetary aspects, thereby 99 broadening the scope of BCA and enhancing the applicability and effectiveness of resilience planning. Moreover, incorporating these tools aligns with GMP, ensuring model fitness for purpose by prioritizing usability, reliability, and feasibility, thereby meeting end-user needs through an accessible and practical resilience planning toolset (Hamilton et al., 2022). The involvement of community members throughout the four-step Co-DECIDR process— from system understanding and shock impact analysis to alternative selection, evaluation, and trade-off analysis—embodies GMP (Gray et al., 2018; Wentworth et al, 2024). This approach aligns with the principles of participatory ensemble modeling as outlined by Schmitt-Olabisi et al. (2020), emphasizing legitimacy, parsimony, and practicality. Such engagement in understanding the system's behavior, identifying the impacts of shocks on the system, selecting and evaluating candidate alternatives, and conducting trade-off analysis ensures the representation of diverse community voices and values in resiliency planning (Legitimacy). Furthermore, it maintains model transparency and accessibility through Co-DECIDR’s four steps (Parsimony), and addresses uncertainties in both process and outcomes with user-friendly modeling tools (Practicality). This community engagement not only fosters community trust and equity but also enhances the strength of resilience planning by understanding what the community members value, ensuring a comprehensive and effective response to systemic shocks. Although the engagement of stakeholders and experts is crucial for the participatory modeling of SES and resilience planning, careful consideration must be given to who is included (Stringer et al., 2006). The sensitivity analysis revealed that diverse expert perceptions in assigning importance weights can alter the outcomes and rankings of alternatives. Engaging various stakeholder groups in FCM can produce different CI models, each reflecting unique system interactions. Therefore, a meticulous selection of stakeholders and experts covering a wide range of perspectives is essential in the Co-DECIDR modeling approach and resilience planning. This deliberate inclusion not only ensures the reliability of modeling outcomes but also captures a comprehensive understanding of system interactions, potential responses to shocks, and the valuation of criteria and their significance. 4-7-Limitations and Future Research In our study, we utilized two straightforward proof-of-concept examples to demonstrate the Co-DECIDR application, focusing on a preliminary evaluation of candidate alternatives. The inherent limitations of this evaluation—particularly regarding the depth of the BCA and the 100 occasional reliance on assumptions due to data unavailability—underscore the need for more comprehensive evaluations in future work. The study's reliance on a limited number of interviews for determining importance weights presents another limitation. Increasing the number of interviews could enhance the reliability of these weights, ensuring they more accurately reflect community priorities and contribute to more robust decision-making processes. To address uncertainties related to benefits, costs, and externalities in BCA, we defined best-case, mean-case, and worst-case scenarios to determine the monetary and final scores. Future research can, however, leverage the advanced probabilistic capabilities of Monte Carlo simulations available within the EDGe$ package for a more nuanced analysis of uncertainties in BCA inputs. Monte Carlo simulations are instrumental in encapsulating the spectrum of uncertainties impacting resilience planning and the outcomes of various scenarios (Kannan et al., 2021; Mavrotas & Makryvelios, 2021). By employing Monte Carlo simulations, the robustness of models can be enhanced, facilitating a more comprehensive evaluation of resilience strategies within the dynamic and uncertain conditions that characterize the SESs. Future expansions of Co-DECIDR could benefit from incorporating Life Cycle Assessment (LCA) to evaluate the environmental impacts of candidate alternatives more thoroughly and effectively (De Luca et al., 2017). Utilizing a publicly available, user-friendly modeling tool for LCA, such as openLCA (Ciroth et al., 2014), would enable a more holistic understanding of the ecological consequences of resilience planning strategies, aligning with sustainability goals. LCA can be integrated with FCM models and BCA models to more comprehensively cover the social, environmental, and economic dimensions of resiliency strategies and facilitate trade-offs and multi-criteria analysis. 4-8-Conclusion Earlier sections illustrated how Co-DECIDR effectively combines the qualitative insights of FCM with the quantitative rigor of BCA and determines the ranking of candidate alternatives through trade-off analysis. This integration allows planners to more clearly and effectively address the complexities of SES. In applying the Co-DECIDR modeling approach to the Flint food system, our focus was on addressing two main types of shocks: pandemics like COVID-19 and extreme weather events. This comprehensive approach encompassed both monetary and non-monetary criteria to assess candidate alternatives for resilience planning. Utilizing tools like the EDGe$ online tool (Helgeson et al., 2017, 2021) and Mental Modeler (Gray et al., 2013) enabled us to 101 analyze the benefit-cost ratio and return on investment of potential strategies, while also exploring how CIVs crucial for the ongoing resilience planning of the Flint food system might be affected. It also sets the boundaries on which options are considered viable for the community. The application of Co-DECIDR yielded significant insights into the trade-offs between monetary and non-monetary criteria, emphasizing the importance of CIVs in resilience planning. By incorporating MCDA (Abdullah et al., 2021), we were able to identify the most suitable resilience strategies based on community priorities, ensuring a process that was equitable, cost- effective, and inclusive. This approach, grounded in the engagement of community representatives throughout all steps of the Co-DECIDR process, underscores its potential for yielding more equitable solutions than BCA methods alone. The Co-DECIDR modeling approach notably advances GMP in the resilience planning of SES and offers a leading practice approach that addresses several limitations of BCA alone. By integrating economic models with participatory modeling techniques, and by embodying principles such as addressing model feasibility and reliability, as well as ensuring stakeholder engagement throughout the modeling process, Co-DECIDR sets a new benchmark for modeling practices in resilience planning. This approach engages stakeholders in a manner that aligns with GMP by promoting inclusiveness, transparency, and robust decision-making under uncertainty. Specifically, the participatory aspect of Co-DECIDR, facilitated through tools like Mental Modeler, fosters a shared understanding of the system under study, ensuring that the modeling process is comprehensive and grounded in community-identified values and priorities. This guarantees that the models are not only technically sound but also socially relevant and accepted. Thus, Co-DECIDR addresses critical aspects of GMP such as meaningful engagement of end- users, systematic elicitation of stakeholder needs, and fostering transparency and traceability in model development. Moreover, by integrating qualitative and quantitative methods, Co-DECIDR effectively navigates the complexities and feedback loops within SES, offering an evaluation of resilience strategies that are reflective of diverse community needs. This alignment with GMP enhances the model's applicability, relevance, and the quality of decision support it provides for resilience planning. Furthermore, the flexibility in Co-DECIDR steps allows for broad application across decision making contexts and decision topics. Demonstrated by the proof-of-concept examples described in this paper, Co-DECIDR's integration of economic modeling with participatory 102 techniques offers valuable perspectives for other socio-ecological systems facing diverse shocks, ranging from wildfire management in forestry systems (McWethy et al., 2019) to water scarcity adaptations in regions affected by climate change (Roach et al., 2018). This adaptability highlights Co-DECIDR's capacity to inform resilience planning across various contexts and scales, making it a proper modeling approach for addressing complex social, economic, and environmental challenges. 103 5-1-Summary of Dissertation Chapters CHAPTER 5: CONCLUSION Food systems are complex adaptive systems (Meter, 2019), and a 'one-size-fits-all' approach cannot adequately address the specific needs of every local and regional food system (Dengerink et al., 2021; Ng'endo & Connor, 2022). Planning for food systems should be carefully tailored and align with what the community worth the most. Through this dissertation, I explored the integration of community-identified values (CIVs) into local food system planning, applying these concepts to the Flint food system in Michigan, USA. With a thorough analysis across three chapters, this research contributes to a clearer understanding of how these CIVs can shape the planning and guide the development of more effective and community-based food system strategies that can pave the way to a more sustainable future. Chapter 1 revealed a rich spectrum of measurement ideas for 15 CIVs through a participatory approach, identifying 21 subthemes under five main themes for what should be measured. As discussed, it matters what is measured to correctly and precisely reflect the status quo and the progress of different strategies. These sub-themes can provide a foundation for the development of multiple measures that can be applicable for monitoring and operationalizing CIVs regarding specific contexts. Chapter 2 provided evidence that highlights the variability in expert predictions regarding food system interventions, showcasing both consensus and divergence among stakeholders about the effectiveness of interventions. Moreover, this chapter demonstrated how FCM can be used for modeling food systems and for understanding and comparing experts' assumptions and causal reasoning in describing the effects of different interventions on CIVs. Finally, Chapter 3 introduced the Co-DECIDR framework, demonstrating its effectiveness in integrating both monetary and non-monetary factors into community resilience planning. Using two proof-of-concept examples for the Flint food system, it showed how trade-offs between monetary and non-monetary considerations can be evaluated and affect the selection of interventions. The research emphasizes the importance of customizing food systems planning to reflect the unique cultural, economic, and environmental contexts of communities, particularly in diverse settings like Flint. Flint has experienced significant challenges such as economic downturns resulting from the decline of the auto industry (Dandaneau, 1996) and public health crises, including the well-documented water contamination issues (Pauli., 2020). These adversities 104 highlight the city's unique characteristics: a community facing post-industrial economic challenges and significant public health needs. These chapters demonstrated that not only for food system planning but also for other sectors, the CIVs that describe the desirable future specific to any other context should be considered at the forefront of every local planning effort. If planning in Flint had been more genuinely community-based, perhaps previous hardships could have been prevented or mitigated by strategies that were more aligned with the specific needs and values of the community, thus fostering greater resilience and sustainability 5-2- Implications of Findings 5-2-1-Policy Implications The findings indicate that policymakers should be attentive to the diverse needs and priorities of communities, which can vary significantly from one community or neighborhood to another. Specific policy recommendations might include: 1) Implement flexible, context-specific measures when monitoring food system policies to ensure they align with local values and needs. This approach acknowledges that one measurement may not capture the complexities of food system interventions, necessitating a set of measurements that consider different aspects of the local food system. 2) Include a wide range of stakeholders in food system planning such as local government, community organizations, food policy councils, representatives from different sectors in food systems, community leaders, and neighborhood advocates. This diversity ensures a comprehensive understanding of the possible impacts of interventions on CIVs. 3) Acknowledge and incorporate opposite perspectives about interventions, as one person may have insights or experiences unique to their context that others may not, potentially opening new pathways that could improve policy implementation. 4) Beyond the effects of interventions on CIVs, evaluate the impacts on other system components that mediate these effects. Understanding these intermediary influences is crucial for developing more effective strategies and mitigating unintended consequences. 5) Conduct resilience assessments with the input of different stakeholders to understand the potential impacts of hazards and systemic disruptions and evaluate the viability of possible intervention candidates. This ensures that resilience planning is informed by a wide array of community perspectives and is tailored to actual needs and vulnerabilities. 105 6) Recognize that not all aspects of policy evaluation can be converted into dollar amounts. Criteria like community empowerment or partnership may be difficult or inappropriate to monetize. Therefore, considering both monetary and non-monetary factors in the evaluation of interventions and in trade-off analysis during resilience assessment is essential. 5-2-2-Theoretical Contributions This dissertation also contributed to our understanding of participatory approaches and their role in monitoring, systems thinking, and resilience assessment within food systems. Specific theoretical advancements might include: 1) By capturing measurement ideas from different food system experts, a diverse pool of brilliant ideas for the operationalization of CIVs was generated. These measurement ideas were cataloged in an online depository accessible to various entities such as food policy councils. 2) By asking perceptions of measurability and data availability, I identified areas that require more purposeful data collection or more innovative and standardized methods of metric and indicator development. 3) Comparing scenario analyses from individual FCMs and a collective intelligence model illustrates the need to consider both. The former identifies extreme predicted impacts and their narratives, while the latter offers a ‘wisdom of the crowd’ version of predicted impacts that integrates diverse perspectives. 4) The predicted impacts in the collective intelligence model may depend on parameter uncertainty—variations in the weights of connections—which suggests that uncertainty analysis should be considered before implementing any policy. 5) The integration of FCM and BCA within the novel Co-DECIDR framework, as discussed in Chapter 3, bridges qualitative and quantitative methodologies. This integration enhances community-based resilience assessments by providing a more comprehensive evaluation framework. 6) The integration of three publicly available online tools—Mental Modeler (Gray et al., 2013), EDGe$ (Helgeson et al., 2017), and AHP-OS (Goepel et al., 2018)—facilitates community engagement, system understanding, economic evaluation, and trade-off analysis, thereby supporting community-based resilience assessments. 106 5-3-Limitations and Future Research In Chapter 2, our survey requested participants to provide measurement ideas for 15 different CIVs, resulting in a lengthy survey. Unfortunately, only 31 participants completed the survey. A shorter version of the survey, asking for ideas on only 5 randomly selected CIVs, might increase participation rates. This condensed survey could also include more detailed questions about the respondents' backgrounds or more explanations about where the necessary data for their measurements can be found. I hypothesized that there would be significant differences between academic members and community-based experts in terms of what and how they measure; however, this hypothesis was not supported by our sample. A larger sample size might provide a more reliable test of this hypothesis, as it would allow for more robust statistical analyses. In Chapter 3, I used FCM to investigate the predicted impacts of selected interventions (in our study, leverage points) on system outcomes (in our study, CIVs). However, scenario analysis through FCMs has certain limitations such as 1) the reliance on expert knowledge, which can introduce bias in the mental models and 2) the challenge of capturing dynamic changes over time, as FCMs typically represent static snapshots of systems. Moreover, although we collected 51 FCMs, only 26 individual FCMs for the first leverage point and 32 individual FCMs for the second leverage point included at least one path from the leverage point to the CIVs and were therefore eligible for the analysis. Collecting FCMs in a more structured way, where participants are explicitly asked to describe the effects of selected leverage points, could increase the number of FCMs that provide meaningful pathways. This, in turn, could result in a more reliable distribution of predicted impacts. Moreover, in chapter 3, for the selection of important paths and intermediary components, various network analysis methods are available. In this chapter, I used path scores and node scores (Chapter 3, Section 2-4). Future studies could compare different methods to determine the most effective approach for recognizing important paths and nodes within the network. Finally, in Chapter 2, I only addressed parametric uncertainty—specifically, variations in the weights of the collective intelligence (CI) model—and its effect on predicted impacts. However, structural uncertainty—variations in system components and their connections—can be analyzed in future studies to provide a more comprehensive understanding of the robustness and reliability of the model's predictions. 107 Finally, in chapter 4, our study used two proof-of-concept examples to illustrate the Co- DECIDR application, focusing on a preliminary evaluation of candidate alternatives. The limitations of this approach, including the shallow depth of the BCA and reliance on assumptions due to data gaps, highlight the need for more thorough evaluations in future work. We defined best-case, mean-case, and worst-case scenarios to address uncertainties in BCA related to benefits, costs, and externalities. Future research could use the advanced probabilistic capabilities of Monte Carlo simulations within the EDGe$ package to better analyze uncertainties in BCA inputs, thereby improving the robustness of models and facilitating a more comprehensive evaluation of resilience strategies under dynamic and uncertain conditions in social-ecological systems. 5-4-Toward a More Desirable Future Centering CIVs in food system planning, coupled with participatory approaches in monitoring, systems modeling, and resilience planning, is pivotal for fostering sustainability, equity, and food sovereignty. Participatory approaches ensure that diverse stakeholders, including marginalized groups, actively contribute to and shape the planning process. This inclusivity not only improves trust and cooperation among community members but also yields more comprehensive data and insights for systems modeling. This participatory approach helps promote food sovereignty by letting communities to collaborate on determining their own food policies and how to grow or get their food (Pimbert, 2009). Additionally, this way of including everyone gives all community members, especially those often left out, a voice in how food is managed, enhancing equity (Mui etal., 2021). Furthermore, such collaborative models are crucial for resilience planning, as they allow communities to identify and strengthen their capabilities to withstand and adapt to natural hazards and societal stressors. By embedding local knowledge and preferences into the heart of food system strategies, communities can tailor solutions that maximize local resources and social capital, ultimately leading to more resilient and adaptive food systems. This approach not only addresses immediate food security concerns but also lays a foundation for long- term sustainability and resilience in the face of increasing environmental and social challenges. While community-centered and participatory approaches in food system planning offer numerous benefits, they also come with significant challenges. These methods are time and resource intensive, often requiring more investment than traditional top-down planning due to the need for extensive community engagement. Additionally, the diversity of community perspectives can sometimes lead to conflicts, complicating the process of reaching agreement on key priorities 108 and strategies. Moreover, the implementation of these approaches can be complex, demanding ongoing coordination and support to ensure that the involvement of various stakeholders remains inclusive and effective. Managing these challenges is crucial for the successful adoption of participatory methods in food system planning. 109 BIBLIOGRAPHY Abdullah, M. F., Siraj, S., & Hodgett, R. E. (2021). An overview of multi-criteria decision analysis (MCDA) application in managing water-related disaster events: analyzing 20 years of literature for flood and drought events. Water, 13(10), 1358. Alaimo, K., Packnett, E., Miles, R. A., & Kruger, D. J. (2008). Fruit and vegetable intake among urban community gardeners. Journal of Nutrition Education and Behavior, 40(2), 94- 101. Allen, T., Prosperi, P., Cogill, B., Padilla, M., & Peri, I. (2019). A Delphi approach to develop sustainable food system metrics. Social Indicators Research, 141, 1307-1339. Ameli, M., Esfandabadi, Z. S., Sadeghi, S., Ranjbari, M., & Zanetti, M. C. (2023). COVID-19 and Sustainable Development Goals (SDGs): Scenario analysis through fuzzy cognitive map modeling. Gondwana Research, 114, 138-155. Aminpour, P., Gray, S. A., Jetter, A. J., Introne, J. E., Singer, A., & Arlinghaus, R. (2020). Wisdom of stakeholder crowds in complex social–ecological systems. Nat Sustain 3: 191–199. Ammann, J., Arbenz, A., Mack, G., Nemecek, T., & El Benni, N. (2023). A review on policy instruments for sustainable food consumption. Sustainable Production and Consumption, 36, 338-353. Anderson, G. B., & Bell, M. L. (2011). Heat waves in the United States: mortality risk during heat waves and effect modification by heat wave characteristics in 43 US communities. Environmental health perspectives, 119(2), 210-218. Arnold, R. D., & Wade, J. P. (2015). A definition of systems thinking: A systems approach. Procedia computer science, 44, 669-678. Atoloye, A., Schouboe, S., Misiaszek, C., Harding, J., Stowers, K. C., Bassarab, K., & Calancie, L. (2023). Developing a food system indicators database to facilitate local food systems assessments: Using a scoping review approach. Journal of Agriculture, Food Systems, and Community Development, 13(1), 1-14. Bakhtavar, E., Valipour, M., Yousefi, S., Sadiq, R., & Hewage, K. (2021). Fuzzy cognitive maps in systems risk analysis: a comprehensive review. Complex & Intelligent Systems, 7, 621- 637. Barbrook-Johnson, P., & Penn, A. S. (2022). Fuzzy cognitive mapping. In Systems Mapping: How to build and use causal models of systems (pp. 79-95). Cham: Springer International Publishing. 110 Barabási, A.-L. (2016). Network Science. Cambridge University Press. Basnet, S., Wood, A., Röös, E., Jansson, T., Fetzer, I., & Gordon, L. (2023). Organic agriculture in a low-emission world: Exploring combined measures to deliver a sustainable food system in Sweden. Sustainability Science, 18(1), 501-519. Bassarab, K., Clark, J. K., Santo, R., & Palmer, A. (2019). Finding our way to food democracy: Lessons from US food policy council governance. Politics and governance, 7(4), 32-47. Béné, C. (2020). Resilience of local food systems and links to food security–A review of some important concepts in the context of COVID-19 and other shocks. Food security, 12(4), 805-822. Belisle-Toler, R., Hodbod, J., & Wentworth, C. (2021). A mixed methods approach to exploring values that inform desirable food-systems futures. Sustainability: Science, Practice and Policy, 17(1), 362-376. Bell, K., & Reed, M. (2022). The tree of participation: a new model for inclusive decision- making. Community Development Journal, 57(4), 595-614. Bell, A., & Taylor, D. E. (2023). A Vanishing Food Infrastructure: The Closure of Food Outlets in Flint in a Pandemic Era. American Behavioral Scientist, 00027642221142202. Benromdhane, S. A. (2021). A multi-attribute utility model for environmental decision-making: an application to casting. Environment Systems and Decisions, 41, 21-32. Berger, L., & Marinacci, M. (2020). Model uncertainty in climate change economics: A review and proposed framework for future research. Environmental and Resource Economics, 77(3), 475-501. Bingham, A. J., & Witkowsky, P. (2021). Deductive and inductive approaches to qualitative data analysis. Analyzing and interpreting qualitative data: After the interview, 133-146. Boardman, Anthony E., David H. Greenberg, Aidan R. Vining, and David L. Weimer. 2018. Cost-Benefit Analysis: Concepts and Practice (5th Edition). Cambridge University Press. Bonacich, P. (1972). Factoring and weighting approaches to status scores and clique identification. Journal of Mathematical Sociology, 2(1), 113-120. Bosomworth, K., & Gaillard, E. (2019). Engaging with uncertainty and ambiguity through participatory ‘Adaptive Pathways’ approaches: Scoping the literature. Environmental Research Letters, 14(9), 093007. 111 Bostick, T. P., Holzer, T. H., & Sarkani, S. (2017). Enabling stakeholder involvement in coastal disaster resilience planning. Risk analysis, 37(6), 1181-1200. Bottero, M., Datola, G., & De Angelis, E. (2020). A system dynamics model and analytic network process: an integrated approach to investigate urban resilience. Land, 9(8), 242. Boyd, H., & Charles, A. (2006). Creating community-based indicators to monitor sustainability of local fisheries. Ocean & Coastal Management, 49(5-6), 237-258. Brandes, U. (2001). A faster algorithm for betweenness centrality. Journal of Mathematical Sociology, 25(2), 163-177 Buchan, R., Cloutier, D., Friedman, A., & Ostry, A. (2015). Local food system planning: The problem, conceptual issues, and policy tools for local government planners. Canadian Journal of Urban Research, 24(1), 1-23. Bull, J. W., Strange, N., Smith, R. J., & Gordon, A. (2021). Reconciling multiple counterfactuals social‐ecological conservation biodiversity evaluating impact in when systems. Conservation Biology, 35(2), 510-521. Bureau of Labor Statistics, U.S. Department of Labor, The Economics Daily, Prices for food at home up 13.5 percent for year ended August 2022 at https://www.bls.gov/opub/ted/2022/prices-for-food-at-home-up-13-5-percent-for-year- ended-august-2022.htm Buzby, J. C., & Hyman, J. (2012). Total and per capita value of food loss in the United States. Food Policy, 37(5), 561-570. Campbell-Arvai, V., & Lindquist, M. (2021). From the ground up: Using structured community engagement to identify objectives for urban green infrastructure planning. Urban Forestry & Urban Greening, 59, 127013. Campbell, C., Papanek, A., DeLong, A., Diaz, J., Gusto, C., & Tropp, D. (2022). Community food systems resilience: Values, benefits, and indicators. Journal of Agriculture, Food Systems, and Community Development, 11(4), 89-113. Carlsson, L., Callaghan, E., Morley, A., & Broman, G. (2017). Food system sustainability across scales: a proposed local-to-global approach to community planning and assessment. Sustainability, 9(6), 1061. Carnegie, M., McKinnon, K., & Gibson, K. (2019). Creating community‐based indicators of gender equity: A methodology. Asia Pacific Viewpoint, 60(3), 252-266. 112 Chollett, I., Escovar‐Fadul, X., Schill, S. R., Croquer, A., Dixon, A. M., Beger, M., ... & Wolff, N. H. (2022). Planning for resilience: Incorporating scenario and model uncertainty and trade‐offs when prioritizing management of climate refugia. Global Change Biology, 28(13), 4054-4068. Ciroth, A., Winter, S., & Berlin, G. (2014). openLCA 1.4 overview and first steps. GreenDelta, Berlin, 174. CJIC. (2021). Statewide Traffic Crash Data Year End Report. Michigan State Police Retrieved from https://www.michigan.gov/msp/-/media/Project/Websites/msp/cjic/Traffic-Crash- Reporting-Unit-Files/Year-End-Reports/CJIC---Annual-Crash-Report-Year-End-2021.pdf Clapp, J., Moseley, W. G., Burlingame, B., & Termine, P. (2022). The case for a six-dimensional food security framework. Food Policy, 106, 102164. Cleveland, D. A., Carruth, A., & Mazaroli, D. N. (2015). Operationalizing local food: Goals, actions, and indicators for alternative food systems. Agriculture and Human Values, 32, 281-297. Dandaneau, S. P. (1996). A town abandoned: Flint, Michigan, confronts deindustrialization. State University of New York Press. De Luca, A. I., Iofrida, N., Leskinen, P., Stillitano, T., Falcone, G., Strano, A., & Gulisano, G. (2017). Life cycle tools combined with multi-criteria and participatory methods for agricultural sustainability: Insights from a systematic and critical review. Science of the Total Environment, 595, 352-370. Dengerink, J., Dirks, F., Likoko, E., & Guijt, J. (2021). One size doesn’t fit all: Regional differences in priorities for food system transformation. Food Security, 13(6), 1455-1466. Dodgson, J. S., Spackman, M., Pearman, A., & Phillips, L. D. (2009). Multi-criteria analysis: a manual. DOE. (2013). Economic Benefits of Increasing Electric Grid Resilience to Weather Outages. Retrieved from https://www.energy.gov/articles/economic-benefits-increasing-electric- grid-resilience-weather- outages#:~:text=Continued%20investment%20in%20grid%20modernization,Americans %20when%20extreme%20weather%20strikes Douthat, T. H., Akhter, F., Sanderson, R., & Penn, J. (2023). Stakeholder Perceptions about Incorporating Externalities and Vulnerability into Benefit–Cost Analysis Tools for Watershed Flood Risk Mitigation. Sustainability, 15(9), 7473. 113 Dubbeling, M., Santini, G., Renting, H., Taguchi, M., Lançon, L., Zuluaga, J., ... & Andino, V. (2017). Assessing and planning sustainable city region food systems: Insights from two Latin American cities. Sustainability, 9(8), 1455. EPA. (2016). What Climate Change Means for Michigan. Retrieved from https://nepis.epa.gov/Exe/ZyPURL.cgi?Dockey=P100QV9I.txt Fanzo, J., Haddad, L., McLaren, R., Marshall, Q., Davis, C., Herforth, A., ... & Kapuria, A. (2020). The Food Systems Dashboard is a new tool to inform better food policy. Nature Food, 1(5), 243-246. Fanzo, J., Haddad, L., Schneider, K. R., Béné, C., Covic, N. M., Guarin, A., ... & Moncayo, J. R. (2021). Rigorous monitoring is necessary to guide food system transformation in the countdown to the 2030 global goals. Food Policy, 104, 102163. Farrow, S., & von Winterfeldt, D. (2020). Retrospective Benefit–Cost Analysis of Security- Enhancing and Cost-Saving Technologies. Journal of Benefit-Cost Analysis, 11(3), 479- 500. Ferreira, L., & Hitchcock, D. B. (2009). A comparison of hierarchical methods for clustering functional data. Communications in statistics-simulation and computation, 38(9), 1925- 1949. Folke, C. (2006). Resilience: The emergence of a perspective for social–ecological systems analyses. Global environmental change, 16(3), 253-267. Forrester, J. W. (1994). System dynamics, systems thinking, and soft OR. System dynamics review, 10(2‐3), 245-256. Freeman, L. C. (1977). A set of measures of centrality based on betweenness. Sociometry, 40(1), 35-41. Garton, K., Kraak, V., Fanzo, J., Sacks, G., Vandevijvere, S., Haddad, L., ... & Swinburn, B. (2022). A collective call to strengthen monitoring and evaluation efforts to support healthy and sustainable food systems:‘The Accountability Pact’. Public health nutrition, 25(9), 2353-2357. Gaviglio, A., Bertocchi, M., Marescotti, M. E., Demartini, E., & Pirani, A. (2016). The social pillar of sustainability: a quantitative approach at the farm level. Agricultural and Food Economics, 4, 1-19. 114 Gilbert, S., & Ayyub, B. M. (2016). Models for the Economics of Resilience. ASCE-ASME Journal of Risk and Uncertainty in Engineering Systems, Part A: Civil Engineering, 2(4), 04016003. Gödecke, T., Stein, A. J., & Qaim, M. (2018). The global burden of chronic and hidden hunger: trends and determinants. Global food security, 17, 21-29. Goepel, K. D. (2018). Implementation of an online software tool for the analytic hierarchy process (AHP-OS). International journal of the analytic hierarchy process, 10(3). Gomes, L. S., Santos, S. P., Coelho, L. S., & Rebelo, E. L. (2023). Using MCDA to assist an Intermunicipal community develop a resilience strategy in face of the pandemic caused by the SARS-CoV-2. Socio-Economic Planning Sciences, 87, 101588. Gray, S. A., Gray, S., Cox, L. J., & Henly-Shepard, S. (2013, January). Mental modeler: a fuzzy- logic cognitive mapping modeling tool for adaptive environmental management. In 2013 46th Hawaii international conference on system sciences (pp. 965-973). IEEE. Gray, S. A., Zanre, E., & Gray, S. R. (2013). Fuzzy cognitive maps as representations of mental models and group beliefs. In Fuzzy cognitive maps for applied sciences and engineering: From fundamentals to extensions and learning algorithms (pp. 29-48). Berlin, Heidelberg: Springer Berlin Heidelberg. Gray, S. A., Gray, S., De Kok, J. L., Helfgott, A. E., O'Dwyer, B., Jordan, R., & Nyaki, A. (2015). Using fuzzy cognitive mapping as a participatory approach to analyze change, preferred states, and perceived resilience of social-ecological systems. Ecology and Society, 20(2). Gray, S., Voinov, A., Paolisso, M., Jordan, R., BenDor, T., Bommel, P., ... & Zellner, M. (2018). Purpose, processes, partnerships, and products: four Ps to advance participatory socio‐ environmental modeling. Ecological applications, 28(1), 46-61. Gray, Steven, Payam Aminpour, Caitie Reza, Steven Scyphers, Jonathan Grabowski, Robert Murphy Jr, Alison Singer et al. "Harnessing the collective intelligence of stakeholders for conservation." Frontiers in Ecology and the Environment 18, no. 8 (2020): 465-472. Gu, D., Dillard, M., Gerst, M., & Loerzel, J. (2023). Validating commonly used indicators for community resilience measurement. Natural hazards review, 24(2), 04023008. Guillaume, J. H., Helgeson, C., Elsawah, S., Jakeman, A. J., & Kummu, M. (2017). Toward best practice framing of uncertainty in scientific publications: A review of W ater R esources R esearch abstracts. Water Resources Research, 53(8), 6744-6762. 115 Gustafson, D., Gutman, A., Leet, W., Drewnowski, A., Fanzo, J., & Ingram, J. (2016). Seven food system metrics of sustainable nutrition security. Sustainability, 8(3), 196. Hamilton, M., Salerno, J., & Fischer, A. P. (2019). Cognition of complexity and trade-offs in a wildfire-prone social-ecological system. Environmental Research Letters, 14(12), 125017. Hamilton, S. H., Pollino, C. A., Stratford, D. S., Fu, B., & Jakeman, A. J. (2022). Fit-for-purpose environmental modeling: Targeting the intersection of usability, reliability and feasibility. Environmental Modelling & Software, 148, 105278. Helgeson, J. F., Grubb, S. A., & Webb, D. H. (2017). The Economic Decision Guide Software (EDGeS) Tool: User Guidance. Helgeson, J., & O’Fallon, C. (2021). Resilience dividends and resilience windfalls: Narratives that tie disaster resilience co-benefits to long-term sustainability. Sustainability, 13(8), 4554. Helgeson, J., & Li, J. (2022). New Ways to Get to High Ground. Issues in Science and Technology, 38(2), 64-67. Hendy, K., Mauri, A. I., & Creary, M. (2023). Bounded Equity: The Limits of Economic Models of Social Justice in Cannabis Legislation. Contemporary Drug Problems, 50(1), 121-135. Hibbard, M., & Lurie, S. (2012). Creating socio-economic measures for community-based natural resource management: a case from watershed stewardship organisations. Journal of Environmental Planning and Management, 55(4), 525-544. Hodbod, J., & Wentworth, C. (2022). Assessing Panarchy in Food Systems: Cross-Scale Interactions in Flint, Michigan. Applied Panarchy: Applications and Diffusion Across Disciplines, 227. Holling, C. S. (1973). Resilience and stability of ecological systems. Horton, P., Banwart, S. A., Brockington, D., Brown, G. W., Bruce, R., Cameron, D., ... & Jackson, P. (2017). An agenda for integrated system-wide interdisciplinary agri-food research. Food Security, 9(2), 195-210. IEEE, systems, and Software engineering vocabulary. (2017) International Standard ISO/IEC/IEEE 24765, Retrieved: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8016712 International Association of Public Participation [IAP2]. (2007). IAP2 Spectrum of Public Participation. Retrieved 116 from:https://cdn.ymaws.com/www.iap2.org/resource/resmgr/pillars/Spectrum_8.5x11_Pri nt.pdf Jablonski, B. B., Casnovsky, J., Clark, J. K., Cleary, R., Feingold, B., Freedman, D., ... & Wentworth, C. (2021). Emergency food provision for children and families during the COVID‐19 pandemic: examples from five US cities. Applied Economic Perspectives and Policy, 43(1), 169-184. Jacobi, J., Mukhovi, S., Llanque, A., Augstburger, H., Käser, F., Pozo, C., ... & Speranza, C. I. (2018). Operationalizing food system resilience: An indicator-based assessment in agroindustrial, smallholder farming, and agroecological contexts in Bolivia and Kenya. Land use policy, 79, 433-446. Jagustović, R., Zougmoré, R. B., Kessler, A., Ritsema, C. J., Keesstra, S., & Reynolds, M. (2019). Contribution of systems thinking and complex adaptive system attributes to sustainable food production: Example from a climate-smart village. Agricultural systems, 171, 65-75. Jakeman, A. J., Letcher, R. A., & Norton, J. P. (2006). Ten iterative steps in development and evaluation of environmental models. Environmental Modelling & Software, 21(5), 602- 614. Jakeman, A. J., El Sawah, S., Cuddy, S., Robson, B., McIntyre, N., & Cook, F. (2018). QWMN Good Modelling Practice Principles. Jaramillo, L. V., Stone, M. C., & Benson, M. H. (2021). Developing Factors Developing factors for Socio-Ecohydrological Resilience. In Handbook of disaster risk reduction for resilience: New frameworks for building resilience to disasters (pp. 387-416). Cham: Springer International Publishing. Jetter, A., & Schweinfort, W. (2011). Building scenarios with Fuzzy Cognitive Maps: An exploratory study of solar energy. Futures, 43(1), 52-66. Jones, A. D., Ngure, F. M., Pelto, G., & Young, S. L. (2013). What are we assessing when we measure food security? A compendium and review of current metrics. Advances in nutrition, 4(5), 481-505. Kannan, D., Moazzeni, S., mostafayi Darmian, S., & Afrasiabi, A. (2021). A hybrid approach based on MCDM methods and Monte Carlo simulation for sustainable evaluation of potential solar sites in east of Iran. Journal of Cleaner Production, 279, 122368. 117 Kenny, D. C., Bakhanova, E., Hämäläinen, R. P., & Voinov, A. (2022). Participatory modelling and systems intelligence: A systems-based and transdisciplinary partnership. Socio- Economic Planning Sciences, 83, 101310. Kiker, G. A., Bridges, T. S., Varghese, A., Seager, T. P., & Linkov, I. (2005). Application of multicriteria decision analysis in environmental decision making. Integrated environmental assessment and management: An international journal, 1(2), 95-108. Knox, C. B., Gray, S., Zareei, M., Wentworth, C., Aminpour, P., Wallace, R. V., ... & Brugnone, N. (2023). Modeling complex problems by harnessing the collective intelligence of local experts: New approaches in fuzzy cognitive mapping. Collective Intelligence, 2(4), 26339137231203582. Kok, K. (2009). The potential of Fuzzy Cognitive Maps for semi-quantitative scenario development, with an example from Brazil. Global environmental change, 19(1), 122-133. Koliou, M., van de Lindt, J. W., McAllister, T. P., Ellingwood, B. R., Dillard, M., & Cutler, H. (2020). State of the research in community resilience: Progress and challenges. Sustainable and resilient infrastructure, 5(3), 131-151. Kugelberg, S., Bartolini, F., Kanter, D. R., Milford, A. B., Pira, K., Sanz-Cobena, A., & Leip, A. (2021). Implications of a food system approach for policy agenda-setting design. Global Food Security, 28, 100451. Lam, S., Dodd, W., Skinner, K., Papadopoulos, A., Zivot, C., Ford, J., ... & IHACC Research Team. (2019). Community-based monitoring of Indigenous food security in a changing climate: global trends and future directions. Environmental Research Letters, 14(7), 073002. Lambrou, N., & Loukaitou-Sideris, A. (2022). Resilience plans in the US: an evaluation. Journal of Environmental Planning and Management, 65(5), 809-832. Larsen, D., Tyndall, J. C., Schulte, L. A., & Grudens-Schuck, N. (2019). Exploring stakeholder consensus for multiple outcomes in agriculture: an Iowa case study. Frontiers in Sustainable Food Systems, 3, 110. Leiter, T. (2015). Linking monitoring and evaluation of adaptation to climate change across scales: avenues and practical approaches. New Directions for Evaluation, 2015(147), 117-127. Leroy, J. L., Ruel, M., Frongillo, E. A., Harris, J., & Ballard, T. J. (2015). Measuring the food access dimension of food security: a critical review and mapping of indicators. Food and nutrition bulletin, 36(2), 167-195. 118 Levin, S., Xepapadeas, T., Crépin, A. S., Norberg, J., De Zeeuw, A., Folke, C., ... & Walker, B. (2013). Social-ecological systems as complex adaptive systems: modeling and policy implications. Environment and Development Economics, 18(2), 111-132. Lewis, M., Herron, L. M., Chatfield, M. D., Tan, R. C., Dale, A., Nash, S., & Lee, A. J. (2023). Healthy food prices increased more than the prices of Unhealthy Options during the COVID-19 Pandemic and Concurrent Challenges to the Food System. International Journal of Environmental Research and Public Health, 20(4), 3146. Liddy, H., Mowlds, S., McKeown, P. C., Lundy, M., & Spillane, C. (2023). Food mapping approaches for understanding food system transformations in rapid-growth city regions in the Global South. Frontiers in Sustainable Food Systems, 7. Maliene, V., Dixon-Gough, R., & Malys, N. (2018). Dispersion of relative importance values contributes to the ranking uncertainty: Sensitivity analysis of Multiple Criteria Decision- Making methods. Applied Soft Computing, 67, 286-298. Manikas, I., Ali, B. M., & Sundarakani, B. (2023). A systematic literature review of indicators measuring food security. Agriculture & Food Security, 12(1), 10. Marani, M., Katul, G. G., Pan, W. K., & Parolari, A. J. (2021). Intensity and frequency of extreme novel epidemics. Proceedings of the National Academy of Sciences, 118(35), e2105482118. Marshall, Q., Fanzo, J., Barrett, C. B., Jones, A. D., Herforth, A., & McLaren, R. (2021). Building a global food systems typology: a new tool for reducing complexity in food systems analysis. Frontiers in Sustainable Food Systems, 5, 746512. Martín, E. G., Giordano, R., Pagano, A., Van Der Keur, P., & Costa, M. M. (2020). Using a system thinking approach to assess the contribution of nature based solutions to sustainable development goals. Science of the Total Environment, 738, 139693. Martone, R. G., Bodini, A., & Micheli, F. (2017). Identifying potential consequences of natural perturbations and management decisions on a coastal fishery social-ecological system using qualitative loop analysis. Ecology and Society, 22(1). Masson-Minock, M., & Stockmann, D. (2010). Creating a legal framework for urban agriculture: Lessons from Flint, Michigan. Journal of Agriculture, Food Systems, and Community Development, 1(2), 91-104. Mavrotas, G., & Makryvelios, E. (2021). Combining multiple criteria analysis, mathematical programming and Monte Carlo simulation to tackle uncertainty in Research and 119 Development project portfolio selection: A case study from Greece. European Journal of Operational Research, 291(2), 794-806. May, C. K. (2022). Complex adaptive governance systems: a framework to understand institutions, in socio-ecological systems. Socio-ecological practice organizations, and people research, 4(1), 39-54. Maynard, D. D. C., Vidigal, M. D., Farage, P., Zandonadi, R. P., Nakano, E. Y., & Botelho, R. B. A. (2020). Environmental, social and economic sustainability indicators applied to food services: A systematic review. Sustainability, 12(5), 1804. McAllister, T. P. (2015). Community resilience planning guide for buildings and infrastructure systems, volume I. McHugh, M. L. (2013). The chi-square test of independence. Biochemia medica, 23(2), 143-149. McKnight, P. E., & Najab, J. (2010). Mann‐Whitney U Test. The Corsini encyclopedia of psychology, 1-1. McWethy, D. B., Schoennagel, T., Higuera, P. E., Krawchuk, M., Harvey, B. J., Metcalf, E. C., ... & Kolden, C. (2019). Rethinking resilience to wildfire. Nature Sustainability, 2(9), 797- 804. MDHHS. (2018). MiTracking: Michigan Environmental Public Health Tracking Accessed at: https://www.michigan.gov/mdhhs/safety-injury-prev/environmental- health/topics/mitracking Meadows, D. H. (2008). Thinking in systems: A primer. Sustainability Institute. Meerow, S., & Newell, J. P. (2021). Urban resilience for whom, what, when, where, and why?. In Geographic Perspectives on Urban Sustainability (pp. 43-63). Routledge. Meter, K. (2019). Assessing food systems as complex adaptive systems: Conceptual views and US applications. In Sustainable Food System Assessment (pp. 66-90). Routledge. Metropolis, N., & Ulam, S. (1949). The Monte Carlo method. Journal of the American Statistical Association, 44(247), 335-341. Miles, S. B. (2018). Participatory disaster recovery simulation modeling for community resilience planning. International Journal of Disaster Risk Science, 9, 519-529. 120 Miller, T. R., Baird, T. D., Littlefield, C. M., Kofinas, G., Chapin III, F. S., & Redman, C. L. (2008). Epistemological pluralism: reorganizing interdisciplinary research. Ecology and Society, 13(2). Mls, K., Kořínek, M., Štekerová, K., Tučník, P., Bureš, V., Čech, P., ... & Triantafyllou, I. (2023). Agent-based models of human response to natural hazards: systematic review of tsunami evacuation. Natural hazards, 115(3), 1887-1908. Monasterolo, I., Pasqualino, R., Janetos, A. C., & Jones, A. (2016). Sustainable and inclusive food systems through the lenses of a complex system thinking approach—a bibliometric review. Agriculture, 6(3), 44. Moragues-Faus, A., & Marceau, A. (2018). Measuring progress in sustainable food cities: An indicators toolbox for action. Sustainability, 11(1), 45. Morone, P., Falcone, P. M., & Lopolito, A. (2019). How to promote a new and sustainable food consumption model: A fuzzy cognitive map study. Journal of cleaner production, 208, 563-574. Morone, P., Yilan, G., & Imbert, E. (2021). Using fuzzy cognitive maps to identify better policy strategies to valorize organic waste flows: An Italian case study. Journal of Cleaner Production, 319, 128722. Mourhir, A. (2021). Scoping review of the potentials of fuzzy cognitive maps as a modeling approach for integrated environmental assessment and management. Environmental Modelling & Software, 135, 104891. MTA. (2022). Financial Report with Supplemental Information. In. Mui, Y., Khojasteh, M., Judelsohn, A., Sirwatka, A., Kelly, S., Gooch, P., & Raja, S. (2021). Planning for regional food equity. Journal of the American Planning Association, 87(3), 354-369. Murtagh, F., & Legendre, P. (2014). Ward’s hierarchical agglomerative clustering method: which algorithms implement Ward’s criterion?. Journal of classification, 31, 274-295. National Institute of Standard and Technology (NIST) (2021), Metrics and Measures, Retrieved: https://www.nist.gov/itl/ssd/software-quality-group/metrics-and- measures#:~:text=The%20terms%20metric%20and%20measure,level%2C%20or%20so mewhat%20subjective%20attributes Neely, C. L., Bourne, M., Chesterman, S., Vågen, T. G., Lekaram, V., Winowiecki, L. A., & Prabhu, R. (2021). Inclusive, cross-sectoral and evidence-based decision-making for 121 resilience planning and decision-making in a devolved context. The European Journal of Development Research, 33(4), 1115-1140. Newman, M. E. J. (2010). Networks: An Introduction. Oxford University Press. Ng'endo, M., & Connor, M. (2022). One size does not fit all—addressing the complexity of food system sustainability. Frontiers in Sustainable Food Systems, 6, 816936. NOAA. (2023). Global Historical Climatology Network-Daily. In: National Centers for Environmental Information Nuno, A., Bunnefeld, N., & Milner-Gulland, E. J. (2014). Managing social–ecological systems under uncertainty: implementation in the real world. Ecology and Society, 19(2). O'Hara, S., & Toussaint, E. C. (2021). Food access in crisis: Food security and COVID-19. Ecological Economics, 180, 106859. Ober Allen, J., Alaimo, K., Elam, D., & Perry, E. (2008). Growing vegetables and values: Benefits of neighborhood-based community gardens for youth development and nutrition. Journal of Hunger & Environmental Nutrition, 3(4), 418-439. Olabisi, L. S., Wentworth, C., Key, K., Wallace, R. V., McNall, M., Hodbod, J., & Gray, S. A. (2023). Defining success in community-university partnerships: lessons learned from Flint. Journal of Responsible Innovation, 10(1), 2102567. Ostertagova, E., Ostertag, O., & Kováč, J. (2014). Methodology and application of the Kruskal- Wallis test. Applied mechanics and materials, 611, 115-120. Pauli, B. J. (2020). The Flint water crisis. Wiley Interdisciplinary Reviews: Water, 7(3), e1420. Pearson, D. E., & Clark‐Wolf, T. J. (2023). Predicting ecological outcomes using fuzzy interaction webs. Ecology, 104(7), e4072. Perz, S. G., Muñoz-Carpena, R., Kiker, G., & Holt, R. D. (2013). Evaluating ecological resilience with global sensitivity and uncertainty analysis. Ecological Modelling, 263, 174-186. Pimbert, M. (2009). Towards food sovereignty (pp. 1-20). London: International Institute for Environment and Development. Proag, V., (2021). Analysis of Environmental Impacts of Infrastructure. Infrastructure Planning and Management: An Integrated Approach, 219-251. 122 Prosperi, P., Moragues-Faus, A., Sonnino, R., & Devereux, C. (2015). Measuring progress towards sustainable food cities: Sustainability and food security indicators. Report of the ESRC financed Project “Enhancing the Impact of Sustainable Urban Food Strategies. Proudfoot, K. (2023). Inductive/deductive hybrid thematic analysis in mixed methods research. Journal of Mixed Methods Research, 17(3), 308-326. Raciborski, R. A., Woodward, E. N., & Painter, J. T. (2022). Economic analyses of behavioral health intervention implementation: Perspective on stakeholder engagement. Frontiers in Psychiatry, 13, 1031325. Rezvani, S. M., de Almeida, N. M., & Falcão, M. J. (2023). Climate Adaptation Measures for Enhancing Urban Resilience. Buildings, 13(9), 2163. Rising, J. A., Taylor, C., Ives, M. C., & Ward, R. E. (2022). Challenges and innovations in the economic evaluation of the risks of climate change. Ecological Economics, 197, 107437. Roach, T., Kapelan, Z., & Ledbetter, R. (2018). A resilience-based methodology for improved water resources adaptation planning under deep uncertainty with real world application. Water resources management, 32, 2013-2031. Rogers, A. A., Dempster, F. L., Hawkins, J. I., Johnston, R. J., Boxall, P. C., Rolfe, J., ... & Pannell, D. J. (2019). Valuing non-market economic impacts from natural hazards. Rubinstein, R. Y., & Kroese, D. P. (2016). Simulation and the Monte Carlo method (Vol. 10). John Wiley & Sons. Saaty, T. L. (2003). Decision-making with the AHP: Why is the principal eigenvector necessary. European journal of operational research, 145(1), 85-91. Sadler, R. C., Arku, G., & Gilliland, J. A. (2015). Local food networks as catalysts for food policy change to improve health and build the economy. Local Environment, 20(9), 1103- 1121. Sarma, P., & Das, D. (2015). Application of Shannon’s index to study diversity with reference to census data of Assam. Asian J. Manag. Res, 5(4), 620-628. Schmeltz, M. T., Petkova, E. P., & Gamble, J. L. (2016). Economic Burden of Hospitalizations for Heat-Related Illnesses in the United States, 2001-2010. Int J Environ Res Public Health, 13(9). Schmidhuber, J., & Tubiello, F. N. (2007). Global food security under climate change. Proceedings of the National Academy of Sciences, 104(50), 19703-19708. 123 Schmitt-Olabisi, L., McNall, M., Porter, W., & Zhao, J. (Eds.). (2020). Innovations in collaborative modeling. MSU Press. Schmitt-Olabisi, L. S., Wentworth, C., Key, K., Wallace, R. V., McNall, M., Hodbod, J., & Gray, S. A. (2023). Defining success in community-university partnerships: lessons learned from Flint. Journal of Responsible Innovation, 10(1), 2102567. Schneider, K. R., Fanzo, J., Haddad, L., Herrero, M., Moncayo, J. R., Herforth, A., ... & Wiebe, K. (2023). The state of food systems worldwide in the countdown to 2030. Nature Food, 4(12), 1090-1110. Sellberg, M. M., Ryan, P., Borgström, S. T., Norström, A. V., & Peterson, G. D. (2018). From resilience thinking to Resilience Planning: Lessons from practice. Journal of environmental management, 217, 906-918. Stein, A. J., & Santini, F. (2022). The sustainability of “local” food: A review for policy- makers. Review of Agricultural, Food and Environmental Studies, 103(1), 77-89. Stewart, T. (2005). Dealing with uncertainties in MCDA. Multiple criteria decision analysis: state of the art surveys, 445-466. Stringer, L. C., Dougill, A. J., Fraser, E., Hubacek, K., Prell, C., & Reed, M. S. (2006). Unpacking “participation” in the adaptive management of social–ecological systems: a critical review. Ecology and society, 11(2). Stover, V. W., & McCormack, E. D. (2012). The Impact of Weather on Bus Ridership in Pierce County, Washington. Journal of Public Transportation, 15(1), 95-110. Swain, D., & Hollar, D. (2003). Measuring progress: Community indicators and the quality of life. International Journal of Public Administration, 26(7), 789-814. Tendall, D. M., Joerin, J., Kopainsky, B., Edwards, P., Shreck, A., Le, Q. B., ... & Six, J. (2015). Food system resilience: Defining the concept. Global food security, 6, 17-23. Thilmany, D., Brislen, L., Edmondson, H., Gill, M., Jablonski, B. B., Rossi, J., ... & Schaffstall, S. (2021). Novel methods for an interesting time: Exploring US local food systems’ impacts and initiatives to respond to COVID. Australian Journal of Agricultural and Resource Economics, 65(4), 848-877. Thomas, D. R. (2003). A general inductive approach for qualitative data analysis. United States Census Bureau (USCB). (2022). QuickFacts: Flint city, Michigan. Census Bureau QuickFacts;. https://www.census.gov/quickfacts/flintcitymichigan 124 Uphomes (2023), “What’s the cost of living in Michigan?”, Accessed at https://uphomes.com/blog/cost-of-living-michigan/ USDOT. (2023). Benefit-Cost Analysis Guidance for Discretionary Grant Programs. Retrieved from https://www.transportation.gov/sites/dot.gov/files/2023- 01/Benefit%20Cost%20Analysis%20Guidance%202023%20Update.pdf Van Berkum, S., & Ruben, R. (2021). Exploring a food system index for understanding food system transformation processes. Food Security, 13, 1179-1191. Van de Lindt, J. W., Kruse, J., Cox, D. T., Gardoni, P., Lee, J. S., Padgett, J., ... & Hamideh, S. (2023). The interdependent networked community resilience modeling environment (IN- CORE). Resilient Cities and Structures, 2(2), 57-66. Van Vliet, O., Hanger-Kopp, S., Nikas, A., Spijker, E., Carlsen, H., Doukas, H., & Lieu, J. (2020). The importance of stakeholders in scoping risk assessments—Lessons from low- carbon transitions. Environmental Innovation and Societal Transitions, 35, 400-413. Vermeulen, S., Campbell, B. M., & Ingram, J. (2012). Climate Change and Food Systems. Annual Review of Environment and Resources, 37, 195-222. Von Bertalanffy, L. (2010). General systems theory. The science of synthesis: exploring the social implications of general systems theory, 103. Wanjiru, I. R., & Xiaoguang, L. (2021). Evaluating University-Community Engagement Through a Community-Based Lens: What Indicators Are Suitable?. Journal of Higher Education Outreach and Engagement, 25(4). Wasserman, S., & Faust, K. (1994). Social Network Analysis: Methods and Kosko, B. (1986). Fuzzy cognitive maps. International Journal of Man-Machine Studies, 24(1), 65-75. Welsh, M. (2014). Resilience and responsibility: governing uncertainty in a complex world. The geographical journal, 180(1), 15-26. Wentworth, C., Hodbod, J., & Gerard, A. (2022). Blurring the boundaries: cross-scale analyses of food systems. Ecology and Society, 27(4). Wentworth, C., Warsaw, P., Isaacs, K., Traore, A., Hammon, A., & Lewis, A. (2023). The resilience and viability of farmers markets in the United States as an alternative food network: case studies from Michigan during the COVID-19 pandemic. Agriculture and Human Values, 40(4), 1481-1496. 125 Wentworth, C., Arroyo, M. T., Lembi, R. C., Feingold, B. J., Freedman, D., Gray, S., ... & Yamoah, O. (2024). Navigating community engagement in participatory modeling of food systems. Environmental Science & Policy, 152, 103645. White, R., Stewart, B., & O’Neill, P. (2010). Access to Food in a Changing Climate. Environmental Change Institute, School of Geography and the Environment. Wilk, R. (2012). The limits of discipline: Towards interdisciplinary food studies. Physiology & behavior, 107(4), 471-475. Wittman, H. (2011). Food sovereignty: a new rights framework for food and nature?. Environment and Society, 2(1), 87-105. Worstell, J. (2020). Ecological resilience of food systems in response to the COVID-19 crisis. Journal of Agriculture, Food Systems, and Community Development, 9(3), 23-30. Xu, Z. (2000). On consistency of the weighted geometric mean complex judgement matrix in AHP. European journal of operational research, 126(3), 683-687. Yi, S. S., Ali, S. H., Russo, R. G., Foster, V., Radee, A., Chong, S., ... & Mezzacca, T. A. (2022). COVID-19 leads to dramatic changes in the food retail environment in New York City: May–July 2020. Journal of Immigrant and Minority Health, 1-7. Yuan, M. H., Chiueh, P. T., & Lo, S. L. (2021). Measuring urban food-energy-water nexus sustainability: Finding solutions for cities. Science of The Total Environment, 752, 141954. Yue, M., & Kahn, G. (2019). A crash course on driving in the rain in L.A. https://xtown.la/2019/03/19/a-crash-course-on-driving-in-the-rain-in-la/ Zhong, Q., Wang, L., & Cui, S. (2021). Urban food systems: a bibliometric review from 1991 to 2020. Foods, 10(3), 662. Zhou, Z., Shi, Q., & Chu, H. (2020). A comprehensive review on Root Mean Square Error (RMSE) in modeling: Recent advances and applications. Applied Energy, 269, 115120. Zurek, M., Hebinck, A., Leip, A., Vervoort, J., Kuiper, M., Garrone, M., ... & Achterbosch, T. (2018). Assessing sustainable food and nutrition security of the EU food system—an integrated approach. Sustainability, 10(11), 4271 Zurek, M., Ingram, J., Sanderson Bellamy, A., Goold, C., Lyon, C., Alexander, P., ... & Withers, P. J. (2022). Food system resilience: concepts, issues, and challenges. Annual Review of Environment and Resources, 47(1), 511-534. 126 APPENDIX A: FOOD SECURITY DIMENTIONS Table A-1. Explanation of each food security dimension and the goals that have been defined for them Availability Access Utilization Stability Agency Sustainability Six Dimensions of Food Security Having a quantity and quality of food sufficient to satisfy the dietary needs of individuals, free from adverse substances and acceptable within a given culture, supplied through domestic production or imports. Having personal or household financial means to acquire food for an adequate diet at a level to ensure that satisfaction of other basic needs are not threatened or compromised; and that adequate food is accessible to everyone, including vulnerable individuals and groups. Having an adequate diet, clean water, sanitation, and health care to reach a state of nutritional well-being where all physiological needs are met. Having the ability to ensure food security in the event of sudden shocks (e.g., an economic, health, conflict or climatic crisis) or cyclical events (e.g., seasonal food insecurity). Individuals or groups having the capacity to act independently to make choices about what they eat, the foods they produce, how that food is produced, processed, and distributed, and to engage in policy processes that shape food systems. Food system practices that contribute to long-term regeneration of natural, social and economic systems, ensuring the food needs of the present generations are met without compromising the food needs of future generations. 127 APPENDIX B: SCENARIO ANALYSIS FOR THE CI MODEL The collective intelligence (CI) model created through FLPP (see Figure B-1) was used as the basis for running “what-if” scenarios for evaluating the impacts of alternatives on Community Identified Values (CIVs) and other system components. For the scenario analysis, the CI model adjacency matrix was loaded in Mental Modeler online tool (Gray et al., 2013). For all the analysis, “hyperbolic tangent” squishing function was selected. Figure B-1. Collective intelligence model generated from 51 FCM interviews with Flint food system experts, adapted from Knox et al., 2023 B-1-Alternatives for COVID-19 To cope with COVID-19, two alternatives were considered: 1) establishing a new open-air farmers market, and 2) opening a food hub with online marketing. Both the "farmers market and "food hub" were one of the system’s components within the CI model; their relationships with other system components in the Flint Food System were mentioned during data collection through fuzzy cognitive maps. Consequently, no modification to the CI model was required. For the analysis evaluating the open-air farmers market, the concepts "Farmers Market," "Seasonality," "Healthy Food Choices," and "Knowledge of How Food Is Grown/Produced" in the 128 CI model were set to one (+1). Figure B-2 illustrates the impacts of this alternative on all system components in the Flint Food System. Similarly, to evaluate the food hub with online marketing, the concepts "Flint Fresh + Food Hub" and "Food Delivery" were set to one (+1). Figure B-3 shows the impacts of this alternative on all the system components in the Flint Food System. Figure B-2. FCM scenario results of increasing "Farmers Market," "Seasonality," "Healthy Food Choices," and "Knowledge of How Food Is Grown/Produced" 129 Figure B-3. FCM scenario results of increasing "Flint Fresh + Food Hub" and "Food Delivery" B-2-Alternatives for Extreme Weather Events To address heat waves and severe precipitation affecting Flint community members' food accessibility, two alternatives were proposed: 1) the construction of 100 new bus shelters and 2) the upgrading of 10 convenience stores to offer healthy, locally sourced fresh products. Unlike the COVID-19 alternatives, where related system components were already included in the CI model, there were no existing concepts for bus shelters or healthy convenience stores. For the bus shelters, we assumed that "Access to Transportation," an existing CI model concept, would improve with the construction of new bus shelters and hence was set to one (+1). Figure B-4 demonstrates the impacts of this alternative on all system components in the CI model for the Flint Food System. While the "convenience store" concept existed in the CI model, it did not capture the dynamics of a "healthy convenience store" as described in the main manuscript section 3-2. Therefore, modifications were made to the convenience store concept and its interactions with other system components within the CI model. Table B-1 represents all these changes. Even with conservative estimates for new interactions and strengths, an increase in various concepts were 130 observed by setting the “healthy convenience store” to one (+1). Figure B-5 demonstrates the impacts of this alternative on all system components in the CI model for the Flint Food System. Table B-1. Modified CI model connection strengths for healthy convenience store scenario Edge: Convenience Stores to Nutritious Foods Convenience Stores to Availability Convenience Stores to Affordability Convenience Stores to Availability of Cultural Foods Convenience Stores to Building Community Convenience Stores to Gardening + Local Agriculture Original Strength: New Strength: -0.6 -0.12 -0.57 0 0 0 +0.2 +0.1 +0.2 +0.1 +0.1 +0.1 131 Figure B-4. FCM scenario results of increasing “Access to Transportation” Figure B-5. FCM scenario results of modified CI model based on Table B-1 132 BIBLIOGRAPHY Gray, S. A., Gray, S., Cox, L. J., & Henly-Shepard, S. (2013, January). Mental modeler: a fuzzy- logic cognitive mapping modeling tool for adaptive environmental management. In 2013 46th Hawaii International Conference on System Sciences (pp. 965-973). IEEE. Knox, C. B., Gray, S., Zareei, M., Wentworth, C., Aminpour, P., Wallace, R. V., ... & Brugnone, N. (2023). Modeling complex problems by harnessing the collective intelligence of local experts: New approaches in fuzzy cognitive mapping. Collective Intelligence, 2(4), 26339137231203582. 133 APPENDIX C: INPUT FOR THE BCA ANALYSIS C-1-Open-Air (Outdoor) Farmers Market This alternative proposes the establishment of a new medium-sized open-air (outdoor) farmers market, with a maximum capacity of 35 vendors (MIFMA, 2017). The market, run by the local government, will offer seasonal services to vendors and customers from mid-April to mid- November (7 months). It will operate in a suitable parking lot, allowing vendors to use the space once a week on Saturdays from 8 am to 3 pm. Vendors will be charged a rental fee and are required to set up their own stalls or booths, payment processing equipment, and other marketing materials. The potential sources of costs, benefits, and externalities have been identified based on available secondary data and preliminary findings from interviews with a limited number of local farmers market directors. Table C-1 summarizes these values for the best-case, mean-case, and worst-case scenarios. Table C-1. Summarized costs, benefits, and externalities for the open-air farmers market alternative Item Rent for the site Initial Supplies and Miscellaneous Market Management and Staffing Licenses and Permits Insurance Utility and Services Marketing and Promotions Revenue from Vendor Fees Health Benefits Spillover Effect on Nearby Businesses Reduction in Losses of Closed Food Outlets Reduction in Food Insecure People t e k r a M s r e m r a F r i A n e p O Classification Cost Cost Cost Cost Cost Cost Cost Best Case $3,000 Value Mean Case $15,000 Worst Case $24,000 $3,500 $6,000 $9,000 $80,000 $100,000 $125,000 $95 $150 $255 $1,000 $2,000 $3,000 $1,750 $4,200 $7,000 $4,500 $5,400 $6,000 Benefit $78,750 $55,500 $26,250 Externality $50,000 $35,000 $20,000 Externality $66,570 $57,500 $44,380 Externality $252,000 $210,000 $170,000 Externality $62,500 $50,000 $37,500 Fatalities Averted Human Life 10 8 6 134 C-1-1-Costs Rent for the site: If the market is set up in rented parking lots, there might be a rental fee. This cost can vary greatly based on location and size but could range from $100 to $800 per market day (estimated based on the interviews). Regarding the operation time of 7 months per year, it can range from $3,000 to $24,000 per year. Initial Supplies and Miscellaneous: Includes costs for items like tables, chairs, tents, and signage for each market day. This might range from $3500 to $9000. [Every 10 years needs to be renewed] (estimated based on the interviews). Market Management and Staffing: Staff costs for setting up, managing, and dismantling the market, along with administrative tasks. This could be around $80,000 to $125,000 per year, depending on the number of staff and hours worked (MIFMA, 2017). Licenses and Permits: Costs for necessary permits and licenses for outdoor farmers market can vary but might be around $95 to $255 (State of Michigan, 2023) Insurance: Liability insurance is essential for protecting against accidents or damage. This could cost approximately $1000 to $2000 per year (estimated based on the interviews). Utility and Services: This includes costs for electricity, water, and waste management. For a market operating once a week, this might range from $50 to $200 per market day, depending on the services needed and the length of the market (estimated based on the interviews). Marketing and Promotions: This is vital for attracting both vendors and customers. Costs for digital and print advertising, social media promotion, and signage could range from $150 to $200 per market day (estimated based on the interviews). C-1-2-Benefits Revenue from Vendor Fees: Vendors might pay a fee to participate in the market, which can range from $25 to $75 per day or more, depending on the market size and location. Based on the assumption of 35 vendors during the operation of outdoor farmers market, it can provide $26,250 to $78,750 per year (MIFMA, 2017; NASS, 2017). C-1-3-Externalities Health Benefits: Shopping at farmers markets is linked to higher fruit and vegetable intake, suggesting that these markets are an effective strategy for enhancing overall consumption of produce in the population (Pitts et al., 2014). Increased fruit and vegetable consumption can lead to better community health outcomes. Quantifying this in dollar terms is complex, but reduced 135 healthcare costs due to improved diets could average from $100 to $250 per person annually. It has been assumed $20,000 to $50,000 for 200 customers per year. Spillover Effect on Nearby Businesses: Farmers' markets not only attract customers to their stalls but also encourage these visitors to spend money at other local businesses (Morckel & Colasanti et al., 2018; Sadler et al., 2013). Regarding the Morckel & Colasanti (2018) study, people who visit the Flint indoor farmers' market typically spend an extra $6.34 at nearby shops and restaurants during their market visit. Considering this, our analysis assumes that 20% of all visitors to the market throughout the year, totaling 45,000 people, will spend this additional amount in the surrounding area. By multiplying 20% of the 35,000 to 52500 annual visitors and by the average extra spending of $6.34, we estimate that the farmers' market could generate an additional $44,380 to $66,570 per year for local businesses. Reduction in Losses of Closed Food Outlets: By selling directly to consumers, farmers can retain a higher percentage of the profit (Hughes et al., 2022, Park et al., 2014). This might increase farmer incomes by around 15-30%, potentially adding up to an extra $5,000-$10,000 per farmer annually. By assuming 35 vendors could continue their business during the COVID-19 and making $160 -$240 per market day, it could prevent a loss of $170,000 to $252,000. Fatalities Averted: During the COVID-19 pandemic, Flint witnessed the loss of approximately 400 lives. Assuming a 1.5% to 2.5% reduction in the number of fatalities due to the lower transmission rate of the virus in open-air spaces, this would imply that engaging more in outdoor activities or having more facilities that operate outdoors could have potentially saved between 6 to 10 individuals. C-2-Food Hub with Delivery Services The second alternative represents the establishment of a new medium-large food hub with a 15,000 square foot space (report 2021 food hub). This food hub employs 10 full-time and 5 part- time staff members. Designed to operate without a traditional storefront, this hub will instead focus on an online marketplace for both fresh and processed food items, including a variety of fruits, vegetables, meat, dairy, and bread, along with specialty products like fruit jams and dried goods. The platform will deliver the products to the individual consumers, while also providing a wholesale distribution channel to serve restaurants, schools, and other organizations. By combining the convenience of online shopping with a comprehensive product range, the food hub 136 aims to support local agriculture, enhance food accessibility, and meet the needs of diverse customers. The potential estimations of costs, benefits, and externalities have been identified based on available reports and preliminary findings from interviews with a limited number of local fresh market directors. Table C-2 summarizes these values for the best-case, mean-case, and worst-case scenarios. C-2-1-Costs Purchasing Property: Initial investment for purchasing a property of 15,000 square foot for the food hub has been estimated between $750,000 and $1,500,000 based on the average price for industrial spaces in Michigan. Equipment and Initial Setup: Depending on the food hub’s operational needs for processing, packaging, and refrigerating the products, the estimated investment for equipment and initial setup for a food hub with 15,000 square foot range between $250,000 and $500,000 (estimated based on the interviews). Staffing: Assuming 10 full-time job and 5 part-time job, the yearly payroll can be estimated between $635,000 and $950,000 (CRFS, 2023) Utility and Services: Monthly utilities and service expenses might range from $2,000 to $5,000. Raw Materials: Costs for sourcing raw materials such as fruits, vegetables, meats, eggs, and bread are projected to be $10,000 to $30,000 per month (CRFS, 2023; CRFS, 2020). Packaging: Costs for packaging materials, including boxes, jars, and other necessary packaging for processed food items, are estimated to range from $3,000 to $6,000 monthly (estimated based on the interviews). Marketing and Promotions: To effectively market the online marketplace and attract both individual and wholesale customers, monthly expenses for digital marketing and website operations are anticipated to be $1,000 to $3,000 (estimated based on the interviews). Licenses and Permits: Costs for necessary permits and licenses for food hub that offers delivery services and include food processing can vary from $2,000 to $5,000 (State of Michigan, 2023) 137 Insurance: It has been estimated that the food hub would need to budget $3,500 to $5,500 annually for liability and property insurance to protect against operational risks (estimated based on the interviews). Table C-2. Summarized costs, benefits, and externalities for the food hub with delivery services alternative Item Classification s e c i v r e S y r e v i l e D h t i w b u H d o o F Purchasing Property Equipment and Initial Setup Staffing Utility and Services 2 Raw Materials Packaging Marketing and Promotions 2 Licenses and Permits 2 Insurance 2 Revenue from Online Sales Revenue from Wholesale Non-Sale Revenue Job Creation Enhanced Income of Small Farm Businesses Waste Management Reduced losses of Increased prices Fatalities Averted Best Case $750,000 Value Mean Case Worst Case $1,100,000 $1,500,000 $250,000 $350,000 $500,000 $210,000 $300,000 $510,000 $24,000 $33,000 $60,000 $220,000 $450,000 $660,000 $18,000 $24,000 $36,000 $12,000 $22,000 $36,000 $2,000 $3,500 $4,000 $4,200 $5,000 $5,500 Cost Cost Cost Cost Cost Cost Cost Cost Cost Benefit Benefit Benefit $420,000 $360,000 $270,000 $576,000 $480,000 $168,000 $100,000 $75,000 $30,000 Externality $510,000 $300,000 $210,000 Externality $700,000 $500,000 $300,000 Externality $35,000 $25,000 $15,000 Externality $750,000 $693,530 $500,000 Human Life 12 10 8 C-2-2-Benefits Revenue from Online Sales: Based on an average order of $40 and receiving between 250 to 400 orders per month, the annual sales can range from $120,000 to $192,000 (CRFS, 2023). Revenue from Wholesale: For wholesale, with the food hub having between 7 to 12 clients making average purchases of $2,000 to $4,000 per month, the annual revenue can range from $168,000 to $576,000 (CRFS, 2023). Non-Sale Revenue: For this alternative, the food hub might make between $30,000 and $100,000 a year from non-sale revenues. This extra money could come from diverse sources, such as federal and state government grants, contributions from individuals, and funds from private 138 foundations. This reflects findings from a 2021 report on food hubs, which indicated that nearly two-thirds of such hubs benefitted from similar non-sales income streams (CRFS 2023, CRFS, 2020). C-2-3-Externalities Job Creation: This food hub is able to create 10 full-time and 5 part-time job opportunities for the local community. The salaries for employees can range from $36,210 to $47,500 and for the managers from $47,843 to $64,827 (CRFS, 2023) Enhanced income of small farm businesses: Food hubs usually source their products from local/regional small farm businesses that cannot compete with larger producers due to various constraints, such as limited access to markets or insufficient volume. The food hub can facilitate the aggregation, distribution, and marketing of food products, thereby providing small farm businesses with a vital link to larger markets. Assuming the food hub collaborates with 40 local/regional small-farm businesses per year, a meaningful estimation for the purchasing amount from them could be in the range of $300,000 to $700,000 per year (estimated based on the interviews). Waste Management: Annually, a significant quantity of fresh products, such as fruits and vegetables, is wasted due to the lengthy supply chains at the national level. Food hubs that engage in close collaboration with local or regional producers can benefit from shorter supply chains, leading to a reduction in the waste rate of fresh produce. By achieving a 5% reduction (Assumed) in waste from fresh produce and assuming annual purchases from producers amounting to between $300,000 and $700,000, this efficiency could translate into savings ranging from $15,000 to $35,000. Fatalities Averted: During the COVID-19 pandemic, Flint witnessed the loss of approximately 400 lives. Assuming a 2% to 3% reduction in the number of fatalities due to the lower transmission rate of the virus as people could benefit from online marketing, this would imply that leveraging online marketing strategies to promote social distancing and reduce physical interactions could have potentially saved between 8 to 12 individuals. 139 C-3-Bus Shelters C-3-1-Costs Construction: Costs of bus shelters are highly variable, with basic models ranging from $10,000-$12,000 (Wesoff, 2011) and higher-end shelters with features like climate control costing $40,000 (Mohl, 2019). For the purposes of this analysis, we will assume a lower cost of $10,000 for a shelter with benches, a roof, and walls ($10,000/shelters * 100 shelters = $1,000,000 for Lifespan: 30 years) Table C-3. Summarized costs, benefits, and externalities for the bus shelter alternative Item Construction Maintenance Classification Best Case Cost Cost Value Mean Case $1,000,000 $100,000 Public Transport Ridership Benefit $448 $265 HRI Cases Food Waste Vehicle Accidents Fatalities Averted Externality Externality Externality $6,137 $1,614 $1,036 $389 $16,903 $10,866 $6,037 Human Life 0.01975 0.007 0.00143 Worst Case $130 $306 $130 s r e t l e h S s u B Maintenance: Yearly maintenance: 20 hours of maintenance per year (Mohl, 2019) * hourly wage: $25/hour (Assumption) * 100 shelters + maintenance materials: $500 (Assumption) * 100 shelters = $100,000/year C-3-2-Benefits Increase in Ridership: During heatwaves: Increase in ridership for stops with shelters, during high temperatures: 0.275% (Miao et al., 2019) * Daily Ridership: 5270 (MTA, 2022) * Ridership that benefits from shelters: 20%-40% (Assumption) * Local Fare Price: $1.75 (MTA, 2022) * Duration of heatwave: 3 days = $15.22-$30.43 (or ~9-17 riders) per heatwave. Lost Ridership in Heatwave: $99 * Ridership that benefits from shelters: 20%-40% (Assumption) = $19.80-$39.60 (or ~11-22 riders) per heatwave. During rain events: Increase in ridership for stops with shelters, during rain: 0.107% (Miao et al., 2019) * Daily Ridership: 5270 (MTA, 2022) * Ridership that benefits from shelters: 20%-40% (Assumption) * Local Fare Price: $1.75 (MTA, 2022) = $1.97-$3.95 (or ~1-3 riders) per storm. Lost Ridership in Rain: $50 (see 2.1.2) * Ridership that benefits from shelters: 20%-40% (Assumption) = $10-$20 (or ~6-12 riders) per storm 140 C-3-3-Externalities Percentage Reductions in HRI cases and food loss/waste: Costs of HRI cases: $15,321- $51,141, mean: $26,895 * 1%-3% = $153-$1,534/heatwave. Mortality from HRI: 0.057-0.1587 * 1%-3% = 0.00057-0.004761/heatwave. Costs of food loss/waste: $12,953 * 0.5%-2% = $65- $259/heatwave Avoided Crashes: Bus travel can be 40 times safer than auto travel (SWRPC, 2001) and using the assumption that the ridership increases from shelters replaced a car trip, there is a marginal decrease in the number of car accidents. Ridership increase: ~7-15. Decrease in the number of car accidents: 0.091%-0.182%. Cost of Avoided Crashes: $1,231-$2,647/storm. Avoided Fatalities: 0.0000594-0.000128/storm C-4-Healthy Convenience Stores This alternative proposes transforming 10 existing convenience stores in different neighborhoods with limited access to fresh markets into "healthy convenience stores". This initiative is designed to enhance infrastructure and facilities within these stores to offer a wide range of fresh local products, including fruits, vegetables, meat, and dairy ensuring higher diversity of choice and affordability. By collaborating with local producers, these health convenience stores will provide consumers--especially the ones without personal vehicles--easy access to healthy food options in their neighborhood, promoting better dietary habits and reducing food deserts. Moreover, shorter distance for purchasing groceries can enhance their accessibility during extreme weather events. 141 Table C-4. Summarized costs, benefits, and externalities for the healthy convenience store alternative Item Classification Infrastructure Upgrades Training and Development Inventory and Sourcing Marketing and Community Engagement Licensing and Compliance Increased Sales Cost Cost Cost Cost Cost Best Case $500,000 Value Mean Case Worst Case $750,000 $1,000,000 $10,000 $15,000 $20,000 $250,000 $350,000 $450,000 $24,000 $33,000 $60,000 $20,000 $35,000 $50,000 Benefit $400,000 $300,000 $200,000 Strengthened Local Economy Externality $400,000 $300,000 $200,000 Health and Nutrition Externality $250,000 $175,000 $100,000 Saving Energy HRI Cases Fatalities Averted Externality $4,800 $3,600 $2,400 Externality $32,272 $26,894 $21,516 Human Life 0.068 0.056 0.044 e r o t S e c n e i n e v n o C y h t l a e H C-4-1-Costs Infrastructure Upgrades: The initial investment for upgrading infrastructure (refrigeration, shelving, display units) is estimated between $50,000 and $100,000 per store, totaling $500,000 to $1,000,000 for all 10 stores (based on the available online prices). Training and Development: Costs for training staff in handling and marketing fresh products might range from $10,000 to $20,000 (Assumption). Inventory and Sourcing: Average costs for sourcing a diverse range of products from the 40 local producers are projected to be $20,000 to $40,000 per store (Qin et al., 2014) Marketing and Community Engagement: Expenses to market the new offerings and engage the community are anticipated to be $5,000 to $10,000 per year (Assumption). Licensing and Compliance: Updating licenses and ensuring compliance with food safety standards could require $2,000 to $5,000 per store, or $20,000 to $50,000 in total (Assumption). C-4-2-Benefits Increased Sales: By offering a wider range of fresh and healthy options, sales per store could increase, potentially adding $30,000 to $50,000 in annual revenue per store. For all 10 healthy convenience stores can range from $300,000 to $500,000 per year (Memphis MPO, 2014). 142 C-4-3-Externalities Strengthened Local Economy: Collaborating with 40 local producers on average can inject approximately $300,000 to $700,000 annually into the local agricultural economy (Miller & McCole, 2014). Health and Nutrition: Enhanced access to local fresh food can improve community health outcomes, potentially reducing healthcare costs in the neighborhood. Quantifying this in dollar terms is complex, but reduced healthcare costs due to improved diets could average $100-$250 per person annually. (Assumed For 1000 customers) Saving Energy: By enhancing the availability of diverse fresh local products in neighborhoods, this alternative has the potential to significantly reduce travel distances for customers. Assuming it can shorten the travel for 5,000 customers per year from an average of 5- 10 miles to 1-2 miles, and considering an average fuel economy of 25 miles per gallon with the price of gas at $3.50 per gallon, it can save $4200 per year. Reduction in HRI: By reducing the travel distance for customers, people need to spend less time in extreme heat weather and it can considerably reduce the heat-related illnesses (HRI) by 20%-30%. (CDC, 2022). Costs of HRI cases: $15,321-$51,141, mean: $26,895 * 30%-40% = $5379-$8,068/heatwave. Mortality from HRI: 0.057-0.1587* 30%-40% = 0.011-0.017/heatwave. 143 BIBLIOGRAPHY Center for regional food systems (CRFS). (2023) 2021 National Food Hub Survey Report. Retrieved from: https://www.canr.msu.edu/foodsystems/uploads/MSU-007-CRFS-2021- National-Food-Hub-Survey-Report-Final1.pdf Center for regional food systems (CRFS). (2020) 2019 National Food Hub Survey Report. Retrieved from: https://www.canr.msu.edu/foodsystems/uploads/files/Findings-of-the- 2019-National-Food-Hub-Survey-Report.pdf Hughes, M., Pressman, A., Oberholtzer, L., Dimitri, C., & Welsh, R. (2022). Selling to Local and Regional Markets: Barriers and Opportunities for Beginning Farmers. National Center for Appropriate Technology. IP595. Michigan Farmers Market Association (MIFMA). (2017). Market Manager Compensation Report 2017. Retrieved from https://mifma.org/wp-content/uploads/2020/07/MIFMA- Market-Manager-Compensation-Report-2017.pdf Miao, Q., Welch, E. W., & Sriraj, P. S. (2019). Extreme weather, public transport ridership and moderating effect of bus stop shelters. Journal of Transport Geography, 74, 125-133. https://doi.org/https://doi.org/10.1016/j.jtrangeo.2018.11.007 Miller, C. L., & McCole, D. (2014). Understanding collaboration among farmers and farmers' market managers in southeast Michigan (USA). Journal of Agriculture, Food Systems, and Community Development, 4(4), 71-95. Memphis MPO. (2014). YMCA Healthy Convenience Store Plan. Retrieved from: https://memphismpo.org/sites/default/files/documents/resources/projects/greenprint/ymca -healthy-convenience-store-plan.pdf Mohl, B. (2019). T notes: Bus shelter maintenance pricey. https://commonwealthmagazine.org/transportation/t-notes-bus-shelter-maintenance- pricey/ Morckel, V., & Colasanti, K. (2018). Can farmers’ markets in shrinking cities contribute to economic development? A case study from Flint, Michigan. Sustainability, 10(6), 1714. MTA. (2022). Financial Report with Supplemental Information. In. National Agricultural Statistics Services (NASS). (2020) National Farmers Market Mangers Report. Retrieved from: https://www.nass.usda.gov/Publications/Todays_Reports/reports/nfar0820.pdf 144 Park, T., Mishra, A. K., & Wozniak, S. J. (2014). Do farm operators benefit from direct to consumer marketing strategies?. Agricultural Economics, 45(2), 213-224. Pitts, S. B. J., Gustafson, A., Wu, Q., Mayo, M. L., Ward, R. K., McGuirt, J. T., ... & Ammerman, A. S. (2014). Farmers’ market use is associated with fruit and vegetable consumption in diverse southern rural communities. Nutrition journal, 13, 1-11. Qin, Y., Wang, J., & Wei, C. (2014). Joint pricing and inventory control for fresh produce and foods with quality and physical quantity deteriorating simultaneously. International Journal of Production Economics, 152, 42-48. Sadler, R. C., Clark, M. A., & Gilliland, J. A. (2013). An economic impact comparative analysis of farmers’ markets in Michigan and Ontario. Journal of Agriculture, Food Systems, and Community Development, 3(3), 61. State of Michigan. (2023). Farmers market FAQ. Retrieved from: https://www.michigan.gov/mdard/food-dairy/farmersmarket/farmers-market-faq SWRPC. (2001). CityExpress Cost/Benefit Analysis. Wesoff, E. (2011). Solar Bus Shelters From GoGreenSolar. https://www.greentechmedia.com/articles/read/solar-bus-shelters-from- gogreensolar#:~:text=A%20traditional%20bus%20stop%20costs,traditional%20bus%20s helter%20without%20solar. 145 APPENDIX D: SENSITIVITY ANALYSIS OF DR AND PH As outlined in Section 4-5-5-1, we performed a sensitivity analysis to examine the effects of variations in the Discount Rate (DR) and Planning Horizon (PH) on the final scores and rankings of the candidate alternatives. This appendix investigates the influence of these parameters across worst-case, mean-case, and best-case scenarios in a more detailed manner. D-1-Worst-Case Scenario Figure D-1 illustrates the heatmaps for the final scores of the four candidate alternatives under various DR and PH within the worst-case scenario. A clear trend emerges from the data: extending the Planning Horizon generally improves the final scores of all alternatives. Conversely, the Discount Rate exerts a less pronounced effect on the final scores. Specifically, the Open-air Farmers Market (a) and the Bus Shelter (c) show negligible sensitivity to changes in the Discount Rate. In contrast, the Food Hub with Online Marketing (b) and the Healthy Convenience Store (d) exhibit a mild sensitivity, where a lower Discount Rate correlates with slightly higher final scores. Figure D-1. Heatmap for the final scores of each alternative candidate regarding different discount rates and planning horizons for the worst-case scenario. 146 To assess the impact of variations in DR and PH on the ranking of alternatives, box plots were utilized to visualize the range of final scores. As depicted in Figure D-2, the distinct and non- overlapping box plots for each alternative reinforce the stability of their rankings. This indicates that the variations in DR and PH do not influence the relative positioning of these alternatives. Figure D-2. Final score ranges for each alternative based on the variation on DR and HP for the worst-case scenario D-2-Mean-Case Scenario Figure A3-3 showcases the heatmaps of final scores for the four alternatives in the context of various DR and PH as part of the mean-case scenario analysis. In contrast to the patterns observed in the worst-case scenario, the data from the mean-case scenario does not reveal a uniform trend. Specifically, for the Open-air Farmers Market (a), the Food Hub with Online Marketing (b), and the Healthy Convenience Store (d), an extended PH corresponds with a decline in final scores. Conversely, for the Bus Shelter (c), an increase in PH continues to result in improved final scores. This divergence in trends could potentially be attributed to the Benefit-Cost Ratio (BCR) for alternatives a, b, and c transitioning from below one to above one, signifying a shift to a more favorable outcome. However, for alternative d, the BCR remained below one. The impact of DR on the final scores was comparatively minor. A varied response was noted where for the Open-air Farmers Market (a), a higher DR was associated with higher final scores, yet for the 147 Food Hub with Online Marketing (b), Bus Shelter (c), and Healthy Convenience Store (d), an increased DR correlated with decreased scores. Figure D-3. Heatmap for the final scores of each alternative candidate regarding different discount rates and planning horizons for the mean-case scenario. To assess the impact of variations in DR and PH on the ranking of alternatives, Figure D- 4 represents the distinct and non-overlapping box plots for each alternative in the mean case scenario. This suggests that the variations in DR and PH do not influence the relative positioning of these alternatives in the mean-case scenario, similar to the worst-case scenario. 148 Figure D-4. Final score ranges for each alternative based on the variation on DR and HP for the mean-case scenario D-3-Mean-Case Scenario Figure D-5 showcases the heatmaps of final scores for the four alternatives in the context of various DR and PH as part of the best-case scenario analysis. Similar to the trends for the mean-case scenario, for the Open-air Farmers Market (a), the Food Hub with Online Marketing (b), and the Healthy Convenience Store (d), an extended PH corresponds with a decline in final scores. Conversely, for the Bus Shelter (c), an increase in PH continues to result in improved final scores—as it has the BCR below 1 even for the best-case scenario. The impact of DR on the final scores was inconsiderable again. A varied response was noted where for the Open-air Farmers Market (a), a higher DR was associated with higher final scores, yet for the Food Hub with Online Marketing (b), Bus Shelter (c), and Healthy Convenience Store (d), an increased DR correlated with decreased scores. Figure D-6 demonstrates the distinct and non-overlapping box plots for each alternative in the best-case scenario. This suggests that the variations in DR and PH do not influence the relative positioning of these alternatives in the best-case scenario, same as the mean-case and worst-case scenarios. It concludes that DR and HP have not impacted the ranking of the alternatives. 149 Figure D-5. Heatmap for the final scores of each alternative candidate regarding different discount rates and planning horizons for the best-case scenario. Figure D-6. Final score ranges for each alternative based on the variation on DR and HP for the best-case scenario 150