BREAKING DOWN BARRIERS TO FOOD ACCESS: AN EVALUABILITY ASSESSMENT OF A MOBILE FARMERS MARKET By Angel Hammon A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Community Sustainability – Master of Science 2023 ABSTRACT Food insecurity is a growing problem in the United States. Approaches have been in place for decades to address it, but food insecurity rates show little change. Federal food assistance programs aim to reduce food insecurity by providing funding and nutrition education to low-income consumers that meet eligibility requirements. Further, work has been done to encourage use of these programs at farmers markets so that low-income consumers can access local products and other benefits like the ability to buy directly from farmers. However, the literature shows that persistent barriers related to transportation, convenience, price, exclusivity, and administrative burden prevent low-income customers from visiting farmers markets. Mobile farmers markets attempt to alleviate these barriers by bringing the farmers market to the customer, thereby increasing healthy food access and food security for vulnerable populations. Through a partnership with a nonprofit mobile farmers market in Michigan, this thesis will evaluate the effectiveness of this particular mobile farmers market toward these ends. Using participant observations, staff and customer interviews, and document analysis, we conducted an evaluability assessment to: (1) provide useful evaluation information about the customer experience to the partner organization, and (2) investigate if and how the mobile market provides traditional farmers market benefits while overcoming documented farmers market barriers for low-income consumers. Chapter 1 will detail the assessment of barriers, benefits, and connections to local food pathways to contribute to literature on food insecurity and food assistance. Chapter 2 will detail the use of evaluability assessment to contribute to staff capacity building, allay problematic organizational dynamics, and improve evaluation use. ACKNOWLEDGEMENTS This thesis was made possible by a partnership with a local nonprofit organization, their mobile farmers market staff, and the customers they serve. I could not thank them enough for sharing their struggles, celebrations, and family recipes with me. I have learned as a person and a researcher from these folks over plentiful baskets of peppers and collard greens. I thank my advisor, Dr. Lissy Goralnik for teaching me how to navigate the research process while maintaining passion for the subjects I love. This work has been reimagined, shaped, edited, and proofread countless times at ungodly hours and it is something I am proud to have crafting alongside her. I will continue in my professional career with a positive attitude for life and a color-coded chart in my toolbox. I am thankful for the guidance of many other academics along the way, particularly my committee members, Dr. Kimberly Chung and Dr. Katherine Alaimo, for providing academic expertise and support through the many iterations of this project. Thank you to Dr. McNall, who has been a great sounding board for evaluation and community engagement ideas. I have also learned so much about perspective and possibility from many others in Community Sustainability and I am so happy I found a home in this department. My time here is characterized by laughter and growth I shared with the CSUS 2021 cohort as well as the hardships we overcame together. Especially, Rafael Lembi and Ellie Shiappa from/with whom I learned so much about community-engaged research and dive bar trivia. And finally, to my friends, family, and partner who provided unending support that the distance could not hinder. iii TABLE OF CONTENTS INTRODUCTION ...........................................................................................................................1 CHAPTER 1: TRADEOFFS BETWEEN TRADITIONAL AND MOBILE FARMERS MARKETS ......................................................................................................................................3 CHAPTER 2: AN EVALUABILITY ASSESSMENT OF A MOBILE FARMERS MARKET.......................................................................................................................................31 BIBLIOGRAPHY ..........................................................................................................................44 APPENDIX A: CUSTOMER INTERVIEW GUIDE ...................................................................49 APPENDIX B: STAFF INTERVIEW GUIDE .............................................................................51 APPENDIX C: THEORY OF CHANGE ......................................................................................52 APPENDIX D: EXECUTIVE DIRECTOR LOGIC MODEL ......................................................53 APPENDIX E: EVALUATION REPORT ....................................................................................54 APPENDIX F: STAFF LOGIC MODEL ......................................................................................69 iv INTRODUCTION This thesis is a collaboration between the primary researcher and a local nonprofit in Michigan that runs a mobile farmers market (MFM). The goal of the collaboration was twofold: (1) to evaluate the effectiveness and impacts of the MFM for its targeted consumers, and (2) compare the tradeoffs between traditional and mobile farmers market to better understand if and how mobile markets overcome documented limitations of traditional farmers markets, including: transportation, convenience, price, exclusivity, and administrative burden (Alkon & McCullen, 2011; Freedman et al., 2016; Mino et al., 2018; Ritter et al., 2019; Webber et al., 2010). Feeding America estimates that there are 1,150,150 food insecure individuals in Michigan as of 2020 (Feeding America, 2023). This nonprofit is aware of food insecurity in their service area and created programs in the area of food outreach and nutrition education to address it (Northwest Initiative, 2023). They started this program seven years ago to bring healthy food to low-income residents. The program was launched with grant support and has continued to secure external funding to maintain operations. They supplement grant funding with market sales, which allows them to break even with the cost of gas, vehicle maintenance, and insurance. The MFM prioritizes customers that use food assistance programs by providing the opportunity to purchase fresh produce with Electronic Benefit Transfer (EBT), Senior Farmers Market Nutrition Program (SFMNP), WIC Project FRESH, and Double Up Food Bucks (DUFB). Originally, they were set up more like a traditional farmers market in which farmers were invited to each of their sites, however this was taxing for all parties and the market transitioned to using a bus drive around to each site. The mobile market still sources produce from farmers, though no farmers are present at the time of selling. Additionally, when customers requested, the market began to supply non-local produce as well, providing fresh fruits and vegetables that customers preferred and would purchase but would not be available at traditional farmers markets (i.e. bananas). The market stops have varied over the years to assure the program is reaching its target customers. The executive director of the program recognized that cash, credit, and debit were largely used at their stops and has adjusted them since. In 2022, the MFM made four weekly stops on Thursdays from July to October. They have attended sites A, B, and C for over 3 years and these sites consistently show redemption of food assistance. These sites are in the parking lots of senior and low-income apartments. Site D, the fourth stop, was added in the 2022 season 1 and was located at a community center. The market serves mostly seniors (55-years-old and older). Using participant observations at the market and interviews with MFM customers and staff, Chapter 1 of this thesis address will investigate the ability of the MFM to provide benefits associated with traditional farmers markets while also overcoming barriers documented in the literature. In particular, this chapter will explore the potential of the MFM to connect low-income and minority customers with positive market experiences while overcoming challenges to accessing fresh, local produce. This chapter ends with lessons learned for future food access interventions and their contribution to improving food security at a small scale. Chapter 2 will use the same data to detail the process of an evaluability assessment, which is the evaluation approach we used for the MFM. Traditionally this approach is used to increase the usability of the evaluation outcomes and prepare an organization for more involved evaluation in the future. It is often an approach chosen for smaller organizations with limited resources for evaluation. We found that this approach both provided useful outcomes for the organization, while also building internal capacity for effective communication and organizational change. 2 CHAPTER 1: TRADEOFFS BETWEEN TRADITIONAL AND MOBILE FARMERS MARKETS Introduction Over 38 million people in the United States face food insecurity – a disproportionate amount of which are minority communities such as seniors, Latino, Black, and Native American people (Feeding America, 2021). The United States Department of Agriculture (USDA) defines food insecurity as the “limited or uncertain availability of nutritionally adequate and safe foods, or limited or uncertain ability to acquire acceptable foods in socially acceptable ways (USDA ERS, 2022). Although food insecurity is not a new phenomenon in the US, it was exacerbated by COVID-19. Reliance on few major producers to supply food for large populations emphasized the weakness of the conventional food system (Hendrickson, 2020) and highlighted the need for stable connection to the food system. Unlike other countries where conflict and climate-crises are the main drivers of food insecurity (FAO, 2021), in the U.S. systemic barriers are often the cause of low food access. For example, low-income consumers and consumers of color are often relegated to neighborhoods where there are more fast food and convenience options than grocery stores, creating a food environment that leads to higher rates of noncommunicable disease in these communities (Bell et al., 2019; Story et al., 2008). The U.S. has launched a number of policy approaches to reduce food insecurity over the last 80 years, most of which primarily provide low-income consumers with money to spend on food (Lang & Barling, 2012). Nutrition education was included as a priority starting in the late 1970s (USDA FNS, 2022a). More recent approaches to increasing food access have combined funding and nutrition education to provide low-income customers opportunities to use government food assistance to purchase fresh fruit, vegetables and other local items at farmers markets (Freedman et al., 2016; Mino et al., 2018). This is an exciting trend, especially because alternative food pathways like farmers markets were able to maintain food supply with little to no pause during the pandemic by making modifications to care for health and safety of vendors and consumers (Klisch & Soule, 2020). Opening this pathway to healthy food through government assistance thus has the potential to increase food security, even in times of uncertainty. Long before COVID impacted markets and supply chains, consumer research has shown that farmers markets have been used in tandem with conventional grocery stores by some 3 consumers because of perceived value, high quality foods, the ability to buy directly from a farmer, and ecological benefits (Warsaw et al., 2021; Wolf et al., 2005). These qualities present in farmers markets and other alternative food pathways like community supported agriculture (CSA) and community gardening foster personal connections to the food system and food producers through social embeddedness which facilitates trust in consumer-producer relationships (Hinrichs, 2000). However, research shows the benefits of farmers markets are not equitably realized. The primary consumer base for these markets is largely white women, 40- years-old or older, who have some college education (Byker et al., 2012), and the literature documents a number of barriers that explain why resources are underutilized by low-income and minority consumers, including transportation, price and perception of price, and exclusivity (Freedman et al., 2016). Mobile farmers markets offer one approach to minimizing these barriers. For one thing, their mobility decreases or eliminates the transportation barrier by bringing the market to the consumer. Some markets also accept food assistance and/or acquire grant funding to bring down food prices and make produce more affordable to low-income consumers. Mobile markets still have barriers of their own such as advertising and consistent funding (Hsiao et al., 2019), though. There are other tradeoffs that make mobile farmers markets differ from traditional farmers markets as well, for example, some mobile markets do not have farmers on site. As well, some mobile markets do not strictly sell local food. Because research shows that consumers are often drawn to farmers markets for the opportunity to connect with local products and growers, this lack of connection to the local food system maybe a limitation of mobile markets for some consumers (Wolf et al., 2005). Better understanding the differences between traditional and mobile farmers markets can highlight tradeoffs that may provide insight to successful food access intervention. Research Questions RQ1: In what ways does the MFM address the barriers associated with traditional farmers markets (convenience, administrative burden, transportation, price, exclusivity)? RQ2: In what ways does the MFM provide the benefits associated with traditional farmers markets (sense of community, food access, flexible market)? 4 Literature Review Food Assistance Programs as Nutrition Interventions Many interventions have been created to address food insecurity in the United States. Federal food assistance programs are a well-established example, the most common of which is the Supplemental Nutrition Assistance Program (SNAP). SNAP is the evolved policy of what was once called Food Stamps, a program created for the dual benefit of offloading surplus crops and providing for unemployed Americans during World War II (USDA FNS, 2022a). SNAP, which provides grocery benefits to qualifying citizens in the form of an Electronic Benefit Transfer (EBT) card, is now used in stores and markets much like a debit card. Funds are allocated every month to the card and consumers can use it at EBT-accepting vendors of their choice. The Special Supplemental Nutrition Program for Women, Infants, and Children (WIC), another federal food security initiative, is used to support pregnant women, new mothers, and their children up to five years old. The program provides vouchers that can be redeemed for specific products (e.g. one gallon of 2% milk). A similar Special Supplemental Nutrition Program for Seniors aged 60 and older also exists. These programs were created to support communities who are particularly vulnerable to food insecurity (Feeding America, 2021). Food assistance programs in the U.S. are collectively housed under the US Department of Agriculture Food and Nutrition Service, the funding and programming for which are determined by the Farm Bill, which renews about every five years. Eligibility requirements, funding limits, and other logistics are modified with each draft of this program. Currently, requirements to qualify for benefits from these programs include income level, hours worked per week, and citizen status of applicants, among other characteristics. Non-citizens and students, two populations also commonly susceptible to food insecurity (Chilton et al., 2009; Payne-Sturges et al., 2018) have restricted access to these funds (USDA FNS, 2022e), which highlights one limiting factor of food assistance programs that can lead to further marginalization of already vulnerable populations. Beyond financial access to healthy foods, a priority of federal food assistance programs is to supply consumers with knowledge about which products can be obtained with program support. SNAP-Ed, the educational component of this food assistance program , helps participants “shop for and cook healthy meals...[and] help people learn how to make their SNAP dollars stretch” (SNAP-Ed, 2022a). SNAP also allows purchases in the category of “seeds and 5 plants that produce food for human consumption” (USDA FNS, 2022a), which is a useful benefit for those who wish and are able to supplement store-bought produce with gardening. The SNAP- Ed website displays hands-on programs that provide education and is intended to serve as a tool for sharing practical agricultural and nutrition knowledge. Examples of these programs include virtual cooking classes, school gardens, and school food banks. Farmers Markets as Nutrition Interventions in Michigan In Michigan, SNAP-Ed is housed under Michigan State University Extension and the Michigan Fitness Foundation (MFF) with the support of other “statewide partners whose work focuses on improving the health of Michigan’s most vulnerable citizens, including children, seniors, families, and communities in crisis” (SNAP-Ed at Michigan Fitness Foundation, 2022). Many farmers markets across the state accept EBT, which allows consumers in the SNAP program to shop at these markets. The Farmers Market Nutrition Program (FMNP) (USDA FNS, 2022c) allows WIC and senior consumers to redeem vouchers for particular foods up to a specified dollar amount at participating farmers markets (USDA FNS, 2022d). Most markets also have an incentive program that allows consumers who use food assistance benefits to double their buying power at the farmers market, like the Double Up Food Bucks program maintained by the Fair Food Network (Fair Food Network, 2022; USDA FNS, 2022b). These programs typically provide tokens worth double the dollar amount that users pay that day up to a certain maximum purchase. The Michigan Farmers Market Association (MIFMA) and a network of food support agencies were integral to the wide-spread acceptance of food assistance at farmers markets in the state and beyond (Chadderdon Bair, 2022). Now more than 160 farmers markets in Michigan accept EBT payment (Wheaton, 2018), and over 3,000 markets accept EBT nationwide (FNS, 2022). The Michigan Fitness Foundation also funds some markets to participate in the Farmers Market Navigator program, which aims to build awareness about farmers markets, assist customers in using food assistance and other payments at the markets, and provides nutrition education through policy, systems, and environmental change interventions (SNAP-Ed, 2022b). Barriers to Farmers Market Access While the application of these food assistance programs within farmers markets has some beneficial outcomes, there has been little change overall to who attends farmers markets (Warsaw et al., 2021). This is likely due to other barriers to farmers market access that still exist 6 for many low-income consumers, including: (1) transportation, (2) convenience, (3) price (real and perceived), (4) exclusivity, and (5) administrative burden. Transportation The first step required to benefit from a farmers market is getting there, and the inability to actually reach the market presents a common physical barrier for many potential consumers. Ritter et al. (2019) discuss transportation as one of the major deterrents for low-income consumers to shop at the farmers market. Not all consumers have their own vehicle, and many public transportation programs – if available – are limited in their usefulness. For example, some buses don’t run on Sundays, a common market day, many have long wait times or numerous transfers, and often, and/or markets are not located directly on a bus line, so customers still have to get from the bus stop to/from the market carrying their bags (Gusto et al., 2020). Buses also sometimes limit the number of bags or packages riders can carry with them, and for many consumers, it is inconvenient or impractical to carry numerous bags to the bus and then home from the stop (Ritter et al., 2019). These travel limitations can lead consumers away from the farmers market and to a different venue to spend their food assistance benefits. Even in cases like WIC or Senior FMNP, where some vouchers can only be redeemed for locally produced goods, the time and energy it takes to navigate to the market and back is not worth it for some consumers, in which case the vouchers often go unused. Convenience Webber et al. (2010) assessed low-income shoppers’ motivations to buy fresh fruits and vegetables in upstate New York and found that farmers markets often proved to be inconvenient for low-income shoppers compared to grocery stores (especially chain stores) by every indicator of convenience. In the study, customers defined convenience as being in physical proximity to where one lives or works given transportation, compatibility of schedules, and logistics (e.g. what items are needed in conjunction with other items on their agenda, or multi-tasking). While physical proximity was primarily discussed in terms of transportation, it is also important to note that there are many more grocery stores than farmers markets, reducing limitations by increasing the number of locations, as well as a much broader range of hours for shopping. While some grocery stores are open 24 hours a day, farmers markets are frequently open seasonally and only one or two days of the week. Many low-income shoppers need to source their food outside of the 9am-5pm workday, which does not often align with farmers market schedules. The final 7 determinate of convenience is the ability to multi-task, which describes customers’ desire to buy most or all of the items on their list at one place and/or near other errands they have to run. Essentially, customers assess if the products they want and need are available. Farmers markets, often limited by local production and vendors, are a weak competitor against conventional grocery stores that import any item regardless of distance or season. In markets where there is limited vendor diversity, the lack of certain staples might cause minority shoppers to go elsewhere. But these limitations do not mean that shoppers do not want produce from the farmers market. In fact, when asked by Webber et al. (2010) to describe an ideal grocery store, low- income consumers routinely stated they would prefer if produce was purchased directly from farmers and presented in a “farmers market model” with ornate displays and booths. And while convenience is a major factor in purchasing decisions, there is research to support that low- income consumers who do have transportation are willing to bypass the closest store to find quality and culturally appropriate goods (Shannon, 2014; Webber et al., 2010). What these consumers are implying is that they may enjoy farmers markets, but they have greater access to grocery stores that are more convenient for them to get to. Price: Real and Perceived In a study about the shopping behaviors of low-income families, Fish et al. (2015) found that price was a key deciding factor. Lower price was equated with buying in larger quantities, which is sometimes harder to do at a farmers market. The ability to use food assistance at the farmers market can address this barrier to an extent. For example, an assessment of six Michigan farmers markets that were early to adopt food assistance programs showed an increase of 33- 325% in SNAP sales during a single market season (Mino et al., 2018), demonstrating the usefulness of these programs for low-income consumer attendance at the market. However, Ritter et al. (2019) found that the perception of higher prices does still hinder consumers from exploring farmers markets even if they accept food assistance and some market items are more affordable than at other venues. Exclusivity The image, and often reality, of alternative food systems, including farmers markets, is that they exist for the most privileged tier of consumers in the United States (Alkon, 2008; Alkon & McCullen, 2011; Lambert‐Pennington & Hicks, 2016; Lowery et al., 2016; Slocum, 2007). As 8 low-income consumers struggle to meet daily nutrition demands, white elite populations are paying more for food considered to be niche and popular (Slocum, 2007). The divide between high-income, white consumers and low-income and minority consumers exists to the point where the former hardly sees or thinks about the latter within the food system. Alkon & McCullen, (2011) explain this as the “white community imaginary.” Due to the portrayal of farmers markets as an inclusive community event, white consumers carry on thinking the markets are representative of their broader community, meaning they assume that the miniscule percentage of low-income and minority consumers attending or vending at the farmers market is representative of the community. On the other hand, low-income and minority consumers are more aware of the upper-class, white consumers that exist in these spaces. For example, customers seeking to use food assistance often must wait in line at the market manager booth to redeem their funds, unable to blend in with consumers shopping with cash or credit card. Consumers report feeling disrespected by vendors that feel inconvenienced by the alternative – and often cumbersome - method of payment (Freedman et al., 2016). In general, low income consumers may be uncomfortable around the “bourgeoisie crowd” (Ritter et al., 2019). While these sentiments aren’t eliminated entirely in grocery store settings, low-income consumers can choose stores that feel more comfortable to them and that serve a consumer base that looks and acts more like them (Shannon, 2014). Consumers of color are impacted by these white pressures regardless of if they use food assistance or not. For example, Alkon and McCullen (2011) also refer to the ‘white farm imaginary,’ to illuminate the illusion that farms - even ones that are well known employers of people of color – are populated and staffed by white workers, because of the image presented at farmers markets. Consumers are greeted by white owners who are assumed to produce and harvest themselves or with the help of their families, e.g. the wholesome image of the family farm. This presumption allows white consumers to continue buying their produce guilt free, but it also makes invisible the faces of those that contribute the physical labor of harvesting the food (Alkon & McCullen, 2011). Faces that may be comforting or relatable for low-income and minority consumers are obscured to maintain the image that food produced by ‘family farms’ at the farmers market is more ethically produced than the produce at the grocery store. Alkon & McCullen (2011) illuminate the need to "recognize and confront the liberal, elite whiteness that 9 pervades [farmers market] discourses and practices" before farmers markets can be seen as a viable alternative food option for people of color. Administrative Burden It is important to note here that all barriers thus far have been described as a hinderance to customers attending or frequenting farmers markets, however, there is also literature explaining burden from the administrative perspective. This is included in this section about barriers because there are administrative hurdles, which ultimately impacts the ability of farmers market managers and staff to serve these customers. For example, where the WIC FMNP and Senior FMNP have specific vouchers that can be collected by farmers and submitted to the market manager at the end of each day, SNAP benefits use Electronic Benefit Transfer, which works like a debit card. Market managers must run the EBT cards at their booth to redeem these funds. Under the Agricultural Act of 2014, it is required that “retailers (with some exceptions) pay for EBT equipment, supplies, implementation and related services” meaning that SNAP benefit use is not guaranteed at every farmers market (USDA FNS, 2022a). Without funding to purchase supplies such as a point of sale device, markets may not be able to participate in food assistance programs. Mino et al. (2018) conducted observations and interviews at six farmers markets that support the use of food assistance programs and found that the administrative hardship was heavy. Markets varied in their ability to support these programs due to organizational capacity, ability to obtain resources (including point of sale devices that run EBT cards and trained staff), and a lack of strong leadership. The authors go on to explain that “federal policy makers are unlikely to be aware of the challenges associated with implementing food assistance programs in these venues” (p. 832). This research finds that the burden of food assistance lies with market managers rather than the federal government, who is encouraging participation in food assistance interventions. Gusto et al. (2020) found that market managers who have good relationships with government entities have additional support that allows for them to implement food assistance, including community relationships, educational resources, and external funding. Administrative burden was heightened during the COVID-19 pandemic, when farmers markets had to make changes to their operations to support continued consumer access. Farmers markets in Michigan that received guidance about safety measures from the Michigan Farmers Market Association (MIFMA) in the form of webinars and educational materials during the 10 height of the COVID-19 pandemic (Taylor et al., 2021). However, not all markets have this type of support. One adaptation many markets made was the ability for customers using debit or credit cards to buy online and conduct contactless pick-up, but customers paying with SNAP or other food assistance could not use these modifications (Taylor et al., 2021). This was not a payment pathway that individual farmers markets could create for food assistance customers as they could for those using credit/debit cards. Markets are handed down the regulations of the program and must implement them as the federal government decides; it is not up to the local government or community organizations to make administrative changes such as these. Addressing These Barriers: Mobile Markets as Interventions Mobile farmers markets are often suggested as an intervention to overcome the barriers between low-income consumers and traditional farmers markets. They build upon the already successful mobile food vending trend – e.g. food trucks, mobile fruit vendors - and combine it with the structure of a farmers market. They vary in motivation from health promotion to social justice (Best & Johnson, 2016; Satin-Hernandez & Robinson, 2016), but most share similar end goals of providing convenient access to fresh fruits and vegetables for low-income and marginalized communities by selling produce and providing nutrition information. Scholarship documents their ability to overcome barriers like transportation and price (Hsiao et al., 2019), however an underexamined aspect of mobile farmers markets is their ability to overcome barriers for their target group. This ability is important as there can still be barriers to mobile farmers markets. Hsiao et al. (2019) conducted a systematic review of the barriers and facilitators to using mobile farmers markets and found that while mobile markets are designed to overcome some of the barriers to more traditional farmers markets, including transportation, convenience, and exclusivity, some customers have a “reasonable and healthy suspicion of price gouging” (Arcadia Center for Sustainable Food & Agriculture, 2012). They also explain administrative difficulties such as lack of knowledge about the produce sold and ineffective promotion of markets (Hsiao et al., 2019; Ylitalo et al., 2019). Additionally, some markets note that a lack of culturally relevant foods or nutrition information (recipes, cooking demonstrations) can turn away customers looking for ethnic-specific produce (Dulin et al., 2022; Ylitalo et al., 2019). Further, the literature provides some distinction between mobile farmers markets that are run by farmers and those that are run by nonprofits. While some mobile markets, typically the 11 ones owned and operated by farmers, must maintain economic viability through market purchases, akin to more traditional farmers markets (Best & Johnson, 2016; Robinson et al., 2016), many others are catalyzed by and maintained with grant support, which support the subsidization of food costs. Therefore, these different funding models can impact the objective of providing food at a reasonable cost to the target consumers for health-based intervention; farmers need to stay in business, so they cannot always provide low-cost options for customers. Best and Johnson (2016) describe the informal social networks formed among customers and between customers and staff at mobile farmers market in Washington, D.C. that sold fruits and vegetables, offered nutrition education, and hosted cooking demonstrations. The authors used the term “affect structures” to describe the “emotionally attentive interactional pattern of respect, reciprocity, and trust that enabled specific types of consumer encounters” (Best & Johnson, 2016). However, over three market seasons the authors found that the market needed to scale up operations to maintain profit margins. The informal networks that were created during earlier seasons suffered. Observations made during subsequent years showed that the market transitioned from a place of learning and community building to a place of transaction, which resulted from standardization of efficient procedures and an inability to continue the cooking demonstrations and informal nutrition education. Maintaining these affect structures was in conflict with business norms, suggesting that mobile farmers markets seeking to encourage community building must intentionally prioritize the formation of affect structures to maintain social benefits. What this case suggests about mobile farmers markets – which could be extended to traditional farmers markets that also need to make a profit – is that food access intervention strategies are about more than the transaction of funds for fresh produce. Rather, the experience in these spaces is what brings customers back, strengthening their connections to local food access. The highlighted barriers and facilitators of mobile farmers markets encourage future studies to further investigate the “influence of the social environment and how to leverage social norms and market staff characteristics when implementing mobile market interventions” (Dulin et al., 2022). This is timely research as Janhonen et al. (2018) call for new ways to teach food- related education in a manner that specifically analyzes the social acts of acquiring and eating food. This chapter will unpack the case of a mobile farmers market in a small city in Michigan to understand the ways in which the program caters to the specific needs of their target audience, 12 therefore overcoming the outlined barriers to traditional farmers markets. This comparison is important, especially if the mobile farmers market is also capable of providing the benefits associated with traditional farmers markets. Methods The following data were collected as part of an evaluation of the mobile farmers market. The purpose of this evaluation was to learn from the perspective of the customers (1) what the mobile farmers market was doing well and (2) how the mobile farmers market could improve. This chapter will focus on information found during participant observation and interviews with customers and staff to discuss the mobile farmers market’s ability to overcome barriers to traditional farmers markets and what benefits they provide. These ethnographic methods allowed us to explore the MFM program as a “multifaceted composition of human experiences permeated with multiple meanings” (Schwandt, 2015). While the original goal of the project was to understand the customer experience at the MFM, , preliminary observations and informal discussion with market staff illuminated organizational challenges beyond the customer experience that were impacting program objectives. Therefore, in addition to the customer interviews we planned for, we also included staff interviews in the evaluation process. Consent and interview protocol were deemed exempt by MSU IRB: STUDY00007880. 13 Interviews Customer Interviews Customer Interviewee Demographics Site Age Gender Ethnic Background A 63 Female White A 53 Female Hispanic A 61 Female African American B 70 Female White B 61 Female African American B 42 Female White C 59 Female White C 75 Female African American C 73 Female White C 64 Female White C 52 Female African American Table 1.1: Customer interviewee demographics including their home site, age, gender, and self- identified ethnic background We used convenience sampling to identify participants, which is “a type of nonprobability or nonrandom sampling where members of the target population that meet certain practical criteria, such as easy accessibility, geographical proximity, availability at a given time, or the willingness to participate” (Etikan, 2016). Convenience as used in this project meant convenience for both the researcher and the participants. For example, when asked how they were by MFM staff, one customer responded “sad,” then went on to explain that a family member had passed away. Though it would have been convenient for the researcher to interview this customer, she deemed it an inconvenient time for the customer to engage as an interviewee. Research can be a burden for participants, especially minority and low-income participants, and this flexible sampling method allowed us to be considerate about participants’ time and capacity. The intention of convenience sampling was to interview anyone willing to participate in an interview. The limitations of this method of sampling are that the customers that want to participate are already in good spirits with the mobile farmers market. This method also occurs without set demographic quotas (See Table 1.1 for customer demographics). While we hoped for 14 a demographic that was similar to that of the market, all the customers that interviewed identify as female. Male customers were approached and asked to interview but denied at the time of sign up or later at the start of interviewing. Customer interviews (n = 11) ranged from 11 to 36 minutes and explored customers’ shopping experiences, produce use, perceptions of fresh produce, and perceptions about how the market functioned through a semi-structured interview guide (Appendix A). Staff Interviews Staff Interviewees Age Gender Ethnic Background 32 Female White 25 Female African American 50 Female White 71 Male African American 40 Female White 57 Male African American 65 Female African American 71 Female White Table 1.2: Staff demographics including age, gender, and self-identified ethnic background Interviews were conducted with MFM staff members and the executive director (n = 8). Both the executive director and one other staff member had worked for the organization for more than seven years; the other six staff members had worked with the MFM between one month and one year. There is a good deal of staff turnover within the organization because many are either AmeriCorps members (<1-year terms) or AARP Senior Community Service Employment Program staff (varying term lengths), which presents a challenge for maintaining historical institutional knowledge. Staff interviews averaged 40 minutes and focused on: experience in the food system, position in the organization, and perspectives of market function (Appendix B). Participant Observation Participant observation data was collected for the purpose of describing activities and interactions at the market as well as to contribute to the evaluation (Schensul & LeCompte, 2013). This occurred throughout the market season in two stages. Early observations were intended to gain familiarity with each site’s layout and customers, as well as the staff activities 15 and market processes. Field notes for these observations were conducted during three full market days, from pre-market preparation to post-market unpacking (9:30am – 6:30pm). Observations during the following three full market days then focused on who was at the market, how long customers spent at the market, and what kinds of interactions (content, length) customers had while there and with whom. We conducted observations during two additional market days, but with the intention of recruiting interviewees, so field notes on those days were less comprehensive and recorded between interviews. At the end of each day, field notes were expanded into narrative memos by the observer. While the evaluator planned to conduct an external evaluation, limited capacity and short staffing at the MFM required the researcher to assist with market activities. Therefore, it became an internal investigation as she prepared produce, ran transactions, and supported staff and volunteers. In some ways, this made data collection more natural, providing opportunities to interact with customers and observe interactions that would have otherwise been awkward for an evaluator. This role also fostered relationships with customers and staff, which helped make the interviews more comfortable. On the other hand, some information was more difficult to collect. For example, quantitative data about the number of customers, length of time customers spent at the market, and demographic dynamics of market customers was difficult to ascertain while working at the market. This information was only recorded at a few markets. Data Analysis All interviews were recorded for accuracy and transcribed into text files to be analyzed during the qualitative coding process. Both deductive and inductive coding were used (Miles et al., 2020). The coding process included an initial read of printed transcripts with margin notes marking salient and recurrent themes guided by the research questions. These notes were condensed and streamlined to create a working codebook for the interviews, which were implemented and refined during two rounds of coding using MAXQDA qualitative coding software. After the second round of coding, no new codes emerged. At this stage, the researcher created a matrix to demonstrate themes across the interviews (Miles et al., 2020). This involved writing summaries for each code and collecting quotes that explain their use. This process occurred for both the customer interviews and staff interviews. A different codebook was created and used for each of these sets of data. Each codebook was peer reviewed by two or more researchers not involved in the project. Finally, a matrix was created to look across interviews to 16 create the categories that appear in results. Observation notes were used to corroborate findings in the customer and staff interviews by adding relevant stories to the matrix related to the those categories. Results Customer Interviews First, we coded deductively using the identified barriers to traditional farmers market participation for low-income consumers: transportation, convenience, price, exclusivity and administrative burden (See Table 1.3). Because there was no mention of exclusivity in the interviews, we dropped this theme dropped from the final codebook and analysis. Instead there was a notion of inclusion included in the theme ‘Community Building’. We also found during the coding process that administrative burden was too broad a concept to be a useful analytical theme. From the customer perspective, administrative burden was seen in two areas: (a) staffing and training and (b) communication. We grouped the themes from the deductive analysis into a category called Traditional Limitations. We also analyzed the data for emergent themes to better understand the customer experience. This inductive coding strategy was informed by our research questions, thus we attended to any mention of benefits of attending the market, locally produced food, grocery shopping, healthy eating habits, food stories, and the market experience. We identified the following themes: community building, health, and food connection. We grouped these themes into a category called Market Benefits. The inductive code also led to a third category we called: Other Market Experience, which describes customers’ experience obtaining food from markets other than the MFM and provides useful data to comparatively understand the impacts of the MFM for customers. 17 Category Associated Definition Themes Transportation Mention of transportation method (to the market and/or other shopping locations) such as having a car, walking, getting a ride, taking the bus, etc. Also note any ease or difficulty with which they mention this transportation. Convenience and Convenience (or inconvenience) described by participants Product to when explaining their process of accessing food. Availability Price Price relating to the cost of food. Mention of food assistance and use. Other related terms often used are cost, deals, discounts, sales, etc. Exclusivity Mention of discomfort, not fitting in, feeling outed by Farmers payment method, or other exclusivity occurring while Market accessing food. Barriers Administrative This overarching theme describes where customers Burden mentioned an administrative decision or areas where they perceived staff to be struggling. Explanation of staff having difficulty in providing the mobile farmers market services to customers in terms of Staffing and the amount of staff at the market and having trained staff Training to answer questions. This relates to communication between staff and Communication customers. Note where customers want more communication e.g. advertisement or explanation of services. Farmers Community Mention of social interactions at the market including Market Building customer-customer and customer-staff relations. Benefits Health Customer mention of health in any terms. May include mental and physical wellness. Food connection Mention of personal connection to food. This could be prior food knowledge or nostalgia for certain produce, generational knowledge or learning about food, or future food interests. Other Other Market Customer explanation of accessing food outside of the Market Experience MFM. Experience Table 1.3: Customer interview coding categories, associated themes, and definitions Staff Interviews All codes for staff interviews were emergent, though some similar themes appeared which corroborate findings from customer interviews including: staffing and training, communication, and community building. From the staff perspective, administrative burden was explained as more than a few tasks, but rather the entire process. Staff frequently discussed day- 18 to-day operations of the MFM and how they could be improved to ease administrative burden. This theme appeared more than any other in the staff interview data. Category Associated Definition Themes Process Use when the staff discusses difficulties in the day-of Process process or when they offer solutions to the day-of process. Staffing and Staffing and Use when the staff mentions how the market is staffed. Training Training Also use when staff expresses a want or need for more training, elaborates what training would be useful, or explains the training they have received to perform this position. Food System Used to gather information about prior experience the Experience staff had related to this role, or a lack thereof. Communication Communication This includes communication with customers such as with Customers advertising, education, newsletters, or explaining services. Communication Communication between the staff members and between with Staff the staff and executive director. Community Community Use when staff speak of relationships with customers. Building Building Table 1.4: Staff interview coding categories, associated themes, and definitions Discussion Our findings show that most barriers to traditional farmers markets are addressed by the services of the MFM, including transportation, convenience, exclusivity, and price. The MFM does still face administrative burden, though it apparently did not hinder the customer experience at the market. As well, the MFM provides a number of benefits associated with traditional farmers markets, save for the opportunity to interact personally with farmers. Therefore our results show there are tradeoffs between traditional farmers markets and the MFM, which we will detail below. Amongst these findings are important lessons for food access interventions in general, as customers relayed that the MFM experience improved their food access compared to typical grocery stores. Other Market Experience The interview guide was set to ask about typical grocery shopping routines which included their experience at other grocery stores and farmers markets. Customers mentioned mostly shopping at Kroger or Meijer, which are supermarkets that carry food and nonfood items. These stores range from 2.6 to 5.7 miles away from these sites. Customers also explained that 19 when necessary, they would go to a smaller market close by. Some of these are options like Family Dollar, or corner stores. Many customers mentioned either never having been to the farmers market or not visiting them for many years. Four customers reported occasionally going to other farmers markets. When asked to compare, one customer mentioned having a different social experience at traditional farmers markets, “everybody was pleasant, but it just wasn't social. People don't seem to hang around. They seem to get in and get out over there”. Four customers said that they had been to farmers markets in the past but they did not currently attend any either because of transportation, moving and not finding one in the area, or the farmers markets themselves changing locations. Three customers mentioned never having been to other farmers markets. Farmers Market Barriers Transportation Transportation is one of the most documented barriers to farmers market participation for low-income customers and, unsurprisingly, transportation was not a barrier in this study since the mobile market brings the market to the customers. Customers lauded the MFM for coming to them and corroborated the established literature about difficulty getting to other grocery stores and farmers markets (Freedman et al., 2016; Gusto et al., 2020; Ritter et al., 2019). Customers often discussed transportation and convenience synonymously. They shared the ease of visiting the MFM compared to the difficulty of visiting other food outlets because they lacked transportation. As one participant shared, “you just walk out the door and boom. There you go”. Another participant explained that the MFM is “very convenient, because a lot of times people can't get out, their families can't get them. And the bus is like a whole $5 [roundtrip]. So then you have to wait a long process for them to come back and get you”. When discussing how she accesses the grocery in the off season or to purchase products the MFM does not carry, one participant shared: “[I] pray for a ride. Sometimes my granddaughter takes me, sometimes my daughter's baby daddy takes me … because I don't have a car anymore”. The MFM removes this uncertainty by going directly to the customer, thereby providing access to fresh fruit and vegetables for customers without the need for transportation. Convenience and Product Availability Customers in this study agreed with the definition of convenience shared by Webber (2010) in the respects of physical proximity and compatibility of schedules, often noting the 20 frequency, consistency, and time of the MFM being convenient. In the off-season, some customers go to the grocery store infrequently due to transportation limitations. One customer explained that the infrequent shopping schedule limited her ability to buy produce for fear of wasting it, sharing, “you know, you try to use [the produce], you don't try to buy too many, because you don't want them to spoil”. The market provided a consistent way for customers to get fresh produce in a timely manner, as another participant explained, “we come to the point where we don't even buy greens, or green beans [at the grocery store], because we depend on you for having them”. The MFM visited once a week which worked well for most customers, though some participants wished it came more often. “[I]n a perfect world,” one shared, “it would just sit out here all day. And we can come out anytime of the day morning, noon, or night”. The timing of the MFM stops was convenient for some customers, and less so for others. One customer shared “this one is just very convenient for me. I get home from work, and it's right here," while another shared “I work Monday through Friday. And like I said, I get off at 3:30pm. And you're gone already”. Comments made by several customers compared the convenience of the MFM to grocery stores in terms of physical accessibility. The MFM customer population, which largely consists of seniors, may use mobility aids like walkers and wheelchairs. Discussing the difficulty she faces when going to the grocery store, one participant shared, “when I use an electric cart, it's not easy to get my products and I feel like I'm in the way and you've got the cart”. However, at the MFM, this concern is minimized. Customers need only to leave their apartment and go through the line, which requires much less movement compared to walking up and down grocery store aisles. Also, staff create a unidirectional flow at the MFM, with plenty of space for customers to move through the line. They handle the produce and walk through the line with customers to add items to their baskets, deliberately trying to create a convenient experience for customers. Staff ask customers if they’d like their bags on their lap or in their walker compartment, or, in some cases, go so far as to bring customers’ groceries inside the apartment building and to their door. A staff member compared the market experience they try to create to other farmers markets they had visited, explaining that at those markets “on Saturday morning it's packed, and it's kids, and…. it's bags, and it's, you know, a lot of things that it's just not feasible” for the customers they see at the MFM. Customers described their use of the MFM as positively contributing to their self-sufficiency because they did not need to rely on a caretaker, friend, or family member 21 to acquire their groceries. The notion of mobile farmers markets being more physically accessible, as was mentioned in our study, is not prevalent in the literature. Future research focused on physical accessibility would be helpful to better understand the experience of physically disabled communities in accessing local food. Product availability was also used to describe convenience by Webber (2010) as the ability for customers to get the items they need. In this study, when customers spoke of product availability, they were mostly happy. Customers requested to see more Michigan-grown products, and mentioned they would be able to spend their DUFB if there were more. As the market accepted recommendations as to what items to bring, they were well suited to provide produce options that reflect the needs of the customers. However, the difficulty was in ensuring product availability across sites. As the market stops were all in one day, stock would dwindle toward the later stops. This means that customers at Site C often had access to fewer options than those at earlier sites and in interviews they brought this up. For example, one customer from Site C said “I know it's not your fault, because people buy it up before you get here. But how about save some for us?” Later in the season, the staff began to hold back items for later sites and regulate the number of certain items that would normally sell out before other customers could get them (e.g. Michigan-grown strawberries). The other improvement customers offered had to do with the physical ability to touch the produce. As a regulation during the peak of COVID-19, the market created a system in which the staff would walk down the line and put produce in a bin for customers with gloves on. This meant that multiple people were not touching the produce. As COVID-19 became less of a worry for these customers in the 2022 season, they brought up the want to handle the produce again themselves. Customers told stories of not being able to inspect the produce thoroughly before taking it home and later noticing a tomato was rotten or their strawberries were already bad. Price The ability to use food assistance at the MFM impacted participants’ perception of price. The market accepts DUFB, Senior and WIC FMNP (although little to no WIC is redeemed year to year), and EBT. In the first two years of the market, the organization took in a high quantity of cash and credit card transactions; since then they have sought to find locations where customers are more likely to use food assistance to make sure they were supporting low income customer needs. Every staff member is trained to use each type of food assistance so they can help 22 shoppers make decisions. For example, DUFB, WIC and Senior FMNP can only be used on Michigan-grown produce whereas EBT can be used to buy other goods (e.g. non-Michigan produce, bread, and jam). One staff member walks through the market with each customer, often beginning by asking “how are you paying today” so they can help guide the customer’s shopping process. The market is set up with all Michigan-grown produce on one side and non-local produce on the other side to make it easy for customers to use particular food assistance types. Senior FMNP comes in the form of coupons worth $5. As staff walk through the market, they count up to $5 with the customer, often offering to adjust their order by adding a jalapeño or onion to reach exactly $5, as they cannot give change back for these coupons. Participants mostly described positive experiences using their food assistance at the MFM. In some cases, food assistance benefits successfully incentivized the customers to visit the market, as one shared, “there's been times that the only way I was able to come was because I had Double Up coins”. Other customers could purchase more because they had food assistance. One participant explained, “I love it because I can get stuff that I might not be able to get if I didn't have enough money”. Participants did describe a learning curve, because they had to figure out which items could be purchased with which benefits. This proved frustrating for some customers. For example, one customer asked for iceberg lettuce, but the staff member knew it was from outside of Michigan, therefore could not be purchased with food assistance. She offered the customer a head of romaine instead, but the customer stormed off and never returned to the MFM. But this is a limitation of the food assistance system, outside of MFM control. Relatedly, though, the MFM does source some of their produce from other grocery stores, rather than from farmers. The MFM must make decisions as to whether to buy produce that is Michigan-grown and produce from the grocery store that may cost less. Though price was less of concern for customers because of the ability to use food assistance, customers did notice when a product was more expensive at the MFM than it would be at the grocery store. They also noted that there were no “sales” at the mobile market (i.e. no buy one get one or reduced prices for overstock). Sometimes customers rationalized the higher prices by highlighting the quality or convenience of the item. One customer complained about the price but noted the unique variety was worth it, “I have been to every grocery store basically in [this city] besides [store name] and no one has Golden Delicious apples”. 23 Exclusivity Though exclusivity was coded for in our data, the sentiment did not come up as defined in the codebook; rather a sense of inclusion was explained. This MFM conducts activities as encouraged in the literature to limit this feeling such as hiring staff that reflect the customers, offering relevant ethnic staples, and encouraging requests (Robinson et al., 2016; Satin- Hernandez & Robinson, 2016). The MFM staff largely reflects the customer base in terms of both ethnic background and age (See Tables 1.1 and 1.2) The staff is 50% White, 50% African American; 75% of the staff identify as female, 25% male; age ranges from 25-71 years old with 75% of the staff over 40 years old. Demographic surveying of participants was not rigorous enough to provide detailed information about demographics of all market customers, but most customers used SFMNP coupons which have an eligibility requirement of 60 years old. The interviewed customers ranged from 42-70 years old. All interviewed customers identified as female. Though there were customers that identified as male at the MFM, these customers did not agree to being interviewed either up front or later on the phone. In any case, they have less of a presence at the MFM comparably. Interviewed customers identified as White (55%), African American (36%), and Hispanic (9%; one customer). Participants shared that they felt a sense of inclusion at the MFM because of the broad and encouraged acceptance of food assistance. As most customers used some form of food assistance at the MFM, participants did not feel singled out when they also used them. A staff member, who had used food assistance at multiple traditional markets in the past, captured this feeling: I feel like at other farmers markets, there's like a stigma [when] you use [food assistance].... [Y]ou're using these…giant like coins that other people might not be using. So it feels kind of stigmatizing ….And I feel like our market [the MFM], since it's geared towards people who are using [assistance] there isn't a stigma involved ….[P]eople seem … pretty comfortable using different forms of payment. As someone who has used SNAP a lot at other farmers markets … I was kind of embarrassed at first to do that. Now, I don't care….I feel like we do a good job with that” S1. At the market, customers frequently talked with each other about food assistance and often taught each other how to use the different types, even if they did not know each other before. 24 Whereas this is explained as a barrier for customers to traditional farmers markets (Freedman et al., 2016; Ritter et al., 2019), this was a point of connection for these customers as food assistance would be a shared topic of conversation upon which customers and staff could connect. This makes sense in this context in which the sole purpose of the MFM is to provide for customers that use food assistance. Conversely, at traditional farmers markets, food assistance usage can be taxing because individual vendors as well as the market managers must know the rules to each type of food assistance (Mino et al., 2018) and their purpose at the market is often to sell their produce. Administrative Burden Though customers found the experience of using food assistance at the MFM to be relatively easy and encouraged, the MFM staff did struggle with some aspects of implementing these benefits. Documenting the information required to report back about food assistance is repetitive and onerous during transactions, which slowed the process of checking out. Customers sometimes complained about lengthy wait times when the line was held up. In the interviews, staff members shared that they found it frustrating that a single transaction required two people to complete, but they admitted that navigating the multiple forms required to document inventory and SNAP/DUFB usage was burdensome and required extra support. This challenge was compounded by staffing issues. During this season, the market manager role was vacant, leaving a leadership vacuum and increasing the administrative burden for the staff. One interviewee explained that. “it almost seemed like…there was no plan. And so …in the absence of a plan, there will be chaos”. Customers observed these staff struggles and one participant recommended that staff be trained to identify each type of produce and how to handle them with care. Staff agreed, but received limited training before the market began and had little time at the market to educate amongst themselves. Mino et al. (2018) reported similar frustrations for traditional farmers market managers with the numerous and repetitive reporting tasks as well as minimal staff to support market activities. These authors suggest “underlying capacity of the markets themselves” as the real source of limitation. We agree with this line of thinking, especially as the roles in this market are often filled by temporary staff. Chapter 2 provides a deeper dive into organizational capacity and working toward program improvement. 25 Farmers Market Benefits Based on these results, the MFM is effectively achieving their goals by getting produce and nutrition education to customers in food deserts. The MFM provides many of the benefits associated with traditional farmers markets, like social interaction and community building, quality produce, and good value (Warsaw et al., 2021; Wolf et al., 2005). The foundational purpose for mobile farmers markets is to bring the farmers market benefits to customers that could not attend due to these barriers and in ultimately, increase access to fresh fruits and vegetables. Beyond the first benefit of accessing produce, customers also appreciated produce quality and in some cases the good value. Community Building Participants characterized their experiences of grocery store shopping as a chore that induced stress. Conversely, they talked about their experience at the MFM more like a social gathering. Customers attended the market with friends and family or made new friends at the market. At one stop, customers noticed a new resident and helped him get adjusted to the market, sharing information and food assistance to help him make the purchases he needed. Participants also discussed sharing produce and recipes from the MFM with friends and family outside of the market. They talked about buying items for a fruit or vegetable platter, making salsa for birthday parties, and inviting others they saw at the market to join their celebrations – including some staff members. The MFM also created a social environment that the customers enjoyed. Bringing the fresh produce to the customers, sharing food, knowledge, and conversation, provides the similar experience that is had by traditional farmers market attendees. The is exemplified in the relationships created between customers at the market and between customers and staff. One customer particularly enjoyed their interactions with the staff at the MFM, sharing that they looked forward to “talking with people in the interaction [of shopping]. I'm a people person. So I really look forward to it”. Customers built relationships with staff over the season, and even across multiple seasons. They had positive perceptions of their interactions with the staff and they appreciated the staff’s dedication to the market. One customer shared, “the girls that sit there writing up our purchases are just as smiley and nice as they can be”. Another described the staff by saying, “they're supportive. Anything I need you know, they're on it.”. Beyond the professional relationship of selling and purchasing produce, the customers and staff also share personal information and experiences. One example of this was observed on a market 26 day in August. One customer cried while going through the line. She was consoled by her caregiver, her friend, and [a staff member]. As she came up to the till to pay, the executive director asked her what was wrong and she sobbed again. She was upset about a medical procedure she was just informed she needed. The executive director consoled her further and asked questions about next steps. Staff at the market kept up with this customer week after week, checking on her medical progress. These deeper connections, mentioned in the interviews with both staff and customers, are sought out by the staff. This was observed during one stop where there was a customer that spoke only Arabic. A staff member went through the line with him, using an app to translate. She then sat with him for a few minutes translating back and forth and they called each other friends. He told her he is looking for places where the “happy people” are. She used her phone to look up local places where he might be able to connect with others and build relationships. Customers and staff did not hesitate to be real with each other, often answering the typical “how are you” with genuine answers such as sad, stressed, busy, or excited, which led to conversation during market transactions. A staff member shared that these relationships are an important part of the nutrition intervention, because staff can leverage personal connections to encourage customers to buy fresh fruits and vegetables and incorporate new things into their diet. She said, “a lot of people in that demographic are not open…They’re not going to try anything, and they’re certainly not gonna buy anything if they don’t trust you”. She shared that the MFM brings produce and nutrition education into customers’ comfort zones, explaining that, “you can be in your wheelchair and slippers and push yourself on up here and we don’t care. You know, it’s like, we’re not going to judge you. And I think that our own level of respect for them has a lot to do with their comfort level too”. Health Participants reported that their visits to the MFM made them feel healthier. One customer referenced her husband’s recent heart surgery and how the mobile farmers market helped them change their diet by “eating more leafy greens and things like that. You get it here. And sometimes you go to the grocery store and they don't have the stuff. Yeah, it's very limited or wilted or sometimes”. Others agreed, sharing that because of the MFM they are eating more fruits and vegetables. One customer attributed this to being able to buy smaller amounts of produce more frequently. She is only able to go to the store once or twice a month, and mentions 27 “it's hard to eat healthy if you don't have that availability”. Our work focused on the perceived benefits of the market so though it was not expressly measured, customers reported eating more produce since shopping at the MFM. Increased fruit and vegetable intake is also proven in the MFM literature, however further research is encouraged to improve the validity of these findings because there is not a standardized measurement procedure (Hsiao et al., 2019). Customers also referenced the more holistic health that they receive at the market, which encourages exercise and social interaction that supports their wellbeing, “[Y]ou physically have to walk to get there, you know? So you have to get out and get some fresh air. And you see it as fun. It's nice to see people you know”. Another customer shared that, “There's a lot of people that don't get out [at the senior building…But] they do for this [market]. They feel safe. And it's close. And everybody's friendly”. Connection to Food Participants discussed gaining a connection to local food at the MFM. One customer shared, “I like that it is fresh and it's from [a] farmer site. I like them to support our local farmers. Because they need support. Yeah, I trust it better. …I feel like it's better for me”. Some customers were familiar with seasonality and others learned about it at the MFM, expressing excitement for strawberry season or frustration when the peaches were gone. While shopping at the MFM, customers talked about their broader connections to the food system, which was seemingly sparked by interacting with fresh and local produce. Customers questioned produce they had never seen before or requested produce that was not present; this was encouraged by the staff. Some customers used to live on farms or tended gardens, and they shared stories of learning how to cook when they were young, teaching their children and grandchildren how to cook, and sharing family recipes handed down over time. They also discussed how they get their food information now, most of it casually from family and friends, media such as the internet or television, and from the MFM. Recipes, cooking methods, and stories connected customers and staff over the produce displays. Limitations There are limitations to this study as qualitative research, which does not allow for much generalization beyond these results. Further, within the mobile farmers market literature, there are distinctions that do not allow them to be lumped into one category. For instance, some mobile farmers markets are run by farmers who need to make profit from sales while others are 28 nonprofit. Some obtain grant funding which allows them to make pricing decisions that other markets cannot make. However, the connection to the expansive farmers market literature allowed us to make connections larger than this study. Our sample population also brings up some limitations. Within the convenience sampling, we only interviewed customers who were at the market, hence there is still missing knowledge about members of this community that do not attend the MFM. Future research for both mobile and traditional farmers markets could innovate this research by targeting populations that do not attend. An example of this could be tracking the FMNP coupons that are not used and ask what could be done to improve their access. Finally, only women were interviewed but the literature reflects this population attends more than others. Efforts were made to interview men but they were not seen through. Due to the limited staff, the researcher played an integral role in running the market, which supported learning about how the market was conducted and impacted how the data was collected. But this made it difficult to collect market data. Finally, it was difficult to ask customers to compare between traditional farmers market and mobile farmers market because not all customers have been to traditional farmers market before, therefore our findings draw largely on the literature to depict experience of low-income consumers at traditional farmers markets. Conclusion: Alternative Approaches to Interventions for Farmers Markets The SNAP program and the movement being made to allow food assistance at farmers markets is an opportunity to connect low-income consumers with local producers. We’ve discussed here why these mobile farmers markets are beneficial. This research encourages the use of programs like these that provide food access intervention with specific care to a small population’s needs. Overall, results show that the MFM is capable of overcoming the barriers of traditional farmers markets while providing the benefits to these customers. This leads to lessons that may be generalized beyond this single case. First, that increased food access and nutrition education is enhanced by the act of community building. The literature suggests that food access interventions ought to be integrated into social settings to have meaningful impacts for customers (Best & Johnson, 2016; Dulin et al., 2022; Hsiao et al., 2019). This MFM provides a sense of community through social connections which incentivizes customers to attend the market and receive nutrition education 29 and fresh produce (Best & Johnson, 2016; Dulin et al., 2022). Beyond the chore of shopping, these customers saw the market as a time to interact with the family, friends, and staff. This avenue for nutrition education is seen as a positive and enjoyable space for these customers. Feelings of comfort and safety were also conveyed, which are an inherent benefit of bringing the market to the customer. Second, that this unique intervention works well for this specific community. The MFM overcame all customer-facing barriers to traditional farmers markets, leaving customers with few requests for program improvement. Specifically for this population of low-income seniors, the MFM worked well as an intervention. The market catered to their food needs by bringing preferential or appropriate food items like sugar-free jam. Additionally, the MFM took care to make the market physically accessible, which is important for this population. This is notable as an intervention. The MFM is aware of their small population and the fact that they cannot serve all food-insecure individuals. However, they can make programmatic changes that cater to the needs of this population. By listening and learning from customers, they can best serve this small population. Finally, programs like this could still use some improvement. This is evidenced by the burdensome administrative lift it takes to manage and accept food assistance programs. The struggles this team had in filling out multiple documents and keeping up with transactions were not unique to mobile farmers markets. This process is taxing for traditional farmers markets as well (Mino et al., 2018). Further work on programs like these could look into streamlining food assistance processes to ease the burden on farmers market managers and staff. 30 CHAPTER 2: AN EVALUABILITY ASSESSMENT OF A MOBILE FARMERS MARKET Introduction This chapter outlines the evaluability assessment used in this project and how it can be used in organizations with communication barriers that are at odds with program goals. We designed the study using the six step framework for evaluability assessment developed by Wholey et al. (2015) and were guided by the four common uses for evaluability assessments identified by Lam & Skinner (2021). By facilitating the creation of a shared understanding of program goals and function, and by preparing staff to use evaluation data moving forward, evaluability assessment can strengthen the potential for evaluation use in the process. Beyond these expected outcomes, we found evaluability assessment especially useful in building staff capacity for internal organizational change and improving organizational communication. Literature Review Evaluability assessment is an evaluation approach made popular by Joseph Wholey and other evaluators concerned with evaluation use. They wanted to “ensure that ‘management basics’ are in place before an intensive evaluation is done” (Strosberg & Wholey, 1983), by clarifying program goals and purpose before a formal evaluation is conducted. This prioritization allows the evaluator and the organization to ask whether a program is ready for evaluation or if other internal work needs to be done first, so that evaluation can be most useful (Wholey et al., 2015). In an evaluability assessment, three elements of a program are assessed: (1) the program “in principle”, (2) the program “in practice”, and (3) the program to use evaluation information (Davies & Payne, 2015). The goal is to compare the program as it is written, i.e. with a mission, logic models, theories of change, etc., with the program as it is implemented to assess whether further evaluation would be useful for a program. Wholey et al. (2015) summarize current research that shows an evaluation is more likely to be used if: (1) the program has realistic and agreed upon program goals, (2) information needs are well defined, (3) evaluation information is obtainable, and (4) evaluation information is (a) available to the intended users and (b) the intended users are willing to use the information. Evaluability assessment allows organizations to gather this information and take small steps toward a larger evaluation. It also provides valuable evaluation feedback while also avoiding the time, labor, and economic expenditure of an elaborate evaluation if the outcomes might not be useful and/or able to be used (Wholey et al., 31 2015). This evaluation approach is often used by community programs, who may have little capacity for evaluation in terms of staff and/or funding. Lam and Skinner (2021) conducted a scoping review of the ways evaluability assessment has been used over the last ten years and found four common purposes: (1) to assess the extent to which programs are ready for evaluation, (2) to support evaluation planning, (3) to identify promising interventions for evaluation and (4) to support program development. This project originally intended to utilize evaluability assessment to support program development. During the process of conducting the assessment, though, we observed internal dynamics that led us to shift our original goal. After concluding that the staff and the executive director (ED) held different, and incompatible, ideas about the program’s objectives that were impacting the program’s effectiveness, we broadened our scope to use the evaluability assessment to work to reconcile program perceptions and reality across stakeholders. As Thomas and Campbell (2020) explain, “There can be multiple realities that are socially constructed by different stakeholders and these realities can be in conflict” (p. 245). The evaluability assessment allowed us to document these multiple realities and work with staff and the ED to identify shared goals for program operations and impacts. Rather than comparing the program in practice and the program in reality, as most evaluability assessments do, we instead compared the ED’s perception of the program (essentially the program as written) compared to staff’s perception of the program (essentially the program as implemented) to identify leverage points for alignment. Like other evaluability assessments, our outcomes included preliminary evaluation findings and recommendations for future evaluation. Additionally, we attended to process use, which prioritizes not only the evaluation findings but also the “impacts on those involved in the evaluation process” (Wholey et al., 2015). Early in the evaluation process, through observations during market preparations, market activities, and post-market unloading, the evaluator noticed a lack of communication between the ED and MFM staff, which appeared to be impacting the customer experience. To address this challenge, we shifted the evaluability assessment from an outcomes-focused project to a process-focused project, allowing for, as Hendricks & Papagiannis (1990) explain, “possibilities for action in other, unforeseen areas.” Our intention with this change was to make staff “feel good about participating and contributing to a process that is meaningful and useful,” which Patton and Campbell-Patton (2022) describe as a “boosting morale” approach to process use. 32 By addressing the internal communication challenges, we intended to improve staff capacity for organizational stewardship and change, as well as prepare the staff to maintain the program beyond the impending retirement of the ED. We found that the process of the evaluability assessment emboldened the staff by supporting effective communication with the ED and providing space for them to voice their ideas about program improvement and to learn more about theory and function. Both of these outcomes, the immediate evaluation results and the lasting evaluation knowledge, improved the chance of relevant evaluation use which is the ultimate goal of evaluability assessment. Background The program at the center of this evaluation is a mobile farmers market (MFM). The theory of change for the MFM program identifies transportation as a barrier to food security, therefore the organization seeks to remove this barrier to increase food access and nutrition education for seniors (Theory of Change, Appendix C). During farmers market season they bring fresh fruits and vegetables to customers in a renovated bus, stopping at four consistent locations around their service area: three senior and low-income housing parking lots and one community center. They source the produce they bring from local farmers and grocery stores, and they sell the produce as close to cost as possible by obtaining grant funding. The profit they do make serves to keep prices low, maintain the vehicle, and cover insurance. Most staff members are funded at least in part by an external agency, including AmeriCorps, the AARP Senior Community Service Employment Program (SCSEP), and state-funded nutrition education programs, therefore they have short or variable-term placements. This creates frequent staff turnover. The ED has been with the program since its establishment in 2016. Over the seven years of market function, the program has not been professionally evaluated, though they have informally collected customer feedback with short surveys or word-of-mouth requests for program changes. In May 2022, the MFM ED developed a partnership with the lead evaluator. Due to an absence of previous evaluation data and limited funds, they decided to conduct an exploratory evaluation, specifically evaluability assessment. Though evaluability assessments inherently ask whether the program is doing what is says it will do and whether a program is ready to be evaluated, the ED also asked for summative information about customer experience. The ED and the evaluator co-created the following evaluation questions: 33 1. In what ways are the services provided by the MFM useful for the customers it serves? 2. In what ways can the MFM services and operations improve to better meet customer needs? Methods This project began in June 2022 and final reporting concluded in March 2023. We followed the six steps of evaluability assessment developed by Wholey et al. (2015): (1) Involve intended users, (2) Clarify program design, (3) Explore program reality, (4) Assess plausibility of program, (5) Discuss any changes, and (6) Discuss future evaluation possibilities. Figure 2.1 shows how these steps align with our data collection, including document review, participant observation, and interviews with customers and staff. Figure 2.1: This timeline of the project from June 2022 to March 2023. The qualitative methods used throughout the project are in orange. The evaluability assessment steps are in blue Document review occurred mostly at the beginning of the process in which we collected program documents to provide context for the program. Participant observation took place throughout the process, most concentrated during our exploration of the program reality. This entailed (1) early observations to gain familiarity with each site’s layout and customers, as well as the staff activities and market processes and (2) later observation to pay attention to interactions at the market. Collectively, this included six full market observation days from (9:30 am to 6:30 pm). The evaluator attended two other markets with the intention of recruiting and interviewing customers, so these observations were less comprehensive. 34 Customer and staff interviews were also used in exploring the program reality. We used convenience sampling to identify customer interviewees (Etikan, 2016). As a participant observer, the evaluator ran transactions at the market. As the customers were leaving, the evaluator asked customers if they would like to interview and offered a $20 MFM voucher in return for their time. This resulted in 11 customer interviews. All customer interviewees were female. Male customers were at the market, but any that were asked to participate declined. These interviews ranged from 11 to 36 minutes and customer experience with the market ranged from 2-7 years. After customer interviews, informal conversations with staff at the market revealed that staff had relevant insight into program change and wanted to share their perspective. All program staff were interviewed (n = 8). These interviews averaged 40 minutes (See Interview Guides Appendices A and B). Evaluability Assessment Process 1. Involve Intended Users This step in the evaluability assessment is used to “define the boundaries of the project to be assessed” and understand who in the organization will utilize the findings of the evaluation (Davies & Payne, 2015). Toward this end, we held a meeting with the ED early in the planning process to learn about available documents and stakeholders that could inform the evaluability assessment. This was a one-on-one discussion to get to know the program ‘in principle’ by collecting information used to describe, monitor, and/or advertise the program. Documents gathered included the organization’s theory of change (Appendix C) and a logic model (Appendix D) the ED had used for grant applications and reports. As the ED spent the most time with this documentation, we used this and informal meetings to craft her perspective of the MFM. We also used this first step to develop the interview guide for our conversations with customers at the MFM. Later in the project, the staff also created a logic model (Appendix F) to articulate their vision of the program reality, which we also used in document review. Other documents received from the organization included fliers, transaction data from the 2021 season, and a customer needs survey from previous seasons. Some of this documentation was inconsistent. For example, customer counts were created by counting transactions, but these numbers did not account for repeat customers or staff purchases. 35 2. Clarify Program Design The task of the evaluator in this step is to “clarify [the] expectations, concerns, and information priorities” (Wholey et al., 2015) of those closely involved with the program. We used the logic model from the ED to clarify program design through the ED’s perspective. We also conducted participant observation of the MFM process before, during, and after the market, which revealed staff discontent and varying perspective about the program design and purpose. This led to the addition of staff interviews in the next step. 3. Explore Program Reality We investigated the customer experience using interviews and observation during this step. Reviewing the transcripts for these interviews led to useful summative information that the staff could use to improve the MFM for the customers. In general, the customers were happy with the way the MFM was running; they were able to get fresh fruit and vegetables with minimal problems and wanted to see the program continue. They did share a few recommendations related to the market process they felt could improve their experience. Their ideas of a better experience included some process-related concerns such as being able to touch the produce themselves (a regulation was still upheld from peak COVID safety) and assuring the MFM had enough produce to provide for each stop during the day. They also recognized differences in the way staff would handle produce and requested that staff be trained, so as to not put tomatoes at the bottom of the bag or to know the difference between cucumber and zucchini. These ideas are further elaborated in the evaluation report (Appendix E). These points of improvement led the evaluator to question training and change making. In informal conversation with staff, they mentioned having ideas about training and program change that had no formal outlet. To further explore program reality, the evaluator conducted interviews with all staff to learn the differences between how the staff and the ED were conceptualizing the program’s design and effectiveness. In these interviews, staff members shared ideas about how changes in market operations related to produce quality, product availability, and the transaction process could improve the customer experience. They mentioned, though despite vastly different backgrounds (some staff being nutrition educators, and others having no prior experience in food), there was no training involved with becoming a staff member at the MFM aside from instruction at the market. Additionally, staff members shared that they felt overburdened due to 36 an unfilled position for the MFM manager, an important role that creates market cohesion and organization, which made more work for the team and impacted team effectiveness. Overall, staff were aware of the constraints of the team due to high turnover and little training or formal knowledge handover. They felt some of these problems could be addressed by making small and feasible changes in planning and operations, but they did not feel there was an appropriate way to communicate that feedback to the ED that would be heard and acted upon. Following the market season, the evaluator hosted a meeting with the staff (and without the ED, as she agreed the staff may not share their concerns freely if she was present) to co- create a logic model to articulate the MFM from their perspective (Appendix F). The two logic models (one created with the ED and the other with the staff) illuminate the disconnect between the staff and ED perspectives about MFM operations. The evaluator used the juxtaposition of these two visual representations of the intended program reality to question the underlying logic of the staff and the ED. The Venn Figure 2.2 summarizes similarities and differences between the two logic models. Figure 2.2: Venn diagram summarizing the similarities and differences between ED- and staff- created logic models Unsurprisingly, both groups want to see the program continue. They also agree that the primary goal of the MFM is to increase food access. But their vision for what this looks like differs. While the ED envisions the program running like a neighborhood grocery store, i.e. providing material goods through transactional exchange, the staff pictures an experience more 37 like a farmers market, i.e. including local foods, relationship building, and even farmer interactions. The logic models also revealed that the ED’s perception is rooted in administrative and legal details, which makes sense given her responsibilities. Her role includes securing grant funds and determining what is possible within grant funding, organizational mission, and partner agreements. While she also works and interacts with customers at every market, she is also responsible for the behind the scenes function of the organization. Staff, on the other hand, are focused on program implementation and customer satisfaction, as their primary responsibilities are related to market logistics and customer interface. Since most of the staff have only been in their positions for one or two seasons, they do not have a historical perspective about what has worked and what has changed over the last seven years, though they do have fresh energy and are excited to bring new ideas to the table. All of this information about diverging perspectives, insight, and visions is important because the ED shared that she hoped to retire at the end of the season, but the staff that they felt they lacked important organizational information to confidently take over market operations. 4. Assess Plausibility of the Program During this step we to create a preliminary findings report for stakeholder review. Interview transcripts and observation notes were both deductively and inductively coded (Miles et al., 2020). The deductive codes related to documented barriers to farmers markets and were used to answer the research questions in Chapter 1. The inductive codes emerged from customer experiences and were used in the evaluation to contribute to the program in practice. Further, we looked across all data sources (interviews, observations, and documents) to understand if the program goals could be reached given the program activities. Comparing the program in principle with the program in practice, we can see that the program is doing what it says it will do. Customers are satisfied with their ability to obtain fresh produce from the MFM. However, due to staff discontent and a prospective change in leadership, the program may not be sustainable into the future. In this comparison process, we determined that the dynamic between the staff and the ED caused the staff to not feel comfortable discussing or enacting change, which was impacting the long-term plausibility of this program. Additionally, with the central goal to continue the program when the ED retired, a shared understanding of how to run the program was essential. To address the differences in management ideas and the lack of 38 communication between the staff and the ED, we decided to prioritize creating communication pathways between the groups, starting with an all staff meeting to build understand of program goals. 5. Discuss Any Changes Findings were shared with the staff and ED during a 2.5 hour meeting facilitated by the primary evaluator; a second researcher took notes and conducted observations about discussion content and staff interactions during the meeting. At the beginning of the meeting, participants were asked to discuss each of the findings shared in a preliminary report and create recommendations that could be implemented in future market seasons. These recommendations were recorded using a facilitation tool called rotating flip charts or carousel adapted by the evaluator from work by Stevahn and King (2016) and Michigan State University Extension (2015). In this activity, staff are broken up to work individually or in partners to work through recommendations one at a time, documenting ideas on a poster for a short period of time and then rotating to add ideas to the next poster until each small group had an opportunity to record responses on each poster. Titles for these posters reflected categories of opportunities for growth identified during data analysis (training, communication with staff, communication with customers, pricing, quality, availability, and market flow). Then the group re-gathered to work together to prioritize these categories. Following the meeting, the evaluator analyzed the staff recommendation list to identify and streamline higher-level themes related to program improvements. This analysis was shared with the ED and the staff in a final report (Appendix E). Observation data collected during this meeting verified the previous conclusions about the ED and staff’s differing perceptions about the MFM purpose and operations. Some of this divergence is rooted in their different roles (i.e. administrator or implementor). For example, during the meeting, when the staff explained programmatic changes they desired, the ED responded by detailing regulations or administrative challenges that precluded the changes. Because the staff was not often included in decision making beyond the day-to-day activities, they did not have this insight. In this exchange the staff felt heard and learned more about how and why decisions are made. One of the highest priorities for organizational change that emerged during the meeting was that more extensive training for the staff was needed. Deeper involvement in MFM operations would ensure that their feedback could be useful and relevant given organizational resources and constraints. Another top priority was the development of 39 structured communication strategies such as planning and check-in meetings. During the meeting the staff identified the root of many of their frustrations was that they did not know the mission or what was in the bounds of the program philosophy. In making assumptions to fill that void of understanding, their vision misaligned with that of the ED. The meeting was a useful platform to discuss how they collectively might best use the evaluation information in the future. 6. Discuss Future Evaluation Possibilities Throughout the findings meeting, the observer noted that the ED was unsurprised about the findings and somewhat dismissive of staff recommendations. The evaluator met with the ED individually to debrief the findings meeting and to co-generate some ideas about how the evaluation findings might be useful for future operations. In this meeting, the ED shared that she felt overwhelmed with her responsibilities and concerned that the staff did not have the experience to help with decision-making. The evaluator used this information from the ED to inform the final report and recommendations. Along with the list of recommendations from the staff, the evaluator provided higher level insight to diminish some of the worries from the ED. This included sections such as what to do with “tasks outside the scope of the [program]”, “tasks that sound overwhelming”, and “tasks that sound tedious”. After the final report was distributed, the ED allowed staff to create agendas for future meetings to prepare for market and decided to revisit the mission with each meeting to help staff gain a rooted understanding of their shared purpose. Discussion The evaluability assessment results are useful in two ways. First, the feedback from both customers and staff is useful as preliminary data to inform changes in future MFM seasons. Second, the results can be used to further develop the reality of the program in practice and continue to work toward alignment with the program in principle. This comparison is an important part of determining a program’s evaluability and creating realistic and usable future evaluation plans. Toward this end, we will use the four standards of evaluability (Wholey et al. 2015, p. 90-91) to discuss the evaluability of the MFM: (1) agreed upon and realistic program goals, (2) well-defined program goals, (3) obtainable evaluation data, and (4) the willingness and ability of staff to use evaluation information. 40 Agreed Upon and Realistic Program Goals As discussed thus far, the program goals were not previously agreed upon. Though it is the task of the ED to create the program in principle, it is also required to communicate them to program implementors. This does not mean necessarily that the staff must agree with every decision made, but the staff in this program felt empowered by at least knowing how and why certain programmatic decisions were made. The organization is currently undergoing changes for the upcoming season using evaluation information to better align the program mission with training opportunities. The program goals are mostly realistic. Their activities align with their desired outcomes of continued funding, reaching more customers, and increasing neighborhood presence. However, one aspect about mobile farmers markets in general that is presented in the logic models is the idea that the MFM has the outcome or impact of increased, long-term food access such as brick and mortar stores in the neighborhoods they serve. This is not supported by the activities of the MFM and is something that should be recognized by the organization as outside the scope of the program. Conversely, activities could be altered to better reflect this desired outcome. Gaining clarity on this point would assist in defining information needs to meet long- term outcomes. Well-Defined Program Goals The understanding of program goals was guided by the logic models created by the organization, and further supplemented by the findings meeting observations. Collectively, the group decided that moving forward, educating the staff about the philosophy or driving purpose of the market would be a priority. This would allow staff to make recommendations within the bounds of what the program is intended to do. This side-by-side also allows us to see that in general, both the staff and ED agreed by setting the highest priority as getting fruits and vegetables in the hands of the target customers. This common ground can be used as a point in future for planning and to find consensus on program goals. Other program goals were not as well defined. There were varying versions of well-defined goals, which became clear throughout this process. The findings meeting was a step toward aligning clarifying where goals were not communicated. Future meetings could work on the definition and collective shaping of program goals. 41 Obtainable Evaluation Data Evaluation data sufficient to answer the evaluation questions here are obtainable. This mobile market focuses on customer satisfaction; thus customer feedback is the most relevant data. Data in terms of transactions, inventory, and customer counts were not reliable. In some cases, inadequate documentation of data can be problematic (Davies and Payne, 2015), however the market is not relying on or asking for that data at this time. If the MFM wanted to ask questions about that data in the future, they should seek to improve consistency in transaction documents. But “there is no right set of performance measures” (Wholey et al. 2015, p. 97), as long as they are not being requested by a funding agency. Many mobile farmers markets have attempted to evaluate based on amount of fruits and vegetables consumed (Hsiao et al., 2019), however there is no standardized method of doing so in the literature and this is a challenging outcome to quantify beyond self-reporting. Willingness and Ability of Staff to Use Evaluation Information The staff are excited to use this information, as demonstrated in their enthusiasm for making recommendations. However, the organization has limited staff and time to enact these changes. Therefore, the challenge will be in integrating these changes through the change in leadership. Overall Evaluability and Recommendations for Future Evaluation In general, the program is evaluable. However, before moving forward with future evaluation, we recommend alignment be found between the staff and the ED first. Specifically, ways that these results could be used include (1) a meeting to discuss staff recommendations to detail what is and what is not possible - and why, (2) a meeting to build out the specific tasks required to make desired changes for the priorities created in the findings meeting, (3) delegation of these tasks, and (4) co-creation of the logic model using both perspectives and a short-term (3- 5 years) plan to align program activities, outputs, and outcomes. To name the type of evaluation to move forward beyond these activities, would require the organization to seek further information. As this evaluation led to internal process work rather than further evaluation questions, further evaluation is not necessary at this time. If the organization wanted to pursue further evaluation when they feel confident about their shared program goals and plan, they could develop new research questions at that time. Alternatively, if they wanted to check in on 42 the progress of their changes made relating to customer experience, it may be helpful to conduct another short, summative evaluation with customer interviews. Conclusion Recommendations created by the staff in the findings meeting and by the evaluator regarding evaluability are useful outcomes of the evaluation, though we believe the deeper outcomes of this evaluation are in staff capacity and enhanced communication. By involving staff in the logic model, preliminary findings, and recommendation processes, evaluability assessment plants the seed for evaluative thinking. These staff members, who may not otherwise be heard, had the opportunity to learn and then apply evaluation knowledge to their own work and learn about the changes they wanted to see in the program. Patton (2018) emphasizes the importance of seeing evaluation capacity building beyond the extent of completing the evaluation. This level of engagement allows the staff to “incorporate evaluative thinking into everything they do as part of ongoing attention to mission fulfillment and continuous improvement” (Patton, 2018). This process focuses on the empowerment of voices of program implementors, which in this case were drowned out by a lack of communication channel to the administrative decision maker. This level of staff involvement may not have occurred if we were not looking to make recommendations outside of the “issue area” or beyond the evaluation questions that were deemed important to start with (Hendricks & Papagiannis, 1990). Using this exploratory type of evaluation allowed us to promote usability with “stakeholder engagement [sic.] flexible enough to permit modifications to the project and/or the evaluation based on stakeholder feedback” (Thomas & Campbell, 2020). It also encourages us to not only ask if the program works but also how it works and why (Schwandt, 2015 p. 38). The use of evaluability assessment allowed exploration and streamlining of these multiple realities that will be helpful for the organization moving forward. 43 BIBLIOGRAPHY Alkon, A. (2008). Paradise or pavement: The social constructions of the environment in two urban farmers’ markets and their implications for environmental justice and sustainability. Local Environment, 13(3), 271–289. https://doi.org/10.1080/13549830701669039 Alkon, A., & McCullen, C. G. (2011). Whiteness and Farmers Markets: Performances, Perpetuations … Contestations? Antipode, 43(4), 937–959. https://doi.org/10.1111/j.1467-8330.2010.00818.x Arcadia Center for Sustainable Food & Agriculture. (2012). Mobile markets: Applying the food Truck model to food access. http://arcadiafood.org/sites/default/files/files/arcadia%20%20mobile%20market%20repor t%20Digital.pdf Bell, C., Kerr, J., & Young, J. (2019). Associations between Obesity, Obesogenic Environments, and Structural Racism Vary by County-Level Racial Composition. International Journal of Environmental Research and Public Health, 16(5), 861. https://doi.org/10.3390/ijerph16050861 Best, A. L., & Johnson, J. L. (2016). Alternate Food Markets, NGOs, and Health Policy: Improving Food Access and Food Security, Trust Bonds, and Social Network Ties. World Medical & Health Policy, 8(2), 157–178. https://doi.org/10.1002/wmh3.190 Byker, C., Shanks, J., Misyak, S., & Serrano, E. (2012). Characterizing Farmers’ Market Shoppers: A Literature Review. Journal of Hunger & Environmental Nutrition, 7(1), 38– 52. https://doi.org/10.1080/19320248.2012.650074 Chadderdon Bair, R. (2022). Top 9 Reasons Why Michigan Farmers Markets Lead the Pack. Fair Food Network. https://fairfoodnetwork.org/from-the-field/top-9-reasons-why-michigan- farmers-markets-lead-the-pack/ Chilton, M., Black, M. M., Berkowitz, C., Casey, P. H., Cook, J., Cutts, D., Jacobs, R. R., Heeren, T., de Cuba, S. E., Coleman, S., Meyers, A., & Frank, D. A. (2009). Food Insecurity and Risk of Poor Health Among US-Born Children of Immigrants. American Journal of Public Health, 99(3), 556–562. https://doi.org/10.2105/AJPH.2008.144394 Davies, R., & Payne, L. (2015). Evaluability Assessments: Reflections on a review of the literature. Evaluation, 21(2), 216–231. https://doi.org/10.1177/1356389015577465 Dulin, A., Mealy, R., Whittaker, S., Cardel, M., Wang, J., Risica, P. M., & Gans, K. (2022). Identifying Barriers to and Facilitators of Using a Mobile Fruit and Vegetable Market Intervention Delivered to Low-Income Housing Sites: A Concept Mapping Study. Health Education & Behavior, 49(1), 159–168. https://doi.org/10.1177/1090198121998287 Etikan, I. (2016). Comparison of Convenience Sampling and Purposive Sampling. American Journal of Theoretical and Applied Statistics, 5(1), 1. https://doi.org/10.11648/j.ajtas.20160501.11 Fair Food Network. (2022). Healthy Food for Michiganders & More Business for Local Farmers. Double Up Food Bucks Michigan. https://doubleupfoodbucks.org/ 44 FAO. (2021). The State of Food Security and Nutrition in the World 2021. https://doi.org/10.4060/CB4474EN Feeding America. (2021). Hunger in America. https://www.feedingamerica.org/hunger-in- america Feeding America. (2023). Overall (all ages) Hunger & Poverty in the United States | Map the Meal Gap. https://map.feedingamerica.org Fish, C. A., Brown, J. R., & Quandt, S. A. (2015). African American and Latino Low Income Families’ Food Shopping Behaviors: Promoting Fruit and Vegetable Consumption and Use of Alternative Healthy Food Options. Journal of Immigrant and Minority Health, 17(2), 498–505. https://doi.org/10.1007/s10903-013-9956-8 FNS. (2022). Farmers’ Markets Accepting SNAP Benefits Food and Nutrition Service. https://www.fns.usda.gov/snap/farmers-markets-accepting-snap-benefits Freedman, D. A., Vaudrin, N., Schneider, C., Trapl, E., Ohri-Vachaspati, P., Taggart, M., Ariel Cascio, M., Walsh, C., & Flocke, S. (2016). Systematic Review of Factors Influencing Farmers’ Market Use Overall and among Low-Income Populations. Journal of the Academy of Nutrition and Dietetics, 116(7), 1136–1155. https://doi.org/10.1016/j.jand.2016.02.010 Gusto, C., Diaz, J., Warner, L., & Monaghan, P. (2020). Advancing Ideas for Farmers Market Incentives: Barriers, Strategies, and Agency Perceptions from Market Managers. Journal of Agriculture, Food Systems, and Community Development, 9(3), Article 3. https://doi.org/10.5304/jafscd.2020.093.022 Hendricks, M., & Papagiannis, M. (1990). Do’s and Dont’s for Offering Effective Recommendations. Evaluation Practice, 11(2), 121–125. https://doi.org/10.1177/109821409001100205 Hendrickson, M. K. (2020). Covid lays bare the brittleness of a concentrated and consolidated food system. Agriculture and Human Values, 37(3), 579–580. https://doi.org/10.1007/s10460-020-10092-y Hinrichs, C. C. (2000). Embeddedness and local food systems: Notes on two types of direct agricultural marketଝ. Journal of Rural Studies, 9. Hsiao, B., Sibeko, L., & Troy, L. M. (2019). A Systematic Review of Mobile Produce Markets: Facilitators and Barriers to Use, and Associations with Reported Fruit and Vegetable Intake. Journal of the Academy of Nutrition and Dietetics, 119(1), 76-97.e1. https://doi.org/10.1016/j.jand.2018.02.022 Janhonen, K., Torkkeli, K., & Mäkelä, J. (2018). Informal learning and food sense in home cooking. Appetite, 130, 190–198. https://doi.org/10.1016/j.appet.2018.08.019 Klisch, S., & Soule, K. (2020). Farmers Markets: Working with Community Partners to Provide Essential Services During COVID-19. Journal of Agriculture, Food Systems, and Community Development, 9(4), Article 4. https://doi.org/10.5304/jafscd.2020.094.035 45 Lam, S., & Skinner, K. (2021). The Use of Evaluability Assessments in Improving Future Evaluations: A Scoping Review of 10 Years of Literature (2008–2018). American Journal of Evaluation, 42(4), 523–540. https://doi.org/10.1177/1098214020936769 Lambert‐Pennington, K., & Hicks, K. (2016). Class Conscious, Color-Blind: Examining the Dynamics of Food Access and the Justice Potential of Farmers Markets. Culture, Agriculture, Food & Environment, 38(1), 57–66. https://doi.org/10.1111/cuag.12066 Lang, T., & Barling, D. (2012). Food security and food sustainability: Reformulating the debate. The Geographical Journal, 178(4), 313–326. https://doi.org/10.1111/j.1475- 4959.2012.00480.x Lowery, B., Sloane, D., Payán, D., Illum, J., & Lewis, L. (2016). Do Farmers’ Markets Increase Access to Healthy Foods for All Communities? Comparing Markets in 24 Neighborhoods in Los Angeles. Journal of the American Planning Association, 82(3), 252–266. https://doi.org/10.1080/01944363.2016.1181000 Michigan State University Board of Trustees. (2015). Facilitative Leadership Participant’s Guide. Miles, M. B., Huberman, A. M., & Saldaña, J. (2020). Qualitative data analysis: A methods sourcebook (Fourth edition.). SAGE. Mino, R., Chung, K., & Montri, D. (2018). A look from the inside: Perspectives on the expansion of food assistance programs at Michigan farmers markets. Agriculture and Human Values: Journal of the Agriculture, Food, and Human Values Society, 35(4), 823. https://doi.org/10.1007/s10460-018-9877-1 Northwest Initative. (2023). About. http://nwlansing.org/about/ Patton, M. Q. (2018). Facilitating Evaluation: Principles in Practice. SAGE Publications, Inc. https://doi.org/10.4135/9781506347592 Patton, M. Q., & Campbell-Patton, C. (2022). Utilization-Focused Evaluation (5th Edition). Thousand Oaks: Sage. Payne-Sturges, D. C., Tjaden, A., Caldeira, K. M., Vincent, K. B., & Arria, A. M. (2018). Student Hunger on Campus: Food Insecurity Among College Students and Implications for Academic Institutions. American Journal of Health Promotion, 32(2), 349–354. Ritter, G., Walkinshaw, L. P., Quinn, E. L., Ickes, S., & Johnson, D. B. (2019). An Assessment of Perceived Barriers to Farmers’ Market Access. Journal of Nutrition Education and Behavior, 51(1), 48–56. https://doi.org/10.1016/j.jneb.2018.07.020 Robinson, J. A., Weissman, E., Adair, S., Potteiger, M., & Villanueva, J. (2016). An oasis in the desert? The benefits and constraints of mobile markets operating in Syracuse, New York food deserts. Agriculture and Human Values: Journal of the Agriculture, Food, and Human Values Society, 33(4), 877–893. https://doi.org/10.1007/s10460-016-9680-9 Satin-Hernandez, E., & Robinson, L. (2016). A Community Engagement Case Study of the Somerville Mobile Farmers’ Market. Journal of Agriculture, Food Systems, and Community Development, 5(4). https://doi.org/10.5304/jafscd.2015.054.015 46 Schensul, J. J., & LeCompte, M. D. (2013). Essential Ethnographic Methods: A Mixed Methods Approach. Rowman Altamira. Schwandt, T. (2015). Evaluation Foundations Revisited: Cultivating a Life of the Mind for Practice. Stanford Business Books. https://search.ebscohost.com/login.aspx?direct=true&db=nlebk&AN=986112&site=eds- live Shannon, J. (2014). What does SNAP benefit usage tell us about food access in low-income neighborhoods? Social Science & Medicine, 107, 89–99. https://doi.org/10.1016/j.socscimed.2014.02.021 Slocum, R. (2007). Whiteness, space and alternative food practice. Geoforum, 38(3), 520–533. https://doi.org/10.1016/j.geoforum.2006.10.006 SNAP-Ed. (2022a). SNAP-Ed Connection. https://snaped.fns.usda.gov/ SNAP-Ed. (2022b). The Farmers Market Food Navigator Program – SNAP-Ed Toolkit. https://snapedtoolkit.org/interventions/programs/the-farmers-market-food-navigator- program/ SNAP-Ed at Michigan Fitness Foundation. (2022). SNAP-Ed at Michigan Fitness. https://snap- ed.michiganfitness.org/ Stevahn, L., & King, J. A. (2016). Facilitating Interactive Evaluation Practice: Engaging Stakeholders Constructively. New Directions for Evaluation, 2016(149), 67–80. https://doi.org/10.1002/ev.20180 Story, M., Kaphingst, K. M., Robinson-O’Brien, R., & Glanz, K. (2008). Creating Healthy Food and Eating Environments: Policy and Environmental Approaches. Annual Review of Public Health, 29(1), 253–272. https://doi.org/10.1146/annurev.publhealth.29.020907.090926 Strosberg, M. A., & Wholey, J. S. (1983). Evaluability Assessment: From Theory to Practice in the Department of Health and Human Services. Public Administration Review, 43(1), 66– 71. https://doi.org/10.2307/975301 Taylor, D. E., Lusuegro, A., Loong, V., Cambridge, A., Nichols, C., Goode, M., McCoy, E., Daupan, S. M., Bartlett, M., Noel, E., & Pollvogt, B. (2021). Racial, Gender, and Age Dynamics in Michigan’s Urban and Rural Farmers Markets: Reducing Food Insecurity, and the Impacts of a Pandemic. AMERICAN BEHAVIORAL SCIENTIST. https://doi.org/10.1177/00027642211013387 Thomas, V. G., & Campbell, P. B. (2020). Evaluation in Today’s World: Respecting Diversity, Improving Quality, and Promoting Usability. https://eds.p.ebscohost.com/eds/ebookviewer/ebook/bmxlYmtfXzMzNjE2MDBfX0FO0? sid=ec321b7a-a78c-4b3e-b339-7ed63e24a203@redis&vid=5&format=EB&rid=1 USDA ERS. (2022). Food Security in the U.S. Measurement. https://www.ers.usda.gov/topics/food-nutrition-assistance/food-security-in-the-u- s/measurement/ 47 USDA FNS. (2022a). A Short History of SNAP | Food and Nutrition Service. https://www.fns.usda.gov/snap/short-history-snap#1939 USDA FNS. (2022b). Double Up National Network. SNAP Education Connection. https://snaped.fns.usda.gov/library/materials/douple-national-network USDA FNS. (2022c). Farmers Market Nutrition Program. https://www.fns.usda.gov/fmnp/wic- farmers-market-nutrition-program USDA FNS. (2022d). Seniors Farmers’ Market Nutrition Program. https://www.fns.usda.gov/sfmnp/senior-farmers-market-nutrition-program USDA FNS. (2022e). SNAP Policy on Non-Citizen Eligibility. https://www.fns.usda.gov/snap/eligibility/citizen/non-citizen-policy Warsaw, P., Archambault, S., He, A., & Miller, S. (2021). The Economic, Social, and Environmental Impacts of Farmers Markets: Recent Evidence from the US. Sustainability, 13(6), Article 6. https://doi.org/10.3390/su13063423 Webber, C. B., Sobal, J., & Dollahite, J. S. (2010). Shopping for fruits and vegetables. Food and retail qualities of importance to low-income households at the grocery store. Appetite, 54(2), 297–303. https://doi.org/10.1016/j.appet.2009.11.015 Wheaton, B. (2018). Bridge Card holders can double their purchasing power at farmers markets. https://www.michigan.gov/mdhhs/inside-mdhhs/newsroom/2018/06/19/bridge- card-holders-can-double-their-purchasing-power-at-farmers-markets_2 Wholey, J., Hatry, H., & Newcomer, K. (2015). Handbook of Practical Program Evaluation (1st ed.). John Wiley & Sons, Ltd. https://doi.org/10.1002/9781119171386 Wolf, M. m., Spittler, A., & Ahern, J. (2005). A profile of farmers’ market consumers and the perceived advantages of produce sold at Farmers’ Markets. Journal of Food Distribution Research, 36(1), 192–201. Ylitalo, K. R., During, C., Thomas, K., Ezell, K., Lillard, P., & Scott, J. (2019). The Veggie Van: Customer characteristics, fruit and vegetable consumption, and barriers to healthy eating among shoppers at a mobile farmers market in the United States. Appetite, 133, 279–285. https://doi.org/10.1016/j.appet.2018.11.025 48 APPENDIX A: CUSTOMER INTERVIEW GUIDE • Any questions before we get started? Mobile Market Experience • How frequently do you go to the market? o How long have you been attending the market? • Please tell me what your grocery shopping routine looks like? Where do you usually shop in addition to the mobile market? o Why do you go where you go? o What do you enjoy about grocery shopping? o What do you not enjoy? • Why do you attend the market in addition to these other shopping locations? How does this experience differ from those experiences? o How do you get to the market? Is it convenient for you to attend the market? o Do you think the prices are fair? Do they differ from your other shopping locations? o Do you see/interact with other people when you visit the market? o Do you use food assistance at the mobile farmers market (SNAP, Senior project fresh) o What is it like using your food assistance at the MFM? • Have you been to another farmers market? Why/why not? o How does this market compare? Would you attend other markets after visiting this one? • Have you interacted with staff at the MFM? o What was your experience like? How would you describe those interactions? • Do you interact with other customers? o How would you describe those interactions? Produce Use (Culturally/Preferentially Appropriate) • Does the market have the foods that you want to buy? • What are your favorite foods to purchase at the mobile o Is there anything you would like to see at the mobile farmers market? 49 • How do you prepare the food you get (eat it fresh, cooked, preserve for later)? Who are you buying and/or cooking for? • Where do you get information about how to prepare new or different items? Perception/Interest in Fresh Produce • Have you purchased foods from the mobile market that are new to you? o What made you try the new food? • Do you feel like the MFM is a healthy shopping experience? o What does healthy food mean to you? Market Function • Are you familiar with the other programs that NWI hosts? Fresh conversations, food distribution, etc? o Have you participated in these programs? Are there other resources or programs you would be interested in? • Is there anything else related to the market you would like to share? Feedback? • Do you have any questions for me regarding the research process or this interview? Knowledge Share I’d like to make a cookbook to share with the MFM community. Do you have any recipes you would be willing to share to contribute? 50 APPENDIX B: STAFF INTERVIEW GUIDE • How long have you been working for NWI? How long have you been involved with the mobile farmers market? • Describe the Mobile Farmers Market program • Overall, do you think it serves the customers well? Do you have any stories to share about customer success with the MFM? • Do you have any stories about customer displeasure? What caused this? How was that handled? • Think about the MFM process from a broad view, e.g., the entire season. Is there anything you would change? o To make it more beneficial for customers? o To make the season run more smoothly (behind the scenes)? o To ensure longevity and sustainability of the program? • Now think more narrowly about the day-of process. Is there anything you would change to make it more beneficial for customers? o To make the market more beneficial for customers? o To make the day run more smoothly? o To ensure longevity and sustainability of the program? • In what ways is the market doing well that should be continued? • Anything else we haven’t discussed that you would like to share? 51 APPENDIX C: THEORY OF CHANGE Many communities in Lansing do not have access to fresh fruits and vegetables necessary for a healthy diet. Inequality in the food system caused by social determinants such as wealth, age, race, and citizen status in the moment causes food insecurity and overtime leads to health disparities. This is especially true of seniors and any others who face difficulties in finding transportation to the grocery store. The mobile market overcomes transportation as a barrier by bringing local and non-local fresh produce and other goods to four low-income housing areas in Lansing. Price, another typical food access barrier, is addressed by accepting food assistance programs and offering the best price possible while maintaining the program. Further the market is committed to providing nutrition education and other services to assist community members in applying for food assistance, health coverage, and more benefits they may are available to them. 52 APPENDIX D: EXECUTIVE DIRECTOR LOGIC MODEL Figure D.1: Logic model created from the perspective of the executive director 53 APPENDIX E: EVALUATION REPORT NorthWest Initiative: Lansing Mobile Farmers Market Evaluation Findings Report Author: Angel Hammon Contributors: NWI Staff Figure E.1: Evaluation report distributed to the organization 54 Figure E.1 (cont’d) Table of Contents Page 3 Introduction Overview Objectives Approach Page 4 Preliminary Findings Overall Findings Points of Celebration Opportunities for Growth Page 8 Recommendations Idea Generation Prioritization Logic Model Conclusion References Appendices Intended Audience and Purpose This report describes the findings from an evaluation of the 2022 Lansing Mobile Farmers Market (LMFM) season. These findings can provide a jumping off point for staff discussion and future planning. This report is intended to spark reflection on the market purpose and function with the goal of supporting continued success of the LMFM. 55 Figure E.1 (cont’d) Introduction Overview NorthWest Initiative has been bringing healthy food to Lansing residents for seven years through the Lansing Mobile Farmers Market (LMFM). This program has adapted over time to best serve the target demographic of low-income and minority customers, particularly in Lansing food deserts. The LMFM provides customers the opportunity to purchase fresh produce with Electronic Benefit Transfer (EBT), Senior Farmers Market Nutrition Program (SFMNP), WIC Project FRESH, and Double Up Food Bucks (DUFB). Though the market has been successfully running for many seasons, there has not yet been a professional evaluation beyond the reporting required by funding agencies. Objectives This evaluation explores the ways the LMFM serves its purpose in the context of NWI and in the broader Lansing food system by asking those closest to the program (customers, staff) about areas of strength and opportunities for improvement. With the executive director, the following questions were created to guide the evaluation activities: ● In what ways are the services provided by the LMFM useful to its customers? ● How could the LMFM improve these services? Approach This evaluation uses an approach called an evaluability assessment, which is the first step toward evaluating a program that allows stakeholders to learn about if and how a program should be evaluated before committing to a more elaborate - and more expensive - evaluation. The evaluability assessment allows an evaluator and those involved to compare how a program is presented on paper with how it actually runs, with the goal of greater alignment. To understand the LMFM process, I conducted: (1) an analysis of market documents, (2) 11 participant interviews, (3) 8 staff interviews, and (4) rigorous observations and field notes during the market. This was followed with qualitative content analysis. Findings from this analysis are presented below in the Overall Findings section. I also gathered staff input during an after-market debrief and analyzed sticky notes created during the logic model development and findings meetings. These were used to create the logic models and evaluation recommendations in the sections below. 56 Figure E.1 (cont’d) Preliminary Findings Overall Findings Preliminary findings show that customers are overall very happy with the market. Most feedback about how to improve the LMFM relates to the internal structure of the market and maintaining the longevity of market services into the future. The tables below summarize the findings across all data sources: (a) document analysis, (b) observation, (c) participant interviews, and (d) staff interviews. Table 1 details the things participants most appreciate and value about the market. These are important to note as they are what bring customers back year to year. Table 2 details opportunities for strengthening the longevity and function of the market into the future, which can guide LMFM planning. 57 Figure E.1 (cont’d) Table 1: Points of Celebration Category Theme Description The LMFM serves the community by bringing produce directly to the customer. In this case, the Transportation transportation barrier between customers and fresh produce is removed. For the population the market serves, accessibility Convenience in terms of mobility is very important. The market does a service by getting as close to the customers as possible. The staff goes beyond that by assisting Mobility customers by putting groceries in their wheelchairs or walkers, making sure they have a clear path, and in some cases, bringing their groceries all the way to their apartment. Weekly interaction gives the staff and customers the ability to build meaningful relationships, some of which are upkept outside of the market. They build With Staff trust this way, which contributes to the overall experience of the market and helps in encouraging Community customers to try new foods or ways of preparing Building them. Community members are interacting with each Among other. Sharing food and conversation about fresh Customers fruits and vegetables they are going to buy. The LMFM caters to the use of food assistance. Though there is a learning curve, customers feel comfortable using food assistance without fear of Food judgment. The staff work hard to make the best use Assistance of food assistance and maximize the amount of Usability fresh produce they can buy. In this way, the barrier of exclusivity or discomfort using food assistance is removed. The customers are trying new things throughout the market. Trust and knowledge are big contributors to Trying New this. Staff knowledge about how to prepare these Things items can be employed when encouraging customers to try new things. Broadly, the market successfully gets produce in the hands of customers by overcoming barriers that would typically be in their way, including transportation and mobility. Once at the 58 Figure E.1 (cont’d) market, the customers feel a sense of community both with the staff and each other. This is important not only because of the age group of most customers, which often struggles with social isolation, but also because the market remained in operation during the height of the COVID pandemic, which exacerbated isolation and loneliness for many groups. From the perspective of participants, the market is doing a great job and they would like to see it continue. 59 Figure E.1 (cont’d) Table 2: Opportunities for Growth Category Theme Description Staff knowledge contributes to the experience of the shopper. The background of the staff at the market is General not always in food, so training is essential for the confidence and ability of the staff. The market is understaffed. The outreach position is not filled, which means no staff member’s sole Training responsibility is the market. When everything is everyone's job, the work is not guaranteed to get done. Specified There are certain tasks that all staff members should Market Roles be trained to do in case it is necessary (displaying produce, transactions) but there should be some guidelines so staff feel comfortable and confident in their position. There are not many viable channels of communication available for staff to bring up concerns or make Communication change. All staff members mentioned having more with Staff meetings could create a consistent conversation where all are heard and included. About Participants are not generally aware of the programs available and services that are available at the market including services, NWI as a whole. Many customers are not aware that other NWI is hosting the market or why. programming, Communication and NWI with Customers Customers would like more notice about the market. Almost all customers interviewed are used to Advertisement communication by flier, a few by Facebook. In any case, this is a large part of the preparation process for the market. The market currently runs in a for-profit manner. The customers have no choice but to pay the market’s Pricing prices. Staff are interested in applying for grants that could further offset costs and bring prices down for participants. 60 Figure E.1 (cont’d) Category Theme Description Since COVID, the market has been set up in a no-touch arrangement in which customers must stand behind cones and are not allowed to inspect their produce. Customers Multiple customers interviewed have a hard time with Handling this as they cannot perceive the quality from far away or Produce Quality just by looking. This is also controversial from the staff perspective as some want customers to wear gloves and others do not. There is not a consistent process for produce before and after the market. This allows for some produce to be left Process for behind or it remains in the office until the next week, both Produce of which lead to waste. Some items kept after a week are not of standard quality. There is no process for holding back certain items for customers at later stops. This leads to competition Availability amongst customers or worry that they will not get what they need even though the market technically supplies it. The market process varies from day to day without specific guidelines. Customers and staff are able to go with the function, but it may cause confusion. Without an Market Flow efficient or explicit process, the staff must expend energy figuring out what to do day to day rather than making broader improvements. Table 2 details opportunities for NWI to ensure the longevity of the LMFM. This was used as a jumping off point for group discussion so staff could co-create solutions that make sense for the needs, resources, and expectations of all involved in the preliminary findings meeting. Three main areas for intervention arose: (1) Staff Experience, including attention to training, expectations, and consistent communication channels to voice concerns and share ideas; (2) Customer Experience, including attention to communications so customers are aware of all market services, as well as opportunities to interact with NWI in other programs; and (3) Market Function, including clear protocol for produce handling to eliminate waste, specified market roles, and understanding of the transaction process to increase efficiency. 61 Figure E.1 (cont’d) Recommendations On February 27th, 2023, staff gathered to make collective sense of the preliminary findings and co-create recommendations. The group concluded this meeting by outlining a priority list of tasks to attend to. These ideas can guide the planning process for the 2023 market season as well as provide the foundation for longer-term goals in future seasons. Below are the staff- generated tasks and ideas for the LMFM created in response to each of the items in the Opportunities for Growth table (Table 2). Idea Generation All of these ideas do not need to be implemented right away! There is a wealth of information here, and some ideas might need to be adapted, tabled for the future, or discarded based on feasibility and program needs. This list is provided as a resource to return when there is time and motivation to evolve the program. - Tasks outside the scope of LMFM demonstrate an opportunity to be transparent with staff about what is possible, and not, and why. The more they know about the how, why, what of the market, the more their feedback will be relevant and implementable. - Tasks that sound overwhelming present an opportunity to scaffold change. For example, a number of participants expressed an interest in more staff meetings, communication, and training. Their eagerness to know what is going on demonstrates their commitment to LMFM activities. Creating opportunities for deeper inclusion and participation will be meaningful, and change toward this end can start small and build over time. - Tasks that sound tedious can be re-framed more broadly. For example, the market does not need to stock every item requested by customers (Table 3), but creating a chance for customers to offer feedback in a systematic way – e.g. on a white board at the market - can provide opportunities for conversation and inclusion. Training Volunteers • Instruction manual • Train to fill specific roles and/or recruit specific people/volunteers for the different roles Staff • Training manual o Create Standard Operating Procedure (SOP) so everyone is on the same page o Dry-run/role play before the season begins 62 Figure E.1 (cont’d) o Context for market mission, pricing philosophy, product selection, vendor selection, and guidelines for product reuse • Provide an opportunity to learn about produce before the market o What it is and how to prepare it o Fun recipes Staff Communication • Pre-meeting a day before the LMFM and post-meeting a day after the LMFM to discuss specific roles, responsibilities ,and plan Bs, as well as debrief what worked or could work better. • Weekly updates from farmers regarding inventory so this can be communicated to customers Customer Communication (+ Promotion) Customer communication • Pre and Post- season surveys o Pre-season produce preferences o Post-season feedback and suggestions • Provide an inventory list to customers • Make sure all programs are represented at each market (Fresh Conversations, cooking demonstrations, and other services offered by NWI) Possibly an NWI-specific table with information about the organization • Designate one person to communication with each site o Work with that site to decide the best way to communicate (email, phone, etc.) • Create a place at the market for customers to make requests, report back to customers on that list Promotion • More flyers going out to buildings; Email information to senior center managers to post in the lobby about market logistics • Create a social media schedule that includes what to post, where, and when before and after the market • Ask local news outlets and newspapers to promote LMFM 63 Figure E.1 (cont’d) • Create/Promote LMFM mission statement/goal • Promotional video on YouTube/Facebook Pricing • Value to customer o Grant funding to limit/reduce mark-up or even make “below market value” possible o Discuss the possibility of regulations or other implications that may come with more grant funding o More private donor funding • Working with vendors on discount pricing for volume/pre-order/seconds • Offer reduced “short-life” items at a deep discount • Bundle items for particular recipes with overall discount and pre-order (e.g. salsa kit, fruit salad, veg pasta salad) • Signage/Clearly displayed prices o Chalkboards/large boards or individual signs o Pricing list set and shared prior to the morning of the market • Grow as much as we can, intertwining Garden Club and LMFM. Those kids can help! • Donations from small groceries with promotion • Obtain cottage industry license for easy stuff we can sell at profit keeps the other prices low (e.g. cookies). Quality • Interact with local farmers to better understand how to store and use produce o Communicate to people about shelf life of certain items • Get produce that will last longer • Better systems for storage/transport/display o Create system for what to do with spoils • Improve process for moving items after each market. “Hold over items” should be standardized and based on quality • Use signs to convey the importance of washing produce Availability • Improve system to determine quantities of food items purchased from farmers or stores. 64 Figure E.1 (cont’d) o The goal is not to have left-over items at the end of the day o Maybe have residents pre-order “hot” commodities or highly perishable items • Meet with farmers for produce “schedule” and create “pre-shop” list • Give out a market produce schedule weekly or bi-weekly o Create a different “produce of the day” menu board like restaurants do o Maybe put both of these options online • Larger vehicle, more space for stuff • Multiple days for market or longer at each stop, focus on broadening market reach Market Flow • Specific roles • Create overview document so new folks can step into a new role • Timeline/checklist o Create minute by minute schedule of the market. How is each hour being performed/tracked. What tasks should be done when? • Start on time every week o Note how long it takes for the set up and take down at the sites • Improve check-out process. o Could incorporate “order” sheets that clients/shoppers fill out • Set up exactly the same way, every time at every location • Better communication with site managers about where we can set up • Eliminate Letts or promote Letts to use the time more productively Prioritization The list of priorities here ranks the larger categories of improvements that staff would like to make. Given more time, these priorities could be further teased out, for example taking another vote on the priorities with tied votes or going further to prioritize certain tasks within the categories. Two tasks were particularly important to the staff and should be addressed before this market season: 1) Hiring or designating a single person to be in charge of the market 2) Educating staff on the philosophy or driving purpose of the market 65 Figure E.1 (cont’d) Future Priorities: 1. Training 2. Communication with Staff and Market Flow 3. Communication with Customers 4. Pricing and Quality 5. Availability 6. Logic Model This logic model reflects the collective perspective of the LMFM executive director, staff, and participants. Keep in mind this model represents what those involved would like the market to do, not only what it presently does. This can be used as a guide for planning in the future and 66 Figure E.1 (cont’d) to evaluate how current activities connect (or not) to outcomes and impacts NWI hopes to achieve. For example, take note that the outcomes and impacts related to increasing food access through viable grocery stores in these neighborhoods does not come as a direct action of the LMFM. Specific steps would need to be taken to see this action occur through earlier outcomes such as awareness through increased neighborhood presence. The logic model can be altered, pared down, or expanded to fit organizational needs over time. Conclusion Using the evaluability assessment approach mentioned in the introduction, the final step of this process is to decide how to move forward with evaluation. This includes if the program is ready for further evaluation and how further evaluation could be useful. Wholey et al. (2015) explain four standards to determine if evaluation would be useful. These are 1) if the goals of the program are agreed upon and realistic, 2) if the information needs are well defined, 3) if the information the evaluation seeks is obtainable, and 4) if those involved in evaluation are willing and able to use the evaluation information. At this point the goals of the program are realistic, but they are not consistently communicated to all participating staff. These goals should guide the program broadly and inform everyday tasks at the market. For now, it seems that the information needed by the LMFM comes from the customers and the staff. This information is readily available and freely given. Any information of this kind (qualitative, meaning gathered through interviews/discussion) is surely obtainable. If there are any evaluation questions that come up regarding transactions, inventory or other numbers (quantitative), this data may not be obtainable as the market currently runs. This is apparent in the Training and Communication section in which the transaction and inventory process need improvement to obtain consistent and accurate information. Finally, staff are excited to have conversations about improving the market, however using the evaluation information may be limited due to staff time and capacity. This should be addressed before further resources could be designated to evaluating the mobile market. Recommendations For the Future 1. A meeting to discuss staff recommendations to detail what is and what is not possible, and why 67 Figure E.1 (cont’d) 2. A meeting to build out specific tasks for the two priority items to be done for this season 3. Delegate tasks to make headway toward those goals 4. Use the logic model to create a 2-year plan to strategize how and when to make early changes and 5-year plan to adapt and align activities, outputs, and outcomes. Supplementary Material Table 3 This table provides a list of items that customers mentioned wanting to see at the market. Table 4 This table provides a list of program suggestions from participants about the LMFM or NWI more broadly. 68 APPENDIX F: STAFF LOGIC MODEL Figure F.1: Logic model from the perspective of the staff 69