HOW SERVICE-LEARNING STUDENTS MAKE MEANING OF CIVIC LEARNING OUTCOMES: A MIXED METHODS APPROACH HIGHLIGHTING THE VOICES OF STUDENTS OF COLOR By Nicole Cheri Springer A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Higher, Adult, and Lifelong Education – Doctor of Philosophy 2023 ABSTRACT Service-learning has been a key pedagogical tool for over 50 years. During that time, research in the field moved from proving the worth and validity of the pedagogy to better understanding how student civic learning was impacted. That research often did not disaggregate the data by race and was therefore incomplete. The purpose of this study is twofold. The first is to use racially disaggregated data from the Civic Attitudes and Skills Questionnaire (Moely et al, 2002b) to compare the civic learning outcomes of students who completed a course with an integrated service-learning component. The second is to use qualitative methods to uncover how students make meaning of their civic learning through the statements used in the CASQ. Special attention was given to how student outcomes differ based on race both in survey responses and in their focus group conversations. Using multiple methods that intersect during the analysis of the data allow for a deeper understanding of the meaning students made of the survey statements. Copyright by NICOLE CHERI SPRINGER 2023 For my father M.Sgt. James E. Springer, USAF Ret. (8.28.1941 – 1.15.2021) Always my biggest cheerleader. You would be so proud. iv ACKNOWLEDGMENTS My professional and academic paths were not always linear. In addition to a changing work life, my personal life took some twists and turns that made for a difficult last couple of years. However, together with the help of those listed below, we made it through these changes, and I have come out on the other side forever changed and deeply grateful. I want to express my sincere gratitude to my dissertation chair and major advisor, Dr. Marilyn Amey, for always being the right parts listening ear, cheerleader, and taskmaster. You helped me think about my role as a practitioner-scholar in new and expansive ways. I am also grateful to my dissertation committee (Dr. Diane Doberneck, Dr. Terry Flennaugh, and Dr. Steve Wieland). Your feedback pushed me to dig deeper and reflect on how important this work is to the field and those professionals who will benefit from these insights. I want to acknowledge others in my life who were often my gravity and my glue. First, I am grateful to my family, both given and chosen, for loving me long before I decided to write this big book report. They may not have always understood why I do what I do, but they always helped me see a gentler truth… always saw me in a good light. Major thank you to my wife, Dr. Susan Grettenberger, who did so much to ensure we both made it through this Ph.D. You have been a constant source of support and love – and clean clothes. I appreciate your letting me talk things over with you even though I never let you read a word. Your influence is present in this labor of love. To others in my life who hold the titles of friends, neighbors, classmates, and other sources of love and kindness – it truly takes a village. You have each, in your own ways, helped me be better than I was. Thank you for allowing me to make you proud. v TABLE OF CONTENTS CHAPTER 1: INTRODUCTION ................................................................................................... 1 CHAPTER 2: LITERATURE REVIEW ...................................................................................... 13 CHAPTER 3: METHODS ............................................................................................................ 34 CHAPTER 4: RESEARCH FINDINGS ....................................................................................... 49 CHAPTER 5: DISCUSSION, IMPLICATIONS, AND CONCLUSION ..................................... 89 REFERENCES ........................................................................................................................... 107 APPENDIX A: QUALTRICS CIVIC ATTITUDES AND SKILLS QUESTIONNAIRE ......... 114 APPENDIX B: FOCUS GROUP PROTOCOL .......................................................................... 122 APPENDIX C: FACTOR DIFFERENCES IN MEAN VALUES .............................................. 125 APPENDIX D: REFLECTIONS ................................................................................................ 131 vi CHAPTER 1: INTRODUCTION Many quotes have been attributed to the emergence of service-learning throughout its history. As a community engagement professional at a Research 1 institution in the Midwest, we often used this quote attributed to Confucius, “I hear, and I forget. I see, and I remember. I do, and I understand.” These fifteen words capture the essence of service-learning as both a philosophy and a pedagogy. Situated under the larger umbrella of experiential education, service- learning integrates service within a community into the course curriculum through intentional reflection (Jacoby, 2015). As service-learning moved from the margins of faculty curriculum building to a celebrated high-impact practice for student learning (Kuh, 2008), it also became integral in answering the call to incorporate civic education more fully into the central goals of higher education. In 2012, the Civic Learning and Democratic Engagement (CLDE) National Task Force submitted a report to the nation entitled A Crucible Moment: College Learning and Democracy’s Future. Using ten indicators of the decline of civic health in the United States, this document brought the charge of civic education to rest more fully with higher education. Based on research during the late 1990s and early 2000s, service-learning was championed as a central element in creating a more civically informed society. With its rich history and various iterations, service-learning has yet to have one definition agreed upon by all who utilize the pedagogy. However, it has key themes of learning, community involvement, and reflection. Using this as a foundation, Barbara Jacoby’s definition guides this research. She defines service-learning as “a form of experiential education in which students engage in activities that address human and community needs, together with structured opportunities for reflection designed to achieve desired learning outcomes” (Jacoby, 2015, p.4). This definition embraces service-learning in its triumvirate state of being a program, pedagogy, 1 and/or philosophy. While this study looks at the pedagogical underpinnings of service-learning, using Jacoby’s definition opens the possibility of (re)framing the field’s view of service-learning in programmatic and philosophical ways. It is valuable to note that Jacoby’s definition is intentionally broad to allow for more instances of community-based learning pedagogies under service-learning's umbrella. The work of this dissertation is to investigate how the philosophy behind service-learning impacts pedagogy through the perceived benefits to student civic learning. One consistent attribute of service-learning that sets it apart from other experiential activities is the intentional integration of reflection into the curricula designed to create space for critical analysis and reflection. Students are asked to think not only about the information presented to them in class and to examine and evaluate that body of work, but they are also invited to critically reflect on their own positionality and connect what they see in the community to the issues raised in the classroom. One of the long-standing principles of quality service-learning practice is connected to integrating critical analysis and reflection. Service- learning should make the distinction between learning in the classroom and learning from the community experience minimal (Howard, 2001). When the two types of learning experiences are as seamless as possible, students can better connect them in ways that produce the civic learning outcomes associated with service-learning. This is connected to another hallmark of service-learning: integrating service into specific course curriculum learning objectives. The learning outcomes can be achieved during one semester with consistent involvement with a community partner, over a full academic year, or even as part of an academic program (Eyler & Giles, 1999). While there are examples of the service-learning experience integrated into a full program curriculum or those spanning short 2 service experiences, they are not included here as they are either not commonly practiced or do not meet the requirements for quality service-learning experiences. Research has shown that students receive the most benefit from service-learning opportunities if they can consistently engage with the community throughout a semester or an academic year (Shumer, 1997; Eyler & Giles, 1999; Tryon et al., 2008). However, assessment of a year-long program or short service experience is beyond the scope of this dissertation due, in part, to the disruption of academic programs because of the rise of the COVID-19 pandemic. A final connection is between service-learning and the civic learning outcomes which reach beyond the classroom and the impact of these civic outcomes on student learning. Civic learning was introduced in the early 20th century during the call for educating students to be active citizens through experiences with a broader community outside the confines of the campus (Dewey, 1916). This specific form of teaching is now related to service-learning’s impact on a student’s ability to understand oneself in relation to others and be a knowledgeable participant in one’s community/society (Moely, 2002a). The specific civic outcomes of service-learning are measurable and have become critical components of the service-learning pedagogy. The considerations of civic education specifically are evolving along with the outcomes connected to civic engagement and service-learning. Civic engagement outcomes and their connection to service-learning are further discussed in the literature review. History Service-learning, as both a concept and a practice, is thought to be over 100 years old and has undergone many changes and refinements of the pedagogy during this time. It has moved from the response to a call to action for including experiential learning to a distinct and well- researched high-impact practice. Campus Compact and the National Survey of Student 3 Engagement championed the collection of national data around student civic and community engagement. Campus Compact, a United States-based coalition of campuses dedicated to the promotion of civic and community engagement established in, has issued annual member engagement surveys since 1986. These reports show two trends in community/civic engagement assessment. The first is around curricular and co-curricular engagement. While many institutions (80% of public four-year) track engagement on campus, a quarter of those do not track data in a way that allows for academic engagement (course connected) to be separated from co-curricular service or volunteering. The second trend from the Campus Compact data is about student learning outcomes. Respondents were able to indicate if their institutions formally tracked community engagement outcomes for their students. In the 2016 report, 47% of public four-year respondents indicated they tracked community engagement outcomes for students in at least one of seven categories: civic/democratic learning, critical thinking, engagement across differences, global learning, media literacy, policy knowledge, or social justice orientation (Campus Compact, 2016, p. 4). Similarly, the National Survey of Student Engagement, completed by over 1,600 higher education institutions in both the United States and Canada since 2000, only provides disaggregated data by institution type and whether a student is a first-year or a graduating senior. Without consistent and high-level data tracking, most service-learning research uses a sample within a specific institution or region (Center for Postsecondary Research, n.d.). Additionally, many qualitative studies investigate the experiences of students from different demographic backgrounds such as race, sex, or income (Green, 2001; Kiely, 2005). However, few studies specifically target civic learning outcomes as they were introduced later during the move toward building a field. 4 Building a field The conversation around moving service-learning from a social movement to a field with identifiable learning outcomes started in the 1990s (Giles & Eyler, 1994). During this time, research on and about service-learning increased exponentially. This research aimed to prove the value of service-learning and support the institutionalization of the practice. Scholars and practitioners worked together to study different ways service-learning increased student learning and contributed to the rigor of the academy. Quantitative research was prevalent during this time resulting in standards of practice, learning outcomes, and assessment instruments. Much of the research was built on each other, but just as there was no standard definition of service-learning, there was no explicit agreement on how to assess it. During field-building, the structure of racial identity in the United States also started to shift, as seen in postsecondary enrollments. Between 1990 and 2013, the United States saw one of the largest undergraduate enrollment increases from Hispanic and African American populations (Musu-Gillette et al., 2016). The increases range from 140% for Black students, 496% for Hispanic students to a 560% increase for students with Asian or Pacific Islander backgrounds. Nonwhite students enrolled in undergraduate education increased from 15.9% in 1990 to 47.1% in 2023 (U.S. Department of Education, 2022). Although Students of Color saw unprecedented increases in postsecondary enrollment, white students saw a drop in undergraduate enrollment going from 77.6% of the undergraduate population to 53.2% in 2017. The combined effect of the changing demographics in higher education and the communities they serve requires new strategies for maximum impact responsive to the needs and unique challenges an increasingly diverse student body brings to its institutions. Additionally, many 5 higher education institutions work to achieve their broader missions as anchors and are committed to working with communities outside campus boundaries (Dubb et al., 2013). Although the overall racial diversity in higher education was changing dramatically, current research suggests student participation in service-learning courses often maintains a similar demographic breakdown as when service-learning became broadly institutionalized within higher education (e.g., Song et al., 2017; Chittum et al., 2022). Additionally, most research was, and continues to be, conducted at predominantly White institutions (PWIs). Research in community-engaged teaching and learning follows the course of service-learning pedagogy as it moved from the margins, in departments such as Ethnic Studies, to becoming more mainstream and incorporated into a general liberal arts curriculum for many institutions. As service-learning use increased in courses across the country, the student population in these courses became increasingly white as there are many more PWIs than other institution types. According to information updated in August 2019, only a little over 12% of degree-granting institutions in 2016-2017 were classified as minority-serving (US Department of Education, 2019. In addition to the lack of diversity within service-learning courses is an overall concern about the experience of Students of Color (SOC) at predominantly White institutions. Research showed that, overall, these students report different experiences than their white classmates. They struggle with both academic achievement (D’Augelli & Hershberger, 1993), persistence (Bennett & Okinaka, 1990), and a sense of belonging (Allen, 1985; Jones, et al., 2002). One of the characteristics of a high impact practice is the ability to increase the likelihood of student retention. Considering Kuh’s (2008) inclusion of service-learning as a high impact practice and further assertion that high-impact practices produce particularly positive results for students in 6 marginalized identity groups, understanding the learning outcomes of students who fall in one of these identity groups (Students of Color) will guide the future of both service-learning and other high impact practices. If the experiences that SOC have in their classes are not producing the desired outcomes, these experiences will not lead to the high impact indicated by Kuh. Purpose of study The purpose of this study is twofold. The first is to use racially disaggregated data to compare the civic learning outcomes of students who completed a course with an integrated service-learning component. Other demographic information has been used in comparing learning outcomes for students, but it has primarily been connected to gender, first-generation, or socioeconomic status. Administering the survey to students with similar service-learning requirements and a focus on understanding of/exposure to diversity as part of the course provides an added dimension of connection to the civic learning outcomes presented in the Civic Attitudes and Skills Questionnaire (CASQ) (Moely et al., 2002b). The second is to use qualitative methods to uncover how students make meaning of their civic learning through the statements used in the CASQ. Using these statements, students were asked to find connections to their service-learning courses in two ways. The first was to connect to the overarching factor themes, and the second was to dig deeper into the individual statements. The use of two methods of research is important as this will be the first effort to do the following: use data from students that have been out of a class for at least one calendar year, disaggregate civic learning outcome data by race, and explore how students make meaning of their service-learning as it connects to civic learning outcomes. Special attention will be given to how student outcomes differ based on race. The study is predicated on two research questions related to the goals: 1) How do students make meaning of statements in the Civic Attitudes and Skills Questionnaire? and 2) Are 7 there quantitative or qualitative differences in student responses based on racial identity? The research questions are designed to be investigated using two different methodological approaches. Most service-learning practitioners, whether they are faculty members working with students in a class or community engagement professionals engaging students connected to academic programs or co-curricular activities, use the CASQ and are primarily concerned with the quantitative data as they connect to the broad civic learning goals attributed to service- learning. This study, however, goes beyond the quantitative data to present information on student understanding using the qualitative method of focus groups. Information about the CASQ instrument is explained further. Civic Attitudes and Skills Questionnaire The research presented here is the first step to both understanding the importance of student perceptions of their service-learning and the meaning students make of the civic learning outcomes based on the statements in the Civic Attitudes and Skills Questionnaire (see Appendix A). The Civic Attitudes and Skills Questionnaire (CASQ) was chosen because it was the first vetted civic learning assessment instrument in the field of service-learning. Although other instruments have been created since the 2002 publication of the CASQ, they all use the CASQ as the baseline for their survey development. Additionally, considering the gap in the quantitative research connecting civic learning outcomes to racial identity, the use of critical race theory will be employed to investigate any variances in civic learning outcomes (skills, values, and attitudes) based on racial identity. The Civic Attitudes and Skills Questionnaire (CASQ) was developed by Moely and associates to assess how participation in a service-learning course impacted civic learning outcomes (Moely et al., 2002b). While creating the instrument, the authors were able to group 8 the associated questions into six themes or factors. While a more robust discussion of the instrument is given in Chapter Three, the six factors are listed here: civic action, interpersonal and problem-solving skills, political awareness, leadership skills, social justice attitudes, and diversity attitudes. Although the words may be different, these six concepts have been used by others to assess civic learning outcomes both connected to service-learning and related to general learning outcomes (Musil, 2009; Prentice & Robinson, 2010; Hurtado, 2019; Hudgins, 2020). Participants in this study were given the CASQ as it was initially created, including the demographic questions, with slight modifications. The changes to demographic questions reflected the commitment to culturally responsive research and the need for options for demographics to reflect the richness of diversity within this new study body. Additional changes were made to the civic action factor due to the proximity of the data collection to the unfolding of the COVID-19 pandemic. All modifications are explained in further detail in chapter three. Significance Using survey data to uncover potential differences in student civic learning outcomes based on the identity marker of race provides insights into three important areas. The first is a better understanding of the demographics of service-learning students. Since the push for institutionalization and its increase in popularity at predominately white institutions, service- learning has a history of sending predominantly white students into communities of color (Mitchell, 2008). In fact, when the Civic Attitudes and Skills Questionnaire was developed, the student population at Tulane University was 79.1% white (Tulane, 2001) which resulted in each sample used in the creation of the instrument being 79% white (Moely et al., 2002b). As the field continues to develop, evaluating the demographic base of students who participate in this practice becomes increasingly important. While it is also important to think about the 9 demographic makeup of the faculty/instructors that integrate service-learning into their curriculum, it is beyond the scope of this study to include that information in a meaningful way. The second insight leads to understanding, through analyzing the data using quantitative methods, the benefits of service-learning as a high-impact practice and its applicability for all students. According to Kuh’s (2008) landmark publication with the American Association of Colleges and Universities, ten teaching and learning practices within higher education are proven to increase student success and retention. Included in this list is service-learning and other forms of community-based learning. The definition used by Kuh closely matches that of Jacoby and contains the three key elements of learning, community, and reflection. Many have researched the impact of these practices on the retention of various groups of students, focusing on first- generation and Students of Color. Still, Swaner and Brownell (2008) support the need for more research on the impact of service-learning on Students of Color beyond retention. Finally, this research can provide direction in refining the pedagogy and implementation of service-learning. Although it is not known if differences exist, it is within reason to consider that the changes in the demographics of the student population require the pedagogy to shift from how service-learning was originally institutionalized within PWIs and modify the expectation of its outcomes. Mitchell (2008) divides service-learning into two camps: traditional and critical. Traditional service-learning leans heavily on two conceptual models of the practice: technical and cultural. The technical model elevates the importance of efficacy through the number of hours students are engaged in the community and a focus on student learning more than community partnership (Butin, 2005). Often this results in students viewing the community as a laboratory meant for taking information rather than connecting to its own expertise. The cultural model focuses more on meaning-making through integrating social learning models of education. 10 Culturally based service-learning upholds the practice’s ability to influence students’ views of diversity (Butin, 2005) by creating opportunities for students to engage with individuals and organizations in communities that may differ from a student’s particular background. In her work in critical service-learning, Mitchell (2008) expands the possibility of service-learning to move forward and adopt a social justice framework with special attention paid to systems and structures of inequity. Student voice is key to understanding how to best connect the civic learning process to service-learning opportunities to provide a high-impact practice that benefits more students (Chittum et al., 2022). Covid 19 Many people consider March 2020 as the point in time when most people in the United States found out about the novel coronavirus 19. In many ways, this was true, but institutions of higher education began monitoring COVID-19 in connection with travel advisories as early as late January 2020 (National Center for Immunization and Respiratory Diseases (NCIRD), 2021). This was connected to travel outside of the United States, with less concern about what might happen on specific campuses. As more information became available about the dangers of the virus, how it was spread, and the best ways to prevent it, institutions of higher education quickly made decisions that more directly affected their faculty, staff, and students. More widespread disruption began in March 2020 when the first documented case was reported in Seattle, WA, and news of other cases quickly spread, people started to question the safety of face-to-face interactions. Out of an abundance of caution, most higher education institutions halted or cut short international programs, and many transitioned to an entirely online learning environment. Not only did institutions cancel in-person classroom learning, but they also halted pedagogical tools such as labs, fieldwork, and service-learning (Filho et al., 2021). The suspension of in- 11 person classes did not necessarily mean the end to all student service-learning options. Many institutions flipped to online platforms, allowing students to continue their education while remaining at home or in their dorm rooms. Mid-Lakes University, where this study was conducted, proceeded in that direction. During the fall semester of 2020, most students enrolled at MLU chose to participate in their classes online. The situation at Mid-Lakes University was not the only factor in how service-learning could be implemented during the fall semester of 2020. The surrounding community relied heavily on the guidance given by both the CDC and county health department officials. Many preventive measures were put in place to require facial coverings, reduce the capacity of restaurants, and limit the number of people at outdoor gatherings. Due to a surge in Covid cases, in July 2020, local county health officials recommended that all MLU students self-quarantine (MLU website, 2020). The original proposal for this research study submitted in July 2020 was to collect data from students at the end of their service-learning course. During this time, the Centers for Disease Control and Prevention regularly issued guidance for the public and, more specifically, institutions of higher education that were increasingly more restrictive (NCIRD 2021). After a conversation with my dissertation committee and following the guidance of the CDC and the institution’s IRB requirements, the decision was made to use the student population from the 2019 fall semester to draw the sample. The data for this study were collected between February 1 and April 30, 2021. During this time, the institution implemented stronger social distancing policies and limited people on campus. Additionally, the institutional review board continued to prohibit any in-person interviews. It is within these parameters that this study was conducted. The impacts of COVID-19 on the design and implementation are discussed in consequent chapters. 12 CHAPTER 2: LITERATURE REVIEW The previous chapter presented the premise for this study, including a brief history of service-learning in the United States and an outline of the significant contributions this research brings to service-learning and civic engagement. This chapter presents foundational scholarship on service-learning, expands on previously mentioned contemporary authors, and introduces the connection to civic education/outcomes. Next is an overview of critical race theory as the theoretical foundation for this study. Using critical race theory to center this work requires both an understanding of white supremacy culture (Okun, 2021) and the examination of the experiences of Students of Color in predominantly white institutions generally and as participants in service-learning. These two components set the stage for exploration and insight into the current research. Foundational Scholarship Modern service-learning scholarship, as it was institutionalized in mainstream higher education, is built on the work of three key scholars: John Dewey, David Kolb, and Paolo Freire. However, it is important to note that service-learning had its earlier roots in historically Black colleges and universities as well as Appalachian schools. These institutions were designed with the intention of contributing to the communities in which they were built. Service-learning in other parts of higher education started as a social movement to reimagine the public purposes of higher education. The change from a social movement to a field is still relatively new as discussions about this started in the mid to late 1990s (Eyler & Giles, 1994). John Dewey provided an early connection between education and democratic learning and emphasized the importance of personal experiences. David Kolb took the connection to personal experience and, through his research, created a four-step learning cycle that 13 incorporates the experiential components of learning. Finally, Paolo Freire (1970) moved the philosophy of teaching from a banking model of depositing information into students' minds to a problem-posing model rooted in the experiences of all. Other contemporary scholars have refined these early works to transform service-learning into an enhanced teaching pedagogy for students, and it is important to review the foundational works of these scholars to properly situate service-learning within the higher education context. Influence of John Dewey. When John Dewey made the connection between personal experiences and learning in his book, Democracy and Education: An Introduction to the Philosophy of Education, first published in 1916, the primary focus was creating an educational model promoting education's connection to the broader society. In this original treatise, he took a holistic approach to education. Student experiences were required to achieve the learning needs of students and to ensure the future of our democracy. While often seen as the beginning of service-learning, Dewey’s call for a different educational philosophy could better be described as the start of civic involvement/engagement in higher education (D’Agostino, 2018). The distinction is important because the intentional connection to course learning outcomes needed to qualify as service-learning was not yet developed. His contribution to the field is important to the development of civic learning outcomes connected specifically to service-learning. Dewey’s second contribution to what would become service-learning is the importance of critical reflection. In a review of Dewey’s work connecting personal experience, knowledge, and democracy, Markus Holdo (2023) shows that the line from Dewey to transformative learning theory is based on critical reflection. Transformative learning theory, in its simplest definition, suggests that people change the way they think about something or how they approach situations because of learning that has occurred (Kiely, 2005). Often, students are presented with the liberal 14 education requirement to improve their critical thinking, which has them focused on what they are learning in class and critiquing theories and information presented by experts in their disciplines. Critical reflection, as grounded in Dewey’s work, is grounded in student positionality and how they orient to the issues surrounding the communities with whom they are working (Rocheleau, 2004). As stated earlier, research about service-learning became more focused in the late 20th century as the practice moved toward institutionalization. The goal of research at this time was to both improve the pedagogy and prove its worth. Giles and Eyler (1994) specifically drew upon the work of Dewey as they recounted the history of service-learning and its search for a solid theoretical grounding and research agenda. Their work, along with other leaders in the field, focused on experiential learning theory. Experiential Learning – Kolb and Taylor. David Kolb was a scholar who started to draw the connection between the student experience and the learning process. Although Dewey first introduced experiential learning as a concept in 1916, it was not until Kolb took Dewey’s call for integration to inform his work on the experiential learning cycle that educators could point to a theory (Shavers & Mitchell, 2019). It is important to note that Astin, a contemporary of Kolb, was also thinking about how experiences contribute to student outcomes, but his work was more closely connected to co-curricular aspects of higher education (Renn & Reason, 2013). Service-learning does have links to co-curricular activities, but the research for this dissertation is focused on what happens in a classroom. Kolb’s (1984) four-step cycle consists of an experience, reflection of that experience, learning from the experience (conceptualization), and then applying that knowledge to a new situation (application). Kolb laid the foundation for other scholars to refine his work to be better 15 suited for a theoretical basis of service-learning. One such person was Marilyn Taylor, who adapted Kolb’s work on experiential learning to create a new four-phase model for adult learning (Mackeracher, 2004). She reconfigured Kolb’s model to consist of disorientation, exploration, reorientation, and equilibrium. Quality practices in service learning resemble this model in the following ways: 1. Disorientation refers to the period of time when students often feel disoriented during the beginning phases of their service-learning experience. This could happen from the time they enroll in a service-learning class and wonder what they will encounter, or it could begin the moment a student becomes aware of their assignment or project. Some of the causes of disorientation are having the community partner play the role of co- educator or the need to leave the confines of a classroom to engage with a community that is not the student’s own. Students are challenged in their beliefs about whose knowledge matters and to what extent. 2. During the time of engagement with the community partner, students explore the various reasons for their disorientation and often start a new process as they continue to work alongside those in the community. Students begin to accept the tacit knowledge the community partner as co-educator brings to the learning experience. Additionally, students use this time to apply what they are learning in class to the situations they may find themselves in. For example, an education student may try different ways to connect with students based on a classroom assignment interrogating racialized social norms. 3. In her 2002 article in the Journal of Social Issues, Janet Eyler reconfirms her position that the hyphen in service-learning is student reflection. Reflection, specifically critical reflection, helps students bridge the gaps between what they learn through readings and 16 lectures, what they believe they already know, and what they experience in the community (Eyler et al., 1996). Processing the information in this way allows students to reorient to a new way of knowing. 4. Learning does not stop once a student has made peace with their experience. True learning is achieved when they can use this knowledge in different settings. The ability to take what they have learned and apply it in other ways does not mean that this is a static position for them. Even though this stage is one of equilibrium, it is dynamic and continues to be refined as students continue to develop their civic identity. Continued exposure to different situations, especially those outside of the classroom, lead student to form their civic identities (Mitchell, 2015). Once research in the field attained the level of rigor necessary to make substantive claims, those within the academy began to move beyond the quest for a theory to a place where they could study the impact of service-learning on the students participating in it. One of the first published works in this area was in 1997 by Janet Eyler, Dwight Giles, Jr., and John Braxton. Their study used quantitative methods to gather information from over 1500 students enrolled in 20 colleges and universities in the United States. These colleges were selected because of their current involvement in service-learning and to have representation from different sectors of higher education (Eyler et al., 1997). Their work and the work to follow focused primarily on the differences between students who chose to participate in service-learning and those who did not. While sociodemographic data were requested, the primary concern was the difference between genders. Part of the reason for this was that the population of Students of Color participating in service-learning at the time was limited. Another reason for this is because the primary goal of 17 this research was to show the impact of service-learning for students that chose to participate not for differences within the population that was participating. Civic Learning Although service-learning is now considered a key pedagogical tool for creating a civically informed society, that connection didn’t come until much later. The research cited above speaks mostly about the importance of service-learning to transform students’ learning into something more meaningful and to enhance students' lives and educational goals. The following research holds importance for the field in its connection to student learning and the bigger picture of civic engagement and education. Before the publication of The Crucible Moment in 2012, Barbara Jacoby and associates published the book Civic Engagement in Higher Education: Concepts and Practices. In this publication, Caryn McTighe Musil presented the civic learning spiral to fully incorporate civic learning outcomes into higher education. She viewed civic engagement as part of a broader set of student learning reform movements, including U.S. diversity and global learning (2009). A decade later, Sylvia Hurtado (2019) acknowledged the path civic engagement had taken and posited that what is needed is a clear alignment of civic learning goals in higher education teaching and learning. All of this requires an agreed-upon set of civic learning goals and an expressed commitment to civic identity development. The civic learning goals of service- learning center on the original goals outlined by John Dewey and continue as 21st-century learning goals today. Most will agree that ten skills make up these goals: creativity and innovation; critical thinking, problem-solving, and decision-making; learning to learn, metacognition; communication; collaboration; information literacy; digital literacy; citizenship; life and career; and personal and social responsibility (Binkley et al., 2012, p. 36). The Civic 18 Attitudes and Skills Questionnaire was created to assess how service-learning corresponds to six civic learning goals, as outlined later in the chapter. Many of the 21st-century learning goals are present in different ways in the CASQ. Assessment Before developing service-learning-specific assessment tools, many researchers relied on survey instruments specific to the construct they wanted to measure (Bringle et al., 2004). Often this meant choosing from a variety of different scales and cobbling them together to create an assessment instrument the researcher and/or faculty member was content to use. These tools did not have the explicit connection to service-learning outcomes required to best understand service-learning's impact on civic development. Below is a review of the Civic Attitudes and Skills Questionnaire constructed to measure six factors of service-learning outcomes. The Civic Attitudes and Skills Questionnaire (CASQ) was designed and vetted by Barbara Moely, professor and Director of Service-Learning at Tulane University, along with her colleagues Sterett Mercer, Vincent Illustre, Devi Miron, and Megan McFarland (2002). Using previous research in service-learning and volunteer outcomes, Moely and colleagues identified scales to assess the degree to which a service-learning experience impacted students' self- reported attitudes and skills related to civic outcomes. The instrument was created to be used as a one-time distribution at the end of a service-learning course to assess how a service-learning course impacted students’ civic learning outcomes but was later administered using a pre/post- test design to measure the change in these outcomes (Moely et al., 2002a). While both are valid uses of the instrument, this study uses the CASQ as a summative assessment. Academic learning goals for service-learning courses are often assessed using reflections and the ability to perform on tests. Those indicators may or may not be influenced by participation in service-learning. 19 Civic learning outcomes, however, are more closely tied to the learning goals and objectives of service-learning (in the community) rather than the academic coursework. Using factor analysis, the questionnaire was reduced from an 84-item questionnaire to the final 45-item instrument. The remaining items were grouped into six factors based on the percent of score variance: civic action, diversity attitudes, interpersonal and problem-solving skills, leadership skills, political awareness, social justice attitudes (Moely et al, 2002). To pass reliability and validity testing, a factor had to have a percent of variance less than 20%. The range of variance was 3.0% for diversity attitudes to 16.9% for civic action. The CASQ allows students to “self-evaluate their skills and personal attitudes regarding civic and social issues” (Moely et al., 2002b, p. 17). It is given to students in the classroom at the beginning of the semester and asks for some basic demographic information such as race, gender, age, year in school, major, GPA, and their goal for educational attainment level. Additionally, students are asked to estimate the number of hours they have participated in volunteer service in religious organizations, throughout high school and in college. The CASQ asks students to use a scale from 1-completely disagree to 5-completely agree in response to each question. Specifically, the CASQ assesses using the following six factors: Civic Action, Interpersonal and Problem-Solving Skills, Political Awareness, Leadership Skills, Social Justice Attitudes, and Diversity Attitudes. The Civic Action factor asks students to think about their intent to be a part of and do service work in their community. Interpersonal and Problem-Solving Skills focus on students’ thinking about their ability to work and communicate with others both individually and in group settings. For the Political Awareness factor, students reflect on their individual awareness of current events locally, nationally, and in the world. The Leadership Skills scale involves students 20 assessing themselves and their effectiveness as leaders and their ability to make a difference. The Social Justice Attitudes scale asks students to think about their power and privilege in relation to socio-economic status, policy, and equity. Finally, the Diversity Attitudes scale asks students to think about who they feel comfortable being around regarding varying backgrounds including race and culture. The purpose of this dissertation is to focus on the civic learning that occurs because of participation in a service-learning class. Using the outcomes for civic learning principles by Musil (2009), the CASQ was chosen for its attention to the six principles through its assessment of students’ knowledge, skills, values, and behaviors associated with participation in service learning (Moely et al., 2002b). Critical Service-Learning According to the National Center for Education Statistics, the U.S. has seen dramatic increases in Students of Color enrolled in degree-granting institutions. The increases range from 140% for Black students to 496% for Hispanic students (term used by NCES) to 560% for students with an Asian or Pacific Islander background (U.S. Department of Education, n.d.). Although this level of increase was seen broadly within higher education, the diversity did not filter down to service-learning classes and became an issue of concern for service-learning practitioners. In addition to the overall lack of diversity within service-learning courses was the overall concern about the experience of Students of Color at predominantly White institutions. Research has shown that these students report different experiences than their white classmates. They struggle with both academic achievement (D’Augelli & Hershberger, 1993), persistence (Bennett & Okinaka, 1990), and a sense of belonging (Allen, 1985; Jones et al., 2002). During this time, George Kuh (2008) published his list of ten high-impact practices that were to enhance student 21 success. Service-learning is one of these practices but may not be as impactful if students are having different experiences. John Dewey stated in his manuscript “Experience and Education” that, “Experience and education cannot be directly equated to each other” (as quoted in Giles & Eyler, 1994). If the experiences that Students of Color (SOC) are having in the community are not enhancing what they are learning because it has a negative climate or does not provide the connection points for each of the factors, their service-learning will not lead to the high impact indicated by Kuh. The changing demographics of American higher education require a shift in how service- learning is operationalized. Traditional service-learning and the research around it remained constant until 2008, when Tania Mitchell differentiated between traditional service-learning and critical service-learning. While the general outcomes for service-learning remained the same, the critical approach sought to expand beyond simply providing an experience that enhanced academic outcomes for students in higher education. Critical service-learning seeks social justice and social change as central goals of the service-learning experience (Mitchell, 2008). While social justice outcomes are documented in the assessment tools used in service-learning across the board, until this delineation between critical and traditional, many service-learning courses still need to actively include issues of social justice in the teaching, explanation, or reflection spaces. This approach also looks at the participants within service-learning and the people who receive the service. The concept of working alongside comes from critical service-learning as it seeks to “redistribute power” so that students can start to see themselves as part of social change and not just fix a community (Mitchell, 2015). Additionally, critical service-learning can be utilized as a way for racially minoritized students to feel a sense of belonging outside of the 22 confines of a predominantly white institution or classroom (Kinzie et al., 2021). It is the connection with the community that has been argued to create civic learning and foster a civic identity within participants. Students of Color at predominantly White institutions The research on Students of Color and their persistence and success on predominantly white campuses dates back several decades, with the literature for this review going back to 1985. The early data looked specifically at Black students, but as time and demographics changed, the research started to include more racial and ethnic groups. The studies mostly looked to decipher if persistence was related to the personal attributes of Students of Color or if they were more impacted by their external environment. The conclusion of these studies is that it is a both/and instead of either/or reality for Students of Color (Allen, 1985; Bennett & Okinaka, 1990; D’Augelli & Hershberger, 1993; Jones et al., 2002). In the review for this study, it is important to note that Bennett and Okinaka examined persistence factors for Asian, Black, Hispanic, and White undergraduates at Indiana University in Bloomington, and Jones, Castellanos, and Cole only engaged with Students of Color. A difficulty in this, and other studies, is the separation of students by race and/or ethnic category. The realities of these students may look different based on how they self-identify (internal) and how they are identified by others (external). Complicating this matter is the common critique of the predominantly White teacher education classes which are the subject of many service-learning studies and the primary participant base for this dissertation. Christine E. Sleeter (2016) states that approximately 80% of new teachers are white. The central issue is not that these students are white, but rather one that is two-fold. The first is that the teacher education classrooms and curriculum spend an 23 overabundance of time working with and toward the white students’ needs. Additionally, even with this attention, the students are underprepared to work with the diverse populations that they will be teaching. This comes from a continuation of viewing diversity as something that is either “other” or something that needs not be addressed because of an adherence to color-blindness as a form a quality teaching (Sleeter, 2016). She posits that the use of critical race theory to start the restructuring of teacher education in a way that strategically partners with local communities is integral to creating change. Gaps in Literature The intersection of the impacts of service-learning on Students of Color in predominantly white classes in predominantly white institutions that continue to perpetuate a color-blind narrative is part of the gap in the literature regarding both research and the history of service- learning. A recent narrative showed how service-learning, at its core, was not designed to include people of color (Bocci, 2015). The first introduction to democratic education and citizenship came at a time when people of color were not allowed to be citizens, let alone active members in democracy. Disenfranchisement in this regard has been difficult to overcome both within higher education and society. Very few empirical studies have been conducted using quantitative methods to look at the intersection of service-learning outcomes and Students of Color. The primary goal for studies that did address racial identity compared civic learning outcomes based on enrollment in a service-learning course and often mentioned that there were too few students to either disaggregate beyond a binary or too few for it to make a significant difference in the outcome (e.g., Eyler et al., 1997; Moely, 2002a; These civic learning outcomes were assumed as uniform across racial lines with any differences occurring as a result of course participation. The research that has been done has used an interpretivist framework that focused on understanding a 24 specific experience from a small subset of individuals and not investigating how Students of Color, as a group (though heterogeneous), report their civic learning outcomes. This gap in the literature led to this current study focused on the examination of civic learning outcomes that both interrogates how the CASQ is received today and highlights the voices of Students of Color. By holding the service-learning component stable, this further investigation can offer a different view on the impact service-learning has on civic learning outcomes for Students of Color. Ideally, one would compare models of service-learning (critical vs. traditional) to see which has the stronger outcomes; however, it is just as valuable to look at who is involved in service-learning, the demographics of the community in which the students are serving, and how the interplay of both components impacts students’ development of civic outcomes. Part of being able to assess this critically is to have the theoretical knowledge of both models of service-learning as informants. The small-scale studies previously conducted attempted to provide an in-depth look into the ways in which specific Students of Color experienced service-learning. This deep dive ignored the broader system influences that are at play and the resulting disadvantage for these students. Student experiences could potentially limit their ability to develop their own civic identity as they move throughout this part of their academic and personal development. This multi-methods research will combine both the quantitative data to show student outcomes with qualitative data to better understand the how students arrived at their acquisition of these civic attitudes and skills that are core components and benefits of using service-learning as a pedagogical model. Finally, this work opens the door to other research to disaggregate the data to see where the differences become pronounced. Additionally, the benefit of having this empirical data is the theoretical base it provides for future qualitative research. 25 Theoretical Underpinning The primary theory employed to guide both the research and analysis of data is Critical Race Theory (CRT). CRT has its roots in legal scholarship and developed in the 1970s to explore the ways in which covert acts of racism impact people of color. In their book Critical Race Theory: An Introduction, Richard Delgado and Jean Stefancic discuss how critical race theorists are interested in both “studying and transforming the relationship among race, racism, and power” (2012, p. 3). A key point of this theory is the recognition of the truth of white privilege and the power structures that have come from it. It is not simply that these structures oppress economically, but they find themselves manifested in who is accepted as knowledge holders and who is epistemically marginalized within structures of higher education. Service-learning becomes complicit in upholding white privilege through its focus on designing curricula that comes from a white perspective and is often implemented by white faculty sending white students into communities of color (Mitchell, 2008). By using CRT as the theoretical framework for this study of civic learning outcomes of Students of Color, I move to uncover differences based on race and present ways to transform the current implementation and assessment approaches of service-learning to create a more culturally responsive pedagogy. Five Tenets of CRT Critical race theory has five central tenets which serve as the analytical lens for this study. Much of what follows speaks to using critical race theory to interrogate the practice of service-learning and the way it has been assessed using quantitative measures. The connection to quantitative critical inquiry becomes apparent throughout the five tenets. Quantitative critical inquiry is an approach to quantitative research that upholds the central beliefs of critical race 26 theory. The following discussion of the tenets gives a brief description of the tenet and its integration into the current research. Counter-storytelling. When centering the voices of those who have been marginalized by society and the academy, it is important to not only create a space to intentionally listen to these voices but also take steps to legitimize the different forms of knowledge they provide. Most often, this is done using qualitative research methods, which allow these individuals to tell stories in their own words. However, the legitimization of these marginalized individuals can also occur in quantitative research. Using Critical Race Theory to analyze quantitative data can provide counter-stories to deracialize statistics (Garcia et al., 2018). The creation of the Civic Attitudes and Skills Questionnaire did not consider differences that may occur based on racial identity. In fact, during the creation of the CASQ, factors five and six (connecting service- learning to diversity and social justice attitudes) were added because the researchers acknowledged that the bulk of their service-learning students were white students entering communities of color (Moely et al., 2002b). Additionally, studies that use the CASQ often do not disaggregate data by racial identity because the number of respondents in different racial categories was typically deemed too small to consider in the final analysis. Critical race theory allows for the legitimization of these small numbers in understanding the impact of service- learning by acknowledging the small numbers, using the lack of data as a data point, and treating the data from the Students of Color as unique information to be analyzed. Permanence of Racism. CRT is predicated on the inherent reality and impact of racism in the United States. In conjunction with this racism is an infiltration of the belief and effects of white supremacy whether it is conscious or unconscious (Hiraldo, 2010). This immersion in White supremacy culture means it is present in institutions of higher education from the 27 administration to the classroom. Understanding that such connections may influence both the implementation and impact of service-learning for Students of Color is important in the analysis of the data from the Civic Attitudes and Skills Questionnaire by looking beyond the overall learning outcome into specific questions within the instrument. CRT lends itself to the use of critical realism, another philosophical concept, to guide the framing of this study and brings validity to the self-reported nature of the CASQ. Critical realism states that there is a reality that exists outside the minds of individuals (Glesne, 2016), and this reality has been shaped by social structures that limit some members more than others (Roberts, 2001). Instead of using qualitative methods to ask for an interpretation of an individual’s reality, critical realism allows for exploration of the magnitude to which power plays a part in people’s interaction with the social world. The use of CRT hones this concept further to adopt Derrick Bell’s “racial realism,” which goes deeper to discuss how this truth is based on racial identity (Delgado, 1991). The interpretation of the way in which a specific statement in the CASQ applies to the student respondent provides insight into their lived reality as it is connected to civic learning. Whiteness as Property. Originally, whiteness as property, especially in its connection to the original work in legal studies, was connected to actual property rights in explaining who has a right to own property and who, then, was also considered the property (Hiraldo, 2010). In higher education research, this tenet is framed differently to examine epistemic injustices, particularly around the right of exclusion. Vetted assessment instruments have undergone a rigorous psychometric analysis to prove their generalizability. That generalizability, however, was often normed on individuals from dominant categories (Leong & Park, 2016). The Civic Attitudes and Skills Questionnaire is no different. Included in the psychometric analysis, the 28 researchers acknowledge that often service-learning experiences are predominantly students who are white and either middle or upper class going into predominantly low-income communities that are most often people of color. This study is designed to critique the instrument based on the lived realities of Students of Color. More important, however, is the discussion of the effects on pedagogical implementation. Interest Convergence. This tenet of CRT recognizes that those who benefit the most from diversity efforts are primarily White people (Hiraldo, 2010). The basis for this rests on the case of affirmative action. Research on affirmative action has shown that White women were most often the beneficiaries of these policies. Service-learning and the assessment of its impacts may prove to be similar. While unpacking the experience of Students of Color in service- learning classes is beyond the scope of this quantitative study, some work has already been done along those lines. According to Green (2003), it is important to understand that service-learning may not be the same for every student based on their background. Often, Students of Color report that their race relegates them to hold one of four positions in a service-learning class: teacher of other students about race, token member of the racial community in which the class is serving (Green, 2003), translator to help white students better understand structural inequalities (Smith, 2019), or traitor because they are somehow not like the community members being served (Simons et al., 2011). If these experiences compete with the civic learning objectives of a course, then it would follow that service-learning has continued to grow in popularity not because of the impact it has on “all students” but because of its impact on white students. Critique of Liberalism. This critique of liberalism is directly connected to the issues of neutrality and colorblindness in education (Hiraldo, 2010). Service-learning has long been upheld as an important challenge to traditional pedagogy that provides a stronger connection to 29 the public purposes of higher education. The concern becomes whether the implementation of service-learning is done in a way that perpetuates white supremacy by ignoring issues of racism within the pedagogy. This study is designed to highlight the importance of racial identity rather than erase it as a factor influencing student civic learning outcomes. Any findings that point to racial differences should lead to a discussion of racism and white supremacy in the discussion. Both the willingness to name the systemic structures that tend to subordinate Black and Brown bodies and proceeding to put it in writing are necessary to avoid diluting the power of using critical race theory (Harper, 2012). Service-Learning as a Conceptual Framework Understanding critical race theory as the theoretical framework for this study is the first part of connecting to the examination of service-learning using that lens. While CRT guides the process, service-learning as a conceptual tool shapes the ways in which the assessed learning occurs. The following section briefly overviews the history of service-learning as a pedagogy. The primary focus is providing a conceptual model adapted from Whitley’s 2014 publication. Experiential learning became the umbrella term for practices that promoted the integration of the learner’s experiences into the act of learning. Kolb’s Experiential Learning Cycle has been used as the foundation for other experiential learning theories. For nearly three more decades, researchers and philosophers came up with several theories of experiential learning, with Kolb’s Experiential Learning Cycle (Kolb, 1984) being well cited. By connecting the work of three earlier theorists (John Dewey, Karl Lewin, and Jean Piaget), Kolb was able to distill four key aspects of experiential learning: concrete experiences, reflection observation, abstract conceptualization, and active experimentation (Kolb, 1984, p.30). At its foundation, service-learning pedagogy take Kolb’s learning and add in key components Paolo Freire’s work 30 on liberation (1970) to move from a theory focuses solely on student outcomes to one that starts to include an emancipatory lens (Kolb, 1984). Emancipation in this context is what allows for the connection to and service within a community in an intentional way. Whitley (2014) created a framework for studying service-learning effects on students that acknowledges four areas of concern: context, service-learning experience, mediating variables around critical reflection, and proximal and distal outcomes. At first glance, this framework can appear to be overwrought with details and theories connected to service-learning. The reality is, however, that all these components exist in a service-learning course experience. The complication becomes deciding where to focus one’s effort during a study. For this dissertation, each part of Whitley’s frame is used in either the quantitative or qualitative methods of study. Whitley’s (2014) framework begins with context, which includes various aspects of a student’s identity. The new framework, which draws from Bronfenbrenner’s socio-ecological model (Onwuegbuzie et al., 2013) acknowledges a broader context that frames the student experience of service learning. This broader context consists of the community where the service-learning is performed and the institutional context, specifically concerning sector and classification. The final area of context concern is student identity. Whitley lists 22 possible student context markers, seventeen of which are connected to student identity, and five are connected to inputs such as prior volunteer experience. The current study is particularly concerned with student racial identity, which includes the context markers of race, ethnicity, culture, and nationality. Culture is an important aspect of the consideration of racial identity. When asking respondents how they identify, this is not based on the concept of biological race; rather, racial identity upholds the notion of the social construction of race, which has a direct connection to cultural affinity (Haney Lopez, 2006). 31 The second area service-learning impacts students is the experience itself. Included in the experience are the course (i.e., whether the course is in a student’s major, how many students are in the course, and how well the service-learning is integrated into the syllabus), student (i.e., interest, motivations, assumptions, and personal identification with social and cultural norms), and community activity variables (i.e., logistics of time, support from community, and feedback from community partners). When assessing student learning outcomes in a study that uses different methods, several of these factors cannot be kept constant. One of the largest limitations of any quantitative service-learning study is the inability to adequately judge these variables using a scale. The Civic Attitudes and Skills Questionnaire (CASQ), however, provides insight into the student variable of social issues interest, civic development, assumptions, beliefs, and values, and identity and cultural awareness. Whitley (2014) lists mediating variables of critical reflection and critical thinking as a separate concern and presents them on the same level as other concerns. The new framework shows these mediating variables occur on the student level in ways that connect the personal to the political aspects of the service-learning experience. These variables remain concerns for understanding the impact service-learning has on civic outcomes, but they also show the interplay between critical reflection and critical thinking as part of the learning experience within the student. The COVID-19 pandemic impacted the timing of this research. Data were collected in April of 2021 when this university was not engaging in face-to-face activities. The participants of this study were enrolled in their service-learning courses during the calendar year of 2019 because most in-person service-learning activities were suspended in March of 2020. Students were not necessarily able to make direct connections to their service-learning courses, however, general course syllabi were obtained. Whitley’s final concerns are the proximal and distal 32 outcomes on personal, academic/career, social/civic, and diversity/intercultural levels. Consistent with research in the field of service-learning and in connection to this study, the first, third, and fourth outcomes can be collapsed into civic learning outcomes. 33 CHAPTER 3: METHODS The previous chapters introduced the research topic focused on student understanding of civic learning outcomes as outlined in the Civic Attitudes and Skills Questionnaire and its importance to higher education, provided background literature on the topic, and laid out the research question underlying this study. This chapter presents the questions used to frame the interrogation of the research topic and methods used in the approach and analysis of data through a brief introduction to the use of multiple methods in this research with detailed quantitative (quantitative critical inquiry) and qualitative (focus group interviews) components underscoring the utilization of each methodological approach and their point of interface (Guest, 2013). Researcher positionality is then provided for understanding the choice of methodology and the impacts on the ways in which data were collected and analyzed. The design segment of the chapter reintroduces the research question and related hypotheses before detailing the aspects of this mixed methods study and the tools utilized for data analysis. Methodology Research methodology provides the tools to systematically address a problem. In any study, the methodology is often based on a paradigm or framework that guides both the researcher’s ontological and epistemological stances. This section outlines and addresses the use of mixed methods research to address the question of student understanding of civic learning outcomes as defined by the Civic Attitudes and Skills Questionnaire (CASQ), introduces quantitative critical inquiry (QuantCrit), provides the qualitative approach used for further clarity, and connects the use of these two methods to inform the research from the procedures and techniques used to analyze the data and the resulting interpretation. 34 Framework The framework is the guide used to align how the research is conceptualized, data are collected, and results are analyzed. The first piece of the framework is ontology, or the way reality is understood. This sets the stage for a better understanding of what can be known and what makes something real (Sears & Cairns, 2015). Is it real because it can be touched, felt, or discovered? If so, the ontological bent is one of positivism. If something’s realness only comes into being because of construction or interpretation, then it follows an interpretivist paradigm (Sipe & Constable, 1996). Ontology is the broader worldview through which the research decides what is real and what the relationship is to this reality. This study is undergirded by the post-positivist view of critical realism, thereby acknowledging the reality of a world that can be known through observation but holds the tension of the role social construction plays in this reality (Cohen et al., 2011). Student understanding of the civic learning outcomes connected to their service-learning course can be observed through the completion of the CASQ, but students enter a service- learning course already molded by their race, gender, socioeconomic status, and their lived experiences (Whitley, 2014). The use of critical realism described below allows these aspects to be used in the analysis of the data. Additionally, during the formation of the focus groups, students were placed with others with similar demographic traits to allow for the possibility of a more open discussion. Recent research has shown that Students of Color are more likely to speak more freely about their experiences if they are with those who are of a similar background (Walls & Hall, 2018). The ability for students to be more open, especially in a context where race isn’t specifically named, became even more important due to the students’ enrollment in a 35 predominantly white institution and their participation in a pedagogical tool steeped in whiteness (Mitchell, 2008). An epistemological grounding follows from and connects to the understanding of reality defined by ontology. The relationship here is between the knower and what is known (Sipe & Constable, 1996). Knowledge may come in the form of generalizable and quantifiable data. Here, knowledge has an objective basis and is known free from context. Another epistemological perspective purports that knowledge may also result from dominant narratives passed down at the exclusion of other forms of knowledge. This is the conflict perspective of epistemology, where knowledge has been naturalized as told through the voice of the victor. Epistemology must be able to relate to the ontological basis of the paradigm. Traditionally, quantitative research follows from a positivist ontology to an objective epistemology that does not allow for the interrogation of privilege. This study, however, utilizes the conflict perspective to provide a way to look beyond the dominant to the voices of the marginalized. While not explicitly stated, service-learning is often positioned as being color- blind and supporting the development of all students who participate. This is evidenced by the assessment tools used with service-learning courses and what is not included in them. Most service-learning assessment tools focus solely on student learning outcomes with little regard for demographic data (Gelmon et al., 2005; Moely et al., 2002a). When race is a factor, it is often framed in a way that centers whiteness by assuming the process will still follow that of predominantly white students going into predominantly communities of color. By providing a space for minoritized students, especially those with non-dominant racial identities, to be part of the conversation around civic learning outcomes, researchers can better understand and potentially provide an alternative way of conceptualizing civic learning. This perspective does 36 not necessarily privilege one way or form of knowledge over another, but it does allow for a more robust knowledge base. Connecting the conflict perspective to the ontological base of critical realism allows for the introduction of a different way to approach quantitative research through quantitative critical inquiry. Researcher positionality Before I decided to pursue a doctoral degree to further investigate racial differences in service-learning, I heard the phrase research is “me-search” somewhere. I remember thinking that I was not going to fall into that category. However, I better understand that researchers do not approach their study from a purely objective stance. I have worked in the field of service- learning and civic engagement for over fifteen years. During this time, a few colleagues and I started looking around at annual conferences and our home institutions and wondered why we saw few scholars/practitioners of color. Included in our musings was the concern of the demographic makeup of service-learning courses. One colleague went on to write about this pedagogy as one being grounded in Whiteness and not designed to be beneficial to all students (Mitchell, 2008). The experiences of Students of Color seemed to vary dramatically from their White classmates. I wanted to go a step further and see if these differing experiences resulted in different civic learning outcomes for Students of Color. My connection to the field of service-learning and civic engagement provides me with a unique connection to this work in general and this research study in particular. The majority of my time as a community engagement professional (CEP) was spent in a large research university in the industrial Midwest. My primary focus during this time was on faculty development and integrating concepts of diversity, equity, and inclusion into service-learning preparation and implementation. Faculty development took the form of community partner connections, syllabus 37 planning, and student preparation for service. Most often the work did not end when the students started at their placement sites. I would often be asked to troubleshoot community partner complications and assist students in understanding their assignments. There was even a time when I read student reflection papers. Many CEPs will tell you similar stories about what a “typical” day looks like for them. My specific connection to this study centers on the courses I worked most closely with. The students who participated in this study were enrolled in two of the courses that I supported for approximately ten years before leaving the university. In fact, the participants of this study were enrolled during the first year of my departure. I had not worked with them directly and had been away from the university for almost two years when the data collection occurred. Jaeger et al. (2013) uses a picture frame analogy as they encourage practitioners to start putting theory into practice. As a practitioner, how I do my work is connected to how I conceive of conducting research. The methodology is connected to a researcher’s ontological and epistemological stances. A continuation of this metaphor would represent methodology as the type of camera used to take the picture. For example, if one believed that reality is something that is tangible and the way to know that is through quantifiable data, they would use a high- powered camera and lens that could show detail in a picture. A reality that is constructed and known through dialogue would be represented in a picture of people talking with a focus on facial expressions and body position. Quality research is done when all three theoretical stances align. Misalignment would cause unclear questions, inadequate or invalid data, and the analysis would be misleading at best and detrimental in the worst-case scenario. To best illustrate the interconnections of ontology, epistemology, and methodology, I turn to an explanation of my personal philosophical commitment/paradigm to include connections to practice. 38 Researchers do not enter their study from a purely objective stance. In their breakdown of four research paradigms, Lawrence Sipe and Susan Constable (1996) state that critical theory holds that reality is related to the interpretivist paradigm of being subjective and constructed, but this reality also consists of power dynamics that influences whose voices are heard. Although critical theorists often use methods of inquiry that lead to the production of varied truths, this does not rule out the use of quantitative methods in the pursuit of illuminating group experiences. Individuals who are experiencing similar conditions may be under enough constraint that they form a more unified reality. This belief is shared by Frances K. Stage, who has written about what it means to be a quantitative criticalist, especially in the social sciences, and how one can use that vantage point to construct research questions and analyze data (2007). Holding a critical-interpretivist view of reality allows me to problematize how the field comes to know what is real. Chang (2019) argued that his experiences and the reflection he did allowed him to compare what he was learning to previous experiences and see if his beliefs held up to the new information. This process is not unique and is the basis of learning. Yet, those who take a more positivist stance on knowledge often discount personal experiences. Beyond this is the concern with the way that marginalized voices are heard in the academy. I am concerned with looking at how those who are affected by policies must enact these changes in their practice (practitioners both inside and outside academia) to make their voices heard and valued. Additionally, my worldview opens the door to assumptions about conflict between those with and, conversely, without power. These power structures include inequalities based on race, gender, and even position within the academy. Specifically, I am interested in how marginalized student learning outcomes play out in a pedagogy developed within the dominant narrative using a vetted instrument used by service-learning faculty to assess student civic learning outcomes. 39 The value I place on these marginalized voices related to the value I place on the language of epistemic justice allows “the opportunity to leave impressions on old and new knowledge” for those who have routinely been left out of knowledge production (Gonzales, 2015, p. 28). These voices have historically been part of minoritized and racialized groups, especially as research participants. Often smaller in numeric representation, their voices are considered too small to count. Delimitations This research study was conducted at a large R1 university in the midwestern region of the United States. Several factors influenced the selection of this institution. First, this institution has a central office that supports service-learning and civic engagement on their campus through faculty development, student orientations, and community partner preparation. Second, by selecting only one type of institution, I was able to mitigate the differences that could occur due to institution type. As mentioned previously, some institutions have been using service-learning philosophies since their inception (e.g. HBCUs) and have different approaches to using service- learning. Finally, the chosen institution offered courses in teacher training and lifespan studies that were offered to multiple sections. Their service-learning component specifically focused on diversity and/or social justice. Sample Usually, students who complete the Civic Attitudes and Skills Questionnaire have just completed their service-learning course. However, the data for this study were collected in 2021 during the height of the COVID-19 pandemic, and in-person service-learning experiences had been suspended since March of 2020. To prevent delay of collection, participants were recruited from students who completed either a TT 222 or LS 227 service-learning course at Near 40 Midwest University during the 2019 fall semester. Each course had a coordinating instructor who emailed students with the link to the online survey, demographic data, and focus group participation option. The first recruitment email was sent out on March 3, 2021, to 16 sections of TT 222 which was approximately 350 students who completed the fall semester offering of that course. When the survey closed at 10pm on April 3, 2021, 52 TT 222 students had completed the CASQ. Of those students 43 were white and 9 identified as Students of Color. To increase the number of BIPoC students who completed the survey and potentially agreed to participate in a focus group, students were then recruited from LS 227. The coordinating professor of this course sent the email out to approximately 45 students who had completed her class during the fall semester of 2019. Six students completed the survey which included an additional four white students and 2 BIPoC students. Although the overall response numbers for the TT 222 students was 15% and the LS 227 students was approximately 7.5%, due to the time constraints of this study and the uncertainty of the COVID 19 pandemic, this became the sample size for the study. This subset of courses was chosen because their service-learning experiences had similar attributes. First, both courses require students to complete a minimum number of direct service hours, maintain a reflective journal, and receive a satisfactory evaluation from their community partner. It is important to note two differences in the course requirements. The first is TT 222 students must complete a minimum of 20 hours while LS 227 students complete 40 hours of direct service. Quality service-learning courses, however, do not give credit on the number of hours completed but rather the evidence of learning (Eyler & Giles, 1999; Howard, 2001). Another difference is connected to their placement locations. TT 222 students were all assigned to an education-based field site, whether that was at a local school, after-school program, or an educational agency. Students enrolled in LS 227 had a wider variety of placement options. Some 41 were at education-based placements while others could be at nursing homes, homeless shelters, or other placements dealing with human services. The second common characteristic was both courses partnered with the university’s community-based learning center, which provided a consistent screening and orientation process for students and community partners. Additionally, this meant that all students were placed at a vetted community partner agency, often with a longstanding partnership with the center. The final commonality important to this study is that both courses specifically addressed issues of diversity and social inequality. While the topic was similar, the courses did take slightly different approaches to integrate this into the service-learning experience and overall class syllabus. Students enrolled in TT 222 were asked to take a critical approach to understanding systems of power and privilege, while those in LS 227 were focused more on learning about diversity as it relates to human services work. Design and Methods The literature review provided several examples of research on student civic learning outcomes based on participation in a service-learning class. One of the most used instruments by those in the field of service-learning, and the seminal work in the field, is the Civic Attitudes and Skills Questionnaire (Moely et al., 2002b). Although other surveys have been developed over the years, their work tends to reflect that of the CASQ, which is still being used in contemporary research articles (Latta et al., 2018). This study used a mixed methods approach to interrogate this instrument, provide critical analysis, and uncover connections to a pedagogy of whiteness. Outlined below are the ways in which quantitative data were used, how qualitative data were used to enrich and supplement the survey data and provides information on the point of interface between the two for a mixed methods approach. 42 Quantitative Critical Inquiry Throughout the use of the CASQ and other surveys designed to assess civic learning, the analysis of the data is often not disaggregated by race. This gap in quantitative research led to the development of the overarching research question: Are there differences in civic learning outcomes for Students of Color who participate in service-learning courses? The use of the critical quantitative process outlines how this was integrated, starting from the study design through the analysis. The research design for this study is based on quantitative criticalist inquiry, also referred to as quantitative criticalism. As stated, traditional quantitative methodology upholds the notions of objectivism and researcher neutrality. However, recent research has shown that individuals have implicit biases that are often unknown to the individual and counter a person’s beliefs. Quantitative criticalism, or what has come to be called QuantCrit, uses a foundation of critical theory to reject these premises and underscore the impossibility of true objectivity by taking a researcher’s lived experiences as a factor in research design and analysis and highlighting the need for self-reflexivity (Gilborn, et al., 2018). Additionally, QuantCrit opens the possibility of using an empirical approach to answer questions of racial differences and the power systems in which they exist. QuantCrit adheres to many of the assumptions of critical theory as outlined by Kincheloe and McLaren (2005), addressing language, the nature of oppression, and the reproduction of oppression through mainstream research (Stage, 2007). Quantitative criticalist research utilizes quantitative methods to accomplish one or any combination of three objectives: interrogation of systemic inequities; problematizing dominant practices; and/or use context to uphold culturally relevant methods of inquiry (Stage & Wells, 2014). This research is focused primarily on problematizing the dominant practices within service-learning. 43 Quantitative Method and Design. The CASQ was used as a summative tool which allowed an assessment of students' civic learning as it is connected to specific attitudes and skills acquisition outlined by the six factors presented in the survey. Although it has been used in subsequent iterations by Moely et al. (2002a) and others who use the questionnaire in their service-learning courses as a pre-post assessment to measure change in civic learning over the course of a semester, the instrument was created using a one-time distribution at the end of a course. While both are valid uses of the instrument, assessing change in student outcomes is beyond the scope of this research. Using psychometric analysis as a guide, students were asked to self-assess their civic-learning outcomes approximately one year after the completion of their course. The CASQ specifically measures civic action, interpersonal and problem-solving skills, political awareness, leadership skills, diversity attitudes, and social justice attitudes. In addition to statements/questions in the questionnaire, students were asked to report the same demographic information such as age, race, gender, and other characteristics from the original instrument such as the number of service hours completed and previous experience with service-learning or volunteering. The original instrument used basic demographic categories, however, updates to the demographic sections were necessary to uphold the standards of culturally responsive evaluation and assessment standards for language use (SenGupta, et al., 2004). New demographic categories are more connected to the language of identity that allowed more participants to see themselves in the research instrument were used based on both inclusive practice and encouraging students to fill out the survey. In the iteration of the tool used for this study, students were allowed to select more than one race, ethnicity was separated into its own category, and more options outside of the gender binary were provided. The results were then disaggregated by race, primarily focusing on students who identify as Black, Indigenous, and/or 44 Person of Color (BIPOC). It is important to acknowledge two things: (1) each category within BIPOC is not monolithic, and people bring with them their own variations of experience (Burke, 2018); Simons, 2011); and (2) while it would be useful to look at each racial identity category individually, service-learning courses are still predominantly White; therefore, analysis was divided into White students and BIPOC students. The questionnaire was distributed using the online platform Qualtrics. Students first completed a randomized version of the CASQ statements as presented in the original article, except for the removal of statements in the Civic Action portion. The deleted statements referred to specific actions that were not available to students during this part of the COVID-19 epidemic. After students completed the CASQ statements, they were asked to complete the demographic section of the survey. Recruitment for the focus groups occurred at the completion of the survey. If students wanted to be eligible to participate in a focus group, they indicated their interest and were required to leave their contact information. This identifying information was removed during the quantitative analysis. Students were incentivized to participate by offering them a $20 gift card by email if they were chosen and actively participated in a focus group. Qualitative Design and Method. However, uncovering differences in descriptive statistics only provides a partial picture of why these differences could have occurred. For this reason, a mixed methods approach to the research was used by engaging in focus group interviews and analysis to work through a secondary research question: How do students, particularly Students of Color, make meaning of the statements in the Civic Attitudes and Skills Questionnaire as connected to their service-learning experience using current measures of civic learning outcomes? The addition of focus groups as a qualitative approach allowed students to unpack the connections between their service-learning experience and the statement from the 45 CASQ. These connections, or disconnections, provide insight into the ways students make meaning of the civic learning outcomes connected to their experiences. The data from these focus groups were later analyzed using the Glaser and Strauss constant comparison method, micro-interlocuter (Onwuegbuzie et al., 2009), and analytical narrative analysis. Focusing on both content and the body language and other non-verbal cues of the focus group participants (micro-interlocuter) provided a more reliable analysis of the qualitative data. Additionally, analytical narrative analysis provides an interpretation of data connected to critical realism, a central paradigm of mixed methods research explained below. Focus group interviews were conducted using the Zoom online platform. Since it had been at least a year since these students completed their service-learning, focus group interviews were the preferred method, as being in a group setting with individuals who went through a similar experience can aid in recall (Denzen & Lincoln, 2000). The questions were based on previous literature regarding the creation of the CASQ, qualitative studies on how Students of Color experienced service-learning, and my personal experience as a community engagement professional. Students, particularly Students of Color, were asked to interpret the statements in the survey and connect them to their service-learning experiences. A semi-structured interview protocol was used to allow for movement between broadly and narrowly focused questions. This method of interview also maintains a higher level of flexibility as participants may introduce different paths of inquiry that could be followed if within the parameters of the study (Hall, 2020). In this case, the questions moved from the overall experience to factor connections and then ended with statement connections. Participants were also asked to assist in the prediction of which factor, if any, would the researcher find the most difference in scores between white students and Students of Color. The role of the 46 interviewer was to act as a facilitator/moderator and keep respondents on topic but still allow for some discussion between group members. Additionally, the interviewer can prevent the domination of one group member at the expense of other voices (Hall, 2020). The focus groups were all recorded using the Zoom platform, allowing me and my peer debriefer to both (re)view the video and audio components of the interviews. Additionally, the first round of audio transcriptions was done through Trint and then further analyzed for accuracy by the researcher. The peer debriefer for this study was chosen because of their social identities and connection to community-engaged learning in higher education. I wanted the person who would be the second set of eyes and ears for this data to understand service-learning history and trajectory and have a different lived experience. Since I identify as a queer Black woman, I selected a peer debriefer who identifies as a straight white man. We have been in the field for about the same time as community engagement professionals at higher education institutions. His work has been primarily in New England in different capacities as a community engagement professional working with students, faculty, and staff. Most of my career has been in a large R1 institution. It was important to me to have a peer debriefer with significant time in the field, but who also had key identities different from my own to help mitigate biased interpretations due to our claimed identities. Mixed Methods The hallmark of mixed methods research is the integration of quantitative and qualitative methods of inquiry in a single research study. Early foundational scholarship in mixed methods research had a prescribed way of performing and writing about research that contained both quantitative and qualitative components. While this current study would not necessarily be classified as using mixed methods, later authors provide a more inclusive stance to mixed 47 methods research. Guest (201e) proposes that researchers move away from the gatekeeping standards of the past and move toward conceptualizing mixed methods based on integrating the work at any point along the research process and limit classification as mixed-methods based on the complete research study. Specifically, Guest suggests that a study can be classified as mixed- methods based on the “timing and purpose of the integration” (p. 147, italics in the original). Research into the nuances of student learning and skill acquisition requires the ability to go deeper into how students understand survey questions. This study was designed to allow students to talk about how they conceptualized and reflected on their service-learning experiences to make connections to civic learning goals. This design's quantitative and qualitative parts are weighted equally and follow an explanatory sequential design where the focus group data is used to explain the CASQ scores (Guest, 2013). Figure 1 shows how and where the two methods were integrated. Figure 1 Integration of quantitative and qualitative methods into the research design 48 CHAPTER 4: RESEARCH FINDINGS The previous chapters gave a substantive overview of the need to better understand how students make meaning of the civic learning outcomes connected to service-learning opportunities. The Civic Attitudes and Skills Questionnaire (CASQ) was the first to answer the call for a more rigorous assessment of attainment of civic goals connected to service-learning classes. The primary research question guiding this dissertation required investigation using both qualitative and quantitative methods. This chapter presents both the quantitative and qualitative analysis of this research study. The quantitative analysis is presented using two hypotheses, and the qualitative analysis provides a discussion of grounded theory and its relationship to the data. The quantitative data were collected first using the CASQ, and the qualitative group interviews occurred within two weeks of the survey closing. Quantitative data Analysis of quantitative data for this research study used the critical approach to quantitative data described in Chapter 3 called QuantCrit. This approach is not a step-by-step guide to analyzing data the “right” way but a process to help researchers consider the reasons why data is collected, the data itself, and the processes used to analyze data (Gilborn, 2008). The data presented below come from the respondents’ answers to each factor of the CASQ. The CASQ measures students’ ability to connect what they learned during their service-learning course to the statements found in the questionnaire. The statements are divided into factors to address the different civic learning outcomes that should be addressed during service-learning. The Likert scale data for the survey results were recorded in SPSS statistical software. The first step was to aggregate the data from both the Teacher Training (TT 222) and Lifespan Studies (LS 227) courses as the surveys were sent out in two groups. Using SPSS software, each factor 49 was analyzed to provide descriptive statistics of the mean for both Students of Color and white students. The number of data points for each factor is provided below as the number of statements in each factor as well as the number of students who responded to all the statements, varies throughout the survey. Disaggregated statement data One of the research questions for this study centered around uncovering any statistically significant differences in the mean values of each factor between Students of Color and white students. After the data for each statement and each factor were analyzed, the data were then divided, by factor, into white students and Students of Color. For the purposes of this study, Students of Color include any student who indicated a racial marker other than white, a non- white ethnicity, or any combination of the two. Hypothesis 1: Students of Color will show a statistically significant difference in individual factor mean scores compared to white students from the same sample. This will be tested using a null hypothesis stating there is no statistical difference between the Students of Color and white students which is both best practice and the belief during the creation of the CASQ. Using an unpaired t-test with Welch’s correction for unequal standard deviations between the data, each factor had an initial analysis of standard deviation and statistical significance. Figure one shows the standard deviations for the three sets of data: Students of Color, White students, and the full sample. Keeping with the quantitative standard for statistical significance of p < .05, factors one through five did not show any statistically significant difference in the factor mean comparison of Students of Color and white students. Factor six, which assessed diversity attitudes did show a statistical difference. 50 Figure 2 Standard deviation and p-values for CASQ factors l s e u a v - p d n a n o i t a i v e D d r a d n a t S 2.0 1.8 1.6 1.4 1.2 1.0 0.8 0.6 0.4 0.2 0.0 Factor 1 Factor 2 Factor 3 Factor 4 Factor 5 Factor 6 SD SOC SD White SD Full p-value 0.852 0.726 0.745 0.907 0.763 0.659 0.674 0.879 0.915 0.889 0.893 0.417 1.043 1.149 1.134 0.412 1.608 1.690 1.677 0.626 1.195 1.597 1.552 0.045 CASQ Factor The following information is divided by factor and shows each statement, the mean of each statement disaggregated by race, and the overall factor mean connected to their corresponding factor. Respondents were asked to indicate the level to which they agreed or disagreed with the statements using a five-point Likert scale. An explanation of each factor’s purpose and the number of statements is presented in the introduction to each chart. Factor One: Civic Action The first factor of the CASQ was designed to understand how students’ participation in service-learning influenced their plans for civic action. The statements are listed below. An * indicates a statement where Students of Color had a higher mean score. The means for the factor 51 overall are nearly equal. Figure 3 shows the mean of each statement and the overall mean for the factor (shown in the seventh column). Statement 1: I plan to become involved in my community.* Statement 2: I plan to participate in a community action program. Statement 3: I plan to become an active member of my community.* Statement 4: In the future, I plan to participate in a community service organization. Statement 5: I am committed to making a positive difference.* Statement 6: I plan to become involved in programs to help clean up the environment. Figure 3 Statement means for factor one disaggregated by race Factor Two: Interpersonal and Problem-Solving Skills The second factor was designed to understand how student participation in service- learning influenced students’ interpersonal and problem-solving skills. The statements are listed below. An * indicates a statement where Students of Color had a higher mean score. The means 52 for the factor overall are nearly equal. Figure 4 shows the mean of each statement and the overall mean for the factor (shown in the seventh column). Statement 1: I can listen to other people’s opinions.* Statement 2: I can work cooperatively with a group of people.* Statement 3: I can think logically in solving problems. Statement 4: I can communicate well with others. Statement 5: I can successfully resolve conflicts with others. Statement 6: I can easily get along with people. Statement 7: I try to find effective ways of solving problems.* Statement 8: When trying to understand the position of others, I try to place myself in their position.* Statement 9: I find it easy to make friends. Statement 10: I can think analytically in solving problems.* Statement 11: I try to place myself in the place of others in trying to assess their current situation.* Statement 12: I tend to solve problems by talking them out. Figure 4 Statement means for factor two disaggregated by race 53 Factor Three: Political Awareness The third factor was designed to understand how student participation in service-learning influenced political awareness. The statements are listed below. Figure 5 shows the mean of each statement and the overall mean for the factor (shown in the seventh column). The statements where Students of Color had higher averages are marked with an *. The means for the factor overall are nearly equal. Statement 1: I am aware of current events. Statement 2: I understand the issues facing this nation. Statement 3: I am knowledgeable of the issues facing the world. Statement 4: I am aware of the events happening in my local community. Statement 5: I plan to be involved in the political process. Statement 6: I understand the issues facing my city’s community. Figure 5 Statement means for factor three disaggregated by race 54 Factor Four: Leadership Skills The fourth factor was designed to understand how student participation in service- learning influenced leadership skills. The statements are listed below. Figure 6 shows each statement's mean and the factor's overall mean (shown in the sixth column). This factor included two statements designed to be reversed scored, making a lower mean more desirable. Those statements are indicated by a ^. The statements where Students of Color had higher averages or more desirable averages are marked with an *. Students of Color show a slightly higher average, which is due to the higher means in the reverse-analyzed statements. Statement 1: I am a better follower than a leader.^ Statement 2: I am a good leader.* Statement 3: I have the ability to lead a group of people. Statement 4: I would rather have somebody else take the lead.^ Statement 5: I feel that I can make a difference in the world.* 55 Figure 6 Statement means for factor four disaggregated by race Factor Five: Social Justice Attitudes The fifth factor was designed to understand how participation in service-learning influenced student attitudes around social justice. The statements are listed below. Figure 7 shows the mean of each statement disaggregated by race and for the full sample and the overall mean for the factor (shown in the ninth column). This factor included two statements designed to be reversed scored, making a lower mean more desirable. Those statements are indicated by a ^. The statements where Students of Color had higher averages or more desirable averages are marked with an *. Students of Color show a slightly lower average, which is due to the lower means in the reverse-analyzed statements. 56 Statement 1: I don’t understand why some people are poor when there are boundless opportunities available to them.^ Statement 2: People are poor because they choose to be poor.^ Statement 3: Individuals are responsible for their own misfortunes.^* Statement 4: We need to look no further than the individual in assessing their problems.^* Statement 5: In order for problems to be solved, we need to change public policy. Statement 6: We need to institute reforms within the current system to change our communities Statement 7: We need to change people’s attitudes in order to solve social problems.* Statement 8: It is important equal opportunity be available to all people. Figure 7 Statement means for factor five disaggregated by race Factor Six: Diversity Attitudes The sixth factor was designed to understand how student participation in service-learning influenced their attitude toward diversity. The statements are listed below. Figure 8 shows the mean of each statement disaggregated by race and for the full sample and the overall mean for 57 the factor (shown in the ninth column). This factor included two statements designed to be reversed scored, making a lower mean more desirable. Those statements are indicated by a ^. The statements where Students of Color had higher averages or more desirable averages are marked with an *. Students of Color show a higher factor average. Since three of the five statements are designed to be reverse-analyzed, a higher mean score for the overall factor does not mean students achieved the civic learning outcome. In fact, Students of Color had an overall mean well above their white peers. Additionally, factor six was the only factor with a statistically significant difference in mean scores between white students and Students of Color. Statement 1: It is hard for a group to function effectively when the people involved come from very diverse backgrounds.^ Statement 2: I prefer the company of people who are very similar to me in background and expression.^ Statement 3: I find it difficult to relate to people from a different race or culture.^ Statement 4: I enjoy meeting people who come from backgrounds very different from my own. Statement 5: Cultural diversity within a group makes the group more interesting and effective. 58 Figure 8 Statement means for factor six disaggregated by race Qualitative Findings Using focus group interviews, ten student participants answered four questions connected to better understanding the meaning that students make of civic learning outcomes as they connect to their service-learning courses. Presented below are the primary themes for each question. Most students interviewed were part of an early teacher education course. One student had taken a course related to family studies. I conducted four group interviews designed to create homogenous groupings whenever possible. The final groupings were three women of color, two white men, a group of three white women with two identified as not straight, and a final pairing of two white women. The first four columns of Table 1 show the salient demographics of participants and how they were divided into focus groups. The last two columns give more 59 information to create a fuller picture of the participants. Students who participated in this study took their service-learning class one academic year prior. Their class standing is based on when the data were collected. Table 1 Characteristics of focus group participants Research into the student benefits of service-learning has been integral in the institutionalization of community-engaged teaching and learning. Additionally, this work has been utilized by both community engagement professionals and faculty alike. The data was coded in collaboration with another community engagement professional involved in service- learning from a student development perspective and coded to find common themes. Whitley’s (2014) conceptual framework was used to connect themes to broader theories in the field. Question One: Most memorable part of your service Since the data were collected approximately one year after the end of their service- learning experience, this question was designed to help students begin to recall their service- learning experience from the courses mentioned above. Students primarily remembered the relationships they had with both the community members and their peers. Every student spoke about these connections when asked to recall something from their experience. Many students touched on the different community activity variables introduced in Whitley’s (2014) conceptual 60 framework connected to the service-learning experience. Student comments in this section connect to the time they were at their site, the quality of their placement, and how they were able to work with their lead teacher or another person of authority. Interactions with community. When students discussed their interactions with the community, most interactions were with children at an education-based placement or clients at their individual sites. Students at an education-based placement spoke primarily about the children they worked with. One student of color, Olivia, who worked at a local elementary school, said: The best, to me, the best part of my service was definitely being there on a weekly basis because I only, I think, I only missed one week, and I was in a third-grade class at an elementary school. And it was just so nice to see the kids, like, remember me and kind of like are excited to see me… but it was great to have, like, kind of an ongoing relationship with the same classroom of kids and like helping them learn about what they’re learning and stuff. This was a common theme that surfaced both during this question and throughout the course of discussing the survey. Sometimes the answer was like Olivia’s, where the participant just started speaking, and as the memory unfolded, so did the connection. Other times the answer was more straightforward, like Riley, who said, “And I think for me, the best part is getting to interact with the kids.” The majority of students interviewed were placed in an educational setting, but even for the student at a different placement, the connection points were similar. Emma, who was in a class focused on human development, also connected with individuals living at the nursing home where she was placed. She was the first to speak up and was enthusiastic when she said: 61 I got to just spend time with that community and the best part for me was just getting to know everyone and taking the time to, like, listen to their stories and conversations. And it was just… it was a good time.” Interactions with peers. In addition to being able to connect with people who were part of the community where the MLU students were serving, another theme that appeared was the connection that students made with their peers. This connection was stronger in students like Lily and Zoe, who were part of a cohort program. One thing that I definitely liked is that I am in the [cohort], so it was like nice and comforting. Through my first semester of college, I got to meet a whole group of people that I knew I was going to have a class with the next semester and possibly my sophomore year. And so I think that comfort really made it a lot easier to be open and honest with these people because they’re only like thirty of us and I knew we were all going to be somewhat close. (Lily) I would say my favorite part of doing the service-learning, I mean, obviously because I’m part of the [cohort] as well. So being, like, being able to learn with, like, a small group of individuals, like, you know well, you get to know pretty well. That was pretty great. (Zoe) One of the benefits of service-learning is its ability to foster high-impact learning spaces for higher education students by creating a sense of community within the class. Connecting with peers in a classroom, partly through this shared experience, provided the building blocks necessary to facilitate social-emotional learning. For example, Liam often mentioned how much he enjoyed this course. He pointed out that while he connected with the students at the 62 elementary school, he appreciated the ability to be part of a larger group of college students going through the experience together: Definitely one of the best classes that I’ve taken in college so far was [this class]. And it was a part of that service-learning component and kind of being able to have that camaraderie of going into the classroom, meeting with people who are all kind of experiencing it for the first time. Other notable connections. It was clear that connecting with people in the community and with their peers were top of mind for the students. It was also unsurprising that two other themes surfaced during the first question. According to the work done by Whitley (2014), there are seventeen student variables that are part of the conceptual framework. Even with the number of possibilities, students indicated that the best part of their service-learning experience was the connection to their chosen profession/degree and exposure to diversity. Service-learning is upheld as a high-impact practice, particularly around its ability to connect students to issues of diversity in the real world (Kuh, 2008). While the evidence of the Civic Attitudes and Skills questionnaire shows that students were able to make that connection, it is the words from the students who participated in the focus groups that really bring this to light. For example, Liam talked about a fellow student in his class that did not have a great experience not necessarily because her placement was not a good fit but because she realized she was not a good fit for teaching. She did not like being here. But at the same time, you know, in my head I’m thinking, well, you got experience as a teacher and you realized it wasn’t your thing, which I don’t think is… is really a bad thing. 63 Other participants were able to connect to their chosen profession through their interaction with the children they worked with. Olivia said, “... it was great to have, like, kind of an ongoing relationship with the same classroom of kids and like helping them learn about what they’re learning and stuff.” Another participant, Noah, was able to connect the best part of his service-learning experience to both emergent themes of connection to a chosen profession and exposure to diversity as shown in the quote below: And more specifically, even, it was coming from a background that was not very diverse. My high school wasn’t incredibly diverse and being able to be placed in a situation where I was exposed to various cultures and various ethnicities and being able to kind of still connect with the students and relate and kind of get to hear about what was a unique experience that, you know, we have to consider when we’re educating and when we’re interacting with other people, that there is different backgrounds and different experience, life experiences that everyone kind of has. Because of the nature of service-learning, it was important to also monitor how these themes continued to show up in the interviews. Question Two: Factor Connection Participants in the focus groups were asked to take some time to go over the larger headings, the factors of the Civic Attitudes and Skills Questionnaire using the condensed version that was given to them and think about the way the sections of questions were divided into factors. The students reflected on how they made connections to their service-learning course. Students were encouraged to be specific about their about how they were able to make the 64 connection using specific examples from their time in the community and in the classroom. This question was designed to have students think primarily about the service-learning portion of their class, even though they could draw from all parts of their course. If the conversation got off track, more probing questions were asked to bring the students back to their time in the community. Additionally, students were able to make as many connections to the overarching themes as they felt appropriate. My peer debriefer and I paid close attention to both the factors that were explicitly stated and in student reflections after. On several occasions, students would mention a factor as a connection or disconnection, but during further discussion would contradict themselves. These instances were coded using their discussion thoughts rather than their initial statement. If it was unclear during the focus group discussion, clarifying prompts were used. Political Awareness – Factor Three. The factor most mentioned was Factor three - Political Awareness. Political awareness in the CASQ is composed of six statements. Five of the six statements focus on current events/issues and the other statement is about involvement in the political process. Reflecting on these six statements, students were able to make this connection in two ways. The first was the connection students were able to make to their course readings and activities through discussions and critical reflection. The second was the way participants were able to have an outsider-insider perspective as they took their class information and saw it come to life, learning from both the students/clients they worked with and from the teachers/or others in positions of authority. Connection to course materials/activities. Critical reflection in service-learning courses encourages students to connect what they are learning in class to what they see during their time with their community partners. The quantitative data showed that many of those surveyed were able to find connections between their service-learning and political awareness, however, that 65 connection came primarily during their time in class. Of the ten participants in the focus groups, seven indicated that their service-learning course had a strong influence on their political awareness. It is important to note these focus groups were held between April 15-April 30, 2021. During this time, not only was the country in the middle of fighting the COVID-19 virus, but we were also contending with a renewed sense of racial discrimination especially connected to police violence. One such case occurred in Minneapolis, MN, and involved a police officer. The guilty verdict of the case against the police officer was announced the day before the focus group Mia was in. She briefly spoke about the case and then connected back to her service-learning course. As well as, kind of, what we discussed in the class of the political awareness was always kind of in the back of my head and always kind of talking about redlining. We talked about how minority groups are held down in all of these intricate ways, briefly discuss them and how it really affects the knowledge gap and the education gap, which was fascinating. And then I ended up doing a after school program and it kind of like dawned on me later on in the semester, like, those of us adults and those of us coming in - not all of us were white, but we were the only white people there. And seeing how some of the students were struggling with some areas of learning that should be simpler for them based on, kind of, my awareness of where age has fallen in education and it just kind of helped display to me like the education gap and potentially along with, like, political awareness and the way that minority groups tend to be discriminated against and not given as much of an opportunity or enough attention from an instructor. 66 Mia was not the only student that connected the political awareness factor with issues that were brought up during time in the classroom and then connected them to what they saw at the placement site. Olivia, one of the Students of Color, not only made direct connections back to the CASQ statements but included her community placement site as a co-teacher in this. I was going to add also political awareness was something that I was able to relate to what I experienced. First off, being that the district that I volunteered in happened to be one of the, unfortunately, one of the highest homelessness rates among families in [this community]. So there are actually a lot of homeless kids, houseless kids, or kids who are experiencing poverty within, you know, their daily experience. And so with that, there are a lot of challenges in the classroom when it comes to absenteeism or just, you know, people being not the correct at the standard grade level reading level… And then so that kind of heightened my own personal political awareness of the needs in that, in the classroom. But also I learned so much from the kids themselves because the kids were also politically aware of their surroundings… And so, you know, kind of the political awareness was happening kind of on multiple fronts. This connection back to the class materials is the ideal situation when it comes to service- learning pedagogy, but there were times when students were not able to make that connection back to their service experience. Service-learning courses get the bulk of their strength from combining academic content with community engagement, but that strength is contingent upon the faculty member/instructor being explicit about the connections and/or the community partner having an explicit role as co-teacher. During the focus group with Liam and Noah, they both indicated that they were able to connect their service learning to four of the six factors, and factor 67 three was included in that list. When this happened in this group, it was presented in list form and always grouped with factor five (social justice attitudes) and factor six (diversity attitudes). Factor five was commonly discussed with factor three and was tied for second place with Factor two - Interpersonal/Problem-solving skills which is discussed next. Interpersonal/Problem-Solving Skills and Social Justice Attitudes. As mentioned above, participants in the focus groups could name as many factor connections as they could explain. Two factors tied for second place with four votes each - Interpersonal and Problem- Solving Skills (factor two) and Social Justice Attitudes (factor five). The primary theme that came out of these factors was personal growth, with a secondary theme of vocational development. Quotes from both factors illustrate the theme and secondary theme below. Theme 1: Personal Growth. Students who discussed connections to either Factor Two and Factor Five consistently brought up that their service-learning experience did not necessarily create their agreement/disagreement or connection/disconnection with the statements, but it made them stronger in their abilities and convictions. For example, Liam connected Factor Two to Factor Five as he explained the way service-learning influenced both civic learning goals for him: It did develop, you know, certain problem-solving skills, certain ways of thinking of things. Because the other thing is when you start to think about things in a different way. So, when you start to think about, you know, black and brown students and systemic racism in the country and in schools, then you start being able to think of problems differently just because you’re expanding your mind, you’re opening up new options. And I think that that really helped expand the problem-solving skills. I don’t know if it created it, but it definitely helped 68 improve it and kind of made it easy to open up to a lot of things that you know are…were scary for me to talk about before then. Emma was the only focus group participant who did not come from an education major and, therefore, was not placed in an education-based placement. Her focus on human development had her serving at a nursing home. When she talked about her connection to factor two, she was very clear about how it connected and to which statements it connected. I think for me, factor two, the interpersonal and problem-solving skills, really came in handy listening to other people’s opinions. For me, the elderly community, they’re very opinionated and obviously I have my own opinions… So just listening to them and what they have to say was a really big factor that I had to learn. And then cooperatively working with a group of people. I had to run like bingo, Wii bowling and just working with them as like a collective group was another thing I had to learn how to do and showed up in this factor. Theme 2: Vocational Development. Many of the students interviewed were engaged in service-learning as part of their teacher education program and could often connect it to their future careers as teachers. Liam, as shown above, made that connection in his discussion of the joint nature of the two factors. Other students, like Sophia, made the connection between social justice, her service-learning class, and what it may be like when she is teaching. We talk a lot about social justice issues and how we can, how the biases that we have. So that was something that we also had conversations about because life is still happening while you are teaching. So, there’s definitely time for you to talk about resumes using essays and everything, but there’s also a great opportunity to 69 have conversations between people and give them the space to, you know, have these conversations because everyone’s different. Even though participants were not asked to talk about specific statements within the factors at this time, Sophia spoke about social justice attitudes in generalities and connected them to personal biases but is also able to connect them to Factor Four - Leadership Skills. This is further indication that many of these factors are not stand-alone entities but, in fact, inform and reinforce each other. Question Three: Statement alignment After participants reflected on how each overall factor connected to their course, they were then asked to dig deeper into the questionnaire. They were asked to talk about specific statements and if a statement was closely related to their service-learning experience. The student responses to this question were first analyzed by the frequency of answers. Those responses were then used to connect to service-learning as a pedagogy, practice, and philosophy. Students could also talk about disconnections, but these answers were not included in this part of the analysis. Factor One Theme: Civic Responsibility. The student responses to the way they were able to connect their service-learning experience to civic action was through the theme of civic responsibility. Students used their previous experiences, or a certain service ethic, and spoke about how this helped them understand the importance of continued involvement. In this factor, statements three and five had the same frequency of positive connection, with four responses each. Chapter three explained how factor one was reduced to six questions due to the ongoing COVID-19 pandemic. Although the United States was offering vaccines when this research was conducted, the future of face-to-face interactions was still unknown. In fact, due to research 70 protocol, these interviews could not be conducted in person. The statements that remained were mostly future-oriented to not skew the data. One student raised this in their response: So I guess for me, yeah, I almost want to group like those first three together because I don’t really know, like what the future holds. And honestly, COVID just puts such a cap in, like, my plans for this last year or so. It's kind of weird. The “cap in the plans” speaks to the overall uncertainty of what civic engagement was going to look like in the future. While not everyone could find a different outlet for staying connected to the community, one student was able to find a connection through a different service-learning class: I think the one that I can relate to the most was “I plan to become an active member of the community.” So, coming to East Lansing only about an hour and fifteen minutes from where I grew up, but it still was a whole different area. It’s not suburban like I’m used to. It's a lot more diverse and more liberal than where I grew up. So that was very different. But coming to a totally new area, it was nice to be involved because now I’m doing service-learning again. I’m doing it online, but I'm tutoring kindergartners and I don’t think that I would have had, like, the self-confidence to sign up and just do it if I hadn’t done it last year because I kind of already knew how it worked. Statement three, “I plan to become an active member of my community”, clearly indicates a future orientation, but it is the way the students talk about these plans by building on previous community service experiences that points toward civic momentum. While it is important that students want to continue engaging in communities, civic responsibility also connects to an increased civic understanding. Civic understanding goes beyond the doing and 71 indicates that the participant better understands the reason why the service is needed or important. Additionally, Olivia reflected on how she understands community: But also the third bullet point, being an active member of my community. I think one thing about being in a service-learning opportunity here is we were… this was an ability for me to get a glimpse into what the greater Lansing area’s public schools are like. I didn’t grow up in this area. However, it does really provide an insight into me wanting to make sure that I am an active member of the education community wherever I end up living permanently. I think it’s really important that people are aware of how their schools look and how we’re investing into the schools. And so I wish everybody was able to take a day or take a couple of weeks out and actually sit down and know what school is like in their area, because I feel like it would really change the way we prioritize education. Students' connection to statement five, “I am committed to making a difference,” and its connection to their service-learning experience was often fraught. This is a tension that should be expected as the overall factor was framed by the student participants as civic responsibility, which includes a deeper understanding of what it means to be engaged with one's community. While Noah chose this statement as one that presented him with a disconnect, it is his rationale that speaks to this tension: On the flipside of that, in terms of one that I don’t feel like connected. I don’t know, I mean, I’m second guessing, but like my initial reaction was, it was the second to last where “I’m committed to making a positive difference”. And it’s not because I’m not committed to making a positive difference. Just it seems so 72 vague and it didn’t seem to quite apply like, yes, positive difference, but in what way? Noah felt this presented a disconnect. Other students wrestled with this statement and how it connected to what they experienced during service learning. Kinsley chose it as a positive connection with a caveat: “But I definitely, I’d probably connect most to “I’m committed to making a positive difference” in whatever way that ends up showing up.” Riley, who was in a different group, also spoke to the connection as something positive but also presented a caution as well. ... these are things I kind of already felt before starting my service-learning and my service-learning just showed me, like, how good intentions to help the community can go really, really bad and I have to do better. Factor Two Theme: Personal Growth. Student responses for this factor spoke to personal growth through a deeper understanding of themselves to both work with others and how they address problems. There were twelve statements in this factor divided into the areas of communication, problem-solving, and working with others. Six students chose the first statement (I can listen to other people’s opinions) as having a connection to their service-learning course, and five students chose the eighth statement (When trying to understand the position of others, I try to place myself in their position). Both statements are connected to communication, but there was a difference between the way white students and Students of Color made the connections and reflected on the statements. White students made the connections to statement one to what happened during the class. The Students of Color, on the other hand, connected this statement to things that happened during their time working with the community. 73 The example that follows is how a white student made the connection to what happened during class. Liam spent time talking about how TT 222 helped him better understand why he believed the things he did. This self-reflection allowed him to be open to listening to other people’s opinions. I started analyzing, kind of like, what I believe and why I believe what I believe and in, kind of, finding out the reasons why I believe what I believe, I was able to more comfortably hear other people’s opinions and then explain why I actually do think differently or whether I think the same too, and, you know, it can be on both sides of the aisle to be talking to a conservative or could be talking to an extreme liberal or an extreme conservative… It really is just by understanding my own beliefs, I was able to understand other people’s better. When talking about statement one, students in the group that was all BIPOC made their connections to what was happening at the site either with the constituents or with someone in a position of authority at the agency. The last thing I would want to give some time to is the first bullet point: I can listen to other people’s opinions. I actually did spend a lot of time, I resonated with that one because I spent a lot of time with the teacher. Once the kids went out to lunch, the teacher would always talk to me about all of the issues that they’re facing right now with teaching and stuff… So I think it was really interesting to get her perspective about teaching as a career and kind of the issues with teaching and stuff like that. Statement eight was about putting yourself in someone else’s shoes. Again, the divide for the students came in the way they made the connections. During this same focus group comprised of Students of Color, Sophia was able to connect both statements to things that happened in class: 74 When you’re teaching, it’s like, it takes a village. And for me, I was working with my peers, like to mentor the kids and do like the teaching activities that we would do. So a lot of the time, we will conflict on what we thought we should talk about or how we should understand or deal with an issue. So that was something that I really had to learn because I’m not always like my leading style would probably be like, I know I can do it rather than like relying on other people first. So allowing myself to let other people communicate to me and realize, like, OK, this might be a better way of doing things. That was something that I always take as a learning experience when I work with others. These examples represent the responses of seven of the ten students. The fourth focus group had a different set of connections to Factor Two, but still within the theme of personal development. The students in that focus group also self-disclosed that they were in cohort programs. Cohorts are often created to allow students to go through a program together with others with similar interest to help establish an intentional learning community (Beck & Kosnik, 2001). For example, if the education program offered a rural education cohort experience, everyone in that section of TT 222 would have indicated an interest in rural education and would move through the program together. Since the differences within the TT 222 course experience are not part of this research, highlighting this difference is to provide additional information regarding this different train of thought. The conversation in focus group four centered on working with others, whether that involved resolving conflicts or working cooperatively. One example of this came from Kinsley, as she talked about working with the other students placed at her service-learning site. I think I’d also say “I can work cooperatively with a group of people,” but for a different reason. Because when we came in, everyone was like, OK, conquer and divide [sic]. Who 75 has what homework… And so, like, just who can do math, who can do social studies? And then I do English… So that was just, like, who can do this? Who can do this? And making sure that everyone has a person who like is going to be able to help, like, the kids with their homework. Another student in that same focus group, Zoe, started out talking about working cooperatively with others and moved a little into resolving conflicts without naming the statement explicitly. I would say the one that I related to the most in Factor Two would be more working cooperatively with a group of people just because in my classroom setting it was a lot of group activity. So being able to go from one group of students to the next group of students and being able to jump in and be like, “OK, what are we doing now? Let’s work together and try to figure it out.” Because if there wasn’t like a, like a teacher, like someone kind of watching over them, they would get a little rowdy, as kids do when they work with other people. It’s like being able to learn those skills and being able to work with a group of students that I didn’t think I’d be able to work with before. It was definitely something that really helped me and my service-learning. Zoe started to talk about students being rowdy with each other and Lily was able to connect to resolving conflicts between students as the way she connected to Factor Two. She did this by talking out the problems (statement 12). A lot of the students that I worked with in the resource room had like emotional disorders. So like talking to them could be a little difficult. They’re also like third, fourth, fifth grade. So it’s. kind of like, you know, you kind of got to think through things with them. So like talking through things with kids, like if a kid was like sent in the hallway because they kept putting their head down and, like, falling asleep on their desk… So, 76 like, talking to them about, like, OK what can we do to fix this? You know, like that was definitely, I saw myself using that in my service-learning. Factor Three Theme: Investment. The six statements in factor three could be divided into three types of statements. Three statements are about awareness (statements one, three, and four), two statements address comprehension (statements two and six), and the final statement is about future involvement broadly (statement five). Students in the focus groups overwhelmingly connected to statement four (I am aware of the events happening in my local community) and statement two (I understand the issues facing this nation). Many students who connected to these two statements were reflecting on the whole of their service-learning course. Lily, who connected more to what happened in class than in the community, made this observation about why it was difficult to make the connection to the local community. I would say that a lot of these I can relate more to my class than service-learning, but one thing that sticks out to me is the “aware of events happening in my local community,” because I wouldn't say that I really lived in the same community as my students. I mean, obviously there’s the [city] divide, but I’m living on a college campus and they’re living with their families somewhere in [city]. So our lives were just so different that I didn’t really know about a lot of the things that they were coming through and a lot of things happening in that community. Noah summed this up when he talked about the connections he made. But with that being said, the class in and of itself, I feel like, I felt like it connected very closely to the events happening in the local community, more specifically how the issues facing the nation applied and had effects in the local community and it made it local, this was real… But I mean, I’m almost, like, ashamed to say it, but like to a certain degree, 77 it’s like it’s not as real until it’s real and tangible and you can see this and you can apply and you can recognize it. Service-learning really helped that component be heightened. This factor is another example of the subtle difference in response between the focus groups with white students and one with Students of Color. The Students of Color focused primarily on what occurred during their time with their community partner. I think it was, it will probably be the awareness of events happening in the local community. I was working in [city] like here in [college city] because the students were here for a summer program at [college], so being aware of not only what was going on in [city] at the time and like what all the different kids school districts were dealing with, but I was also hearing about what their home life was like. Additionally, Olivia connected to statement four and presents an interesting segue into the overall theme for this factor, which is investment: All of them, honestly, I identify with all of the bullet points… not just what I’ve learned from students talking to me, not just from the things that the teachers told me, but also just me observing the classroom and the issues that a lot of the students and the family, and the students’ families face kind of have motivated me to be, well, I was already motivated, but to just is kind of further heightened the need for us to be aware of current events, issues in the local community, and then for the plan to be involved in the political process. While not every student pointed directly toward statement five (I plan to be involved in the political process), many students indicated a commitment to civic engagement at some point in their reflection. As noted above, the theme for this factor came not just from the primary 78 statements but also from the conversation the students had in their focus groups. For example, Riley found that her service-learning experience led her to become more invested in doing her own work: This might be the area that to me helped the most with, I think, just being raised in the community where I was kind of in my own little white bubble. It felt like a lot of times I was very, very ignorant and very unaware of, like, I had never heard of redlining until TT 222 and that one especially stood out to me as like, oh my gosh, like this. So this is calculated and it’s not yet been resolved. Like this is still how the world is, even though, like, the laws might not be there, we didn't really do anything to fix it. And that’s just seeing issues like that made me more aware. And I think it also has helped me get involved in other ways and plan on continuing education, because I know that, like, one semester there's so many issues and you can’t cover them in one semester. I kind of need to keep doing my own work there. And I think I have tried to continue that since taking this class. For some students, this investment was connected to their future careers as teachers. During a conversation in the fourth focus group, Lily and Kinsley both spoke about the importance of understanding what’s happening in the local community and situating it in the context of the United States in order to be better teachers. Zoe’s thoughts on this section illuminate this importance: Like I am knowledgeable of issues facing this nation. I feel like especially now like that’s super important just because with the Derek Chauvin trial, the verdict and everything, and then Adam Toledo being shot by police… As an educator, you need to be really aware of all of these issues that are happening right now. 79 Factor Four Theme: Purpose. The Civic Attitudes and Skills Questionnaire uses five statements to assess how a student’s participation in service-learning influenced their leadership skills. Statement 1 (I am a better follower than a leader) and statement 4 (I would rather have somebody else take the lead in formulating a solution) are reverse-scored when analyzed quantitatively. Taking a qualitative approach, I am more interested in how the students reflected on the connections they made. During the focus groups, seven of the ten students connected to statement five (I feel that I can make a difference in the world), and four students felt statement four (I would rather have somebody else take the lead in formulating a solution) connected to their service-learning experience. This statement was most often used in connection to a student’s career choice. Throughout the conversations, the overarching theme was about having or finding a sense of purpose. Olivia connected her sense of purpose to making a difference in the world on a small scale. She said, I only really resonate with the “I am a good leader” and then I can, “I feel that I can make a difference in the world.” This made me realize, you know, how large some issues are that one person can’t, you know, completely eradicate the problem, obviously. But I can at least, I can make a positive difference in someone’s day. And that’s what is important to me. Mia went on to connect to a sense of purpose in being a teacher. “... I have always been more of a leader than a follower, but this has given me a platform to do so with students and kind of trying to lead them towards understanding and learning and looking towards a brighter future.” Mia then commented, “And through that, I feel like I am making a difference in the world by making a difference in their world, by kind of helping this younger generation start thinking broader and brighter.” 80 Noah also connected his sense of purpose to his decision to become a teacher but could see it as his responsibility based on his positionality. He said, What service-learning really did for that, in my opinion, was it, I feel now because of this TT 222 course, that I can make a difference in the world for people who don’t look like me, who… people who are marginalized and discriminated against and advocate, because for screwed up reasons, the world listens to people who look like me more than they listen to people who look like them or whatever that is, and kind of be that safe haven in the school, you know. Factor Five Theme: Critical Consciousness. This factor relies on eight statements to assess if student participation in service-learning influenced their attitudes toward social justice. The statements could be divided into two types of statements. The first type of statement was based on individual responsibility, and the second type was connected to policy or structural changes. Students presented a strong connection to critical consciousness when talking about this factor. Critical consciousness in this case, most closely aligns with Paolo Freire’s (1970) conceptualization of “learning to perceive social, political, and economic contradictions and to take action against the oppressive elements of reality.” Most students, six out of ten, felt they were able to connect their service-learning in a positive way to statement eight, “It is important equal opportunity be available to all people.” Emma, the Lifespan Studies major, connected to statement eight in a way that became the baseline for how students connected. She said, “I think, just thinking inside my community and where I was at just, I was taught that you offer our services to every single person that you see… people need to make sure that they’re, they feel like they’re included and have the opportunity to 81 be involved.” Students often spoke about inclusion in this factor, but equality was just the beginning for many students. When talking about equal opportunity, students were able to both connect to that statement and take it a step further. That extra step was supplemented by the coursework. For example, Mia said specifically, “it was more in the class and seeing how things evolved and how the opportunities presented and given to us and the story shared were always to try and teach us and train us to start thinking more holistically. The only thing I can kind of think of in the bottom is equal opportunity is wonderful, but equal opportunity and understanding that equity is sometimes more important than equity.” Kinsley also indicated that she hadn’t understood the systemic side of the issue until her TT 222 class when she said, “I didn’t understand the need for institutional reforms or the need, like or what like, what equal opportunities for people meant versus like equity”. Noah’s words summed up the equality versus equity debate nicely. It’s important for equal opportunity to be available for all people, that is absolutely like, yes, we need to do our best to overcompensate, to compensate for the inadequacies and the systemic racism or systemic everything: sexism, xenophobia, whatever have you, that exist because you know, it should be fair to that degree… are not fair to use the, to use the word. It should be equitable, you know, and fairness is not equality is the distinction there. Factor Six Theme: Value of Diversity. This section was designed to understand how students’ participation in service-learning influenced their attitudes toward diversity. This factor consisted of five statements. Three of the statements were connected to personal attitudes about diversity, and the other two were about group diversity. The overall theme from the student focus groups was about the value of diversity, with two statements with the most connection points. 82 Seven participants said they were able to connect their service-learning to statement four: I enjoy meeting people who come from backgrounds different from my own. Six students were also able to connect to statement five: Cultural diversity within a group makes the group more interesting and effective. The definition or operationalization of diversity may change from student to student, but students repeatedly discussed how that connection to diversity allowed them to see an issue differently. Olivia, one of the Students of Color, said, “I think in the classroom that I was in was predominantly Black. However, there were also Latino students and it was really, really fun because I feel like having a diverse group makes the learning more interesting and effective for other people… And I think it was really cool to see students teaching other students about their own culture”. She was able to connect to the value of diversity not necessarily for herself, but for the benefit of the students in the class she was placed in. When asked to talk about factor six, Liam was quick to talk about his exposure to different people, the importance to his class, and how he sees the value in what he learned: … Because in some different way, I was exposed to groups of people that I may have not been exposed to. And, it helped, kind of, open up my eyes to the fact that, you know, you don’t need to be from, like the same community or whatever to really connect with people or to connect with students. There’s really universal things that you can still connect with. And you can always find something from a student or from an adult that they love, that if you want to invest in learning about it or if you want to invest and learn about it, then you can absolutely do that…And I think that is something that service learning really helped me with. 83 Some students reflected on how their class makeup was not as diverse as the places they were going in the community. Lily noted, I think that in a lot of cases, diversity is actually better. But the one thing that makes me think about is, I found a lot of the people who are in this department, we all kind of tend to have a lot of the same beliefs, even if we do come from different backgrounds… so that, I don’t know, I’m kind of iffy on that because, like, I think diversity is awesome, but also I think that, like, maybe my class wasn’t as diverse as I would like to claim that it is… Kinsley chimed in to add, “I definitely get what you mean about how teachers are like, “yay! Diversity” and we’re all like white middle-class females that live within an hour of each other…”. Question Four: Differences in scores The final question regarding the factors connected specifically to individual factors, and asked students to consider all the factors and talk about which factors might be different for students based on their race. Overall, students believed that researchers could find variance in primarily social justice attitudes and/or diversity attitudes. Every group mentioned diversity attitudes to some degree and three discussed how social justice attitudes would show difference. The reasons given for this are similar between the Students of Color and the white students. The following quotes from the students show where the subtle differences occurred. In the focus group with the three Students of Color, each mentioned that both social justice attitudes and diversity attitudes would be different based on experiences that Students of Color (broadly) may have before entering a service-learning experience. 84 Specifically for social justice attitudes, Olivia started talking about the factor by saying, “I think all of us were talking about how we already had kind of experiences, but then it was enhanced for the service-learning experience.” Sophia went on to add some ways that these experiences going into a service-learning course/experience could differ. While she didn’t specifically state Students of Color do things one way and white students do things a different way, but by connecting her reaction to this question to how adamant she was when talking about the need for structural change during the discussion of factor five statements and how she connected her comments to what Olivia said about student experiences made it more apparent. She said, And you do find most people are pretty aware, but you can’t discount the few that are not. And just because people are aware, they might not necessarily… some people have more faith in government than others. Some people have, you know, they feel that people should have a greater sense of accountability than others. So they’ll blame people for things that they’re experiencing. The Students of Color in this group focused on the connections to social justice from personal experience because of their connection to the need for social justice in a way that their white classmates may not have. Factor five was also named by students in the remaining focus groups, but their connection to this factor was not as strong. The question specifically asked about differences between BIPOC students and white students. Liam had been vocal during the focus group with very firm opinions about his positions. When asked to think about where differences could be, he said: 85 I really… I’m having trouble with this, honestly, because I really, you know. I guess this is what comes out of trying to put myself in other people’s shoes. But, you know, I think it’s difficulty because every single person is so individual, you know, and for me to say that as a… as a white man that I would do all of these things differently than maybe like a black or brown person is kind of… Difficult for me to say. When this group discussed social justice attitudes, it was Noah who spoke to Liam’s discomfort by talking about his privilege as a white man. I have the privilege of being neutral. My identity gives me the privilege of not having a completely disagree or completely agree opinion, and I feel like that for a variety of reasons. Whether that’s just because I… like, one of the things that has come to my attention is I have the privilege of not being aware of a lot of these social justice issues because they don’t negatively affect me. This connects back to the way the students in the first focus group of all Students of Color discussed how their experiences as Students of Color enhanced or even affirmed their positions. This, for them, is a different kind of lived experience that enters the student service-learning equation (Whitley, 2014). The other factor students felt would be different between white students and Students of Color was in diversity attitudes. Starting again with the first focus group, students talked about how their classes were fairly homogenous. Olivia said, “I did notice in my classes like those mostly white women in my classes, but I was also there because a lot of the TT 222 are courses for like the education major, which is predominantly white women so…” The other students in the group gave nonverbal signs of agreement. Emma followed up, saying, “The ones that there 86 would be like most of the difference in just because of how people perceive these sort of things, some people have, like, a different outlook than others.” Although the Students of Color did not spend much time talking about this factor, it was clear that their primary concern was about how/if a group of people who are so similar would feel about diversity. The students in the other focus groups were also cognizant of both the lack of diversity in their courses and people’s perceptions of different topics. Some focus groups had already acknowledged the lack of diversity within their classes. During focus group four, which had already talked about the homogeneity of their class, some of the conversation centered on different types of marginalized identities (e.g., LGBTQ+ or international students), and they would come back to the comparison to white students and how experiences for “black or brown” students would be different. When prompted to speak on the topic, Zoe connected the difference in student experiences and how the field itself is homogenous. So I was just thinking about factor six, the diversity attitudes. I feel like if we were to ask, like a group of people of color, I feel like they would have a little bit of a different like opinion on this factor just because… Obviously, they’ve had a lot of different experience than I have because I’m a straight white woman, like I obviously have not experienced a lot of the things they have. So I feel like that factor would definitely provide a lot of interesting answers in comparison to the majority of educators, which would be white women. Riley, when talking about factor six as one that might be different between white students and Students of Color, touched on people’s perceptions when she said: I feel like even just the word diversity itself probably has a different meaning to someone who’s not white, because when I hear… It’s just like I’ve always been surrounded and 87 immersed in white privilege, and when I hear, like people… I wish it wasn’t this way, but I think of like people who look like me and when I hear diversity, that’s like what triggers the alarm of people who look different from me. These observations from both the Students of Color and the white students are not surprising. Whitley (2014) synthesized the work of different leaders in the field who wrote about the importance of the context of the students participating in service-learning. Context in this case consists primarily of the different intersectional identities students have as they enter into their service-learning courses. The majority of research has not yet considered how these intersections influence how students engage with their service-learning experiences and the impact they have on their civic learning outcomes. 88 CHAPTER 5: DISCUSSION, IMPLICATIONS, AND CONCLUSION The purpose of this study was to better understand the way students make meaning of the civic learning outcomes connected to participation in a service-learning class. Using the Civic Attitudes and Skills Questionnaire, students were asked to both complete the instrument and were given the opportunity to participate in a focus group for further exploration. Additionally, I wanted to disaggregate the data to look for differences in responses of Students of Color and white students. Both the quantitative and qualitative portions of this study were informed by relevant service-learning theory (Whitley, 2014) and pedagogy (Furco, 1996; Kiely, 2005; Butin, 2005; Jacoby, 2015), critical race theory (CRT), which allowed for critical quantitative analysis, and my curiosity and experience as a practitioner-scholar. These combined factors led to the overarching research question guiding this work: How do students both assess and make meaning of the civic learning outcomes associated with their service-learning class one year after completion? To connect student civic learning outcomes (quantitative data) to how students make meaning of the survey and the service, this research utilized two methods of inquiry. First, students were asked to complete a modified Civic Attitudes and Skills Questionnaire (CASQ) with certain statements removed due to the ongoing COVID-19 pandemic. At the end of the survey, students could choose to participate in a focus group held approximately two weeks after the survey's close. Ten students were chosen to participate based on their demographics and availability and then divided into four focus groups. Each group was designed to have as close to a homogeneous group as possible. One group consisted of all women of color, two groups were all white women, and the final group was all white men. Each focus group met for about an hour to an hour and a half and went over the different factors and statements of the CASQ. 89 Additionally, participants were asked to identify factors where researchers might find differences between white students and Students of Color. Based on other research on civic learning outcomes and service-learning, I had a basic idea of where the differences could occur. This question started to provide possible reasons for the differences and if other factors might also be included. Connection to Theory The data collection and analysis were guided by critical race theory and connected to a conceptual framework that acknowledges the differences in student context and recognizes the positive civic outcome goals of service-learning for all students. Critical race theory (CRT) provided the basis for including the voices of Students of Color who were often overlooked because their enrollment in service-learning courses, specifically at predominantly white institutions, did not provide sufficient numbers to run more sophisticated quantitative analyses. Larger data sets were important to the institutionalization of service-learning and contributed to the current state of service-learning (Bringle & Hatcher, 2000) and led me to question its benefit for all students, especially those with minoritized or marginalized identities (Finley et al., 2022). Additionally, CRT was used to provide a basis for a critical approach to quantitative data as well as potential explanations for the nuances of the qualitative portion of the research study. The conceptual framework was adapted from Meredith Whitley’s 2014 work. This adapted framework pulls out the salient portions of her work with a primary focus on race. The use of critical race theory as a starting point for inquiry and a method to collect and analyze data led to a second question about the differences in mean scores and meaning-making for Students of Color. Student responses to both the CASQ and the focus group questions were examined, keeping the voices of BIPOC (Black, Indigenous, and People of Color) students at the 90 forefront. The survey responses were disaggregated by race in such a way that it upheld the dichotomy of Students of Color and white students, while it also allowed students to self-identify when appropriate and the quantitative data can be further analyzed by others. The focus groups were designed to be as homogenous as possible to allow for the creation of a safe(r) space for student interaction. Chapter Three provided more information on both the quantitative and qualitative methodologies as well as how the two were integrated to create a mixed-methods research design. Chapter Four presented the data for both the survey data and the focus groups separately. In the following sections, the findings are summarized by factors outlined by the Civic Attitudes and Skills Questionnaire (CASQ). Each factor is analyzed by mean scores when comparing Students of Color to white students, white students to the full sample, and Students of Color to the full sample, with a focus on statistical significance in the difference between Students of Color and white students for the overall mean score of the factor. The focus group data are very important as they can show not only where connections or disconnections are coming from but they can also illuminate why some connections were not made. These student discussions point to student meaning-making of their service-learning experience and the statements in the case as possible contributors to their scores. Critical Race Theory Critical race theory (CRT) and its application in education research (Ladson-Billings & Tate, 1995) was used as the overarching theory for this study. From that lens, the analysis of the data becomes richer as it allows for certain assumptions to be made. Those assumptions connect to the five tenets of CRT in the following ways. 91 First, the tenet of counter-storytelling is typically presented in providing those who have been historically marginalized the opportunity to tell their stories using their own words. While this is prevalent in qualitative methods and shown in chapter four by always providing an example from the Students of Color focus group, counter-storytelling can also occur in quantitative analysis. In the discussion below, survey analysis is explained and presented so the nuanced differences in the data can be seen. Additionally, counter-storytelling is also present when using participant responses to illuminate or counter what is expected from the survey data. Another tenet is the critique of liberalism that leads to issues of neutrality and color-blind pedagogy. Therefore, the quantitative analysis of the pedagogical practice of service-learning cannot be color-blind. The CASQ was designed under the premise that any student who participated in a service-learning course would have similar results (Moely et al., 2002b). While the original creators of the CASQ did note differences based on factors such as previous experience, age, and gender, the probability of a difference in race was only important when thinking about the third tenet of CRT. A third tenet of CRT is connected to the permanence and everyday occurrence of racism. Racism is part of the everyday and should be considered the rule, not the exception. In the interest of this study, this tenet speaks to two parts of the analyses. Using qualitative methods, one does not necessarily have to call out racism for the experience to be included in a conversation. Additionally, because racism is part of our everyday lives, sometimes what is not said is as important as what is. In quantitative research, this connects to constructs that are not assessed, often due to low numbers. The fourth tenet is interest convergence which was described earlier as the way in which the institution of whiteness makes it acceptable to promote racial justice if it serves white people. 92 This can be seen in the CASQ statements as the goal is for students to better understand themselves in relation to an “other” who is not part of the majority group. It leaves out the connection students might have to non-majority identities, specifically that of race. While this is specific to the added factors of social justice and diversity attitudes, this interest convergence can be seen in some of statements in other factors. Service-learning theory Service-learning has many connection points to pedagogical theories within higher education. Many practitioners cite connections to Dewey, Freire, and Mezirow in their work in service-learning. Using the well-documented theories from Whitley’s 2014 article that pulled service-learning theories together in a comprehensive framework, the focus group interviews were grouped into the themes seen in Chapter Four. Most important in this study are the proximal and distal outcomes impacting students on a personal level, in their academic lives and career choices, the ways they interact with social and civic responsibility, and their attitudes toward diversity. This study limited many of the mediating variables connected to the critical reflection and critical thinking components of service-learning by primarily drawing from two courses with similar service-learning requirements at the same Midwestern R1 institution. While the student contexts were varied, they were representative of student demographic diversity in service-learning even with the changing demographic landscape of higher education. Summary of Findings The Civic Attitude and Skills Questionnaire was the first vetted instrument to assess student civic learning outcomes in the field of community-engaged teaching and learning (Moely et al., 2002a). The quantitative data for the current study were collected during April 2021. Since higher education institutions had dealt with the ramifications of the COVID-19 pandemic for 93 over twelve months, the student participants were collected from a pool of students who had taken either a Teaching Training 222 (TT 222) course or Lifespan Studies 227 (LS 227) course during the 2019 calendar year. The CASQ is often administered at the end of a service-learning course. Since all in-person service-learning courses were suspended starting March 2020 and had not restarted, the participants were reflecting on service that occurred between 24-28 months prior. The CASQ was administered in nearly its entirety. As mentioned in Chapter 3, some questions from Civic Action were removed due to the uncertain nature of COVID-19. The data were analyzed both by factor and by individual statements. Comparisons were made between the difference in means between Students of Color and White students, Students of Color and the full sample, and White students and the full sample. Additionally, I could note any directionality of the difference by looking at the means for each group. This analysis allowed for the discussion of three main points within each factor: the statement with the biggest difference in means, the statement that was most similar, and if the difference in means between Students of Color and White students were statistically significant. The graphs for each factor can be found in the appendix. In general, the visualization of the difference in mean scores by statement showed the line for Students of Color/White (s/w) students, and the line for Students of Color/Full (s/f) students run parallel to each other with the s/w line being higher (have more difference in the means) than the s/a line. The line showing the difference in means between white students and all students (w/f is typically a flattened version of the other lines. It is rare for this line to be over 0.1. It seems reasonable for these differences to be seen in this way. The original CASQ was based on a predominantly white and female population. Additionally, based on the different variables in play for Students of Color, it seems reasonable that they would have a greater 94 difference in statement means because there are fewer constants. Qualitative information is included in the quantitative summary of findings when it is relevant. When appropriate a richer summary of qualitative findings will be included in the overall discussion of the factor. During the group interviews, students were asked several questions asking them to specifically connect statements and whole factors to their service-learning experiences. The themes discussed in chapter four are used here for further context. Factor One. This factor assessed how participation in service-learning influenced students' plans for civic action. The largest difference in mean scores for this factor was on statement six between white students and Students of Color. This statement was connected to a student’s ability to connect their participation in service-learning to plan to become involved in programs to help clean up the environment. White students had the larger average mean of this statement. It is important to note that students in the focus group unanimously found this statement to be the one with the greatest disconnect to the overall course. During the discussion of this statement in the focus groups, students often talked about how it wasn’t that they were against working to improve the environment; it just didn’t connect to their service-learning course. The statements that students connected to the most were statements connected to their civic responsibility. Of these statements, Students of Color had higher mean scores than white students. Students of Color had a lower average when connecting their service-learning to their intention to be involved in their community. The difference in the factor means could suggest that Students of Color are a little more invested in civic action than their white classmates. I do not believe this correlation can be made. Starting with Whitley’s conceptual framework (2014) around student context, two interesting connections were made during the student focus groups. The first was around prior volunteer 95 experience. Students often talked about how they had previous volunteer experience, whether that was in high school or within their religious organizations. It was then difficult for students to determine whether their service-learning course was the reason behind their feelings of civic responsibility. As service-learning becomes more institutionalized within both secondary and higher education, connecting one specific course to civic learning outcomes will become more difficult. The Civic-Minded Graduate scale (Steinberg et al., 2011) attempted to mitigate some of these biases by having students think about their entire higher education experience and how they connect to particular learning outcomes. While this can be helpful overall, it does not add to the assessment and evaluation of class learning outcomes. Factor Two. The second factor of the CASQ connected service learning to interpersonal and problem-solving skills. This factor has almost double the number of statements as the other factors. Statement nine had the biggest difference between Students of Color and white students. This connected their participation in service-learning to the statement, “I find it easy to make friends”. Students of Color averaged almost one whole point lower than their white classmates. This statement was also the one that students found had the greatest disconnect with their service-learning experience. During the focus groups, both Students of Color and white students did not see this as an outcome of their service-learning experience. The first statement, however, was indicated as the one with the greatest connection to service-learning and was the statement that had the smallest gap in mean scores. The difference between Students of Color and white students was .08 with Students of Color having the higher average. This factor also presents an anomaly not found in the other factors. The tenth statement that connects service-learning to the ability to think analytically in solving problems had a change in the usual trend where the line graph for the difference in means for Students of 96 Color/white students and Students of Color/full group ran parallel to each other. The difference in means for Students of Color and white students was lower than that of Students of Color and the full group. Additionally, the difference in means for white students and the full group was closer to the difference in means for Students of Color and white students. Visually the line for the Students of Color/white difference dipped below the Students of Color/full group line. Additionally, the line for white/full went up to meet the Students of Color/White students group line. Students of Color rated themselves much higher than their white counterparts, therefore impacting the full group mean. During the focus groups, students did not find a connection to the tenth statement. They primarily focused on listening to other people’s opinions (statement one) and placing themselves in the position of someone else (statement eight). Both statements are connected to the overall theme of communication for the full group, but white students made the connection to activities during the class, while Students of Color connected these statements to experiences in the community. Factor Three. Factor three connects students’ service-learning experience to political awareness. This factor visually followed the trend of s/w and s/f except for statement four where the s/f dipped below the w/f line. This will be explored below. The largest gap in the difference in means was in statement six connecting service-learning to the “issues facing my city’s community.” It is also important to look at the second largest difference in means which is also between Students of Color and white students for statement one connected to awareness of current events. In both cases, Students of Color self-reported scores a little over .4 lower than their white classmates. The only difference in these two statements was that more students found a disconnect for statement six than for statement one. An overall explanation for the difference in 97 scores is that students were not able to make the connection to their awareness of current events at the time of their service-learning because of the amount of time that had passed since their experience. However, that does not explain why there was such a large gap between the two groups of students and why Students of Color rated themselves so much lower than their white classmates. The overall difference in means was not statistically significant. A discussion of the lower scores by Students of Color is covered below. The statement with the most similar response average was statement four between Students of Color and the full sample. Statement four connects service learning to the events happening in a student’s local community. This occurrence is interesting for several reasons. First, the difference in means for Students of Color and the full group generally follows parallel to the difference in means for Students of Color and white students. In this instance, the s/a line is the lowest. Second, the difference in means for white students and the full sample rises above the s/f line and almost touches the s/w line. Third, the average means for white students is lower than the other two average means. Finally, during the focus groups, students found that statement four had the most connection to their service-learning experience. Factor Four. This factor assessed students’ development of leadership skills as a result of their service-learning experience. It is important to note that several of these statements were intended to be reverse-scored. Instead of doing that, as explained in chapter four, statement values on the Likert scale remained the same during the data collection and analysis. When the data was interpreted, however, the rationale for not reverse scoring these statements was discussed. Additionally, this factor statement analysis followed the expected trend with the largest differences in means existing between Students of Color and white students. The statement with the largest difference in mean score was statement four which was about whether 98 they would rather have somebody else take the lead. Students of Color rated themselves almost a half a point higher than their white counterparts. The original CASQ had this as a statement to be reverse scored, which means that the “correct” answer is to prefer to take the lead. This has its own complications, which are discussed below. However, what is important to note here is that Students of Color would most often have someone else take the lead at their placement site. The statement with the most similar response was statement three connecting the service- learning experience to the ability to lead a group of people. While this is technically the statement where white students are most closely aligned with the full group ranking, it is important to note that there is only a .05 range between the difference in scores between statements one, two, and three. Additionally, the difference for the other two group comparisons is also very close together for statements 1-3. Students of Color have the higher averages for statements one and two and the lower average for statement three. Statement one is reverse interpreted, which means Students of Color are slightly more likely to indicate they are better followers than they are leaders. Again, the differences in these three averages are very similar, and the overall factor p-value does not indicate a statistically significant difference in the overall factor mean for Students of Color and white students. Factor Five. Factor five evaluated the extent to which their service-learning experience influenced their social justice attitudes. The statement differences for this factor had some variation from the general trend of the other statements. The difference in means for white students and the full group had greater variance than that of other factors. Additionally, the differences between Students of Color and white students showed greater highs and lows. The first four statements in this factor were all reverse analyzed. For this factor, the statement with the greatest difference in the mean score was between white students and the full group on 99 statement two. Statement two is connected to the belief that people choose to be poor. As this is one of the statements that is reverse analyzed, a “good” score would be lower than a high score. In this instance, white students agreed with this statement more frequently than their classmates of color. The statement with the least amount of difference between means was between white students and the full group on statement three. This is a change from statement two, where the differences between these two means were greatest. Statement three asks students to reflect on how their service-learning experience influenced their agreement with a statement about individuals being responsible for their own misfortune. This statement was also reverse analyzed, and the goal for this was for students to have a lower score. The difference here between White students and the full group was only .004. It should be noted that this statement did have the least amount of difference between the three sets of student populations overall. Additionally, the difference between Students of Color and either White students or the full group were similar. In this case, students mostly disagreed with the statement. The mean score for this state for Students of Color was 2.5, which is closer to a neutral response than one of disagreement. Factor Six. The final factor connects service-learning to students' attitudes toward diversity. Factor six only had five statements in the instrument. The statement trend for the differences in means followed the same parallel pattern as many of the other factors. The greatest difference in means was between Students of Color and white students on statement two. This statement asked students whether they preferred the company of people who were very similar to them. This is another statement designed to be reverse coded and was, therefore, reverse analyzed. For students to have achieved the appropriate learning outcome, the original developers of the instrument wanted students to lean more toward disagreement and therefore 100 show means closer to one. Students of Color, however, had a mean of 3.6, which is above that of a neutral stance. The mean for white students was a full point below that. Previous research indicates that Students of Color often feel they are asked to play the roles of either teacher (Walls & Hall, 2018) or translator (Smith, 2019) in service-learning courses that have students that are predominantly White, especially if they are completing their service in communities of color. It is not surprising, then, that Students of Color may leave a service-learning experience with the desire to be around people who have a similar background. The statement with the least amount of difference was connected to the level to which students got enjoyment from meeting people who come from backgrounds different from their own. The mean for this statement for all groups was 4.7 and ranged from Students of Color with a mean of 4.71 to white students with a mean of 4.73. This factor is the only one with a p-value less than 0.05 showing that the differences in means for the entire factor for Students of Color and White students were statistically significant. Limitations The limitations of this research design are four-fold: student self-reported data, lack of control, time since course completion, and the COVID-19 pandemic. Self-reported data is, by its nature, subjective. However, using a vetted instrument such as the CASQ, which is tested for reliability as it pertains to the tendency of participants to answer questions they feel are more socially desirable (Moely et al, 2002b), should have mitigated some limitations of self-reporting. Additionally, using quantitative critical inquiry allows for the acceptance and appreciation of the subjective as it speaks to the lived reality of the individual. The second limitation is the lack of control over the implementation of the service-learning component. Broadly speaking, the gauge for quality service-learning implementation requires adherence to the three main components of 101 service-learning: community engagement, academic study, and reflection. An additional data point for this study could have been course syllabi. Unfortunately, due to the number of faculty/instructors who teach TT 222 and in connection with the third limitation, time since course completion, it was not possible to include syllabi in this study. The CASQ is designed to be administered at the end of a service-learning class. However, the COVID-19 pandemic required canceling all face-to-face activities. Some students could find a way to continue serving in a community online through an academic class or volunteering on their own. After the time they were away from their class, some students indicated that it was difficult to say if their responses were connected to their service-learning class specifically or if they were influenced by other activities they did in the interim. Implications Service-learning has been upheld as a high-impact practice for students in higher education without a substantial deep dive into the differences in outcomes for students based on different identity markers. In this case, the racial background of the student was the key marker when looking for differences. The data presented only shows a few statistically significant differences in civic learning. The analysis of this data was conducted to better understand how students make meaning of their service-learning experiences as they are connected to the varied civic learning outcomes. These outcomes are what hold service learning up as a high-impact practice as they allow students to take a deep dive into their discipline (academic learning) and provide a way to have broad application for success after college (Kilgo et al., 2015). The metrics to determine success after college are connected to the six factors presented in the Civic Attitudes and Skills Questionnaire (Moely et al., 2002b). This study uncovered that two key areas should be addressed for service-learning to hold as a high-impact practice for students from 102 diverse backgrounds. First, the focus should be on the experiences of the student and being cognizant of the various intersectional identities held by students. In addition, practitioners need to be aware of both the context/environment of the classroom and community partners. Secondly, training and support for faculty and other community engagement professionals need to be part of the institutional structure for service-learning. The following discussion covers these implications for the field connected to student civic learning outcomes. Students enter higher education with identities, backgrounds, and abilities that are both unique to the individual student and connected to the larger field of student development. When dealing with race as the primary context, it is important to understand racial identity development theories and how they influence student service-learning experiences, impacting how they achieve civic learning outcomes. A broader context outside of the identities and personal backgrounds of students enrolled in service-learning classes draws from both key tenets of critical race theory and the characteristics of white supremacy culture (Okun, n.d.). Critical race theory and white supremacy culture give us a deeper understanding of how systems were not created to support Students of Color. Higher education must work harder to ensure equitable access to quality practices. Critical service-learning is a first step in removing some of the institutional barriers and creating a new structure (Mitchell, 2008). Student development theories assist educators in creating useful syllabi to guide students along. Additionally, research has been done on quality practices for undergraduate teaching and learning (Chickering & Gamson, 1987). While it would be difficult to mandate faculty to understand racial development theory in addition to their content area, student affairs practitioners could be a solution to this. Since the institutionalization of service-learning on many college and university campuses, there has been an increase in centers specializing in community 103 and civic engagement. Faculty members often seek out the services of these centers to help them find partners, create syllabi, and learn quality service-learning practices (Welch & Plaxton- Moore, 2019). The community engagement professionals (CEPs) at these centers and offices can then provide information to the faculty on student identity development, including racial identity, and how it impacts implementing a service-learning course. Further Research As noted in the delimitations section, this study was conducted at a large, predominantly White, research one institution in the Midwest. One direction this research could go is to take samples from different institution types and/or institutions in different areas of the United States. If the focus was on institution type, other researchers may find different results due to the way different institutions support and promote faculty who engage in teaching versus those who focus on research. Additionally, this study was done at a public 4-year institution, but there is much to be explored within both community colleges and private and/or religiously affiliated institutions. In addition, as the United States continues to see pushback on issues of diversity, equity, and inclusion as well as the use of critical race theory, the location of an institution may have a greater influence on student civic learning. Aside from institution type and location, researching courses within different disciplines could also influence students' civic learning outcomes even as it is disaggregated by race. I chose two courses within the social sciences that were explicitly focused on diversity and social justice. Other disciplines, even with a similar focus, may approach service-learning differently and therefore produce different outcomes. As another qualitative data point in course comparisons would be the inclusion of course syllabi in the analysis process. Researchers could use this to 104 assess how these different classes address the different learning objectives. Again, it will be important to look for what isn’t there as much as what is included. Research about service-learning and, specifically, its connection to civic learning needs to move beyond individual or community impact and think about how the broader context of the society in which we live is impacting student learning outcomes. Students’ perceptions of diversity and social justice attitudes as outlined in the CASQ were shown to be problematic. Additionally, the CASQ itself presented many concepts rooted in white supremacy culture as the positive or desired outcome for students. If the field maintains that one of the central goals of service-learning is to impact societal change, then there needs to be an understanding of how the institution of higher education, itself, is complicit in maintaining a culture that inhibits social change through the “reproduction of white supremacy” (Rosa & Diaz, 2019, p. 121). New research in assessment should reckon with a pedagogical tool that comes from a history of racist ideas. Researchers should ask themselves, “How do notions of racism and white supremacy culture show up in the way we implement and evaluate service-learning?” Conclusion Using a mixed methods approach to assess civic learning outcomes provided a different insight into the acquisition of the skills, behaviors, and attitudes needed for positive civic identity development for college students. The foundation set by the Civic Attitudes and Skills Questionnaire (CASQ) gives current and future scholars a solid place to start in their assessment of the ways service-learning contributes to these efforts. This study shows that service-learning remains a high-impact practice and continues to be crucial in this development. By highlighting the voices of racially minoritized students who have been historically excluded from deeper research on how service-learning experiences impact students, it becomes apparent that people in 105 the service-learning field need to advocate for the inclusion of more voices. These voices, once small in number but now growing exponentially, represent a new, more nuanced direction of implementation, assessment, and research in the field. 106 REFERENCES Allen, W.R. (1985). Black student, White campus: Structural, interpersonal, and psychological correlates of success. Journal of Negro Education, 54(2), 134-147. Beck, C. & Kosnik, C. (2001). From cohort to community in a preservice teacher education program. Teaching and Teacher Education, 17(8), 925-948. https://doi.org/10.1016/S0742-051X(01)00041-5 Bennett, C. & Okinaka, A.M. (1990). Factors related to persistence among Asian, Black, Hispanic, and White undergraduates at a predominantly White university: Comparison between first and fourth year cohorts. The Urban Review, 22(1), 33-60. Binkley, M., Erstad, O., Herman, J., Ripley, M., Rumble, M., & Miller-Ricci, M. (2012). Defining twenty-first century skills. In P. Griffin, B. McGraw, and E. Care (Eds.). Assessment and teaching of 21st century skills. Springer, Dordrecht. Bocci, M. (2015). Service-learning and White normativity: Racial representation in service- learnings historical narrative. Michigan Journal of Community Service Learning, 2(1), 5- 17. Bringle, R.G. & Hatcher, J. (2000). Institutionalization of service-learning in higher education. The Journal of Higher Education, 71(3): 273-290. Bringle, R.G., Phillips, M.A., & Hudson, M. (2004). The measure of service-learning: Research scales to assess student experiences. American Psychological Association. Burke, P.J. (2018). Trans/Forming pedagogical spaces: Race, belonging and recognition in higher education. In J. Arday & H.S. Mirza (Eds.), Higher Education: Racism, whiteness and decolonizing the academy. https://doi.org/10.1007/978-3-319-60261-5_21 Butin, D.W. (2005). Service-learning as postmodern pedagogy. In D.W. Butin (Ed.) Service- learning in higher education: Critical issues and directions. Palgrave Macmillan. Campus Compact. (2016). Revitalizing our democracy: building on our assets, 2016 summary of public four-year institutions. Center for Postsecondary Research. (n.d.). National survey of student engagement. https://nsse.indiana.edu/nsse/about-nsse/index.html Chang, B. (2019). Reflection in learning. Online Learning, 23(1), 95-110. Chickering, A.W. & Gamson, Z.F. (1987). Seven principles for good practice in undergraduate education. American Association for Higher Education Bulletin, March: 3-7. Chittum, J.R., Enke, K.A., & Finley, A.P. (2022). The effects of community-based and civic engagement in higher education: What we know and questions that remain. American Association of Colleges and Universities. 107 Cohen, L., Manion, L., & Morrison, K. (2011). Research methods in education (7th ed.). Routledge. D’Agostino, M.J. (2018). Fostering a civically engaged society: The university and service learning. Journal of Public Affairs Education, 14(2), 191-204. D’Augeli, A.R. & Hershberger, S.L. (1993). African American undergraduates on a predominantly White campus: Academic factors, social networks, and campus climate. The Journal of Negro Education, 62(1), 67-81. Delgado, R. (1991). Derrick Bell’s racial realism: A comment on white optimism and Black despair commentary on racial realism. Connecticut Law Review, 27, 527-532. Delgado, R. & Stefancic, J. (2012). Critical race theory: An introduction. New York University Press. Denzin, N. K., & Lincoln, Y. S. (2000). Introduction: The Discipline and Practice of Qualitative Research. In N. K. Denzin, & Y. S. Lincoln (Eds.), Handbook of Qualitative Research (2nd ed., pp. 1-34). Sage Publications. Dewey, J. (1916). Democracy and education: An introduction to the philosophy of education. Free Press. Dubb, S., McKinley, S., Howard, T., Ferris, D., Serang, F. (2013). Achieving the anchor promise: Improving outcomes for low-income children, families and communities. The Annie E. Casey Foundation. Eyler, J. (2002). Reflection: Linking service and learning – linking students and communities. Journal of Social Issues, 58(3), 517-534. Eyler, J. & Giles, Jr., D.E. (1999). Where’s the learning in service-learning? Jossey-Bass. Eyler, J., Giles, Jr., D.E., & Braxton, J. (1997). The impact of service-learning on college students. Michigan Journal of Community Service Learning, 4(1), 5-15. Eyler, J., Giles, Jr., D.E., Schmiede, A. (1996). A practitioner’s guide to reflection in service- learning: Student voices & reflections. Corporation for National Service. Filho, W.L., Wall, T., Rayman-Bacchus, L., Mifsud, M., Pritchard, D.J., Lovren, V.O., Farinha, C., Petrovic, D.S., & Balogun, A. (2021). Impacts of COVID-19 and social isolation on academic staff and students at universities: a cross-sectional study. BMC Public Health, 21(1213), 1-19. Finley, A. McNair, T., & Clayton-Pedersen, A. (2022). Introduction. In Delivering on the promise of high-impact practices: Research and models for achieving equity, fidelity, impact, and scale. (J. Zilvinskis, J. Kinzie, J. Daday, K. O’Donnell, & C. Vande Zande). Routledge. 108 Furco, A. (1996). Service-learning: A balanced approach to experiential education. Expanding Boundaries: Serving and Learning. Washington, DC: Corporation for National Service, 2-6. Freire, P. (1970). Pedagogy of the Oppressed. Continuum International Publishing Group. Garcia, N.M., Lopez, N., & Velez, V. N. (2018). QuantCrit: Rectifying quantitative methods through critical race theory. Race Ethnicity and Education, 21(2), 149-157. https://doi.org/10.1080/13613324.2017.1377675 Gelmon, S.B., Agre-Kippenhan ,S., & Cress, C.M. (2005). Learning through serving: A student guidebook for service-learning across the disciplines. Stylus Publishing. Gilborn, D., Warmington, P., & Demack, S. (2018). QuantCrit: Education policy, ‘Big Data’ and principles for a critical race theory of statistics. Race, Ethnicity, and Education, 21(2), 158-179. Giles, Jr., D.E. & Eyler, J. (1994). The impact of a college community service laboratory on students’ personal, social, and cognitive outcomes. Journal of Adolescence, 17(4), 327- 339. Glesne, C. (2016). Becoming qualitative researchers: An introduction. 5th edition. Pearson: Boston, MA. Green, A.E. (2003). Difficult stories: Service-learning, race, class, and whiteness. College Composition and Communication, 55(2), 276-301. Green, A.E. (2001). “But you aren’t white:” Racial perceptions and service-learning. Michigan Journal of Community Service Learning, 8(1), 18-26. Guest, G. (2013). Describing mixed methods research: An alternative to typologies. Journal of Mixed Methods Research, 7(2), 141-151. Hall, J.N. (2020). Focus groups: Culturally responsive approaches for qualitative inquiry and program evaluation. Myers Education Press. Haney López, I. (2006). White by law: The legal construction of race. New Your University Press. Harper, S.R. (2012). Race without racism: How higher education researchers minimize racist institutional norms. The Review of Higher Education, 36(1), 9-29. Hiraldo, P. (2010). The role of critical race theory in higher education. The Vermont Connection, 31(1). https://scholarworks.uvm.edu/tvc/vol31/iss1/7 Holdo, M. (2023). Critical reflection: John Dewey’s relational view of transformative learning. Journal of Transformative Education, 21(1), 9-25. 109 Howard, J. (2001). Service-learning course design workbook. Michigan Journal of Community Service Learning. Hudgins, A. (2020). Civic identity development in a critical service-learning context: A critique of the Civic-Minded Graduate Rubric 2.0. Journal of Community Engagement and Higher Education, 12(1), 66-81. Hurtado, S. (2019). “Now is the time”: Civic learning for a strong democracy. Daedalus, the Journal of the American Academy of Arts & Sciences, 148(4), 94-107. Jacoby, B. (2015). Service-learning essentials: Questions, answers, and lessons learned. Jossey- Bass. Jaeger, A. J., Dunstan, S., Thornton, C., Rockenbach, A. B., Gayles, J. G., & Haley, K. J. (2013). Put Theory into Practice. About Campus, 17(6), 11–15. https://doi.org/10.1002/abc.21100 Jones, L., Castellanos, J., & Cole, D. (2002). Examining the ethnic minority student experience at predominantly White institutions: A case study. Journal of Hispanic Higher Education, 1(1), 19-39. Kincheloe, J., & McLaren, P. (2005). Rethinking critical theory and qualitative research. In N. Denzin & Y. S. Lincoln (Eds.), Handbook of Qualitative Research (3rd ed.). Sage. Kiely, R. (2005). A transformative learning model for service-learning: A longitudinal case study. Michigan Journal of Community Service Learning, 12(2), 5-22. Kilgo, C.A., Sheets, J.K.E., Pascarella, E.T. (2014) The link between high-impact practices and students learning: Some longitudinal evidence. Higher Education, 69, 509-525. Kinzie, J., Silberstein, S., McCormick, A.C., Gonyea, R.M., & Dugan, B. (2021). Centering racially minoritized student voices in high-impact practices. Change. Kolb, D.A. (1984). Experiential learning: Experience as the source of learning and development. New Jersey: Prentice Hall Kuh, G.D. (2008). High-impact educational practices: What they are, who has access to them, and why they matter. Association of American Colleges and Universities. Latta, M., Kruger, T.M., Payne, L., Weaver, L., & VanSickle, J.L. (2018). Approaching critical service-learning: A model for reflection on positionality and possibility. Journal of Higher Education Outreach and Engagement, 22(2), 31-55. Leong, F.T.L. & Park, Y.S. (2016). Introduction. In Council of National Psychological Associations for the Advancement of Ethnic Minority Interests. Testing and assessment with persons & communities of color. American Psychological Association. Retrieved from https://www.apa.org/pi/oema 110 Mitchell, T.D. (2008). Traditional vs. critical service-learning: Engaging the literature to differentiate two models. Michigan Journal of Community Service-Learning, 50-65. Mitchell, T.D., (2015). Using a critical service-learning approach to facilitate civic identity development. Theory into Practice, 54(1), 20-28. Moely, B.E., McFarland, M., Miron, D., Mercer, S., & Ilustre, V. (2002). Changes in college students’ attitudes and intentions for civic involvement as a function of service-learning experiences. Michigan Journal of Community Service Learning, 9(1), 18-26. Moely, B. E., Mercer, S. H., Ilustre, V., Miron, D., & McFarland, M. (2002). Psychometric properties and correlates of the Civic Attitudes and Skills Questionnaire (CASQ): A measure of students' attitudes related to service-learning. Michigan Journal of Community Service Learning, 8(2), 15–26. Musil, C.M. (2009). Education students for personal and social responsibility: The civic learning spiral. In Barbara Jacoby and Associates (Eds.), Civic engagement in higher education: Concepts and practices (pp. 49-68). Jossey-Bass. Musu-Gillette, L., Robinson, J., McFarland, J., KewalRamani, A., Zhang, A., & Wilkinson- Flicker, S. (2016). Status and Trends in the Education of Racial and Ethnic Groups 2016 (NCES 2016-007). U.S. Department of Education, National Center for Education Statistics. http://nces.ed.gov/pubsearch. National Center for Immunization and Respiratory Diseases (U.S.). Division of Viral Diseases. (2021). Guidance for institutions of higher education (IHEs). The National Task Force on Civic Learning and Democratic Engagement. (2012). A crucible moment: College learning and democracy’s future. Okun, T. (2021). White supremacy culture – still here. http://www.whitesupremacyculture.info/ Onwuegbuzie, A.J., Dickinson, W.B., Leech, N.L., & Zoran, A.G. (2009). Toward more rigor in focus group research: A new framework for collecting and analyzing focus group data. International Journal of Qualitative Methods, 8(3), 1-21. Onwuegbuzie, A.J., Freis, R.K., and Collins, K.MT. (2013). Foreword: Using Bronfenbrenner’s ecological systems theory to frame quantitative, qualitative, and mixed research. International Journal of Multiple Research Approaches, 7(1), 2-8. Prentice, M. & Robinson, G. (2010). Improving student learning outcomes with service learning. Higher Education. Renn, K.A. & Reason, R.D. (2013). College students in the United States: Characteristics, experiences, and outcomes. Jossey-Bass. Roberts, J.M. (2001). Critical realism and the dialectic1. The British Journal of Sociology, 52(4), 667-685. 111 Rocheleau, J. (2004) Theoretical roots of service-learning: Progressive education and the development of citizenship. In B.W. Speck & S.L. Hoppe (Eds.). Service-learning: History, theory, and issues (pp. 3-22). Praeger. Rosa, J. & Diaz, V. (2019). Raciontologies: Rethinking anthropological accounts of institutional racism and enactments of white supremacy in the United States. American Anthropologist, 122(1), 120-132. Schön, D.A. (1995). Knowing-In-Action: The new scholarship requires a new epistemology. Change: The magazine of higher learning, 27(6), 27-34. Sears, A. & Cairns, J. (2015). A good book in theory (3rd ed.). University of Toronto Press. SenGupta, S., Hopson, R., & Thompson-Robinson, M. (2004). Cultural competence in evaluation: An overview. New Directions for Evaluation, Summer (102), 5-20. Shavers, C.L. & Mitchell, R.B. (2019). High-impact educational experiences in colleges of business. e-Journal of Business Education & Scholarship of Teaching, 13(1), 1-12 Shumer, R. (1997). What research tells us about designing service learning programs. National Association of Secondary School Principals (NASSP), 81(591): 18-24. Simons, L., Fehr, L., Black, N., Hogerwerff, F., Georgana, D, and Russell, B. (2011). The applications of racial identity development in academic-based service learning. International Journal of Teaching and Learning in Higher Education, 23(1), 72-83. Sipe, L. & Constable, S. (1996). A chart of four contemporary research paradigms: Metaphors for modes of inquiry. Taboo, Journal of Culture and Education, 1, 153-163. Sleeter, C.E. (2017). Critical race theory and the Whiteness of teacher education. Urban Education, 52(2), 155-169. Smith, S. (2019). Students’ of Color experiences in a service-learning program at a community college: A critical study. Ph.D. dissertation, University of Tennessee, 2019. https://trace.tennessee.edu/utk_graddiss/5713 Song, W., Furco, A. ,Lopez, I. & Maruyama, G. (2017). Examining the relationship between service-learning participation and the educational success of underrepresented students. Michigan Journal of Community Service Learning, 24(1), 23-37. Stage, F.K. (2007). Answering critical questions using quantitative data. New Directions for Institutional Research. Spring (133), 5-16. Stage, F.K., & Wells, R.S. (2014). Critical Quantitative Inquiry in Context. New Directions for Institutional Research, 2013, 1-7. Stanton, T.K., Giles, Jr., D.E., Cruz, N.I. (1999). Service-learning: A movement’s pioneers reflect on its origins, practice, and future. Jossey-Bass. 112 Steinberg, K.S., Hatcher, J.A., Bringle, R.G. (2011). Civic-minded graduate: A north star. Michigan Journal of Community Service Learning, Fall, 19-33. Swaner, L.E. & Brownell, J.E. (2008). Outcomes of high impact practices for underserved students: A review of the literature. Association of American Colleges and Universities. Tryon, E., Stoeker, R. Martin, A., Seblonka, K., Hilgendorf, A., & Nellis, M. (2008). The challenges of short-term service-learning. Michigan Journal of Community Service Learning, 14(2): 16-26. Tulane University (n.d.) 2001 Fall Tulane University Profile. https://registrar.tulane.edu/sites/default/files/2001_Fall_Tulane_University_Profile.pdf U.S. Department of Education. Institute of Education Sciences, National Center for Education Statistics (2019). Spotlight B: Characteristics of postsecondary institutions serving specific minority racial/ethnic groups. https://nces.ed.gov/programs/raceindicators/spotlight_b.asp U.S. Department of Education. Institute of Education Sciences, National Center for Education Statistics. (2022). Fall enrollment of U.S. residents in degree-granting postsecondary institutions by race/ethnicity: Selected years 1976 through 2031. https://nces.ed.gov/programs/digest/d22/tables/dt22_306.30.asp Walls, J.K. & Hall, S.S. (2018). A focus group study of African American students’ experiences with classroom discussions about race at a predominantly White university. Teaching in Higher Education, 23(1), 47-62. https://doi.org/10.1080/13562517.2017.1359158 Welch, M. & Plaxton-Moore, S. (2019). The craft of community-engaged teaching and learning: A guide for faculty development. Campus Compact. Whitley, M.A. (2014). A draft conceptual framework of relevant theories to inform future rigorous research on student service-learning outcomes. Michigan Journal of Community Service Learning, 20(2), 19-40. 113 APPENDIX A: QUALTRICS CIVIC ATTITUDES AND SKILLS QUESTIONNAIRE Civic Attitudes and Skills Questionnaire Relaunch Q1 You are being asked to participate in a research study of civic learning outcomes based on participation in a service-learning course. This requires you to fill out the following questionnaire. In order to participate in this study, you must be 18 years of age or older Participation in this research study is completely voluntary. You have the right to say no. You may change your mind at any time during the survey and withdraw from the survey. Whether or not you participate will have no effect on your standing within your higher education institution. The completion of this survey should take no more than 15 minutes. Q2 I voluntarily consent to participate in this research study. o Yes o No Q3 Please indicate your class rank: o Freshman o Sophomore o Junior o Senior Q4 Please indicate your current grade point average: ________________________________________________________________ Q5 What is your major? ________________________________________________________________ 114 Q6 How many hours of service were required for your course? Q7 Please indicate your ethnicity: o Hispanic/Latino/Latinx (A person of Cuban, Mexican, Puerto Rican, South or Central American, or other Spanish culture or origin, regardless of race.) o Not Hispanic/Latino/Latinx Q8 Select one or more of the following options that best describes your race: ▢ ▢ ▢ ▢ American Indian or Alaska Native: A person having origins in any of the original peoples of North and South America (including Central America) who maintains cultural identification through tribal affiliation or community attachment. Asian: A person having origins in any of the original peoples of the Far East, Southeast Asia, or the Indian Subcontinent, including, for example, Cambodia, China, India, Japan, Korea, Malaysia, Pakistan, the Philippine Islands, Thailand, and Vietnam. Black or African American: A person having origins in any of the Black racial groups of Africa. Native Hawaiian or Other Pacific Islander: A person having origins in any of the original peoples of Hawaii, Guam, Samoa, or other Pacific Islands. ▢ White: A person having origins in any of the original peoples of Europe, the Middle East, or North Africa. ▢ I prefer to self-identify: 115 Q9 Are you an international student: Possession of an F-1, J-1 ,O-1, TN, or or E3 Visa. o Yes o No Q10 What is your current gender identity? o Woman o Man o Transgender Woman o Transgender Man o Genderqueer o Gender-nonconforming o Nonbinary o Not listed (please specify) ________________________________________________ o Prefer not to answer 116 Q11 This section is designed to understand how your participation in a service-learning course influenced your plans for civic action. Because of my service-learning course: Completely disagree 2 3 4 Completely agree I plan to become involved in my community. I plan to participate in a community action program. I plan to become an active member of my community. In the future, I plan to participate in a community service organization. I am committed to making a positive difference I plan to become involved in programs to help clean up the environment. o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o 117 Q12 Because of my participation in service-learning during Fall 2019: Completely Disagree 2 3 4 Completely Agree I can listen to other people's opinions. I can work cooperatively with a group of people. I can think logically in solving problems. I can communicate well with others. I can successfully resolve conflicts with others. I can easily get along with people. I try to find effective ways of solving problems. When trying to understand the position of others, I try to place myself in their position. I find it easy to make friends. I can think analytically in solving problems. I try to place myself in the place of others in trying to assess their current situation. I tend to solve problems by talking them out. o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o 118 Q13 Because of my participation in service-learning during Fall 2019: Completely disagree 2 3 4 Completely agree I am aware of current events. I understand the issues facing this nation. I am knowledgeable of the issues facing the world. I am aware of the evens happening in my local community. I plan to be involved in the political process. I understand the issues facing my city's community. o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o Q14 Because of my participation in service-learning during Fall 2019: Completely disagree 2 3 4 Completely Agree I am a better follower than a leader. I am a good leader. I have the ability to lead a group of people. I would rather have somebody else take the lead in formulating a solution. I feel that I can make a difference in the world. o o o o o o o o o o o o o o o o o o o o o o o o o 119 Q15 Please indicate the degree to which you agree or disagree to the following statements: Completely Disagree (1) (2) (3) (4) Completely Agree (5) I don't understand why some people are poor when there are boundless opportunities available to them. People are poor because they choose to be poor. Individuals are responsible for their own misfortunes. We need to look no further than the individual in assessing their problems. In order for problems to be solved, we need to change public policy. We need to institute reforms within the current system to change our communities. We need to change people's attitudes in order to solve social problems. It is important that equal opportunity be available to all people. o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o 120 Q16 Based on your participation in service-learning during Fall 2019, please indicate the level to which you agree or disagree with the following statements: Completely disagree (1) (2) (3) (4) Completely Agree (5) It is hard for a group to function effectively when the people involved come from very diverse backgrounds. I prefer the company of people who are very similar to me in background and expression. I find it difficult to relate to people from a different race or culture. I enjoy meeting people who come from backgrounds very different from my own. Cultural diversity within a group makes the group more interesting and effective. o o o o o o o o o o o o o o o o o o o o o o o o o 121 APPENDIX B: FOCUS GROUP PROTOCOL 1) Opening Welcome Thank you for allowing me to talk with you. My name is Nicole Springer, and I am going to act as the facilitator for today’s focus group session. Today is a chance for you to share your thoughts on your service-learning experience in TE 250 and how it connects to the civic learning outcomes in the survey you filled out. The purpose of this research is to document your thoughts to help faculty and other community engagement professionals utilize service- learning to its full civic potential. 2) Ground Rules Before today’s focus group, you received a copy of the statements from the survey you filled out. They are grouped by the civic learning outcome the statement assessed. Please keep this handy as you will be asked to refer to it from time to time. 3) Reaffirmation of consent Before we get started, I would like to remind you of the information that was on the consent form you signed. That form acknowledges that I will do my best to keep your answers confidential. That means when I write about what you say, I will limit any identifying information and you will be given a pseudonym to hide your identity. I, however, also need some help from you. In order to make sure every participant feels comfortable to share their truth, please remember that what is said here stays here. Additionally, if ever I ask a question you would rather not answer, you can choose to pass. This conversation should take about 90 minutes and will be both video and audio recorded through the Zoom platform. Please verbally state if you are willing to continue with the focus group. 122 4) Introductory Question: Think back to your service-learning experience. What was the best part of that experience? a) If students have trouble recalling the “best part” ask them to recall something memorable that happened. b) This question is designed to get participants situated in the experience of service- learning. 5) Topic 1: Civic Attitudes and Skills Questionnaire (CASQ) connection to service-learning course a) I would first like to get your opinions about the questionnaire I asked you to fill out regarding the service-learning portion of your TE 250 course. b) Factor Alignment i) Take a minute and look at the larger groupings of questions on the survey and think about your service-learning experience. ii) How does the way the sections of the questions are divided connect to what you learned from service-learning. For example, did service-learning help you learn something about how you feel about social justice? iii) Which parts of your service-learning experience are you able to connect to the themes on the survey? c) Statement Alignment i) Let’s move into each section of the survey and look at specific statements. (1) Which statements seem closely related to your service-learning experience? 123 (a) For example, did your service-learning experience help you understand how cities function? (b) How were you able to make that connection? (2) Which parts did not seem related to your service-learning experience? (may need to wrap this up in question 1) 6) Topic 2: Difference in scores i) It is often suspected that students of color would answer parts of the survey differently than white students. Are there aspects of your service-learning experience that led you to agree or disagree with that statement? Why? ii) Where do you think I would find the most difference, if any? 7) Topic 3: Improvement (These questions get at connecting to service-learning in a different way. Students will be allowed to talk about how the questionnaire could better connect to their experience in service-learning.) i) Thank you for talking with me about your experience with the survey as it connects to your service-learning experience. My next set of questions are for you to be able to give input about how you would improve the questionnaire. (1) Overall, how well did the questions on the survey let you critically reflect on your service-learning experience? Why? Why not? (2) If you could change anything about the survey, what would it be? 124 APPENDIX C: FACTOR DIFFERENCES IN MEAN VALUES Table C1 Likert scale for Civic Attitudes and Skills Questionnaire Completely disagree 1 Somewhat disagree 2 Neutral Somewhat agree 3 4 Completely agree 5 Figure C1 Differences in mean scores for factor one disaggregated by race Note: The corresponding statements are listed below. 1) I plan to become involved in my community. 2) I plan to participate in a community action program. 3) I plan to become an active member of my community. 4) In the future, I plan to participate in a community service organization. 5) I am committed to making a positive difference. 6) I plan to become involved in programs to help clean up the environment. 125 Figure C2 Differences in statement mean values of factor two disaggregated by race Note: The corresponding statements are below. 1) I can listen to other people’s opinions. 2) I can work cooperatively with a group of people. 3) I can think logically in solving problems. 4) I can communicate well with others. 5) I can successfully resolve conflicts with others. 6) I can easily get along with people. 7) I try to find effective ways of solving problems. 8) When trying to understand the position of others, I try to place myself in their position. 9) I find it easy to make friends. 10) I can think analytically in solving problems. 11) I try to place myself in the place of others in trying to assess their current situation. 12) I tend to solve problems by talking them out. 126 Figure C3 Differences in statement mean values of factor three disaggregated by race Note: The corresponding statements are below. 1) I am aware of current events. 2) I understand the issues facing this nation. 3) I am knowledgeable of the issues facing the world. 4) I am aware of the events happening in my local city. 5) I plan to be involved in the political process. 6) I understand the issues facing my city’s community. 127 Figure C4 Differences in statement mean values of factor four disaggregated by race Note: The corresponding statements for Factor Four – Leadership Skills are below. 1) I am a better follower than a leader. 2) I am a good leader. 3) I have the ability to lead a group of people. 4) I would rather have somebody else take the lead in formulating a solution. 5) I feel that I can make a difference in the world. 128 Figure C5 Differences in statement mean values of factor five disaggregated by race e c n e r e f f i D n a e M 1.2 1 0.8 0.6 0.4 0.2 0 SOC/WHITE SOC/FULL WHITE/FULL 1 0.1311 0.1097 0.0214 2 0.9321 0.0578 0.9899 3 0.4932 0.42 0.0732 4 0.7665 0.6547 0.1118 5 0.0326 0.0284 0.0042 Statements 6 0.3241 0.2768 0.0473 7 0.3333 0.2907 0.0426 8 0.1637 0.1399 0.0238 Note: The corresponding statements are below. 1) I don’t know why some people are poor when there are boundless opportunities available to them. 2) People are poor because they choose to be poor. 3) Individuals are responsible for their own misfortunes. 4) We need to look no further than the individual in assessing their problems. 5) In order for problems to be solved, we need to change public policy. 6) We need to institute reforms within the current system to change our communities. 7) We need to change people’s attitudes in order to solve social problems. 8) It is important equal opportunity be available to all people. 129 Figure C6 Differences in statement mean values of factor six disaggregated by race 130 APPENDIX D: REFLECTIONS As a community engagement scholar-practitioner, I spend much of my time lifting up the importance of reflection. With students, we connect it to some of the concepts and theories I mentioned in my dissertation such as critical reflection, the importance of student experiences in the learning process, and how incorporating community engagement can lead to transformative learning. When working with other practitioner-scholars, I often talk to them about the work of Donald Schön. Schon (1995) is known for uplifting practitioners' knowledge as valid and important to the overall way we think about epistemology. He speaks on both “knowing-in- action” or tacit knowledge and reflection in action, which creates new ways of knowing specifically within a person’s profession. Right now, this all seems highly academic for reflection, but stay with me. As a doctoral student who has been in the field of community-engaged teaching and learning for over 20 years, I was able to experience moments of both critical reflection and reflection-in-action. Much of what I read, either as a professional or as a student, was interpreted through the lens of a community engagement practitioner who wanted to understand what the numbers meant. I focused on professional development for most of my career and was interested in how more (quantitative) data could yield better results and deeper knowledge. Year after year, I witnessed Students of Color participate in service-learning and remain on the margins of that experience. I was convinced that the only way to understand how Students of Color were impacted by their service-learning experiences was to administer the Civic Attitudes and Skills Questionnaire to over 2,000 students nationwide with the hope that there would be enough Students of Color to make deeper quantitative analysis possible. This was my first proposal, but everything changed because of COVID-19. 131 The change in my first proposed research study due to the suspension of in-person activities due to the COVID-19 epidemic was a blessing in disguise. These shifts in service- learning courses created the necessity of adding a second method of inquiry to my research toolbelt. People are complex beings, and simply assessing the outcomes only provided part of a picture. These are the parts of the characteristics that were easily seen and became the focal point of the artwork. However, using qualitative inquiry and the student participants' responses about how they connected their time with the community to their time in the classroom and combined with their own lived experiences, I could use their words and non-verbal cues to create a more holistic interpretation. Changing how I thought about inquiry was a lesson in critical reflection. This change, this twist of events, also influenced the way I thought about my work. This was reflection-in- action. Schon says reflection-in-action “begins when a spontaneous performance is interrupted by a surprise” (1995, p. 30). I was surprised by COVID-19 and saw the immense benefit of qualitative inquiry. Amid this study that was so connected to my work and my lived experience, I, in the moment, had to reflect on and write about this work in an academic way, knowing that it had implications for my work as a community engagement professional. Over time I have become more and more connected to national organizations dedicated to civic and community engagement and community-engaged scholarship. The conversations in and among these organizations often circle back to the need for reciprocity of sorts. The scholarship should inform the practice, and practice should provide richness to the scholarship. There is much work to be done on both sides of this coin, but my focus remains on professional development and the connection to the scholarship in the field. National organizations can play a role by having intentional conversations that remove the silos and begin to bridge the research 132 and practice divide. While some community engagement practitioners do not see themselves as scholars, they do need to acknowledge that their work has a theoretical base built on researchers from multiple disciplines. National organizations can play a role in translating that work into more accessible professional development opportunities. This work does not rest within one organization or one association. Just as our field is interdisciplinary, the professional and research organizations of the field need to collaborate for this change to happen. We, as community engagement practitioner-scholars, must find a way to do our own boundary-spanning and lead the charge for integrating theory into practice. 133